Generative AI-assisted Participatory Modeling in Socio-Environmental Planning under Deep Uncertainty

Technical Deep Dive: Generative AI-assisted Participatory Modeling in Socio-Environmental Planning under Deep Uncertainty

Overview

This paper proposes a workflow that uses large language models (LLMs) to facilitate the initial problem conceptualization process in socio-environmental planning under deep uncertainty. The key contributions are:

• An LLM-assisted workflow that guides researchers through translating stakeholders' intuitive problem descriptions into a computational model specification, including identifying model components, exploring multiple perspectives, composing a unified model, and implementing it in Python.
• Demonstration of the workflow on two socio-environmental planning problems - the lake problem and an electricity market problem. The results show that the LLM can effectively extract relevant information, complement the specification with its common-sense knowledge, and produce satisfactory outputs after a few iterations of human validation and refinement.
• Discussion of the strengths and limitations of applying the workflow for initial problem conceptualization in socio-environmental planning under deep uncertainty.

Problem & Context

• Socio-environmental planning often involves complex uncertainties, ranging from well-characterized ones to deep uncertainties that cannot be fully eliminated.
• Decision-Making under Deep Uncertainty (DMDU) approaches have been developed to support planning in such contexts, involving three iterative steps: problem conceptualization, exploratory analysis, and plan deployment.
• The problem conceptualization step, where stakeholders' understandings are translated into a formal model, is often complex and time-consuming, especially when non-expert stakeholders are involved.
• Large language models (LLMs) have shown exceptional capabilities in natural language processing and generation, suggesting they could facilitate this problem conceptualization process.

Methodology

• The proposed LLM-assisted workflow guides researchers through 4 steps:
  1. Initial formalization: Translate the problem description into a model specification.
  2. Multi-perspective formalization: Identify different stakeholder perspectives and formalize them.
  3. Composition: Compose the perspectives into a unified model.
  4. Python implementation: Implement the unified model in Python.
• Validation, verification and refinement are performed at each step through iterative interaction with the LLM.
• Experiments were conducted using ChatGPT 5.2 Instant on two socio-environmental planning problems: the lake problem and an electricity market problem.

Data & Experimental Setup

• The lake problem is a canonical benchmark, while the electricity market problem has been implemented differently in prior studies.
• For the electricity market problem, two sub-cases were considered:
  1. Comprehensive problem description provided
  2. Brief problem description provided

Results

• In all sub-cases, the LLM was able to correctly extract the key model components by the end of the first step, with only minor errors in subsequent steps.
• The number of refinement iterations required was low, indicating the LLM maintained consistency in model formalization when following the workflow.
• The final Python implementations exhibited consistent performance with the baseline problem descriptions, though the LLM struggled with some complex functionalities like implementing the log-normal distribution and merit-order market clearing.

Interpretation

• The LLM-assisted workflow can effectively facilitate the initial problem conceptualization process in socio-environmental planning under deep uncertainty, by:
  • Translating stakeholders' intuitive descriptions into a formal model specification
  • Exploring diverse stakeholder perspectives and factors
  • Composing a unified, modular model
  • Implementing the model in Python
• LLMs can complement researchers' understanding with their common-sense knowledge, improving the completeness and consistency of the model specification.
• However, LLMs are still prone to errors in implementing complex mechanisms, requiring human validation and refinement.

Limitations & Uncertainties

• The workflow is subject to limitations of LLMs, such as limited long-horizon reliability, prompt sensitivity, and lack of guaranteed code correctness.
• The experiments were conducted on hypothetical problems, not real-world applications. Further evaluation is needed in more complex, multi-actor contexts.
• The specific prompts and workflow structure used may not generalize to all socio-environmental planning problems. Customization may be required.

What Comes Next

• Applying and evaluating the workflow in more complex, real-world socio-environmental planning problems involving multiple stakeholders and deeper system dynamics.
• Investigating techniques to further improve the reliability and robustness of the LLM-assisted workflow, such as prompt engineering, hierarchical decomposition, and verification methods.
• Exploring how the LLM-assisted workflow can be integrated with existing DMDU approaches to support the overall decision-making process.