PolicyPad: Collaborative Prototyping of LLM Policies

Technical Deep Dive: PolicyPad - Collaborative Prototyping of LLM Policies

Overview

PolicyPad is an interactive system that enables small groups of domain experts to collaboratively design and test policies for governing the behavior of large language models (LLMs). The key contributions are:

• A conceptualization of "LLM policy prototyping" - an emerging practice that draws from user experience (UX) prototyping methods to enable rapid, iterative policy design and testing.
• The design and implementation of the PolicyPad system, which supports policy drafting, scenario-based testing, and real-time collaboration.
• An evaluation showing that PolicyPad fostered more collaborative dynamics and led to more novel policy insights compared to a baseline approach.

Problem & Context

As LLMs are increasingly deployed in high-stakes domains like mental health and law, there is growing recognition that their behaviors should be governed by clear, domain-specific policies. However, existing policy design efforts are often insular, lacking input from relevant domain experts. This can lead to policies that fail to address key safety concerns or produce irresponsible model outputs.

Methodology

The researchers first conducted a 15-week observational study with 9 mental health experts. They observed experts collaboratively drafting policies and taxonomy while seeking ways to rapidly test and iterate on them through experimentation with model behavior on realistic scenarios. Based on these observations, the researchers conceptualized "LLM policy prototyping" - an emerging practice that draws from UX prototyping methods.

The researchers then designed and developed the PolicyPad system to support this practice. PolicyPad enables small groups of experts to:

1. Collaboratively draft LLM policies in a real-time editor.
2. Test the policy-informed model's behavior against example scenarios.
3. Iteratively refine the policy based on insights from testing.

Evaluation

The researchers evaluated PolicyPad through 8 policy prototyping sessions with 22 experts in mental health and law. They found that PolicyPad's design decisions:

• Fostered collaborative dynamics among experts during policy design.
• Enabled experts to prototype more novel policy insights compared to a baseline approach.

Novel policy insights included:

• More specific guidelines on when the model should defer to human experts
• Procedures for handling high-risk situations (e.g. suicidal ideation)
• Recommending the model proactively elicit key information from users before providing assistance

Limitations & Uncertainties

• The perspectives integrated into the policies were heavily influenced by American mental health and legal systems, and may not generalize to other countries.
• The selection of starting scenarios can alter policy outcomes by steering discussions towards certain topics.
• It is unclear whether recent improvements in LLMs' reasoning capabilities translate to more effective information elicitation from users.

What Comes Next

Future work can explore scaling up policy prototyping beyond small expert groups to include more diverse stakeholders, while preserving the richness of synchronous, deliberative discussions. This may involve drawing inspiration from multi-stage citizens' assemblies. New tooling and infrastructure will likely be needed to facilitate policy prototyping at scale.