Multi-Agent LLM Experimentation Framework

Overview

Designing scalable multi-agent LLM experimentation frameworks to study causal effects in identity disclosure and stance change, targeting ACL 2026 submission on identity-conditioned persuasion dynamics.

Key Technologies

• LLMs: 12 frontier models (GPT-4, Claude, Gemini, etc.)
• Automation: Playwright, Chromium/Electron orchestration
• Experimentation: Controlled R1-R2 role structures, Coordinator-LLM consistency scoring

Research Contributions

• Designed experiments across 12 frontier models with 1.9K sessions
• Built Chromium/Electron orchestration layer unifying browser chatbot sessions and direct API inference
• Developed controlled R1-R2 role structures for studying identity disclosure effects
• Implemented coordinator-LLM consistency scoring at scale

Technical Implementation

• Browser Automation: Playwright automation for parallel prompting
• Response Capture: DOM-based extraction from web interfaces
• API Integration: Unified interface for direct model API calls
• Consistency Scoring: LLM-based evaluation of multi-agent interactions

Research Goals

• Study causal effects in multi-agent LLM interactions
• Analyze identity-conditioned persuasion dynamics
• Understand stance change in controlled conversational settings
• Target publication: ACL 2026

Advisor

Prof. Mingfeng Lin, Georgia Institute of Technology