Persona Evolution

Persona evolution is an automated A/B testing system that continuously improves agent system prompts by testing variants against each other and promoting the best performers. Agents gradually evolve more effective personas without manual prompt engineering.

How it works

Persona evolution operates in a continuous cycle:

1. Variant generation — The evolution system generates 2–4 alternative versions of the agent's system prompt, with controlled variations in tone, verbosity, reasoning style, or domain emphasis
2. A/B assignment — Incoming sessions are randomly assigned to the current champion prompt or one of the challenger variants
3. Metric collection — User satisfaction signals (explicit ratings, implicit signals like conversation length and task completion) are collected for each variant
4. Statistical evaluation — After reaching a minimum sample size, variants are evaluated for statistical significance
5. Promotion — If a challenger variant outperforms the champion by the configured margin, it is automatically promoted as the new champion

Configuration

evolution:
  enabled: true
  evaluationWindowDays: 7      # Collect data over 7 days before evaluating
  minSampleSize: 50            # Minimum sessions per variant before evaluation
  improvementThreshold: 0.05   # Require 5% improvement to promote (avoids noise)
  maxVariants: 4               # Max concurrent variants to test
  autoPromote: true            # Automatically promote winners without human review
  rollbackOnDrop: true         # Revert if champion performance drops after promotion

Evaluation metrics

Persona evolution uses a composite score based on several signals:

Viewing and managing variants

# List all active variants for an agent
GET /agents/:id/evolution/variants

# Get the current champion prompt
GET /agents/:id/evolution/champion

# Manually promote a specific variant
POST /agents/:id/evolution/promote/:variantId

# Pause evolution for an agent
POST /agents/:id/evolution/pause

Safety guards

To prevent persona evolution from producing harmful or off-brand prompts, several safety guards are applied:

• Generated variants are evaluated against SOUL.md constraints before being activated
• No variant can change the agent's core identity, tool permissions, or spawn rules
• If rollbackOnDrop is enabled and the promoted variant shows declining metrics within 24 hours, it is automatically rolled back
• All variant changes are logged and auditable via GET /agents/:id/evolution/history