Runtime Optimization

Techniques for optimizing LeafMesh performance in production.

LLM Response Caching

LeafMesh includes built-in LLM caching that deduplicates identical requests — reusing only the model's text; your agent function and decorators always run fresh (see Caching). The cache lives in the configured store, which from 2.4.482 is set only in .env, never in the YAML:

# .env
LEAFMESH_STORE=redis

Cache hits are served directly, saving both latency and cost. Hit rate and estimated savings are visible in the dashboard.

# Check cache effectiveness
cache_stats = Studio's Dashboard (HTTP)
print(f"Hit rate: {cache_stats['hit_rate']}")
print(f"Cost savings: ${cache_stats['estimated_savings']}")

Model Selection Optimization

Use the adaptive model selection to automatically pick the best model for each request:

agents:
  # Fast, cheap model for simple classification
  triage:
    name: "triage"
    model: "gpt-4o-mini"
    optimization_strategy: "cost"

  # More capable model for complex analysis
  specialist:
    name: "specialist"
    model: "gpt-4o"
    optimization_strategy: "performance"

  # Balanced approach
  general:
    name: "general"
    model: "claude-sonnet-4-6"
    optimization_strategy: "performance"

LeafMesh categorizes requests and selects models based on the configured strategy. See Adaptive Execution for details.

Redis TTL Tuning

Adjust TTL values based on your access patterns:

# .env
LEAFMESH_STORE=redis
# 1 hour for agent yields
REDIS_DEFAULT_TTL=3600
# 7 days for sessions (the default)
REDIS_SESSION_TTL=604800

From 2.4.482 the store is set only in .env, never in the YAML.

Short TTL (< 1h): High-throughput systems where data is consumed quickly
Moderate TTL (1-2h): Standard interactive sessions
Long TTL (24h+): Baseline data and configurations that change slowly
No TTL: Global configuration data that should persist indefinitely

Token Optimization

Control token usage per agent:

agents:
  classifier:
    name: "classifier"
    max_tokens: 500           # Short responses for classification
    temperature: 0.1          # Low randomness for consistent output

  writer:
    name: "writer"
    max_tokens: 2000          # Longer responses for content
    temperature: 0.7          # Higher creativity

Reducing Mesh Latency

Use `call_immediately`

For chains that should always fire, skip waiting for the current response to fully process:

can_call:
  - agent: "logger"
    condition: "true"
    call_immediately: true    # Fire without waiting

Limit Call Depth

Prevent deep chains that add latency:

manager:
  coordination_rules:
    max_agent_calls: 10       # Limit total calls per session

Monitoring Performance

# Overall system analytics
analytics = Studio's Dashboard (HTTP)
print(f"Events processed: {analytics['total_events']}")
print(f"Error rate: {analytics['error_rate']}")

# Per-agent performance
agent_stats = leafmesh.get_agent_stats()
for agent, stats in agent_stats.items():
    print(f"{agent}: avg {stats['avg_response_ms']}ms, errors: {stats['error_count']}")

Performance Characteristics

Component	Overhead
Request routing	~10-50ms
Condition evaluation	~1ms
LLM call	500ms-5s (provider-dependent)
Tool execution	Varies by tool

The LLM call dominates request latency. Optimization efforts should focus on model selection, prompt efficiency, and caching rather than framework-level tuning.

Next Steps

Adaptive Execution — Automatic model selection
Redis Integration — Redis configuration
Scaling — Horizontal scaling patterns

LeafMesh — Production performance tuning