Scaling Strategies

Approaches for scaling LeafMesh deployments to handle increased load.

Per-Instance Concurrency

Each LeafMesh instance runs as an async Python process. Within one instance:

All agents run in that process
Concurrency comes from async I/O (LLM calls, Redis operations, tool execution)
CPU-bound work blocks that instance's event loop

To go beyond one instance's capacity, scale out horizontally (below).

Horizontal Scaling with Shared Redis

Multiple LeafMesh instances can share the same Redis backend. Running more than one instance requires workers/HA mode: set LEAFMESH_WORKERS_MODE=1 on every instance and point them all at the same Redis. In that mode internal events are delivered exactly once across instances, run-once jobs (the scheduler, event listeners) run on a single instance at a time, session state is locked across instances, and human-in-the-loop waits survive an instance restart. Without it, run a single instance.

                    ┌──────────┐
                    │  Redis   │
                    │ (shared) │
                    └────┬─────┘
                    ┌────┴─────┐
              ┌─────┤          ├─────┐
              │     │          │     │
         ┌────┴───┐│    ┌────┴───┐  │
         │ LeafMesh││    │ LeafMesh│  │
         │Instance││    │Instance│  │
         │   1    ││    │   2    │  │
         └────────┘│    └────────┘  │
                   │                │
              ┌────┴───┐      ┌────┴───┐
              │ LeafMesh│      │ LeafMesh│
              │Instance│      │Instance│
              │   3    │      │   4    │
              └────────┘      └────────┘

Each instance:

Loads the same YAML configuration
Connects to the same Redis instance/cluster
Handles different sessions independently
Shares auto-stored yields, mesh data, and session state

Redis Cluster

For high-volume deployments, use Redis Cluster. From 2.4.482 the store is set only in .env, never in the YAML; listing the nodes in REDIS_CLUSTER_NODES is what turns cluster mode on:

# .env
LEAFMESH_AGENCY_NAME=my-agency
LEAFMESH_STORE=redis
REDIS_CLUSTER_NODES=redis-1:6379,redis-2:6379,redis-3:6379
REDIS_PASSWORD=your-password

Every instance should get the same .env values. Redis TLS (REDIS_SSL, REDIS_SSL_CA_CERTS, ...) and Postgres (POSTGRES_*, with POSTGRES_POOL_MIN / POSTGRES_POOL_MAX sizing the connection pool) are set the same way; see Persistence for the full list. Studio shows whether the store is reachable under Infrastructure → Datastores, which also shows its latency, the server, the settings in force and which .env names set them — never the address or password.

LeafMesh handles node routing transparently. Data is distributed across cluster nodes based on key hash slots.

Load Balancing Entry Points

Use the FastAPI services behind a load balancer:

from leafmesh import LeafMesh

leafmesh = LeafMesh.from_store()

# Each instance runs its own API server
# Load balancer distributes requests across instances
await leafmesh.start()
api_server = leafmesh.api_server

Entry points provide stable names for external systems, while the load balancer distributes across LeafMesh instances.

Scaling Considerations

Concern	Approach
Workers mode	Required for >1 instance — `LEAFMESH_WORKERS_MODE=1` on every instance + one shared Redis
Lease window	`runtime.lease_seconds` `(default 30) — singleton-runner lease TTL. Raise it (e.g.` 120`) when Redis is remote over a flaky link, so connectivity blips shorter than the window never flap the leased runners
Session affinity	Not required — any instance can serve any session; workers mode locks session state across instances. Place instances behind any round-robin load balancer.
Agent state	Shared via Redis — any instance can process any session
Event processing	In workers mode, internal events are delivered exactly once across the fleet
LLM rate limits	Distribute agents across instances to spread API calls

Provider-Based Scaling

Distribute load across LLM providers:

agents:
  # High-volume agents on fast, cheap models
  triage_1:
    name: "triage_1"
    model: "gpt-6-luna"
  triage_2:
    name: "triage_2"
    model: "gemini-3.6-flash"

  # Complex agents on capable models
  specialist:
    name: "specialist"
    model: "claude-sonnet-5-5"

Different providers have independent rate limits, so spreading agents across providers increases total throughput.

Current Limitations

No built-in load balancer — front instances with your platform's load balancer (App Runner / Container Apps provide one; on Kubernetes use a Service + Ingress).
Multi-instance requires LEAFMESH_WORKERS_MODE=1 and one shared Redis; without it, run a single instance.
Horizontal scaling is driven by your orchestrator (instance count on App Runner / Container Apps via leafmesh-deploy, or replicas on Kubernetes).
Redis becomes the bottleneck for extremely high-volume deployments.

Next Steps

Docker Deployment — Container deployment
Kubernetes Deployment — Kubernetes orchestration
Redis Integration — Redis configuration

LeafMesh — Scaling multi-agent deployments