LLM Agents

LLM agents are the default agent type in LeafMesh. They send a prompt to an LLM provider, parse the response against a yields schema, and evaluate can_call conditions for downstream routing. Intelligence functions add deterministic Python logic on top of the LLM response.

Basic Configuration

Execution Pipeline

When an LLM agent is called via leafmesh.mesh_call(), the following pipeline executes:

Input Data
    │
    ▼
Pre-compose pipeline (if registered)
    │  context_processor → input_processor → others_processor
    ▼
Prompt assembly
    │  System prompt + yields schema + conversation history + user input
    ▼
LLM call
    │  Calls the provider (OpenAI, Anthropic, etc.)
    │  Handles the tool-calling loop if tools are configured
    ▼
Yields parsing
    │  Parses LLM response against the yields schema
    ▼
Intelligence function (if registered)
    │  Function name matches agent name — deterministic Python
    ▼
Yields persisted
    │
    ▼
can_call evaluation
    │  AST-safe condition evaluation against yields
    ▼
Downstream agents (via mesh)

Model Selection

Specify any supported model directly in the agent's YAML:

Cloud gateway providers (Bedrock, Vertex AI, Microsoft Foundry) require project-level configuration in the mesh section. See LLM Providers for setup details.

You just specify the model name — LeafMesh routes to the correct provider for you. See LLM Providers for the full supported list.

Prompt Engineering

The prompt field defines the agent's system prompt. LeafMesh assembles it with the yields schema and conversation history into the message array sent to the LLM.

Context Parts

Beyond the main prompt, agents can include context_parts — additional system messages that shape agent behavior without cluttering the primary task instructions:

Each part is injected as a separate system message after the main prompt, with a bracketed label. The four reserved keys are:

All are optional. Custom keys are also accepted and auto-labelled. See Guardrails for patterns.

Effective Prompts

Tips:

• Be specific about the agent's role and output format
• Mention the yields fields by name so the LLM formats correctly
• Lower temperature (0.0–0.3) for deterministic tasks
• Higher temperature (0.5–0.8) for creative or generative tasks
• Keep prompts under 2000 characters for optimal performance

Intelligence Functions

Intelligence functions run after the LLM call. They receive the raw LLM response and can modify, enrich, or replace it entirely.

Enhancing LLM Output

Validating and Enriching

Context-Aware Intelligence Functions

Use input_data and context for session-aware responses. Previous agent yields are automatically passed as input_data through the mesh:

Session state, yields, and mesh communications are managed automatically by the mesh and persisted in Redis. Query state via the REST API at :18820.

Tool Integration

LLM agents can use tools through the YAML tools array. The LLM decides when and how to use tools based on the user's request.

Tool Execution Flow

1. Tool schemas are included in the LLM request
2. LLM generates a response that may include tool calls
3. LeafMesh executes the tool calls
4. Tool results are fed back to the LLM for a follow-up response
5. This loop continues until the LLM responds without tool calls, or max_tool_calls_per_message is reached

Two cases never end as an empty success, on every provider:

• A garbled tool call. When the model returns a tool call it could not format (Gemini can do this), it is told that its call was not run and asked for valid arguments, up to two more times. A call that is still garbled ends the turn as a failure: "The model's tool call could not be parsed…".
• Tools ran, but no answer. When the model's reply comes back empty after it ran tools, it is told so with its tools still on, and asked to carry on with the next step or write its answer. Two empty replies in a row is a failure. Only when no tools are left to call (the tool-call limit is reached, the budget is spent, or a tool kept failing) is it asked for its answer with tools off. A turn where no tool ran is unchanged.

A failed turn never signs off its board ticket.

Tool Choice Modes

See Tools for registering custom tools.

Tool blocks (zapier, composio, n8n, mcp), the playbook, follow-ups and their limits, image output, eager skills and the sandbox are for LLM agents only. Set on a human, external or programmatic agent, they are refused at load.

Running Code (Sandbox)

An LLM agent runs code only when you give it a sandbox. thinking: true does not run code: it offers three text tools, chain_of_thought, metacognitive_reflection and apply_edits.

The agent gets one tool, sandbox_exec(code, language). language is python (the default) or shell, and code is the complete program. The result is {stdout, stderr, exit_code, error, truncated}. Inside the sandbox the agent may run code, write files in its own fresh workspace and use the installed packages.

To widen a limit or allow a host, write a map. It changes only the fields it names:

What the sandbox is. A locked-down process on the same server, separate from the mesh. It has no environment (so no credentials or vault key), its fresh workspace is the only place it can write, and it cannot reach the mesh's files, memory, other processes or other tenants' data. It never runs in the mesh process.

Network is denied by default. With no network.allow, the code has no network. A listed host is reached through the mesh's own egress proxy; every other destination is refused.

The host decides the strength, and says so at boot. The mesh checks what the server can enforce and logs one line:

Linux and macOS are supported. On macOS the system sandbox gives the same guarantees, and a Mac reports full. Studio shows the current strength (GET /api/registry?category=sandbox). A Playground run cannot widen an agent's sandbox.

Upgrade note: an agent that ran code under thinking: true needs sandbox: true to run code again.

Conditional Routing (can_call)

After an agent produces yields, LeafMesh evaluates can_call conditions to determine which downstream agents are called:

Supported Condition Operators

Conditions are evaluated using the Python ast module — never eval().

Accessing Nested Object Yields

When yields: declares a single object field (e.g. qualification: object), conditions read it via dot-path: qualification.match_score. If your LLM prompt asks for a flat JSON shape without the wrapping key, the SDK auto-wraps the output so the dot-path still works. See Accessing Yields in Conditions for the exact rules and gotchas.

call_immediately

When call_immediately: true, the downstream agent is called synchronously as part of the current pipeline. Without it, downstream calls are dispatched asynchronously.

Direct Mesh Calls

From inside an intelligence function, call another agent directly:

Pre-Compose Pipeline

Run deterministic Python before the LLM call to control what the LLM sees:

Execution order:

1. context_processor — assemble business context
2. input_processor — clean and sanitize input
3. others_processor — supplementary data
4. Prompt is assembled from the prepared data
5. LLM call executes
6. Intelligence function runs

Dual Response Pattern

For agents that need to respond immediately while triggering background work:

With communication_type: "dual", LeafMesh returns the agent's response to the caller immediately, then dispatches can_call chains as background tasks.

Streaming Output

Dual-mode LLM agents can stream their response token-by-token to the caller via Server-Sent Events. The caller passes stream=True to mesh_call (or stream=true on POST /api/mesh/request); the SDK opens an SSE connection and tokens flow out as they're generated.

Three chunk kinds — token (incremental text), final (assembled response dict at the end), error (terminal failure with diagnostic).

Picking the yield to stream:

• If the agent declares one yield key, the SDK auto-picks it and stream_yield can stay unset.
• If the agent declares multiple yield keys, set stream_yield: "<key>" to pick which one streams.
• Other yields are still produced — they arrive in the final chunk's data dict.

Fallback behavior: Non-dual-mode agents (chain, execute) and non-LLM agents (programmatic / human / external) ignore the stream flag and return their full result as a single final chunk. Your SSE consumer code stays identical — no branching by agent type.

UI surface: Studio's Test Modal and Playground's session history both have a single toggle that flips the same stream=true flag when debugging an agent. Tokens land in real time so you can see the model's pace.

Configuration Reference

v2.2.24 — reasoning and thinking swapped (BREAKING)

As of v2.2.24, the two YAML flags have been swapped to align with how each major LLM provider names the feature. The keys themselves are unchanged, but their meanings have flipped:

Why: OpenAI / DeepSeek / xAI Grok all call the native feature "reasoning" (reasoning_effort, deepseek-reasoner, reasoning_content). Anthropic / Google call theirs "thinking". The swap aligns the dominant ecosystem term (reasoning) with the native-API concept; thinking becomes the in-the-loop SDK chain-of-thought scaffolding, matching the informal "think out loud" meaning.

No automatic config migration. The semantic flip means we can't safely guess what an existing reasoning: true was meant to do. On SDK startup, the v2.2.24 validator emits a loud warning when it sees reasoning: true on a model that isn't reasoning-capable — that's the strongest signal that a pre-v2.2.24 config needs to be flipped.

Migrating an existing config

For each LLM agent in your YAML:

From 2.4.481 the configuration lives in the agency's store. Make these changes in Studio's YAML view, or run this on a config.yaml before it is imported. Quick sed snippet (review the diff before committing):

The reasoning_budget rename is backward-compatible — if you leave thinking_budget in your YAML, the loader reads it as reasoning_budget and logs a deprecation warning once (numeric rename, no semantic change). If both are set, reasoning_budget wins.

Adaptive Model Selection

Adaptive Model Selection

When optimization_strategy is set, the SDK ignores the configured model field on every call and routes the request to a model picked by the adaptive predictor instead. The configured model is kept and used only as a fallback if the predictor's selection fails for any reason.

Strategies

Model catalog

The predictor considers every model in its calibrated benchmark catalog. Calibration is refreshed each SDK release; the current set covers OpenAI (GPT-6, GPT-5.6, GPT-5.x, GPT-4.1, gpt-4o, o3, o4-mini), Anthropic (Fable 5/5.1, Opus, Sonnet and Haiku 5.5, plus the 4.x models still offered), Google (Gemini 4 Argon, 3.8/3.6 Flash, 3.5 Flash-Lite, 3.1, 2.5), DeepSeek (deepseek-v4-pro, deepseek-flash), Mistral, xAI Grok (4.3 to 4.7), and the vendors with their own models (Kimi, Qwen, GLM, MiniMax, Mercury, Muse, StepFun, Arcee). Retired models such as deepseek-chat, grok-4 or gemini-2.0-flash are never proposed.

Only models from loaded providers are considered — if you only have an OpenAI API key configured, the predictor will only pick from OpenAI models, even with cost selected.

Observing the picked model

The Studio agent inspector and the Playground agent panel both display the model the predictor last picked for an agent. Programmatically, query the HTTP endpoint:

Aggregated analytics, cache metrics, and per-session event streams are available in Studio's Dashboard and session inspector — operator UI, not part of the customer Python or HTTP surface.

The endpoint returns "selection": null until the first adaptive call lands.

Next Steps

• Agent Types — All four agent types (LLM, programmatic, human, external)
• Tools — Extend agents with real-world capabilities
• Agent Configuration — Complete YAML reference
• LLM Providers — Multi-provider architecture

LeafMesh — LLM agents with deterministic control