# Liquid LFM2.5-2.6B: On-Device Agent Guide

> Run LFM2.5-2.6B locally with llama.cpp, MLX, or vLLM; connect an agent harness, budget context, test tool calls, and preserve data boundaries.

- **Published**: 2026-08-07
- **Category**: AI Infrastructure
- **URL**: https://agentpedia.codes/blog/liquid-lfm2-5-2-6b-on-device-agent-guide

---

**Liquid LFM2.5-2.6B** is a compact text model post-trained for tool calling and multi-step agent work. Liquid AI distributes native, GGUF, MLX, and ONNX variants and documents connecting the model to agent harnesses through a local OpenAI-compatible endpoint.

The practical opportunity is not "a frontier model on a phone." It is a small, responsive model for bounded local workflows where privacy, latency, offline operation, or high request volume matters more than maximum reasoning and coding quality.

> **Note callout**

Liquid AI released LFM2.5-2.6B on August 4, 2026. The launch speed and benchmark results are vendor-run. "On-device" describes model inference; an agent can still transmit data through web search, email, telemetry, remote MCP servers, or other tools.

**Fast decision:**

- Start with `Q4_K_M` on constrained hardware and increase precision after measuring tool reliability.
- Serve less than the full 128K context when memory or time-to-first-token matters.
- Enable the runtime's LFM2 tool-call parser or chat template explicitly.
- Use a larger fallback model for difficult coding, ambiguous planning, or repeated tool failures.

## What LFM2.5-2.6B is

LFM2.5-2.6B is the post-trained member of a two-checkpoint release:

| Checkpoint | Intended role |
| --- | --- |
| `LiquidAI/LFM2.5-2.6B-Base` | Pre-trained base for fine-tuning and research |
| `LiquidAI/LFM2.5-2.6B` | Post-trained model for chat, tools, and agent workflows |

The model card lists 2.69 billion parameters, 30 layers, a 128,000-token vocabulary, and a 131,072-token context window. The architecture combines 22 double-gated short-convolution blocks with eight grouped-query-attention layers. It is a **text-only** model; image or document understanding requires a separate tool or model.

> Today we release LFM2.5-2.6B, an agentic model that runs entirely on-device. It plans, calls tools, and works through multi-step tasks on phones, laptops, PCs, and robots.
>
> -- [@liquidai, August 4, 2026](https://x.com/liquidai/status/2084640701669613906)

Liquid says pre-training used approximately 34 trillion tokens. Its post-training pipeline combines supervised fine-tuning, specialist teachers, on-policy distillation, and reinforcement learning inside agent environments. The company names Hermes Agent, OpenClaw, and other harnesses as training or integration targets.

That does not guarantee identical behavior across harnesses. System prompts, tool schemas, context trimming, error messages, runtime parsers, and sampling parameters all influence whether a small model calls tools correctly or merely describes an intended action. The model card also describes an internal reasoning section before the answer; applications should parse only the documented assistant and tool-call structure, avoid exposing raw reasoning by default, and never treat hidden reasoning as an auditable decision record.

The Hugging Face repository identifies the license as the **LFM Open License v1.0**, a custom model license rather than Apache 2.0, MIT, or an OSI software license. Its published threshold requires organizations with annual revenue of **US$10 million or more** to obtain a commercial license from Liquid AI; redistribution also carries license and notice obligations. Review the current terms for your organization and deployment.

## Choose a local runtime

Liquid publishes several formats because "local" covers different hardware and operational goals.

| Runtime or format | Best fit | Important trade-off |
| --- | --- | --- |
| llama.cpp / GGUF | Laptops, desktops, CPU and mixed CPU-GPU use | Quantization and served context affect quality and memory |
| LM Studio / GGUF | Interactive desktop evaluation | Convenient UI, but configuration should be exported and versioned for repeatability |
| MLX | Apple Silicon | Apple-specific runtime and quantization |
| ONNX | Edge accelerators, supported CPUs/GPUs, WebGPU experiments | Backend support and tool-template behavior vary |
| vLLM | GPU server and concurrent OpenAI-compatible serving | Requires explicit LFM2 tool-call parsing |
| SGLang | GPU serving and agent workloads | Requires the matching tool-call parser and supported version |

Liquid's local-agent guide recommends `Q4_K_M` as the starting GGUF, with a documented file size of 1.67 GB. It lists `Q6_K` at 2.22 GB, `Q8_0` at 2.87 GB, and BF16 at 5.4 GB. Weight size is not total runtime memory: the key-value cache, context, runtime buffers, concurrency, and GPU offload add overhead.

Serving the full 128K window can be unnecessary and expensive. Begin with 8K or 32K for bounded tasks, observe truncation and cache use, and raise the limit only when traces show the workflow needs it.

## Serve it with llama.cpp

A practical cross-platform starting point is the official GGUF repository. Liquid's guide uses the following pattern:

```bash
llama-server \
  -hf LiquidAI/LFM2.5-2.6B-GGUF:Q4_K_M \
  --jinja \
  --host 127.0.0.1 \
  --port 8080 \
  -c 32768 \
  --temp 0.1 \
  --top-k 50 \
  --repeat-penalty 1.1
```

Expected output: llama.cpp downloads the selected quant if needed, loads it, and exposes an OpenAI-compatible endpoint on the loopback interface at port `8080`.

The `--jinja` flag matters because it enables the model's chat template and tool-call formatting. A local server that returns natural-language answers but malformed or missing tool calls is often a parser or template configuration problem, not just a model-quality problem.

Confirm the endpoint before connecting an agent:

```bash
curl -fsS http://127.0.0.1:8080/v1/models
```

Expected output: a JSON model list containing the served LFM2.5 identifier.

For GPU runtimes, Liquid documents these key flags:

```bash
vllm serve LiquidAI/LFM2.5-2.6B \
  --enable-auto-tool-choice \
  --tool-call-parser lfm2
```

```bash
sglang serve \
  --model-path LiquidAI/LFM2.5-2.6B \
  --host 127.0.0.1 \
  --port 30000 \
  --tool-call-parser lfm2
```

Pin known-good runtime versions for production. "Latest" is useful during evaluation but makes regressions in tokenization, templates, parser behavior, or kernels harder to reproduce.

## Connect an agent harness

Any harness that supports a custom OpenAI-compatible endpoint can use the local server. The essential configuration is:

- provider type: custom or OpenAI-compatible;
- base URL: the local server's `/v1` path;
- model ID: the identifier the server returns;
- context limit: no larger than the server actually exposes;
- API mode: compatible chat completions;
- tool-use behavior: enabled and tested.

Liquid's Hermes example uses the model selector for an interactive setup:

```bash
hermes model
```

Choose a custom self-hosted endpoint, enter the loopback interface on port `8080` with the `/v1` path, leave the API key empty for a loopback-only keyless server, and use the model identifier returned by `/v1/models`.

Before granting real tools, create a synthetic canary set:

1. one tool with two required string arguments;
2. one read-only tool returning structured JSON;
3. one intentionally failing tool;
4. one request that should not call any tool;
5. one ambiguous request that should ask a clarifying question;
6. one sensitive write action that requires approval.

Measure whether the model chooses the right tool, emits valid arguments, respects errors, avoids duplicate calls, and stops after success. Tool-call syntax alone is not task success.

## Budget context and tools

A 128K advertised window is a capacity ceiling, not a recommendation to fill every request. Small local models can spend substantial time ingesting long prompts and may become less reliable when stale traces and irrelevant tools dominate the context.

Use these controls:

- Load only tool schemas relevant to the current task.
- Summarize completed tool traces and retain structured outcomes.
- Cap large web pages, logs, and documents before they enter the prompt.
- Keep immutable system and safety instructions close to the model's expected template.
- Reject recursive or repeated tool calls with the same arguments.
- Set per-task tool-call, token, time, and network budgets.
- Create a larger-model escalation path after repeated parse or reasoning failures.

A routing policy can preserve local operation for routine work:

| Task | Default route |
| --- | --- |
| Local file classification or extraction | LFM2.5 locally |
| Simple calendar or note action with approval | LFM2.5 locally |
| Bounded research using read-only tools | LFM2.5 with strict budgets |
| Repository-wide architecture change | Larger coding model |
| High-impact financial, account, or publishing action | Human approval regardless of model |
| Repeated malformed or looping tool calls | Stop and escalate; do not retry indefinitely |

## Read the benchmarks correctly

Liquid reports competitive results against several larger Gemma and Qwen checkpoints across instruction following, tool use, and agent benchmarks. The launch table includes BFCLv4, ToolSandbox, Claw-Eval, PinchBench, BrowseComp+, math, coding, and instruction-following evaluations.

Liquid also reports 220 output tokens per second on an Apple M5 Max, 113 on a Ryzen AI Max+ 395, approximately 30 on a phone, and high-throughput H100 results. The three on-device figures used `Q4_K_M` with a 4K-token input context; the associated sub-2.5 GB memory measurements do not describe a full 128K-context deployment. These are useful indications that the architecture can be efficient, but they are not independent cross-platform guarantees.

When reproducing performance, record:

- exact device, power mode, runtime, commit or version, and quantization;
- prompt length, output length, concurrency, and context limit;
- time to first token and decode throughput separately;
- peak resident and GPU memory;
- correct tool selection and valid arguments;
- end-to-end task completion, retries, and human interventions;
- quality changes between Q4, Q8, and full precision.

Vendor benchmark leadership can coexist with poor performance on a specific task. Liquid itself notes that larger models retain an edge on more complex agentic or coding-heavy work.

## Preserve local data boundaries

A model server on `127.0.0.1` keeps inference traffic local. The agent around it may not.

Audit every component:

- **Tools:** web search, email, cloud drives, remote browsers, hosted MCP servers, and analytics can transmit content.
- **Model downloads:** fetch from the canonical publisher and pin a revision or digest for repeatability.
- **Server exposure:** do not bind a keyless OpenAI-compatible endpoint to a public interface.
- **Prompt injection:** treat documents, pages, email, and tool results as untrusted data, not instructions.
- **Filesystem scope:** allowlist directories and keep secrets outside the agent's readable workspace.
- **Writes:** require approval for publishing, deleting, purchasing, messaging, account changes, and code execution outside a sandbox.
- **Logs:** prompts and tool results can contain personal data, tokens, or proprietary documents; minimize and redact retention.
- **Updates:** re-run tool and safety canaries when the model, quant, runtime, template, or harness changes.

"Data never leaves the device" is true only when the model, prompts, retrieval corpus, tools, telemetry, and outputs all remain local. Document the actual boundary instead of treating local inference as a blanket privacy guarantee.

## Verdict

LFM2.5-2.6B is a strong candidate for bounded local agents: personal automation, private extraction, routine tool use, offline workflows, and high-volume tasks where a cloud token bill or network round trip is undesirable.

Its size is also its boundary. Use measured tool canaries, modest context, strict budgets, and a larger-model fallback. A good deployment does not force the local model to solve every task; it routes the work that the model can complete reliably while keeping high-impact actions behind explicit authorization.

## FAQ

## FAQ

### What is LFM2.5-2.6B?

LFM2.5-2.6B is Liquid AI's 2.69-billion-parameter text model post-trained for tool use and multi-step agent tasks. It supports a 128K context window and is distributed in native, GGUF, MLX, and ONNX forms.

### Can LFM2.5-2.6B run on a phone or laptop?

Yes, depending on the runtime, quantization, available memory, and desired context. Liquid documents a 1.67 GB Q4_K_M GGUF as a practical starting point and reports a phone demonstration, but real speed and tool reliability should be measured on the target device.

### Does a local LFM2.5 agent keep all data private?

Only if its entire execution path is local. Model inference can stay on the device, while web search, email, cloud storage, telemetry, remote MCP servers, and other tools may still transmit data. Audit every tool and network route separately.

### Which runtime should I use for LFM2.5-2.6B?

Start with llama.cpp or LM Studio for cross-platform local use, MLX on Apple Silicon, ONNX for supported edge accelerators and WebGPU experiments, and vLLM or SGLang for GPU serving and higher concurrency.

### Are Liquid AI's LFM2.5 benchmark and speed claims independent?

No. The launch benchmark, model comparison, and device throughput figures are first-party Liquid AI results. Reproduce latency, memory, task success, malformed tool calls, and recovery behavior on your own hardware and agent workload.


## Sources and links

- [Liquid AI: LFM2.5-2.6B launch](https://www.liquid.ai/blog/lfm2-5-2-6b)
- [Liquid AI model documentation](https://docs.liquid.ai/lfm/models/lfm25-2.6b)
- [Liquid AI local agent-harness guide](https://docs.liquid.ai/examples/agent-harnesses)
- [LFM2.5-2.6B model card](https://huggingface.co/LiquidAI/LFM2.5-2.6B)
- [Liquid AI launch post on X](https://x.com/liquidai/status/2084640701669613906)


---

- [All articles](https://agentpedia.codes/blog)