Inkling-Small Guide: Open Weights, Hardware and Agent Benchmarks

Thinking Machines released Inkling-Small on July 30, 2026. The model combines text, image and audio input, text output, variable reasoning effort, and a context window of up to one million tokens. This guide turns the launch and model card into an evaluation plan. It does not independently reproduce the provider's benchmarks or validate every serving recipe.

The practical verdict

Inkling-Small fits a fairly specific deployment envelope. It is more accessible than the provider's larger Inkling model, but it is not a workstation model. Its sparse Mixture-of-Experts design activates about 12B parameters per token while retaining 276B total parameters. Sparse activation lowers per-token compute; it does not remove the need to store the expert weights.

Use this decision table before downloading a checkpoint:

For nearby deployment comparisons, the Poolside Laguna S 2.1 guide shows how another sparse coding model's active-parameter label differs from its real memory footprint.

Model and license boundaries

The official model card is the best source for architecture facts:

The Hugging Face pages identify the repositories as Apache-2.0, but that metadata is not the whole review surface. The provider's model card separately links the Thinking Machines Model Acceptable Use Policy, which says it governs access to and use of the model weights, parameters, associated materials and modified versions and may be updated. Do not state unqualified Apache rights or assume the AUP is irrelevant; counsel and policy owners should review the pinned repository materials together with the AUP in effect for the intended use.

The Hugging Face interface may also show generated parameter metadata that disagrees with the provider's disclosed count. Use 276B total / 12B active from the model card as authoritative. Interface metadata is useful for discovery, not for overriding a provider architecture statement.

“Open weights” also has a narrower meaning than “the complete system is reproducible.” Thinking Machines describes training data as a mixture of public, third-party and synthetic or augmented sources. The release does not provide a complete training corpus or a recipe that lets a third party recreate the checkpoint bit-for-bit.

Choose a deployment path

The first choice is not SGLang versus vLLM. It is whether your evaluation needs weight custody.

A sensible sequence is hosted screening, NVFP4 validation, then BF16 only if quantization deltas matter enough to justify the cluster. Keep the prompt set, tool schema and grading rubric stable across those stages.

The model card names SGLang, vLLM, TokenSpeed, Unsloth and Hugging Face as supported deployment frameworks or recipes. That list establishes provider-described routes; it does not establish equivalent feature coverage, performance or security defaults across them.

Hardware and memory without the 12B trap

Thinking Machines publishes minimum aggregate VRAM figures rather than a promise about usable context or batch size:

These are lower bounds for a supported configuration, not a batch-and-context guarantee. Loaded weights compete with KV cache, modality preprocessing, communication buffers, CUDA graphs, allocator fragmentation and the inference engine itself.

Use this worksheet before provisioning:

checkpoint mode:
GPU model and count:
aggregate physical VRAM:
free VRAM after driver/runtime:
target input tokens:
maximum generated tokens:
concurrent sequences:
image/audio inputs:
tensor/expert parallelism:
measured peak VRAM:
failure margin:

Start with a short text-only request and one sequence. Add tools, modalities, context and concurrency one axis at a time. A server that loads successfully but OOMs during a long tool trajectory is not a valid deployment.

Serving framework boundaries

The official vLLM Inkling-Small recipe requires a nightly vLLM build with Inkling support. Its B300 NVFP4 W4A4 starting configuration is:

# Official B300 W4A4 starting configuration; not executed for this article.
export VLLM_USE_V2_MODEL_RUNNER=1
export FLASH_ATTENTION_CUTE_DSL_CACHE_ENABLED=1

vllm serve thinkingmachines/Inkling-Small-NVFP4 \
  --tokenizer-mode inkling \
  --reasoning-parser inkling \
  --tool-call-parser inkling \
  --enable-auto-tool-choice \
  --tensor-parallel-size 1 \
  --kernel-config.enable_flashinfer_autotune=False \
  --trust-remote-code

This command was not executed for the article. The same recipe says H200 uses TP2 in W4A16 mode, with the NVFP4 weights dequantized to BF16 on the fly. Before production, pin the model revision and nightly runtime build, bind the service to a private interface, require authentication, and set explicit limits for input size, output tokens and concurrency. Confirm that the chosen runtime correctly preserves reasoning/tool state and supports the modality mix you need.

Read the agent benchmarks as system results

Thinking Machines reports its own evaluations and also imports some external scores. The key coding results are system measurements, not model-only constants.

The provider says all its evals run at effort 0.99 and temperature 1.0, and coding evals allow a 256K maximum-token trajectory. Long trajectories can materially change success, latency and cost. The Terminal-Bench footnote also says a small number of solutions found to be contaminated through web search were scored zero. That is useful disclosure, but it does not turn the overall result into an independent audit.

For a stronger internal study, use the validation structure in the coding agents for scientific software guide. For security-oriented agent claims, the Fugu orchestration and security benchmark guide explains why model, scaffold, permissions and containment must be reported together.

Run a bounded smoke test

The following is a verification plan, not tested output from this article.

1. Pin the exact checkpoint, tokenizer, processor and serving container.
2. Start with text-only generation at low context and one concurrent request.
3. Record startup time, idle VRAM, first-token latency, tokens per second and peak VRAM.
4. Submit a deterministic repository-navigation task with no write tools.
5. Add a read-only shell tool and verify arguments against an allowlist.
6. Run an image-understanding fixture, then a short 16 kHz WAV fixture.
7. Increase reasoning effort and record output/reasoning-token changes.
8. Run the same task at 32K, 128K and your intended context; do not jump straight to 1M.
9. Introduce a recoverable tool failure and check retry, timeout and state preservation.
10. Only then allow writes in a disposable repository with tests and diff review.

Use a compact result sheet:

Deploy with bounded authority

The model card reports internal safety evaluations and external red teaming. Those provider-reported results do not replace deployment controls, especially when the model can call tools.

• Run inference and tools in separate, least-privilege security boundaries.
• Keep credentials out of prompts, model-visible files and tool output.
• Default network egress to denied; allow only task-required destinations.
• Require human approval for destructive, financial, identity or production actions.
• Validate tool arguments independently of model text.
• Cap steps, wall time, output tokens, concurrency and spend per run.
• Make mutation tools idempotent or attach unique operation keys.
• Log prompts, tool decisions, executed actions and results with appropriate redaction.
• Test prompt injection through repository files, web results, images and audio.
• Provide a server-side kill switch that revokes active credentials and stops workers.
• Re-run the smoke suite after any checkpoint, runtime, prompt or tool-schema change.

Open weights move some controls into your hands. They do not remove the risks of unsafe permissions, untrusted inputs or poorly contained tools.

Adopt, evaluate or wait

Evaluate now if you have a documented 180 GB or larger GPU path, need multimodal input plus agentic tool use, and can run a controlled comparison against an existing system.

Adopt after evidence if the pinned checkpoint meets your task-success, latency, memory, safety and operational-cost thresholds under the same harness you will deploy.

Wait or use hosted access if your hardware is smaller than the provider minimum, you need an independently reproduced leaderboard result, or your runtime cannot yet support the required NVFP4 mode. A model that fits only by relying on an undocumented quantization is a separate artifact with a separate quality and licensing review.

Official sources

This guide uses first-party launch, model-card and model-repository sources. Benchmark values remain provider-reported unless the source row says otherwise.

Thinking Machines Lab

• Introducing Inkling-Small — July 30 release, architecture summary, capabilities, evaluation settings and benchmark table
• Inkling-Small model card — Apache license label, linked Model AUP, modalities, architecture, minimum hardware, distribution and safety disclosures
• Thinking Machines Model Acceptable Use Policy — separate use restrictions and update terms for model materials
• Inkling-Small on Hugging Face — BF16 repository, Apache-2.0 metadata and framework starting examples
• Inkling-Small NVFP4 on Hugging Face — quantized repository and framework starting examples
• Official vLLM Inkling-Small recipe — nightly requirement and architecture-specific B300, B200 and H200 configurations

Related AgentPedia guides

• Poolside Laguna S 2.1 serving and benchmark guide
• Validating coding agents for scientific software
• Fugu orchestration security and benchmark guide