# IFM Documentation (full)

---

<!-- Source: https://docs.ifm.ai/docs/overview.md -->

# Build with IFM models

Explore IFM's foundation model portfolio. Review model capabilities, access methods, licensing terms, and deployment options to identify the right model for your needs.

[Get API key](https://platform.ifm.ai/api-keys) · [Explore models →](https://docs.ifm.ai/docs/model-catalog.md)

## Featured models

### K2 Horizon · K2 family · New

Frontier agentic reasoning and tool use. Six open-weight sizes.

[View model →](https://docs.ifm.ai/docs/introduction.md) · [Weights →](https://huggingface.co/collections/IFM/k2-horizon)

### Jais 3 · Jais family · New

K2 Horizon architecture specialized for Arabic — multiple Arabic dialects plus four more languages.

[View model →](https://docs.ifm.ai/docs/jais-3.md)

## Next steps

- [Quickstart](https://docs.ifm.ai/docs/quickstart.md): Project key, base URL, and your first chat completion in three steps.
- [Explore IFM research](https://ifm.ai): Research, model releases, and publications from the Institute of Foundation Models.

_Last updated: October 14, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/quickstart.md -->

# Quickstart

Start building with hosted IFM models in three steps.

## Create an API key

Create an API key in the [IFM Platform](https://platform.ifm.ai/api-keys). Store it securely.

```
export IFM_API_KEY="IFM-xf..."
```

## Configure your environment

All IFM models share the same base URL. Choose a hosted model that best fits your needs from the [Model overview](https://docs.ifm.ai/docs/model-catalog.md).

```
export IFM_BASE_URL="https://api.ifm.ai/v1"
```

```
export IFM_MODEL="IFM/K2-Horizon-375B-A23B"
```

## Make the request

See [Parameters](https://docs.ifm.ai/docs/chat-completions.md) for more details about the requests.

```
curl $IFM_BASE_URL/chat/completions \
  -H "Authorization: Bearer $IFM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "'$IFM_MODEL'", "messages": [{"role": "user", "content": "hello"}]}'
```

## Response

A JSON response — the reply text is at `choices[0].message.content`:

```
{
  "choices": [{
    "message": {
      "content": "Hello! How can I help you today?"
    }
  }]
}
```

The full response schema is defined in [Chat completions API](https://docs.ifm.ai/docs/chat-completions.md).

_Last updated: October 13, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/model-catalog.md -->

# Model overview

Explore IFM models designed for diverse needs — from compact, efficient models to larger, more powerful options. Compare capabilities, model sizes, and multilingual support to find the right fit for your use case.

**K2-Think-V2** API access retired on **3 October 2026**. Please migrate to **IFM/K2-Horizon-375B-A23B**. Model weights remain available on Hugging Face. See [Deprecation](https://docs.ifm.ai/docs/deprecation.md) for migration steps.

## Model list

| Series | Model | Context | Languages | API |
| --- | --- | --- | --- | --- |
| [K2 Horizon](https://docs.ifm.ai/docs/introduction.md) ([Hugging Face](https://huggingface.co/collections/IFM/k2-horizon)) | `IFM/K2-Horizon-375B-A23B` | 512K | English, Arabic, 30+ more | Yes |
| K2 Horizon | `IFM/K2-Horizon-MoVA-36B-A4B` | 512K | English, Arabic, 30+ more | No |
| K2 Horizon | `IFM/K2-Horizon-32B` | 512K | English, Arabic, 30+ more | No |
| K2 Horizon | `IFM/K2-Horizon-7B` | 512K | English, Arabic, 30+ more | No |
| K2 Horizon | `IFM/K2-Horizon-3.7B` | 512K | English, Arabic, 30+ more | No |
| K2 Horizon | `IFM/K2-Horizon-0.9B` | 128K | English, Arabic, 30+ more | No |
| [Jais 3](https://docs.ifm.ai/docs/jais-3.md) ([Hugging Face](https://huggingface.co/collections/IFM/jais-3)) | `IFM/Jais-3-375B-A23B` | 512K | Arabic + Multiple Arabic Dialects + English | No |
| Jais 3 | `IFM/Jais-3-MoVA-36B-A4B` | 512K | Arabic + Multiple Arabic Dialects + English | Yes |
| Jais 3 | `IFM/Jais-3-32B` | 512K | Arabic + Multiple Arabic Dialects + English | No |
| Jais 3 | `IFM/Jais-3-7B` | 512K | Arabic + Multiple Arabic Dialects + English | No |
| Jais 3 | `IFM/Jais-3-3.7B` | 512K | Arabic + Multiple Arabic Dialects + English | No |
| Jais 3 | `IFM/Jais-3-0.9B` | 128K | Arabic + Multiple Arabic Dialects + English | No |
| [Jais ASR](https://docs.ifm.ai/docs/jais-asr.md) ([Hugging Face](https://huggingface.co/collections/IFM/jais-speech)) | `IFM/Jais-ASR` | — | Arabic + Multiple Arabic Dialects, +4 more | Yes |

Older series (open weights only, no hosted API): [K2 V2](https://docs.ifm.ai/docs/k2-v2.md), [Jais 2](https://docs.ifm.ai/docs/jais-2.md).

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/introduction.md -->

# K2 Horizon

K2-Horizon is IFM's frontier reasoning series, built for agentic tool use and long-horizon tasks. Six open-weight sizes run from on-device to datacenter; the 375B sparse MoE is the model served on the hosted API.

- **Weights license:** Apache 2.0
- **Hugging Face collection:** [IFM/K2-Horizon](https://huggingface.co/collections/IFM/k2-horizon)
- **Pre-train data cutoff:** December 2025

## Model series

Six sizes spanning dense, MoVA + MoE, and sparse MoE architectures. Every size is released as open weights; the 375B is the one served on the hosted API. [Explore the collection on Hugging Face →](https://huggingface.co/collections/IFM/k2-horizon)

| Model | Architecture | Context | Best for | API | Weights |
| --- | --- | --- | --- | --- | --- |
| IFM/K2-Horizon-375B-A23B | Sparse MoE | 512K | Frontier reasoning and long-horizon agents | [Yes](https://docs.ifm.ai/docs/chat-completions.md) | [View](https://huggingface.co/IFM/K2-Horizon-375B-A23B) |
| IFM/K2-Horizon-MoVA-36B-A4B | MoVA + MoE | 512K | Production-style serving experiments | No | [View](https://huggingface.co/IFM/K2-Horizon-MoVA-36B-A4B) |
| IFM/K2-Horizon-32B | Dense decoder-only | 512K | Long-context and reasoning experiments | No | [View](https://huggingface.co/IFM/K2-Horizon-32B) |
| IFM/K2-Horizon-7B | Dense decoder-only | 512K | Fine-tuning and cost-conscious deployment | No | [View](https://huggingface.co/IFM/K2-Horizon-7B) |
| IFM/K2-Horizon-3.7B | Dense decoder-only | 512K | Efficient research and single-node serving | No | [View](https://huggingface.co/IFM/K2-Horizon-3.7B) |
| IFM/K2-Horizon-0.9B | Dense decoder-only | 128K | Local development, evaluation dry runs | No | [View](https://huggingface.co/IFM/K2-Horizon-0.9B) |

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/jais-3.md -->

# Jais 3

**Jais 3 is built to act, not just chat.** Jais 3 is trained to work as an agent: it plans multi-step tasks, calls tools, edits Arabic documents and searches the web, while talking to the user in Arabic. **Jais 3** is built on the [K2 Horizon](https://docs.ifm.ai/docs/introduction.md) architecture and ships in the same six sizes.

- **Hosted API:** Available
- **Hugging Face collection:** [IFM/Jais-3](https://huggingface.co/collections/IFM/jais-3)
- **Language coverage:** Arabic + Multiple Arabic Dialects + English

## Model series

| Model | Architecture | Context | API | Weights |
| --- | --- | --- | --- | --- |
| IFM/Jais-3-375B-A23B | Sparse MoE | 512K | No | [View](https://huggingface.co/collections/IFM/jais-3) |
| IFM/Jais-3-MoVA-36B-A4B | MoVA + MoE | 512K | [Yes](https://docs.ifm.ai/docs/chat-completions.md) | [View](https://huggingface.co/collections/IFM/jais-3) |
| IFM/Jais-3-32B | Dense decoder-only | 512K | No | [View](https://huggingface.co/collections/IFM/jais-3) |
| IFM/Jais-3-7B | Dense decoder-only | 512K | No | [View](https://huggingface.co/collections/IFM/jais-3) |
| IFM/Jais-3-3.7B | Dense decoder-only | 512K | No | [View](https://huggingface.co/collections/IFM/jais-3) |
| IFM/Jais-3-0.9B | Dense decoder-only | 128K | No | [View](https://huggingface.co/collections/IFM/jais-3) |

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/jais-asr.md -->

# Jais ASR

**IFM/Jais-ASR** is IFM's state-of-the-art speech recognition model for Arabic as it is spoken. It is based on IFM/Jais-3-4B.

- **Transcriptions API:** Available
- **Hugging Face:** [IFM/Jais-ASR](https://huggingface.co/collections/IFM/jais-speech)
- **Language coverage:** Arabic + Multiple Arabic Dialects + English, French, Spanish, and Italian

## Capabilities

- **Dialects.** Transcribes Modern Standard Arabic and dialects from across the Arab world, and identifies the language or dialect being spoken.
- **Code-switching.** Follows speakers as they switch between Arabic and English, French, Spanish, or Italian, keeping those words in their original script.
- **Diacritization.** Optionally adds Arabic vowel marks to the transcript.
- **No-speech detection.** Returns an explicit signal when there is no speech to transcribe.

## Limits

- Maximum file size: 32 MB
- Maximum audio length: 15 minutes

See [Transcriptions API](https://docs.ifm.ai/docs/transcription.md) for request parameters and response fields.

## Model

| Model | Architecture | API | Weights |
| --- | --- | --- | --- |
| IFM/Jais-ASR | Speech-to-text | [Yes](https://docs.ifm.ai/docs/transcription.md) | [View](https://huggingface.co/collections/IFM/jais-speech) |

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/k2-v2.md -->

# K2 V2

K2-V2 is built for research, fine-tuning, and long-context work.

**K2-Think-V2** API access retired on **3 October 2026**. Please migrate to **IFM/K2-Horizon-375B-A23B**. Model weights remain available on Hugging Face. See [Deprecation](https://docs.ifm.ai/docs/deprecation.md) for migration steps.

- **Weights license:** Apache 2.0
- **Hugging Face collection:** [IFM/K2-V2](https://huggingface.co/collections/IFM/k2-v2)

## Model series

The K2-V2 collection ships a base model (for research), an instruction-tuned variant (aligned, conversational), and a reasoning-focused release. Only K2-Think-V2 was served on the hosted API, and that access ends 3 October 2026; all weights stay open. [Explore the collection on Hugging Face →](https://huggingface.co/collections/IFM/k2-v2)

| Model | Type | Params | Context | API | Weights |
| --- | --- | --- | --- | --- | --- |
| K2-Think-V2 | Reasoning | 70B | 128K | No | [View](https://huggingface.co/IFM/K2-Think-V2) |
| K2-V2-Instruct | SFT / aligned | 70B | 128K | No | [View](https://huggingface.co/IFM/K2-V2-Instruct) |
| K2-V2 | Base (pretrained) | 70B | 128K | No | [View](https://huggingface.co/IFM/K2-V2) |

## Intended use

- Research on large language models and reasoning.
- Downstream fine-tuning — instruction following, agents, domain models.
- Long-context architecture experiments.
- Open, reproducible benchmarking of LLM scaling.

For aligned conversational use and any production traffic, use [K2 Horizon](https://docs.ifm.ai/docs/introduction.md) (see [Deprecation](https://docs.ifm.ai/docs/deprecation.md)).

_Last updated: October 14, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/jais-2.md -->

# Jais 2

**Jais 2** left the hosted API on **21 August 2026**. Weights remain available on Hugging Face.

The second generation of the Jais family — Arabic-centric bilingual chat models developed by MBZUAI, Cerebras, and Inception, trained from scratch on 600B+ curated Arabic tokens.

- **Weights license:** Apache 2.0
- **Hugging Face collection:** [Jais 2](https://huggingface.co/collections/inceptionai/jais-2-family)

## Model series

Jais 2 ships in two decoder-only sizes, both trained from scratch with a custom 150K-token Arabic-centric vocabulary, RoPE positional encoding, and ReLU² activations. [Explore the models on Hugging Face →](https://huggingface.co/collections/inceptionai/jais-2-family)

| Model | Architecture | Active params | Context | Weights |
| --- | --- | --- | --- | --- |
| Jais-2-70B-Chat | Dense decoder-only | 70B | 8K | [View](https://huggingface.co/inception42/Jais-2-70B-Chat) |
| Jais-2-8B-Chat | Dense decoder-only | 8B | 8K | [View](https://huggingface.co/inception42/Jais-2-8B-Chat) |

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/reasoning.md -->

# Reasoning

IFM models offer multiple reasoning modes. Learn how to set the effort level, interpret the resulting trace, and replay it across turns.

## Reasoning effort

Set the thinking budget with `reasoning_effort` — one of `low`, `medium`, or `high` — passed through `chat_template_kwargs` in the request body.

```
{
  "model": "IFM/K2-Horizon-375B-A23B",
  "messages": [{ "role": "user", "content": "hello" }],
  "chat_template_kwargs": {
    "reasoning_effort": "high"
  }
}
```

Effort support differs by model family: K2 Horizon accepts `low`, `medium`, and `high`; Jais 3 (`IFM/Jais-3-MoVA-36B-A4B`) does not support `medium`.

## Reasoning tokens

The trace comes back in `reasoning_content`, separate from `content` — see [Chat completions API](https://docs.ifm.ai/docs/chat-completions.md) for the full response shape.

Each effort level closes the trace with its own token:

| Effort | Token |
| --- | --- |
| low | `<ifm\|think_faster>` |
| medium | `<ifm\|think_fast>` |
| high (recommended) | `<ifm\|think>` |

Every assistant turn you send back must replay its reasoning trace verbatim — see [Multi-turn conversations](https://docs.ifm.ai/docs/multi-turn.md).

_Last updated: October 14, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/custom-function-calling.md -->

# Custom function calling

Tool calling enables IFM models to use functions implemented by an application. Interfaces are defined with the standard OpenAI tool schema, so existing tool definitions work unchanged.

1. **The application** implements and executes the function.
2. **The function interface** — including its name, description, and parameters — is defined in the `tools` array.
3. **The model** selects the appropriate function and generates its arguments but does not execute the function itself.

## Declare the interface

Pass an array of function definitions in `tools`. Names, descriptions, and JSON Schema parameters are all standard.

In the example below, `lookup_order_status` is a function your application already implements.

```
{
  "type": "function",
  "function": {
    "name": "lookup_order_status",
    "description": "Current status of a customer order.",
    "parameters": {
      "type": "object",
      "properties": { "order_id": { "type": "string" } },
      "required": ["order_id"]
    }
  }
}
```

## The tool loop

The model answers with `finish_reason: "tool_calls"` and an empty `content`. Execute the function, append the result, and re-send — looping until `finish_reason: "stop"`.

```
messages = [
  {"role": "user", "content": "Where is my order A-10432?"},
  {
    "role": "assistant",
    "content": "",
    "reasoning_content": "User wants live order status; call lookup_order_status with the order id.",
    "tool_calls": [{"id": "call_1", "type": "function",
      "function": {"name": "lookup_order_status", "arguments": "{\"order_id\":\"A-10432\"}"}}]
  },
  {"role": "tool", "tool_call_id": "call_1", "content": "{\"status\": \"shipped\", \"eta\": \"2026-09-17\"}"},
  # the next assistant message answers using the tool result
]
```

## Choosing when tools run

`tool_choice` controls whether the model is allowed — or required — to call a function on a given turn.

| Value | Behavior |
| --- | --- |
| "auto" | Default when tools are present. The model decides whether to call a tool or answer directly. |
| "none" | Tools stay visible to the model but are never called. Useful for a final summarizing turn. |
| "required" | The model must call at least one tool. Use when your flow cannot proceed without data. |
| {"type": "function", …} | Forces one named function. The model still chooses the arguments. |

The model may return several entries in `tool_calls` in a single turn — each one needs its own `tool` message back.

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/streaming.md -->

# Streaming

Stream tokens as they are generated using server-sent events in the OpenAI-compatible chunk schema. Existing SDK streaming code works unchanged.

## Enable streaming

Set `stream=True` to receive the response incrementally. Reasoning and content arrive as separate deltas — buffer each on its own, then append the finished turn when the stream ends.

```
stream = client.chat.completions.create(
    model=os.environ["IFM_MODEL"], messages=messages, stream=True,
)

content, reasoning = "", ""
for chunk in stream:
    delta = chunk.choices[0].delta
    reasoning += getattr(delta, "reasoning_content", None) or ""
    content   += getattr(delta, "content", None) or ""

messages.append({"role": "assistant", "content": content, "reasoning_content": reasoning})
```

## Streaming tool calls

Tool calls stream too, and `arguments` arrives in fragments across many chunks. Accumulate the fragments per `tool_calls[i].index` and only parse the JSON once the stream has finished — a partial fragment is almost never valid JSON on its own.

```
for chunk in stream:
    for tc in (chunk.choices[0].delta.tool_calls or []):
        buf = calls.setdefault(tc.index, {"name": "", "args": ""})
        if tc.function.name:      buf["name"] += tc.function.name
        if tc.function.arguments: buf["args"] += tc.function.arguments

# parse only after the stream ends
args = json.loads(calls[0]["args"])
```

## Ending a stream

The last content chunk carries `finish_reason` — `"stop"`, `"length"`, or `"tool_calls"` — followed by the `[DONE]` sentinel. Treat a stream that ends without a `finish_reason` as truncated and retry it; do not append a half-finished turn to your history.

Streams cannot be resumed. If the connection drops mid-generation, re-send the request from the last complete turn. Sending the same `X-Session-ID` makes the retry land on the compute node that still holds the prefix — see [Session routing](https://docs.ifm.ai/docs/session-routing.md).

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/structured-output.md -->

# Structured output

Constrain the reply to machine-readable JSON with `response_format` — either valid JSON of any shape, or JSON matching a schema you supply.

## JSON mode

`{ "type": "json_object" }` guarantees the reply parses as JSON, but says nothing about its shape. Describe the fields you want in the prompt as well — the constraint enforces syntax, not structure.

```
{
  "model": "IFM/K2-Horizon-375B-A23B",
  "messages": [{ "role": "user", "content": "List three UAE emirates with populations." }],
  "response_format": { "type": "json_object" }
}
```

## JSON Schema mode

`{ "type": "json_schema" }` pins the exact shape. Decoding is constrained to your schema, so the reply validates by construction and you can skip defensive parsing.

```
"response_format": {
  "type": "json_schema",
  "json_schema": {
    "name": "emirate_list",
    "strict": true,
    "schema": {
      "type": "object",
      "properties": {
        "emirates": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": { "type": "string" },
              "population": { "type": "integer" }
            },
            "required": ["name", "population"]
          }
        }
      },
      "required": ["emirates"]
    }
  }
}
```

Keep schemas shallow and name fields the way you would in a prompt — deeply nested or cryptically named schemas cost tokens and degrade the content of the answer even though it stays valid.

## With reasoning

The constraint applies to `content` only. `reasoning_content` is free-form prose and is never JSON — parse the two fields separately, and keep replaying the trace as described in [Multi-turn conversations](https://docs.ifm.ai/docs/multi-turn.md).

## Failure modes

A schema the model cannot satisfy usually surfaces as a truncated reply with `finish_reason: "length"` rather than an error — the constraint keeps the output valid right up to the token limit. Raise `max_tokens`, then simplify the schema. Unsupported JSON Schema keywords are rejected at request time with a `400`; see [Errors](https://docs.ifm.ai/docs/errors.md).

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/multi-turn.md -->

# Multi-turn conversations

The model is stateless: each request must carry the full conversation history. Append every assistant message back into `messages` exactly as returned, then append the next user turn. This applies uniformly to plain replies, tool calls, and streamed responses. Parameters are documented on the [Chat completions API](https://docs.ifm.ai/docs/chat-completions.md) page.

## Continue a conversation

Reasoning models return `reasoning_content` alongside `content`. Replay it verbatim — an empty string `""` is a valid value, so keep it rather than dropping the field or sending `null`.

```
msg = resp.choices[0].message

messages.append({
    "role": "assistant",
    "content": msg.content,
    "reasoning_content": msg.reasoning_content,   # keep "" as-is
})
messages.append({"role": "user", "content": "Now double it."})

resp = client.chat.completions.create(model=os.environ["IFM_MODEL"], messages=messages)
```

| Field | Meaning |
| --- | --- |
| reasoning_content | The canonical field, matching vLLM and SGLang. Use this one. |
| reasoning | Mirror with the same value, kept for older clients. Send either; sending both is harmless. |

## Tool turns and streaming

The same replay rule covers both. Tool turns interleave in the one `messages` array — see [Custom function calling](https://docs.ifm.ai/docs/custom-function-calling.md). Streamed turns are assembled from deltas before being appended — see [Streaming](https://docs.ifm.ai/docs/streaming.md). To keep a long conversation on the compute node that already holds its prefix, see [Session routing](https://docs.ifm.ai/docs/session-routing.md).

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/session-routing.md -->

# Session routing

Hosted requests are load-balanced across many compute nodes. Sending a session identifier keeps a conversation on the node that already holds its KV cache, so the shared prefix is not recomputed on every turn.

## Send X-Session-ID

Set the `X-Session-ID` header to a value that is stable for the life of one conversation or agent run. Requests carrying the same value are consistently routed to the same worker.

```
curl https://api.ifm.ai/v1/chat/completions \
  -H "Authorization: Bearer $IFM_API_KEY" \
  -H "X-Session-ID: conv_8f2c14ab" \
  -H "Content-Type: application/json" \
  -d '{"model": "IFM/K2-Horizon-375B-A23B", "messages": [...]}'
```

With the OpenAI SDKs, pass it per request via `extra_headers`:

```
resp = client.chat.completions.create(
    model="IFM/K2-Horizon-375B-A23B", messages=messages,
    extra_headers={"X-Session-ID": session_id},
)
```

## Choosing a value

| Rule | Why |
| --- | --- |
| Stable per conversation | The whole point is that turn N+1 reaches the node that served turn N. |
| Opaque and unguessable | Use a random ID you mint — never an email, user name, or raw account ID. |
| Never shared across users | Two users on one ID land on one node and share nothing useful, while concentrating load. |
| New ID per agent run | Long-lived agent loops reuse a large prefix; a fresh run has a fresh prefix. |

## What it does and does not change

This is a **latency and efficiency** optimization, and it is best-effort. Results are identical with or without the header — no state is stored on the node between requests, and nothing about your conversation is retained server-side. Omit the header and requests are distributed by load as usual; every turn simply re-reads the full prompt.

Affinity can break at any time — nodes are added, drained, and restarted, and a cache entry can be evicted under memory pressure. Treat a cache hit as a bonus, never a guarantee, and never as a substitute for sending the full conversation history on each request.

Where it pays off most: long multi-turn chats, agent loops that resend a growing tool transcript, and any workload with a large shared system prompt.

## Under the hood

The hosted API balances traffic with the [vLLM Router](https://github.com/vllm-project/router) using its `consistent_hash` policy, which routes requests carrying the same session identifier to the same backend worker. See the router's [load-balancing documentation](https://github.com/vllm-project/router/tree/main/docs/load_balancing#consistent-hash) for the routing-key priority order and the full policy set.

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/chat-completions.md -->

# Chat completions API

The primary text-generation endpoint for IFM chat and reasoning models, fully compatible with the OpenAI Chat Completions API. New to the API? Follow the [Quickstart](https://docs.ifm.ai/docs/quickstart.md) to make your first call.

`POST https://api.ifm.ai/v1/chat/completions`

## Parameters

For quotas and failure handling, see [Limits](https://docs.ifm.ai/docs/limits.md) and [Errors](https://docs.ifm.ai/docs/errors.md).

| Parameter | Type | Status | Notes |
| --- | --- | --- | --- |
| model | string | Required | e.g. IFM/K2-Horizon-375B-A23B, IFM/Jais-3-MoVA-36B-A4B |
| messages | array | Required | system / user / assistant / tool roles. Text content only. |
| temperature | float | Supported | 0–2, default 0.7. Use ≤0.3 for agentic workloads. |
| top_p | float | Supported | 0–1, default 1. Prefer this or temperature, not both. |
| max_tokens | int | Supported | Cap on completion length. Defaults to remaining context. |
| stop | string \| array | Supported | Up to 4 stop sequences. |
| seed | int | Supported | Best-effort determinism for identical requests. |
| stream | bool | Supported | SSE streaming — see [Streaming](https://docs.ifm.ai/docs/streaming.md). |
| tools / tool_choice | array \| str | Supported | Standard OpenAI function schema — no changes needed. |
| frequency_penalty | float | Supported | Accepted for compatibility but has no effect. |
| logprobs | bool | Supported | Accepted for compatibility; not returned. |
| reasoning_effort | string | Extended | "low" · "medium" · "high", default "high". Per-family behavior differs — see [Reasoning](https://docs.ifm.ai/docs/reasoning.md). |
| response_format | object | Extended | { "type": "json_object" } for guaranteed JSON. K2 Horizon also accepts a json_schema for structured output. |

## Response

```
{
  "id": "chatcmpl-acd8436fe38e4520",
  "model": "IFM/K2-Horizon-375B-A23B",
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "\nHello! Is there anything I can help you with today?",
      "reasoning_content": "The user just says hello, with no request. Respond warmly and…",
      "reasoning": "The user just says hello, with no request. Respond warmly and…",
      "tool_calls": []
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 214,
    "total_tokens": 225,
    "prompt_tokens_details": { "cached_tokens": 0 },
    "completion_tokens_details": { "reasoning_tokens": 184 }
  }
}
```

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/transcription.md -->

# Transcriptions API

Converts speech in an audio file to text. The endpoint is compatible with the OpenAI audio transcription API.

`POST https://api.ifm.ai/v1/audio/transcriptions`

## Request

The request body is `multipart/form-data`: the audio file plus the fields below. Leave `language` out and the model detects the spoken language automatically.

```
curl https://api.ifm.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer $IFM_API_KEY" \
  -F file=@meeting.wav \
  -F model="IFM/Jais-ASR"
```

### Body parameters

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `file` | file | required | The audio file to transcribe. See Supported formats. |
| `model` | string | required | Transcription model ID, e.g. `IFM/Jais-ASR`. |
| `language` | string | — | Language or dialect hint. Case-insensitive. Omitting it is recommended; see Language. |
| `code_switch` | boolean | `true` | When `true`, foreign words spoken within Arabic speech (such as English terms) are written in Latin script. When `false`, they are transliterated into Arabic script. |
| `diacritics` | boolean | `false` | When `true`, Arabic output includes diacritics (tashkeel). |
| `max_completion_tokens` | integer | `1024` (max `2048`) | Maximum number of tokens the model can generate for each chunk of up to 30 seconds of audio. The default is sufficient in almost all cases; see `truncated` under Response. |

### Passing extra parameters with the OpenAI SDK

`code_switch`, `diacritics`, and `max_completion_tokens` are not part of the OpenAI SDK's method signature, so pass them through `extra_body`. With curl, send them as ordinary form fields.

```
curl https://api.ifm.ai/v1/audio/transcriptions \
  -H "Authorization: Bearer $IFM_API_KEY" \
  -F file=@meeting.wav \
  -F model="IFM/Jais-ASR" \
  -F language=ar \
  -F code_switch=false \
  -F diacritics=true
```

## Language

**Leave `language` unset.** The model identifies the spoken language or dialect on its own and returns it in the response's `language` field. This is the recommended path for every workload, including dialect audio.

Set `language` only when you are certain of the input. Setting it turns off detection: the model transcribes the audio as the language you named, and the response's `language` field echoes your hint instead of a detected value. The right hint, such as `language=eg` for Egyptian Arabic, can improve accuracy.

The exception is `language=ar`. It tells the model the audio is Arabic but keeps detection between Modern Standard Arabic and the dialects, so the response still returns the detected variety, such as `[msa]` or `[EG]`. Use it when you know the audio is Arabic but not which dialect.

When `language` is unset, detection returns a single label per segment: the majority language in that stretch of audio. Speech that mixes languages is not labeled language by language. To control how foreign words within Arabic speech are written, use `code_switch`.

### Language codes

Codes are case-insensitive, so `eg`, `EG`, and `Eg` are equivalent.

| Language | Code |
| --- | --- |
| Arabic, including all supported dialects | ar |
| Modern Standard Arabic | msa |
| English | en |
| French | fr |
| Spanish | es |
| Italian | it |

Use `ar` when the audio is Arabic but you don't know the dialect (the response still reports the detected dialect or MSA), `msa` for Modern Standard Arabic, or one of the dialect codes below:

| Code | Dialect |
| --- | --- |
| `DZ` | Algerian |
| `BH` | Bahraini |
| `EG` | Egyptian |
| `IQ` | Iraqi |
| `JO` | Jordanian |
| `KW` | Kuwaiti |
| `LB` | Lebanese |
| `LY` | Libyan |
| `MR` | Mauritanian |
| `MA` | Moroccan |
| `OM` | Omani |
| `PS` | Palestinian |
| `QA` | Qatari |
| `SA` | Saudi |
| `SD` | Sudanese |
| `SY` | Syrian |
| `TN` | Tunisian |
| `AE` | Emirati |
| `YE` | Yemeni |

## Response

`200 · application/json`

```
{
  "text": "نبدأ الاجتماع بمراجعة أرقام الربع الثالث.",
  "language": "[msa]",
  "no_speech": false,
  "truncated": false
}
```

| Field | Type | Description |
| --- | --- | --- |
| `text` | string | The transcript. Empty when no speech is detected. |
| `language` | string \| null | Detected language tag. Always wrapped in brackets (e.g. `[msa]`, `[EG]`) to mark it as a tag emitted by the model; strip the brackets if you want to reuse it as a `language` hint. `null` when no speech is detected. |
| `no_speech` | boolean | `true` if no speech was detected in the audio. |
| `truncated` | boolean | `true` if the model's token budget ran out before it finished transcribing a chunk, so part of the speech is missing from `text`. This is rare at the default budget; if it happens, raise `max_completion_tokens`. |

With the OpenAI SDK, `language`, `no_speech`, `truncated`, `duration_seconds`, and `segments` are not typed attributes. Read them from `result.model_extra`, e.g. `result.model_extra["language"]`. The SDK's own `languages` attribute is not populated and will show as `None`.

### Long audio (over 30 seconds)

Audio longer than 30 seconds is split into chunks automatically. Split points are placed at low-energy moments, such as pauses, using vLLM's default chunking, so each chunk is at most about 30 seconds.

The response then includes two additional fields, and the top-level `text` is the transcript of all chunks concatenated:

| Field | Type | Description |
| --- | --- | --- |
| `duration_seconds` | number | Total duration of the audio, in seconds. |
| `segments` | array | One entry per chunk, in order. Each has `start` and `end` (in seconds) plus `text`, `language`, `no_speech`, and `truncated`, with the same meaning as above but for that chunk only. |

Segments with no speech stay in the array, with an empty `text`, `no_speech: true`, and `language: null`. The segments therefore always cover the full duration of the audio without gaps.

## Supported formats

| Container | Extensions | Notes |
| --- | --- | --- |
| FLAC | flac | Lossless |
| MP3 | mp3 | Any bitrate |
| OGG | ogg, oga | Vorbis or Opus |
| WAV | wav | PCM, 16-bit or 24-bit |
| WEBM | webm | Opus audio track |

Stereo audio is downmixed to mono before transcription. The maximum file size is **32 MB**, and the maximum audio length is **15 minutes**.

_Last updated: October 14, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/limits.md -->

# Limits

Usage is governed by a daily token allowance per API key. You can monitor your daily usage in the [IFM Platform](https://platform.ifm.ai).

## Daily token cap

Every API key gets **10,000,000 tokens per day**, counting prompt and completion tokens together. The counter resets at 00:00 UTC. Requests past the cap return `429` with `rate_limit_error` until the next reset.

| Limit | Value |
| --- | --- |
| Daily token cap | 10,000,000 |
| Reset | Daily, 00:00 UTC |
| Scope | Per API key |

Per-minute request and token guards also run on the platform to protect shared capacity. They are set well above normal application traffic and are not published — build against the daily cap, and handle `429` responses with the `Retry-After` header as described below.

## Rate-limit headers

| Header | Meaning |
| --- | --- |
| `x-ratelimit-limit-tokens` | Current allowance applied by the platform guard. Informational — not a published quota. |
| `x-ratelimit-remaining-tokens` | Tokens left against that guard in the current window. |
| `x-ratelimit-limit-tokens-daily` | Daily token cap (10,000,000), resetting at 00:00 UTC. |
| `x-ratelimit-remaining-tokens-daily` | Tokens left in today's allowance. |
| `x-ratelimit-reset-tokens` | Seconds until the guard window refills. |
| `Retry-After` | Seconds to wait before retrying (present on 429s). |

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/errors.md -->

# Errors

Status codes, error types, and which failures are safe to retry.

## Error responses

Every error carries an HTTP status and a JSON body with a stable `error.type`. Branch on `type`, not on the message text — messages may change.

| Status | Type | Notes | Retry |
| --- | --- | --- | --- |
| 400 | invalid_request_error | Malformed JSON, missing required field, or an invalid parameter value. Fix the payload — retrying unchanged will fail again. | No |
| 401 | authentication_error | Missing or invalid API key. Check the `Authorization: Bearer` header. | No |
| 403 | permission_error | The key is valid but not entitled to this model, endpoint, or region. | No |
| 404 | not_found_error | Unknown model ID or endpoint path. Verify the model name and base URL. | No |
| 408 | timeout_error | The request exceeded the server-side time limit. Shorten the prompt or reduce `max_tokens`. | Backoff |
| 410 | deprecated_model_error | The requested model has reached its sunset date and is no longer served. The error body carries the migration-guide URL — switch to the replacement model. | No |
| 413 | request_too_large | Request body exceeds the size limit, or the prompt exceeds the model context window. | No |
| 422 | validation_error | Semantically invalid request — e.g. a malformed tool schema or an unsupported parameter combination. | No |
| 429 | rate_limit_error | Rate or quota limit exceeded. Wait the `Retry-After` interval, then retry with exponential backoff. | Backoff |
| 499 | request_cancelled | The client disconnected before the response completed. Not billed. | Yes |
| 500 | api_error | Unexpected internal error. Retry with exponential backoff and jitter. | Backoff |
| 502 / 504 | upstream_error | A gateway or upstream inference node failed or timed out. Transient — retry with backoff. | Backoff |
| 503 | service_unavailable | The endpoint is temporarily unable to serve traffic, typically during a deploy or capacity event. | Backoff |
| 529 | overloaded_error | The service is at capacity. Back off aggressively and retry with jitter to avoid a thundering herd. | Backoff |

## Example bodies

Invalid request — fix the payload before retrying:

```
# HTTP/1.1 400 Bad Request
{
  "error": {
    "type": "invalid_request_error",
    "code": "missing_required_parameter",
    "param": "messages",
    "message": "Field 'messages' is required."
  }
}
```

Token rate 429 — wait `Retry-After` seconds before retrying:

```
# HTTP/1.1 429 Too Many Requests
# Retry-After: 37
{
  "error": {
    "type": "rate_limit_error",
    "code": "token_rate_limit",
    "message": "Per-minute token budget exhausted."
  }
}
```

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/deployment.md -->

# Deploy K2 Horizon

K2 Horizon models can be self-hosted using SGLang or vLLM. Follow the guides below for deployment configurations and instructions:

- [Serve with SGLang](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon#hw=h200&variant=0.9b&quant=bf16&strategy=balanced&nodes=single)
- [Serve with vLLM](https://recipes.vllm.ai/IFM)

_Last updated: September 3, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/deploy-jais-3.md -->

# Deploy Jais 3

Jais 3 models can be self-hosted using vLLM. Follow the guide below for deployment configurations and instructions:

- [Serve with vLLM](https://recipes.vllm.ai/IFM)

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/opencode.md -->

# OpenCode

OpenCode, from SST. See the [OpenCode documentation](https://opencode.ai/docs) for its current Node and dependency requirements before installing, since those change independently of this page.

## Install OpenCode

```
npm install -g opencode-ai   # or: curl -fsSL https://opencode.ai/install | bash
```

## Configure the provider

In `~/.config/opencode/opencode.json`, set `"interleaved": "reasoning_content"` on the model so reasoning is carried through multi-step turns.

```
# ~/.config/opencode/opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ifm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "IFM",
      "options": {
        "baseURL": "{env:IFM_BASE_URL}",
        "apiKey": "{env:IFM_API_KEY}"
      },
      "models": {
        "{env:IFM_MODEL}": { "interleaved": "reasoning_content" }
      }
    }
  },
  "model": "ifm/{env:IFM_MODEL}"
}
```

`interleaved` is only valid inside a model entry. At provider level OpenCode ignores it, and conversations break as soon as the model returns an empty reasoning trace.

Export the variables. Point `IFM_BASE_URL` at the hosted endpoint (see [Quickstart](https://docs.ifm.ai/docs/quickstart.md)) or at a model running in your own environment. Then start OpenCode and start building:

```
export IFM_BASE_URL="https://api.ifm.ai/v1"
export IFM_API_KEY="sk-ifm-..."
export IFM_MODEL="IFM/K2-Horizon-375B-A23B"

opencode
```

`opencode` opens the interactive session. For a scripted conversation, use `opencode run "your prompt"`, then `opencode run --continue "next prompt"` to continue it.

## Self-hosting

Serve the model with vLLM 0.30.0 or later. Take the GPU and parallelism flags for your hardware from the [vLLM recipe](https://recipes.vllm.ai/IFM/K2-Horizon-375B-A23B), and keep the flags below:

```
vllm serve IFM/K2-Horizon-MoVA-36B-A4B \
  --trust-remote-code \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice --tool-call-parser k2_horizon \
  --chat-template-content-format string
```

Then point `IFM_BASE_URL` at your server and leave the rest of the config unchanged:

```
export IFM_BASE_URL="http://<host>:8000/v1"
export IFM_API_KEY="placeholder"
export IFM_MODEL="IFM/K2-Horizon-375B-A23B"
```

`IFM_MODEL` must match the name vLLM serves, which is the model id it was started with. If you start it from a local path, add `--served-model-name IFM/K2-Horizon-375B-A23B`. A server started without `--api-key` accepts any value for `IFM_API_KEY`.

`--chat-template-content-format string` is required. Without it, vLLM misreads the K2 Horizon chat template and sends user messages to the model empty.

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/openclaw.md -->

# OpenClaw

OpenClaw, a multi-channel AI gateway. See the [OpenClaw documentation](https://docs.openclaw.ai) for its current Node and dependency requirements before installing, since those change independently of this page.

## Install OpenClaw

```
npm install -g openclaw
openclaw --version
```

## Register the provider

Add IFM under `models.providers`. Pass the block as strict JSON and merge it so the rest of the config is left alone:

```
export IFM_API_KEY="sk-ifm-..."

openclaw config set models.providers.ifm '{
  "baseUrl": "https://api.ifm.ai/v1",
  "apiKey": { "source": "env", "provider": "default", "id": "IFM_API_KEY" },
  "api": "openai-completions",
  "models": [{ "id": "IFM/K2-Horizon-375B-A23B", "name": "K2 Horizon 375B",
               "input": ["text"], "contextWindow": 512000, "maxTokens": 32000,
               "reasoning": true }]
}' --strict-json --merge
```

The command reports `Updated models.providers.ifm`. If it asks you to restart the gateway, do so.

| Field | Meaning |
| --- | --- |
| baseUrl | The OpenAI-compatible root — `https://api.ifm.ai/v1` for the hosted API, or your own server's address when self-hosting. |
| apiKey | A secret reference, not a literal key: `{"source":"env","provider":"default","id":"IFM_API_KEY"}` reads the value from the environment at request time. |
| api | Wire protocol. Use `openai-completions`. |
| models | One entry per model you intend to call — `id`, a display `name`, `input` modalities, `contextWindow`, and `maxTokens`. |
| reasoning | Set to `true`. Without it, OpenClaw discards the model's reasoning between turns and the next request is rejected. |
| compat | Keep `requiresStringContent: true` and `thinkingFormat: "openrouter"`. Together they make OpenClaw send message text in the format IFM expects and return the model's reasoning on later turns. |

The `apiKey.provider` field names a registered _secret_ provider — use `"default"`. It is a separate namespace from the model-provider id, so `"provider": "ifm"` fails with `Secret provider "ifm" is not configured`.

## Run a request

Call the model by its ID:

```
openclaw infer model run --local \
  --model "IFM/K2-Horizon-375B-A23B" \
  --prompt "Count the .txt files in the current directory."
```

For a conversation, use the agent command and reuse the same `--session-id` to continue it:

```
openclaw agent --local --session-id my-chat \
  --model "IFM/K2-Horizon-375B-A23B" -m "Your message"
```

## Gateway and TUI

The `--local` commands above run without a gateway. For the interactive terminal UI, start the gateway, set IFM as the default model, then open the TUI in a second terminal.

### 1. Start the gateway

Start it from the shell where `IFM_API_KEY` is exported, because the key is read from the gateway's environment.

```
export IFM_API_KEY="sk-ifm-..."
openclaw gateway

# in another terminal: confirm it is running
openclaw gateway status
```

### 2. Set the default model

```
openclaw models set "IFM/K2-Horizon-375B-A23B"
openclaw gateway restart
```

### 3. Open the TUI

```
openclaw tui
```

The TUI connects to the running gateway. Type `/model` to confirm or switch the active model.

If the TUI reports that it can't reach the gateway, check `openclaw gateway status`. If requests fail with a missing-secret error, restart the gateway from a shell where `IFM_API_KEY` is set. See [OpenClaw documentation](https://docs.openclaw.ai) for daemon install and remote gateways.

## Self-hosting

Serve the model with vLLM 0.30.0 or later. Take the GPU and parallelism flags for your hardware from the [vLLM recipe](https://recipes.vllm.ai/IFM/K2-Horizon-375B-A23B), and keep the flags below:

```
vllm serve IFM/K2-Horizon-MoVA-36B-A4B \
  --trust-remote-code \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice --tool-call-parser k2_horizon \
  --chat-template-content-format string
```

Then register the provider as above with `baseUrl` set to your server, for example `http://<host>:8000/v1`. The model id stays `IFM/K2-Horizon-375B-A23B`, because vLLM serves the model under the name it was started with. If you start it from a local path, add `--served-model-name IFM/K2-Horizon-375B-A23B`.

## Notes

- Only `baseUrl` changes between the hosted API and a self-hosted server; a local server that needs no auth can drop `apiKey` entirely.
- `--chat-template-content-format string` is required.
- Models absent from the `models` array are rejected before a request is sent. Add an entry per model you plan to call.
- OpenClaw's built-in tools use the standard tool-calling schema; see [Custom function calling](https://docs.ifm.ai/docs/custom-function-calling.md).

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/pi.md -->

# Pi

Pi, from Earendil Works. See the [Pi documentation](https://github.com/earendil-works/pi/tree/main/packages/coding-agent/docs) for its current Node and dependency requirements before installing, since those change independently of this page.

## Install

```
npm install -g --ignore-scripts @earendil-works/pi-coding-agent
pi --version
```

## Configure the provider

Pi reads custom providers from `~/.pi/agent/models.json`. One entry covers both hosted and self-hosted endpoints — only `baseUrl` changes.

`~/.pi/agent/models.json`

```
{
  "providers": {
    "ifm": {
      "baseUrl": "https://api.ifm.ai/v1",
      "api": "openai-completions",
      "apiKey": "$IFM_API_KEY",
      "compat": {
        "supportsDeveloperRole": false,
        "requiresReasoningContentOnAssistantMessages": true
      },
      "models": [
        {
          "id": "IFM/K2-Horizon-375B-A23B",
          "name": "K2 Horizon 375B",
          "reasoning": true,
          "input": ["text"],
          "contextWindow": 512000,
          "maxTokens": 32000
        }
      ]
    }
  }
}
```

| Field | Meaning |
| --- | --- |
| baseUrl | The OpenAI-compatible root, e.g. `https://api.ifm.ai/v1`. A self-hosted server substitutes its own address; nothing else in the profile changes. |
| api | Wire protocol. `openai-completions` for every IFM endpoint. Required — Pi refuses to load a provider without it. |
| apiKey | Read literally, or expanded from the environment when written as `$IFM_API_KEY`, so the key itself never goes in the file. |
| compat.supportsDeveloperRole | Must be `false`. IFM rejects the `developer` role that Pi otherwise sends to reasoning models. |
| compat.requiresReasoningContentOnAssistantMessages | Must be `true`. Makes Pi replay the reasoning field on every assistant message, including when the trace is empty. |
| models[].id | Model id. Must be an object with an `id`, not a bare string. |
| models[].reasoning | Must be `true`, or Pi treats the model as non-thinking and never carries a trace. |
| models[].contextWindow, maxTokens | Planning limits. Omit them and Pi silently assumes a 128K context and a 16.4K output cap. |

Self-hosting: change `baseUrl` to your own server address. `apiKey` must still resolve to a non-empty value; a server that does not check authentication accepts any placeholder.

Startup defaults live in a separate file, `~/.pi/agent/settings.json`, as `defaultProvider` and `defaultModel`:

`~/.pi/agent/settings.json`

```
{
  "defaultProvider": "ifm",
  "defaultModel": "IFM/K2-Horizon-375B-A23B"
}
```

## Model routing

A model reference is `<provider>/<model>`, split on the **first** slash — so `ifm/IFM/K2-Horizon-375B-A23B` resolves to provider `ifm` and model `IFM/K2-Horizon-375B-A23B`, whose own id contains a slash.

```
export IFM_API_KEY="sk-ifm-..."

# explicit provider and model
pi --provider ifm --model IFM/K2-Horizon-375B-A23B

# combined reference
pi --model ifm/IFM/K2-Horizon-375B-A23B

# falls back to the settings.json defaults
pi
```

## Reasoning pass-through

Preserve and resend the model's reasoning field in every assistant turn; otherwise Pi drops it and multi-turn requests can fail with HTTP 400.

See [Reasoning](https://docs.ifm.ai/docs/reasoning.md) and the [multi-turn guide](https://docs.ifm.ai/docs/multi-turn.md).

## Tool calling

Pi's file and shell tools go over the standard tool-calling schema — no IFM-specific adapter layer. The model selects a declared tool and produces its arguments; executing the call stays with the agent. See [Custom function calling](https://docs.ifm.ai/docs/custom-function-calling.md).

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/hermes.md -->

# Hermes

Hermes Agent, from Nous Research. See the [Hermes documentation](https://hermes-agent.nousresearch.com/docs/) for its current Python and dependency requirements before installing, since those change independently of this page.

Verified with Hermes 0.19.0 for K2 Horizon.

## Install

```
pip install hermes-agent
hermes --version
```

If pip says `No matching distribution found for hermes-agent`, your Python version is unsupported. The reason is in pip's first line, `Ignored the following versions that require a different python version`.

## Install the IFM provider plugin

Hermes needs a small provider plugin to use IFM. It registers the endpoint and tells Hermes that IFM requires the reasoning trace to be echoed back on every replayed assistant message. Without it Hermes strips the trace and every turn after the first is rejected.

Create `~/.hermes/plugins/model-providers/ifm/` and put both files in it.

`~/.hermes/plugins/model-providers/ifm/plugin.yaml`

```
name: ifm-provider-profile
kind: model-provider
version: 1.0.0
description: IFM K2 Horizon — OpenAI-compatible endpoint with reasoning echo-back
author: IFM
```

`~/.hermes/plugins/model-providers/ifm/__init__.py`

```
from __future__ import annotations

from providers import register_provider
from providers.base import ProviderProfile

IFM_HOSTS = ("api.ifm.ai",)
IFM_PROVIDER_NAMES = ("ifm", "custom:ifm")

ifm = ProviderProfile(
    name="ifm",
    display_name="IFM",
    description="IFM — K2 Horizon reasoning models",
    signup_url="https://ifm.ai/",
    env_vars=("IFM_API_KEY",),
    base_url="https://api.ifm.ai/v1",
    fallback_models=("IFM/K2-Horizon-375B-A23B",),
    default_aux_model="IFM/K2-Horizon-375B-A23B",
)

register_provider(ifm)

def _enable_reasoning_echo_back() -> None:
    """Make _needs_thinking_reasoning_pad() true for IFM endpoints."""
    try:
        import run_agent
    except Exception:
        return
    agent_cls = getattr(run_agent, "AIAgent", None)
    if agent_cls is None:
        return
    original = getattr(agent_cls, "_needs_thinking_reasoning_pad", None)
    if original is None or getattr(original, "_ifm_patched", False):
        return

    def _needs_thinking_reasoning_pad(self):
        provider = (getattr(self, "provider", "") or "").strip().lower()
        base_url = (getattr(self, "base_url", "") or "").strip().lower()
        if provider in IFM_PROVIDER_NAMES:
            return True
        if any(host in base_url for host in IFM_HOSTS):
            return True
        return original(self)

    _needs_thinking_reasoning_pad._ifm_patched = True
    agent_cls._needs_thinking_reasoning_pad = _needs_thinking_reasoning_pad

_enable_reasoning_echo_back()
```

Self-hosting: see Self-hosting below. `IFM_HOSTS` is only a fallback for setups that rename the provider away from `ifm`, and `display_name` and `signup_url` are cosmetic strings you can edit freely.

## Configure the provider

Hermes reads YAML from `~/.hermes/config.yaml`. The plugin already supplies the base URL, the API-key variable and the model list, so the config only selects which provider and model to use.

```
# ~/.hermes/config.yaml
model:
  provider: "ifm"
  default: "IFM/K2-Horizon-375B-A23B"
```

| Field | Meaning |
| --- | --- |
| model.provider | Provider to use. `ifm` matches the plugin directory name. |
| model.default | Model id used at startup. Override per run with `--model`. |

If you already have a `config.yaml`, add or replace only the `model:` block and leave the rest of the file alone.

## Run

Export the key named by the plugin's `env_vars`, then run:

```
export IFM_API_KEY="sk-ifm-..."

hermes -z "your prompt"
```

`-z` is one-shot mode: it sends a single prompt and prints only the final response, for scripts and pipes. Omit it for the interactive session.

To continue a conversation from a script, run `hermes chat -Q -q "your prompt"`, which prints a `session_id`, then pass `--resume <session_id>` on later calls. Each `-z` call starts a new session.

Confirm the wiring with `hermes status`, which should report `Model: IFM/K2-Horizon-375B-A23B` and `Provider: IFM`.

## Self-hosting

Serve the model with vLLM 0.30.0 or later. Take the GPU and parallelism flags for your hardware from the [vLLM recipe](https://recipes.vllm.ai/IFM/K2-Horizon-375B-A23B), and keep the flags below:

```
vllm serve IFM/K2-Horizon-MoVA-36B-A4B \
  --trust-remote-code \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice --tool-call-parser k2_horizon \
  --chat-template-content-format string
```

Then change `base_url` in the plugin to your server, for example `http://<host>:8000/v1`. The plugin and config use the model id `IFM/K2-Horizon-375B-A23B`, which vLLM serves by default when started as above. If you start it from a local path, add `--served-model-name IFM/K2-Horizon-375B-A23B`.

`--chat-template-content-format string` is required.

## Reasoning pass-through

Preserve and resend the model's reasoning field in every assistant turn; otherwise Hermes drops it and multi-turn requests can fail with HTTP 400.

See [Reasoning](https://docs.ifm.ai/docs/reasoning.md) and the [multi-turn guide](https://docs.ifm.ai/docs/multi-turn.md).

_Last updated: October 15, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/migration.md -->

# Migration to K2 Horizon

This page shows how to move from a K2 V2 model to K2 Horizon, including the differences in the context window, tool-format settings, and multi-turn conversation state.

## What changes

| Area | Required change |
| --- | --- |
| Model name | Replace `MBZUAI-IFM/K2-V2-Instruct` or `MBZUAI-IFM/K2-Think-V2` with `IFM/K2-Horizon-375B-A23B`. |
| Context window | Grows from 128K to 512K tokens. |
| Conversation state | Assistant turns now carry a `reasoning_content` field that must be replayed verbatim. See below. |
| Custom function calling | Set `tool_presentation_format` and `tool_call_format` to `"xml"`; assistant tool turns keep both `tool_calls` and `reasoning`. |
| Sampling parameters | Recommended: `temperature` 1.0 and `top_p` 0.95 for K2 Horizon. |
| Token budget | Replayed reasoning counts as prompt tokens, so long threads consume the daily cap faster than the visible transcript suggests. |

## Multi-turn conversations

K2 Horizon is stateless, like K2 V2: each request carries the full history. The change is what an assistant turn contains. K2 Horizon returns `reasoning_content` alongside `content`, and both must be appended back into `messages` exactly as returned. An empty string `""` is a valid value — keep it rather than dropping the field or sending `null`. Code that rebuilt history from `content` alone will still run, but drops the model's reasoning context between turns.

```
# K2 V2 — content only
messages.append({"role": "assistant", "content": msg.content})

# K2 Horizon — replay reasoning too
msg = resp.choices[0].message
messages.append({
    "role": "assistant",
    "content": msg.content,
    "reasoning_content": msg.reasoning_content,   # keep "" as-is
})
```

Streaming follows the same rule: reasoning and content arrive as separate deltas, so buffer each on its own and append the finished turn when the stream ends. Full patterns, including tool loops, are on [Multi-turn conversations](https://docs.ifm.ai/docs/multi-turn.md).

## Context window

The ceiling moves from 128K to 512K tokens. Existing requests are unaffected, and client-side history trimming written against the old limit can usually be loosened or removed. Per-size context is listed on [K2 Horizon](https://docs.ifm.ai/docs/introduction.md).

## Custom function calling

Existing OpenAI function definitions carry over unchanged. What is new is the two format parameters K2 Horizon adds — set both to `"xml"`, documented on [Custom function calling](https://docs.ifm.ai/docs/custom-function-calling.md).

```
resp = client.chat.completions.create(
    model=os.environ["IFM_MODEL"],
    messages=messages,
    tools=tools,
    extra_body={
        "tool_presentation_format": "xml",
        "tool_call_format": "xml",
    },
)
```

Responses keep the usual OpenAI shape — `tool_calls` with `finish_reason: "tool_calls"` — so parsing code does not change.

## Sampling parameters

Recommended values differ from K2 V2 — the table above lists them. Requests that pass `temperature` and `top_p` explicitly are unaffected; requests that omit them may sample differently, so re-check any output that depended on the K2 V2 behavior.

## Before you switch

- Move the model name into an environment variable so the cutover is a config change, not a code change.
- Re-check any prompt that depended on K2 V2 phrasing — K2 Horizon reasons before answering and may need less instruction scaffolding.
- Set both tool-format parameters to `"xml"` before benchmarking agentic workloads.
- Log `usage` per call during the first week to size the new token profile.
- Retirement dates for the K2 V2 models are on [Deprecation](https://docs.ifm.ai/docs/deprecation.md).

_Last updated: September 2, 2026_

---

<!-- Source: https://docs.ifm.ai/docs/deprecation.md -->

# Deprecation

Retirement dates and the code changes each one requires.

## Retired models

| Model | Replacement | Effective on |
| --- | --- | --- |
| `MBZUAI-IFM/K2-Think-V2` | `IFM/K2-Horizon-375B-A23B` | 3 October 2026 |
| `inceptionai/Jais-2-8B-Chat` | `IFM/Jais-3-MoVA-36B-A4B` | 14 October 2026 |
| `inceptionai/Jais-2-70B-Chat` | `IFM/Jais-3-MoVA-36B-A4B` | 14 October 2026 |

## Retired base URLs

| Base URL | Replacement | Effective on |
| --- | --- | --- |
| `api.jaischat.ai/v1` | `api.ifm.ai/v1` | 14 October 2026 |
| `api.k2think.ai/v1` | `api.ifm.ai/v1` | 3 October 2026 |

Open weights remain available on [IFM Hugging Face →](https://huggingface.co/IFM).

_Last updated: October 19, 2026_