# UniLM.jl

> One Julia API for two kinds of model. A language model writes — an answer, a call of one of your functions, JSON that fits a schema — on OpenAI, Anthropic, Google Gemini, Ollama and any OpenAI-compatible server (Azure OpenAI, DeepSeek, Mistral, vLLM, LM Studio, llama.cpp). A decision model reads a piece of state and answers the questions you list about it with a probability for each answer, writing no text: TypeSafe's Jev, OpenAI's Decisions API, and local decision models through Ollama.
> The provider is one argument: `service` names its endpoint and `model` its model. A language model is reached through `Chat` + `chatrequest!` — and `respond` and `Embeddings` + `embeddingrequest!` on the providers that serve them —, a decision model through `ask`; the typed results, token and cost accounting and `RequestConfig` timeouts and retries are the same whichever provider answers. Each provider's own setup, rules, limits and failures are on its entry, linked under Providers below.

Install: `using Pkg; Pkg.add("UniLM")` (registered in Julia's General registry; requires Julia 1.13+). Source: https://github.com/algunion/UniLM.jl

## Start here (for LLMs / coding agents)

- [UniLM.jl — LLM Reference](https://algunion.github.io/UniLM.jl/dev/llm/): the types, signatures and defaults of the chat, Responses, image, embeddings, FIM, tool-loop, MCP, decision and audio APIs, the result types, the provider contracts, the request configuration and the complete exports list, on one page. Read this first. The OpenAI platform APIs (files, vector stores, conversations, uploads, containers, batch, fine-tuning, moderation, webhooks, realtime) are there by name, with their result types and shared contracts: their signatures and defaults are on the API reference pages listed under Reference below.
- [Language models](https://algunion.github.io/UniLM.jl/dev/providers/): choose your provider; the concepts every provider shares; the pages across providers.
- [Decisions](https://algunion.github.io/UniLM.jl/dev/decisions/): the same, for decision models.

<!-- generated:providers -->

## Providers

- [OpenAI](https://algunion.github.io/UniLM.jl/dev/providers/openai/): `service = OPENAIServiceEndpoint`; reads `OPENAI_API_KEY`; default model `gpt-6-sol`
- [Anthropic](https://algunion.github.io/UniLM.jl/dev/providers/anthropic/): `service = ANTHROPICServiceEndpoint`; reads `ANTHROPIC_API_KEY`; default model `claude-opus-5-5`
- [Google Gemini](https://algunion.github.io/UniLM.jl/dev/providers/gemini/): `service = GEMINIServiceEndpoint`; reads `GEMINI_API_KEY`; default model `gemini-3.8-flash`
- [Ollama](https://algunion.github.io/UniLM.jl/dev/providers/ollama/): `service = OllamaEndpoint()`; reads `OLLAMA_HOST`, `OLLAMA_API_KEY`; no default model (a call names one)
- [vLLM, LM Studio & llama.cpp](https://algunion.github.io/UniLM.jl/dev/providers/local_servers/): `service = GenericOpenAIEndpoint("http://localhost:8000", "")`; reads no environment variable; no default model (a call names one)
- [Azure OpenAI](https://algunion.github.io/UniLM.jl/dev/providers/azure/): `service = AZUREServiceEndpoint`; reads `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_BASE_URL`, `AZURE_OPENAI_API_VERSION`, `AZURE_OPENAI_DEPLOY_NAME_<MODEL>`; default model `gpt-5.2`
- [DeepSeek](https://algunion.github.io/UniLM.jl/dev/providers/deepseek/): `service = DeepSeekEndpoint()`; reads `DEEPSEEK_API_KEY`; default model `deepseek-flash`
- [Mistral & OpenAI-compatible services](https://algunion.github.io/UniLM.jl/dev/providers/openai_compatible/): `service = MistralEndpoint()`; reads `MISTRAL_API_KEY`; no default model (a call names one)
- [Jev (TypeSafe)](https://algunion.github.io/UniLM.jl/dev/decisions/jev/): `service = TYPESAFEServiceEndpoint`; reads `TYPESAFE_API_KEY`, `TYPESAFE_BASE_URL`, `TYPESAFE_DEFAULT_MODEL`; default model `jev-latest`
- [OpenAI Decisions API](https://algunion.github.io/UniLM.jl/dev/decisions/openai/): `service = OPENAIServiceEndpoint`; reads `OPENAI_API_KEY`; default model `gpt-6-luna`
- [Local decision models](https://algunion.github.io/UniLM.jl/dev/decisions/ollama/): `service = OllamaEndpoint()`; reads `OLLAMA_HOST`, `OLLAMA_API_KEY`; no default model (a call names one)

<!-- /generated:providers -->

## Concepts (every language-model provider)

- [Conversations with Chat](https://algunion.github.io/UniLM.jl/dev/guide/chat_completions/): a `Chat` holds a conversation and `chatrequest!` sends it, turn by turn
- [Agentic respond](https://algunion.github.io/UniLM.jl/dev/guide/agentic/): `respond` sends one request of a provider's agentic API and returns its output items
- [Tool Calling](https://algunion.github.io/UniLM.jl/dev/guide/tool_calling/): `Tool` on a `Chat`, `FunctionTool` on a `Respond`, and `tool_loop!` / `tool_loop` to run the round trips
- [Streaming](https://algunion.github.io/UniLM.jl/dev/guide/streaming/): `stream=true` and a `callback` on a `Chat`, a `do` block on `respond`
- [Structured Output](https://algunion.github.io/UniLM.jl/dev/guide/structured_output/): `response_format` on a `Chat`, `text` on a `Respond`
- [Attachments](https://algunion.github.io/UniLM.jl/dev/guide/attachments/): `ImageAttachment`, `AudioAttachment` and `DocumentAttachment` on a user message
- [Embeddings](https://algunion.github.io/UniLM.jl/dev/guide/embeddings/): `Embeddings` and `embeddingrequest!`
- [Image Generation](https://algunion.github.io/UniLM.jl/dev/guide/image_generation/): `generate_image` and `edit_image`
- [FIM & Prefix Completion](https://algunion.github.io/UniLM.jl/dev/guide/completions/): `fim_complete` and `prefix_complete`
- [Results](https://algunion.github.io/UniLM.jl/dev/guide/results/): the typed success, failure and call error a request returns, the four lifecycle verbs that return a `Dict` on success, and the tool loops' `ToolLoopResult`
- [Cost Tracking](https://algunion.github.io/UniLM.jl/dev/guide/cost_tracking/): `TokenUsage`, `estimated_cost`, `cumulative_cost` and `DEFAULT_PRICING`
- [Timeouts & Retries](https://algunion.github.io/UniLM.jl/dev/guide/timeouts/): `RequestConfig`, the typed timeout failures, retry contracts
- [Concurrency, Tasks and Cancellation](https://algunion.github.io/UniLM.jl/dev/guide/concurrency/): what a concurrent call may share (`fork` for `Chat`), fan-out patterns, streaming into a `Channel`, `CancelToken` / `with_cancel`, MCP sessions under load
- [Testing and recordings](https://algunion.github.io/UniLM.jl/dev/guide/testing/): `with_recorded_answers` records a call once and replays it with no key and no network
- [Model Context Protocol (MCP)](https://algunion.github.io/UniLM.jl/dev/guide/mcp/): an MCP client whose tools join a tool loop, and an MCP server of your own

## Across providers

- [One API across providers](https://algunion.github.io/UniLM.jl/dev/guide/multi_backend/): what every provider shares, how to switch, and where the providers differ
- [Capabilities by provider](https://algunion.github.io/UniLM.jl/dev/guide/capabilities/): what each provider's endpoints serve (`provider_capabilities`, `has_capability`)
- [Write your own backend](https://algunion.github.io/UniLM.jl/dev/guide/custom_backends/): a new `ServiceEndpoint` on the seams the built-in backends use

## Concepts (every decision provider)

- [Route and Decide](https://algunion.github.io/UniLM.jl/dev/guide/system_one/): `ask` with `choice`, `score` and `noul` questions, reading the answers, many questions in one request, a struct filled in one request, decision rules on the probabilities (a confidence threshold, a price for each mistake, a "none of these" option, the risk of the worst level), designing good questions
- [Dispatch on Meaning](https://algunion.github.io/UniLM.jl/dev/guide/natural_language_dispatch/): `nl_classify`; `nl_dispatch` with a table of sentences (`texts =`) or `nl"..."` meanings in method signatures, composed with ordinary dispatch (a state machine), hierarchies, several slots in one request, `min_confidence` and `decide`, the `@branch` switch
- [Many Items at Once](https://algunion.github.io/UniLM.jl/dev/guide/semantic_algorithms/): `ask_each` (one request per item, concurrent, results in input order), a batch judged in one request, ranking, finding the first line where something happens, matching records across two lists, stopping early
- [Decisions with LLMs](https://algunion.github.io/UniLM.jl/dev/guide/jev_with_llms/): a decision before a language model's call (route the message, pick the model, screen what it reads, offer only the tools it needs) and after it (check the draft and its claims against the source, approve its tool calls)
- [Test and Develop](https://algunion.github.io/UniLM.jl/dev/guide/jev_testing/): previews with `meanings` / `meaning_gaps`, `with_recorded_answers` for decision and LLM calls, tests of the methods, the resolution and the judgment, a decision + LLM pipeline tested offline, an audit trail

## Across decision providers

- [Choosing a decision provider](https://algunion.github.io/UniLM.jl/dev/decisions/choosing/): the same call on each provider, per call (`service =`) or per scope (`with_system_one`), what each takes, a threshold tuned per provider
- [Write your own decision provider](https://algunion.github.io/UniLM.jl/dev/decisions/custom/): a gateway that speaks TypeSafe's wire, or a protocol of your own

## Gotchas for code generation

- A `Chat` must start with a **system** message: `push!` **throws** `InvalidConversationError` for a non-system message pushed onto an empty `Chat`, and for consecutive same-role messages except `tool`; sending a history-keeping `Chat` that ends with an assistant message throws it too, before any request. One turn without a system message is `respond(input; …)`, on an endpoint that declares `:responses` or `:agentic` ([Capabilities by provider](https://algunion.github.io/UniLM.jl/dev/guide/capabilities/)).
- A call that names no `model` gets its provider's default model, listed under Providers above; a `Chat` for a provider with none throws `ArgumentError` when it is built. A provider's defaults for its other verbs — images, embeddings, speech, transcription — are on its entry, such as OpenAI's [Images, audio & realtime](https://algunion.github.io/UniLM.jl/dev/providers/openai/media/) and [Embeddings](https://algunion.github.io/UniLM.jl/dev/providers/openai/#openai_embeddings), and Google Gemini's [Which API for what](https://algunion.github.io/UniLM.jl/dev/providers/gemini/#gemini_apis).
- A `Chat` reaches every language-model provider, and `respond` those with an agentic API; which API each endpoint speaks is compared in [Where the providers differ](https://algunion.github.io/UniLM.jl/dev/guide/multi_backend/#multi_backend_compare), and each entry under Providers above says which of its verbs reach it.
- Before sending, a verb refuses an endpoint whose declared capabilities lack it, and the provider's encoder refuses what the rules in its entry's Limits and refusals section name, each with an `ArgumentError`; what those rules let through goes to the provider, whose answer is the check. A `callback` or `on_tool_call` given without `stream=true` is an `ArgumentError` too.
- Request failures — HTTP errors, timeouts (`cause::UniLMTimeout`), cancellations (`cause::UniLMCancelled`) — come back as typed results (`*Success` / `*Failure` / `*CallError`), except from the natural-language decision verbs (below). The accessors `text`, `output_text`, `embedding_vectors`, `image_data` and `fim_text` throw `LLMResultError` on a non-success result, so branch on `issuccess` (or the result type) first. `delete_response`, `list_input_items`, `compact_response` and `count_input_tokens` return a `Dict` on success.
- `estimated_cost` prices a result at its model's `DEFAULT_PRICING` row (USD per token) — the exact id, else the id without a dated snapshot suffix —, and a model with no row at `0.0`, with one warning per model id (pass `pricing=`, or add a row with `DEFAULT_PRICING[model] = row`). Where a provider prices otherwise — local models, versioned ids, tiers, long prompts — its entry's cost section says so, as Ollama's [Cost and usage](https://algunion.github.io/UniLM.jl/dev/providers/ollama/#ollama_cost) and Jev's [Cost](https://algunion.github.io/UniLM.jl/dev/decisions/jev/operations/#jev_cost) do.
- `Chat(n=…)` must be 1. A `Chat` (and an `Embeddings`) is mutated by its request: use one per in-flight call and `fork(chat, n)` to fan out. Cancel a call from any task with a `CancelToken` (`cancel=tok`, or `with_cancel(tok) do … end`) and `cancel!(tok)`; a token stays cancelled, so use a fresh one per unit of work.
- `ask` **returns** `SystemOneSuccess` / `SystemOneFailure` / `SystemOneCallError`, but the natural-language control flow built on it **throws**: `@branch`, `nl_dispatch` and `nl_classify` raise `SystemOneError` on a failed call and — when there is no `_` line or `fallback` — `LowConfidenceError` for an answer below `min_confidence` and `DecisionDeclinedError` when a `decide` policy returns `nothing` or the provider declines the question (a `RefusalAnswer`). An endpoint that serves decisions only, as [Jev (TypeSafe)](https://algunion.github.io/UniLM.jl/dev/decisions/jev/)'s does, is refused by `chatrequest!`, `respond`, `embeddingrequest!` and the other language-model verbs with an `ArgumentError`.
- A decision call's provider is its `service = …`, else the provider of the `with_system_one(service; model = …) do … end` scope it runs in (spawned tasks too), else Jev; its model is its `model = …`, else — when it names no `service` — the scope's, else the provider's default under Providers above. Each provider's wrapper refuses, before any request, what its API cannot carry, as Jev's [Limits and refusals](https://algunion.github.io/UniLM.jl/dev/decisions/jev/#jev_limits), the OpenAI Decisions API's [Limits and refusals](https://algunion.github.io/UniLM.jl/dev/decisions/openai/#openai_decisions_limits) and the local decision models' [Limits and refusals](https://algunion.github.io/UniLM.jl/dev/decisions/ollama/#ollama_decisions_limits) list; [What each provider takes](https://algunion.github.io/UniLM.jl/dev/decisions/choosing/#backends_table) compares their questions, limits, pictures, options and `ask_each` widths. Provider options are `ask` keywords, and `options = (…)` on the natural-language verbs. A reply that does not fit the request is a `SystemOneCallError`. `min_confidence` must be a finite number in 0 … 1, and one tuned on one provider does not carry over to another. `ask_each(states, questions...)` returns one typed result per state, in input order.
- With `texts = TABLE`, `nl_dispatch` dispatches on keys instead of `nl"..."` meanings: a `NamedTuple` of sentences (each `Symbol` key reaches its method as `Val(key)`) or a vector of `key => sentence` pairs (your own instances, types or enum values, passed as written); only the sentences are sent, in table order, a `Dict` is refused (no order), and a key no method takes is an `ArgumentError` before any request. `nl_classify(text, TABLE)` returns the chosen key as the table writes it (`:billing`, not `Val(:billing)`). `on_response = r -> …` on `nl_dispatch`, `nl_classify` and `@branch` is called once with the `SystemOneSuccess` before the decision policy runs, never for a failed call. `nl_dispatch` refuses, before any request, a default state holding a function, type, module, `Ref`, `Task`, `Channel`, lock, `IO`, `Expr` or a value JSON.jl cannot write (an `AudioAttachment`); an `ImageAttachment` argument is data, which a provider takes there or refuses ([What each provider takes](https://algunion.github.io/UniLM.jl/dev/decisions/choosing/#backends_table)); every field of a struct argument is sent — pass `state = …` to choose what the model reads. A `Meaning` type prints as `Meaning{Symbol("…")}` (a one-word sentence as `Meaning{:x}`).
- `with_recorded_answers` records and replays `ask`, one model listing (Jev's: [Testing and recordings](https://algunion.github.io/UniLM.jl/dev/decisions/jev/operations/#jev_testing)), `chatrequest!` and `respond`, streamed or not, `embeddingrequest!` (and so the tool loops), the Responses lifecycle calls (`get_response`, `delete_response`, `list_input_items`, `cancel_response`, `compact_response`, `count_input_tokens`), `fim_complete` and `prefix_complete`; images, audio, files, MCP, the other model listings and every other verb reach the network as usual. In its replay scope a request with no recording throws `ReplayMissError` out of the verb itself — and so out of `@branch`, `nl_dispatch`, `nl_classify` and the tool loops; for a later request of a streamed turn, inside the call's task, so `fetch` throws a `TaskFailedException` around it — never a call-error result.

## Reference

- [Chat Completions API](https://algunion.github.io/UniLM.jl/dev/api/chat/): the first page of the API reference for language models
- The API reference of the OpenAI platform APIs: [Files API](https://algunion.github.io/UniLM.jl/dev/api/files/), [Vector Stores API](https://algunion.github.io/UniLM.jl/dev/api/vector_stores/), [Conversations API](https://algunion.github.io/UniLM.jl/dev/api/conversations/), [Batch API](https://algunion.github.io/UniLM.jl/dev/api/batch/), [Fine-tuning API](https://algunion.github.io/UniLM.jl/dev/api/fine_tuning/), [Moderations API](https://algunion.github.io/UniLM.jl/dev/api/moderations/), [Containers API](https://algunion.github.io/UniLM.jl/dev/api/containers/), [Uploads API](https://algunion.github.io/UniLM.jl/dev/api/uploads/), [Webhooks API](https://algunion.github.io/UniLM.jl/dev/api/webhooks/), [Realtime API](https://algunion.github.io/UniLM.jl/dev/api/realtime/)
- [System One API (Jev, OpenAI Decisions, Ollama)](https://algunion.github.io/UniLM.jl/dev/api/system_one/): the API reference for decision models
- [Repository](https://github.com/algunion/UniLM.jl)
