# UniLM.jl

> A Julian, type-safe interface to LLM providers with first-class native backends — OpenAI (Chat + Responses), Anthropic (Messages), and Google Gemini (generateContent + agentic Interactions) — plus any OpenAI-compatible provider (Azure, DeepSeek, Mistral, Ollama, vLLM, LM Studio). One cross-provider agentic `respond` verb, an MCP client and server, and built-in token/cost accounting.
> It also speaks TypeSafe's System One model Jev, which answers enumerated questions about a piece of state with a typed, calibrated judgment instead of generated text — and which can select a Julia method by natural-language meaning.

Install: `using Pkg; Pkg.add("UniLM")` (registered in Julia's General registry; requires Julia 1.13+). Source: https://github.com/algunion/UniLM.jl

## Start here (for LLMs / coding agents)

- [Single-file LLM reference](https://algunion.github.io/UniLM.jl/dev/llm/): the whole API — types, signatures, defaults, and the complete exports list — on one page. Read this first.

## Guides

- [Getting Started](https://algunion.github.io/UniLM.jl/dev/getting_started/): install, API keys, first request
- [Chat Completions](https://algunion.github.io/UniLM.jl/dev/guide/chat_completions/): stateful `Chat` + `chatrequest!`
- [Responses API](https://algunion.github.io/UniLM.jl/dev/guide/responses_api/): the newer `respond` API
- [Agentic Workflows](https://algunion.github.io/UniLM.jl/dev/guide/agentic/): cross-provider `respond` (OpenAI Responses + Gemini Interactions)
- [Tool Calling](https://algunion.github.io/UniLM.jl/dev/guide/tool_calling/): function tools + automated `tool_loop`
- [Retrieval & File Search](https://algunion.github.io/UniLM.jl/dev/guide/retrieval/): end-to-end RAG (files → vector store → `file_search`)
- [Streaming](https://algunion.github.io/UniLM.jl/dev/guide/streaming/): callback / `do`-block, across providers
- [Timeouts & Retries](https://algunion.github.io/UniLM.jl/dev/guide/timeouts/): `RequestConfig`, the typed timeout failures, retry contracts
- [Concurrency, Tasks and Cancellation](https://algunion.github.io/UniLM.jl/dev/guide/concurrency/): what a concurrent call may share (`fork` for `Chat`), fan-out patterns, streaming into a `Channel`, `CancelToken` / `with_cancel`, MCP sessions under load
- [Structured Output](https://algunion.github.io/UniLM.jl/dev/guide/structured_output/): JSON-schema-constrained generation
- [Multi-Backend](https://algunion.github.io/UniLM.jl/dev/guide/multi_backend/): native + OpenAI-compatible providers
- [Custom Backends](https://algunion.github.io/UniLM.jl/dev/guide/custom_backends/): implement a new `ServiceEndpoint`
- [MCP](https://algunion.github.io/UniLM.jl/dev/guide/mcp/): Model Context Protocol client & server
- [Start Here: Jev in Five Minutes](https://algunion.github.io/UniLM.jl/dev/guide/jev_start/): what TypeSafe's System One model Jev decides, a first `nl_classify` call, and which Jev page answers which question
- [Jev with LLMs](https://algunion.github.io/UniLM.jl/dev/guide/jev_with_llms/): Jev before an LLM call (route the message, pick the model, screen what it reads, shortlist its tools) and after it (check the draft and its claims against the source, approve its tool calls)
- [Route and Decide](https://algunion.github.io/UniLM.jl/dev/guide/system_one/): `ask` with `choice`/`score`/`noul`, many questions in one request, a struct filled in one request, confidence gates, loss-matrix decisions, a catch-all option
- [Dispatch on Meaning](https://algunion.github.io/UniLM.jl/dev/guide/natural_language_dispatch/): `nl_classify`, `nl_dispatch` with a keyed table (`texts =`) or `nl"..."` meanings in method signatures, type hierarchies and state machines, `decide`, `on_response`, the `@branch` switch
- [Many Items at Once](https://algunion.github.io/UniLM.jl/dev/guide/semantic_algorithms/): many items in one request (keyed, not indexed), ranking, finding an event in a long sequence, matching records, stopping early
- [Test and Develop](https://algunion.github.io/UniLM.jl/dev/guide/jev_testing/): previews with `meanings`/`meaning_gaps`, `with_recorded_answers` record/replay for Jev and LLM calls, a Jev + LLM pipeline tested offline, an audit trail
- [Cost Tracking](https://algunion.github.io/UniLM.jl/dev/guide/cost_tracking/): token usage + USD estimation
- [Embeddings](https://algunion.github.io/UniLM.jl/dev/guide/embeddings/)
- [Image Generation](https://algunion.github.io/UniLM.jl/dev/guide/image_generation/): create + edit
- [FIM & Prefix Completion](https://algunion.github.io/UniLM.jl/dev/guide/completions/)

## Gotchas for code generation

- A `Chat` must start with a **system** message; `push!` **throws** `InvalidConversationError` for a non-system message pushed onto an empty `Chat` (and for consecutive same-role messages, except `tool`). Use `respond("...")` for a single turn without a system prompt.
- Real defaults: OpenAI `gpt-5.6-sol`; native Gemini `gemini-3.8-flash`; native Anthropic `claude-opus-5-5` (auto `max_tokens` 16000); DeepSeek `deepseek-flash`; TypeSafe System One `jev-latest`; image `gpt-image-2`; embeddings `text-embedding-3-small`.
- Prefer the Responses API / `respond` for new code. Native Anthropic = `ANTHROPICServiceEndpoint`; native Gemini = `GEMINIServiceEndpoint`; the OpenAI-compat Gemini shim (needed for embeddings) = `GEMINIOpenAIServiceEndpoint`.
- Local validation throws before any request: an unsupported capability, an option the provider/model cannot express, or a `callback`/`on_tool_call` given without `stream=true` is an `ArgumentError`, and sending a history-keeping `Chat` that ends with an assistant message is an `InvalidConversationError`. Request failures — HTTP errors, timeouts (`cause::UniLMTimeout`), cancellations (`cause::UniLMCancelled`) — come back as typed results (`*Success` / `*Failure` / `*CallError`). The accessors `text`, `output_text`, `embedding_vectors`, `image_data` and `fim_text` throw `LLMResultError` on a non-success result, so branch on `issuccess` (or the result type) first. `estimated_cost` returns `0.0` (logging one warning per model id) for any model not in `DEFAULT_PRICING` (pass `pricing=`, or add a row with `DEFAULT_PRICING[model] = row`), except that a dated snapshot uses its base model's row and a versioned `jev-X.Y.Z` id without its own row uses the `jev-latest` row.
- `Chat(n=…)` must be 1. A `Chat` (and an `Embeddings`) is mutated by its request: use one per in-flight call and `fork(chat, n)` to fan out. Cancel a call from any task with a `CancelToken` (`cancel=tok`, or `with_cancel(tok) do … end`) and `cancel!(tok)`; a token stays cancelled, so use a fresh one per unit of work.
- `TYPESAFEServiceEndpoint` is **not** a chat backend: it declares only `:system_one` and `:models`, so the verbs `chatrequest!`, `respond`, `embeddingrequest!` and the other platform verbs reject it with an `ArgumentError` — constructing a `Chat` or an `Embeddings` that names it is fine, and omitting `model=` is refused the same way. `ask` and `list_models` follow the package convention and **return** `SystemOneSuccess`/`SystemOneFailure`/`SystemOneCallError`, but the natural-language control flow built on them **throws**: `@branch`, `nl_dispatch` and `nl_classify` raise `SystemOneError` on a failed call, and — when there is no `_` line / `fallback` — `LowConfidenceError` for an answer below `min_confidence` and `DecisionDeclinedError` when a `decide` policy returns `nothing`. `nl_dispatch` raises an `ArgumentError` before any request when no natural-language method accepts the types of its ordinary arguments, or when a combination of the meanings it would offer has no method (a gap; `meaning_gaps(f, argtypes)` lists them). With `texts = TABLE`, `nl_dispatch` dispatches on keys instead of `nl"..."` meanings: a `NamedTuple` of sentences (each `Symbol` key reaches its method as `Val(key)`) or a vector of `key => sentence` pairs (your own instances, types or enum values, passed as written); only the sentences are sent, in table order, a `Dict` is refused (no order), and a key no method takes is an `ArgumentError` before any request. `nl_classify(text, TABLE)` returns the chosen key as the table writes it (`:billing`, not `Val(:billing)`). `on_response = r -> …` on `nl_dispatch`, `nl_classify` and `@branch` is called once with the `SystemOneSuccess` (request id, model, raw body) before the decision policy runs, never for a failed call. `with_recorded_answers` records and replays `ask` and `list_models` and the non-streaming `chatrequest!`, `respond` and `embeddingrequest!` (and so the tool loops); streamed calls and every other verb reach the network as usual. In its replay scope a request with no recording throws `ReplayMissError` out of the verb itself — and so out of `@branch`, `nl_dispatch`, `nl_classify` and the tool loops — never a call-error result. Default model `jev-latest` (`TYPESAFE_API_KEY`; `TYPESAFE_DEFAULT_MODEL` overrides).

## Reference

- [API reference (start)](https://algunion.github.io/UniLM.jl/dev/api/chat/)
- [Repository](https://github.com/algunion/UniLM.jl)
