What Kosong is

Kosong is a Python library from Moonshot AI that acts as an LLM abstraction layer for modern agentic applications. It is designed to sit between your agent logic and multiple LLM providers, unifying message structures, streaming semantics and asynchronous tool orchestration so teams do not have to hard-wire business logic to a single provider API. Kosong also powers Moonshot's Kimi CLI.

Core API and design goals

The public surface of Kosong is intentionally small. At the top level you typically import kosong.generate, kosong.step and the result types GenerateResult and StepResult. Supporting modules include kosong.chat_provider, kosong.message, kosong.tooling and kosong.tooling.simple.

These components wrap provider-specific streaming formats, token accounting and tool call details behind a consistent interface so agent code can remain stable while providers and models evolve.

ChatProvider and message model

The central integration point is the ChatProvider abstraction. Moonshot provides an example implementation for Kimi in kosong.chat_provider.kimi. A Kimi provider is instantiated with base_url, api_key and a model name such as kimi-k2-turbo-preview. That provider instance is then passed into kosong.generate or kosong.step along with a system prompt, available tools and a message history.

Messages are represented by the Message class from kosong.message. A Message is constructed with a role like user and a content payload. Content can be a simple string or a list of content parts, enabling richer multimodal payloads while keeping the basic chat workflow simple for new users. During streaming, providers emit StreamedMessagePart items via kosong.chat_provider, and the library merges them into the final Message object.

Kosong also exposes an optional TokenUsage structure attached to result objects for provider-independent token accounting useful for logging and monitoring.

Tooling and toolsets

Most agents need external tools such as search, code execution or database access. Kosong models tools using the tooling module. The examples show defining a tool by subclassing CallableTool2 with a Pydantic parameter model. Tools return a ToolReturnType such as ToolOk when execution succeeds.

Tools are registered into a SimpleToolset from kosong.tooling.simple. In the examples the toolset is augmented with tools using +=. The toolset is responsible for resolving tool calls produced by the model and routing them to the correct asynchronous functions.

generate for plain chat completions

Use kosong.generate for single-shot chat completions. You provide the chat_provider, a system_prompt, an explicit list of tools (which can be empty) and a history of Message objects. Kosong supports streaming via an on_message_part callback. The function returns a GenerateResult containing the merged assistant message and an optional TokenUsage structure after streaming completes. This lets applications present incremental output while still working with a clean final Message object.

step for tool-using agents

For agent flows that invoke tools, use kosong.step. step is called with a ChatProvider, a SimpleToolset that contains the tools the agent can call, a system prompt and a user history. step returns a StepResult; the example shows printing result.message and then awaiting result.tool_results() to collect all tool outputs produced during the step.

Kosong handles orchestration of tool calls, including parsing arguments into Pydantic models and converting outputs into ToolReturnType results. This removes the need for agent authors to implement a provider-specific dispatch loop for each backend.

Demo agent and Kimi CLI integration

Kosong includes a built-in demo agent that can run locally. The repo's README documents environment variables KIMI_BASE_URL and KIMI_API_KEY, and shows a launch command using python -m kosong kimi --with-bash under uvicorn. The demo uses Kimi as the chat provider and can expose a terminal agent that calls tools, including shell commands when the with bash option is enabled.

Why this matters

By centering the integration surface on ChatProvider, Message and Toolset, Kosong provides a minimal but practical infrastructure for building long-lived agent systems that can swap providers, models and tooling without rewriting orchestration logic. It is a pragmatic move from Moonshot AI to separate agent intent and flow from provider-specific mechanics, and it underpins the Kimi CLI experience while remaining extensible for other backends.