An MIT-licensed Swift package that brings per-step tool calling, agent loops, NPC dialogue with memory, and an OpenAI-compatible local server to Apple's on-device Foundation Models on iOS, iPadOS, macOS and visionOS 27. Pre-release: no tagged releases yet, and all measurements so far come from macOS 27.
Measured on macOS 27 before building anything: Apple's fm serve returned 0 tool calls in 54 requests, and the framework's own .required mode never stopped calling tools. Details in RESEARCH.md.
| Feature / Scenario | Apple Native (fm serve / SPM) |
OpenAppleModels Runtime |
|---|---|---|
| Tool Calls Emitted (Tool Mode) | ā 0 / 54 Emitted (Always answered directly) | ā Real tool_calls from oam serve and the Swift API |
Enforcing tool_choice: "required" |
ā HTTP 500 Internal Server Error | ā Forced on the first step, then the model answers |
| Multi-Turn Loop Behavior | ā .required loops (40+ calls, 3 of 3 runs) |
ā Round, call and per-step tool limits |
| Direct Response Fallback | ā .allowed often skips tools and invents facts |
ā .explicit: a tool or respond_directly |
| Game Engine Bridging | ā Not Provided (Swift framework only) | ā JSON-RPC 2.0 over stdio or a C ABI (C, Python, Unity bindings) |
| NPC Personas & Memory | ā No High-Level Dialogue Primitive | ā Personas, facts, relationship score, secrets |
| CI/CD Mock Testing | ā Not included | ā ScriptedLanguageModel (tests still need a macOS 27 host) |
Six library products and the oam CLI, for game developers who want on-device NPCs and AI decisions, and for anyone who needs real tool calls from Apple's model.
Force a tool call on step 1 to ground queries against game inventory, SQLite, or player stats, then seamlessly transition to natural language generation or structured schemas.
.explicit)
The model must call a tool or a built-in respond_directly tool, which grounds lookups without forcing a pointless call on small talk. On eight tavern lines it chose the right action 8/8, against 7/8 for .auto.
OpenAppleModelsGame provides NPC personas with memory (facts, a relationship score from -100 to 100, a summary), secrets that unlock with the relationship, world state, decisions and content generation. A blocked reply is retried once as plain text, then replaced by a fallback line.
A replacement for fm serve on http://127.0.0.1:1976/v1 (model "system") whose tool_calls work, with tool_choice, streaming and json_schema responses.
JSON-RPC 2.0 protocol v1.0 over oam stdio or the C ABI (OpenAppleModelsFFI), with tools executed by the host. Bindings for C, Python and Unity; integration notes for Godot and Unreal.
By default everything runs on the device's own model, with no API key or cloud service, and the package sends no telemetry. Use of the model is subject to Apple's acceptable use requirements.
Swift, the oam CLI, any OpenAI client, or a game engine over JSON-RPC. These examples are taken from the README, where they compile against the current code.
import OpenAppleModels
let inventory = try AgentTool(
name: "check_inventory",
description: "Look up how many of an item the blacksmith has and its price in gold.",
parameters: .object(["item": .string(description: "Item name")])
) { call in
let item = try call.string("item")
return .json(["item": .string(item), "stock": 3, "price_gold": 45])
}
let gorm = try Agent(
instructions: "You are Gorm, a grumpy blacksmith in a fantasy game. Reply in at most two sentences.",
tools: [inventory])
// .required: the model must call a tool on the first step, then answers from its output.
let reply = try await gorm.respond(to: "Got any iron swords? How much?",
policy: ToolPolicy(choice: .required))
print(reply.text, reply.toolCalls.count)
import OpenAppleModelsGame
let world = WorldState(["player": ["name": "Aria", "gold": 60], "time_of_day": "evening"])
let smith = try NPC(
persona: Persona(
name: "Gorm", role: "the village blacksmith",
personality: "Gruff and proud, but fair", speakingStyle: "Short, blunt sentences. Calls people 'lad'."),
tools: [inventory], // any AgentTool, local or external
world: world, // adds a read_world_state tool
options: NPCOptions(
groundingTool: "check_inventory", // look up stock on every turn
worldContextPaths: ["player.name", "player.gold", "time_of_day"]))
let turn = try await smith.talk("Evening! Got any iron swords? How much?")
print(turn.emotion, turn.line, turn.playerOptions)
// Live run of a similar setup (docs/GAMES.md): called check_inventory, then
// "Aria, I've got three iron swords for 45 gold. Take one if you're keen." in 4.4-5.2 s.
# Start the server first: oam serve (127.0.0.1:1976)
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:1976/v1", api_key="unused")
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Current weather for a city.",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
},
}]
response = client.chat.completions.create(
model="system",
messages=[{"role": "user", "content": "What is the weather in Paris?"}],
tools=tools,
tool_choice="required",
)
for tool_call in response.choices[0].message.tool_calls:
print(tool_call.function.name, tool_call.function.arguments)
# Force a tool call with an inline tool. A tool without a command is external:
# oam exits with code 10 and prints the pending calls and a transcript path.
oam respond 'How many potions do I have?' --tool-choice required \
--tool-json '{"name":"get_potions","description":"Count potions in the inventory","parameters":{"type":"object","properties":{}}}'
# OpenAI-compatible local server (127.0.0.1:1976)
oam serve
# Talk to Mira, an NPC innkeeper with a menu tool and a till
oam demo tavern
# JSON-RPC bridge over stdin/stdout for game engines
oam stdio
// A client tool call over oam stdio, after initialize (id 1); some session/event notifications left out
ā {"jsonrpc":"2.0","id":2,"method":"session/create","params":{"session":"guard","instructions":"You are a castle guard. Use tools to act.","tools":[{"name":"open_gate","description":"Open a named gate.","parameters":{"type":"object","properties":{"gate":{"type":"string"}}}}],"options":{"toolChoice":"required"}}}
ā {"jsonrpc":"2.0","id":3,"method":"session/respond","params":{"session":"guard","prompt":"Please open the north gate.","stream":true}}
ā {"jsonrpc":"2.0","id":"t-1","method":"tool/call","params":{"session":"guard","requestId":3,"call":{"id":"call_NYH7ā¦","name":"open_gate","arguments":{"gate":"north"}}}}
ā {"jsonrpc":"2.0","id":"t-1","result":{"output":{"opened":false,"reason":"the portcullis chain is jammed"}}}
ā {"jsonrpc":"2.0","id":3,"result":{"session":"guard","text":"The north gate remains shut because the portcullis chain is jammed.",ā¦}}
Comprehensive guides covering architecture, engine bridging, protocols, and benchmarking.
Layers, a turn step by step, steering, errors and concurrency.
WorldState, Persona, NPC, DecisionEngine, ContentGenerator, prompting tips, guardrails and measured performance.
The JSON-RPC protocol v1.0: framing, ordering, session/respond, tool/call, npc/talk, decision/decide and the scripted model.
Command flags, environment variables, exit codes, and pipeline examples for the oam binary.
Request mapping, endpoints, errors, configuration, security, and a comparison with fm serve.
Measurements of fm, fm serve and the framework that shaped the design, all on macOS 27.