LLM has become an essential tool for building software. But integrating them remains an unnecessary pain for Apple developers.
Developers building AI-powered apps typically take a hybrid approach, employing a combination of:
Local models using Core ML or MLX for privacy and offline features Cloud providers like OpenAI or Anthropic for frontier features Apple’s underlying model as a system-level fallback Each comes with different APIs, different requirements, and different integration patterns. Since the amount is large, it will add up quickly. When we interviewed developers about building AI-powered apps, friction around model integration quickly surfaced. One developer put it bluntly:
I immediately thought of using the demo for testing and doing a quick and dirty build, but ended up wasting a lot of time instead. I almost went crazy.
High experimentation costs prevent developers from discovering that local open source models may actually work best for their use cases.
Today we are announcing AnyLanguageModel. This is a Swift package that provides a drop-in replacement for Apple’s Foundation Models framework that supports multiple model providers. Our goal is to reduce the friction of working with LLM on the Apple platform and make it easier to adopt an open source model that runs locally.
solution
The central idea is simple. Replace the import statements and keep the same API.
– Import FoundationModel
+ Import AnyLanguageModel
Here’s how it actually works: Start with Apple’s built-in model.
Let me model = system language model. default
Let me session = language model session(Model:Model)
Let me response = try wait session.respond(to: “Explain quantum computing in one sentence”)
print(response.content)
Next, try an open source model running locally through MLX.
Let me model = MLXL language model(Model ID: “mlx-community/Qwen3-4B-4bit”)
Let me session = language model session(Model:Model)
Let me response = try wait session.respond(to: “Explain quantum computing in one sentence”)
print(response.content)
AnyLanguageModel supports a variety of providers.
Apple Foundation Models: Native integration with Apple’s system models (macOS 26+ / iOS 26+) Core ML: Run transformed models using Neural Engine acceleration MLX: Efficiently run quantized models on Apple Silicon llama.cpp: Load GGUF models via the llama.cpp backend Ollama: Connect to locally served models via Ollama’s HTTP API OpenAI, Anthropic, Google Gemini: Cloud provider for comparison and fallback Hug Face Inference Provider: Hundreds of cloud models powered by world-class inference providers.
We focus on local models that can be downloaded from Hugging Face Hub. Cloud providers are included to lower the barrier to entry and provide a migration path. Make it work and then fix it properly.
Why use the foundation model as the base API?
When designing AnyLanguageModel, I was faced with the choice of creating a new abstraction that tries to capture everything, or building on top of an existing API. We chose the latter and used Apple’s Foundation Models framework as a template.
This may seem counterintuitive. Why are you bound by Apple’s choices? There are several reasons:
Foundation Models are really well designed. It leverages Swift features such as macros for an ergonomic developer experience, and the abstractions around sessions, tools, and generation map well to how LLM works in practice.
It’s intentionally restricted. Foundation Models represent something like the least common denominator of language model features. Rather than seeing this as a weakness, treat it as a stable foundation (hyukhyuk). All Swift developers targeting Apple platforms will encounter this API, so building it directly has less conceptual overhead.
It keeps us grounded. Each additional layer of abstraction takes you further away from the problem you’re actually solving. Abstractions are powerful, but stacking too many of them becomes a problem in itself.
As a result, switching between providers requires minimal code changes, and the core abstractions remain clean and predictable.
Package features: Include only what you need
One of the challenges with multi-backend libraries is dependency bloat. If you only want to run MLX models, you don’t need to include llama.cpp and all its dependencies.
AnyLanguageModel uses properties of Swift 6.1 packages to solve this. Opt in to only the backends you need.
Dependencies: ( .package( url: “https://github.com/mattt/AnyLanguageModel.git”from: “0.4.0”characteristics: (“MLX”) ) )
Available traits include CoreML, MLX, and Llama (for llama.cpp / llama.swift). By default, heavy dependencies are not included. Get basic APIs and cloud providers. This requires only standard URLSession networking.
For Xcode projects (that don’t yet directly support trait declarations), you can create a small internal Swift package that depends on AnyLanguageModel with the traits you want, and add that package as a local dependency. The README contains detailed instructions.
Image support (and API design tradeoffs)
Vision language models are very powerful and widely used. Describe images, extract text from screenshots, analyze charts, and answer questions about visual content. Unfortunately, Apple’s Foundation Models framework currently does not support sending images with prompts.
Building on an existing API means accepting its limitations. Apple will probably add image support in a future release (probably iOS 27), but the vision language model will be more useful the longer you wait. That’s why we’ve expanded beyond what Foundation Models currently offers.
Here is an example of sending an image to Claude.
Let me model = human language model(APIKey: Process information.processInfo.environment(“ANTHROPIC_API_KEY”)!Model: “Claude Sonnet-4-5-20250929”
)
Let me session = language model session(Model:Model)
Let me response = try wait session.respond( to: “What is in this image?”image: .Initialization(URL: URL(fileURLWithPath: “/path/to/image.png”)))
You’re taking a calculated risk here. You may design something that is inconsistent with Apple’s final implementation. But that’s what deprecation warnings are for. Sometimes you need to create an API for a framework that doesn’t yet exist.
Try: chat-ui-swift
To see AnyLanguageModel in action, check out chat-ui-swift, a SwiftUI chat application that demonstrates the library’s capabilities.
The app includes:
Apple Intelligence integration via Foundation Models (macOS 26+) OAuth authentication to access Hugging Face gate models Streaming responses Chat persistence
This is meant as a starting point and can be forked, extended, and swapped out for different models. See how each piece fits together and adjust it to suit your needs.
what’s next
AnyLanguageModel is currently below 1.0. While the core API is stable, we are actively working to bring the full feature set of the Foundation model to all adapters.
Tool invocation across all providers MCP integration for tools and elicitation Guided generation of structured output Local inference performance optimization
This library is a first step towards a larger library. The Unified Inference API provides the scaffolding needed to build seamless agent workflows on Apple platforms, applications where models can use tools, access system resources, and perform complex tasks. More on that in a moment. 🤫
Let’s participate
We’d love to help you make this better:
Give it a try — build something, get the wheels rolling Share your experience — what works? What frustrates you? Hear about the challenges you face when integrating AI into your apps Open issues — feature requests, bug reports, questions Contributions — PRs are welcome
link
I can’t wait to see what you build 🦾
