Every AI feature we shipped in Ionic apps in 2024 and 2025 went to a cloud model. In 2026 that is no longer the only option: iOS 26 exposes Apple's on-device Foundation Models to third-party apps, and Android ships Gemini Nano through ML Kit GenAI and the AICore system service on supported devices. Both run locally, cost nothing per token, work in airplane mode, and never send the user's text off the handset.
Neither is exposed to a web view, so an Ionic app reaches them the same way it
reaches any native capability: through a Capacitor plugin. This tutorial builds
that plugin, wires a three-tier strategy behind one TypeScript call
(on-device → cloud → graceful degradation), and covers the parts that bite in
production — capability detection, token budgets, streaming into ion-content
without jank, and what to tell the app stores.
Examples are Ionic React with Capacitor 7/8. The plugin code is identical for Angular and Vue; only the component at the end changes.
When on-device is the right call
On-device models are small. Treat them as good at bounded, text-in/text-out work and bad at open-ended knowledge:
| Good fit | Send it to the cloud |
|---|---|
| Summarising a note, ticket or thread the user already has | Anything needing your product knowledge or live data |
| Rewriting / tone change / "make this shorter" | Long multi-turn reasoning |
| Classifying free text into your own enum (triage, sentiment, priority) | Tool calling against your API |
| Extracting fields from a scanned receipt or message | Anything you must audit or log centrally |
| Suggested replies in a chat | Image generation |
The economics matter for the apps we consult on: a field-service app that summarises 40 job notes per technician per day is a real cloud bill and a real privacy review. The same feature on-device is free and needs no data processing agreement.
1. Scaffold the plugin
npm init @capacitor/plugin@latest
# name: capacitor-local-llm, id: com.example.locallm
Define the API first — this is the contract the rest of the app codes against, and the one thing you should not change later.
// src/definitions.ts
export type LocalLlmAvailability =
| 'available'
| 'downloading' // Android: AICore is fetching the feature
| 'unsupported-device'
| 'unsupported-os'
| 'disabled'; // user turned off Apple Intelligence
export interface GenerateOptions {
prompt: string;
instructions?: string; // system prompt / role
maxTokens?: number;
temperature?: number;
}
export interface LocalLlmPlugin {
availability(): Promise<{ status: LocalLlmAvailability; model?: string }>;
prepare(): Promise<void>; // warm the session / trigger download
generate(o: GenerateOptions): Promise<{ text: string }>;
generateStream(o: GenerateOptions & { requestId: string }): Promise<void>;
addListener(
event: 'localLlmChunk',
cb: (d: { requestId: string; delta?: string; done?: boolean; error?: string }) => void,
): Promise<{ remove: () => Promise<void> }>;
}
Two things to notice. availability() is a first-class call, not an
afterthought: on-device AI is available on a minority of installed devices
for the next couple of years, so every feature you build must have a
non-AI path. And streaming is modelled as events keyed by requestId, because
Capacitor plugin calls resolve once — partial output has to come back over the
event bridge.
2. iOS: Apple Foundation Models
Requires Xcode 17+, an iOS 26 deployment target for the AI path, and a device with Apple Intelligence enabled. Guard the import so the plugin still compiles for older targets.
// ios/Sources/LocalLlmPlugin/LocalLlm.swift
import Foundation
#if canImport(FoundationModels)
import FoundationModels
#endif
@objc public class LocalLlm: NSObject {
#if canImport(FoundationModels)
private var session: LanguageModelSession?
#endif
public func availability() -> (String, String?) {
#if canImport(FoundationModels)
if #available(iOS 26.0, *) {
switch SystemLanguageModel.default.availability {
case .available:
return ("available", "apple-foundation-on-device")
case .unavailable(.deviceNotEligible):
return ("unsupported-device", nil)
case .unavailable(.appleIntelligenceNotEnabled):
return ("disabled", nil)
case .unavailable(.modelNotReady):
return ("downloading", nil)
@unknown default:
return ("unsupported-os", nil)
}
}
#endif
return ("unsupported-os", nil)
}
@available(iOS 26.0, *)
private func makeSession(_ instructions: String?) -> LanguageModelSession {
if let s = session { return s }
let s = instructions == nil
? LanguageModelSession()
: LanguageModelSession(instructions: instructions!)
session = s
return s
}
@available(iOS 26.0, *)
public func generate(prompt: String, instructions: String?,
maxTokens: Int?, temperature: Double?) async throws -> String {
let opts = GenerationOptions(
temperature: temperature,
maximumResponseTokens: maxTokens
)
let response = try await makeSession(instructions)
.respond(to: prompt, options: opts)
return response.content
}
@available(iOS 26.0, *)
public func stream(prompt: String, instructions: String?,
onDelta: @escaping (String) -> Void) async throws {
let stream = makeSession(instructions).streamResponse(to: prompt)
var emitted = ""
for try await partial in stream {
// partial is cumulative; emit only what is new
let text = partial.content
if text.count > emitted.count {
onDelta(String(text.dropFirst(emitted.count)))
emitted = text
}
}
}
}
Three field notes:
- Partials are cumulative. Apple's stream yields the whole response so far, not a delta. Emitting it raw makes the UI repeat itself; diff as above.
- Reuse the session for a conversation, but throw it away when the topic
changes. Context windows are small (a few thousand tokens) and
exceededContextWindowSizeis a real error you must catch and restart from. - Guardrails fire. Apple's safety layer rejects some prompts and some outputs. Surface a neutral fallback, never a raw error string.
Bridge it:
@objc(LocalLlmPlugin)
public class LocalLlmPlugin: CAPPlugin, CAPBridgedPlugin {
public let identifier = "LocalLlmPlugin"
public let jsName = "LocalLlm"
public let pluginMethods: [CAPPluginMethod] = [
CAPPluginMethod(name: "availability", returnType: CAPPluginReturnPromise),
CAPPluginMethod(name: "prepare", returnType: CAPPluginReturnPromise),
CAPPluginMethod(name: "generate", returnType: CAPPluginReturnPromise),
CAPPluginMethod(name: "generateStream", returnType: CAPPluginReturnPromise),
]
private let impl = LocalLlm()
@objc func availability(_ call: CAPPluginCall) {
let (status, model) = impl.availability()
call.resolve(["status": status, "model": model ?? NSNull()])
}
@objc func generateStream(_ call: CAPPluginCall) {
guard #available(iOS 26.0, *) else {
call.reject("unsupported-os"); return
}
let id = call.getString("requestId") ?? UUID().uuidString
let prompt = call.getString("prompt") ?? ""
let instructions = call.getString("instructions")
call.resolve()
Task { [weak self] in
do {
try await self?.impl.stream(prompt: prompt, instructions: instructions) { delta in
self?.notifyListeners("localLlmChunk", data: ["requestId": id, "delta": delta])
}
self?.notifyListeners("localLlmChunk", data: ["requestId": id, "done": true])
} catch {
self?.notifyListeners("localLlmChunk",
data: ["requestId": id, "error": "\(error)", "done": true])
}
}
}
}
3. Android: Gemini Nano via ML Kit GenAI
On Android the model lives in AICore and is delivered per feature (summarise, rewrite, proofread, image description) rather than as a raw prompt API on every device. Use the ML Kit GenAI Summarization/Rewriting APIs where they fit; they cover the majority of real requests and handle the model download for you.
// android/src/main/java/com/example/locallm/LocalLlm.kt
import com.google.mlkit.genai.common.FeatureStatus
import com.google.mlkit.genai.summarization.Summarization
import com.google.mlkit.genai.summarization.SummarizerOptions
import com.google.mlkit.genai.summarization.SummarizationRequest
class LocalLlm(private val context: Context) {
private val summarizer by lazy {
Summarization.getClient(
SummarizerOptions.builder(context)
.setInputType(SummarizerOptions.InputType.ARTICLE)
.setOutputType(SummarizerOptions.OutputType.THREE_BULLETS)
.setLanguage(SummarizerOptions.Language.ENGLISH)
.build()
)
}
suspend fun availability(): String =
when (summarizer.checkFeatureStatus().await()) {
FeatureStatus.AVAILABLE -> "available"
FeatureStatus.DOWNLOADABLE, FeatureStatus.DOWNLOADING -> "downloading"
else -> "unsupported-device"
}
suspend fun prepare() {
if (summarizer.checkFeatureStatus().await() == FeatureStatus.DOWNLOADABLE) {
summarizer.downloadFeature(null).await() // ~hundreds of MB, Wi-Fi only in practice
}
}
fun stream(text: String, onDelta: (String) -> Unit, onDone: (String?) -> Unit) {
val request = SummarizationRequest.builder(text).build()
summarizer.runInference(request) { partial -> onDelta(partial) }
.addOnSuccessListener { onDone(null) }
.addOnFailureListener { e -> onDone(e.message ?: "inference-failed") }
}
}
Android specifics worth budgeting for:
- The feature download is large and asynchronous. Never block first launch on
it — call
prepare()behind a user action ("Enable on-device summaries") or on an idle Wi-Fi callback, and keep the cloud path live until it finishes. checkFeatureStatus()must be re-read at runtime. A device can go fromDOWNLOADABLEtoAVAILABLEbetween two screens.- Coverage is narrow: current Pixel and recent flagship Samsung/Xiaomi devices.
Treat
unsupported-deviceas the default case, not the exception. - Call
close()on the client inonDestroy()or you leak an AICore session.
4. One TypeScript entry point, three tiers
The app should never branch on platform. Put the policy in one service:
// src/ai/generate.ts
import { LocalLlm } from 'capacitor-local-llm';
type Tier = 'on-device' | 'cloud' | 'none';
let cached: Promise<Tier> | null = null;
async function tier(): Promise<Tier> {
cached ??= (async () => {
try {
const { status } = await LocalLlm.availability();
if (status === 'available') return 'on-device';
} catch { /* plugin missing on web */ }
return navigator.onLine ? 'cloud' : 'none';
})();
return cached;
}
export async function summarise(text: string, signal?: AbortSignal) {
switch (await tier()) {
case 'on-device': {
const { text: out } = await LocalLlm.generate({
prompt: text,
instructions: 'Summarise the note in three short bullets. No preamble.',
maxTokens: 220,
});
return { text: out, source: 'on-device' as const };
}
case 'cloud': {
const r = await fetch('/api/summarise', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ text }),
signal,
});
if (!r.ok) throw new Error('cloud-failed');
return { text: (await r.json()).summary, source: 'cloud' as const };
}
default:
throw new Error('offline-and-unsupported');
}
}
Invalidate cached on resume (Capacitor's App plugin) — users toggle Apple
Intelligence and Android finishes downloads while your app is backgrounded.
Do not silently fall back from on-device to cloud for content the user was told stays local. If your privacy copy says "summarised on your device", the cloud tier must be opt-in, and the UI has to say which one ran. We label it with a small chip on the result.
5. Streaming into the UI without jank
The mistake we see most often: appending each delta straight to React state.
On-device models emit dozens of small chunks a second, and an ion-content
with a re-rendering markdown block will drop frames on exactly the mid-range
devices that most need the help. Buffer to one animation frame:
function useLocalStream() {
const [text, setText] = useState('');
const buf = useRef('');
const raf = useRef<number | null>(null);
const flush = () => {
raf.current = null;
setText(buf.current);
};
useEffect(() => {
let handle: { remove: () => Promise<void> } | undefined;
LocalLlm.addListener('localLlmChunk', ({ delta, done }) => {
if (delta) buf.current += delta;
if (done && raf.current) { cancelAnimationFrame(raf.current); flush(); return; }
raf.current ??= requestAnimationFrame(flush);
}).then(h => { handle = h; });
return () => { handle?.remove(); if (raf.current) cancelAnimationFrame(raf.current); };
}, []);
const run = (prompt: string) => {
buf.current = ''; setText('');
return LocalLlm.generateStream({ prompt, requestId: crypto.randomUUID() });
};
return { text, run };
}
Two more UI rules we apply on every engagement: render plain text while
streaming and only parse markdown on done (incremental markdown parsing is
the other half of the jank), and always give the user a visible Stop. Small
models loop on ambiguous prompts, and a hard maxTokens plus a Stop button is
the only thing standing between that and a hot phone.
6. Testing and store review
- Simulators and emulators have no model.
availability()returnsunsupported-device. Add a dev-only mock tier so the whole team is not blocked on one physical device, and put a real Pixel and a real iPhone 16+ in CI's manual smoke pass. - Test the three tiers explicitly. Airplane mode + unsupported device is the state that generates support tickets.
- Store metadata. On-device inference is not a third-party data collection, so it does not add Data Safety or privacy-manifest entries — but the cloud tier does, and both stores want AI-generated content labelled and a report mechanism if output is user-visible and free-form. If you removed the cloud tier entirely, go back and delete the stale declarations.
- Binary size. The plugin adds kilobytes, not megabytes: the model belongs to the OS. That is the whole point, and it is worth saying in your release notes.
Where this leaves the stack
The pattern that has held up for us: one typed plugin surface, an availability gate, on-device for bounded text work, cloud for anything that needs your data or your reasoning, and a UI that never pretends AI is present when it is not. Build it that way and the coverage question answers itself over time — each OS release moves more of your users into the free, private tier without you shipping anything.
If you are weighing an on-device AI feature for an existing Ionic app and want a second opinion on scope, device coverage or the privacy story, get in touch — our Ionic consultants do this work daily.