+1 (415) 843-4662

On-device AI in Ionic apps: Apple Foundation Models, Gemini Nano and a Capacitor plugin that falls back cleanly

Every AI feature we shipped in Ionic apps in 2024 and 2025 went to a cloud model. In 2026 that is no longer the only option: iOS 26 exposes Apple's on-device Foundation Models to third-party apps, and Android ships Gemini Nano through ML Kit GenAI and the AICore system service on supported devices. Both run locally, cost nothing per token, work in airplane mode, and never send the user's text off the handset.

Neither is exposed to a web view, so an Ionic app reaches them the same way it reaches any native capability: through a Capacitor plugin. This tutorial builds that plugin, wires a three-tier strategy behind one TypeScript call (on-device → cloud → graceful degradation), and covers the parts that bite in production — capability detection, token budgets, streaming into ion-content without jank, and what to tell the app stores.

Examples are Ionic React with Capacitor 7/8. The plugin code is identical for Angular and Vue; only the component at the end changes.

When on-device is the right call

On-device models are small. Treat them as good at bounded, text-in/text-out work and bad at open-ended knowledge:

Good fitSend it to the cloud
Summarising a note, ticket or thread the user already hasAnything needing your product knowledge or live data
Rewriting / tone change / "make this shorter"Long multi-turn reasoning
Classifying free text into your own enum (triage, sentiment, priority)Tool calling against your API
Extracting fields from a scanned receipt or messageAnything you must audit or log centrally
Suggested replies in a chatImage generation

The economics matter for the apps we consult on: a field-service app that summarises 40 job notes per technician per day is a real cloud bill and a real privacy review. The same feature on-device is free and needs no data processing agreement.

1. Scaffold the plugin

npm init @capacitor/plugin@latest
# name: capacitor-local-llm, id: com.example.locallm

Define the API first — this is the contract the rest of the app codes against, and the one thing you should not change later.

// src/definitions.ts
export type LocalLlmAvailability =
  | 'available'
  | 'downloading'      // Android: AICore is fetching the feature
  | 'unsupported-device'
  | 'unsupported-os'
  | 'disabled';        // user turned off Apple Intelligence

export interface GenerateOptions {
  prompt: string;
  instructions?: string;   // system prompt / role
  maxTokens?: number;
  temperature?: number;
}

export interface LocalLlmPlugin {
  availability(): Promise<{ status: LocalLlmAvailability; model?: string }>;
  prepare(): Promise<void>;                 // warm the session / trigger download
  generate(o: GenerateOptions): Promise<{ text: string }>;
  generateStream(o: GenerateOptions & { requestId: string }): Promise<void>;
  addListener(
    event: 'localLlmChunk',
    cb: (d: { requestId: string; delta?: string; done?: boolean; error?: string }) => void,
  ): Promise<{ remove: () => Promise<void> }>;
}

Two things to notice. availability() is a first-class call, not an afterthought: on-device AI is available on a minority of installed devices for the next couple of years, so every feature you build must have a non-AI path. And streaming is modelled as events keyed by requestId, because Capacitor plugin calls resolve once — partial output has to come back over the event bridge.

2. iOS: Apple Foundation Models

Requires Xcode 17+, an iOS 26 deployment target for the AI path, and a device with Apple Intelligence enabled. Guard the import so the plugin still compiles for older targets.

// ios/Sources/LocalLlmPlugin/LocalLlm.swift
import Foundation
#if canImport(FoundationModels)
import FoundationModels
#endif

@objc public class LocalLlm: NSObject {

  #if canImport(FoundationModels)
  private var session: LanguageModelSession?
  #endif

  public func availability() -> (String, String?) {
    #if canImport(FoundationModels)
    if #available(iOS 26.0, *) {
      switch SystemLanguageModel.default.availability {
      case .available:
        return ("available", "apple-foundation-on-device")
      case .unavailable(.deviceNotEligible):
        return ("unsupported-device", nil)
      case .unavailable(.appleIntelligenceNotEnabled):
        return ("disabled", nil)
      case .unavailable(.modelNotReady):
        return ("downloading", nil)
      @unknown default:
        return ("unsupported-os", nil)
      }
    }
    #endif
    return ("unsupported-os", nil)
  }

  @available(iOS 26.0, *)
  private func makeSession(_ instructions: String?) -> LanguageModelSession {
    if let s = session { return s }
    let s = instructions == nil
      ? LanguageModelSession()
      : LanguageModelSession(instructions: instructions!)
    session = s
    return s
  }

  @available(iOS 26.0, *)
  public func generate(prompt: String, instructions: String?,
                       maxTokens: Int?, temperature: Double?) async throws -> String {
    let opts = GenerationOptions(
      temperature: temperature,
      maximumResponseTokens: maxTokens
    )
    let response = try await makeSession(instructions)
      .respond(to: prompt, options: opts)
    return response.content
  }

  @available(iOS 26.0, *)
  public func stream(prompt: String, instructions: String?,
                     onDelta: @escaping (String) -> Void) async throws {
    let stream = makeSession(instructions).streamResponse(to: prompt)
    var emitted = ""
    for try await partial in stream {
      // partial is cumulative; emit only what is new
      let text = partial.content
      if text.count > emitted.count {
        onDelta(String(text.dropFirst(emitted.count)))
        emitted = text
      }
    }
  }
}

Three field notes:

  • Partials are cumulative. Apple's stream yields the whole response so far, not a delta. Emitting it raw makes the UI repeat itself; diff as above.
  • Reuse the session for a conversation, but throw it away when the topic changes. Context windows are small (a few thousand tokens) and exceededContextWindowSize is a real error you must catch and restart from.
  • Guardrails fire. Apple's safety layer rejects some prompts and some outputs. Surface a neutral fallback, never a raw error string.

Bridge it:

@objc(LocalLlmPlugin)
public class LocalLlmPlugin: CAPPlugin, CAPBridgedPlugin {
  public let identifier = "LocalLlmPlugin"
  public let jsName = "LocalLlm"
  public let pluginMethods: [CAPPluginMethod] = [
    CAPPluginMethod(name: "availability", returnType: CAPPluginReturnPromise),
    CAPPluginMethod(name: "prepare", returnType: CAPPluginReturnPromise),
    CAPPluginMethod(name: "generate", returnType: CAPPluginReturnPromise),
    CAPPluginMethod(name: "generateStream", returnType: CAPPluginReturnPromise),
  ]
  private let impl = LocalLlm()

  @objc func availability(_ call: CAPPluginCall) {
    let (status, model) = impl.availability()
    call.resolve(["status": status, "model": model ?? NSNull()])
  }

  @objc func generateStream(_ call: CAPPluginCall) {
    guard #available(iOS 26.0, *) else {
      call.reject("unsupported-os"); return
    }
    let id = call.getString("requestId") ?? UUID().uuidString
    let prompt = call.getString("prompt") ?? ""
    let instructions = call.getString("instructions")
    call.resolve()
    Task { [weak self] in
      do {
        try await self?.impl.stream(prompt: prompt, instructions: instructions) { delta in
          self?.notifyListeners("localLlmChunk", data: ["requestId": id, "delta": delta])
        }
        self?.notifyListeners("localLlmChunk", data: ["requestId": id, "done": true])
      } catch {
        self?.notifyListeners("localLlmChunk",
          data: ["requestId": id, "error": "\(error)", "done": true])
      }
    }
  }
}

3. Android: Gemini Nano via ML Kit GenAI

On Android the model lives in AICore and is delivered per feature (summarise, rewrite, proofread, image description) rather than as a raw prompt API on every device. Use the ML Kit GenAI Summarization/Rewriting APIs where they fit; they cover the majority of real requests and handle the model download for you.

// android/src/main/java/com/example/locallm/LocalLlm.kt
import com.google.mlkit.genai.common.FeatureStatus
import com.google.mlkit.genai.summarization.Summarization
import com.google.mlkit.genai.summarization.SummarizerOptions
import com.google.mlkit.genai.summarization.SummarizationRequest

class LocalLlm(private val context: Context) {

  private val summarizer by lazy {
    Summarization.getClient(
      SummarizerOptions.builder(context)
        .setInputType(SummarizerOptions.InputType.ARTICLE)
        .setOutputType(SummarizerOptions.OutputType.THREE_BULLETS)
        .setLanguage(SummarizerOptions.Language.ENGLISH)
        .build()
    )
  }

  suspend fun availability(): String =
    when (summarizer.checkFeatureStatus().await()) {
      FeatureStatus.AVAILABLE -> "available"
      FeatureStatus.DOWNLOADABLE, FeatureStatus.DOWNLOADING -> "downloading"
      else -> "unsupported-device"
    }

  suspend fun prepare() {
    if (summarizer.checkFeatureStatus().await() == FeatureStatus.DOWNLOADABLE) {
      summarizer.downloadFeature(null).await()   // ~hundreds of MB, Wi-Fi only in practice
    }
  }

  fun stream(text: String, onDelta: (String) -> Unit, onDone: (String?) -> Unit) {
    val request = SummarizationRequest.builder(text).build()
    summarizer.runInference(request) { partial -> onDelta(partial) }
      .addOnSuccessListener { onDone(null) }
      .addOnFailureListener { e -> onDone(e.message ?: "inference-failed") }
  }
}

Android specifics worth budgeting for:

  • The feature download is large and asynchronous. Never block first launch on it — call prepare() behind a user action ("Enable on-device summaries") or on an idle Wi-Fi callback, and keep the cloud path live until it finishes.
  • checkFeatureStatus() must be re-read at runtime. A device can go from DOWNLOADABLE to AVAILABLE between two screens.
  • Coverage is narrow: current Pixel and recent flagship Samsung/Xiaomi devices. Treat unsupported-device as the default case, not the exception.
  • Call close() on the client in onDestroy() or you leak an AICore session.

4. One TypeScript entry point, three tiers

The app should never branch on platform. Put the policy in one service:

// src/ai/generate.ts
import { LocalLlm } from 'capacitor-local-llm';

type Tier = 'on-device' | 'cloud' | 'none';
let cached: Promise<Tier> | null = null;

async function tier(): Promise<Tier> {
  cached ??= (async () => {
    try {
      const { status } = await LocalLlm.availability();
      if (status === 'available') return 'on-device';
    } catch { /* plugin missing on web */ }
    return navigator.onLine ? 'cloud' : 'none';
  })();
  return cached;
}

export async function summarise(text: string, signal?: AbortSignal) {
  switch (await tier()) {
    case 'on-device': {
      const { text: out } = await LocalLlm.generate({
        prompt: text,
        instructions: 'Summarise the note in three short bullets. No preamble.',
        maxTokens: 220,
      });
      return { text: out, source: 'on-device' as const };
    }
    case 'cloud': {
      const r = await fetch('/api/summarise', {
        method: 'POST',
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ text }),
        signal,
      });
      if (!r.ok) throw new Error('cloud-failed');
      return { text: (await r.json()).summary, source: 'cloud' as const };
    }
    default:
      throw new Error('offline-and-unsupported');
  }
}

Invalidate cached on resume (Capacitor's App plugin) — users toggle Apple Intelligence and Android finishes downloads while your app is backgrounded.

Do not silently fall back from on-device to cloud for content the user was told stays local. If your privacy copy says "summarised on your device", the cloud tier must be opt-in, and the UI has to say which one ran. We label it with a small chip on the result.

5. Streaming into the UI without jank

The mistake we see most often: appending each delta straight to React state. On-device models emit dozens of small chunks a second, and an ion-content with a re-rendering markdown block will drop frames on exactly the mid-range devices that most need the help. Buffer to one animation frame:

function useLocalStream() {
  const [text, setText] = useState('');
  const buf = useRef('');
  const raf = useRef<number | null>(null);

  const flush = () => {
    raf.current = null;
    setText(buf.current);
  };

  useEffect(() => {
    let handle: { remove: () => Promise<void> } | undefined;
    LocalLlm.addListener('localLlmChunk', ({ delta, done }) => {
      if (delta) buf.current += delta;
      if (done && raf.current) { cancelAnimationFrame(raf.current); flush(); return; }
      raf.current ??= requestAnimationFrame(flush);
    }).then(h => { handle = h; });
    return () => { handle?.remove(); if (raf.current) cancelAnimationFrame(raf.current); };
  }, []);

  const run = (prompt: string) => {
    buf.current = ''; setText('');
    return LocalLlm.generateStream({ prompt, requestId: crypto.randomUUID() });
  };

  return { text, run };
}

Two more UI rules we apply on every engagement: render plain text while streaming and only parse markdown on done (incremental markdown parsing is the other half of the jank), and always give the user a visible Stop. Small models loop on ambiguous prompts, and a hard maxTokens plus a Stop button is the only thing standing between that and a hot phone.

6. Testing and store review

  • Simulators and emulators have no model. availability() returns unsupported-device. Add a dev-only mock tier so the whole team is not blocked on one physical device, and put a real Pixel and a real iPhone 16+ in CI's manual smoke pass.
  • Test the three tiers explicitly. Airplane mode + unsupported device is the state that generates support tickets.
  • Store metadata. On-device inference is not a third-party data collection, so it does not add Data Safety or privacy-manifest entries — but the cloud tier does, and both stores want AI-generated content labelled and a report mechanism if output is user-visible and free-form. If you removed the cloud tier entirely, go back and delete the stale declarations.
  • Binary size. The plugin adds kilobytes, not megabytes: the model belongs to the OS. That is the whole point, and it is worth saying in your release notes.

Where this leaves the stack

The pattern that has held up for us: one typed plugin surface, an availability gate, on-device for bounded text work, cloud for anything that needs your data or your reasoning, and a UI that never pretends AI is present when it is not. Build it that way and the coverage question answers itself over time — each OS release moves more of your users into the free, private tier without you shipping anything.

If you are weighing an on-device AI feature for an existing Ionic app and want a second opinion on scope, device coverage or the privacy story, get in touch — our Ionic consultants do this work daily.