"Can we add a chat assistant to the app?" is the most common feature request we hear in 2026. The model call is the easy part. The parts that decide whether the feature is good are streaming (nobody waits four seconds for a blank bubble), a backend that keeps your API keys and costs under control, and sensible behaviour when the phone is offline. This tutorial builds all three for an Ionic app on Capacitor. The examples use Ionic React, but the backend and the patterns are identical for Angular and Vue.
Architecture
Ionic app --POST /api/chat (SSE)--> your API --streaming call--> model provider
The app never holds a provider key. The API adds the system prompt, retrieves any grounding content (your help docs, the user's account data), calls the provider with streaming enabled, and forwards tokens to the app as server-sent events. Everything about the model — which one, how much context, what it is allowed to say — lives on the server and can change without an app release.
1. The streaming backend
A minimal Node/Express handler. The provider call is abstracted behind
streamCompletion(); implement it with whichever provider SDK you use — all
the major ones expose an async iterator of text deltas.
import express from 'express';
import { streamCompletion } from './provider'; // your provider adapter
import { checkBudget, recordUsage } from './budget';
const app = express();
app.use(express.json());
app.post('/api/chat', async (req, res) => {
const userId = req.auth.userId; // from your auth middleware
const { messages } = req.body as { messages: { role: string; content: string }[] };
if (!(await checkBudget(userId))) {
return res.status(429).json({ error: 'daily_limit' });
}
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.flushHeaders();
const system = 'You are the in-app assistant for Example. Answer only from the ' +
'provided help content. If unsure, say so and offer to contact support.';
let outputTokens = 0;
const abort = new AbortController();
req.on('close', () => abort.abort()); // client cancelled: stop paying
try {
for await (const delta of streamCompletion({ system, messages, signal: abort.signal })) {
outputTokens += 1;
res.write(`data: ${JSON.stringify({ text: delta })}\n\n`);
}
res.write('data: [DONE]\n\n');
} catch (err) {
res.write(`data: ${JSON.stringify({ error: 'upstream' })}\n\n`);
} finally {
await recordUsage(userId, outputTokens);
res.end();
}
});
Two details matter. req.on('close') aborts the upstream call when the user
hits stop or backgrounds the app — otherwise you pay for tokens nobody reads.
And checkBudget runs before the provider call, so a runaway client cannot
run up a bill.
2. Consuming the stream in the app
Capacitor's WebView supports fetch with a readable body, so you do not need
a native plugin for SSE. A small reader that yields text chunks:
export async function* streamChat(
messages: { role: string; content: string }[],
signal: AbortSignal,
): AsyncGenerator<string> {
const res = await fetch(`${API_BASE}/api/chat`, {
method: 'POST',
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${await getToken()}` },
body: JSON.stringify({ messages }),
signal,
});
if (res.status === 429) throw new Error('daily_limit');
if (!res.ok || !res.body) throw new Error('chat_failed');
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const events = buffer.split('\n\n');
buffer = events.pop() ?? '';
for (const evt of events) {
const line = evt.replace(/^data: /, '');
if (line === '[DONE]') return;
const payload = JSON.parse(line);
if (payload.error) throw new Error(payload.error);
yield payload.text as string;
}
}
}
Set server.allowNavigation or CORS appropriately if your API is on a
different origin from the WebView — on native the origin is
capacitor://localhost (iOS) or http://localhost (Android).
3. The Ionic chat UI
The naive approach — append every token to React state and let
IonContent re-render a growing list — stutters on mid-range Android by the
second paragraph. Three fixes: batch token updates with
requestAnimationFrame, keep completed messages in a memoised list and only
the in-flight message in fast-changing state, and scroll with
scrollToBottom() at a throttled interval rather than on every token.
import { useRef, useState, useCallback } from 'react';
import {
IonPage, IonHeader, IonToolbar, IonTitle, IonContent, IonFooter,
IonItem, IonInput, IonButton, IonIcon, IonSpinner,
} from '@ionic/react';
import { send, stop } from 'ionicons/icons';
import { streamChat } from './stream-chat';
import { MessageList } from './MessageList'; // memoised, renders completed messages
type Msg = { role: 'user' | 'assistant'; content: string };
export function ChatPage() {
const [history, setHistory] = useState<Msg[]>([]);
const [draft, setDraft] = useState('');
const [live, setLive] = useState<string | null>(null); // in-flight assistant text
const contentRef = useRef<HTMLIonContentElement>(null);
const abortRef = useRef<AbortController | null>(null);
const ask = useCallback(async () => {
const question = draft.trim();
if (!question || abortRef.current) return;
const next = [...history, { role: 'user', content: question } as Msg];
setHistory(next);
setDraft('');
setLive('');
const abort = new AbortController();
abortRef.current = abort;
let pending = '';
let raf = 0;
const flush = () => { setLive((cur) => (cur ?? '') + pending); pending = ''; raf = 0;
contentRef.current?.scrollToBottom(0); };
try {
for await (const chunk of streamChat(next, abort.signal)) {
pending += chunk;
if (!raf) raf = requestAnimationFrame(flush);
}
} catch (e) {
pending += e instanceof Error && e.message === 'daily_limit'
? '\n\nYou have reached today\u2019s assistant limit.'
: '\n\nSorry, something went wrong. Please try again.';
} finally {
if (raf) cancelAnimationFrame(raf);
flush();
setLive((final) => {
setHistory((h) => [...h, { role: 'assistant', content: final ?? '' }]);
return null;
});
abortRef.current = null;
}
}, [draft, history]);
return (
<IonPage>
<IonHeader><IonToolbar><IonTitle>Assistant</IonTitle></IonToolbar></IonHeader>
<IonContent ref={contentRef}>
<MessageList messages={history} />
{live !== null && (
<div className="bubble assistant">{live || <IonSpinner name="dots" />}</div>
)}
</IonContent>
<IonFooter>
<IonItem lines="none">
<IonInput value={draft} placeholder="Ask a question"
onIonInput={(e) => setDraft(e.detail.value ?? '')}
onKeyDown={(e) => e.key === 'Enter' && ask()} />
{abortRef.current
? <IonButton fill="clear" onClick={() => abortRef.current?.abort()}><IonIcon icon={stop} /></IonButton>
: <IonButton fill="clear" onClick={ask}><IonIcon icon={send} /></IonButton>}
</IonItem>
</IonFooter>
</IonPage>
);
}
Render assistant text through a Markdown component that supports streaming
partial input (most do; test with an unterminated code fence). Keep the
MessageList memoised on history so completed bubbles never re-render
while tokens stream.
4. Offline fallback
Field apps lose connectivity. Use @capacitor/network to check before
sending, queue the question locally, and — for the high-value case — ship a
small set of canned answers or a tiny on-device model for FAQs:
import { Network } from '@capacitor/network';
const { connected } = await Network.getStatus();
if (!connected) {
const hit = searchLocalFaq(question); // prebuilt index shipped with the app
return hit ?? 'You are offline. Your question will be sent when you reconnect.';
}
Tell the user which mode they are in; a silent downgrade to canned answers is the kind of thing that ends up in a one-star review.
5. Cost controls that ship with v1
- Per-user daily budget enforced server-side (the
checkBudgetcall above), with a clear in-app message when it is reached - Context trimming — send the last N turns plus a running summary, not the whole history
- Model routing — a small, cheap model answers first; escalate to a larger one only for flagged question types
- Prompt caching where your provider supports it for the static system prompt and grounding content
- Usage dashboard — tokens and cost per day and per feature, from the
recordUsagetable, reviewed weekly for the first month
6. Before you submit to the stores
Generated content and third-party AI services affect your App Store privacy labels and Google Play data-safety form, and Apple's review guidelines address AI-generated content explicitly. Document what data leaves the device (the question, the grounding content, identifiers), where it goes, and how long the provider retains it, and make sure the in-app experience says the answers are AI-generated and may be wrong. Doing this at design time is an hour; doing it during review is a week.
This is the build we deliver on our AI features for Ionic & Capacitor apps engagements — usually two to four weeks from discovery to a feature-flagged beta. Talk to us about the workflow in your app that would benefit most.