Can AI Work Offline in Flutter? Here's What's Possible
Most Flutter apps with AI features assume a connection: type a message, call Gemini or an API, wait for a response. That assumption breaks the moment a user loses signal — and for a large share of the world's users, especially outside major cities, unreliable connectivity isn't the exception you design around. It's the default you should be designing for.
So: can AI actually work offline in a Flutter app? Yes — but not in the single way most people picture when they hear the question. There isn't one answer, there are three levels, and knowing which one a feature actually needs is most of the engineering work.
The assumption worth dropping first
The instinctive picture of "AI" is a large model, a live API call, cloud compute — which quietly becomes the belief that AI requires an internet connection by definition. That was closer to true a few years ago. It isn't anymore. Lightweight on-device models, aggressive caching, and plain rule-based logic can each deliver something a user would call "smart" without a live connection, and a well-designed app usually needs all three, not just one.
Level 1: Fully offline, on-device models
The closest thing to true offline AI is running a model directly on the device instead of calling an API. Google's Gemma family is built specifically for this — small enough to run on a phone's CPU or NPU, capable of summarization, question-answering, and structured text generation, without a server in the loop.
Flutter itself doesn't run these models; it's the UI and orchestration layer sitting on top of a native inference engine. The most direct path today is the flutter_gemma package, which wraps Google's on-device runtime and handles the model-loading and inference calls for you:
dependencies:
flutter_gemma: ^0.8.0
final gemma = FlutterGemmaPlugin.instance;
Future<void> initModel() async {
// Model file is downloaded once and stored locally — no network needed after this.
await gemma.modelManager.installModelFromAsset('gemma-2b-it.bin');
}
Future<String> summarizeOffline(String text) async {
final model = await gemma.createModel(modelType: ModelType.gemmaIt);
final session = await model.createSession();
await session.addQueryChunk(Message.text(text: 'Summarize in one sentence: $text'));
final response = await session.getResponse();
await session.close();
return response;
}
Everything after installModelFromAsset runs entirely on-device — no fallback to a server, no latency spike from a bad connection, no data leaving the phone. That last property matters as much as the offline capability itself for anything privacy-sensitive: a journaling app, a health-tracking feature, anything where sending user text to a cloud API is a harder sell than running it locally.
The honest limitations: a 2B-parameter on-device model is meaningfully less capable than a cloud model at complex reasoning, inference is slower on older or lower-end hardware, and the model file itself adds real weight to your app's install size. Gemma-class models don't replace Gemini — they make a specific slice of AI features possible without one.
Level 2: Hybrid — the approach most production apps actually need
Most real systems shouldn't choose between offline and online; they should use both, switching based on what's actually available right now.
Future<String> getAiResponse(String prompt) async {
final connectivity = await Connectivity().checkConnectivity();
final isOnline = connectivity != ConnectivityResult.none;
if (isOnline) {
try {
return await cloudModel.generate(prompt); // Gemini — higher quality
} catch (_) {
// Network reported available but the call still failed — degrade, don't crash.
return await localModel.summarizeOffline(prompt);
}
}
return await localModel.summarizeOffline(prompt); // Gemma — always available
}
The try/catch inside the "online" branch matters as much as the connectivity check itself: Connectivity().checkConnectivity() tells you whether a network interface is up, not whether the actual API call will succeed. A user on a weak or captive-portal connection reads as "online" and then times out anyway — treating connectivity as a hint rather than a guarantee is what keeps that case from becoming a crash instead of a graceful fallback.
This pattern is the right fit for exactly the features you'd expect: a study assistant that gives richer explanations online and simpler ones offline, a finance app that gets sharper insights from a cloud model but never goes silent without one, anything where "slightly less capable" beats "completely unavailable."
Level 3: Smart without a model at all
The most underused approach, and often the highest-leverage one: many features that feel like AI don't need a model running at all, online or off.
If a user asks the same handful of questions repeatedly, cache the previous response and serve it instantly instead of re-generating it. If a request maps cleanly onto a small set of known intents ("show my progress," "what's next"), route it to a predefined action with if/switch logic instead of paying model latency for something deterministic. If you can precompute a recommendation from data you already have — yesterday's most-skipped lesson, a study streak — do that ahead of time and just display it.
final _responseCache = <String, String>{};
Future<String> getCachedOrGenerate(String prompt, Future<String> Function() generate) async {
if (_responseCache.containsKey(prompt)) {
return _responseCache[prompt]!; // instant, no model call, works offline
}
final response = await generate();
_responseCache[prompt] = response;
return response;
}
Store the cache in Hive or Isar rather than an in-memory map if it needs to survive app restarts. None of this is AI in the model sense — but the user experiencing it can't tell the difference between "the model figured this out" and "the app remembered the answer from last time," and the second one costs nothing and never fails offline.
Designing for offline from the start
Retrofitting offline support onto a cloud-only feature is much harder than building the seam in from day one. A few decisions make the difference:
Keep the cloud and local paths as genuinely separate layers, not one function with a network check bolted in — the hybrid example above works because cloudModel and localModel share a return type and neither knows the other exists.
Cache aggressively and specifically — not just final responses, but the intermediate data (embeddings, parsed structures) that's expensive to regenerate, so a reconnect doesn't mean redoing work you already paid for.
Queue rather than fail for anything that genuinely requires a network — a message send, a sync, a purchase. Store the pending action locally and retry when connectivity returns, instead of showing an error and discarding the user's input:
final _pendingActions = <Map<String, dynamic>>[];
Future<void> submitAction(Map<String, dynamic> action) async {
if (await isOnline()) {
await api.send(action);
} else {
_pendingActions.add(action); // persist this to Hive/Isar in a real app
}
}
// Call this when connectivity is restored (e.g. via a Connectivity listener).
Future<void> flushPendingActions() async {
final toRetry = List<Map<String, dynamic>>.from(_pendingActions);
_pendingActions.clear();
for (final action in toRetry) {
await submitAction(action); // re-queues automatically if still offline
}
}
Design for latency even when there's no network round-trip — on-device inference isn't instant either, so a loading state and progressive feedback still matter at Level 1, not just Level 2.
The constraints that don't go away
None of this makes offline AI free. Device hardware varies enormously — a five-year-old Android phone and a current iPhone will run the same on-device model at very different speeds, and you have to test on the low end, not just your own device. Smaller models genuinely reason less well and hold less context than their cloud counterparts, which limits what Level 1 alone can responsibly promise a user. iOS and Android differ in how they expose native inference, which shows up as platform-specific bugs you won't hit with a pure API-based feature. And debugging is harder across the board — a failure buried inside on-device inference gives you far less observability than a failed HTTP request with a status code and a response body.
What this actually comes down to
Offline isn't an edge case to handle after the "real" feature ships — for a meaningful share of users, it's the default condition the feature has to work under from day one. Pure cloud AI and pure offline AI both fail this test for different reasons: one goes silent without a connection, the other can't match a cloud model's reasoning. Hybrid systems, backed by aggressive caching and a fallback to simple rule-based logic when no model is warranted at all, are what actually hold up. The bar a user applies isn't "is this AI" — it's whether the app kept working when their connection didn't.
References
