Adding On-Device AI to an Android App with Gemini Nano

By Michael Samuel Naeem · February 10, 2026 · 5 min read

On-device AI finally crossed the line from demo to shippable on Android. With Gemini Nano running through AICore, you can summarize a chat thread, rewrite a message in a different tone, proofread text, or describe an image — all without a network call, without a per-request bill, and without user data leaving the phone. For a class of features that used to require a cloud round-trip, that's a meaningful shift.

The catch, and the thing most tutorials gloss over, is that on-device AI on Android is a conditional capability. It exists on a subset of devices, the model is small and task-specific, and you must design for graceful absence. Here's how to add Gemini Nano features properly, what the APIs actually give you, and where the sharp edges are.

The pieces: AICore, Gemini Nano, and ML Kit GenAI

Three layers matter:

• AICore is a system service (updated via Google Play system updates) that manages the on-device foundation model — downloading it, running inference on the NPU, and isolating it from apps. Your app never ships the model; AICore owns it.
• Gemini Nano is the model AICore runs. It's small by design, tuned for on-device latency and power, not for open-ended reasoning.
• ML Kit GenAI APIs are the high-level, stable entry point for most apps. Rather than prompting the raw model, you call purpose-built APIs: Summarization, Rewriting, Proofreading, and Image Description. These wrap Gemini Nano with task-specific tuning and are the path I'd recommend for production.

For lower-level, prompt-based access there's the experimental generative APIs, but the ML Kit GenAI feature APIs are where reliability lives today.

Always check availability first

The single most important habit: never assume the feature exists. Device support is narrow (Pixel 9 family, recent Galaxy S flagships, and expanding), and even on a supported device the model may still need to download. Check status and drive UI from it.

val summarizer = Summarization.getClient(
    SummarizerOptions.builder(context)
        .setInputType(InputType.ARTICLE)
        .setOutputType(OutputType.THREE_BULLETS)
        .setLanguage(Language.ENGLISH)
        .build()
)

when (summarizer.checkFeatureStatus().await()) {
    FeatureStatus.AVAILABLE -> runSummary(summarizer)
    FeatureStatus.DOWNLOADABLE -> {
        // trigger download, show progress, then run
        summarizer.downloadFeature(downloadCallback)
    }
    FeatureStatus.DOWNLOADING -> showDownloadingState()
    FeatureStatus.UNAVAILABLE -> useCloudFallbackOrHideFeature()
}

UNAVAILABLE is not an edge case — it's the majority of the installed base right now. Decide up front whether the feature degrades to a cloud call or simply hides, and make that decision per feature.

Running a summarization

Once available, the API is stream-friendly, which matters for perceived latency because even on-device generation isn't instant:

val request = SummarizationRequest.builder(longText).build()

summarizer.runInference(request) { result ->
    // streamed chunks; append to UI as they arrive
    appendToUi(result.summary)
}

On a Pixel 9 I've seen short summaries complete in a couple of seconds with the model already resident. The first run after a cold start is slower because AICore loads the model into memory; subsequent runs are quick. Stream the output so the user sees progress rather than a spinner.

What the model is good at (and not)

Gemini Nano is a small model. Calibrate expectations accordingly:

The task-specific ML Kit APIs exist precisely because a small model does far better on a narrow, well-defined job than on an open prompt. Lean into that. If your feature needs genuine reasoning or long context, that's a cloud job — possibly with RAG — and trying to force it onto Nano will disappoint users on the few devices that even run it.

Why bother, given the device limits

Three reasons make on-device worth the conditional complexity:

Privacy. The text or image never leaves the phone. For messaging, health, journaling, or anything sensitive, that's a feature you can put in marketing copy — and it sidesteps a pile of mobile privacy and GDPR obligations that come with shipping user content to a server.

Offline. Summarization on the subway, proofreading on a plane. No connectivity dependency.

Cost. Zero marginal cost per inference. At scale, moving even a fraction of requests on-device meaningfully dents a cloud LLM bill.

Production checklist

Things I'd verify before shipping a Gemini Nano feature:

• Model download UX. The feature model can be hundreds of megabytes and downloads via AICore. Handle metered-connection preferences and show honest progress; don't block the UI waiting for it.
• Fallback strategy. Explicitly decide cloud-fallback vs. hide per feature, and test the UNAVAILABLE path on a device that doesn't support Nano — which is most of them.
• Thermal and battery. Sustained inference warms the device. For one-shot actions this is fine; don't build a feature that runs Nano in a tight loop.
• Localization. Language support is limited and set at client creation. Verify your target languages are covered before promising the feature.
• Testing. On-device behavior varies by hardware. Test on real supported devices, not just the emulator.

On-device AI is a great addition to the Android toolbox — it's not a replacement for cloud models, it's a complementary tier for well-scoped, privacy-sensitive, offline-friendly tasks. Treat it as one, design for absence, and it becomes a differentiator rather than a support headache. It pairs naturally with the broader on-device AI for Android privacy story.

Resources

• Android Developers — AICore and on-device GenAI
• ML Kit — GenAI APIs
• Google AI Edge
• Android Developers — ML and AI overview
• Google — Gemini Nano announcement
• Android Developers — Google Play system updates