Why “Classic RAG” Breaks on Android

On paper, retrieval-augmented generation is straightforward: embed the query, retrieve the top chunks, stuff them into a prompt, and generate an answer with citations. On Android, that “classic” flow runs into real constraints:

Latency budgets are tight. Users feel delays instantly, especially inside chat-like UIs.
Networks are unreliable. RAG becomes brittle when your retrieval depends on a perfect connection.
Privacy expectations are higher. Users assume mobile experiences are local-first, especially for enterprise or personal data.
Resources are limited. Battery, memory, and storage don’t tolerate “just cache everything.”
Cold start is unforgiving. If the first answer is slow or wrong, you lose trust quickly.

So the goal isn’t “RAG everywhere.” The goal is first to find a helpful answer quickly, then to upgrade the grounding when the cloud is available. That’s exactly what a two-tier system provides.