IlyasKaGpt – 100% Offline On-Device AI
A fully private, serverless on-device AI assistant running quantized LLMs & VLMs directly on mobile CPU/GPU/NPU — zero internet, zero cloud costs.
100%
Offline — zero internet
4-bit
GGUF quantized models
0$
Cloud / server cost
The Challenge
Running large language models on constrained mobile hardware with no cloud fallback, while keeping the UI responsive and supporting multimodal inputs like images, PDFs, and voice.
Our Solution
Used llama.cpp's GGUF 4-bit quantization via llama.rn to drastically reduce model memory footprint. Offloaded inference to CPU/GPU/NPU using platform-native acceleration layers, and built async streaming so the UI stays live during generation.
IlyasKaGpt is a 100% offline, serverless, and private on-device AI assistant built with React Native and llama.rn (llama.cpp). It runs 4-bit GGUF quantized models (Qwen2.5 LLM and Qwen2-VL Vision) natively on mobile CPU/GPU/NPU with real-time token streaming.
Features include offline image and camera inspection, local PDF parsing, voice chat (STT/TTS), and an in-app background model downloader from HuggingFace — all with zero internet dependency.