On-device AI eliminates latency and privacy concerns of cloud inference. Key frameworks: 1) Apple CoreML: optimized for Apple Neural Engine (ANE), supports quantized models, excellent for iOS/macOS deployment. Core ML Tools converts PyTorch/TensorFlow models. 2) MLC-LLM (Machine Learning Compilation): runs full LLMs (Llama, Mistral) on iPhone, Android, WebGPU via TVM compilation. Achieves 20-30 tokens/sec on iPhone 15 Pro for 3B models. 3) ONNX Runtime: cross-platform, supports DirectML (Windows), CoreML (Apple), NNAPI (Android). 4) LiteRT (formerly TensorFlow Lite): embedded and microcontroller friendly. Model optimization for edge: Quantization reduces 7B model from 14GB to 4GB (INT4). Knowledge distillation: train small student model to mimic large teacher. Structured pruning: remove low-importance weights. Deployment considerations: iPhone 15 Pro has 8GB RAM supporting 4B INT4 models. Android varies significantly. Apple Silicon Mac supports 70B models via Metal GPU. Privacy advantage: user data never leaves device. Use cases: offline translation, personal health monitoring, on-device document processing, real-time image analysis.