What is Local AI?

Local AI refers to running artificial intelligence models directly on your own device rather than using cloud-based services. This approach offers unique advantages for privacy, control, and customization.

• Nothing leaves your computer
• You can build customized workflows (for example calling AI from Python scripts)
• RAG (Retrieval Augmented Generation) using your local documents

Key Concepts

Cloud AI vs. Local AI

Cloud AI: (ChatGPT, Claude etc…)

• Models run on remote servers
• Requires internet connection
• Data sent to external services (some countries like the USA allow authorities to ask access to the logs)
• Generally more powerful models
• Pay-per-use or subscription pricing

Local AI:

• Models run on your device (CPU/GPU)
• Works offline
• Data stays on your machine
• Limited by hardware capabilities (large model require high-spec computers)
• One-time setup cost

Apple computers now feature a chip (Apple Silicon) with a relatively powerful GPU, that can run reasonably large models if there is enough memory available.

---