Offline AI
A private AI assistant that runs without the internet
Project NOMAD is free, open-source software that turns a PC you already own into an offline knowledge server. It includes an optional AI assistant that runs entirely on that computer, so once it is set up you can unplug the internet and keep asking questions.
How offline AI works
Cloud assistants like ChatGPT run on someone else's servers, so every question you ask travels over the internet. A local AI model is a file on your own drive. Your computer loads that file and does the work itself, which means no connection, no account, and nothing leaving the machine.
Project NOMAD sets that up for you. It installs Ollama to run the models, lets you download open-source models such as Llama, Qwen, and Gemma while you are online, and gives you a chat window in the same dashboard as the rest of your library. If you have an NVIDIA or AMD graphics card, Project NOMAD detects it and sets up GPU acceleration.
Self-hosted AI usually means wiring several tools together and keeping them running. Here the install is two commands and a setup wizard.
What it's actually good for
Having a copy of Wikipedia is one thing. Having an assistant that can read through your stored references and pull out what matters for your situation is another. We call that applied reasoning: the AI works through your question using what's on the box, instead of handing you a list of search results.
You can also add your own files to the Knowledge Base, such as manuals, PDFs, and notes, and ask the assistant about them. Those files stay on your drive with everything else.
Local models are smaller than cloud models and they do get things wrong. Treat the assistant as a fast way into your library, and treat the library itself as the source of truth.
How fast is it on real hardware?
Speed is measured in tokens per second. A token is roughly three-quarters of a word, so 10 tokens per second already reads faster than most people do. These are measured results on an 8-billion-parameter model (llama3.1:8b). Smaller models run several times faster on the same hardware.
AMD Radeon 780M (integrated)
Minisforum UM890 Pro
12.3
AMD Radeon 890M (integrated)
Minisforum AI X1 Pro
15.8
NVIDIA RTX 3060 12 GB
Community builds
~65
NVIDIA RTX 5060 8 GB
Minisforum MS-02 Ultra
84.2
NVIDIA RTX 3090 24 GB
Community builds
~150
No dedicated graphics card? A refurbished mini PC with integrated graphics runs small 1 to 3 billion parameter models at 5 to 20 tokens per second, which is fine for basic questions. The hardware guide has full builds at three price points, and the NOMAD Benchmark leaderboard shows results from community builds.
The AI is optional
Project NOMAD is a knowledge library first. Wikipedia, offline maps, Project Gutenberg, medical references, and Khan Academy courses all work without the AI. Some people want an assistant and some don't, so it's a checkbox in the setup wizard.
A few things to know before you start. Installing Project NOMAD and downloading models needs an internet connection, so do it before you need it. It runs on x86 PCs with Ubuntu or Debian, not on a Raspberry Pi, and not natively on macOS.
Offline AI questions
Can you run AI without an internet connection?
Yes. A local AI model is a file on your own drive, and once it is downloaded your computer runs it without contacting anyone. Project NOMAD downloads the models you pick while you are online, then the AI assistant keeps working with the internet unplugged.
Is offline AI private?
Project NOMAD runs the AI entirely on your own hardware. There is no account, no API key, and nothing you type is sent to a company or a data center. Your questions and your documents stay on your machine.
Is it as good as ChatGPT?
No, and we would rather say so up front. The models that fit on home hardware are smaller than the ones cloud services run, so they know less and make more mistakes. Where a local model earns its place is working through your own offline library, where it can read what is stored and help you find the part that answers your question.
What hardware do I need for offline AI?
A regular x86 PC running Ubuntu or Debian Linux. Integrated graphics will run small models. An AMD Radeon 780M or 890M runs an 8-billion-parameter model at 12 to 16 tokens per second, and an NVIDIA GPU with 12 GB or more of memory runs it several times faster. Project NOMAD detects NVIDIA and AMD GPUs and sets up acceleration for you.
Do I have to use the AI?
No. The AI assistant is completely optional. Wikipedia, maps, books, medical references, and courses all work without it. Leave it out during setup and nothing else changes.
How much does Project NOMAD cost?
Nothing. Project NOMAD is free and open source under the Apache 2.0 license, with no subscription and no telemetry. You supply the computer.