I'm going to show you something I do in my talks that always surprises executives: you can run a language model locally, on a Mac, without internet, without an account, without a license, without paying anything... and it takes literally three commands. And well that's local AI, and it's much simpler than most CIOs imagine.
The hardware: you probably already have everything
If you have a recent Mac with an Apple Silicon chip, an M1, M2, M3, or M4, well you already have what you need. Apple's unified memory is a free GPU, we don't think about it enough: the memory that serves your system also serves the model, there's no need to buy a separate graphics card. And then if you want to scale up, a server with a 16 to 24 GB VRAM GPU costs between 2,000 and 15,000 euros depending on what you choose... an RTX 4090, an L40S, or a used data center GPU.
And to test before investing a single euro, well even a Raspberry Pi 5 with 8 GB of RAM can run a small quantized model. I have one on my desk, it does text summarization, information extraction, classification... the same tasks that companies pay for at the API every day, for every employee, on every query. On the Pi, it costs nothing.
The software: Ollama, free
On the software side, you need Ollama, it's a runtime that serves open source models through a local API, and it's free. And then you need a model, and that's free too: Llama from Meta, Mistral, Qwen... all these models are open source, quantized, downloadable. You download them once, after that they're on your machine, nobody can cut off your access, nobody can change the price.
Step 1: install Ollama
And well there you go, Ollama is installed. No account to create, no license to accept, no cloud to configure... one command and it's done.
Step 2: download a model
The model is on your machine. It's 2 GB, it was downloaded once, and now it's there, it stays there, and it won't leave. That's a point I want to emphasize because it's fundamental: you no longer depend on a remote server, an internet connection, or a decision by an American company to maintain access.
Step 3: query the local API
The API runs on localhost, meaning on your machine. The data never leaves your network, the cost is zero, and you can cut the wifi, put the computer in airplane mode, and well it keeps running... I do it as a demo in my talks, and every time executives are surprised to see that it works without a network.
What you didn't have to do
And this is where I want us to take a moment to realize what didn't happen: you didn't have to ask a provider for permission, negotiate a contract, wait for a validation, send your data somewhere, sign a confidentiality clause, check the GDPR compliance of a transfer outside the EU... none of that. And well that's technical sovereignty, and it's accessible to any company that has a Mac or a server.
After the purchase: electricity
Once you have the hardware and the model, what it costs is electricity. No token, no per-user subscription, no surprise at the end of the month, no bill that triples in 30 days. The technical barrier is lower than people think... it's the mental barrier that's higher, and that's the one you have to bring down first.
If you want to see it run on your machine and you don't know where to start, well that's exactly the kind of thing I show in my workshops, and I'm at your disposal to organize one.