To run AI on your own PC for free, install Ollama on Windows, then raise the context length from its 4K default to at least 64K in Settings before you download anything. Pull a model sized for your graphics card with a command like ollama pull qwen3.5:4b, and chat with it in the Ollama app. Everything runs locally — no account, no subscription, and nothing leaves your machine.
Applies to: Windows 10 (22H2) and Windows 11 (23H2, 24H2, 25H2) | Last updated: July 31, 2026
Key Takeaways
- Ollama installs in about a minute and does not need admin rights — it goes into your local app data folder, so it installs for your user only
- The context length defaults to 4K, and that single setting is why most people think local AI is broken — raise it to 64K in Settings before you download a model
- Your VRAM decides which models you can run, not your system RAM — check it in Task Manager under Performance > GPU > Dedicated GPU memory
- Match the model’s download size to your VRAM — roughly 4B models on an 8 GB card, 9B on a 12 GB card, and larger models only on 16 GB or 24 GB cards
- Tokens per second is not the whole story — a thinking model can be the fastest per token and still take twice as long to finish an answer
Quick Steps:
- Download and install Ollama from ollama.com/download/windows
- Open Settings in the Ollama app and drag the context length slider to 64K
- In the same Settings screen, click Browse and move your model folder off the C drive
- Check your VRAM in Task Manager > Performance > GPU > Dedicated GPU memory
- Download a model that fits:
ollama pull qwen3.5:4b - Talk to it with
ollama run qwen3.5:4b, or just pick it in the Ollama app - Optionally enable Expose Ollama to the network to use it from other devices at home
How to Install Ollama on Windows
Ollama is the app that actually runs AI models on your machine, it is completely free, and it takes about a minute to set up. Head to the Ollama downloads page and click Download for Windows. You can also use the PowerShell command shown on their homepage if you prefer, but the installer is the simpler route. Ollama’s Windows documentation has the finer detail if you want it.
The installer is a big one — around 1.3 GB at the time of writing — so give it a few minutes depending on your connection. Once it has downloaded, run the setup file. There is no wizard asking you twenty questions here. You click Install, and it puts everything in your local app data folder, which means it installs for your user account only and does not need admin rights.
When the install finishes, Ollama launches on its own and drops you straight into a chat window. You could start typing right away — but do not. There is one setting to change first, and getting it wrong is the single most common reason people give up on local AI.
Note: The chat window is Ollama’s newer desktop app. If you have used Ollama before and only remember the terminal, the app is a recent addition — the commands still work exactly as they always did.
Fix the Context Length Before You Download a Single Model
Context length is the model’s working memory — how much of your conversation it can actually see at one time. On most graphics cards, Ollama sets this to just 4K out of the box, which is about 4,000 tokens. That fills up remarkably quickly.
The moment you paste in a large document or your chat runs a bit long, the model can no longer see the beginning of the conversation and starts forgetting what you told it. This is exactly why so many people try local AI, decide the model is broken or stupid, and give up — when the model was never the problem. Ollama’s own documentation states that coding tools, agents and web search want at least 64K of context.
To fix it, open the Settings tab on the left of the Ollama app and find the context length slider. Set it to 64K. It saves automatically — you will see a small confirmation at the bottom of the window, and there is no save button to hunt for.
You can push this to 128K or even 256K if you work with really big documents, but the higher you set it, the more memory it uses. 64K is a good middle ground for most people and it is where I keep mine.
If the Context Length Slider Is Greyed Out
If the slider will not move, you can set the context length with an environment variable instead. First, fully close Ollama — right-click the Ollama icon in your system tray and click Quit Ollama. Closing the window is not enough; it keeps running in the tray.
Then right-click your Start button, open Windows Terminal (Admin) or PowerShell as Admin, and run:
setx OLLAMA_CONTEXT_LENGTH 64000You will get a “SUCCESS: Specified value was saved” message, and Ollama picks the new value up the next time it starts.
There are two things I ran into here that are worth knowing in advance, so they do not throw you:
- The slider can still show the old value after you set the variable. The app and the environment variable do not talk to each other, so do not let the mismatch confuse you — the variable is what takes effect.
- The error
listen tcp ... only one usage of each socket addresssimply means Ollama is already running in the background. Nothing is broken. Quit the one in the system tray and the error goes away.
Move Your Ollama Models Off the C Drive
By default, Ollama saves downloaded models inside your user profile folder on the C drive. Individual models range from a couple of gigabytes to tens of gigabytes each, and they add up faster than you would expect — so it is worth changing this before you start downloading rather than after.
In the same Settings screen, you will see the model storage location. Click Browse, pick another drive, and create a folder for them. I put mine on my D drive in a folder called Ollama models. Like the context length, it saves automatically with no save button.
Beyond the space saving, there is a second reason I do this: models stored on a separate drive survive a Windows reinstall. When I eventually format my C drive, my model library is still sitting there untouched. Ollama’s FAQ covers the storage location alongside the context and network settings if you want to dig further.
Tip: If your C drive is already tight on space, it is worth reading my guide on how to clear space in Windows 10 and 11 before you start pulling multi-gigabyte models.
How to Download and Run Your First AI Model
To download a model, right-click your Start button, open Windows Terminal or PowerShell, and use ollama pull followed by the model name. I use Qwen 3.5 in its 4 billion parameter version:
ollama pull qwen3.5:4bAlways type the model name with the size tag at the end, exactly like the example above. If you type just the plain name, Ollama grabs that model’s default version, and that is usually a much bigger download than you were expecting.
The download starts by pulling the manifest and then fetches the model itself. This particular one is about 3.4 GB, which took roughly eight to ten minutes on my connection. When it is done, it verifies the digest and gives you a success message.
To actually talk to it, swap pull for run:
ollama run qwen3.5:4bIt loads the model and prompts you to send a message. I asked it “what is DISM” as a nice Windows question. It paused for a moment because this particular model thinks before it answers — you can watch its reasoning appear on screen — and then gave me a correct explanation. That is a real AI model running entirely on my own PC, answering my question with no internet involved.
To leave the chat, type /bye and press Enter.
You do not have to live in the terminal, though. The Ollama app itself is a chat window, so now that the model is downloaded you can simply select it in the app and chat away exactly like you would with any of the online services.
The Other Ollama Commands Worth Knowing
These four cover almost everything you will need day to day. The full CLI reference has the rest.
ollama rm qwen3.5:4b— deletes a model and gives you the disk space backollama pull qwen3.5:4b— run it again on a model you already have to update it; it only downloads what changedollama stop qwen3.5:4b— unloads a model from memory immediatelyollama run qwen3.5:4b --verbose— shows speed statistics after every answer
One behaviour that surprises people: a model stays loaded in memory for about five minutes after you use it. That is deliberate, so your next question answers faster. If you want it out of memory right away, ollama stop does exactly that.
Which AI Model Can Your PC Actually Run?
This is the question I had when I started running AI locally, and the honest answer is that it depends on your hardware. When you browse models on Ollama’s website you will see numbers like 4B, 9B or 30B. The B stands for billion, and it is how many parameters the model has — think of parameters as the model’s brain cells. More of them usually means smarter answers, but also a bigger model that needs more memory.
Here is the part almost nobody explains: what matters most for speed is not your normal system RAM, it is the memory on your graphics card, your VRAM. To check how much you have, right-click your Start button, open Task Manager, go to Performance and click your GPU. The figure labelled dedicated GPU memory is your VRAM. Mine reads 12 GB because I am on an RTX 3060, a fairly normal mid-range card.
The rule is simple. Look at the model’s download size on the Ollama website — it is listed next to every version. If that size fits comfortably inside your VRAM with a couple of gigabytes of breathing room for the context you set earlier, it will run fast. If the model is bigger than your VRAM, Ollama splits it between your graphics card and your normal RAM. It still works, it just gets a lot slower.
Three Models Tested on a 12 GB RTX 3060
I gave all three the same question — “what is DISM” — using --verbose to get the speed statistics. The 30B coder model is a 19 GB download that does not fit in 12 GB of VRAM at all, and I ran it anyway to show you exactly what happens.
| Model | Download size | Fits in 12 GB VRAM? | Speed | Time to full answer |
|---|---|---|---|---|
| 4B (thinking) | 3.4 GB | Yes, easily | ~73 tokens/sec | 34 seconds |
| 9B | 6.6 GB | Yes, comfortably | ~47 tokens/sec | 17 seconds |
| 30B coder | 19 GB | No | ~22 tokens/sec | 17 seconds |
Two results here are worth explaining, because neither is obvious.
First, the 30B model managed about 22 tokens per second despite not fitting in VRAM, which is far better than its size suggests. That is because it only uses roughly 3 billion of its 30 billion parameters at a time, so even split between GPU and system RAM there is much less work per answer than the number implies.
Second, the 4B model was the fastest per token and still took the longest to produce a finished answer — 34 seconds against 17 for the other two. It is a thinking model, so before it answers it writes out a whole reasoning process for itself. For my simple question it wrote nearly 2,500 tokens to get there, where the coder model answered the same thing in about 240. Tokens per second is not the whole story: a thinking model gives you more careful answers, but you wait for them.
My Recommendations by Graphics Card
- 8 GB card — stay with the small models, around 4B
- 12 GB card — the 9B range is the sweet spot
- 16 GB or 24 GB card — the really big models start making sense here
- No graphics card at all — Ollama still runs on just your processor. It is a lot slower, but a small model remains usable
AMD cards work too — this is not an Nvidia-only thing.
For a more detailed breakdown by exact hardware tier, I keep a separate always-updated page: what local AI model can your PC run? You enter your RAM and VRAM and it tells you which models are genuinely worth running on your machine.
Tip: If a model ever feels frozen, or your whole PC starts crawling the moment you send a message, that is almost always a model that is too big for your hardware. Nothing is broken — switch to a smaller model and it will come right. If you are deliberately running a model that spills into system RAM, clearing out background bloat gives it more room to work with, which is exactly what I built Winhance to do.
How to Use Your Local AI From Other Devices on Your Network
Once Ollama is set up on one PC, every other device in your house can use those same models. This is the part most people do not realise is possible, and it turns one machine into a private AI server for your whole home.
In the Ollama Settings, enable Expose Ollama to the network. That allows other devices and services on your network to reach it. Two things then need to be true for it to actually work.
Your network must be set to Private. Right-click the Start button, go to Network Connections in Settings, open your Ethernet or Wi-Fi adapter, and confirm the network profile is Private and not Public.
The Windows Firewall must allow the traffic. Even with the setting enabled, the firewall can still block other devices from reaching your PC. Run this in an admin PowerShell window to allow Ollama on your private network only:
New-NetFirewallRule -DisplayName "Ollama" -Direction Inbound -Protocol TCP -LocalPort 11434 -Action Allow -Profile PrivateImportant: This is for your home network only. Do not do this on public Wi-Fi at a coffee shop, an airport or a hotel. The
-Profile Privatepart of the rule is what keeps it limited to networks you have marked as private.
Next, run ipconfig in that same window and look at whichever adapter you are connected through. Note down the IPv4 address — that is your PC’s address on your network.
Mine is a static address I assigned on my router. If you plan to use this setup long term, I would suggest doing the same so the address does not change on you. The steps differ from router to router, so look up how to set a static IP on your specific model.
Confirming It Works
On the PC running Ollama, open a browser and go to http://localhost:11434. You should see the message Ollama is running, which is Ollama’s way of telling you the server is up.
Now do the same from another machine on your network, replacing localhost with the IP address you noted — for example http://192.168.1.50:11434. If you get the same message, anything on your home network can now use your models: your laptop, another PC, a phone, whatever you have.
Here is where it gets genuinely interesting. That address does not just work in a browser. Ollama exposes an OpenAI-compatible API, which means any app that knows how to talk to ChatGPT’s API can be pointed at your Ollama instead — completely free and completely private. You normally just paste your address with /v1 on the end into the app’s custom endpoint field.
That is exactly how I gave a local AI agent full control of a Windows 11 virtual machine. If you want to see what these models can really do once they are wired into a proper agent, that test is worth a watch:
You can read the written version in I gave a local AI full control of Windows 11, and if you would rather set up the agent yourself, my guide on running a free local AI agent on Windows with Ollama and OpenCode walks through the whole thing.
Frequently Asked Questions
Is Ollama actually free?
Yes. Ollama is free to download and use, and the models in its library are free to download as well. There is no account to create and no subscription. The only cost is disk space for the models and the electricity your PC uses while a model is running.
Does my data leave my PC when I use Ollama?
No. Once a model is downloaded, it runs entirely on your own hardware and your conversations never leave your machine. You only need an internet connection to download Ollama itself and to pull models. After that you can disconnect completely and it still works.
Can I run local AI without a graphics card?
Yes. Ollama falls back to running on your processor when there is no suitable graphics card available. It is considerably slower than running on a GPU, but a small model in the 4B range is still usable for questions and drafting. AMD graphics cards are supported as well, so this is not limited to Nvidia hardware.
Why does my local AI model keep forgetting what I told it?
That is almost always the context length sitting at its 4K default rather than a fault with the model. Raise the context length slider to 64K in the Ollama Settings, or set OLLAMA_CONTEXT_LENGTH to 64000 with the setx command if the slider is greyed out, then restart Ollama.
How do I completely uninstall a model I no longer want?
Run ollama rm followed by the model name, for example ollama rm qwen3.5:4b. This deletes the model files and returns the disk space immediately. You can always pull the model again later if you change your mind.
Is it safe to expose Ollama to my network?
It is fine on a home network you control, which is why the firewall rule above is limited to the Private profile. Do not enable it on public Wi-Fi, because anyone on that network could then send requests to your machine. If you no longer need it, turn off Expose Ollama to the network in Settings and the access closes again.
