Two separate questions hide inside this one, and they have different answers.
Will a model load on it? Usually yes, if the box has the memory.
Will it be fast enough to enjoy? Usually no, because a NAS has no graphics card and generation speed is set by memory bandwidth.
The interesting part is the third question nobody asks, which is what it costs to find out. That one has a surprisingly good answer.
What will fit
A NAS is not allowed to spend all its memory on you. It has an actual job: serving files, holding a filesystem cache, running whatever containers you already put on it. Reserve something for that first.
Assuming 4 GiB left to the NAS itself, 4k of context, and a gibibyte of runtime overhead, at 4 bits per weight:
| NAS memory | Free for a model | What fits |
|---|---|---|
| 4 GiB | 0 GiB | nothing |
| 8 GiB | 4 GiB | up to 3B |
| 16 GiB | 12 GiB | up to 13B |
| 32 GiB | 28 GiB | up to 32B |
| 64 GiB | 60 GiB | up to 70B |
The small end of that table is more useful than it looks. A 3B model at 4 bits is 1.4 GiB of weights, and a 1B is 0.47 GiB. Those are not toys for classification, extraction and summarising, which is most of what a NAS would sensibly be asked to do.
One warning specific to this hardware. On a ZFS system, free memory is not free: the filesystem cache expands to fill it and gives it back under pressure, so a box that reports plenty available may be reporting cache. Cap the cache deliberately rather than discovering the interaction during a transfer, and put your own numbers through the model fit calculator against the memory you have actually set aside. Which system you are running changes how much of this you control, and the three common ones differ structurally here.
What it will not do is be fast
This is the honest part and I am not going to dress it up.
Generation speed is set by how quickly the machine can read the model’s weights out of memory for every token. A graphics card has memory bandwidth measured in hundreds of gigabytes per second. A NAS has system memory, often in a low-power processor with fewer channels, and that gap is the whole story.
A model that fits will still answer you. It will answer slowly, and the larger the model the worse it gets, because there are more weights to read per token. What “slowly” means in practice is a measurement on a NAS, and I have not made it. The benchmark database now has a Spark inference row, which is a different machine. What I would do before buying anything is put a candidate speed into the tokens per second simulator and watch text stream at that rate, because the difference between 3 and 30 tokens per second is something to feel rather than read about.
The practical shape of it: small models on a NAS are usable for things you are not sitting and watching. Large models on a NAS are a demonstration.
The reason to do it anyway
Here is the argument that actually decides this, and it is not about performance.
The NAS is already switched on. Its electricity is a cost you have already accepted, and it does not increase when the machine is idle rather than thinking. So the only new cost of running a model on it is the difference between idling and working, for the hours it works.
Take a NAS drawing 55 W at rest and 90 W while generating, at 30 cents per kWh:
| Inference per day | Annual electricity | New cost over doing nothing |
|---|---|---|
| none | $144.64 | $0.00 |
| 30 minutes | $146.56 | $1.92 |
| 1 hour | $148.47 | $3.84 |
| 2 hours | $152.31 | $7.67 |
| 4 hours | $159.98 | $15.34 |
Now the same work on a machine bought for it, which was off before and now is not, at 40 W idle and 300 W under load:
| Inference per day | Annual electricity |
|---|---|
| 30 minutes | $119.44 |
| 1 hour | $133.68 |
| 2 hours | $162.17 |
| 4 hours | $219.15 |
An hour a day costs $3.84 on the NAS and $133.68 on the new box. Not because the NAS is efficient, but because its idle hours were already paid for and the new box’s are not. That arithmetic is worked through generally in what a homelab really costs in electricity, and this is the clearest case of it on the site.
So the trade is speed against a running cost of almost nothing. For a model you interact with all day, buy the machine. For anything that runs on a schedule, or that you kick off and come back to, the NAS is very hard to argue with.
What a NAS is genuinely good at here
- Classification and tagging. Small model, short output, runs while you are asleep.
- Summarising documents you already store there. The data is on the box already, which is the other half of the argument.
- Embedding for search. Run once per document, then the expensive part is over.
- Anything asynchronous. A queue that drains overnight does not care about tokens per second.
- Finding out whether you like local AI at all, before spending anything. That is a real use and it is the one I would start with.
And what it is not good at: interactive chat with a large model, anything with a person waiting, and long documents, since filling a long context is compute before the first word appears and this is the machine least able to spare it.
What it costs you elsewhere
Three things worth knowing before you install anything.
Memory contention. The model’s memory is memory the NAS is not using for its job. On a busy array this is the change you will notice first, and it shows up as the file server getting slower rather than the model failing.
Heat and noise while it works. Sustained inference is sustained load, and a machine that was quiet at idle may not be. If the NAS shares a room with anything else, how those sources combine is less intuitive than it looks.
Drive wear, mildly. Models are gigabytes and they accumulate on the array you built for other things. That is capacity and standing watts, and the NAS capacity calculator will price both.
When to buy a separate machine instead
- Someone is waiting for the answer, routinely.
- You want a model above about 13B at a speed you would call interactive.
- The NAS is already working hard and its actual job is the one that matters.
- You have measured what you need and a NAS cannot reach it.
Before committing to that, work out whether owning the machine beats renting the same tokens at all, because for a lot of usage patterns the arithmetic does not favour buying: cloud GPU vs owning hardware has the numbers, and three of the nine cells in its table say never.
FAQ
Can a Synology or QNAP NAS run a local LLM?
If it has the memory and a processor that can run containers, generally yes, and slowly. Memory is the constraint on what loads: reserve about 4 GiB for the NAS itself and a model up to roughly 3B fits in what an 8 GiB box has left. Speed is the constraint on whether you enjoy it, and without a graphics card it will be modest.
How much RAM does a NAS need to run AI?
For a small model that is genuinely useful, 8 GiB total is the floor and 16 GiB is where it stops feeling tight. At 16 GiB you have around 12 GiB free, which holds a 13B model at 4 bits with short context. Above that the question becomes speed rather than fit, and more memory stops helping.
Will running a model slow down my file server?
It can, in two ways. The model’s memory is memory the filesystem cache no longer has, which on ZFS in particular is a real effect. And sustained inference is sustained processor load, competing with whatever the NAS does for a living. Both are worse on a small box and both are reasons to run inference on a schedule rather than on demand.
Is a GPU worth adding to a NAS?
If the chassis takes one and you have the slot and the power, it changes the speed answer completely, since generation is bandwidth-bound and a card has far more of it. Whether that is the best use of the money depends on what else you would put it toward, and a small dedicated machine is often a cleaner answer than turning a file server into a workstation.
Is it cheaper than paying an API?
The electricity is nearly free, since the NAS was already on, but that is not the whole comparison: your time and the slower answers are real costs the arithmetic does not show. For light use, an API is frequently cheaper overall even against a machine whose marginal cost is a few dollars a year. The break-even calculator will say so plainly when it applies to you.
What model should I start with on a NAS?
The smallest one that does your task, which is almost always smaller than you expect. Start at 3B at 4 bits, see whether the output is good enough, and only move up when it demonstrably is not. That direction is much less frustrating than starting large and working down through disappointment.
Sources
- Memory fit, context cost and the electricity figures on this page are computed by the same code that runs the model fit calculator and the power cost calculator.
- Memory sizes, wattages and tariffs here are illustrative inputs, not measurements of any product. Nothing on this page was measured on hardware. Take the wattages from a plug meter on your own NAS, at rest and under load, since the gap between those two readings is the entire marginal cost argument above.
- No generation speed appears on this page, because a NAS’s speed is a measurement and I have not made one. When runs are published to the benchmark database, this article gains real figures for the machines tested and its updated date moves.