I earn affiliate commission on hardware, so take the conclusion of this article in that light: for a lot of people, buying a machine to run models locally will never be cheaper than paying an API, and the gap does not close over time. The arithmetic below says when it does close, and when it does not.
Three numbers decide it. What the hardware costs, what the electricity costs to run it at your duty cycle, and what the same tokens would cost from a provider. Only the third is genuinely uncertain, which is why nothing here quotes an API price: put in whatever the provider you would actually use charges today, and blend the input and output rates in the proportion your work uses them.
Where the money actually goes at low volume
Start with the finding that surprises people. A machine drawing 40 W at idle, at 30 cents per kWh, costs $105.19 a year to leave switched on. That is $8.77 a month before it does anything at all.
Now generate 50,000 tokens a day on it, which at 20 tokens per second is about 42 minutes of work. Total electricity: $10.41 a month. Of that, $8.77 is idle.
At light usage you are not paying to run models. You are paying to keep a computer on. Every argument about efficiency, quantisation and throughput is rearranging the $1.64 and ignoring the $8.77. The power cost calculator makes that split visible for your own numbers, and it is usually the more useful lever.
That finding is not specific to AI hardware. It is how a homelab power bill works generally: standing draw dominates, and what the equipment does while you are watching it barely registers.
Three usage levels
Same machine throughout: $2,000, 40 W idle, 300 W under load, 30 cents per kWh. The three columns are what your provider might charge per million tokens. Break-even is how long until the electricity saving has repaid the purchase.
| Your usage | At $0.50/M | At $3/M | At $15/M |
|---|---|---|---|
| 50k tokens/day, 20 tok/s | never | never | 161 months |
| 500k tokens/day, 20 tok/s | never | 98 months | 10 months |
| 2M tokens/day, 60 tok/s | never | 13 months | 2 months |
Read the top-left corner first, because it is the one nobody writes about. Light use against a cheap model does not pay back in a hundred years. The monthly electricity alone, $10.41, is more than the $4.57 of API calls it replaces. There is no crossover to wait for.
Now read the bottom-right. Heavy use against an expensive model pays back in two months and then saves money every month after. Over three years that is $3,107 of local cost against $32,873 of API calls.
Both of those are the same calculation. The answer moved by two orders of magnitude because the inputs did.
The line that decides it
Local cost is nearly flat: you pay for the machine once and then pay electricity whether you use it or not. Cloud cost is nearly linear: nothing when idle, and proportional to what you generate.
So the crossover is set by one thing. Below some volume, the fixed cost of owning wins the race to the bottom and never gets caught. Above it, the linear cost of renting overtakes and keeps climbing.
That is why “is local cheaper” has no general answer, and why anyone giving you one without asking your volume is selling something. Put your numbers into the break-even calculator and watch which side of the line you land on. It says “never” in as many words when the answer is never.
What the local column leaves out
Everything below is real, unmodelled, and pushes break-even further away:
- Your time. Setup, updates, a runtime that changes its flags, a model that needs re-downloading. Price it at whatever your hour is worth and it dwarfs the electricity.
- Storage. Models are gigabytes each and they accumulate. That is drives, and drives are more power and more money.
- Failure. A power supply, a fan, a drive. Consumer hardware run hard for three years is not free of maintenance.
- The rest of the lab. Networking, a UPS, cooling.
One thing cuts the other way and is also unmodelled: a machine sold after three years recovers part of its purchase price. The tables above count residual value as zero, which favours the cloud column.
What the cloud column leaves out
- One blended price is a simplification. Input and output tokens are priced differently, often several times apart. Caching and batch discounts move the effective rate again.
- Prices have moved, mostly downward. A break-even three years out is a statement about a market that will not sit still, and it is a statement in the cloud’s favour.
- Rate limits and availability are not a cost until the day they are.
When to buy anyway
If cost is genuinely the deciding factor and the calculator says never, then never is the answer. But cost is frequently not the deciding factor, and none of these appear in any of the arithmetic above:
- Data that cannot leave the building, for policy or law
- Work that has to continue without a network
- Latency a round trip cannot meet
- A model nobody offers as a service, or one you have fine-tuned
- Wanting to, which is a perfectly good reason to buy a computer
What the arithmetic gives you is an honest price for those reasons, rather than a story about saving money that does not survive a spreadsheet.
If you do land on buying, work out how much memory the models you want actually need before choosing a card. A machine that cannot hold the model is not cheap at any price, and it is the mistake that costs the most to undo.
FAQ
Is running AI locally cheaper than using an API?
At high volume, usually yes. At low volume, usually no, and often never: a machine idling at 40 W costs about $8.77 a month whether you use it or not, and light usage against a cheap model does not generate enough API spend to repay that, let alone the hardware. The crossover depends on how many tokens you actually produce in a day, which is the number to measure before deciding.
How many tokens a day do I need to justify buying hardware?
There is no single figure, because it depends on what your provider charges. The pattern holds though: at a low price per million, even heavy local use struggles to beat renting; at a high price per million, moderate use pays back inside a year. Work out your own crossover with the break-even calculator rather than trusting a number from an article, including this one.
Does a faster machine change the break-even point?
Yes, and in a direction people find counterintuitive. A faster machine finishes the same work sooner and spends more of the day idling, so it uses less electricity for identical output. In the table above, moving from 20 to 60 tokens per second at 2M tokens a day cuts the duty cycle from an impossible 28 hours to a comfortable 9, and the monthly electricity barely moves.
What about buying a used GPU to lower the hardware cost?
It moves the one number the calculator is most sensitive to, so it is worth modelling rather than guessing. Halving the purchase price roughly halves the break-even period. Weigh that against a shorter remaining life and no warranty, and remember that a cheaper card with less memory may not fit the model you wanted, which the model fit calculator will tell you before you buy.
Should I count my electricity at the average rate or the marginal rate?
The marginal rate, meaning the unit price per kWh, not an average that includes standing charges. The standing charge does not change when you add a server, so including it overstates what the machine costs you. If you are on a time-of-use tariff and can schedule heavy jobs into the cheap window, use that rate, and be aware the annual figure then depends on the scheduling holding.
Sources
- The electricity arithmetic is the definition of the kilowatt hour and is shared with the power cost calculator. Tariff and watt figures are yours to supply: measure at the wall with a plug meter, because a power supply rating is a maximum it can deliver rather than a draw.
- API pricing is deliberately not quoted here. Take the current rate from the provider you would actually use.
- Every figure in this article was computed by the same code that runs the break-even calculator. None of it was measured on any hardware. When runs are published to the benchmark database, the tokens-per-second inputs gain measured values and this article’s updated date moves.