Lexicon

How many bits of storage each of a model's parameters gets after quantization, which sets the file size.

Multiply a model's parameter count by its bits per weight and divide by eight to get bytes. It is the first term in every memory calculation, and the one people quote most loosely: a format named for four bits stores rather more than four, because it also has to store the per-block scales that let those four bits be decoded. The nominal figure is a floor, and the file on disk is the truth.

Work it out for your own hardware: /tools/model-fit

How many tokens of conversation a model can hold at once, counting both what you wrote and what it has answered.

Context is not free and it is not fixed cost. Every token in the window occupies KV cache, so memory grows linearly with it: a model that loads comfortably at 4k can run out of memory at 32k without a single thing else changing. A model advertised with a long context window will still only reach it if you have the memory to hold it, which is a different question from whether the weights fit.

Work it out for your own hardware: /tools/model-fit

A sound level measured on a scale weighted to approximate how human hearing responds, which is how fan and drive noise is normally quoted.

The A weighting rolls off low frequencies, because ears are less sensitive to them at the levels involved. That makes it a reasonable single number for broadband fan hiss and a poor one for a rack's low rumble or a tonal whine, both of which people find more annoying than their A-weighted rating suggests. Decibels are logarithmic, so they do not add: two machines at 38 and 41 dB(A) make about 43 together, not 79.

Work it out for your own hardware: /tools/noise-combiner

The fraction of a battery's rated capacity a device will actually draw before shutting off, which is never all of it.

Running a battery flat shortens its life sharply, so a UPS cuts off well before empty. That is why nominal capacity overstates what you get: multiply volts by amp-hours, then by the depth of discharge, then again by inverter efficiency, and the usable figure is a good deal smaller than the label suggests. It is one of two reasons a calculated runtime is an upper bound rather than a promise, the other being that a battery delivers less when it is discharged quickly.

Work it out for your own hardware: /tools/ups-sizing

Sound behaving as though nothing reflects it, which is the assumption behind every simple distance calculation and never quite true indoors.

In free field the level falls by six decibels each time the distance doubles, because the energy spreads over a growing sphere and none of it comes back. A real room reflects, so the level falls more slowly than that, and a small hard room holds noise up noticeably. Treat free-field arithmetic as the quietest the room could be rather than a prediction, and expect soft furnishings, a rug and an open door to move the answer in your favour.

Work it out for your own hardware: /tools/noise-combiner

The single-file model format used by llama.cpp and the tools built on it, holding the weights, the tokenizer and the metadata together.

GGUF replaced the older GGML formats and is what most quantized models are distributed as for local use. One file carries everything needed to run the model, which is why local runtimes can load a download with no further setup. A GGUF k-quant mixes precisions between tensors rather than storing every weight at one width, which is why its real size runs above the nominal bits-per-weight figure its name implies.

An attention design where several query heads share one set of keys and values, cutting KV cache size by that ratio.

Without it, every attention head keeps its own keys and values and the cache scales with the model's full width. With four query heads per key-value head, the cache costs a quarter as much per token. This is the single biggest reason a modern model of a given size holds far more context than an older one of the same width. Look for num_key_value_heads in a model's config.json: when it equals num_attention_heads, the model has no grouped-query attention.

Work it out for your own hardware: /tools/model-fit

Software whose primary job is running virtual machines, with storage and networking arranged around that rather than the other way round.

The word matters when choosing a platform, because it names an assumption rather than a feature. Proxmox is a hypervisor that can store things, so its unit of work is a virtual machine or a container. TrueNAS is a storage system that can run things, so its unit of work is a dataset. Both will do both jobs, and the one whose assumption matches what your machine is actually for is the one that will stop fighting you.

What a machine pulls from the wall while doing nothing, which for an always-on server is most of its yearly bill.

Idle is the number that matters for a home lab and the one specifications least often give. It runs for every hour of the year, where peak draw runs for the hours you actually use the machine, so ten watts saved at idle usually beats a hundred saved under load. Measure it at the wall with a plug meter rather than adding up component ratings: a power supply's rating is the maximum it can deliver, not what anything is drawing.

Work it out for your own hardware: /tools/power-cost

Running a trained model to get an answer out of it, as opposed to training, which is the process that produced the model.

Everything this site benchmarks is inference. It is far cheaper than training and it is what local hardware is realistically for: a machine that can run a large model perfectly well may be hopeless at training even a small one. The two also stress different things, inference being dominated by memory bandwidth and capacity rather than raw compute.

Further training that teaches a model to follow instructions rather than simply continue text, which is what makes it answer a question instead of extending it.

A base model is a completion engine: give it a question and it may write three more questions, because continuing text is exactly what it was trained to do. That reads as the model being broken when it is working correctly. Instruction-tuned variants, usually marked instruct or chat in the name, have been trained to treat your input as a request. For anything conversational this is the difference between usable and not, and it has nothing to do with parameter count or quantization.

The GGUF quantization family that stores weights in blocks with a scale per block, and keeps some tensors at higher precision than the name advertises.

This is why a file labelled four bits is never four bits. The per-block scales have to be stored alongside the weights, and the embedding and output layers are usually spared the full squeeze because they cost a lot of quality and save little size. The S, M and L suffixes say how much extra precision the mixture spends. Treat the nominal figure as a floor and the file size on the model card as the truth.

Work it out for your own hardware: /tools/model-fit

One thousand watts drawn for one hour, and the unit every electricity bill is priced in.

Watts are a rate and kilowatt hours are an amount, which is the distinction that makes power bills confusing. A 40 W machine left on all year uses about 350 kWh; a 300 W machine used four hours a day uses about 440. Multiply kilowatt hours by your unit rate to get money, and use the marginal rate rather than an average that includes standing charges, because the standing charge does not change when you add a server.

Work it out for your own hardware: /tools/power-cost

The keys and values a model has already computed for the conversation so far, kept in memory so it does not recompute them for every new token.

It is the memory term people forget, and the reason a model that loaded fine can run out of memory an hour into a long conversation. The cache costs a fixed number of bytes per token, set by the model's layer count, width and key-value head count, and it grows linearly with context: double the context and you double the cache. On a model with grouped-query attention it is far cheaper than the model's width alone would suggest.

Work it out for your own hardware: /tools/model-fit

How fast a machine can read data out of memory, which is what sets generation speed once a model fits at all.

Producing a token means reading the model's weights, so the speed of that read is the ceiling rather than raw compute. It is why capacity and speed are separate questions with separate answers: a machine can hold a model comfortably and still generate slowly, which is the usual story for unified memory and for anything without a graphics card. It is also why a smaller or harder-quantized model runs faster, since there is less to read per token.

Work it out for your own hardware: /tools/tokens-per-second

The learned numbers inside a model, counted in billions, and the figure in names like 8B or 70B.

Parameter count sets the floor on memory, but only alongside the precision each parameter is stored at: the same 8B model is roughly 16 GB at full precision and roughly 4 GB at four bits. It is a rough proxy for capability within a model family and a poor one across families, since a well-trained recent model of a given size routinely beats a larger older one.

Work it out for your own hardware: /tools/model-fit

Redundant data computed across an array's drives, letting the array rebuild a failed drive's contents from the survivors.

One parity drive survives one failure, two survive two, and each costs a whole drive of capacity. The risk parity does not cover is the rebuild window: while a replacement resilvers, a single-parity array has no redundancy left and is reading every remaining drive end to end, and those drives are the same age and model as the one that just died. That window grows with drive size, which is the argument for double parity on large disks.

Work it out for your own hardware: /tools/nas-capacity

The rule that a battery delivers less total energy the harder you draw from it, so its rated capacity only holds at a slow discharge.

It is why a UPS runtime calculated by dividing battery watt hours by load watts is always optimistic, and why the vendor's published runtime chart is authoritative over any calculator. The effect is pronounced in the sealed lead-acid batteries most UPS units use, and it gets worse as a battery ages. Treat a computed runtime as a ceiling, and halve it for a battery a few years old.

Work it out for your own hardware: /tools/ups-sizing

The ratio of real power in watts to apparent power in volt-amps, which is why a UPS carries two different ratings.

Watts do the work; volt-amps are what the wiring and the UPS actually have to carry. Where current and voltage are out of step, or the load draws current in a distorted way, apparent power exceeds real power and the ratio between them is the power factor. Computer supplies with active power factor correction sit near 0.95 to 0.99 under load. A UPS has to satisfy both its VA and its watt rating, and the watt rating is the one people overlook.

Work it out for your own hardware: /tools/ups-sizing

Storing a model's weights at reduced precision to shrink it, trading some output quality for a large saving in memory.

Going from sixteen bits per weight to four cuts a model's size by roughly four, which is usually the difference between a model fitting your hardware and not. The quality cost is real but smaller than the size saving, and it is not linear: the drop from eight bits to four is far gentler than from four to two. The practical question is usually whether a larger model quantized harder beats a smaller one quantized lightly, and that is not settled by memory arithmetic alone.

Work it out for your own hardware: /tools/model-fit

Combining several drives so the array keeps serving data when one of them fails.

RAID buys uptime, not safety. It does nothing about a deleted file, a ransomware run, a power supply that takes several drives with it, a filesystem bug, or the building burning down, because every one of those reaches all the drives at once. The rule it does not replace is three copies of anything you care about, on two kinds of media, with one of them off site, and a restore you have actually tested.

Work it out for your own hardware: /tools/nas-capacity

ZFS's parity layouts, numbered by how many drives of parity they carry and therefore how many failures they survive.

RAIDZ1, Z2 and Z3 carry one, two and three drives of parity. They differ from traditional RAID 5 and 6 in being integrated with the filesystem, which lets ZFS write data and parity as a single transaction and so avoid the write hole entirely. ZFS also checksums everything it stores, so a rebuild verifies what it reads rather than trusting it.

Work it out for your own hardware: /tools/nas-capacity

Rebuilding redundancy onto a replacement drive after a failure, which is the most dangerous period in an array's life.

ZFS calls it resilvering and only copies data that actually exists, where traditional RAID rebuilds every sector whether it holds anything or not. Either way the array is reading all its surviving drives hard, for hours or days, at exactly the moment it has least redundancy left. Rebuild time scales with drive size, which is why the case for double parity strengthens as disks grow.

Work it out for your own hardware: /tools/nas-capacity

A scheduled pass that reads everything stored, verifies it against its checksums, and repairs anything wrong from redundancy.

Scrubbing is how silent corruption gets caught before a rebuild needs the data. Without it, a bad block can sit unread for years and then be discovered at the worst possible moment, when it is the only copy left during a resilver. Monthly is a common cadence for home arrays. A scrub moves the whole array to active power for its duration, which is worth knowing if you are watching the meter.

Letting idle drives stop spinning to save power, at the cost of a delay and a start cycle when something next touches them.

It helps most on a large array that genuinely sits idle, and barely at all on one that is touched regularly, because anything reaching the array wakes it and the drives spin up again for nothing. Start cycles are not free either. Treat it as a setting to match against how the array is actually used rather than a default worth turning on everywhere, and remember that a scheduled scrub wakes everything for its duration.

Work it out for your own hardware: /tools/nas-capacity

The power a lab pulls continuously whether or not anything is being asked of it, which is most of what the annual bill pays for.

Idle draw describes one machine at rest; standing draw is the total that never switches off, drives and switch and UPS overhead included. It is the number worth optimising, because it is billed for every hour of the year while the working hours are a small fraction of them. One continuous watt is 8.766 kWh a year, so converting watts to money is a single multiplication once you know your unit rate.

Work it out for your own hardware: /tools/power-cost

A terabyte is a trillion bytes as drives are sold; a tebibyte is 2 to the power 40 as operating systems count, about 9 percent larger.

This is why a new array reports less capacity than the box promised, on top of whatever parity already took. Eight terabytes of usable capacity shows up as about 7.3 TiB before a single byte is written, with no one at fault: the drive maker counts in powers of ten and the operating system counts in powers of two. Filesystem metadata and reserved space then take a little more.

Work it out for your own hardware: /tools/nas-capacity

A processor's thermal design figure, describing what its cooling has to handle rather than what the machine will draw from the wall.

It gets used as a power input constantly and it is the wrong number for the job, in the same way a power supply's rating describes what it can deliver rather than what it takes. Both sit well above a real idle reading and well away from a real load reading, and an annual cost built on either is fiction. Two readings from a plug meter, one at rest and one doing the work you actually do, replace both.

Work it out for your own hardware: /tools/power-cost

How long you wait after pressing enter before any of the answer appears, which grows with the length of your prompt.

It is a separate wait from generation speed and the two trade differently. A short question with a long answer is dominated by tokens per second; a long document pasted in with a one-line answer is dominated by time to first token. Both belong in any honest benchmark, which is why the run files here record them separately.

Work it out for your own hardware: /tools/tokens-per-second

The chunk of text a model actually reads and writes, roughly three quarters of an English word.

Models do not work in words or characters but in tokens from a learned vocabulary, so a common word is usually one token and a rare one splits into several. The ratio holds reasonably for English prose and gets much worse for code, for numbers, and for languages the tokenizer was not mostly trained on, where a single word can cost several tokens. Everything downstream is counted in them: speed, context length, memory, and API pricing.

Work it out for your own hardware: /tools/tokens-per-second

How fast a model produces its answer once it has started, and the headline number in most local inference benchmarks.

Judge it against your own reading speed rather than a leaderboard. Average adult reading works out to roughly five tokens per second, so anything below that means waiting for words you have already caught up with. A quoted rate without its conditions attached cannot be compared to anything: context length, quantization, batch size and thermal state all move it, usually downward from whatever was quoted.

Work it out for your own hardware: /tools/tokens-per-second

A single pool of memory shared by CPU and GPU, so a model's size is limited by total system memory rather than by a separate graphics card.

It is what lets some compact machines run models far larger than a discrete GPU of similar price could hold, since there is no fixed VRAM ceiling to hit. The trade is bandwidth: unified memory is usually slower than the dedicated memory on a high-end graphics card, so the same model may fit comfortably and still generate more slowly. Capacity and speed are separate questions and this is where they most visibly diverge.

A battery between the mains and your equipment, whose job in a home lab is usually to allow a clean shutdown rather than to keep working through an outage.

The expensive mistake is buying runtime. For most home labs a few minutes is the whole requirement, because the value is in the machines noticing the outage and shutting down cleanly before the battery gives out. Spend on a unit that can signal that shutdown, and then test it by pulling the plug on a day you choose rather than a day the weather chooses.

Work it out for your own hardware: /tools/ups-sizing

Voltage times current, the apparent power a supply has to carry, which is equal to or larger than the real power in watts.

UPS units are rated in both VA and watts and you have to satisfy both, which is why a unit sold as 1500 VA is often only 900 W. The watt figure is usually the binding constraint for a lab full of computers, and it is the one buyers skip. Divide your measured load in watts by the power factor to get the VA the unit needs to carry.

Work it out for your own hardware: /tools/ups-sizing

The memory on a graphics card, which sets the hard ceiling on what a model can load onto that GPU.

Capacity is the constraint people hit first and bandwidth is the one that sets speed afterwards. A card sold as 24 GB holds 24 GiB, which is what the driver reports, so the two numbers agree despite the units differing elsewhere in computing. A model that does not fit can often still be run by splitting it between GPU and system memory, at a cost in speed that is a measurement rather than a formula.

Work it out for your own hardware: /tools/model-fit

Starting a machine over the network, which is what makes switching a server off between uses practical rather than annoying.

It matters here because turning things off beats every other lever on a power bill, and the usual objection is convenience rather than cost. A machine that can be woken remotely removes most of that objection: an unused server at zero watts costs nothing at all. Whether it works depends on the network card, the firmware and what the operating system does on shutdown, so it is worth testing before you rely on it.

Work it out for your own hardware: /tools/power-cost

The window in which a parity array loses power partway through a write, leaving data and parity inconsistent with no record of which is wrong.

It happens because traditional RAID updates data and parity as separate operations, so an interruption between them leaves the stripe self-contradictory. The array cannot tell which half is stale, and a later rebuild may reconstruct from the wrong one. Hardware controllers address it with battery-backed write cache; ZFS avoids it structurally by writing data and parity in one transaction, which is a large part of why RAIDZ exists.

A filesystem that manages the array itself, checksumming everything it stores and writing data and parity as a single transaction.

The two properties that matter in practice are both about failures rather than normal running. Because a write is one transaction there is no window in which data and parity can disagree, so the write hole does not exist. And because everything carries a checksum, a rebuild verifies what it reads instead of trusting it, and a scrub can find corruption long before anything needs the data. The cost is a less flexible array: drives in a group want to match, and the cache will expand to fill whatever memory you leave it.