Make the memory fit before the model runs.

Compare weight footprints across formats against your cluster. Fit is necessary for serving, not proof of quality or runtime success.

Format runway

Load the sample and plan memory.

weight GB = parameters (trillions) x 1,000,000,000,000 x bytes / 1,000,000,000 | required GPUs = ceiling(weight GB / GPU memory)

How to read it

Quantization reduces weight footprint. A fit here only means the weights fit the stated capacity; activations, KV cache, runtime overhead, bandwidth, and accuracy remain separate questions.

Source boundary

The source supplies an illustrative hardware and parameter example. This planner does not benchmark formats, predict quality, or claim a model will serve successfully.

Weight footprints, reserves and ceiling device counts

Your browser does not support video.

Read the explanation

At the saved zero point eight trillion parameter assumption, two bytes per parameter gives sixteen hundred decimal gigabytes for BF16. One byte gives eight hundred gigabytes for FP8. Bars share three tenths of a pixel per gigabyte. Half a byte gives four hundred gigabytes for FP4. This is idealized weight storage arithmetic; quantization metadata, activations, key value cache, accuracy, bandwidth and serving success are outside this tool. Eight devices with one hundred eighty decimal gigabytes each provide fourteen hundred forty gigabytes before reserve. Reserving two hundred leaves twelve hundred forty for weights. Bars share three tenths of a pixel per gigabyte. At the saved zero point eight trillion parameters, BF16 still exceeds capacity while FP8 and FP4 fit the stated weights-only budget. Fit does not prove that sharding, software or an actual inference workload will work. Dividing sixteen hundred BF16 gigabytes by one hundred eighty gigabytes per device and taking the ceiling gives nine devices. FP8 eight hundred divided by one hundred eighty gives five devices. Bars share fifty pixels per device. These required device counts ignore the separate total reserve, so they are not a full runtime deployment requirement. The best-fit label selects the smallest fitting footprint, not the highest quality model format.