Jetson Edge AI Memory Budgeter Orin / Xavier

OOM RISK: SAFE Sufficient unified RAM available for OS, CUDA context, and KV cache.
ALLOCATED / TOTAL 6886 / 8192 MB
HEADROOM 1306 MB
UNIFIED MEMORY ALLOCATION STACK Est. Bandwidth: ~32.4 GB/s
OS Overhead
CUDA Context
Model Weights
KV Cache
Tensor Workspace
DETAILED BUDGET BREAKDOWN
Total Board RAM 8192 MB
OS & Driver Reserved 1500 MB
CUDA / TensorRT Context 1024 MB
Model Weights 3850 MB
KV Cache Memory 512 MB
Runtime Workspace 1000 MB
Total Allocated 6886 MB
Free Headroom 1306 MB
Board: Jetson Orin Nano 8GB
Status: SAFE
Allocated: 6886 MB
Headroom: 1306 MB
KV Cache: 512 MB

Trace the saved unified-memory budget and warning threshold

Read the explanation

The executable INT4 lookup uses point five five bytes per parameter, despite initial text showing point five. For seven billion parameters, division by two to the twentieth, a five percent allowance, and rounding give three thousand eight hundred fifty-five binary mebibytes. FP16 at two bytes gives fourteen thousand nineteen. Both bars use point zero three pixels per mebibyte. These are assigned tensor estimates, not verified checkpoint sizes or real device allocations. The cache multiplies two for keys and values, thirty-two layers, full hidden dimension four thousand ninety-six, two thousand forty-eight tokens, batch one, and two bytes per value. Dividing by two to the twentieth yields one thousand twenty-four binary mebibytes. Doubling context gives two thousand forty-eight. Both bars use point two pixels per mebibyte. This uses the full hidden dimension without a separate grouped-head reduction, so it is the saved architecture assumption, not a measured cache shape. The default sums fifteen hundred OS reserve, one thousand twenty-four CUDA context, three thousand eight hundred fifty-five weights, one thousand twenty-four cache and five hundred seventy-eight workspace. Total is seven thousand nine hundred eighty-one against eight thousand one hundred ninety-two, leaving two hundred eleven. Both bars use point zero five pixels per assigned binary memory unit. Positive headroom below twelve percent triggers WARN rather than SAFE. Preset specifications and the bandwidth-derived token estimate are assumptions, not current hardware or performance verification.

Super generates helpful tools and automates fact-checking across the internet proactively. If you enjoyed this tool, build your own with Super and share it with a friend.