A GGUF repo lists a dozen quants. The file sizes already pick one.

Every GGUF repo on HuggingFace lists the same wall of options: Q2_K, Q4_K_M, Q5_K_M, Q6_K, Q8_0, on down the line. The model card explains what each one is, a bits-per-weight and quality tradeoff, and says nothing about what any of them means for your machine. So you guess. You pick Q4 because Q4 feels safe, download several gigabytes, load it, and either it OOMs outright or, worse, it loads and quietly spills to disk and every token takes a second. Delete, guess smaller, repeat. The bandwidth is gone and you still don't know if you left quality on the table.

Snug skips the loop. One command probes your hardware, reads the repo's quant list, and tells you the largest build that actually fits, then offers to download it:

$ snug recommend TheBloke/Llama-2-7B-Chat-GGUF
Probing hardware...
Recommendation: Q5_K_M (5.1 GB)
  fits entirely within budget with 3.4 GB headroom for context + KV cache
Download this build? [Y/n]

It answers before it downloads anything

The trick is that every number the decision needs already exists without pulling a single weight. Your VRAM and RAM are measurable in well under a second. The quant file sizes are published in the HuggingFace API right next to the file names. Snug fetches just that listing, sizes and names, and does arithmetic.

For each quant it asks a three-way question: does this file fit entirely inside the accelerator budget with room left for context and KV cache, does it fit only with part of the model offloaded to CPU, or does it force a swap? Fits-fully beats partial-offload beats swap, and within the winning tier it takes the largest file, because a bigger quant is a better quant right up until it stops fitting. The "headroom" in that output line is the part people forget: a model that exactly fills VRAM has no space for the context window's KV cache and will fall over the moment you feed it a long prompt. By default Snug reserves a flat slice for that. Pass --context 8192 and it goes further, resolving the model's architecture from its own config.json (or its tagged base model's) and computing the exact KV-cache bytes that context length needs.

That scoring logic lives in its own crate, auto-quantize-core, with no HTTP and no OS calls in it at all. It takes a HardwareProfile and a list of quant options and returns a recommendation, which means the one part with real logic worth trusting is trivial to unit-test across the fits/offload/swap cases without a network or a specific GPU in the loop.

Probing hardware without a vendor SDK

The other half is reading the machine, and I wanted that to work on a fresh install with nothing bundled. So Snug leans on signals the OS already exposes: /proc/meminfo and nvidia-smi output on Linux, sysctl and unified-memory queries on macOS, GlobalMemoryStatusEx and DXGI adapter enumeration on Windows. No CUDA, ROCm, or Metal toolkit to install, and the whole probe finishes in under a second on every platform. snug probe just prints what it sees.

The honest gap: effective memory bandwidth isn't probed yet, so today's recommendation is a budget-fit answer, not a throughput one. And when a platform won't give up its VRAM or bandwidth, Snug says unknown and falls back to a more conservative pick rather than inventing a number. It's a static binary with a distinct exit code per failure, so --json and $? drop it straight into a setup script.

Try it

cargo install --git https://github.com/ctkrug/snug, then snug recommend <any-gguf-repo> and read the one line it prints. Run snug probe first if you just want to see what it thinks your machine has. Say Y and it downloads the file, resuming from where it left off if the connection drops.