LocalNodeOps

Synced from Hugging Face — 2026-09-14

How much VRAM does gemma-2-9b-it-GGUF need?

Exact on-disk weight sizes below are pulled directly from Hugging Face. Total VRAM figures also include an estimated KV cache and runtime overhead — see the breakdown per quantization, and the calculator below to adjust context length.

By quantization

gemma-2-9b-it-IQ2_M

3.20 GB exact, on disk

~4.07 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-IQ2_S

2.99 GB exact, on disk

~3.84 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-IQ2_XS

2.86 GB exact, on disk

~3.70 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-IQ3_M

4.19 GB exact, on disk

~5.16 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-IQ3_XS

3.86 GB exact, on disk

~4.80 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-IQ3_XXS

3.54 GB exact, on disk

~4.44 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-IQ4_XS

4.83 GB exact, on disk

~5.86 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q2_K

3.54 GB exact, on disk

~4.44 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q2_K_L

3.75 GB exact, on disk

~4.67 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q3_K_L-Q8

4.99 GB exact, on disk

~6.04 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q3_K_L

4.78 GB exact, on disk

~5.81 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q3_K_M

4.43 GB exact, on disk

~5.42 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q3_K_S

4.04 GB exact, on disk

~4.99 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q3_K_XL

4.99 GB exact, on disk

~6.04 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q4_K_L

5.57 GB exact, on disk

~6.68 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q4_K_M-fp16

6.37 GB exact, on disk

~7.56 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q4_K_M

5.37 GB exact, on disk

~6.46 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q4_K_S

5.10 GB exact, on disk

~6.16 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q5_K_L

6.40 GB exact, on disk

~7.59 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q5_K_M

6.19 GB exact, on disk

~7.36 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q5_K_S

6.04 GB exact, on disk

~7.19 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q6_K-Q8

7.27 GB exact, on disk

~8.55 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q6_K-f32

9.78 GB exact, on disk

~11.31 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q6_K

7.07 GB exact, on disk

~8.33 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q6_K_L

7.27 GB exact, on disk

~8.55 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q8_0-f16

9.95 GB exact, on disk

~11.49 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q8_0

9.15 GB exact, on disk

~10.62 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-Q8_0_L

9.95 GB exact, on disk

~11.49 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

gemma-2-9b-it-f32

34.43 GB exact, on disk

~38.42 GB total estimated VRAM at 4,096 tokens context (KV cache + overhead estimated, not exact)

VRAM calculator

Estimate total VRAM for a given model size, quantization, and context length. Estimates only — see the note at the bottom.

7B13B70B120B
2K4K8K16K32K64K128K
Model weights
—
KV cache
—
Runtime overhead (10%)
—
Total VRAM
—

Fits on:

    Live changes

    • No changes yet — adjust a control above.

    Generic estimate: weight size is bits-per-weight × parameter count; KV cache uses a reference architecture for the selected size bucket (7B/13B/70B: real published models; 120B: extrapolated, no real model exists at that exact size). Model-specific (HF-synced): weight size is the exact .gguf file size fetched from Hugging Face — no estimation. KV cache is still estimated: this schema doesn't carry layer count, head count, or head dimension, so the calculator parses an approximate parameter count from the model's title (e.g. "8B") and reuses the nearest size bucket's reference architecture for the KV math, same caveats as the generic mode. If no parameter count can be parsed from the title, it falls back to the 7B architecture and says so next to the quantization dropdown. Batch size is fixed at 1 in both modes. Treat all figures here as a starting estimate, not a guarantee. Full formula and known limitations: /methodology.

    FAQ

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-IQ2_M?

    The gemma-2-9b-it-IQ2_M quantization is 3.20 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.07 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-IQ2_S?

    The gemma-2-9b-it-IQ2_S quantization is 2.99 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 3.84 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-IQ2_XS?

    The gemma-2-9b-it-IQ2_XS quantization is 2.86 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 3.70 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-IQ3_M?

    The gemma-2-9b-it-IQ3_M quantization is 4.19 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.16 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-IQ3_XS?

    The gemma-2-9b-it-IQ3_XS quantization is 3.86 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.80 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-IQ3_XXS?

    The gemma-2-9b-it-IQ3_XXS quantization is 3.54 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.44 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-IQ4_XS?

    The gemma-2-9b-it-IQ4_XS quantization is 4.83 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.86 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q2_K?

    The gemma-2-9b-it-Q2_K quantization is 3.54 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.44 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q2_K_L?

    The gemma-2-9b-it-Q2_K_L quantization is 3.75 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.67 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q3_K_L-Q8?

    The gemma-2-9b-it-Q3_K_L-Q8 quantization is 4.99 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.04 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q3_K_L?

    The gemma-2-9b-it-Q3_K_L quantization is 4.78 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.81 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q3_K_M?

    The gemma-2-9b-it-Q3_K_M quantization is 4.43 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 5.42 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q3_K_S?

    The gemma-2-9b-it-Q3_K_S quantization is 4.04 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 4.99 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q3_K_XL?

    The gemma-2-9b-it-Q3_K_XL quantization is 4.99 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.04 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q4_K_L?

    The gemma-2-9b-it-Q4_K_L quantization is 5.57 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.68 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q4_K_M-fp16?

    The gemma-2-9b-it-Q4_K_M-fp16 quantization is 6.37 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 7.56 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q4_K_M?

    The gemma-2-9b-it-Q4_K_M quantization is 5.37 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.46 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q4_K_S?

    The gemma-2-9b-it-Q4_K_S quantization is 5.10 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 6.16 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q5_K_L?

    The gemma-2-9b-it-Q5_K_L quantization is 6.40 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 7.59 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q5_K_M?

    The gemma-2-9b-it-Q5_K_M quantization is 6.19 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 7.36 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q5_K_S?

    The gemma-2-9b-it-Q5_K_S quantization is 6.04 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 7.19 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q6_K-Q8?

    The gemma-2-9b-it-Q6_K-Q8 quantization is 7.27 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 8.55 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q6_K-f32?

    The gemma-2-9b-it-Q6_K-f32 quantization is 9.78 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 11.31 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q6_K?

    The gemma-2-9b-it-Q6_K quantization is 7.07 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 8.33 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q6_K_L?

    The gemma-2-9b-it-Q6_K_L quantization is 7.27 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 8.55 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q8_0-f16?

    The gemma-2-9b-it-Q8_0-f16 quantization is 9.95 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 11.49 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q8_0?

    The gemma-2-9b-it-Q8_0 quantization is 9.15 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 10.62 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-Q8_0_L?

    The gemma-2-9b-it-Q8_0_L quantization is 9.95 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 11.49 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.

    How much VRAM does gemma-2-9b-it-GGUF need at gemma-2-9b-it-f32?

    The gemma-2-9b-it-f32 quantization is 34.43 GB on disk (exact, synced from Hugging Face). At a 4,096-token context, using the 7B reference architecture for the KV cache estimate, total estimated VRAM is approximately 38.42 GB. This total includes a KV cache estimate and a 10% runtime overhead margin — only the weight size is exact.