Qwantize

Optimal quantization methods for block-scaled formats.

Installation

pip install qwantize

Requires PyTorch (>=2.0) and Triton (>=3.0).

Repository

• GitHub: github.com/ayghri/qwantize

Formats

• INT8 – Symmetric INT8 with FP8 E4M3 scales (block sizes 32, 64, 128, 256)

• NVFP4 – FP4 E2M1 with FP8 E4M3 scales (block sizes 16, 32)

• MXFP4 – FP4 E2M1 with UE8M0 (power-of-2) scales (block sizes 16, 32)

Quick Start

from qwantize import nvfp4_naive, nvfp4_optimal, nvfp4_dequantize, compute_metrics

# W has shape (..., block_size) where block_size is 16 or 32
# dim specifies which dimension is the block dimension (default: -1)
W_blocked = W.reshape(M, K // 32, 32)

# Quantize: returns (scales, quants)
scales, quants = nvfp4_optimal(W_blocked, dim=-1)

# Dequantize separately
W_dq = nvfp4_dequantize(scales, quants, dim=-1)

# Or get dequantized output directly
scales, quants, W_dq = nvfp4_optimal(W_blocked, dim=-1, return_dequant=True)

metrics = compute_metrics(W, W_dq.reshape(M, K), X)

Methods

• Optimal Scale Search
  • 1. Problem Formulation
  • 2. Baseline Error Anchor
  • 3. Lower Bound (Clipping Dominance)
  • 4. Upper Bound (Dead-Zone Dominance)
  • 5. The Optimized Search Algorithm
• Hessian-Aware Optimal Scale Search
  • 1. Motivation
  • 2. Block Hessian Structure
  • 3. Hessian-Weighted Error
  • 4. Adapting the Bounded Search
  • 5. Efficient Computation
  • 6. Closed-Form Optimal Continuous Scale
  • 7. Results
• SPGL1 LASSO Compensation
  • 1. Motivation
  • 2. Problem Setup
  • 3. Reduced (Gram) Form
  • 4. The Full Pipeline
  • 5. What It Replaces in GPTQ
  • 6. Cost
  • 7. Results
• Scale Distance Analysis
  • Motivation
  • NVFP4: FP8 E4M3 Scale Distances
  • MXFP4: UE8M0 Scale Distances
  • Summary
• Triton Kernels
  • Architecture
  • NVFP4 Kernels
  • MXFP4 Kernels
  • Zero-Divergence Design
• Custom Codebook Optimization
  • 1. Problem Statement
  • 2. Sign Reduction
  • 3. Scale Normalization
  • 4. The Optimization Problem
  • 5. Per-Block Scale Selection
  • 6. Discussion

Results

• Results
  • INT8 (FP8 E4M3 scales)
  • NVFP4 (FP8 E4M3 scales)
  • MXFP4 (UE8M0 power-of-2 scales)
  • Why NVFP4 benefits much more from Hessian-awareness
  • Correctness notes
  • GPTQ Quantization
  • Exotic Scales
  • Larger blocks: bs=64 (E4M3 scales) and bs=128 (FP16 scales)

API Reference

• INT8
  • Reference Implementation
  • Constants
• NVFP4
  • Reference Implementation
  • Triton Kernels
  • Constants
• MXFP4
  • Reference Implementation
  • Triton Kernels
  • Constants
• Metrics
  • compute_metrics()