Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Data Science and Artificial Intelligence

Escaping Rounding Barriers in Large Language Model Quantization via Transverse-Field Ising Optimization

Anh Nguyen Duc, Linh Ngo Van, Sang Dinh Viet, Hoang Ta Duy, Tung Nguyen Son, Canh Dinh The, Tu Vu Van

Published Oct 15, 2026• 0 citations
PDF (CC BY)Publisher

Abstract

Weight-only post-training quantization (PTQ) pipelines such as AWQ ultimately commit to a deterministic rounding decision for every scalar weight, and recent discrete coordinate-descent quantizers refine those decisions one weight at a time. We show that both are blocked by the same obstacle: correlated activation channels create multi-spin energy barriers in the discrete rounding landscape — configurations in which no single rounding flip helps, yet a small group of coordinated flips strictly reduces the layer reconstruction error. We characterize these barriers exactly under the empirical activation metric and prove that they arise only from frustration between local quantization fields and cross-channel couplings. Building on this analysis we propose Transverse-Field Ising Correction (TFIC), a barrier-aware discrete optimizer that post-processes any AWQ-style pipeline. TFIC casts fixed-grid rounding as a classical Ising ground-state problem, selects the spins near a rounding boundary as a mobile pool, drives that pool to a single-flip local minimum by exact coordinate descent, and then crosses barriers with cluster moves that flip small groups of coupled spins together. Every accepted move is scored against the exact reconstruction energy, and a robust per-layer noise-floor acceptance rule prunes flips whose apparent gain is within calibration-estimation noise; together these guarantee that each layer’s calibration reconstruction error is monotonically non-increasing. TFIC requires no gradients and no architectural changes. On Mistral-7B and Llama-3-8B, applying TFIC on top of a 4-bit AWQ baseline consistently reduces WikiText-2 and C4 perplexity, at a per-layer cost of tens of seconds and with no inference-time overhead.

DOI
10.51316/jst.194.dsai.2026.36.1.1

Related papers

OJBKQ: Objective-Joint Babai-Klein Quantization

Xinyu Wang, Ziyu Zhao, Peng Lu +2 more

Post-training quantization (PTQ) is widely used to compress large language models without retraining. However, many existing weight-only methods rely on heuristic objectives and greedy rounding strategies, leading to noticeable degradation under low-bit quantization. We propose OJBKQ, a layer-wise PTQ method that formulates weight quantization as a joint optimization problem over activations and weights, yielding a multiple-right-hand-side box-constrained integer least squares (BILS) problem per layer. To solve this NP-hard problem, we extend Klein’s randomized variant of Babai’s nearest-plane algorithm to handle box constraints and run the extended algorithm K times to generate K box-constrained Klein points. The minimum-residual candidate among these points and the box-constrained Babai point is selected as a suboptimal solution. Experiments on large language models show that OJBKQ achieves lower perplexity at 3–4 bits than existing PTQ methods, while maintaining comparable computational cost.

arXiv:2602.08376•International Joint Conference on Artificial Intelligence•Sep 1, 2026•2 citations

PEQuant: 거대 언어 모델의 양자화 오류를 최소화하기 위한 실용적이고 경험적인 방법

최동훈

Post-training quantization (PTQ) is simpler and faster to implement and consumes fewer resources than quantization-aware training (QAT), which requires retraining the entire model. Existing PTQ methods adjust the quantization granularity, modify the model architecture, or apply equivalent transformations to reduce quantization errors. However, these PTQ approaches often increase computational overhead, complicate implementation, and require resource-consuming efforts to find the optimal quantization configuration, diminishing PTQs advantages over QAT. To address these challenges, we propose PEQuant, a set of effective yet simple techniques that exploit model characteristics and quantization features to quantize large language models (LLMs). Our method uses Per-tensor Static Uniform Symmetric Round-to-Nearest Quantization with minimal quantization errors, eliminating the need for intricate quantization schemes. PEQuant comprises three main techniques. First, we identify the optimal sweet spot for clipping activation outliers for each model. Second, based on our observation that certain layers exhibit significant outliers, we apply selective quantization to layers sensitive to quantization while using W8A8 (weight INT8, activation INT8) for the remaining layers, substantially reducing quantization errors. Finally, we introduce a calibration layer at the models output, trained via knowledge distillation, which further mitigates quantization errors. We implement PEQuant with NVIDIA CUTLASS and evaluate it on various LLMs, including OPT, LLaMA2, and LLaMA3, across perplexity and zero-shot downstream tasks. We also compare PEQuant with two state-of-the-art PTQ methods, LLM.int8() and SmoothQuant. The experimental results indicate that PEQuant is practical and efficient in saving memory spaces, increasing inference speed, and reducing quantization errors.

•Jan 1, 2025•0 citations

Pushing the Limits of Block Rotations in Post-Training Quantization

Sai Sanjeet, Ian Colbert, Pablo Monteagudo-Lago +3 more

Recent post-training quantization (PTQ) methods have adopted block rotations to diffuse outliers prior to rounding. While this reduces the overhead of online full-vector rotations, the effect of block structure on outlier suppression remains poorly understood. To fill this gap, we present the first systematic, non-asymptotic analysis of outlier suppression for block Hadamard rotations. Our analysis reveals that outlier suppression is fundamentally limited by the geometry of the input vector. In particular, in the deterministic worst case, post-rotation outliers are minimized when the pre-rotation $\ell_1$ norm mass is evenly distributed across blocks. Guided by these insights, we introduce PeRQ (Permute, Rotate, then Quantize), a PTQ framework that redistributes activation mass via permutations prior to rotation. We propose a greedy mass diffusion algorithm to calibrate permutations by equalizing the expected blockwise $\ell_1$ norms. To avoid adding inference overhead, we identify permutation-equivariant regions in transformer architectures to merge these permutations into model weights before deployment. Experiments show that PeRQ consistently improves accuracy across all block sizes, recovering up to 90% of the full-vector rotation perplexity when quantizing Llama3 1B to INT4 with block size 16, compared to 46% without permutations.

arXiv:2601.22347•Jan 29, 2026•10 citations

Rethinking Practical and Efficient Quantization Calibration for Vision-Language Models

Zhenhao Shang, Haizhao Jing, Guoting Wei +4 more

Post-training quantization (PTQ) is a primary approach for deploying large language models without fine-tuning, and the quantized performance is often strongly affected by the calibration in PTQ. By contrast, in vision-language models (VLMs), substantial differences between visual and text tokens in their activation distributions and sensitivities to quantization error pose significant challenges for effective calibration during PTQ. In this work, we rethink what PTQ calibration should align with in VLMs and propose the Token-level Importance-aware Layer-wise Quantization framework (TLQ). Guided by gradient information, we design a token-level importance integration mechanism for quantization error, and use it to construct a token-level calibration set, enabling a more fine-grained calibration strategy. Furthermore, TLQ introduces a multi-GPU, quantization-exposed layer-wise calibration scheme. This scheme keeps the layer-wise calibration procedure consistent with the true quantized inference path and distributes the complex layer-wise calibration workload across multiple RTX3090 GPUs, thereby reducing reliance on the large memory of A100 GPUs. TLQ is evaluated across two models, three model scales, and two quantization settings, consistently achieving performance improvements across all settings, indicating its strong quantization stability. The code will be released publicly.

arXiv:2602.07899•Feb 8, 2026•2 citations

One Qubit Can Beat One Bit: Quantum Advantage for Post-Training Quantization

Yuma Ichikawa, Moeto Mishima

One-bit post-training quantization represents each weight using only its sign, requiring all deployment contexts to share the same binary weight matrix even when their activation statistics favor different sign patterns. We study this shared-sign constraint and introduce Quantum Random Access Quantization (QRAQ). This framework encodes context-dependent signs in a quantum random-access code and retrieves them via context-matched Pauli measurements. Under an explicit fresh-copy logical readout model, QRAQ produces an unbiased, context-specific binary surrogate with a tractable shot-noise penalty. We prove a row-wise separation from shared-sign one-bit PTQ with signed per-row scales. When the optimal context-wise signs are incompatible, QRAQ achieves a strictly lower ideal reconstruction risk. We also derive finite-shot and calibrated-noise conditions under which this separation is retained. Fixed-readout quantum schemes are classically simulable, so the relevant resource in this model is measurement incompatibility rather than quantization alone. Finally, we characterize the role of scale granularity, provide finite-sample certificates, and evaluate the predicted ideal, finite-shot, noisy, and multi-context regimes in simulator experiments.

quant-pharXiv:2608.05240•Aug 6, 2026•0 citations

Tree-Ising: Repairing Language Models with Bit-Level Ising Edits

Hadi Salloum, Ammar Ali

Weight-space programming, introduced in recent MIT work by Takehi et al., enables a frontier LLM agent to read a network's weights and write a repair. Their GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B without gradients, using 158 agent turns, 89.7 million tokens, and 50 forward-pass requests. Our method, Tree-Ising, casts this repair as a sparse discrete optimization problem over the model's own BF16 weights. Forward statistics define an objective that weakens a loop direction, strengthens a concluding direction, and preserves the layer's output on productive tokens. Within a state-tree representation, where lossless BF16 storage acts as a special case, the admissible edits are sign flips and exponent shifts that preserve all shared group exponents and mantissas. We decompose this objective exactly into independent QUBO (Ising) problems, one per output row, which we solve on a GPU using simulated annealing or quantum-inspired simulated bifurcation. On the agent's two layers, Tree-Ising achieves 54.6% on MATH-500, outperforming the agent's edit (51.7%) under the same evaluation protocol, while the base model scores 7.9% as reported. The simulated bifurcation solver yields a statistically indistinguishable repair at 55.1%. Both solvers outperform the agent edit across all five math benchmarks, with statistically significant improvements on MATH-500 and GSM8K after correcting for multiple comparisons. Our approach modifies 13% to 17% fewer weights and reduces BF16 bit flips by a factor of 2.2 to 2.3. It requires only four forward-pass requests and zero LLM tokens, with the annealing recipe costing about $0.90 in GPU time. On Qwen3.5-2B, with layers and dose selected using only the base model, the same recipe increases greedy GSM8K accuracy from 21.6% to 69.4%. As a proof of concept, we execute an eight-variable repair subproblem using QAOA on an IBM Heron processor.

arXiv:2611.14210•Nov 10, 2026•0 citations
Weight-only post-training quantization (PTQ) pipelines such as AWQ ultimately commit to a deterministic rounding decision for every scalar weight, and recent discrete coordinate-descent quantizers refine those decisions one weight at a time. We show that both are blocked by the same obstacle: correlated activation channels create multi-spin energy barriers in the discrete rounding landscape — configurations in which no single rounding flip helps, yet a small group of coordinated flips strictly reduces the layer reconstruction error. We characterize these barriers exactly under the empirical activation metric and prove that they arise only from frustration between local quantization fields and cross-channel couplings. Building on this analysis we propose Transverse-Field Ising Correction (TFIC), a barrier-aware discrete optimizer that post-processes any AWQ-style pipeline. TFIC casts fixed-grid rounding as a classical Ising ground-state problem, selects the spins near a rounding boundary as a mobile pool, drives that pool to a single-flip local minimum by exact coordinate descent, and then crosses barriers with cluster moves that flip small groups of coupled spins together. Every accepted move is scored against the exact reconstruction energy, and a robust per-layer noise-floor acceptance rule prunes flips whose apparent gain is within calibration-estimation noise; together these guarantee that each layer’s calibration reconstruction error is monotonically non-increasing. TFIC requires no gradients and no architectural changes. On Mistral-7B and Llama-3-8B, applying TFIC on top of a 4-bit AWQ baseline consistently reduces WikiText-2 and C4 perplexity, at a per-layer cost of tens of seconds and with no inference-time overhead.
Post-training quantization (PTQ) is widely used to compress large language models without retraining. However, many existing weight-only methods rely on heuristic objectives and greedy rounding strategies, leading to noticeable degradation under low-bit quantization. We propose OJBKQ, a layer-wise PTQ method that formulates weight quantization as a joint optimization problem over activations and weights, yielding a multiple-right-hand-side box-constrained integer least squares (BILS) problem per layer. To solve this NP-hard problem, we extend Klein’s randomized variant of Babai’s nearest-plane algorithm to handle box constraints and run the extended algorithm K times to generate K box-constrained Klein points. The minimum-residual candidate among these points and the box-constrained Babai point is selected as a suboptimal solution. Experiments on large language models show that OJBKQ achieves lower perplexity at 3–4 bits than existing PTQ methods, while maintaining comparable computational cost.
Post-training quantization (PTQ) is simpler and faster to implement and consumes fewer resources than quantization-aware training (QAT), which requires retraining the entire model. Existing PTQ methods adjust the quantization granularity, modify the model architecture, or apply equivalent transformations to reduce quantization errors. However, these PTQ approaches often increase computational overhead, complicate implementation, and require resource-consuming efforts to find the optimal quantization configuration, diminishing PTQs advantages over QAT. To address these challenges, we propose PEQuant, a set of effective yet simple techniques that exploit model characteristics and quantization features to quantize large language models (LLMs). Our method uses Per-tensor Static Uniform Symmetric Round-to-Nearest Quantization with minimal quantization errors, eliminating the need for intricate quantization schemes. PEQuant comprises three main techniques. First, we identify the optimal sweet spot for clipping activation outliers for each model. Second, based on our observation that certain layers exhibit significant outliers, we apply selective quantization to layers sensitive to quantization while using W8A8 (weight INT8, activation INT8) for the remaining layers, substantially reducing quantization errors. Finally, we introduce a calibration layer at the models output, trained via knowledge distillation, which further mitigates quantization errors. We implement PEQuant with NVIDIA CUTLASS and evaluate it on various LLMs, including OPT, LLaMA2, and LLaMA3, across perplexity and zero-shot downstream tasks. We also compare PEQuant with two state-of-the-art PTQ methods, LLM.int8() and SmoothQuant. The experimental results indicate that PEQuant is practical and efficient in saving memory spaces, increasing inference speed, and reducing quantization errors.
Recent post-training quantization (PTQ) methods have adopted block rotations to diffuse outliers prior to rounding. While this reduces the overhead of online full-vector rotations, the effect of block structure on outlier suppression remains poorly understood. To fill this gap, we present the first systematic, non-asymptotic analysis of outlier suppression for block Hadamard rotations. Our analysis reveals that outlier suppression is fundamentally limited by the geometry of the input vector. In particular, in the deterministic worst case, post-rotation outliers are minimized when the pre-rotation ℓ1\ell_1ℓ1​ norm mass is evenly distributed across blocks. Guided by these insights, we introduce PeRQ (Permute, Rotate, then Quantize), a PTQ framework that redistributes activation mass via permutations prior to rotation. We propose a greedy mass diffusion algorithm to calibrate permutations by equalizing the expected blockwise ℓ1\ell_1ℓ1​ norms. To avoid adding inference overhead, we identify permutation-equivariant regions in transformer architectures to merge these permutations into model weights before deployment. Experiments show that PeRQ consistently improves accuracy across all block sizes, recovering up to 90% of the full-vector rotation perplexity when quantizing Llama3 1B to INT4 with block size 16, compared to 46% without permutations.
Post-training quantization (PTQ) is a primary approach for deploying large language models without fine-tuning, and the quantized performance is often strongly affected by the calibration in PTQ. By contrast, in vision-language models (VLMs), substantial differences between visual and text tokens in their activation distributions and sensitivities to quantization error pose significant challenges for effective calibration during PTQ. In this work, we rethink what PTQ calibration should align with in VLMs and propose the Token-level Importance-aware Layer-wise Quantization framework (TLQ). Guided by gradient information, we design a token-level importance integration mechanism for quantization error, and use it to construct a token-level calibration set, enabling a more fine-grained calibration strategy. Furthermore, TLQ introduces a multi-GPU, quantization-exposed layer-wise calibration scheme. This scheme keeps the layer-wise calibration procedure consistent with the true quantized inference path and distributes the complex layer-wise calibration workload across multiple RTX3090 GPUs, thereby reducing reliance on the large memory of A100 GPUs. TLQ is evaluated across two models, three model scales, and two quantization settings, consistently achieving performance improvements across all settings, indicating its strong quantization stability. The code will be released publicly.
One-bit post-training quantization represents each weight using only its sign, requiring all deployment contexts to share the same binary weight matrix even when their activation statistics favor different sign patterns. We study this shared-sign constraint and introduce Quantum Random Access Quantization (QRAQ). This framework encodes context-dependent signs in a quantum random-access code and retrieves them via context-matched Pauli measurements. Under an explicit fresh-copy logical readout model, QRAQ produces an unbiased, context-specific binary surrogate with a tractable shot-noise penalty. We prove a row-wise separation from shared-sign one-bit PTQ with signed per-row scales. When the optimal context-wise signs are incompatible, QRAQ achieves a strictly lower ideal reconstruction risk. We also derive finite-shot and calibrated-noise conditions under which this separation is retained. Fixed-readout quantum schemes are classically simulable, so the relevant resource in this model is measurement incompatibility rather than quantization alone. Finally, we characterize the role of scale granularity, provide finite-sample certificates, and evaluate the predicted ideal, finite-shot, noisy, and multi-context regimes in simulator experiments.
Weight-space programming, introduced in recent MIT work by Takehi et al., enables a frontier LLM agent to read a network's weights and write a repair. Their GPT-6 Astra agent corrected the reasoning loops of Qwen3.5-0.8B without gradients, using 158 agent turns, 89.7 million tokens, and 50 forward-pass requests. Our method, Tree-Ising, casts this repair as a sparse discrete optimization problem over the model's own BF16 weights. Forward statistics define an objective that weakens a loop direction, strengthens a concluding direction, and preserves the layer's output on productive tokens. Within a state-tree representation, where lossless BF16 storage acts as a special case, the admissible edits are sign flips and exponent shifts that preserve all shared group exponents and mantissas. We decompose this objective exactly into independent QUBO (Ising) problems, one per output row, which we solve on a GPU using simulated annealing or quantum-inspired simulated bifurcation. On the agent's two layers, Tree-Ising achieves 54.6% on MATH-500, outperforming the agent's edit (51.7%) under the same evaluation protocol, while the base model scores 7.9% as reported. The simulated bifurcation solver yields a statistically indistinguishable repair at 55.1%. Both solvers outperform the agent edit across all five math benchmarks, with statistically significant improvements on MATH-500 and GSM8K after correcting for multiple comparisons. Our approach modifies 13% to 17% fewer weights and reduces BF16 bit flips by a factor of 2.2 to 2.3. It requires only four forward-pass requests and zero LLM tokens, with the annealing recipe costing about $0.90 in GPU time. On Qwen3.5-2B, with layers and dose selected using only the base model, the same recipe increases greedy GSM8K accuracy from 21.6% to 69.4%. As a proof of concept, we execute an eight-variable repair subproblem using QAOA on an IBM Heron processor.