A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work

This note clarifies the relationship between the recent TurboQuant work and the earlier DRIVE (NeurIPS 2021) and EDEN (ICML 2022) schemes. DRIVE is a 1-bit quantizer that EDEN extended to any \(b>0\) bits per coordinate; we refer to them collectively as EDEN. First, TurboQuant\(_{text{mse}}\) is a special case of EDEN obtained by fixing EDEN’s scalar scale parameter to \(S=1\). EDEN supports both biased and unbiased quantization, each optimized by a different \(S\) (chosen via methods described in the EDEN works). The fixed choice \(S=1\) used by TurboQuant is generally suboptimal, although the optimal \(S\) for biased EDEN converges to $1$ as the dimension grows; accordingly TurboQuant\(_{text{mse}}\) approaches EDEN’s behavior for large \(d\). Second, TurboQuant\(_{text{prod}}\) combines a biased \((b-1)\)-bit EDEN step with an unbiased 1-bit QJL quantization of the residual. It is suboptimal in three ways: (1) its \((b-1)\)-bit step uses the suboptimal \(S=1\); (2) its 1-bit unbiased residual quantization has worse MSE than (unbiased) 1-bit EDEN; (3) chaining a biased \((b-1)\)-bit step with a 1-bit unbiased residual step is inferior to unbiasedly quantizing the input directly with \(b\)-bit EDEN. Third, some of the analysis in the TurboQuant work mirrors that of the EDEN works: both exploit the connection between random rotations and the shifted Beta distribution, use the Lloyd-Max algorithm, and note that Randomized Hadamard Transforms can replace uniform random rotations. Experiments support these claims: biased EDEN (with optimized \(S\)) is more accurate than TurboQuant\(_{text{mse}}\), and unbiased EDEN is markedly more accurate than TurboQuant\(_{text{prod}}\), often by more than a bit (e.g., 2-bit EDEN beats 3-bit TurboQuant\(_{text{prod}}\)). We also repeat all accuracy experiments from the TurboQuant paper, showing that EDEN outperforms it in every setup we have tried.