Beyond Real Weights: Hypercomplex Representations for Stable Quantization

Multimodal language models (MLLMs) demand immense parameter capacity to align high-dimensional visual features with linguistic representations, making them computationally heavy and difficult to deploy efficiently. We introduce a progressive reparameterization strategy that physically compresses these models by progressively replacing dense feed-forward network blocks with compact Parameterized Hypercomplex Multiplication (PHM) layers. A residual interpolation schedule, together with lightweight reconstruction and knowledge distillation losses, ensures that the PHM modules inherit the functional behavior of their dense counterparts throughout training. This transition yields substantial parameter and FLOP reductions while preserving strong multimodal alignment, leading to faster inference without degrading output quality. We evaluate the approach on vision-language models (VLMs). Our approach preserves strong performance comparable with base VLMs, while delivering substantial reductions in model size and inference latency. Progressive PHM substitution thus offers an architecture-compatible path toward more efficient multimodal reasoning and complements existing low-bit quantization techniques. Codes are available at https://github.com/milab-nsu/PHM.