Neural Solver of Dichromatic Reflection Model for Specular Highlight Removal

The Dichromatic Reflection Model (DRM), a widely used physical image formation model, has been extensively applied to specular highlight removal. However, traditional DRM solvers fail to effectively recover the missing content underneath specular highlights and are prone to producing visual artifacts. Additionally, existing deep learning-based methods do not fully exploit the full set of the DRM variables. Instead, they typically learn to translate an input image into its diffuse image (and specular residue image). As a result, their performance remains somewhat limited. To overcome these limitations, we propose a neural DRM solver for specular highlight removal. Our pipeline consists of three networks: Highlight Detection Network (HDNet), Alpha-Chrom Estimation Network (ACENet), and Refinement Network (RNet). Specifically, HDNet is first used to detect specular highlights. Meanwhile, guided by multi-level contextual contrasted features from HDNet, ACENet estimates the DRM variables. These estimated variables are then used by our reconstruction models to generate the specular-free and specular residue images. Finally, we introduce RNet, a model-free network, to further refine the results. Extensive experiments on existing datasets and our collected images demonstrate that our neural solver is superior to previous methods.