CF3: Compact and Fast 3D Feature Fields

3D Gaussian Splatting (3DGS) has begun incorporating rich information from 2D foundation models. However, most approaches rely on a bottom-up optimization process that treats raw 2D features as ground truth, incurring increased computational costs. We propose a top-down pipeline for constructing compact and fast 3D feature fields, namely, CF3. We first perform a fast weighted fusion of multi-view 2D features with pre-trained Gaussians. This approach enables training a per-Gaussian autoencoder directly on the lifted features, instead of training autoencoders in the 2D domain. As a result, the autoencoder better aligns with the inference-time feature distribution, achieving high compression fidelity and storage efficiency. Finally, we introduce an adaptive sparsification method that prunes and merges Gaussians, constructing an efficient representation while maintaining details. Our approach achieves a competitive 3D feature field while using as little as 5% of the Gaussians compared to Feature-3DGS.