Decomposed Geometry & Reflectance as UV Maps
The polarised data enables the decomposition of facial reflectance in the form of albedo, normal and specular in the UV space.
The polarised data enables the decomposition of facial reflectance in the form of albedo, normal and specular in the UV space.
The decomposed reflectance information allows us to relight the head avatars using arbitrary environment maps.
Our method allows us to edit the appearane of the subject by modifying the albedo. Additionally, our head representation allows us to edit the facial structure by changing the shape of the underlying parametric head model.
We propose LightHeadEd, a novel, cost-effective approach for creating high-005 quality relightable head avatars using only a smartphone equipped with polaroid filter. Our approach involves simultaneously capturing cross-polarized and parallel-polarized video streams in a dark room with a single point-light source, enabling the separation of skin's diffuse and specular components during dynamic facial performances. We introduce a hybrid representation that embeds 2D Gaussian Splats (2DGS) in the UV space of a parametric head model, facilitating efficient real-time rendering while preserving high-fidelity geometric details. Our learning-based neural analysis-by-synthesis pipeline decouples pose and expression-dependent geometrical offsets from appearance, decomposing the surface into albedo, normal, and specular UV texture maps without requiring pre-trained texture decomposition models.
We collect a dataset of 30 subjects performing diverse facial expressions and head movements using the proposed capture process. Our approach aims to bring scalability to polarized facial performance capture while significantly reducing the complexity and cost of the setup, democratizing high-quality relightable head avatar creation.
We collect a dataset of multiple subjects performing diverse facial expressions and head movements using the proposed capture process. Our approach aims to bring scalability to polarized facial performance capture while significantly reducing the complexity and cost of the setup, democratizing high-quality relightable head avatar creation.