Blog

Researchers at USC reconstruct a full-body 3D avatar from a single image

By Jessica Xieyang Qiao

Given a single image of a person from the frontal view, researches at the University of Southern California can automatically reconstruct a complete and texted 3D clothed body shape by leveraging the deep learning technology.

The Institute for Creative Technologies (ICT) of the University of Southern California recently introduced a silhouette-based technology for modeling clothed human bodies using deep learning. By simply taking a picture of a person as an image input, researchers are able to predict and digitize a 3D clothed subject that is comparable to those obtained from multi-view input.

Researchers are able to extract 2D silhouettes, feed that into an image translation network driven by deep learning, and use that input to infer non-visible body textures and joints for 3D reconstruction. The ability to match side views in 2D with geometry in 3D opens doors to immense real-world applications such as virtual try-on and 3D printing, said Weikai Chen, researcher at Vision and Graphics Lab at the University of Southern California.

“A system that could generate a full-body 3D avatar of a person by simply taking a picture as input would significantly impact the scalability of producing virtual humans for immersive content creation, as well as its attainability by the general population,” Chen said.

This technology is far from perfect and still faces glitches. One hurdle facing researchers at ICT is modeling that fails to identify accessories worn by the person because of limited or biased data. Researchers said they would continue to refine algorithms behind this technology and extend the methods to predict output with high accuracy and details in the near future.

The Institute for Creative Technologies of the University of Southern California dedicates to the advancement of artificial intelligence, virtual reality, and immersive technologies. Recent breakthroughs include silhouette-based reconstructions of the human body, face, and even hair.