awesome_head_avatar_3d.md

3D Head Avatar

• 3D Head Avatar
  • Generating
  • Editing
  • Animation
  • Datasets
  • Extra

---

Generating

• TextToon: Real-Time Text Toonify Head Avatar from Single Video, arXiv, 2410.07160, arxiv, pdf, cication: -1

  Luchuan Song, Lele Chen, Celong Liu, Pinxin Liu, Chenliang Xu · (songluchuan.github) · (songluchuan.github)

• Portrait Video Editing Empowered by Multimodal Generative Priors, arXiv, 2409.13591, arxiv, pdf, cication: -1

  Xuan Gao, Haiyao Xiao, Chenglai Zhong, Shimin Hu, Yudong Guo, Juyong Zhang · (ustc3dv.github)

• SPARK: Self-supervised Personalized Real-time Monocular Face Capture, arXiv, 2409.07984, arxiv, pdf, cication: -1

  Kelian Baert, Shrisha Bharadwaj, Fabien Castan, Benoit Maujean, Marc Christie, Victoria Abrevaya, Adnane Boukhayma · (kelianb.github)

• GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations, arXiv, 2409.11951, arxiv, pdf, cication: -1

  Kartik Teotia, Hyeongwoo Kim, Pablo Garrido, Marc Habermann, Mohamed Elgharib, Christian Theobalt

  · (vcai.mpi-inf.mpg) · (GaussianHeads - Kartik-Teotia)

• GenCA: A Text-conditioned Generative Model for Realistic and Drivable Codec Avatars, arXiv, 2408.13674, arxiv, pdf, cication: -1

  Keqiang Sun, Amin Jourabloo, Riddhish Bhalodia, Moustafa Meshry, Yu Rong, Zhengyu Yang, Thu Nguyen-Phuoc, Christian Haene, Jiu Xu, Sam Johnson

• Barbie - 2017211801

  · (Barbie - 2017211801)

• HeadGAP: Few-shot 3D Head Avatar via Generalizable Gaussian Priors, arXiv, 2408.06019, arxiv, pdf, cication: -1

  Xiaozheng Zheng, Chao Wen, Zhaohu Li, Weiyi Zhang, Zhuo Su, Xu Chang, Yang Zhao, Zheng Lv, Xiaoyuan Zhang, Yongjie Zhang · (headgap.github)

• Gaussian Eigen Models for Human Heads, arXiv, 2407.04545, arxiv, pdf, cication: -1

  Wojciech Zielonka, Timo Bolkart, Thabo Beeler, Justus Thies · (zielon.github)

• EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Head, arXiv, 2408.00297, arxiv, pdf, cication: -1

  Qianyun He, Xinya Ji, Yicheng Gong, Yuanxun Lu, Zhengyu Diao, Linjia Huang, Yao Yao, Siyu Zhu, Zhan Ma, Songcen Xu · (nju-3dv.github)

• Bridging the Gap: Studio-like Avatar Creation from a Monocular Phone Capture, arXiv, 2407.19593, arxiv, pdf, cication: -1

  ShahRukh Athar, Shunsuke Saito, Zhengyu Yang, Stanislav Pidhorsky, Chen Cao · (shahrukhathar.github)

• 3D Gaussian Parametric Head Model, arXiv, 2407.15070, arxiv, pdf, cication: -1

  Yuelang Xu, Lizhen Wang, Zerong Zheng, Zhaoqi Su, Yebin Liu · (yuelangx.github)

• RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models, arXiv, 2407.06938, arxiv, pdf, cication: -1

  Bowen Zhang, Yiji Cheng, Chunyu Wang, Ting Zhang, Jiaolong Yang, Yansong Tang, Feng Zhao, Dong Chen, Baining Guo · (rodinhd.github)

• FAGhead: Fully Animate Gaussian Head from Monocular Videos, arXiv, 2406.19070, arxiv, pdf, cication: -1

  Yixin Xuan, Xinyang Li, Gongxin Yao, Shiwei Zhou, Donghui Sun, Xiaoxin Chen, Yu Pan

• Portrait3D: 3D Head Generation from Single In-the-wild Portrait Image

  · (jinkun-hao.github)

• GGHead: Fast and Generalizable 3D Gaussian Heads, arXiv, 2406.09377, arxiv, pdf, cication: -1

  Tobias Kirschstein, Simon Giebenhain, Jiapeng Tang, Markos Georgopoulos, Matthias Nießner · (tobias-kirschstein.github)

• Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head Capture, arXiv, 2406.00440, arxiv, pdf, cication: -1

  X. Li, Y. Cheng, X. Ren, H. Jia, D. Xu, W. Zhu, Y. Yan · (xuanchenli.github)

• ChatDirector: Enhancing Video Conferencing with Space-Aware Scene Rendering and Speech-Driven Layout Transition | Proceedings of the CHI Conference on Human Factors in Computing Systems

  · (research)

• NPGA: Neural Parametric Gaussian Avatars, arXiv, 2405.19331, arxiv, pdf, cication: -1

  Simon Giebenhain, Tobias Kirschstein, Martin Rünz, Lourdes Agapito, Matthias Nießner

• GGAvatar: Geometric Adjustment of Gaussian Head Avatar, arXiv, 2405.11993, arxiv, pdf, cication: -1

  Xinyang Li, Jiaxin Wang, Yixin Xuan, Gongxin Yao, Yu Pan

• GGAvatar: Geometric Adjustment of Gaussian Head Avatar, arXiv, 2405.11993, arxiv, pdf, cication: -1

  Xinyang Li, Jiaxin Wang, Yixin Xuan, Gongxin Yao, Yu Pan

• Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping, arXiv, 2405.12069, arxiv, pdf, cication: -1

  Tianhao Wu, Jing Yang, Zhilin Guo, Jingyi Wan, Fangcheng Zhong, Cengiz Oztireli · (gaussian-head-shoulders.netlify)

• NeRFFaceSpeech: One-shot Audio-diven 3D Talking Head Synthesis via Generative Prior, arXiv, 2405.05749, arxiv, pdf, cication: -1

  Gihoon Kim, Kwanggyoon Seo, Sihun Cha, Junyong Noh

• GSTalker: Real-time Audio-Driven Talking Face Generation via Deformable Gaussian Splatting, arXiv, 2404.19040, arxiv, pdf, cication: -1

  Bo Chen, Shoukang Hu, Qi Chen, Chenpeng Du, Ran Yi, Yanmin Qian, Xie Chen

• MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing, arXiv, 2404.19026, arxiv, pdf, cication: -1

  Cong Wang, Di Kang, He-Yi Sun, Shen-Han Qian, Zi-Xuan Wang, Linchao Bao, Song-Hai Zhang · (conallwang.github)

• GaussianTalker: Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splatting, arXiv, 2404.16012, arxiv, pdf, cication: -1

  Kyusun Cho, Joungbin Lee, Heeji Yoon, Yeobin Hong, Jaehoon Ko, Sangjun Ahn, Seungryong Kim · (GaussianTalker - KU-CVLAB) · (ku-cvlab.github)

• TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting, arXiv, 2404.15264, arxiv, pdf, cication: -1

  Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, Lin Gu · (fictionarry.github)

• GaussianTalker: Speaker-specific Talking Head Synthesis via 3D Gaussian Splatting, arXiv, 2404.14037, arxiv, pdf, cication: -1

  Hongyun Yu, Zhan Qu, Qihang Yu, Jianchuan Chen, Zhonghua Jiang, Zhiwen Chen, Shengyu Zhang, Jimin Xu, Fei Wu, Chengfei Lv

• Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Prior, arXiv, 2404.10394, arxiv, pdf, cication: -1

  Yiqian Wu, Hao Xu, Xiangjun Tang, Xien Chen, Siyu Tang, Zhebin Zhang, Chen Li, Xiaogang Jin

• Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table Blendshapes, arXiv, 2404.01543, arxiv, pdf, cication: -1

  Ziqian Bai, Feitong Tan, Sean Fanello, Rohit Pandey, Mingsong Dou, Shichen Liu, Ping Tan, Yinda Zhang · (augmentedperception.github)

• GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image, arXiv, 2404.02152, arxiv, pdf, cication: -1

  Chong Bao, Yinda Zhang, Yuan Li, Xiyu Zhang, Bangbang Yang, Hujun Bao, Marc Pollefeys, Guofeng Zhang, Zhaopeng Cui · (zju3dv.github)

• ICE: Interactive 3D Game Character Editing via Dialogue, arXiv, 2403.12667, arxiv, pdf, cication: -1

  Haoqian Wu, Yunjie Wu, Zhipeng Hu, Lincheng Li, Weijie Chen, Rui Zhao, Changjie Fan, Yu Xin · (iceedit.github)

• HeadEvolver: Text to Head Avatars via Locally Learnable Mesh Deformation, arXiv, 2403.09326, arxiv, pdf, cication: -1

  Duotun Wang, Hengyu Meng, Zeyu Cai, Zhijing Shao, Qianxi Liu, Lin Wang, Mingming Fan, Ying Shan, Xiaohang Zhan, Zeyu Wang

• SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting, arXiv, 2403.05087, arxiv, pdf, cication: -1

  Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, Zeyu Wang · (SplattingAvatar - initialneil)

• HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting, arXiv, 2402.06149, arxiv, pdf, cication: -1

  Zhenglin Zhou, Fan Ma, Hehe Fan, Yi Yang

• PSAvatar: A Point-based Morphable Shape Model for Real-Time Head Avatar Animation with 3D Gaussian Splatting, arXiv, 2401.12900, arxiv, pdf, cication: -1

  Zhongyuan Zhao, Zhenyu Bao, Qing Li, Guoping Qiu, Kanglin Liu

• UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Textures, arXiv, 2401.11078, arxiv, pdf, cication: -1

  Mingyuan Zhou, Rakib Hyder, Ziwei Xuan, Guojun Qi

• Tri$^{2}$-plane: Volumetric Avatar Reconstruction with Feature Pyramid, arXiv, 2401.09386, arxiv, pdf, cication: -1

  Luchuan Song, Pinxin Liu, Lele Chen, Celong Liu, Chenliang Xu · (Tri2plane - Songluchuan)

• Make-A-Character: High Quality Text-to-3D Character Generation within Minutes, arXiv, 2312.15430, arxiv, pdf, cication: -1

  Jianqiang Ren, Chao He, Lin Liu, Jiahao Chen, Yutong Wang, Yafei Song, Jianfang Li, Tangli Xue, Siqi Hu, Tao Chen · (Make-A-Character - Human3DAIGC) · (human3daigc.github)

• DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesis, arXiv, 2312.13016, arxiv, pdf, cication: -1

  Yuming Gu, You Xie, Hongyi Xu, Guoxian Song, Yichun Shi, Di Chang, Jing Yang, Linjie Luo · (freedomgu.github) · (DiffPortrait3D - FreedomGu)

• HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation, arXiv, 2312.07539, arxiv, pdf, cication: -1

  Hongyu Liu, Xuan Wang, Ziyu Wan, Yujun Shen, Yibing Song, Jing Liao, Qifeng Chen · (kumapowerliu.github)

• IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing, arXiv, 2312.05210, arxiv, pdf, cication: -1

  Shaofei Wang, Božidar Antić, Andreas Geiger, Siyu Tang · (neuralbodies.github)

• Gaussian3Diff: 3D Gaussian Diffusion for 3D Full Head Synthesis and Editing, arXiv, 2312.03763, arxiv, pdf, cication: -1

  Yushi Lan, Feitong Tan, Di Qiu, Qiangeng Xu, Kyle Genova, Zeng Huang, Sean Fanello, Rohit Pandey, Thomas Funkhouser, Chen Change Loy · (nirvanalan.github)

• 360° Volumetric Portrait Avatar, arXiv, 2312.05311, arxiv, pdf, cication: -1

  Jalees Nehvi, Berna Kabadayi, Julien Valentin, Justus Thies · (jalees018.github)

• A Local Appearance Model for Volumetric Capture of Diverse Hairstyle, arXiv, 2312.08679, arxiv, pdf, cication: -1

  Ziyan Wang, Giljoo Nam, Aljaz Bozic, Chen Cao, Jason Saragih, Michael Zollhoefer, Jessica Hodgins

• SEEAvatar: Photorealistic Text-to-3D Avatar Generation with Constrained Geometry and Appearance, arXiv, 2312.08889, arxiv, pdf, cication: -1

  Yuanyou Xu, Zongxin Yang, Yi Yang

• Artist-Friendly Relightable and Animatable Neural Heads, arXiv, 2312.03420, arxiv, pdf, cication: -1

  Yingyan Xu, Prashanth Chandran, Sebastian Weiss, Markus Gross, Gaspard Zoss, Derek Bradley

• Text-Guided 3D Face Synthesis -- From Generation to Editing, arXiv, 2312.00375, arxiv, pdf, cication: -1

  Yunjie Wu, Yapeng Meng, Zhipeng Hu, Lincheng Li, Haoqian Wu, Kun Zhou, Weiwei Xu, Xin Yu

• FlashAvatar: High-Fidelity Digital Avatar Rendering at 300FPS, arXiv, 2312.02214, arxiv, pdf, cication: -1

  Jun Xiang, Xuan Gao, Yudong Guo, Juyong Zhang

• InvertAvatar: Incremental GAN Inversion for Generalized Head Avatars, arXiv, 2312.02222, arxiv, pdf, cication: -1

  Xiaochen Zhao, Jingxiang Sun, Lizhen Wang, Yebin Liu

• MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar, arXiv, 2312.04558, arxiv, pdf, cication: -1

  Yufan Chen, Lizhen Wang, Qijing Li, Hongjiang Xiao, Shengping Zhang, Hongxun Yao, Yebin Liu

• Relightable Gaussian Codec Avatars, arXiv, 2312.03704, arxiv, pdf, cication: -1

  Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, Giljoo Nam · (jiqizhixin)

• GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians, arXiv, 2312.02069, arxiv, pdf, cication: -1

  Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, Matthias Nießner · (jiqizhixin) · (shenhanqian.github)

• Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians, arXiv, 2312.03029, arxiv, pdf, cication: -1

  Yuelang Xu, Benwang Chen, Zhe Li, Hongwen Zhang, Lizhen Wang, Zerong Zheng, Yebin Liu

• DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars, arXiv, 2311.18635, arxiv, pdf, cication: -1

  Tobias Kirschstein, Simon Giebenhain, Matthias Nießner

• Text-Guided 3D Face Synthesis -- From Generation to Editing, arXiv, 2312.00375, arxiv, pdf, cication: -1

  Yunjie Wu, Yapeng Meng, Zhipeng Hu, Lincheng Li, Haoqian Wu, Kun Zhou, Weiwei Xu, Xin Yu · (faceg2e.github)

• GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar, arXiv, 2311.13655, arxiv, pdf, cication: -1

  Berna Kabadayi, Wojciech Zielonka, Bharat Lal Bhatnagar, Gerard Pons-Moll, Justus Thies · (ganavatar.github)

• BakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesis, arXiv, 2311.05521, arxiv, pdf, cication: -1

  Hao-Bin Duan, Miao Wang, Jin-Chuan Shi, Xu-Chuan Chen, Yan-Pei Cao · (BakedAvatar - buaavrcg)

• CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformer, arXiv, 2311.06443, arxiv, pdf, cication: -1

  Haoyu Ma, Tong Zhang, Shanlin Sun, Xiangyi Yan, Kun Han, Xiaohui Xie

• Towards Realistic Generative 3D Face Models, arXiv, 2304.12483, arxiv, pdf, cication: -1

  Aashish Rai, Hiresh Gupta, Ayush Pandey, Francisco Vicente Carrasco, Shingo Jason Takagi, Amaury Aubel, Daeil Kim, Aayush Prakash, Fernando de la Torre

• FLARE: Fast Learning of Animatable and Relightable Mesh Avatars, arXiv, 2310.17519, arxiv, pdf, cication: -1

  Shrisha Bharadwaj, Yufeng Zheng, Otmar Hilliges, Michael J. Black, Victoria Fernandez-Abrevaya · (flare - sbharadwajj)

• Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generation, arXiv, 2208.12550, arxiv, pdf, cication: -1

  Jichao Zhang, Aliaksandr Siarohin, Yahui Liu, Hao Tang, Nicu Sebe, Wei Wang

• High-Fidelity 3D Head Avatars Reconstruction through Spatially-Varying Expression Conditioned Neural Radiance Field, arXiv, 2310.06275, arxiv, pdf, cication: -1

  Minghan Qin, Yifan Liu, Yuelang Xu, Xiaochen Zhao, Yebin Liu, Haoqian Wang

• Towards Practical Capture of High-Fidelity Relightable Avatars, arXiv, 2309.04247, arxiv, pdf, cication: -1

  Haotian Yang, Mingwu Zheng, Wanquan Feng, Haibin Huang, Yu-Kun Lai, Pengfei Wan, Zhongyuan Wang, Chongyang Ma · (huggingface)

• Text2Control3D: Controllable 3D Avatar Generation in Neural Radiance Fields using Geometry-Guided Text-to-Image Diffusion Model, arXiv, 2309.03550, arxiv, pdf, cication: -1

  Sungwon Hwang, Junha Hyung, Jaegul Choo

• AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections, arXiv, 2309.02186, arxiv, pdf, cication: -1

  Yue Wu, Sicheng Xu, Jianfeng Xiang, Fangyun Wei, Qifeng Chen, Jiaolong Yang, Xin Tong

• InstantAvatar: Efficient 3D Head Reconstruction via Surface Rendering, arXiv, 2308.04868, arxiv, pdf, cication: -1

  Antonio Canela, Pol Caselles, Ibrar Malik, Eduard Ramon, Jaime García, Jordi Sánchez-Riera, Gil Triginer, Francesc Moreno-Noguer

• Neural Point-based Volumetric Avatar: Surface-guided Neural Points for Efficient and Photorealistic Volumetric Head Avatar, arXiv, 2307.05000, arxiv, pdf, cication: -1

  Cong Wang, Di Kang, Yan-Pei Cao, Linchao Bao, Ying Shan, Song-Hai Zhang

• AvatarFusion: Zero-shot Generation of Clothing-Decoupled 3D Avatars Using 2D Diffusion, arXiv, 2307.06526, arxiv, pdf, cication: -1

  Shuo Huang, Zongxin Yang, Liangting Li, Yi Yang, Jia Jia

• Articulated 3D Head Avatar Generation using Text-to-Image Diffusion Models, arXiv, 2307.04859, arxiv, pdf, cication: -1

  Alexander W. Bergman, Wang Yifan, Gordon Wetzstein

• NOFA: NeRF-based One-shot Facial Avatar Reconstruction, arXiv, 2307.03441, arxiv, pdf, cication: -1

  Wangbo Yu, Yanbo Fan, Yong Zhang, Xuan Wang, Fei Yin, Yunpeng Bai, Yan-Pei Cao, Ying Shan, Yang Wu, Zhongqian Sun

• SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling, arXiv, 2307.00804, arxiv, pdf, cication: -1

  Zhongjin Luo, Dong Du, Heming Zhu, Yizhou Yu, Hongbo Fu, Xiaoguang Han

• PanoHead: Geometry-Aware 3D Full-Head Synthesis in 360$^{\circ}$, arXiv, 2303.13071, arxiv, pdf, cication: -1

  Sizhe An, Hongyi Xu, Yichun Shi, Guoxian Song, Umit Ogras, Linjie Luo · (sizhean.github) · (panohead - sizhean)

• Generalizable One-shot Neural Head Avatar, arXiv, 2306.08768, arxiv, pdf, cication: -1

  Xueting Li, Shalini De Mello, Sifei Liu, Koki Nagano, Umar Iqbal, Jan Kautz

• Learning Landmarks Motion from Speech for Speaker-Agnostic 3D Talking Heads Generation, arXiv, 2306.01415, arxiv, pdf, cication: -1

  Federico Nocentini, Claudio Ferrari, Stefano Berretti

• HeadSculpt: Crafting 3D Head Avatars with Text, arXiv, 2306.03038, arxiv, pdf, cication: -1

  Xiao Han, Yukang Cao, Kai Han, Xiatian Zhu, Jiankang Deng, Yi-Zhe Song, Tao Xiang, Kwan-Yee K. Wong

• StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation, arXiv, 2305.19012, arxiv, pdf, cication: -1

  Chi Zhang, Yiwen Chen, Yijun Fu, Zhenglin Zhou, Gang YU, Billzb Wang, Bin Fu, Tao Chen, Guosheng Lin, Chunhua Shen · (StyleAvatar3D - icoz69)

• DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models, arXiv, 2304.00916, arxiv, pdf, cication: -1

  Yukang Cao, Yan-Pei Cao, Kai Han, Ying Shan, Kwan-Yee K. Wong · (yukangcao.github)

• Towards Realistic Generative 3D Face Models, arXiv, 2304.12483, arxiv, pdf, cication: -1

  Aashish Rai, Hiresh Gupta, Ayush Pandey, Francisco Vicente Carrasco, Shingo Jason Takagi, Amaury Aubel, Daeil Kim, Aayush Prakash, Fernando de la Torre · (aashishrai3799.github)

• PV3D: A 3D Generative Model for Portrait Video Generation, arXiv, 2212.06384, arxiv, pdf, cication: -1

  Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Wenqing Zhang, Song Bai, Jiashi Feng, Mike Zheng Shou

• Self-Supervised Geometry-Aware Encoder for Style-Based 3D GAN Inversion, arXiv, 2212.07409, arxiv, pdf, cication: -1

  Yushi Lan, Xuyi Meng, Shuai Yang, Chen Change Loy, Bo Dai

• Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video, acm transactions on graphics (tog), 2022, arxiv, pdf, cication: -1

  Xuan Gao, Chenglai Zhong, Jun Xiang, Yang Hong, Yudong Guo, Juyong Zhang · (ustc3dv.github) · (NeRFBlendShape-code - USTC3DV)

Editing

• Instant Facial Gaussians Translator for Relightable and Interactable Facial Rendering, arXiv, 2409.07441, arxiv, pdf, cication: -1

  Dafei Qin, Hongyang Lin, Qixuan Zhang, Kaichun Qiao, Longwen Zhang, Zijun Zhao, Jun Saito, Jingyi Yu, Lan Xu, Taku Komura · (dafei-qin.github)

• Text-Guided Generation and Editing of Compositional 3D Avatars, arXiv, 2309.07125, arxiv, pdf, cication: -1

  Hao Zhang, Yao Feng, Peter Kulits, Yandong Wen, Justus Thies, Michael J. Black

• SketchFaceNeRF: Sketch-based Facial Generation and Editing in Neural Radiance Fields

  · (SketchFaceNeRF - IGLICT) · (dl.acm) · (jiqizhixin)

• FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fields, arXiv, 2307.11418, arxiv, pdf, cication: -1

  Sungwon Hwang, Junha Hyung, Daejin Kim, Min-Jung Kim, Jaegul Choo

• ClipFace: Text-guided Editing of Textured 3D Morphable Models, arXiv, 2212.01406, arxiv, pdf, cication: -1

  Shivangi Aneja, Justus Thies, Angela Dai, Matthias Nießner

  · (shivangi-aneja.github)

• AlteredAvatar: Stylizing Dynamic 3D Avatars with Fast Style Adaptation

• AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars

• Instruct-Video2Avatar - lsx0101

Animation

• 3DFacePolicy: Speech-Driven 3D Facial Animation with Diffusion Policy, arXiv, 2409.10848, arxiv, pdf, cication: -1

  Xuanmeng Sha, Liyun Zhang, Tomohiro Mashita, Yuki Uranishi

• EmoFace: Emotion-Content Disentangled Speech-Driven 3D Talking Face with Mesh Attention, arXiv, 2408.11518, arxiv, pdf, cication: -1

  Yihong Lin, Liang Peng, Jianqiao Hu, Xiandong Li, Wenxiong Kang, Songju Lei, Xianjia Wu, Huang Xu

• DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation, arXiv, 2408.06010, arxiv, pdf, cication: -1

  Jisoo Kim, Jungbin Cho, Joonho Park, Soonmin Hwang, Da Eun Kim, Geon Kim, Youngjae Yu

• GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer, arXiv, 2408.01826, arxiv, pdf, cication: -1

  Yihong Lin, Lingyu Xiong, Xiandong Li, Wenxiong Kang, Xianjia Wu, Liang Peng, Songju Lei, Huang Xu, Zhaoxin Fan

• JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Model, arXiv, 2408.01627, arxiv, pdf, cication: -1

  Farzaneh Jafari, Stefano Berretti, Anup Basu

• UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Model, arXiv, 2408.00762, arxiv, pdf, cication: -1

  Xiangyu Fan, Jiaqi Li, Zhiqian Lin, Weiye Xiao, Lei Yang · (UniTalker - X-niper) · (x-niper.github)

• EmoFace - SJTU-Lucy

• MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset, arXiv, 2406.14272, arxiv, pdf, cication: -1

  Kim Sung-Bin, Lee Chae-Yeon, Gihun Son, Oh Hyun-Bin, Janghoon Ju, Suekyeong Nam, Tae-Hyun Oh · (multi-talk.github) · (MultiTalk - postech-ami)

• FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models, arXiv, 2312.08459, arxiv, pdf, cication: -1

  Shivangi Aneja, Justus Thies, Angela Dai, Matthias Nießner · (shivangi-aneja.github) · (youtu)

• DiffSpeaker: Speech-Driven 3D Facial Animation with Diffusion Transformer, arXiv, 2402.05712, arxiv, pdf, cication: 2

  Zhiyuan Ma, Xiangyu Zhu, Guojun Qi, Chen Qian, Zhaoxiang Zhang, Zhen Lei · (DiffSpeaker - theEricMa)

• 3D Gaussian Blendshapes for Head Avatar Animation, arXiv, 2404.19398, arxiv, pdf, cication: -1

  Shengjie Ma, Yanlin Weng, Tianjia Shao, Kun Zhou · (gapszju.github)

• Learn2Talk: 3D Talking Face Learns from 2D Talking Face, arXiv, 2404.12888, arxiv, pdf, cication: -1

  Yixiang Zhuang, Baoping Cheng, Yao Cheng, Yuntao Jin, Renshuai Liu, Chengyang Li, Xuan Cheng, Jing Liao, Juncong Lin

• AnimateMe: 4D Facial Expressions via Diffusion Models, arXiv, 2403.17213, arxiv, pdf, cication: -1

  Dimitrios Gerogiannis, Foivos Paraperas Papantoniou, Rolandos Alexandros Potamias, Alexandros Lattas, Stylianos Moschoglou, Stylianos Ploumpis, Stefanos Zafeiriou

• EmoVOCA: Speech-Driven Emotional 3D Talking Heads, arXiv, 2403.12886, arxiv, pdf, cication: -1

  Federico Nocentini, Claudio Ferrari, Stefano Berretti

• DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models, arXiv, 2310.00434, arxiv, pdf, cication: 2

  Zhiyao Sun, Tian Lv, Sheng Ye, Matthieu Gaetan Lin, Jenny Sheng, Yu-Hui Wen, Minjing Yu, Yong-jin Liu

  · (diffposetalk.github)

• Motion Mamba: Efficient and Long Sequence Motion Generation with Hierarchical and Bidirectional Selective SSM, arXiv, 2403.07487, arxiv, pdf, cication: -1

  Zeyu Zhang, Akide Liu, Ian Reid, Richard Hartley, Bohan Zhuang, Hao Tang · (steve-zeyu-zhang.github)

• Video-Driven Animation of Neural Head Avatars, arXiv, 2403.04380, arxiv, pdf, cication: -1

  Wolfgang Paier, Paul Hinzer, Anna Hilsmann, Peter Eisert

• AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation, arXiv, 2402.16124, arxiv, pdf, cication: -1

  Yasheng Sun, Wenqing Chu, Hang Zhou, Kaisiyuan Wang, Hideki Koike

• Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance, arXiv, 2401.15687, arxiv, pdf, cication: -1

  Qingcheng Zhao, Pengyu Long, Qixuan Zhang, Dafei Qin, Han Liang, Longwen Zhang, Yingliang Zhang, Jingyi Yu, Lan Xu · (sites.google)

• Fast Registration of Photorealistic Avatars for VR Facial Animation, arXiv, 2401.11002, arxiv, pdf, cication: -1

  Chaitanya Patel, Shaojie Bai, Te-Li Wang, Jason Saragih, Shih-En Wei

• SAiD: Speech-driven Blendshape Facial Animation with Diffusion, arXiv, 2401.08655, arxiv, pdf, cication: -1

  Inkyu Park, Jaewoong Cho

• HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs, arXiv, 2312.14140, arxiv, pdf, cication: -1

  Artem Sevastopolsky, Philip-William Grassal, Simon Giebenhain, ShahRukh Athar, Luisa Verdoliva, Matthias Niessner · (seva100.github)

• VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment, arXiv, 2312.04651, arxiv, pdf, cication: -1

  Phong Tran, Egor Zakharov, Long-Nhat Ho, Anh Tuan Tran, Liwen Hu, Hao Li

• 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting, arXiv, 2312.09228, arxiv, pdf, cication: -1

  Zhiyin Qian, Shaofei Wang, Marko Mihajlovic, Andreas Geiger, Siyu Tang

• FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models, arXiv, 2312.08459, arxiv, pdf, cication: -1

  Shivangi Aneja, Justus Thies, Angela Dai, Matthias Nießner · (shivangi-aneja.github)

• Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks, Methods, and Applications, arXiv, 2311.18168, arxiv, pdf, cication: -1

  Karren D. Yang, Anurag Ranjan, Jen-Hao Rick Chang, Raviteja Vemulapalli, Oncel Tuzel

• Artist-Friendly Relightable and Animatable Neural Heads, arXiv, 2312.03420, arxiv, pdf, cication: -1

  Yingyan Xu, Prashanth Chandran, Sebastian Weiss, Markus Gross, Gaspard Zoss, Derek Bradley · (studios.disneyresearch)

• DiffusionTalker: Personalization and Acceleration for Speech-Driven 3D Face Diffuser, arXiv, 2311.16565, arxiv, pdf, cication: -1

  Peng Chen, Xiaobao Wei, Ming Lu, Yitong Zhu, Naiming Yao, Xingyu Xiao, Hui Chen

• 3DiFACE: Diffusion-based Speech-driven 3D Facial Animation and Editing, arXiv, 2312.00870, arxiv, pdf, cication: -1

  Balamurugan Thambiraja, Sadegh Aliakbarian, Darren Cosker, Justus Thies · (balamuruganthambiraja.github) · (youtube) · (balamuruganthambiraja.github)

• Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians, arXiv, 2312.03029, arxiv, pdf, cication: -1

  Yuelang Xu, Benwang Chen, Zhe Li, Hongwen Zhang, Lizhen Wang, Zerong Zheng, Yebin Liu

• An Implicit Physical Face Model Driven by Expression and Style | Disney Research Studios

• DualTalker: A Cross-Modal Dual Learning Approach for Speech-Driven 3D Facial Animation, arXiv, 2311.04766, arxiv, pdf, cication: -1

  Guinan Su, Yanwu Yang, Zhifeng Li · (iadf.git - sabrina-su)

• LaughTalk: Expressive 3D Talking Head Generation with Laughter, arXiv, 2311.00994, arxiv, pdf, cication: -1

  Kim Sung-Bin, Lee Hyun, Da Hye Hong, Suekyeong Nam, Janghoon Ju, Tae-Hyun Oh

• Breathing Life into Faces: Speech-driven 3D Facial Animation with Natural Head Pose and Detailed Shape, arXiv, 2310.20240, arxiv, pdf, cication: -1

  Wei Zhao, Yijun Wang, Tianyu He, Lianying Yin, Jianxin Lin, Xin Jin

• CorrTalk: Correlation Between Hierarchical Speech and Facial Activity Variances for 3D Animation, arXiv, 2310.11295, arxiv, pdf, cication: -1

  Zhaojie Chu, Kailing Guo, Xiaofen Xing, Yilin Lan, Bolun Cai, Xiangmin Xu

• AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animation, arXiv, 2310.07236, arxiv, pdf, cication: -1

  Liyang Chen, Weihong Bao, Shun Lei, Boshi Tang, Zhiyong Wu, Shiyin Kang, Haozhi Huang

• Controllable Dynamic Appearance for Neural 3D Portraits, arXiv, 2309.11009, arxiv, pdf, cication: -1

  ShahRukh Athar, Zhixin Shu, Zexiang Xu, Fujun Luan, Sai Bi, Kalyan Sunkavalli, Dimitris Samaras

• FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion, arXiv, 2309.11306, arxiv, pdf, cication: -1

  Stefan Stan, Kazi Injamamul Haque, Zerrin Yumak

• DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models, arXiv, 2310.00434, arxiv, pdf, cication: -1

  Zhiyao Sun, Tian Lv, Sheng Ye, Matthieu Gaetan Lin, Jenny Sheng, Yu-Hui Wen, Minjing Yu, Yong-jin Liu

• HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field, arXiv, 2309.17128, arxiv, pdf, cication: -1

  Xiaochen Zhao, Lizhen Wang, Jingxiang Sun, Hongwen Zhang, Jinli Suo, Yebin Liu

• Emotional Speech-Driven Animation with Content-Emotion Disentanglement, arXiv, 2306.08990, arxiv, pdf, cication: -1

  Radek Daněček, Kiran Chhatre, Shashank Tripathi, Yandong Wen, Michael J. Black, Timo Bolkart

• 扎克伯格在元宇宙里一小时的「真人对话」，惊艳了全世界 | 机器之心

• Controllable Dynamic Appearance for Neural 3D Portraits, arXiv, 2309.11009, arxiv, pdf, cication: -1

  ShahRukh Athar, Zhixin Shu, Zexiang Xu, Fujun Luan, Sai Bi, Kalyan Sunkavalli, Dimitris Samaras

• Speech-Driven 3D Face Animation with Composite and Regional Facial Movements, arXiv, 2308.05428, arxiv, pdf, cication: -1

  Haozhe Wu, Songtao Zhou, Jia Jia, Junliang Xing, Qi Wen, Xiang Wen

• SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces, arXiv, 2306.10799, arxiv, pdf, cication: -1

  Ziqiao Peng, Yihao Luo, Yue Shi, Hao Xu, Xiangyu Zhu, Jun He, Hongyan Liu, Zhaoxin Fan

• Audio-Driven 3D Facial Animation from In-the-Wild Videos, arXiv, 2306.11541, arxiv, pdf, cication: -1

  Liying Lu, Tianke Zhang, Yunfei Liu, Xuangeng Chu, Yu Li

• EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation, ICCV, 2023, arxiv, pdf, cication: 52

  Ziqiao Peng, Haoyu Wu, Zhenbo Song, Hao Xu, Xiangyu Zhu, Jun He, Hongyan Liu, Zhaoxin Fan · (EmoTalk_release - psyai-net)

Datasets

• VGGHeads: A Large-Scale Synthetic Dataset for 3D Human Heads, arXiv, 2407.18245, arxiv, pdf, cication: -1

  Orest Kupyn, Eugene Khvedchenia, Christian Rupprecht

• MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation, arXiv, 2303.09797, arxiv, pdf, cication: -1

  Haozhe Wu, Jia Jia, Junliang Xing, Hongwei Xu, Xiangyuan Wang, Jelo Wang · (wuhaozhe.github)

Extra

• HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles, arXiv, 2312.11666, arxiv, pdf, cication: -1

  Vanessa Sklyarova, Egor Zakharov, Otmar Hilliges, Michael J. Black, Justus Thies