Facial Landmarks and Expression Label Guided Photorealistic Facial Expression Synthesis
Facial expression manipulation plays an increasingly important role in the field of computer graphics and has been widely used in generating facial animations. However, it is still a very challenging task as it needs full understanding of the input face and very depending on the facial appearance. I...
Saved in:
| Published in | IEEE access Vol. 9; pp. 56292 - 56300 |
|---|---|
| Main Authors | , , |
| Format | Journal Article |
| Language | English |
| Published |
Piscataway
IEEE
2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | Facial expression manipulation plays an increasingly important role in the field of computer graphics and has been widely used in generating facial animations. However, it is still a very challenging task as it needs full understanding of the input face and very depending on the facial appearance. In this paper, we present an end-to-end generative adversarial network for facial expression synthesis. Given the facial landmarks and the expression label of a target image, our method automatically generates a corresponding expression facial image with the identity information and facial details well preserved. Both qualitative and quantitative experiments are conducted on the CK+ and Oulu-CASIA datasets. Experimental results show that our method has the compelling perceptual results even there exist large differences in facial shapes for unseen subjects. |
|---|---|
| AbstractList | Facial expression manipulation plays an increasingly important role in the field of computer graphics and has been widely used in generating facial animations. However, it is still a very challenging task as it needs full understanding of the input face and very depending on the facial appearance. In this paper, we present an end-to-end generative adversarial network for facial expression synthesis. Given the facial landmarks and the expression label of a target image, our method automatically generates a corresponding expression facial image with the identity information and facial details well preserved. Both qualitative and quantitative experiments are conducted on the CK+ and Oulu-CASIA datasets. Experimental results show that our method has the compelling perceptual results even there exist large differences in facial shapes for unseen subjects. |
| Author | Qi, Wenqian Li, Dejian Sun, Shouqian |
| Author_xml | – sequence: 1 givenname: Dejian orcidid: 0000-0001-9258-5437 surname: Li fullname: Li, Dejian organization: Zhejiang Key Laboratory of Design and Intelligence and Digital Creativity, School of Computer Science, Zhejiang University, Hangzhou, China – sequence: 2 givenname: Wenqian surname: Qi fullname: Qi, Wenqian organization: Zhejiang Key Laboratory of Design and Intelligence and Digital Creativity, School of Computer Science, Zhejiang University, Hangzhou, China – sequence: 3 givenname: Shouqian surname: Sun fullname: Sun, Shouqian email: ssq@zju.edu.cn organization: Zhejiang Key Laboratory of Design and Intelligence and Digital Creativity, School of Computer Science, Zhejiang University, Hangzhou, China |
| BookMark | eNp9UU1rGzEUFCWBpkl-QS4LPdvV08fu6hiMkwYMLbghR_FWetvI3a5caQ3Nv6_cdUvoobpoeG9mmMe8Y2djHImxG-BLAG4-3K5W6-12KbiApeSN4Lp5wy4E1GYhtazPXuG37DrnHS-vLSPdXLCnO3QBh2qDo_-O6VuuCqjWP_eJcg5xLIuOhur-EDz56vNznGIiHEKegqtO2lfs7cs4PVMO-Yqd9zhkuj79l-zxbv1l9XGx-XT_sLrdLJzi7bTwXW84SXDQQKM1FmRAGcE5dr32RCDQaWo9Aley7pxQvjMkagmgBIG8ZA-zr4-4s_sUyhEvNmKwvwcxfbWYStaBLPpOagGOeKuVUjWKHnXnta8b6HR_9Ho_e-1T_HGgPNldPKSxxLdCg26U5m1bWGZmuRRzTtRbFyacyvVTwjBY4PZYi51rscda7KmWopX_aP8k_r_qZlYFIvqrMNIYJbn8BVE7mng |
| CODEN | IAECCG |
| CitedBy_id | crossref_primary_10_1016_j_procs_2023_10_011 crossref_primary_10_1016_j_iswa_2022_200139 crossref_primary_10_1007_s41060_024_00683_x |
| Cites_doi | 10.1145/2010324.1964956 10.1109/CVPR42600.2020.00308 10.1016/j.cviu.2006.10.019 10.1109/TIP.2020.3048629 10.1109/CVPR.2017.578 10.1145/383259.383289 10.1145/1964921.1964955 10.1109/ACII.2017.8273626 10.1109/CVPR.2014.241 10.7551/mitpress/1140.001.0001 10.1109/ICCV.2015.411 10.1177/0023830909103181 10.1016/j.imavis.2011.07.002 10.1145/3240508.3240612 10.1109/ICCV.2017.244 10.1109/CVPR.2018.00917 10.1109/CVPR.2017.741 10.1109/CVPRW.2010.5543262 10.5244/C.29.41 10.1109/CVPR.2018.00068 10.1145/258734.258880 10.1145/1185657.1185856 10.1145/3240508.3240647 10.1111/1467-8659.t01-1-00712 10.1145/3065386 10.1109/ICCV.2017.267 10.1109/CVPR.2017.632 10.1145/1198555.1198589 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021 |
| DBID | 97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D DOA |
| DOI | 10.1109/ACCESS.2021.3072057 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE Xplore Open Access IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Engineered Materials Abstracts METADEX Technology Research Database Materials Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional DOAJ Open Access Full Text |
| DatabaseTitle | CrossRef Materials Research Database Engineered Materials Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace METADEX Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Materials Research Database |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: RIE name: IEEE/IET Electronic Library url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2169-3536 |
| EndPage | 56300 |
| ExternalDocumentID | oai_doaj_org_article_adb3521ce0854446a2fa5bd5d671b5f1 10_1109_ACCESS_2021_3072057 9399430 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Scientific Innovation 2030–Project of Artificial Intelligence in the Next Generation grantid: 2018AAA0100700 – fundername: Provincial Key Research and Development Plan of Zhejiang Province grantid: 2019C03137 |
| GroupedDBID | 0R~ 4.4 5VS 6IK 97E AAJGR ABAZT ABVLG ACGFS ADBBV AGSQL ALMA_UNASSIGNED_HOLDINGS BCNDV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL GROUPED_DOAJ IPLJI JAVBF KQ8 M43 M~E O9- OCL OK1 RIA RIE RNS AAYXX CITATION RIG 7SC 7SP 7SR 8BQ 8FD JG9 JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c408t-dbf90e31c171755a31c9149200abf5dee12ac5e8da10436bc24db9e2631142e13 |
| IEDL.DBID | RIE |
| ISSN | 2169-3536 |
| IngestDate | Wed Aug 27 01:29:41 EDT 2025 Mon Jun 30 04:18:01 EDT 2025 Tue Jul 01 04:03:26 EDT 2025 Thu Apr 24 23:00:27 EDT 2025 Wed Aug 27 02:30:53 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Language | English |
| License | https://creativecommons.org/licenses/by-nc-nd/4.0 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c408t-dbf90e31c171755a31c9149200abf5dee12ac5e8da10436bc24db9e2631142e13 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-9258-5437 |
| OpenAccessLink | https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/document/9399430 |
| PQID | 2515745088 |
| PQPubID | 4845423 |
| PageCount | 9 |
| ParticipantIDs | crossref_citationtrail_10_1109_ACCESS_2021_3072057 crossref_primary_10_1109_ACCESS_2021_3072057 doaj_primary_oai_doaj_org_article_adb3521ce0854446a2fa5bd5d671b5f1 proquest_journals_2515745088 ieee_primary_9399430 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 20210000 2021-00-00 20210101 2021-01-01 |
| PublicationDateYYYYMMDD | 2021-01-01 |
| PublicationDate_xml | – year: 2021 text: 20210000 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE access |
| PublicationTitleAbbrev | Access |
| PublicationYear | 2021 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref13 ref12 ref15 ref36 ref14 ref31 heusel (ref37) 2017 ref30 ref33 ref2 susskind (ref9) 2008 ref1 ref39 ref17 ref38 ref16 ding (ref11) 2017 yeh (ref19) 2016 larsen (ref25) 2016 wu (ref40) 2015; 4 kemelmacher-shlizerman (ref4) 2011; 30 mirza (ref23) 2014 ref24 ref20 ref41 qiao (ref34) 2018 ref21 ref28 simonyan (ref32) 2014 ref27 ref29 ref8 ref7 goodfellow (ref22) 2017 ref3 ref6 ref5 cheung (ref10) 2014 reed (ref18) 2014 kingma (ref26) 2013 |
| References_xml | – year: 2017 ident: ref11 article-title: ExprGAN: Facial expression editing with controllable expression intensity publication-title: arXiv 1709 03842 – year: 2018 ident: ref34 article-title: Geometry-contrastive GAN for facial expression transfer publication-title: arXiv 1802 01822 – start-page: 1558 year: 2016 ident: ref25 article-title: Autoencoding beyond pixels using a learned similarity metric publication-title: Proc Int Conf Mach Learn – year: 2014 ident: ref32 article-title: Very deep convolutional networks for large-scale image recognition publication-title: arXiv 1409 1556 – year: 2014 ident: ref10 article-title: Discovering hidden factors of variation in deep networks publication-title: arXiv 1412 6583 – volume: 30 start-page: 1 year: 2011 ident: ref4 article-title: Exploring photobios publication-title: ACM Trans Graph doi: 10.1145/2010324.1964956 – ident: ref39 doi: 10.1109/CVPR42600.2020.00308 – ident: ref1 doi: 10.1016/j.cviu.2006.10.019 – ident: ref38 doi: 10.1109/TIP.2020.3048629 – ident: ref15 doi: 10.1109/CVPR.2017.578 – ident: ref5 doi: 10.1145/383259.383289 – ident: ref7 doi: 10.1145/1964921.1964955 – ident: ref35 doi: 10.1109/ACII.2017.8273626 – ident: ref30 doi: 10.1109/CVPR.2014.241 – ident: ref2 doi: 10.7551/mitpress/1140.001.0001 – ident: ref8 doi: 10.1109/ICCV.2015.411 – year: 2017 ident: ref37 article-title: GANs trained by a two time-scale update rule converge to a local Nash equilibrium publication-title: arXiv 1706 08500 – volume: 4 year: 2015 ident: ref40 article-title: A light CNN for deep face representation with noisy labels publication-title: arXiv 1511 02683 – ident: ref6 doi: 10.1177/0023830909103181 – year: 2017 ident: ref22 article-title: NIPS 2016 tutorial: Generative adversarial networks publication-title: arXiv 1701 00160 – ident: ref29 doi: 10.1016/j.imavis.2011.07.002 – ident: ref21 doi: 10.1145/3240508.3240612 – ident: ref24 doi: 10.1109/ICCV.2017.244 – ident: ref13 doi: 10.1109/CVPR.2018.00917 – start-page: 1431 year: 2014 ident: ref18 article-title: Learning to disentangle factors of variation with manifold interaction publication-title: Proc Int Conf Mach Learn – year: 2013 ident: ref26 article-title: Auto-encoding variational Bayes publication-title: arXiv 1312 6114 – ident: ref20 doi: 10.1109/CVPR.2017.741 – ident: ref28 doi: 10.1109/CVPRW.2010.5543262 – ident: ref41 doi: 10.5244/C.29.41 – ident: ref36 doi: 10.1109/CVPR.2018.00068 – ident: ref3 doi: 10.1145/258734.258880 – year: 2016 ident: ref19 article-title: Semantic facial expression editing using autoencoded flow publication-title: arXiv 1611 09961 – ident: ref16 doi: 10.1145/1185657.1185856 – ident: ref33 doi: 10.1145/3240508.3240647 – ident: ref17 doi: 10.1111/1467-8659.t01-1-00712 – ident: ref31 doi: 10.1145/3065386 – ident: ref14 doi: 10.1109/ICCV.2017.267 – ident: ref27 doi: 10.1109/CVPR.2017.632 – ident: ref12 doi: 10.1145/1198555.1198589 – start-page: 421 year: 2008 ident: ref9 article-title: Generating facial expressions with deep belief nets publication-title: Affective Computing Emotion Modelling Synthesis and Recognition – year: 2014 ident: ref23 article-title: Conditional generative adversarial nets publication-title: arXiv 1411 1784 |
| SSID | ssj0000816957 |
| Score | 2.2136128 |
| Snippet | Facial expression manipulation plays an increasingly important role in the field of computer graphics and has been widely used in generating facial animations.... |
| SourceID | doaj proquest crossref ieee |
| SourceType | Open Website Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 56292 |
| SubjectTerms | Computer graphics Faces Facial expression synthesis Gallium nitride Generative adversarial networks Generators Shape Synthesis Target recognition Three-dimensional displays Two dimensional displays |
| SummonAdditionalLinks | – databaseName: DOAJ Open Access Full Text dbid: DOA link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3NS8MwFA-ykx5EneJ0Sg8erWu-uvY4x-YQFUGHu4V8FYezk3UD_e99abNREPTiLbRJ2rz3mvd7afJ7CF1oxiS4QRsmWULd0k0cShxlYUwBymmpicrcaeT7h3g0ZrcTPqml-nJ7wip64EpwHWkUYASsLWADBrGLJJnkynATd7HiWRn4gM-rBVPlHJzgOOVdTzOEo7TT6_dhRBAQEnwFdk0i55Bqrqhk7PcpVn7My6WzGe6hXY8Sg171dvtoy-YHaKfGHdhEL0PpVruDO5mbd7l4KwIoBINPv7E1hxvKzoKb1dRYEzy-ziG4to7tELoMfNta7aevHKBgMS0O0Xg4eO6PQp8lIdQsSpahUVkaWYo1hsiMcwmlFMIesH6pMm6sxURqbhMDOmA0Vpowo1JLYuqO0VpMj1Ajn-f2GAXa4q7UllCmFDMEK2pklqQmxkn5P7WFyFpgQnsKcZfJYibKUCJKRSVl4aQsvJRb6HLT6KNi0Pi9-rXTxKaqo78uL4BRCG8U4i-jaKGm0-OmkxRgGKNRC7XXehX-Uy0EADzeZQ6nnvzHo0_RthtOtUrTRo3lYmXPALcs1Xlpot9pbedX priority: 102 providerName: Directory of Open Access Journals |
| Title | Facial Landmarks and Expression Label Guided Photorealistic Facial Expression Synthesis |
| URI | https://ieeexplore.ieee.org/document/9399430 https://www.proquest.com/docview/2515745088 https://doaj.org/article/adb3521ce0854446a2fa5bd5d671b5f1 |
| Volume | 9 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT9wwEB5RTu2hpaVVtzyUA0eyxK9scoQVC6oAIbWo3Cw_JiqCZit2VwJ-PWPHG61KVfVmJbbl6Btnvhl7ZgD2nJSG1CDmVVOJ4Lopc8OKJi8FUTlnHLdNiEY-vyhPr-TXa3W9Bvt9LAwixstnOAzNeJbvp24RXGUHNWlTKchAf0WGWxer1ftTQgGJWo1SYiFW1AeH4zF9A5mAnA1JknkRVNCK8ok5-lNRlRd_4qheJu_gfLmw7lbJ7XAxt0P39EfOxv9d-Qa8TTwzO-wE4z2sYfsB3qxkH9yEHxMT_OXZmWn9L3N_O8uokR0_pKuxLb2weJedLG48-uzy55TMcwz5EmnKLI1d6f3tsSUyObuZfYSryfH38Wme6izkThbVPPe2qQsUzDGy7ZQy1KrJcKL9Y2yjPCLjximsPKEoRWkdl97WyEsRAnGRiU-w3k5b_AyZQzYyDrmQ1krPmRXeNFXtS1bFE9kB8CUA2qUk5KEWxp2OxkhR6w41HVDTCbUB7PeDfnc5OP7d_Sgg23cNCbTjA0JEp_2ojbdEPZlDopySTGLDG6OsV74cMasaNoDNgGI_SQJwANtLOdFps880UUQ1koHpfvn7qC14HRbYeW62YX1-v8Ad4jJzuxt9ALtRlJ8BFlbxgA |
| linkProvider | IEEE |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3db9MwED9N42HwABsD0bFBHnhcuvgrTR5HtVKgnZC2ib1Z_riIaSOd1lYC_nrOiRtVDE17sxLbcvQ753539t0BfHBSGlKDmBZVIYLrJk8Ny6o0F0TlnHHcViEaeXqajy_kl0t1uQGHXSwMIjaXz7Afms1Zvp-5ZXCVHZWkTaUgA_0J6X3F22itzqMSSkiUahBTC7GsPDoeDukryAjkrE-yzLOghNbUT5OlP5ZVufcvbhTM6AVMV0tr75Vc95cL23d__sna-Ni1b8PzyDST41Y0dmAD65fwbC3_4C58H5ngMU8mpvY_zd31PKFGcvIrXo6t6YXFm-TT8sqjT779mJGBjiFjIk2ZxLFrvc9-10Qn51fzV3AxOjkfjtNYaSF1MisWqbdVmaFgjpF1p5ShVkmmE-0gYyvlERk3TmHhCUcpcuu49LZEnosQiotMvIbNelbjG0gcsoFxyIW0VnrOrPCmKkqfs6I5k-0BXwGgXUxDHqph3OjGHMlK3aKmA2o6otaDw27QbZuF4-HuHwOyXdeQQrt5QIjouCO18ZbIJ3NIpFOSUWx4ZZT1yucDZlXFerAbUOwmiQD2YH8lJzpu97kmkqgGMnDdvf-Peg9b4_PpRE8-n359C0_DYls_zj5sLu6WeEDMZmHfNQL9FyOf89U |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Facial+Landmarks+and+Expression+Label+Guided+Photorealistic+Facial+Expression+Synthesis&rft.jtitle=IEEE+access&rft.au=Li%2C+Dejian&rft.au=Qi%2C+Wenqian&rft.au=Sun%2C+Shouqian&rft.date=2021&rft.pub=IEEE&rft.eissn=2169-3536&rft.volume=9&rft.spage=56292&rft.epage=56300&rft_id=info:doi/10.1109%2FACCESS.2021.3072057&rft.externalDocID=9399430 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2169-3536&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2169-3536&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2169-3536&client=summon |