WO2002030171A3 - Facial animation of a personalized 3-d face model using a control mesh - Google Patents

Facial animation of a personalized 3-d face model using a control mesh Download PDF

Info

Publication number
WO2002030171A3
WO2002030171A3 PCT/IB2001/002363 IB0102363W WO0230171A3 WO 2002030171 A3 WO2002030171 A3 WO 2002030171A3 IB 0102363 W IB0102363 W IB 0102363W WO 0230171 A3 WO0230171 A3 WO 0230171A3
Authority
WO
WIPO (PCT)
Prior art keywords
face
face model
person
texture
images
Prior art date
Application number
PCT/IB2001/002363
Other languages
French (fr)
Other versions
WO2002030171A2 (en
Inventor
Tanju A Erdem
Original Assignee
Tanju A Erdem
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tanju A Erdem filed Critical Tanju A Erdem
Priority to EP01986569A priority Critical patent/EP1334470A2/en
Publication of WO2002030171A2 publication Critical patent/WO2002030171A2/en
Publication of WO2002030171A3 publication Critical patent/WO2002030171A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation

Abstract

A method for generating a 3-D model of a person's face is disclosed. The 3-D face model carries both the geometry (shape) and the texture (color) characteristics of the person's face. The shape of the face model is represented via a 3-D triangular mesh (geometry mesh), while the texture of the face model is represented via a 2-D composite image (texture image). The geometry mesh is obtained by deforming a predefined standard 3-D triangular mesh based on the dimensions and relative positions of the person's facial features, such as eyes, nose, ears, lips, chin, etc. The texture image is obtained by compositing a set of 2-D images of the person's face which are taken from particular directions such as front, right, left, etc. and modifying them along region boundaries to achieve seamless stitching of color on the 3-D face model. The directional images are taken while the mouth is closed and the eyes are open. In order to capture the color information of the facial regions that are not visible in the directional images, i.e., the inside of the mouth and the outside of the eyelids, additional 2-D images are also taken and included in the texture image.
PCT/IB2001/002363 2000-10-12 2001-10-09 Facial animation of a personalized 3-d face model using a control mesh WO2002030171A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP01986569A EP1334470A2 (en) 2000-10-12 2001-10-09 Facial animation of a personalized 3-d face model using a control mesh

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/689,186 US6664956B1 (en) 2000-10-12 2000-10-12 Method for generating a personalized 3-D face model
US09/689,186 2000-10-12

Publications (2)

Publication Number Publication Date
WO2002030171A2 WO2002030171A2 (en) 2002-04-18
WO2002030171A3 true WO2002030171A3 (en) 2002-10-31

Family

ID=24767388

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2001/002363 WO2002030171A2 (en) 2000-10-12 2001-10-09 Facial animation of a personalized 3-d face model using a control mesh

Country Status (3)

Country Link
US (1) US6664956B1 (en)
EP (1) EP1334470A2 (en)
WO (1) WO2002030171A2 (en)

Families Citing this family (80)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7920143B1 (en) * 1997-03-27 2011-04-05 At&T Intellectual Property Ii, L.P. Method for defining animation parameters for an animation definition interface
US6970591B1 (en) * 1999-11-25 2005-11-29 Canon Kabushiki Kaisha Image processing apparatus
US6807290B2 (en) * 2000-03-09 2004-10-19 Microsoft Corporation Rapid computer modeling of faces for animation
FR2811792B1 (en) * 2000-07-13 2002-12-06 France Telecom FACIAL ANIMATION PROCESS
US6950104B1 (en) * 2000-08-30 2005-09-27 Microsoft Corporation Methods and systems for animating facial features, and methods and systems for expression transformation
US6664956B1 (en) 2000-10-12 2003-12-16 Momentum Bilgisayar, Yazilim, Danismanlik, Ticaret A. S. Method for generating a personalized 3-D face model
US7127081B1 (en) 2000-10-12 2006-10-24 Momentum Bilgisayar, Yazilim, Danismanlik, Ticaret, A.S. Method for tracking motion of a face
US7020305B2 (en) * 2000-12-06 2006-03-28 Microsoft Corporation System and method providing improved head motion estimations for animation
US7016824B2 (en) * 2001-02-06 2006-03-21 Geometrix, Inc. Interactive try-on platform for eyeglasses
GB2382289B (en) 2001-09-28 2005-07-06 Canon Kk Method and apparatus for generating models of individuals
JP4082021B2 (en) * 2001-11-29 2008-04-30 コニカミノルタホールディングス株式会社 Texture image generation program, texture image data and data processing apparatus
GB2389498B (en) * 2002-04-30 2005-06-29 Canon Kk Method and apparatus for generating models of individuals
GB2389289B (en) * 2002-04-30 2005-09-14 Canon Kk Method and apparatus for generating models of individuals
JP2004046793A (en) * 2002-05-17 2004-02-12 Nintendo Co Ltd Image processing system
JP4357155B2 (en) * 2002-05-28 2009-11-04 株式会社セガ Animation image generation program
JP3780512B2 (en) * 2002-07-30 2006-05-31 株式会社光栄 PROGRAM, RECORDING MEDIUM, AND GAME DEVICE
JP2004252603A (en) * 2003-02-18 2004-09-09 Canon Inc Three-dimensional data processing method
US8363951B2 (en) * 2007-03-05 2013-01-29 DigitalOptics Corporation Europe Limited Face recognition training method and apparatus
KR100682889B1 (en) * 2003-08-29 2007-02-15 삼성전자주식회사 Method and Apparatus for image-based photorealistic 3D face modeling
US7285047B2 (en) * 2003-10-17 2007-10-23 Hewlett-Packard Development Company, L.P. Method and system for real-time rendering within a gaming environment
TWI220234B (en) * 2003-10-21 2004-08-11 Ind Tech Res Inst A method to simulate animated images for an object
US8326086B2 (en) * 2003-12-11 2012-12-04 Koninklijke Philips Electronics N.V. Elastic image registration
US20050149258A1 (en) * 2004-01-07 2005-07-07 Ullas Gargi Assisting navigation of digital content using a tangible medium
JP3648535B1 (en) * 2004-03-31 2005-05-18 コナミ株式会社 GAME DEVICE, COMPUTER CONTROL METHOD, AND PROGRAM
US7454039B2 (en) * 2004-07-12 2008-11-18 The Board Of Trustees Of The University Of Illinois Method of performing shape localization
US9578345B2 (en) 2005-03-31 2017-02-21 Euclid Discoveries, Llc Model-based video encoding and decoding
US9532069B2 (en) 2004-07-30 2016-12-27 Euclid Discoveries, Llc Video compression repository and model reuse
WO2008091483A2 (en) * 2007-01-23 2008-07-31 Euclid Discoveries, Llc Computer method and apparatus for processing image data
US9743078B2 (en) 2004-07-30 2017-08-22 Euclid Discoveries, Llc Standards-compliant model-based video encoding and decoding
US8077179B2 (en) * 2005-07-11 2011-12-13 Pandoodle Corp. System and method for creating animated video with personalized elements
US8139068B2 (en) * 2005-07-29 2012-03-20 Autodesk, Inc. Three-dimensional animation of soft tissue of characters using controls associated with a surface mesh
US9492750B2 (en) * 2005-07-29 2016-11-15 Pamela Leslie Barber Digital imaging method and apparatus
US7567251B2 (en) * 2006-01-10 2009-07-28 Sony Corporation Techniques for creating facial animation using a face mesh
US8411914B1 (en) * 2006-11-28 2013-04-02 The Charles Stark Draper Laboratory, Inc. Systems and methods for spatio-temporal analysis
US8452160B2 (en) * 2007-06-20 2013-05-28 Sony Online Entertainment Llc System and method for portrayal of object or character target features in an at least partially computer-generated video
KR100940862B1 (en) * 2007-12-17 2010-02-09 한국전자통신연구원 Head motion tracking method for 3d facial model animation from a video stream
US8244066B2 (en) * 2008-03-24 2012-08-14 The Aerospace Corporation Adaptive membrane shape deformation system
US9901829B2 (en) * 2008-07-15 2018-02-27 Pamela Barber Digital imaging method and apparatus
US8493286B1 (en) 2009-04-21 2013-07-23 Mark T. Agrama Facial movement measurement and stimulation apparatus and method
JP5463866B2 (en) * 2009-11-16 2014-04-09 ソニー株式会社 Image processing apparatus, image processing method, and program
US8692867B2 (en) * 2010-03-05 2014-04-08 DigitalOptics Corporation Europe Limited Object detection and rendering for wide field of view (WFOV) image acquisition systems
CN101853523B (en) * 2010-05-18 2012-01-04 南京大学 Method for adopting rough drawings to establish three-dimensional human face molds
WO2011149558A2 (en) 2010-05-28 2011-12-01 Abelow Daniel H Reality alternate
KR20120005587A (en) * 2010-07-09 2012-01-17 삼성전자주식회사 Method and apparatus for generating face animation in computer system
CN101958007B (en) * 2010-09-20 2012-05-23 南京大学 Three-dimensional animation posture modeling method by adopting sketch
KR101681538B1 (en) * 2010-10-20 2016-12-01 삼성전자주식회사 Image processing apparatus and method
US9082222B2 (en) 2011-01-18 2015-07-14 Disney Enterprises, Inc. Physical face cloning
US8723959B2 (en) 2011-03-31 2014-05-13 DigitalOptics Corporation Europe Limited Face and other object tracking in off-center peripheral regions for nonlinear lens geometries
US8947501B2 (en) * 2011-03-31 2015-02-03 Fotonation Limited Scene enhancements in off-center peripheral regions for nonlinear lens geometries
US8982180B2 (en) * 2011-03-31 2015-03-17 Fotonation Limited Face and other object detection and tracking in off-center peripheral regions for nonlinear lens geometries
US10509466B1 (en) * 2011-05-11 2019-12-17 Snap Inc. Headwear with computer and optical element for use therewith and systems utilizing same
WO2013008305A1 (en) * 2011-07-11 2013-01-17 トヨタ自動車株式会社 Eyelid detection device
US9236024B2 (en) 2011-12-06 2016-01-12 Glasses.Com Inc. Systems and methods for obtaining a pupillary distance measurement using a mobile computing device
EP2803963A4 (en) * 2012-01-12 2015-09-30 Mission Infrared Electro Optics Technology Co Ltd Thermal imagery device and normalized thermal imagery shooting method
US10230908B2 (en) * 2012-01-12 2019-03-12 Mission Infrared Electro Optics Technology Co., Ltd Thermal imaging device and thermal image photographing method
JP5620414B2 (en) * 2012-01-18 2014-11-05 株式会社スクウェア・エニックス Game device
US9408686B1 (en) 2012-01-20 2016-08-09 Conformis, Inc. Devices, systems and methods for manufacturing orthopedic implants
US8605972B2 (en) * 2012-03-02 2013-12-10 Sony Corporation Automatic image alignment
KR101927886B1 (en) 2012-03-19 2018-12-11 삼성전자주식회사 Image processing apparatus and method there of
US9286715B2 (en) 2012-05-23 2016-03-15 Glasses.Com Inc. Systems and methods for adjusting a virtual try-on
US9311746B2 (en) 2012-05-23 2016-04-12 Glasses.Com Inc. Systems and methods for generating a 3-D model of a virtual try-on product
US9483853B2 (en) 2012-05-23 2016-11-01 Glasses.Com Inc. Systems and methods to display rendered images
US8712737B1 (en) * 2012-07-20 2014-04-29 Google Inc. Use of physical deformation during scanning of an object to generate views of the object
CN102999929A (en) * 2012-11-08 2013-03-27 大连理工大学 Triangular gridding based human image face-lift processing method
US9304332B2 (en) 2013-08-22 2016-04-05 Bespoke, Inc. Method and system to create custom, user-specific eyewear
US10438631B2 (en) 2014-02-05 2019-10-08 Snap Inc. Method for real-time video processing involving retouching of an object in the video
US10091507B2 (en) 2014-03-10 2018-10-02 Euclid Discoveries, Llc Perceptual optimization for model-based video encoding
US10097851B2 (en) 2014-03-10 2018-10-09 Euclid Discoveries, Llc Perceptual optimization for model-based video encoding
US9621917B2 (en) 2014-03-10 2017-04-11 Euclid Discoveries, Llc Continuous block tracking for temporal prediction in video encoding
US9699123B2 (en) 2014-04-01 2017-07-04 Ditto Technologies, Inc. Methods, systems, and non-transitory machine-readable medium for incorporating a series of images resident on a user device into an existing web browser session
KR101635730B1 (en) * 2014-10-08 2016-07-20 한국과학기술연구원 Apparatus and method for generating montage, recording medium for performing the method
US9953247B2 (en) * 2015-01-29 2018-04-24 Samsung Electronics Co., Ltd. Method and apparatus for determining eye position information
US10116901B2 (en) * 2015-03-18 2018-10-30 Avatar Merger Sub II, LLC Background modification in video conferencing
US9906884B2 (en) * 2015-07-31 2018-02-27 The University Of North Carolina At Chapel Hill Methods, systems, and computer readable media for utilizing adaptive rectangular decomposition (ARD) to generate head-related transfer functions
US9959672B2 (en) * 2015-11-23 2018-05-01 Adobe Systems Incorporated Color-based dynamic sub-division to generate 3D mesh
CN106570822B (en) * 2016-10-25 2020-10-16 宇龙计算机通信科技(深圳)有限公司 Face mapping method and device
US11232617B2 (en) * 2018-01-11 2022-01-25 Pamela L. Barber Digital imaging method and apparatus
CN110060287B (en) * 2019-04-26 2021-06-15 北京迈格威科技有限公司 Face image nose shaping method and device
CN110136236B (en) * 2019-05-17 2022-11-29 腾讯科技（深圳）有限公司 Personalized face display method, device and equipment for three-dimensional character and storage medium
CN111709302B (en) * 2020-05-21 2023-08-25 青岛小鸟看看科技有限公司 Facial expression detection method and device, head-mounted display equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1998001830A1 (en) * 1996-07-05 1998-01-15 British Telecommunications Public Limited Company Image processing
WO1999006962A1 (en) * 1997-08-04 1999-02-11 Pixar Animation Studios Realistic surface simulation in computer animation
EP0926628A2 (en) * 1997-12-19 1999-06-30 Fujitsu Limited mpparatus and method for generating progressive polygon data

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4975960A (en) 1985-06-03 1990-12-04 Petajan Eric D Electronic facial tracking and detection system and method and apparatus for automated speech recognition
GB9019538D0 (en) 1990-09-07 1990-10-24 Philips Electronic Associated Tracking a moving object
US5645077A (en) 1994-06-16 1997-07-08 Massachusetts Institute Of Technology Inertial orientation tracker apparatus having automatic drift compensation for tracking human head and other similarly sized body
US6020892A (en) 1995-04-17 2000-02-01 Dillon; Kelly Process for producing and controlling animated facial representations
JP2894241B2 (en) 1995-04-21 1999-05-24 村田機械株式会社 Image recognition device
US5805745A (en) 1995-06-26 1998-09-08 Lucent Technologies Inc. Method for locating a subject's lips in a facial image
US5802220A (en) 1995-12-15 1998-09-01 Xerox Corporation Apparatus and method for tracking facial motion through a sequence of images
US5774591A (en) 1995-12-15 1998-06-30 Xerox Corporation Apparatus and method for recognizing facial expressions and facial gestures in a sequence of images
US5982909A (en) 1996-04-23 1999-11-09 Eastman Kodak Company Method for region tracking in an image sequence using a two-dimensional mesh
US5923337A (en) 1996-04-23 1999-07-13 Image Link Co., Ltd. Systems and methods for communicating through computer animated images
EP0912965A1 (en) 1996-07-26 1999-05-06 Mirage Technologies Apparatus and method for representation of expression in a tissue-like system
US5907626A (en) 1996-08-02 1999-05-25 Eastman Kodak Company Method for object tracking and mosaicing in an image sequence using a two-dimensional mesh
US5744953A (en) 1996-08-29 1998-04-28 Ascension Technology Corporation Magnetic motion tracker with transmitter placed on tracked object
US6028960A (en) 1996-09-20 2000-02-22 Lucent Technologies Inc. Face feature analysis for automatic lipreading and character animation
US6009210A (en) 1997-03-05 1999-12-28 Digital Equipment Corporation Hands-free interface to a virtual reality environment using head tracking
US6031539A (en) 1997-03-10 2000-02-29 Digital Equipment Corporation Facial image method and apparatus for semi-automatically mapping a face on to a wireframe topology
US6016148A (en) 1997-06-06 2000-01-18 Digital Equipment Corporation Automated mapping of facial images to animation wireframes topologies
US6037949A (en) 1997-08-04 2000-03-14 Pixar Animation Studios Texture mapping and other uses of scalar fields on subdivision surfaces in computer graphics and animation
US6047078A (en) 1997-10-03 2000-04-04 Digital Equipment Corporation Method for extracting a three-dimensional model using appearance-based constrained structure from motion
US6052132A (en) 1998-02-06 2000-04-18 Digital Equipment Corporation Technique for providing a computer generated face having coordinated eye and head movement
US6664956B1 (en) 2000-10-12 2003-12-16 Momentum Bilgisayar, Yazilim, Danismanlik, Ticaret A. S. Method for generating a personalized 3-D face model
US6731287B1 (en) 2000-10-12 2004-05-04 Momentum Bilgisayar, Yazilim, Danismanlik, Ticaret A.S. Method for animating a 3-D model of a face
WO2002031772A2 (en) 2000-10-13 2002-04-18 Erdem Tanju A Method for tracking motion of a face

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1998001830A1 (en) * 1996-07-05 1998-01-15 British Telecommunications Public Limited Company Image processing
WO1999006962A1 (en) * 1997-08-04 1999-02-11 Pixar Animation Studios Realistic surface simulation in computer animation
EP0926628A2 (en) * 1997-12-19 1999-06-30 Fujitsu Limited mpparatus and method for generating progressive polygon data

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LANDE C ET AL: "An MPEG-4 facial animation system driven by synthetic speech", MULTIMEDIA MODELING, 1998. MMM '98. PROCEEDINGS. 1998 LAUSANNE, SWITZERLAND 12-15 OCT. 1998, LOS ALAMITOS, CA, USA,IEEE COMPUT. SOC, US, 12 October 1998 (1998-10-12), pages 203 - 212, XP010309519, ISBN: 0-8186-8911-0 *
LAVAGETTO F ET AL: "THE FACIAL ANIMATION ENGINE: TOWARD A HIGH-LEVEL INTERFACE FOR THE DESIGN OF MPEG-4 COMPLIANT ANIMATED FACES", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, IEEE INC. NEW YORK, US, vol. 9, no. 2, March 1999 (1999-03-01), pages 277 - 289, XP000805532, ISSN: 1051-8215 *

Also Published As

Publication number Publication date
US6664956B1 (en) 2003-12-16
WO2002030171A2 (en) 2002-04-18
EP1334470A2 (en) 2003-08-13

Similar Documents

Publication Publication Date Title
WO2002030171A3 (en) Facial animation of a personalized 3-d face model using a control mesh
CN109410298B (en) Virtual model manufacturing method and expression changing method
CN101324961B (en) Human face portion three-dimensional picture pasting method in computer virtual world
CN100474876C (en) Image processing method and apparatus thereof
CN105955456A (en) Virtual reality and augmented reality fusion method, device and intelligent wearable equipment
KR100327541B1 (en) 3D facial modeling system and modeling method
CN107851299B (en) Information processing apparatus, information processing method, and program
US7567251B2 (en) Techniques for creating facial animation using a face mesh
US20140254939A1 (en) Apparatus and method for outputting information on facial expression
US20100079491A1 (en) Image compositing apparatus and method of controlling same
CN106774856A (en) Exchange method and interactive device based on lip reading
KR20040047623A (en) Method and apparatus for image processing
CN101795375B (en) Device for displaying projection and method for controlling projection
CN109035413B (en) Virtual fitting method and system for image deformation
JP2001522108A (en) Method and apparatus for displaying at least a part of a human body whose appearance has been changed
KR20060064553A (en) Method, apparatus, and computer program for processing image
Breton et al. FaceEngine a 3D facial animation engine for real time applications
CN107491165A (en) A kind of vr eyewear faces 3d images, plane picture capture and gesture capture systems
CN104408702B (en) A kind of image processing method and device
JP2534617B2 (en) Real-time recognition and synthesis method of human image
CN206066469U (en) A kind of portable telepresence interaction robot
KR20200024105A (en) Computer graphics synthesis system and method thereof
CN108762508A (en) A kind of human body and virtual thermal system system and method for experiencing cabin based on VR
JP4351023B2 (en) Image processing method and apparatus
JPH09305798A (en) Image display device

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): JP

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
AK Designated states

Kind code of ref document: A3

Designated state(s): JP

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

WWE Wipo information: entry into national phase

Ref document number: 2001986569

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2001986569

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: JP