WO1996003717A1

WO1996003717A1 - Method and system for the placement of texture on three-dimensional objects

Info

Publication number: WO1996003717A1
Application number: PCT/US1995/009209
Authority: WO
Inventors: Peter C. Litwinowicz
Original assignee: Apple Computer, Inc.
Priority date: 1994-07-22
Filing date: 1995-07-21
Publication date: 1996-02-08
Also published as: US5995110A; AU3139295A

Abstract

Morphing techniques are employed to enable a user to locally deform a texture image on a three-dimensional object, and thereby align image features in a texture map with geometric features of the object. An affine transformation can be applied to the texture map to provide a rough fit of the texture image to the geometric model. The transformed texture image is placed on the geometric model and the user designates corresponding features on the texture image and the geometric model. Once the designations have been completed, the texture image is warped into a final, deformed image, which is then mapped onto the three-dimensional object. With this approach, the user can apply pre-existing images to three-dimensional models, without being required to repaint any portion of the image.

Description

METHOD AND SYSTEM FOR THE PLACEMENT OF TEXTURE ON THREE-DIMENSIONAL OBJECTS

BACKGROUND OF THE INVENTION

The present invention is directed to the computer generation of images, and more particularly to the placement of texture on three-dimensional objects. In the generation of images on a computer, it is desirable to place texture on three-dimensional objects, to enhance surface detail. For example, to generate an image of a globe, a map of the world can be placed on a sphere. Similarly, it may be desirable to place a two-dimensional image of a person from a scanned photograph onto a three-dimensional model of a human head. Generally speaking, the placement of texture from a two dimensional image onto a three-dimensional object involves mapping appearance values from the image to corresponding locations on the surface of the object. Typically, the appearance values represent color, although they can also pertain to other visual aspects, for example surface perturbations such as bumps or the like. If the texture image is specifically designed for the object, the mapping of the appearance values can be carried out in a straightforward manner and produce an acceptable result. Where the texture image and the three-dimensional object are created independently of one another, however, a direct mapping from the texture image onto the surface of the object may not result in the desired effect. For example, if a scanned photograph of an animal is mapped onto a three- dimensional model of that animal, the size and/or orientation of the animal in the photograph may not correspond with those of the model. In such a case, the texture image must be revised to fit onto the model. In one approach to this problem, the texture image is globally manipulated by applying user-designated offsets, rotation and scaling. Examples of this approach are described in Rogers, D. , Procedural Elements for Computer Graphics. McGraw Hill Book Co., 1985, Section 5-11, pp. 354- 355. The purpose of this global manipulation is to resize and reorient the texture image to provide a good fit with the surface of the model. The manipulated texture image is then applied to the three-dimensional model. This approach does not assure that all of the features in the texture image will be aligned with corresponding features on the three-dimensional model, however. For example, if a texture image of a person's face is manipulated so that, overall, the face fits onto a three-dimensional mask, individual features of the face, such as the eyes or the mouth, may not line up exactly with corresponding features on the mask. In such a case, the user may be required to manually repaint portions of the texture image, until the features match. This painting of the image is carried out in a separate two-dimensional window, rather than on the three-dimensional object itself. As a result, this procedure can be tedious, since the changes that need to be made are not always apparent to the user.

In an alternative approach, texture can be painted directly onto the surface of the three-dimensional object. An example of this approach is described in P. Hanrahan et al, "Direct WYSIWYG Painting and Texturing on 3D Shapes", Computer Graphics. Vol. 24, No. 4, August 1990, pp. 215-223. While this technique is useful, it requires painting skills on the part of the user. Furthermore, it does not facilitate the application of pre-existing textures to a new model.

Accordingly, it is desirable to provide a technique which allows users to apply previously scanned or painted images to three-dimensional models, without the need to repaint any portion of the image to align particular features in the texture and the model.

BRIEF STATEMENT OF THE INVENTION

In accordance with the present invention, image warping or "morphing" techniques are employed by a user to locally deform an image on a three- dimensional object, and thereby align features in a texture image with geometric features of the object. In the implementation of the invention, an affine transformation can be first applied to the texture map to provide a rough fit of the texture map to the geometric model. The transformed texture image is placed on the geometric model and the user then designates corresponding portions of the texture image and the geometric model. Once the designations have been completed, the transformed texture image is warped into a final, deformed image, which is then mapped onto the three-dimensional object.

With this approach, the user can apply pre-existing images to three- dimensional models, without being required to repaint any portion of the image. The user is not required to work on an untextured object to do so. Rather, the interaction between the user and the image is carried out by dragging texture in an interactive fashion.

Further features of the invention, as well as the advantages offered thereby, are explained in detail hereinafter with reference to specific embodiments of the invention illustrated in the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a block diagram for a general computer system of the type in which the present invention can be implemented; Figure 2 is an illustration of a three-dimensional object;

Figure 3 is an illustration of a texture image, comprising a picture of an animal;

Figure 4 is another illustration of the three-dimensional object, with a UV parameterization grid overlaid on its surface; Figure 5 is the UV parameterization grid for the texture image;

Figure 6 is an illustration of a direct mapping of the texture image of Figure 3 onto the object of Figure 2;

Figure 7 is an illustration of an affine transformed version of the texture image of Figure 3 mapped onto the surface of the three-dimensional object; Figure 8 is another illustration of the texture image of Figure 3, showing user-designated feature points;

Figure 9 is an illustration of the corresponding feature points on the three-dimensional surface;

Figure 10 is an illustration of the warped texture image; Figure 11 is an illustration of the warped texture image as mapped onto the three-dimensional surface; and

Figures 12a-12d are illustrations of the steps in a warping technique based upon Delaunay triangulation.

DETAILED DESCRIPTION

The present invention is directed to the generation of images in a computer, for display on a display device, such as a monitor, and/or printing in a document. While the particular hardware components of a computer system do not form a part of the invention itself, they are briefly described herein to provide a thorough understanding of the manner in which the features of the invention cooperate with the components of a computer system to produce the desired results.

Referring to Figure 1, the computer system includes a computer 10 having a variety of external peripheral devices 12 connected thereto. The computer 10 includes a central processing unit 14 and associated memory. This memory generally includes a main memory which is typically implemented in the form of a random access memory 16, a static memory that can comprise a read only memory 18, and a permanent storage device, such as a magnetic or optical disk 20. The CPU 14 communicates with each of these forms of memory through an internal bus 22. The peripheral devices 12 include a data entry device such as a keyboard 24, and a pointing or cursor control device 26 such as a mouse, trackball, pen or the like. A display device 28, such as a CRT monitor or an LCD screen, provides a visual display of the information that is being processed within the computer, for example the contents of a document or a computer-generated image. A hard copy of this information can be provided through a printer 30, or similar such device. Each of these external peripheral devices communicates with the CPU 14 by means of one or more input/output ports 32 on the computer.

To facilitate an understanding of the principles which underlie the present invention, it will be described hereinafter with reference to a particular example in which a texture image of a cat's face is placed on a three- dimensional model of a cat's head. An example of a suitable three-dimensional model is depicted in Figure 2, and the two-dimensional texture image is shown in Figure 3. In this particular example, the texture image of Figure 3 is a scanned photograph. It will be appreciated, however, that images derived from other suitable sources, such as those generated with painting and drawing programs, can also be used.

Each of the three-dimensional surface and the texture image can be characterized by UV parameters. In essence, locations on the three-dimensional surface and within the texture image are specified according to their position in a UV coordinate space. Figure 4 illustrates the three-dimensional object of Figure 2 with a UV coordinate grid overlaid on its surface. Figure 5 illustrates the UV coordinate grid for the texture image of Figure 3. Each location on the surface of the three-dimensional object and in the texture image can be represented by three parameters, namely its U coordinate, its V coordinate and a display value. For the three-dimensional object, its display value can be an illumination brightness that represents the light and shaded portions of the surface, as determined by a light source located at an arbitrary position. If it is assumed that the light source remains fixed relative to the object, these values can be precomputed and stored in a buffer contained within the memory 16 of the computer system.

For the texture image, the display values can represent texture color or other texture parameters, such as displacement vectors or normal surface perturbations. In the following description, only color will be referred to as the display value, although it will be appreciated that other characteristics can be taken into account as well. Again, these display values can be stored in a memory buffer, along with the associated U and V coordinate values.

To apply the texture image to the three-dimensional surface, the display value, e.g. color, for each U,V location in the texture image is mapped to a corresponding U,V location in the coordinate space of the three-dimensional model. The resulting display value is determined by multiplying the illumination brightness for the object surface by the texture color for the texture image. These values are used to control the individual pixels in a display device, e.g., the computer monitor 28, for display of the resulting image. Figure 6 illustrates the result of a direct one-to-one mapping of the texture image of Figure 3 onto the surface model of Figure 2. As can be seen, the image of the cat's face does not fit properly on the model. This is due to the fact that the texture image, i.e. the photograph of the cat, was created independently of the three-dimensional model. To correct for differences in the size and/or shape of the texture image relative to the geometric model in this type of situation, an affine transformation can be applied to the texture image. In essence, an affine transformation comprises any combination of rotation, scaling and translation of the texture image. In the specific example illustrated in Figs. 2 and 3, it is desirable to enlarge and rotate the image so that the area of the face corresponds to the size and position of the object. To do so, the user can place the texture image on the object and interactively designate the appropriate scaling, rotation and translation that cause the desired portion of the texture image to fit into the corresponding portion of the object, as described in the previously cited reference by D. Rogers.

When this transformed image is applied to the three-dimensional model, the results are as shown in Figure 7. As can be seen therefrom, the overall size and orientation of the texture image corresponds to that of the three- dimensional model. However, individual features within the image do not match the corresponding features in the model. For example, the cat's nose is not shown at the proper location on the object. Similarly, the shapes of the ears do not match those on the object.

In the past, in order to correct for misalignment between the texture image and the three-dimensional surface, it was necessary to repaint portions of the texture image. For example, using a painting program, the position of the cat's nose could be moved to the left, until it coincided with that of the object. Similarly, the shapes of the ears could be changed with the painting program. In accordance with the present invention, however, the need for painting to align features of the texture image with those on the three-dimensional surface can be avoided, by using image warping techniques of the type conventionally applied to two-dimensional images. To do so in the context of the invention, the user designates feature on the original texture image. This can be done with the cursor control device 26 while the image is displayed on the monitor 28. These features can be defined by individual points, lines, splines or regions of the texture image. In the following description, features are designated by points, although it will be appreciated that other delineations of the image can be employed for the same purpose.

Representative feature points for the texture image are illustrated in Figure 8. For example, the feature points could comprise the corners of the eyes and mouth, the edges of the ears, and the centers of the nose and forehead. Corresponding feature points are also designated in the coordinate space for the three-dimensional surface, as shown in Figure 9. Preferably, the texture image and the three-dimensional object are shown side-by-side on the display monitor, to facilitate the user's designation of corresponding features on each. In the example of Figure 9, the object is shown with the af fine- transformed texture image displayed on its surface while the user designates points on the object. If the texture image obscures features on the object, however, it may be desirable to designate the points on the object without the image being displayed thereon. Most preferably, the user can be provided with an option to toggle between the two views, namely the object with the texture image on its surface and without the texture image. Such toggling can be carried out through actuation of a designated key on the keyboard 24, for example.

The designated points on the three-dimensional object represent the amount of displacement that is needed for the corresponding points in the texture image. For example, the user might designate U,V position (1,1) on the texture image, and location (2,3) on the three-dimensional object. These designations indicate that the display value at position (1,1) in the texture image must be displaced one unit in the U direction and two units in the V direction. After the corresponding points are designated on both the texture image and the three-dimensional surface in this manner, the original texture image is warped. In the warping process, the designated points on the texture image comprise source locations, and the corresponding designated points on the object form the respective destination locations. The warped two-dimensional image for the examples of Figures 2 and 3 is illustrated in Figure 10. This warped image can be mapped onto the surface of the three-dimensional object, with the final result as depicted in Figure 11. Preferably, the texture image is warped as each pair of corresponding points is respectively designated on the image and the object. In this manner, the user is provided instantaneous, interactive feedback for each designation as it occurs.

Any suitable warping technique can be employed to conform the texture image to the three-dimensional surface. In one implementation of the invention, Delaunay triangulation can be used to interpolate the displacement of the pixels within the image. An example of this type of interpolation is illustrated with reference to Figs. 12a-12d. Figure 12a illustrates a picture of a checkerboard, which defines a texture pattern. Six feature points are defined on this pattern, namely the four corners and two points within the checkerboard, respectively depicted by white and black dots. The Delaunay triangulation can be calculated for these six feature points, and is illustrated in Figure 12b. Reference is made to F. Preparata et al, "Computational Geometry, an Introduction", Springer Verlag, 1985, for a description of the manner in which the Delaunay triangulation is calculated.

The user can specify that the two internal feature points be moved, as indicated by the arrows in Figure 12a. Their destination positions are depicted in Figure 12c. The image is then warped using Delaunay triangulation, for example, as described in Wolberg, G., Digital Image Warping. 1990, IEEE Computer Society Press. The resulting warped image is depicted in Figure 12d. The Delaunay triangulation provides a rather simple interpolate for the waφing technique. As such, it may be suitable for systems where the available processing power is limited. In other environments, where more computing power is available, it may be desirable to use other waφing techniques which provide a smoother result. Examples of smoother waφing techniques are described in the foregoing article by G. Wolberg, as well as in Litwinowicz, P. et al, "Animating Images With Drawings", SIGGRAPH 94 Proceedings.

From the foregoing, it can be seen that a combination of UV image buffering and image deformation make it possible to waφ a texture image directly onto the surface of a three-dimensional object. In practice, the user first designates a feature on the texture image. The user then designates a corresponding feature on the object itself, rather than moving the feature in the image plane as in traditional waφing applications. When the user designates the feature on the object, the displacement coordinate for the corresponding feature in the texture image is determined, and the image is waφed before being mapped onto the object. In practice, the user can drag a feature around on the surface of the object, and see the resulting texture mapped onto the object at interactive speeds.

It will be appreciated by those of ordinary skill in the art that the present invention can be embodied in other forms without departing from the spirit or essential characteristics thereof. For example, waφing techniques other than the Delaunay triangulation approach described above can be employed. Similarly, it is not necessary to apply an affine transformation to the texture image prior to waφing. Rather, the user can designate corresponding features along the edge of the image and the boundary of the object, for example, to provide an initial fit of the image on the object. The scope of the invention, therefore, is indicated by the appended claims, rather than the foregoing description, and all changes and equivalents that come within their scope are intended to embraced therein.

Claims

WHAT IS CLAIMED IS:

1. A method for interactively placing texture on a geometric surface, comprising the steps of: defining a texture image and a geometric surface each within a respective coordinate space; designating corresponding features on each of the texture image and the geometric surface; waφing the texture image so that the designated features on the image are located at the same positions within the coordinate space of the image as the corresponding designated features are located in the coordinate space for the geometric surface; mapping the waφed texture image onto the geometric surface; and controlling a display device in accordance with the mapped image to display a resulting three-dimensional image.

2. The method of claim 1 wherein each coordinate point within the geometric surface has an illumination value and each coordinate point within the texture image has a display value, and wherein said step of mapping comprises multiplying the display value for a coordinate point of the texture image by the illumination value for the corresponding coordinate point of the geometric surface.

3. The method of claim 2 wherein said display value represents color.

4. The method of claim 2 wherein said display value represents a displacement vector.

5. The method of claim 2 wherein said display value represents a normal surface perturbation.

6. The method of claim 1 wherein said waφing step employs Delaunay triangulation to inteφolate displacements between corresponding designated points on the texture image and the geometric surface.

7. The method of claim 1 further including the step of globally transforming the texture image to fit it to the geometric surface prior to designating the corresponding features on the texture image and the geometric surface.

8. The method of claim 1 wherein said designated features comprise points on the texture image and the object.

9. The method of claim 1 wherein said designated features comprise lines on the texture image and the object.

10. The method of claim 1 wherein said designated features comprise regions of the texture image and the object.

11. A system for generating an image composed of texture placed on the surface of a geometric object, comprising: means for storing a parametric representation of the geometric object; means for storing a parametric representation of a texture image; means for designating corresponding features on each of the geometric object and the texture image; means for determining a displacement value for each pair of corresponding features respectively located on the geometric object and the texture image; means for waφing the texture image in accordance with said displacement values; means for mapping the waφed texture image on the surface of the geometric object; and means for displaying the mapped image.

12. The system of claim 11 wherein the parametric representation of the geometric object comprises position coordinates with associated illumination values, and the parametric representation of the texture image comprises position coordinates with associated display values.

13. The system of claim 12 wherein said display values represent color.

14. The system of claim 12 wherein said display values represent displacement vectors.

15. The system of claim 12 wherein said display values represent normal surface perturbations.

16. The system of claim 12 wherein said mapping means includes means for multiplying the display value for a coordinate point of the texture image by the illumination value for a corresponding coordinate point of the geometric object.

17. The system of claim 11 wherein said designated features comprise points on the texture image and the object.

18. The system of claim 11 wherein said designated features comprise lines on the texture image and the object.

19. The system of claim 11 wherein said designated features comprise regions of the texture image and the object.