Images

Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/50—Lighting effects
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/06—Ray-tracing
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T15/00—3D [Three Dimensional] image rendering
  • G06T15/10—Geometric effects
  • G06T15/40—Hidden part removal

Definitions

• Such a method and such a device relate to a method and a device for generating a two-dimensional representation of a three-dimensional world.
• the invention particularly relates to the generation of images using the method of ray tracing, which is known colloquially among persons skilled in the art as “ray tracing.”
• the present invention is based on the task of proposing a method as well as a device for implementing the method, with which the generation of a two-dimensional representation of a three-dimensional world is supposed to be simplified.
• resources of a computer are supposed to be utilized more efficiently than was the case with previously known methods.
• the geometrical description of a virtual three-dimensional world contains at least the following components:
• complex objects can in turn possess a tree-like data structure, which divides the complex object up spatially, or divides it into simpler partial objects, either spatially or logically.
• no differentiation is made between simple basic objects and complex objects such as composite objects or those having a tree-like structure. Both cases are referred to as “object.”
• objects can also possess a transformation.
• This transformation can describe an affine transformation of the corresponding object in the three-dimensional world, for example using a transformation matrix, and thereby changes the coordinates of the object.
• a sequence of instructions can also be specified for an object, in addition.
• This sequence of instructions can change the properties of the object. This includes the geometrical properties (in the case of a sphere, its radius, for example) and the material properties (its color, for example).
• the lighting situation in the virtual world can also be specified.
• the tree-like data structure is composed of the following components:
• KD trees which in turn are related to BSP trees, come closest to the present invention.
• the expansion to include placeholders in connection with ray tracing was discovered within the scope of the present invention.
• Another integral part of the invention is the compilation of the definition of objects. As a totality that supports and combines all possible properties equally, this definition is new.
• the aforementioned task is accomplished, according to the invention, according to claim 1 , according to which a ray tracing method is used.
• the ray tracing method means that in the case of its implementation, at least part of the geometrical description of the world is intersected with rays, whereby the two-dimensional representation is put together from intersection points of objects of the world with the rays. Furthermore, several rays are brought together into a packet of discrete rays. In this connection, an association of a data structure with each packet of discrete rays then takes place. A status-related assignment to each ray is stored in memory for this data structure. This status indicates at least whether the indicated ray participates in the operation that is being applied to the packet at this time.
• the method is characterized by a new technique for data reduction, without having to perform unnecessary calculations.
• individual rays are brought together into packets of rays, as was already described in Ingo Wald, Carsten Benthin, Markus Wagner, Phillip Slusallek: “Interactive Rendering with Coherent Ray-Tracing,” Computer Graphics Forum/Proceedings of the EUROGRAPHICS 2001, Manchester, United Kingdom, Sep. 3-7, 2001, http://graphics.cs.uni-sb.dde/Publications/2001/InteractiveRenderingWithCoherentRayTracing.pdf.
• a novel and inventive expansion as compared with the method described in the cited prior publication consists of the association of a data structure with each of these packets.
• this data structure contains the status of each ray of a packet.
• the individual work steps of the ray tracing method are structured in such a manner that they optimally support the new technique, and at the same time are particularly suitable for being implemented in terms of equipment technology.
• a packet can also be assigned to several work steps at the same time, however, in order to thereby achieve a better capacity utilization of the data processing system. For example, the processing of rays for which valid intersection points were already calculated in step E could continue with step F, while the calculation is continued with step B for the other rays. As soon as all the rays of the packet have then carried out the calculations in step F, the packet could then continue as a whole again, with the processing of step G.
• the rays generated in work step G are brought together in new packets by means of another work step.
• all the newly generated rays are collected, sorted and compiled into new packets of discrete rays.
• a new packet does not necessarily contain only rays that were generated on the basis of the calculation results of precisely one predecessor ray packet, and instead, the work step can involve collecting and resorting newly generated rays of several predecessor ray packets that are logically related, to combine them into new packets.
• traversal of the packets does not result in any data reduction as compared with traversal of the individual rays.
• claim 2 advantageously describes a work step that expands the method described in claim 1 , and brings newly generated rays together in packets in such a manner that a data reduction is achieved in the traversal of the packets, as compared with traversal of the individual rays.
• a multi-threading method is used, whereby a packet of rays is considered as a thread, in each instance.
• the multi-threading method is used within a work step in order to allow functional units to perform useful work while they are waiting for results from other functional units.
• each work step holds a supply of packets that are alternately worked off by the functional units of the work step.
• the method is configured in such a manner that the amount of the data of the three-dimensional world is managed in such a manner that only part of these data are held in the memory at a particular time.
• the method can be structured in such a manner that an independent work step is implemented for this, in which it is recorded what (partial) objects were accessed and how often. Furthermore, in this work step, the collected information can be used in order to manage the memory automatically and to load new parts of the three-dimensional world into the memory and remove them, as needed.
• a virtual world can therefore consist of a plurality of objects.
• the description of the virtual world can contain so many data that it requires very much memory space.
• memory space For example, in a data processing system, there are several memories of different sizes and speeds. Since large memories are generally significantly slower than smaller ones, but the speed of the memory plays an important role for the time required to work off the method, it is desirable to hold all of the data important for the method in a fast memory.
• the method records what objects or partial objects were read out when and how often, and loads new (partial) objects out of a slow memory into the fast memory as needed. If the fast memory is full, the method can find (partial) objects that are no longer required, on the basis of the bookkeeping, and delete them from the fast memory, so that room is created for the required data.
• the difference as compared with the variant explained above, in connection with claim 4 is that it is not entire objects or partial objects that are read from the slow memory or deleted in the fast memory, but instead the data of the virtual world are divided into blocks, and these blocks are read into the fast memory or deleted from it as needed.
• an object can be broken down into several blocks by means of this subdivision.
• Claim 5 relates to a device for implementing one of the aforementioned methods, whereby the individual work steps are represented by independent functional groups.
• the functional groups can be additionally equipped with management functions, so that several similar functional groups can be combined in a device, and thereby the performance capacity of the device can be increased.
• the functional groups In order to achieve the result that an increase in the performance capacity of the device can be achieved by adding similar functional groups, the functional groups must be disposed in a suitable manner, so that the required calculation steps are uniformly distributed over all of the available units.
• the functional groups can additionally use the pipelining method, in which several packets are in the method at the same time.
• the pipelining method individual packets are not calculated one after the other, but instead, several packets can be in the method sequence at the same time, in different work steps.
• packet 1 can be generated in work step A. While the calculations for packet 1 are afterwards being carried out in work step B, packet 2 could be generated in step A. Once the calculation of packet 1 in step B is finished, packet 1 goes to work step C, and packet 2 can be processed in step B, while packet 3 is generated in step A, etc.
• the pipelining method can also be used directly in connection with the multi-threading method.
• every work step is then given a number of threads that are processed alternately by the step. After the processing of one thread in a work step has been completed, this thread is passed on to the next work step, and this step than carries out this thread together with those that have already been assigned to it, one after the other.
• the two methods together allow a very great capacity utilization of the functional units of a device.
• Claim 6 relates to a device for implementing one of the aforementioned methods, whereby the individual method steps are brought together into functional groups, as follows:
• the functional groups are shown in FIG. 7 .
• the functional groups Slave, Traversal, and Intersection are structured in such a manner that they contain several sub-units, which perform calculations at the same time, and simultaneously keep a supply of packets available automatically, to which they apply the multi-threading method.
• the functional units are structured in such a manner that the performance capacity of the device can be increased by adding additional similar groups.
• the number of slave functional units used is always precisely equal to the RTC functional groups (which consist of Traversal, List, and Intersection functional units).
• the functional units T-SR, L-SR, and I-SR are structured in such a manner that they can produce the connections between all of the RTC functional groups and the memories.
• the Master functional unit is the highest controlling instance of the device. As input, it receives the parameters for the virtual camera (indicated with UI in the figure), and thereupon assigns descriptions of packets to one or more slave units. These perform the camera ray generation, using these descriptions, and send the calculated packets to the associated RTC unit.
• the slave unit receives the data of the ray/object intersections as a response (some time later), and then performs the calculations for the work steps F and G. If subsequent rays are calculated, these are again sent to the associated RTC unit. In their calculations, the slave units must read data out of a memory and also write them back there, if applicable.
• the memory accesses of all the slave units are coordinated by the MemInt unit, which is connected with one or more memory modules (indicated with RAM in the figure).
• the slave units write to a special memory (indicated with FB in the figure), using the MemInt unit; they can also read data out of this memory again.
• This special memory could additionally be connected with a functional group that is connected to a monitor, in order to display the calculated two-dimensional representation.
• the RTC functional group implements the working steps tree traversal, read-in of the tree leaf, intersecting all objects, and intersection evaluation. If a unit of the RTC group would like to load a datum out of the memory, it directs a corresponding request to the unit assigned to it, in other words to the T-SR unit, the L-SR unit, or the I-SR unit, respectively.
• All of the memory requests from all of the RTC groups come together at the T-cache, L-cache, and I-cache units. These units have a data structure in which the data or prior memory requests are noted. If a request is made and its response from the memory was already noted, this request is answered with the datum from the data structure. In this connection, it is advantageous that all requests only want to read data from the memory. As a result, the content of the data structure can simply be deleted as needed, without having to write it back to the memory. This is of particular interest if the calculation for a new image is starting and objects have changed in this connection.
• the M-SR functional unit passes all requests from T-cache, L-cache, and I-cache on to the MemCtrl unit.
• MemCtrl is the functional unit that is directly connected to the memory module(s) (in the figure: RAM).
• the MemCtrl functional group contains several mechanisms for avoiding possible waiting times that occur during access to the memory modules. These mechanisms include: address hashing, in order to distribute the requests uniformly among all the memory modules, and re-sorting of requests, so that waiting times caused by the internal structure of memory modules are avoided.
• the breakdown of the method into device components, as described here, has the advantage that no functional group has to access data that are stored in another unit. As a result, the data can be stored predominantly locally, and it is not necessary to shift large amounts of data among the units, thereby greatly simplifying the implementation in terms of equipment technology.
• T-SR, L-SR, I-SR functional groups would be to equip them with a mechanism that brings similar requests that are made by different units at the same time together into one memory request, and sends the response of the memory back to all the requesters.
• “at the same time” can also be interpreted as “within a short period of time.” This mechanism can further reduce the number of memory requests.
• T-cache, L-cache, and I-cache functional groups would be to equip them with lists in which all the requests that have already been passed on to the memory but whose result is not yet known are noted. If a request is now made that is already noted in the list, another note would simply be added to the list, but no request would be sent to the memory. If the response then comes from the memory, the datum would be reported as a response to all the requests, in accordance with the information in the list. This technique can further reduce the number of memory requests.
• a device such as those described above could additionally be equipped with a functional group that automatically calculates the tree-like data structure for a number of objects. In this way, the device could react automatically to moving objects, for example.
• FIG. 1 an example to illustrate the fundamentals of the ray tracing method
• FIG. 2 an example of implementation of the ray tracing method
• FIG. 3 a representation to illustrate the definitions of terms
• FIG. 4 a representation to explain the use of placeholders
• FIG. 5 an example of a geometry-changing sequence of instructions
• FIG. 6 the tree structure of a simple world, consisting of squares, triangles, and circles, and
• FIG. 7 a representation of functional groups in a device.
• the ray tracing method will be explained below.
• This can be used, for example, for the production of a two-dimensional representation, as follows: A paper divided with a pattern of small squares is held in front of the virtual camera and then a ray from the camera is sent through each of the small squares. If a ray hits an object, the corresponding small square is colored with the color of the object. Once this has taken place for all the small squares, the piece of paper contains a two-dimensional representation of what the virtual camera sees. If these small squares are made small enough, the representation loses its block-like character and becomes a high-resolution image.
• sending a ray into the world takes place by means of intersecting the ray with all the or individually selected objects of this world.
• an intersection of a ray with an object then either yields an intersection point on the surface of the object, or it does not.
• FIG. 2 shows the calculations for a ray R 1 that is sent into the virtual world by the virtual camera (CAM).
• the ray R 1 intersects the object O 1 (a sphere) at the point P 1 .
• all that would be checked is the color that the object O 1 possesses at the point P 1 , and then the corresponding pixel would be colored this color.
• the lighting situation at the point P 1 could be investigated.
• another ray L 1 would be sent from a light source LS to the point P 1 . If the ray L 1 does not intersect an object between LS and P 1 , there is no object that covers the light source, seen from the point P 1 , so that the light from LS arrives at the point P 1 and illuminates it.
• an object 04 lies in the virtual space, so that the point P 3 lies in shadow.
• the material properties of the intersected object could be considered more closely. Assuming that object O 1 has a reflective surface, in order to determine the color of the object O 1 at the point P 1 , it would have to be checked what object is reflected at the point P 1 . In this case, as well, another ray R 2 , now proceeding from point P 1 , would be sent into the virtual world.
• R 2 intersects the object O 2 at the point P 2 .
• the ray L 2 is calculated.
• the object O 2 is a pane of glass, so that in order to determine the color of the point P 2 , it must be determined what object can be seen through the pane of glass O 2 .
• the ray R 3 is sent into the virtual world, proceeding from the point P 2 .
• the intersection point that results is the point P 3 on the object O 3 .
• the lighting situation is also calculated for P 3 , in order to then be able to determine its color.
• the color for the pixel that belongs to ray R 1 is then composed of the weighted colors at the points P 1 , P 2 , and P 3 .
• the world consists of the objects O 1 , O 2 , O 3 , and O 4 , as well as the light source (LS) and the virtual camera (CAM).
• the ray R 1 it can be seen that the ray intersects the object O 1 , if it intersects anything at all.
• LS light source
• CAM virtual camera
• FIG. 3 shows an example for a data structure corresponding to the definitions of the introduction to the specification, on the left.
• a two-dimensional world was chosen for the example, which can be visualized as simply being a top view of a three-dimensional world. This world can be seen in the center of FIG. 3 .
• the node N 1 forms the root of the tree.
• the partition plane (here also indicated with N 1 ) described by the node N 1 is also evident in the top view of the world. Simply stated, the plane described by N 1 divides the world into a left half that contains the triangle, the sphere, and the placeholder (P 1 ), and a right half that contains a star. Analogously, N 2 and N 3 divide the world into additional halves.
• the ray R 1 is compared with the partition plane N 1 .
• it is determined that the ray crosses the partition plane from the left to the right in other words it would first like to examine the left half and then the right half of the world.
• the information is stored in memory that the right half might also be interesting, but the evaluation in the left half is continued there with the node N 2 , starting by way of the edge E 1 .
• it is then found that only the half with the leaf V 1 which contains two objects, a triangle and a circle, is interesting for the ray.
• the ray R 1 is intersected with these objects, and in this connection it is determined that the ray does not impact any of these objects.
• the evaluation proceeds with the noted right half of N 1 , in accordance with what was stored in memory.
• the evaluation is continued with the node N 3 , by way of the edge E 2 .
• the ray R 1 would first like to go to the left half, and then possibly into the right half.
• the leaf V 2 does not contain any objects, so that the ray does not have to be intersected with any objects. Instead, the further evaluation takes place with the noted right half. This is where the leaf V 3 is located, which contains a star. The intersection of R 1 with the star yields an intersection point, thereby ending the calculation for R 1 .
• the calculation for the ray R 2 again begins with the node N 1 . There, it is decided that only the right half is interesting for the ray. The node N 3 is located in the right half. A comparison between R 2 and the plane N 3 shows that only the left half is interesting for the ray. But since this half only contains the empty leaf V 2 , the calculation for the ray R 2 is therefore finished, without the ray intersecting with an object.
• intersection The method of procedure described, in which a tree-like data structure is run through, is also called traversal (English traversal).
• traversal English traversal
• intersection of an object with a ray is referred to in English as intersection.
• the tree node N 1 is loaded from a memory, and it is calculated for all the rays of the packet where, and if yes how, the partition plane N 1 is crossed.
• ray R 1 would first like to go into the left half and then into the right half of the world.
• Ray R 2 would only like to go into the right half.
• the decision for the packet from this work step looks as follows: It is noted in the statuses of the rays that ray R 1 would like to go into both halves and ray R 2 would only like to go into the right half.
• each of the nodes being considered has to be loaded from the memory for each ray. If, however, entire packets of rays are traversed, each node being considered by one of the rays is loaded from the memory only once for the entire packet.
• the ray R 1 considers the tree nodes and tree leaves ⁇ N 1 , N 2 , V 1 , N 3 , V 2 , V 3 ⁇ and the ray R 2 considers the tree nodes and tree leaves ⁇ N 1 , N 3 , V 2 ⁇ . If the rays are traversed individually, data are loaded from the memory nine times. If the rays are brought together in a packet, the packet only visits the tree nodes and tree leaves ⁇ N 1 , N 3 , V 1 , N 3 , V 2 , V 3 ⁇ and data are loaded from the memory only six times.
• the new type of data structure as compared with previously known methods fulfills the purpose of avoiding unnecessary calculations. Without this data structure, the objects triangle and circle in the leaf V 1 would also be intersected with the ray R 2 , because no information would be available for a packet as to what rays of the packet would like to participate in the current calculation step, and thereby every ray would have to participate in all the calculations. While it is true that this would not result in errors, because such unnecessary intersections do not yield any valid intersection points, it does result in unnecessary calculation steps.
• a Camera ray generation Generation of a packet of rays, which are supposed to be intersected with the geometry of the virtual world, proceeding from virtual camera. For example, a rectangular region of small squares on the page in front of the virtual camera could be combined in a group and all the rays that pass through these small squares could be brought together in a group.
• B Tree traversal In accordance with the spatial orientation of the rays, the tree data structure for the geometry of the three-dimensional world is run through, until a tree leaf or a placeholder is reached. For this purpose, tree nodes and placeholders are read out of a memory, and calculations are carried out. If a placeholder is reached, the corresponding sequence of instructions of the substitute object is carried out. In this connection, the placeholder is replaced with an object that can also be a complex one.
• placeholders particularly the technique of the evaluation of placeholders as needed, which is shown here, is new in the sector of ray tracing.
• the evaluation of placeholders as needed offers the possibility of specifying virtual worlds having millions of objects implicitly, by means of placeholders, and nevertheless requiring only little memory and computer effort for calculating the image, since only those parts of the virtual world for which the method has found out that they are relevant for the calculation of the image are included in the calculation.
• FIG. 6 a simple world having squares, circles, and a triangle is shown.
• a tree structure is supposed to be generated here, as well.
• no partition plane that divides the world into a left and a right half but does not at the same time divide an object into two halves can be determined.
• the plane divides the world into one half having squares and one half having circles, but also divides the triangle into two halves.
• An expansion of the invention would consist, for example, of keeping a list, for every packet, of the objects that were already intersected with the packet. If the references to objects are loaded from a tree leaf, a comparison is made as to whether the object was already intersected, and if so, the intersection with the same object is not performed again. In this connection, however, it must be noted that possibly some rays that participate in the current object/ray intersection did not participate in the object/ray intersection that was entered into the list, and therefore the intersection now has to be carried out for these particular rays, after all. Another advantage of this referencing is presented here: An object, for example a car, is specified and stored in the memory.
• two additional objects are defined, which function as containers, in each instance, and possess references to the car object and, in addition, a transformation matrix and a material-changing sequence of instructions. In this manner, it is possible to represent two cars of the same model, but with different colors and at different positions, without having to store the data of the car in the memory multiple times.
• a transformation was specified for one of the objects, this is applied before the intersection is calculated. If the object is a simple object without its own tree structure, the object is transformed and the intersection is calculated. Otherwise, the rays are transformed and thereby adapted to the relative coordinate system of the object, before traversal of the tree of this object begins with work step B. If it was calculated whether any, and if so, what intersection points exist between the rays and the object, then the results and the rays are transformed back into the absolute coordinate system of the world.
• a geometry-changing sequence of instructions is specified for an object, this is carried out before the intersection with the object is calculated.
• This geometry-changing sequence of instructions can be implemented, for example, in order to model water surfaces.
• a tree structure that contains an object with a geometry-changing sequence of instructions should consider the possible effects of this sequence of instructions, since otherwise errors of the representation could occur.
• An example is shown in FIG. 5 .
• a geometry-changing sequence of instructions was specified for a triangle, which shifts the corner point P to the point P′ and P′′, in accordance with clock time.
• the tree structure is now supposed to be such that the object is referenced in all the tree leaves in which it can maximally occur.
• the triangle should therefore be referenced in the left tree leaf, and the sphere, the rectangle, and the triangle should be referenced in the right tree leaf. If a ray R were only intersected with the right half, and the triangle were not referenced there, an incorrect intersection point would be calculated for this ray at a certain clock time, namely the one with the rectangle instead of the one with the triangle.
• E Intersection evaluation In this work step, an evaluation takes place as to whether valid intersection points exist for the packet of rays. For example, in the preceding work step D, a ray can have been intersected with several objects, and these intersections can also have several intersection points as their result. A valid intersection point, in this connection, would be the intersection point that lies closest to the origin of the ray. If there are still elements of the tree for which a ray intersection is possible, but which have not yet been examined, and the prior work step has not resulted in valid intersection points in a sufficient amount, then the method starts again at step B, including the previous results, otherwise it continues with step F.
• the material-changing sequence of instructions can also be used to make a marbled object out of a colorless object, for example.
• the pattern of the marble can be calculated by means of the sequence of instructions specified for the object, so that the pattern is only implicitly indicated with a mathematical formula, and is only calculated if a ray hits the object.
• step F If it is calculated in step F that one or more additional rays are supposed to be intersected with the geometry of the three-dimensional world, then corresponding packets of rays are calculated in this step, and the method starts again at step B. Otherwise, it starts again at step A, if additional camera rays still have to be generated for forming the two-dimensional representation.
• FIG. 4 Another example for the use of placeholders is shown in FIG. 4 .
• the world with the related tree structures can be seen, from left to right, in a time progression, as it is expanded or changed in the implementation of placeholder P 1 or P 2 , respectively.
• the virtual world comprises the buildings A and B.
• the camera is located in building A, and building B cannot be seen by the camera. Therefore it is sufficient to specify building B by means of a simple placeholder. If the virtual camera leaves building A and approaches building B, then the sequence of instructions for the placeholder B is carried out with the first ray that hits the placeholder for building B during traversal. This sequence of instructions could now specify that building B is loaded from a memory medium and is supposed to be exchanged for the placeholder for building B.

Abstract

Description

Claims (16)

Priority Applications (1)

Applications Claiming Priority (5)

Publications (1)

Family

ID=31502078

Family Applications (2)

Family Applications After (1)

Country Status (6)

Families Citing this family (60)

Citations (24)

• 2002
  • 2002-08-26 DE DE10239672A patent/DE10239672B4/en not_active Expired - Fee Related
• 2003
  • 2003-08-20 US US12/608,864 patent/USRE44347E1/en not_active Expired - Lifetime
  • 2003-08-20 JP JP2004531696A patent/JP4348728B2/en not_active Expired - Fee Related
  • 2003-08-20 EP EP03790730A patent/EP1532586A2/en not_active Withdrawn
  • 2003-08-20 WO PCT/DE2003/002801 patent/WO2004021274A2/en active Application Filing
  • 2003-08-20 AU AU2003266174A patent/AU2003266174A1/en not_active Abandoned
  • 2003-08-20 US US10/526,055 patent/US7289118B2/en not_active Expired - Lifetime

Patent Citations (26)

Non-Patent Citations (43)

Also Published As

Similar Documents

Legal Events