WO2012083960A1

WO2012083960A1 - System and method for scanning objects being modified

Info

Publication number: WO2012083960A1
Application number: PCT/DK2011/050496
Authority: WO
Inventors: Rune Fisker; Henrik ÖJELUND; Kristian Evers HANSEN
Original assignee: 3Shape A/S
Priority date: 2010-12-22
Filing date: 2011-12-19
Publication date: 2012-06-28

Abstract

The present invention relates to a system and a method of scanning an object being modified comprising the steps of: - providing a first 3D representation of the object prior to modification, - obtaining at least a second 3D representation by scanning at least a modified part of the object after modification, - aligning said first and second 3D representations, and - replacing a first subset of the first 3D representation with data from the second 3D representation, said first subset substantially corresponding to the modified part, thereby updating the first 3D representation to represent the modified object. In particular the invention may be applied within the dental field.

Description

System and method for scanning objects being modified

The present invention relates to a system and a method for scanning an object where a part of the object is being modified or has been modified. In particular the invention may be applied within the dental field.

Background of the invention

WO 2010/105628 by the same applicant disclosed a method for planning, visualizing and designing future dental restorative work based on the 3D models of the teeth prior to the normal preparatory work. Because the dental restoration CAD model is created based on a model of the teeth before preparation, the dentist is able to visualize possible future dental restorative work for the patient even before the dentist has started preparing the patient's teeth. However, the normal procedure is still that the dental restoration CAD model is designed based on a model of the prepared teeth. The present invention provides further advantages of scanning the teeth before preparation. The normal procedure of a dental restoration, such as the application of a crown, is that the patients tooth is prepared by removing material in order to constitute a solid base when the customized crown is applied. After the preparation an impression of the prepared tooth and the surrounding teeth is provided, typically the impression covers all the teeth. Subsequently a crown can be designed digitally by 3D scanning the impression or the crown can be designed from a gypsum model based on the impression.

Summary of the invention In general it can be a source of frustration when having a complete model, image or representation, 2D or 3D, of an object and subsequently the object is modified, possibly only marginally modified. The representation will then not any longer constitute a true representation of the present status of the (modified) object and a new model, image or representation must be obtained. One object of the invention is therefore to provide a better method for obtaining a representation of an object being modified. In other situations it may be difficult to obtain a true representation of the object after modification, for example within the dental field. Making impressions or intra oral scanning after preparation of the teeth can be one of the most challenging phases of a complex restoration. There are many problems that can lead to a frustrating experience, such as: margin location; soft tissue quality; location of the teeth to be impressed; patient's inability to open the mouth sufficiently; excess salivary flow; and a large tongue. It can be difficult to gain control of the environment and soft tissue before attempting the impression. A further object of the invention is therefore to ease the process for obtaining a true representation of a set of teeth after preparation.

A first embodiment of the invention relates to a method of scanning an object being modified comprising the steps of:

- providing a first 3D representation of the object prior to modification,

- obtaining at least a second 3D representation by scanning at least a modified part of the object after modification,

- aligning said first and second 3D representations, and

- replacing a first subset of the first 3D representation with data from the second 3D representation, said first subset substantially corresponding to the modified part, thereby updating the first 3D representation to represent the modified object.

In the preferred embodiment of the invention the procedure of aligning and/or the procedure of replacing is provided automatically. Further, the first 3D representation, the second 3D representation and the updated first 3D representation are preferably substantially continuous surface representations.

A second embodiment of the invention relates to a method of scanning an object being modified comprising the steps of:

- providing a first 3D representation of the object prior to modification,

- selecting a first subset of the first 3D representation after modification of the object, said first subset substantially corresponding to a modified part of the object,

- deleting / removing (data corresponding to) said first subset from the first 3D representation,

- obtaining at least a second 3D representation by scanning the modified part of the object after modification,

- aligning said first and second 3D representations, and

- automatically replacing the first subset of the first 3D representation with data from the second 3D representation, said first subset substantially corresponding to the modified part, thereby updating the first 3D representation to represent the modified object.

The first 3D representation (being a representation of the object prior to any modifications) may be predetermined and may already be available. Otherwise the first 3D representation can be obtained by scanning the object prior to modification. The first 3D representation can be seen as the "reference" or the original representation of the object and may in the following be referred to as the reference 3D representation, the original 3D representation or the reference 3D model or the original 3D model. The second 3D representation is provided by scanning of the modified object and typically the second 3D representation is a representation of only a part of the object, namely the modified part of the object. However, preferably more than the modified part of the object is scanned because if only the modified part was scanned it would be difficult to combine the data representing the original and the modified object. Thus, preferably the modified part and part of the adjacent surroundings of the modified part of the object is scanned to provide scan data representing areas that are common to the original object and the modified object.

A 3D representation, aka a 3D digital representation, can be either point clouds, surface, such as faceted and/or meshed, or volumetric. A 3D model or 3D digital model can be generated from a 3D representation. Faceted/meshed representations are preferred over point clouds, but faceted/meshed representations can be generated from point clouds, for example by triangulation. Volumetric models can be obtained with a scanner applying penetrating radiation, such as CT scanners.

Registration or alignment of e.g. images, 3D representations, 3D scans or 3D models is the process of transforming different sets of data into one coordinate system. Data may be multiple photographs, scans, data from different sensors, from different times, or from different viewpoints. Registration is necessary in order to be able to compare or integrate the data obtained from these different measurements.

A dental restoration CAD model is a virtual computer model of a restoration. CAD models are created in a software program and can be based on one or more 3D models or one or more 3D representations of the patient's teeth. Thus, whereas a 3D model is typically a digital representation of a physical object, a CAD model is a virtual digital model, however possibly at least partly comprising a digital representation of at least a part of a physical object. A dental restoration is a classical fixed restoration such as inlays/onlays, veneers, crowns, bridges, implant-retained structures etc, but by analogy also removable restorations such as dentures. A dental restoration requires dental restorative work.

A retraction cord is placed around the gum line during a crown procedure because it eliminates blood from surrounding the tooth during the preparation. This allows the dentist to see the margins of the tooth. A patient is the person for whom a restoration is designed. There may be medical indications for dental treatment of this patient, but also cosmetic considerations can be a relevant motivation for having a dental restoration designed. Scanning of the object to obtain the 3D representations, i.e. 3D scanning, can be performed by a number of methods and by means of many commercially available 3D scanner systems. Scanning may for example be provided intra orally, scanning an impression of a set of teeth and/or the antagonist, scanning a cast of a set of teeth and/or the antagonist, CT scanning and/or the like scanning methods.

There are several commercial systems available for obtaining 3D representations of teeth, e.g. from 3Shape, Cadent and 3M. Among these are intra-oral scanners, e.g. 3Shape Trios^®, and scanners for dental impressions or casts thereof, e.g., 3Shape D640, D700 and D710. Scanners can for example be optical scanners using for example laser, structured light etc.. Optical scanners generally obtain a 3D digital model of an object's surface. While this model describes geometry, it does not differentiate between any materials or sub-objects that make up the surface. Potentially, scanners with penetrating radiation such as (cone beam) CT scanners, from e.g. Imaging Science International's i-CAT, Kodak/lmtec's lluma, can be used. They have the advantage of providing volumetric models showing also decay inside the teeth, while disadvantages include concerns about radiation dose or high price of treatment. The idea behind the present invention is that if only a part of a scanned object has been modified it is not necessary to rescan the whole object. By means of the present invention it is only necessary to scan the modified part of the object and subsequently update the original representation of the object with data representing the modified part. The first subset is preferably the part of the first 3D representation being replaced with new data. This first subset may be a single closed subset of the first 3D representation. The first subset may however also be a plurality of closed subsets of the first 3D representation where these closed subsets are mutually interconnected or they are not mutually interconnected.

Image registration and adaptive scanning is known in the art. Image registration is the process of transforming different sets of data, e.g. from two or more images, e.g. target and reference images, into one coordinate system. Typically a correspondence between a number of points in the images is established. A transformation may then be determined to map the target image to the reference image, thereby establishing a point-by-point correspondence between the reference and target images. The image resulting from the registration is thus a combination of the single images which may result in an improved signal to noise ratio of the resulting image.

In adaptive 3D scanning (known from e.g. WO 2006/007855) the coverage of a scan is evaluated to perform subsequent scans in order to fill out missing parts of the model which was the result of the first scan. Thus, in adaptive scanning a reference model is analyzed for lack of coverage and subsequent scans are supposed to fill out and close holes in the reference model. The present invention differs from these known techniques in that there are typically no areas with lacking coverage in the reference model, and if there are areas with lacking coverage these areas are not interesting, and the purpose of the present invention is not to improve the signal to noise ratio of the resulting model. The purpose of the present invention is not to compare models of different objects for quality control or material testing. Instead the present invention relates to a method for efficiently updating a 3D representation of an object where only a part of the object has been modified. And instead of rescanning the entire object the present invention provides a solution where only the modified part is scanned and subsequently inserted in the original reference model. The crucial step of the present invention is thus the replacement of data in the reference model with data representing the modified part of the object. This replacement may be provided automatically, preferably implemented in software.

A modification of the object may be a single modification, e.g. a single tooth has been modified, it may be a plurality of modifications adjacent to each other, i.e. modification of a plurality of adjacent teeth. The modification of the object may also be a plurality of discrete modifications, i.e. modifications at different locations of the object and these locations are not necessarily adjacent to each other, e.g. modification of three non-adjacent teeth in the lower jaw. These situations can be handled differently. In one embodiment of the invention the step of obtaining at least a second 3D representation by scanning a modified part of the object after modification may be repeated at least once. Thereby a plurality of (second) 3D representations is obtained prior to updating the first 3D representation, where said plurality of (second) 3D representations represent different modified parts of the modified object, such as discrete non-adjacent modified parts of the object. Thus, in one embodiment of the invention "the second 3D representation" is actually a plurality of 3D representations of modified parts of the modified object. In a further embodiment of the invention the step of obtaining at least a second 3D representation by scanning a modified part of the object after modification is repeated for each new modification of the object. I.e. once a modification is executed this newly modified part is subsequently scanned. A new modification is then executed and then scanned. This may be further repeated thereby obtaining a 3D representation of each modification prior to updating the first 3D representation.

In a further embodiment of the invention a modified part of the object is divided into sections and each section is scanned to obtain a 3D representation of said section that is subsequently aligned with and replaced into the first 3D representation, i.e. the method according to the invention is executed for each section. These sections are not necessarily discrete and the sections may well constitute a mutually interconnected part of the object. However it may be advantageous to only scan and update the first 3D representation for one section at a time, i.e. the first 3D representation is updated with data corresponding to one section before continuing with the following section. One section may e.g. correspond to one tooth. In a further embodiment of the invention the method is executed for each modification, i.e. for each modification the modified part is scanned to obtain the second 3D representation which is subsequently aligned with and then replaced into the first 3D representation. Thereby the original 3D representation is sequentially updated for each new modification.

The steps of the methods as specified in the present invention are not necessarily executed in the stated consecutive order. As an example the aligning procedure of the first and second 3D representations may be carried out during the scanning of the modified part. In a further embodiment of the invention the first and second 3D representations are aligned during scanning, preferably when scanning an unmodified part of the objects which is therefore common to first and second 3D representations. Once the aligning is in place substantially all data obtained during scanning of the modified object is replaced into the first 3D representation.

Aligning

Aligning the first and second 3D representations can be a crucial step of the present invention. Preferably the step of aligning comprises aligning the coordinate systems of the first and second 3D representations, i.e. the local coordinate systems of the different 3D representations must be aligned to a common coordinate system. The preferred embodiment of the invention comprises some sort of registration of the first and second 3D representations. The registration procedure may be an independent step of the present invention. However a registration procedure may be part of the alignment procedure and/or the replacement procedure. A further embodiment of the invention comprises calculation of distances between corresponding points. The aligning of the representation may then subsequently be the result of minimizing the square root of the distance between these corresponding points.

In one embodiment of the invention aligning is at least partly provided by the configuration of the hardware, such as the configuration of the scanner hardware. This may be the case if the object is kept in the same coordinate system during modification and/or scanned in the same coordinate system, e.g. by means of arranging the impression or cast of teeth in a fixed holder in a scanner when scanning the impression or cast of teeth. However, even when the aligning is primarily provided by the hardware it may further be assisted by the software.

In yet a further embodiment of the invention the aligning of 3D representations is at least partly provided by software. This is typically necessary if a handheld scanner, such as an intra oral scanner, is used for obtaining at least one of the 3D representations. Further, the aligning may at least partly be manually assisted.

In a further embodiment of the invention aligning may be provided by means of matching and/or comparing one or more specific features, such as one or more specific features common to the first and second representation. Within the dental area the margin line may be unaffected by the modification, thus the margin line may be used for aligning the 3D representations. Aligning may further be provided by means of texture comparison, e.g. color, such as by aligning matching textures. Within the dental area the teeth and the gingival can be easy to distinguish due to differences in texture. Aligning may further be provided by means of matching and/or comparing one or more peripheral features of the second 3D representation. As noted previously the unmodified surroundings of the modified part is preferably included in the second 3D representation. As these features are not modified they will typically be common to the first and second 3D representations and can thereby be used in the aligning procedure.

In a further embodiment of the invention the aligning procedure and/or the registration of the first and second 3D representations comprises, or is followed by, some sort of data comparison between the first and second 3D representations. The second 3D representation may comprise unmodified parts of the object and these unmodified parts will typically be common with parts in the first 3D representation. Comparison of these common parts will provide a small difference or small error. However, comparison of the modified parts in the second 3D representation with the corresponding parts in the first 3D representation, i.e. before modification, will provide a larger difference or larger error. Thus, in a further embodiment of the invention a predefined criterion for maximum allowed error in the alignment and/or registration may be applied. Applying such a criterion may help to automatically define the modified part(s) of the object that distinguishes the first and second 3D representations. The error criterion can for example be based on integration of the registration error of a closed subset of a predetermined size. Replacing

Replacing the data in the first 3D representation with data from the second 3D representation that represents the modified part of the object is a crucial part of the invention. In the preferred embodiment of the invention the procedure of automatically replacing comprises a selection of the first subset of the first 3D representation. This first subset is where replacement is intended. Further a selection of a second subset of the second 3D representation is typically necessary. This second subset is the subset that is to be inserted in the first 3D representation. The selections of the first and second subsets are preferably provided automatically. In one embodiment of the invention these subset selections are based on the previously mentioned error criterion.

Further to selection of the subsets on the basis of the error criterion the second subset may be expanded, e.g. expanded with a safe zone to provide an additional buffer towards the boundary. In a further embodiment of the invention the second subset is excluded from the second 3D representation during registration of the first and second 3D representations. Furthermore, the second subset and the boundary of the second subset may be excluded from the second 3D representation during registration of the first and second 3D representations. When the second subset and possibly also the closed boundary of the second subset is excluded from the registration between the first and second 3D representations it is ensured that the registration is provided between two 3D representations where each point in the second 3D representation has a corresponding point in the first 3D representation, because the data representing the modified part of the object has been excluded from the second 3D representation during registration. Once the registration between the first and second 3D representations has been provided data from the second subset can be replaced into the first 3D representation, thereby updating the first 3D representation with data representing the modified part of the object.

The second subset of the second 3D representation represents the modified part of the object and data in the second subset is replaced into the first 3D representation. There will be a boundary between the original data in the first 3D representation and the new data from the second 3D representation. In a further embodiment of the invention this boundary is "smeared out" or softened by combining data from the first and second 3D representations across or adjacent to the border. There will typically be data in both the first and second 3D representations that are common and almost identical. In a further embodiment of the invention transitional subsets of the first 3D representation and the second 3D representation are selected, preferably selected automatically. These transitional subsets may for example define the area adjacent to or around the first subset and the second subset, e.g. a peripheral area around the first and second subsets. In this area data from either the first 3D representation or the second 3D representation may be used to define the transition between the original data and data obtained from scanning the modified object. Thus, there will typically be a "blending zone" between two subsets from the first and second 3D representations. To address this issue an interpolation between data from different subsets may be applied. In one embodiment of the invention a weight function is applied to said transitional subsets to define the transition in the common area between the first and second 3D representations. The weight applied to a specific data point may depend on the location of said data point relative to the peripheral boundary of the first and second subsets, respectively.

The data replacement procedure may also be supplemented by morphing of the first and second 3D representations. Morphing may advantageously be applied in the transitional areas between the different 3D representations.

Replacement of the data from the first 3D representation, or first scan, with data from the second 3D representation, or second scan, may be performed for example by:

- obtaining a first scan of a patient's teeth comprising a tooth which should be prepped, where a dentist may perform the first scanning of the teeth;

- deleting the area in the first scan comprising the tooth which should be prepped, where the dentist may mark the area manually on a touch screen showing the scan;

- obtaining a second scan of the tooth after it has been prepped, i.e. now called the prep, where the dentist may perform the second scanning of the prep, after he has made the prep, and where the second scan may be performed on the same day as the first scan or on a later day, and where the second scan may be performed on the same day as the prep was performed or on a later day;

- inserting the second scan of the prep into the deleted area on the first scan. Instead of deleting the area where the prep will be, the area may be marked, e.g. by the dentist, and when obtaining the second scan, only data in the marked area will be replaced with data from the second scan.

Alternatively, no area is deleted from the first scan, so when obtaining the second scan, the scanner may deal with this as if the scanning is just continued and the second scan is then added to and/or replaced in the first scan as a normal sub-scan or the first scan of the area is overwritten with the second scan. Thus the first scan may only be replaced with the second scan in the area which the second scan is covering.

A rule may be defined in the software, saying that if the difference between two scans of the same area is larger than X, then the second scan replaces the first scan in that area.

If the first scan and the second scan are obtained on different days or obtained with a time span such that the scanner and system has been used for scanning other patients, then the first scan of the patient is loaded on the screen and the second scan is then obtained, so that the second scan is connected, linked, associated to the first scan.

The deletion or replacement in the first scan and/or the adding of the second scan may be performed manually e.g. by the dentist pointing areas on a touch screen, or may be performed automatically by computer software, computer processors etc..

The automatic deletion, replacement or addition may be performed by means of a detection of the difference between the first and the second scan. In an area where a difference is detected, the last or new data, i.e. the second scan, may replace the old data, i.e. the first scan.

Dental applications

The invention may in particular be applied within the dental field. Thus, in a further embodiment of the invention the object is a dental object, such as at least a part of the upper jaw and/or lower jaw, such as one or more teeth. A dental restoration can be one or more inlays, onlays, veneers, crowns, bridges or combinations thereof and/or a dental restoration can be a removable partial denture framework and/or an implant-retained structure.

In yet a further embodiment of the invention the modification is a preparation of one or more teeth prior to dental restoration. A preparation for a dental restoration typically requires grinding, drilling, removal, endodontic treatment and/or the like, of relevant tooth/teeth. Thus, in one embodiment of the invention the second subset represents one or more preparations. The modification may also be a healing abutment. The modification may also be the insertion or removal of a retraction cord. The modification may also be the injection/addition of adrenalin to the gingiva near a preparation. A retraction cord may be soaked in adrenalin. In case of retraction cord or adrenalin the modification is provided in the gingiva area. The modification may also be a combination of any of the above listed modifications. However, please note that the actual preparation/modification, e.g. of a tooth, is not necessarily a part of the present invention.

In a further embodiment of the invention margin lines are calculated and/or determined for the first and second 3D representations. Further, the aligning procedure may at least partly be based on detecting and/or demarcating and/or aligning margin lines of the first and second 3D representations. Morphing may be applied near the margin line of the first and second 3D representations. Further segmentation, at least partially, of hard and soft material of the objects, such as teeth and tissue, tissue such as gingiva, may be provided in the first and second 3D representations. The segmentation may be based on texture comparison, e.g. based on color information in the first and second 3D representations.

In a further embodiment of the invention segmentation is at least partly provided by means of a computer implemented algorithm, such as a shortest- path algorithm applied on a 3D matrix representing curvature of the tooth surface.

Managing the soft tissue around the teeth is often a difficulty that dentists face. A variety of possible conditions can result in significant inflammation, which is problematic for preparing teeth and making accurate impressions. A problem may be gingiva shrinkage when the tissue heals caused by gingiva inflammation. This may lead to inaccurate impressions and badly fitting dental restorations.

In some cases the gingival around a preparation may collapse during scanning of the preparation, such that the preparation line is not visible everywhere for an intra oral scanner, and hereby the preparation line may not be obtained in the first 3D representation of the patient's teeth. In this situation the dentist/assistant may push the gingival back again and away from the preparation line, and then the dentist scans again for example only at the preparation line thereby obtaining the second 3D representation, where the gingival is modified, and hereby the entire preparation line is obtained, which is important when later designing the restoration for the prepped tooth. In some cases the tooth which should be restored may be visually nice, and in such cases the look of the tooth may be copied for the design of the restoration. In this case the tooth may be scanned before preparation to obtain the first 3D representation, and the scan of the tooth shape from the first 3D representation is then copied into the software program used to design the restoration. After the tooth has been prepped by the dentist, the preparation may be scanned thereby obtaining the second 3D representation of the modification. The look of the tooth from the first representation may then the used when designing the restoration to fit on the preparation from the second representation.

Retraction cords are a standard tool when making teeth preparations for dental restorations. A retraction cord will typically prevent bleeding from the gingiva and will expose the boundary/border of the preparation, i.e. the border between the preparation and the gingiva. Two retraction cords may be used: an inner cord for stopping the bleeding and an outer cord for exposing the preparation border. The retraction cord may be soaked in or impregnated with adrenalin. The adrenalin causes the gingiva to temporarily open or retract around the prepared tooth thereby exposing the preparation border and making it possible to position the retraction cord around the tooth. A correct model of the specific preparation border of a prepared tooth is extremely important to obtain a tight attachment of the subsequent dental restoration. Thus, typically the dentist will remove the retraction cord or at least the outer retraction cord. Adrenalin may help to keep the preparation border exposed, however this is just a brief respite before the gingiva begins to fall back towards the preparation border. Thus, after removal of the retraction cord the dentist must provide an impression of the teeth immediately. This can be a very stressful situation, in particular if there are several prepared teeth, e.g. for a bridge or a denture. With several prepared teeth all retraction cords must be removed and an impression of the teeth must be provided when the preparation border is exposed before the gingiva starts to fall back. By means of the present invention this stressful situation may be avoided. A detailed overview scan may be provided before any preparation has been performed, i.e. thereby obtaining the first 3D representation, e.g. by intra oral scanning. After the preparations have been performed and retraction cords are placed around each preparation the retraction cord may be removed from a single preparation which is subsequently scanned, e.g. by intra oral scanning, thereby obtaining a second 3D representation. The original scan may then be updated with the new scan of the preparation. Subsequently the retraction cord may be removed from another preparation which is then scanned and the updated original scan may be further updated with data representing another preparation, etc. Thereby the dentist avoids removing retraction cords from all preparations at once prior to scanning or taking an impression. Another situation could be a set of teeth with a plurality of preparations where the preparation border is not exposed, either because there are retraction cords around the preparations or the gingiva is covering the borders. A detailed overview scan may be provided of the set of teeth with unexposed/hidden preparation borders, thereby obtaining the first 3D representation. In order to obtain data representing the preparation borders a retraction cord may be removed from a preparation and immediately scanned, thereby obtaining a second 3D representation, for subsequent updating of the original overview 3D representation. As previously indicated the preparation border may be exposed by adding adrenaline to gingiva. This procedure may be repeated for each preparation thereby consecutively updating the original overview 3D representation with data for the crucial preparation borders.

In this case of missing information of the preparation border the data representing a preparation in the first and second 3D representation will actually be common to the 3D representations because the actual preparation is not modified between obtaining the first and second 3D representation. It may therefore be advantageous to use the preparation, or the center of the preparation, as a means for aligning the first and second 3D representations. Thus, in a further embodiment of the invention a preparation, or the central part of a preparation is used for aligning the first and second 3D representations.

If the patient has one or more dental implants in his mouth, then the method may be used such that the patient's dental arch or mouth is scanned with scan flags in the implants and scanned without scan flags in the implants. The object being modified may then be understood as being the implant modified with a scan flag. Alternatively and/or additionally the object being modified may still be a tooth, e.g. at tooth which is prepared for a crown or a tooth being extracted for inserting an implant instead of the tooth, and scanning the implant with and without the scan flag may then be a further step of the method. Instead of a scan flag, a healing abutment or the like in the implant may be scanned for obtaining the position and/or orientation of the implant. When the position and/or orientation of the implant is/are obtained, a suitable restoration can be designed for the implant. The process may be as follows:

- a scan flag, an abutment or the like is arranged by the dentist or his assistant in the implant(s) in the patient's mouth;

- the dentist scans teeth and implant(s) with scan flags in the mouth of the patient;

- the scanning result is shown on a screen, e.g. a touch-screen for receiving input from a user;

- the dentist or his assistant marks where the implant is, e.g. by pressing with a finger on the relevant position of the implant on the scanning result on the touch screen;

- a virtual cylinder is then arranged around the implant on the screen;

- the scan flag is then removed from the implant by the dentist or his assistant;

- the dentist scans the teeth and implants again without the scan flag.

Hereby the position and orientation of the implant is obtained by means of scanning the scan flag in the implant, and the 3D shape of the visible part of the implant or healing abutment and of the neighbor teeth and of the antagonist is scanned by means of scanning the implant without the scan flag.

Alternatively the teeth and implants are first scanned without the scan flags to obtain the 3D shape of teeth etc. Then the scan flags are inserted and only the implants with the scan flags are scanned again to obtain the position and orientation of the implants.

The method may be used for performing a clinical validation of the whether the preparation of a tooth is performed satisfactorily. In some cases a virtual preparation may be designed before the dentists start prepping the tooth. The dentist should then physically prep the tooth so that is corresponds to the virtually designed prep. Thus the dentist/assistant may alternately remove material from the preparation and scan the preparation. When scanning the prep, a clinical validation comparing the designed virtual prep and the scanned physical prep is performed, and by comparing the two, it is determined whether enough tooth material was removed from the prep. When enough tooth material has been removed according to the clinical validation, the dentist stops prepping. If enough tooth material has not been removed yet according to the clinical validation, the software program informs the dentist to prep more. After the dentist has prepped more, he may scan the prep again and perform the clinical validation of whether the prep is accepted now, etc..

In some cases a virtual 3D preparation guide may be designed before the dentist start prepping the patient. The preparation may be the preparation of a single tooth for a crown, a number of teeth for a bridge, the extraction of a tooth, the insertion of an implant etc. The virtual prep guide may be a file with 3D CAD images, computer simulations, text instructions etc.. The prep guide may be loaded into a computer with a screen in the dental clinic, and the prep guide may be shown on the screen, while the dentist preps and/or scans the patient with an intra oral scanner. So while the dentist is prepping a tooth, he can look on the screen with the prep guide to see how he should perform the preparation. After prepping, the dentist may perform a scanning of the prep he has just made, and the scan of the prep and the prep guide may be seen simultaneously on the screen to verify, either automatically or by eye measure, whether the preparation was suitable.

If the patient should have a larger procedure made, e.g. replaced several front teeth, then a diagnostic wax-up, try-on/in or temporary restoration may be produced in a fast and cheap way, and the patient may then wear this temporary restoration to check if he likes the visual appearance and/or the functionality of the restoration before a final, expensive restoration is produced. When a temporary restoration is produced, the temporary restoration may be arranged on the prepped teeth in the patient's mouth, and the temporary restoration and remaining teeth in the mouth may then be scanned as a first scanning. Afterwards the temporary restoration is removed from the mouth and the preps can now be seen, and the preps are then scanned as a second scanning. Instead of deleting the temporary restoration from the first scan, the first scan and the second may be seen simultaneously on a computer screen, and a dental technician may virtually design the final restoration based on the temporary restoration and the preparations.

Computer and system implementation

The methods according to the present invention are preferably implemented in software. Various embodiments of the invention relates to methods executed during procedures carried out on patients. But the scope of the present invention does not cover the actual procedures carried out on patients. Procedures may therefore be referred to in some embodiments, indicating that the different parts of the invention are executed before or after a specific procedure. However, this specific procedure is not an essential or integrated part of the invention.

The present invention relates to different aspects including the methods described above and in the following, and corresponding methods, systems, devices, apparatuses, uses and/or product means, each yielding one or more of the benefits and advantages described in connection with the first mentioned aspect, and each having one or more embodiments corresponding to the embodiments described in connection with the first mentioned aspect and/or disclosed in the appended claims.

The invention furthermore relates to a system comprising means for carrying out any of the herein listed methods. In particular, disclosed is a system for scanning an object being modified comprising:

- means for providing a first 3D representation of the object prior to modification,

- means for obtaining at least a second 3D representation by scanning at least a modified part of the object after modification,

- means for aligning said first and second 3D representations, and

- means for replacing a first subset of the first 3D representation with data from the second 3D representation, said first subset substantially corresponding to the modified part.

The means for providing and obtaining the 3D representations may be a 3D scanner and software for creating a 3D model; or if the 3D representations are acquired at another location than where the rest of the method is performed or at another location than where the rest of the system is located, then the means may be a processor or digital storage or connection for opening and manipulating the 3D representations.

The means for aligning and replacing may be processing means or a processor in a computer.

The invention furthermore relates to a computer program product having a computer readable medium, said computer program product comprising means for carrying out any of the listed methods.

Furthermore, the invention relates to a computer program product comprising program code means for causing a data processing system to perform any one or more of the methods above, when said program code means are executed on the data processing system, and a computer program product, comprising a computer-readable medium having stored there on the program code means. Description of drawings

The above and/or additional objects, features and advantages of the present invention, will be further elucidated by the following illustrative and non- limiting detailed description of embodiments of the present invention, with reference to the appended drawings, wherein:

Fig. 1 shows a 3D representation of a set of teeth before and after preparation,

Fig. 2 shows the modified part of a set of teeth replaced into the original representation of the set of teeth obtained before preparation,

Fig. 3 shows an example of segmentation of teeth and tissue,

Fig. 4 is an illustration of a sagital section of a schematic tooth, and Fig. 5 is an illustration of a zoomed sagital section of a schematic tooth. Fig. 6 shows an example of a flowchart of a method of scanning an object being modified.

Fig. 7 shows a schematic example of the method.

Detailed description of drawings

In the following description, reference is made to the accompanying figures, which show by way of illustration how the invention may be practiced. There are several commercial systems available for obtaining 3D representations of teeth, e.g. from 3Shape, Cadent and 3M. Among these are intra-oral scanners, e.g. 3Shape Trios^®, and scanners for dental impressions or casts thereof, e.g., 3Shape D640, D700 and D710. Scanners can be for example be optical scanners, such as laser and/or structured light scanner etc.. Potentially, scanners with penetrating radiation such as (cone beam) CT scanners, e.g. from Imaging Science International's i-CAT, Kodak/lmtec's lluma can be used. They have the advantage of providing volumetric models showing also decay inside the teeth, while disadvantages include concerns about radiation dose or high price of treatment.

A 3D representation of a complete set of pre-prepared teeth 101 is shown in fig. 1 a. This was acquired before any preparation of the teeth was initiated, i.e. before the patient was tired and stressed. Fig. 1 b shows the same set of teeth 102 but now one tooth 104 has been removed and the two adjacent teeth 103 have been prepared for mounting of a bridge. All teeth except the three modified teeth are common to the sets of teeth in figs. 1 a and 1 b, thus it is advantageous that by applying the present invention only the modified part of the set of teeth have to be rescanned and the data representing this modified part will be replaced into the original 3D representation, thereby reducing discomfort for the patient.

Figs. 2a and 2b is a coarse illustration (using 2D images of 3D models) of the basic idea of the present invention. Fig. 2a shows a cut-out of the modified part of the set of teeth from fig. 1 b corresponding to that only the modified part and the part of the adjacent surroundings have been scanned. Thus, whereas fig. 1 a may be seen as corresponding to the first 3D representation as referred to in this application; fig. 2a may be seen as corresponding to the second 3D representation as referred to in this application. In fig. 2b a subset of fig. 2a has been replaced into fig. 1 a thereby updating fig. 1 a to represent the modified object shown in fig. 1 b. This subset of fig. 2a may be seen to correspond to the "second subset" as referred to in this application and has been provided with a frame in fig. 2b to better emphasize the subset to the reader. The alignment of the first 3D representation 201 (fig. 1 a) and the second 3D representation 202 (fig. 2a) can be provided because they share common areas namely the unmodified teeth adjacent to the modified teeth 203 and 204. Optical scanners generally obtain a 3D digital model of an object's surface. While this model describes geometry, it does not differentiate between any materials or sub-objects that make up the surface. Specifically for dental applications, the 3D model does not differentiate between teeth and gingiva, some of which inadvertently will be included in a teeth scan. For visualization and CAD design of dental restorations, it can therefore be advantageous to segment the 3D models into teeth and tissue, respectively. And it may further be advantageous to segment the individual teeth in the 3D models. Segmentation can be applied by means of an algorithm implemented in software. Segmentation into teeth and tissue is illustrated in fig. 3 with the original 3D model illustrated in fig. 3a) and the result of the segmentation illustrated in fig. 3b, where the teeth part of the 3D model is marked in a different color than the gingival.

In one embodiment of the invention, the segmentation algorithm uses vectors perpendicular to each tooth, or a single vector, perpendicular to the whole model, and one point in the middle of each tooth or two points on the distal and mesial sides of the tooth. A preferred version of the separation algorithm is based on using a 3D shortest path algorithm, preferably capable of handling negative weights, for example the Bellman-Ford algorithm. The algorithm is preferably applied on a 3D matrix with elements representing curvature of the surface of the tooth model shown in fig. 3a. In another embodiment, the scanner used to generate the model shown in fig. 3a can capture color as well. Segmentation can then be based on color information. Teeth are typically white and gingival is typically red or flesh colored. Dental restorations that can be designed in a CAD system include inlays, onlays, veneers, crowns, bridges, combinations thereof, and others. By analogy, the term "restoration" also covers removable partial denture frameworks and implant-retained structures. Several dental CAD software packages that allow such design are available, for example 3Shape DentalDesigner.

Preparation of the teeth is typically performed by the dentist grinding down the teeth such that the dental restoration can be glued on. This step of the actual physical preparation of the teeth is not necessarily part of the present invention, which is mainly computer implemented. The difference between prior to and after preparation of a tooth is illustrated schematically in fig. 4. Prior to preparation, i.e. pre-preparation, the tooth may be more or less complete illustrated by solid line C100. However, if the tooth is broken before the preparation, the tooth may not be complete along its entire surface, and the solid line C100 would then be different than seen in fig. 4, such as showing that a corner of the tooth is missing. The tooth should however still be prepared by the dentist before fixing the dental restoration. The preparation removes material from the tooth and after preparation the outer contour of the teeth may be like the C102.

The line C300 may indicate the inside of the dental restoration, e.g. a crown, and as can be seen there may be a small gap between the inside of the restoration C300 and the preparation contour C102 of the tooth. Cement or glue or the like may be filled in the gap for fixing the restoration securely to the tooth preparation. Fig. 5 is a "close-up" schematic illustration of a sagital section of a tooth. The pre-prepared tooth is illustrated by the solid line C100. Depending on the tooth and the type of restoration different types of preparations may be performed on the tooth. This is exemplified by the dotted lines, C300, C102, C301 and C402. The preparation border, 600 or 601 , is indicated for different types of preparations. As may be imagined from fig. 5 gingiva might cover the preparation border, 600, 601 , after preparation of the tooth because the tooth material between the solid line and the dotted lines has been removed.

After preparation a 3D digital model of prepared teeth and possibly gingiva can be obtained. The segmentation of teeth and gingiva in the prepared model can be executed analogously as described above.

Fig. 6 shows an example of a flowchart of a method of scanning an object being modified.

In step 601 a first 3D representation of the object prior to modification is provided or obtained. The first 3D representation may be obtained by means of a scanner, such as an intra-oral scanner, a desktop scanner etc. The scanning may be performed directly in the mouth of a patient using an intra oral scanner, or performed on a physical impression on the patient's teeth using a desktop scanner, or performed on a physical model of the patient's teeth using a desktop scanner.

In step 602 at least a second 3D representation is obtained by scanning at least a modified part of the object after modification. The second 3D representation may be obtained in the same way as the first 3D representation.

In step 603 the first and second 3D representations are aligned. The first and second 3D representations may be virtually aligned using a computer software program where the two 3D representations are loaded in.

In step 604 a first subset of the first 3D representation is replaced with data from the second 3D representation, where the first subset substantially corresponds to the modified part, thereby updating the first 3D representation to represent the modified object.

Fig. 7 shows a schematic example of the method.

In fig. 7a) a schematic example of a first 3D representation 701 is shown. Three teeth 704, 705 are shown. In fig. 7b) a schematic example of a second 3D representation 702 is shown.

A preparation 703 is shown, and parts of two teeth 704 are shown.

In fig. 7c) the first 3D representation 701 and the second 3D representation

702 are aligned. The second 3D representation 702 is indicated by dotted lines and corresponds to the prep 703.

In fig. 7d) a part of the first 3D representation 701 is replaced with the second 3D representation 702 corresponding to the modified part, which is the prep

703 replacing the tooth 705. Fig. 7a) corresponds to step 601 in fig. 6, fig. 7b) corresponds to step 602 in fig. 6, fig. 7c) corresponds to step 603 in fig. 6, and fig. 7d) corresponds to step 604 in fig. 6.

Although some embodiments have been described and shown in detail, the invention is not restricted to them, but may also be embodied in other ways within the scope of the subject matter defined in the following claims. In particular, it is to be understood that other embodiments may be utilised and structural and functional modifications may be made without departing from the scope of the present invention.

In device claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims or described in different embodiments does not indicate that a combination of these measures cannot be used to advantage.

It should be emphasized that the term "comprises/comprising" when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof. The features of the method described above and in the following may be implemented in software and carried out on a data processing system or other processing means caused by the execution of computer-executable instructions. The instructions may be program code means loaded in a memory, such as a RAM, from a storage medium or from another computer via a computer network. Alternatively, the described features may be implemented by hardwired circuitry instead of software or in combination with software.

Claims

1 . A method of scanning an object being modified comprising the steps of: a) providing a first 3D representation of the object prior to modification, b) obtaining at least a second 3D representation by scanning at least a modified part of the object after modification,

c) aligning said first and second 3D representations, and

d) replacing a first subset of the first 3D representation with data from the second 3D representation, said first subset substantially corresponding to the modified part, thereby updating the first 3D representation to represent the modified object.

2. The method according to any one or more of the preceding claims, wherein aligning is performed automatically and/or wherein replacing is performed automatically.

3. The method according to any one or more of the preceding claims, wherein the first 3D representation is obtained by scanning the object prior to modification.

4. The method according to any one or more of the preceding claims, wherein the first 3D representation, the second 3D representation and the updated first 3D representation are substantially continuous surface representations.

5. The method according to any one or more of the preceding claims, wherein scanning can be scanning intra orally, scanning an impression of a set of teeth and/or the antagonist, scanning a cast of a set of teeth and/or the antagonist, CT scanning and/or the like scanning methods.

6. The method according to any one or more of the preceding claims, wherein the first subset represents parts of the object, wherein said parts are mutually interconnected or wherein said parts are not mutually interconnected.

7. The method according to any one or more of the preceding claims, wherein step b) is repeated at least once thereby obtaining a plurality of 3D representations prior to updating the first 3D representation, said plurality of 3D representations representing different modified parts of the modified object.

8. The method according to any one or more of the preceding claims, wherein step b) is repeated for each new modification of the object thereby obtaining a 3D representation of each modification prior to updating the first 3D representation.

9. The method according to any one or more of the preceding claims, wherein a modified part of the object is divided into sections, and steps b), c) and d) are executed for each section, thereby updating the first 3D representation with data corresponding to one section before continuing with the following section.

10. The method according to claim 9, wherein one section corresponds to one tooth.

1 1 . The method according to any one or more of the preceding claims, further comprising the step of repeating steps b), c) and d) for each new modification of the object, thereby sequentially updating the 3D reference model.

12. The method according to any one or more of the preceding claims, wherein the step of aligning comprises aligning the coordinate systems of the first and second 3D representations.

13. The method according to any one or more of the preceding claims, where aligning is at least partly provided by hardware.

14. The method according to any one or more of the preceding claims, where aligning is at least partly provided by implementation in software.

15. The method according to any one or more of the preceding claims, where aligning is at least partly manually assisted.

16. The method according to any one or more of the preceding claims, wherein the step of aligning comprises aligning by means of matching / comparing one or more specific features, such as one or more specific features common to the first and second 3D representations, such as the margin line.

17. The method according to any one or more of the preceding claims, wherein the step of aligning comprises aligning by means of matching / comparing one or more peripheral features of the second 3D representation, said features are assumed to be unmodified.

18. The method according to any one or more of the preceding claims, further comprising the step of registration of the first and second 3D representations.

19. The method according to any one or more of the preceding claims, wherein the step of aligning and/or the step of replacing comprises registration of the first and second 3D representations.

20. The method according to any one or more of the preceding claims, wherein the step of aligning comprises applying a predefined criterion for maximum allowed error in the registration.

21 . The method according to any one or more of the preceding claims, wherein the step of replacing comprises any of the steps of:

- selecting the first subset of the first 3D representation where replacement is intended,

- selecting a second subset of the second 3D representation that is to be inserted in the first 3D representation, said second subset preferably defined as a closed boundary of the second 3D representation, and

- replacing the first subset of the first 3D representation with the second subset of the second 3D representation.

22. The method according to any one or more of the preceding claims, wherein selection of the first subset and/or the second subset is based on said error criterion.

23. The method according to any one or more of the preceding claims, wherein the second subset is expanded with a safe zone.

24. The method according to any one or more of the preceding claims, wherein the second subset is excluded from the second 3D representation during registration of the first and second 3D representations.

25. The method according to any one or more of the preceding claims, wherein the second subset and the closed boundary of the second subset are excluded from the second 3D representation during registration of the first and second 3D representations.

26. The method according to any one or more of the preceding claims, wherein the step of replacing comprises the steps of:

- selecting transitional subsets of the first 3D representation and the second 3D representation, said transitional subsets defining a common area of the first and second 3D representations, and

- applying a weight function / interpolation to said transitional subsets to define the transition in the common area between the 3D models.

27. The method according to any one or more of the preceding claims, wherein said transitional subsets are adjacent and/or at least partly surrounding the first and second subset, respectively.

28. The method according to any one or more of the preceding claims, wherein said error criterion is based on integration of the registration error of a closed subset, preferably a closed subset of predetermined size.

29. The method according to any one or more of the preceding claims, wherein the weight function applies a varying weight to the combined data in the common area of the first and second 3D representations, wherein the applied weight to a specific data point depends on the position of said specific data point relative to the peripheral boundary of said first and second subsets, respectively.

30. The method according to any one or more of the preceding claims, wherein the step of replacing comprises morphing part of the second 3D representation to the first 3D representation.

31 . The method according to any one or more of the preceding claims, wherein the object is a dental object, such as a set of teeth, such as at least a part of the upper jaw and/or lower jaw, such as one or more teeth.

32. The method according to any one or more of the preceding claims, wherein the modification is a preparation of one or more teeth prior to dental restoration.

33. The method according to any one or more of the preceding claims, wherein the modification is the exposure of a preparation border of one or more prepared teeth.

34. The method according to any one or more of the preceding claims, wherein the second subset represents one or more preparations.

35. The method according to any one or more of the preceding claims, wherein the modification is a healing abutment.

36. The method according to any one or more of the preceding claims, further comprising the step of calculating margin lines of the first and second 3D representations.

37. The method according to any one or more of the preceding claims, wherein the step of aligning is at least partly based on detecting and/or demarcating and/or aligning margin lines of the first and second 3D representations.

38. The method according to any one or more of the preceding claims, wherein morphing is applied near the margin line of the first and second 3D representations.

39. The method according to any one or more of the preceding claims, further comprising the step of at least partially segmenting hard and soft material of the objects, such as teeth and tissue, tissue such as gingiva, in the first and second 3D representations.

40. A method according to any one or more of the preceding claims, wherein segmentation is at least partly provided by means of a computer implemented algorithm, such as a shortest-path algorithm applied on a 3D matrix representing curvature of the tooth surface.

41 . A method according to any one or more of the preceding claims, wherein segmentation is at least partly based on texture information in the first and second 3D representations.

42. The method according to any one or more of the preceding claims, wherein a dental restoration can be one or more inlays, onlays, veneers, crowns, bridges or combinations thereof and/or a dental restoration can be a removable partial denture framework and/or an implant-retained structure.

43. A system for scanning an object being modified comprising:

- means for aligning said first and second 3D representations, and

44. The system according to claim 43 comprising the means for carrying out the method according to any one or more of claims 1 to 42.

45. A computer program product having a computer readable medium, said computer program product providing a system for scanning an object being modified, said computer program product comprising means for carrying out any of the steps of any one or more of the methods according to any of claims 1 to 42.

46. A computer program product comprising program code means for causing a data processing system to perform the method of any one or more of claims 1 -42, when said program code means are executed on the data processing system.

47. The computer program product according to the preceding claim, comprising a computer-readable medium having stored there on the program code means.