WO2005008283A1 - Method for 3d-reconstruction of images acquired by means of laser radar, and endoscopic apparatus using such method - Google Patents

Method for 3d-reconstruction of images acquired by means of laser radar, and endoscopic apparatus using such method Download PDF

Info

Publication number
WO2005008283A1
WO2005008283A1 PCT/IT2004/000379 IT2004000379W WO2005008283A1 WO 2005008283 A1 WO2005008283 A1 WO 2005008283A1 IT 2004000379 W IT2004000379 W IT 2004000379W WO 2005008283 A1 WO2005008283 A1 WO 2005008283A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
points
scanning
scanning head
axis
Prior art date
Application number
PCT/IT2004/000379
Other languages
French (fr)
Other versions
WO2005008283B1 (en
Inventor
Marcello Marzoli
Original Assignee
Marcello Marzoli
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Marcello Marzoli filed Critical Marcello Marzoli
Publication of WO2005008283A1 publication Critical patent/WO2005008283A1/en
Publication of WO2005008283B1 publication Critical patent/WO2005008283B1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/10Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • G01S17/89Lidar systems specially adapted for specific applications for mapping or imaging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • G06T7/521Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30028Colon; Small intestine

Definitions

  • the present invention refers to a method for 3D-reconstruction of images acquired by means of laser radar, and endoscopic apparatus using such method. More in particular, the method according to the invention allows a three dimensional map of the inner surface of examined cavities, in particular body cavities, to be produced, on the basis of a relative distance, detected through a scanning head comprising a laser radar scanning optics, between the scanning head inserted into the cavity and the inner surface of this, as well as some physical parameters identifying the same surface.
  • the endoscopic apparatus uses such method.
  • the laser radar, lidar, ladar or optical radar acts in a way similar to an ordinary radar, with the difference that it sends short pulses or light rays instead of radio waves (they are both electromagnetic waves, but radar waves present wavelengths from 10.000 to 100.000 times longer than the light ones).
  • a receiving system picks up and processes the reflected light. The distance of the object (or “target") hit by the light ray is then calculated on the basis of the delay from emission to reception (also called "time of flight").
  • Laser radars may be with "continuous wave” (CW) or pulsed, focalized or collimated. In case of close target, the focalized CW one is generally preferred, with lower peak power. Also, it may be possible to have separated sites for signal transmission and reception, or they may be coincident in a unique site (as generally preferred in scientific environment).
  • a common laser radar records the intensity of picked up radiations and their delay with respect to transmission.
  • Knowledge of delay (time of flight) and scanning optics tracking direction allows the target to be located with respect to the same scanning optics (and, hence, to the scanning head).
  • the "Coherent (Heterodyne) Laser Radar” also capable to record the phase of the same reflected radiation. From the phase, it is possible to obtain information on the mean frequency, the frequency spectrum and the photon polarization: from the mean frequency, it is possible to obtain the relative speed between scanning head and target.
  • the scanning head is not always still, and sometimes it may move stepping, and rigid body rotations and advances are hence added to its intrinsic scanning rotation (for instance, rotation round its own axis).
  • it will be not enough to precisely detect the wall position with respect to the scanning head, but it will be necessary to simultaneously track the rigid body movements with the same precision.
  • suitable location and navigation techniques will have to be used.
  • different location and navigation techniques exist. One of these is the navigation technique called as “dead reckoning" (developed in the sailing field starting from the “deduced reckoning").
  • Document WO97/32182 concerns a "method and apparatus for performing optical measurements using a fiber optic imaging guidewire, catheter or endoscope".
  • the apparatus and the method perform the coherent optical tomography three dimensional scanning of body cavities, through rotational and longitudinal scanning of cavity walls.
  • Document US6133989 concerns a "3D imaging laser radar”. Thanks to ramp voltage switching, singly and for each pixel of the scanned object, the reflected pulse arrival time is recorded. This time is in relation with the third dimension of the object.
  • the sensor receiving the return pulse is made of units arranged on a two dimensional surface.
  • a computer reconstructs the 3D images starting from two dimensional data and arrival time (3rd dimension).
  • Document US5877851 concerns a "scannerless ladar architecture employing focal plane detector arrays and FM-CW ranging theory".
  • a laser transmitter is amplitude modulated for illuminating the whole field of view.
  • the return signal is mixed by means of an electro- optical modulator placed just in front of the focal plane detector array.
  • the array detects and integrates the mixed light signals, on different fields of view. Numerous image frames are recorded along the FM ("Frequency Modulation") period. Fourier transform taken on all the pixels establishes the detected object distance and gives the object 3D image in the field of view.
  • Document WO00/55642 concerns an optical radar applied to three dimensional imaging.
  • a modular light source sends a plurality of light signals on the object to be detected in three dimensional way.
  • Document JP10262973 concerns an endoscope. It comprises a probe sending and receiving ultrasounds while it simultaneously rotates and moves forward and backward. The probe is placed at the endoscope end having an imaging optical device. The ultrasonic images are stored in a particular memory, and an image processor reconstructs the three dimensional image. Image processing is carried out depending on the reciprocal distance of the optical detector and the probe.
  • Document US5249046 concerns a "method and apparatus for three dimensional range resolving imaging".
  • the apparatus uses Fresnel- Risley prisms and an arrangement of part of the frequencies emitted by the light source in times determined with respect to those of reflection, for avoiding a too extended sensor array when the two dimensional surface to be scanned is too extended.
  • Document US6414746 concerns a "3-D imaging multiple target laser radar”. The laser radar is used for producing 3D images of far objects - placed behind reflecting or absorbing penetrable barriers, as masks or darkening smoke.
  • Document WO01/04576 concerns a "method for operating a laser scanner”. The scanner laser is used for creating maps of building structures which will be then subject to subsequent processing, including comparison with previous maps.
  • Document EP0680271 describes an endoscope capable to give three dimensional images through stereoscopy.
  • step A for each sub-set of points according to step A, calculating a reference surface approximating the sub-set, and a reference system, and projecting the sub-set points onto the reference surface along with calculating the coordinates of the points due to projection; C. for each sub-set of points, determining possible zone of overlap between the sub-set and one or more sub-sets preceding in time, determination being based on the comparison of the intensities / for one or more points; D. in the case when overlap zones are found in step C, positioning the reference surfaces of overlapping sub-sets so that the overlap zones coincide; E. returning to the coordinates of the detected points of each sub-set with respect to its own reference system, starting from the point coordinates obtained with the projection of step B.
  • step B may comprise sub-step: B.1 calculating a reference surface, choosing a type of surface from a predetermined set of surface types and adjusting it on the surface formed by the sub-set of points. Still according to the invention, step B may comprise sub-steps: B.2 choosing a reference system and a reference axis not lying on such surface and passing through the origin of the reference system, B.3 calculating the coordinates of the scanning head and of the detected points according to the new reference system and projecting the sub-set points onto the reference surface.
  • the set of reference axes of the sub-sets form a broken linear coordinate.
  • the aim direction of the scanning signal is advantageously obtained starting from the time coordinate of the detected points.
  • the projection of step B.3 is done orthogonally to the reference surface.
  • the surface types of step B at least partially correspond to surfaces of bodies of revolution.
  • the sub-sets of step A have size such that, in step B, the reconstructed surface is from time to time approximated by said surface type, the axis of the body of revolution being said reference axis.
  • step C comprises determining points of each sub-set belonging to a predetermined topological feature of the examined object, and comparing the intensities / of the points of said topological feature.
  • the topological feature is the surface layer vascular system of a body cavity.
  • the intensities / are related to the response of an optical signal in the green band.
  • the topological feature is made of crypts of surface biological tissue of an intestinal cavity.
  • step C may comprise determining overlap of said zones by using a correlation index.
  • the reference axis of step B.2 is obtained as straight line fitting the centroids of scanning sections corresponding to the points orthogonal to a given axis.
  • step B comprises sub-step: B.4 for each sub-set, positioning the arbitrary triad origin into the median point of the reference axis.
  • step B.3 comprises sub-step: B.3.1 determining position and attitude of the scanning head with respect to the reference system.
  • step B.3 comprises sub- step: B.3.2 determining the position of the detected points with respect to the reference system, as sum of the detected related positions with the scanning head position with respect to the reference system.
  • step B.3 comprises sub-step: B.3.3 orthogonally projecting the points of the sub-set under consideration onto the related reference surface.
  • step A the sub-sets of detected points are further subdivided in a plurality of point portions ordered in time, still more preferably portions successive in the time order are at least two by two overlapping.
  • the overlap of portions is of at least 1 point along at least one of the two scanning orthogonal directions.
  • step C may be performed by taking the portions of a sub-set and comparing each portion of this sub-set with one or more different sub-sets. According to the invention, each portion may be compared with the one preceding in the time order.
  • the three dimensional reconstruction is made by processing the sub-sets starting from a first sub- set and proceeding with the successive sub-sets according to time order.
  • the reference axis is obtained as straight line fitting the centroids of the scanning sections corresponding to orthogonal scanning directions of the scanning head scanning optics.
  • step C comprises sub-step: C.1 set as initial reference system, axis and surface of a sub-set processed after the first one, the corresponding reference system, axis and surface determined according to step B for the preceding sub-set.
  • step B.3.4 projecting the portion under consideration onto the reference surface of the preceding sub-set.
  • step C may comprise sub-step: C.2 comparing the projected portion under consideration with the preceding projected sub-set or a plurality of preceding projected sub-sets.
  • step D may comprise sub-step: D.1 positioning the projected portion under consideration by using the zone of overlap with the preceding projected sub-set or with a plurality of preceding projected sub-sets.
  • step D comprises sub-step: D.2 calculating the scanning head position due to positioning of the handled portion, with respect to the reference system of the preceding sub-set.
  • step D comprises sub-step: D.3 calculating the position of the points of the portion in the new position, with respect to the reference system of the preceding sub-set.
  • the reference surface and the reference axis may be calculated according to step B for the sub-set.
  • the detection device is a laser radar.
  • the detection device may also supply the relative speed vector of the acquired points, along the aim direction of the scanning optics.
  • the detection device is a coherent laser radar.
  • step B for the first sub-set under consideration may be performed by considering the sub-set portion by portion, the initial reference system and axis being corresponding to the scanning head position related to the first portion and to the axis of the same head, the scanning head position being considered as fixed for each portion.
  • the speed vector of sum of all the speed vectors for each point may be calculated.
  • the scanning head position in the portion successive to the one under consideration is calculated by using said sum speed vector.
  • step C in case of non effective comparison, the position of the scanning head and of the handled sub-set points may be determined by using said sum speed.
  • the obtained point cloud may be converted into a continuous surface through a smoothing algorithm.
  • said continuous surface may be displayed by means of a three dimensional display module.
  • a computer program characterised in that it comprises code means adapted to execute, when running on a computer, the method according to what described above.
  • a memory medium readable by a computer, storing a program, characterised in that the program is the computer program according to what described above.
  • an endoscopic apparatus comprising a scanning head, a flexible unit, a detection electro-magnetic or electro-optical device, a processing unit, characterised in that the processing unit processes the detected data by using the method according to the invention.
  • figure 1 shows a continuous data flow coming from a scanning head
  • figure 2 shows the arrangement of the continuous data in subsets or "frames” and portions of sub-set or “slices” according to the invention
  • figure 3 shows the two slices of figure 2, separately
  • figure 4 shows overlapping slices related to a body particular
  • figure 5 shows the flow diagram of the initial scanning module in a first embodiment of the method for reconstruction of three dimensional images according to the invention
  • figure 6 shows a scanned surface, with coordinates relative to the scanning head
  • figure 7 shows the calculation of the centroids of the surface of figure 6, i.e.
  • figure 8 shows the reference cylinder overlapping the first frame, wherein the scanning head position is pointed out; figure 9 shows composition of the relative coordinates of a point with those of the scanning head with respect to with respect to the reference cylinder, for obtaining the absolute coordinates of the same point; figure 10 shows a particular of the projection of the points of the surface of the first frame onto the reference cylinder; figure 11 shows the flow diagram of the "slice" scanning module in the first embodiment of the method for reconstruction of three dimensional images according to the invention; figure 12 shows the scanning head with respect to a slice detected by it; figure 13 shows the scanning head and the slice of figure 12, oriented with respect to the reference cylinder of the previous frame; figure 14 shows the projection of the slice of figures 12 and 13 onto the reference cylinder at the top, and the overlap of the same slice with the previous frame at the bottom; figure 15 shows the slice of figures from 12 to 14 with respect to the present position of the scanning head; figure 16 shows the flow diagram of the reference correction module in the first embodiment of the method for reconstruction of three dimensional images according
  • the method according to the present invention is applicable to any examination of even partial cavities. However, it is particularly intended for endoscopic examination of animal or human body cavities. In fact, in this case the method solves problems otherwise unsolvable nowadays except at the cost of making the examination complex and more dangerous. Consequently, in the following, we will refer only to endoscopy of body cavities, in particular la recto-colonoscopy and laparoscopy. In case of 3D recto-colonoscopy, it is deemed that the most proper topological features for univocal recognition of a tissue portion correspond to the vascular map of the examined tissues (similarly to human eye retina scanning, the configuration or "pattern" formed by the vascular network is stable and univocal from point to point).
  • An endoscopic apparatus for recto-colonoscopy or laparoscopy is made of a unit comprising a laser radar, a processing unit, a flexible unit that has to be inserted into the cavity, comprising a fibre optics, and a scanning unit or head comprising at least a laser radar scanning optics and possible other sensors (for example an optical charge coupled device or CCD).
  • the scanning unit or head preferably in the shape of a small cylinder, will be housed at the end of a flexible body very similar to those of the present endoscopes and it will comprise one or more sensors or scanning optics (such for example camera, spectrometer, microscope, mirror system).
  • scanning parameters will be adjusted so as to ensure the requested frequency decoupling, or it may more simply be calculate the time interval within which the scanning head and the surrounding surfaces may be considered as reciprocally "still” and, at the expiry of the interval, the navigation techniques described in the following have to be adopted for referring the detected points and integrating the resulting surface (slice or frame) with the previous ones.
  • the scanning head rotates round its own axis for carrying out the scanning along a direction.
  • the scanning head cylinder is a moving rigid body, within the body cavity under examination, and hence there may be an additional rotation, which may be positively added to or subtracted from the first one.
  • a vertical 360 degrees scanning of the cavity from the scanning optics i.e., when the cavity is approximated to a right cylindrical pipe, along its axis
  • the scanning head it is rather preferable to arrange the scanning head so as to have references or marks of scanning start and end. The reason for this choice are substantially two: first, when the axis direction is reached, too far parts are detected whereby the reconstruction of the gradually scanned zone becomes hard; secondly, without data operating as cyclic reference it would be hard to suitably rearrange the various pixels, as specified below.
  • the laser beam coming from the endoscope axis will be so capable to be aimed by the scanning optics so that tissues are scanned along the direction parallel to the axis of the same cylinder.
  • the simultaneous rotation of the same cylinder will enable the scanning of a helicoidal band of the surrounding surfaces.
  • scanning start and end references could be eliminated through a rocking mirror or galvanometric elements scanning optics capable to provide the detection angle with sufficient precision.
  • the scanning head optics continually detects data (data of intervals beyond the aforesaid references will be automatically eliminated), obviously with respect to its reference system that is herein defined as "relative”.
  • the data processing unit to which the scanning head is connected, receives a continuous data flow.
  • the scanning head it is still to be noted that it could comprise a charge coupled device (CCD) watching for example along the direction of the cylinder axis.
  • CCD charge coupled device
  • the processing unit sends the laser pulse through the fibre optics at a time t and receives from the same means the response of intensity / at a time t + ⁇ .
  • the signal is subjected to an optical and electronic processing, up to obtaining the delay r and the frequency spectrum of the detected elementary area: taking the whole optical path into account, the measured distance from the scanning unit is obtained from the first one, while the mean value of the frequency spectrum gives the frequency shift and hence the leaving/approaching speed v of the same elementary area with respect to the scanning unit.
  • the processing unit hence receives a data string and associates the angles of elevation ⁇ and azimuth ⁇ of the scanning mirror with respect to the scanning unit reference axes, as well as the angles of elevation ⁇ s and azimuth ⁇ s of the upper scanning optics with respect to the same reference axes, with the detection time t Hence, the process described above gives, for each laser pulse sent at the time t, the values /, r, v, ⁇ , ⁇ , ⁇ s , ⁇ s related to the elementary area or "pixel" reached by the pulse.
  • the process is repeated for each pixel, but in order to be able to represent the detected surfaces as a spatial continuum, it is necessary to know with high accuracy the position and orientation of the scanning unit at each time interval t with respect to an independent reference system, wherein the surfaces may be represented as a whole.
  • the detection from the scanning head will correspond to a continuous data flow that will arrive at the external processing unit and will be processed by this.
  • Processing will have, among others, the task of calculating and tracking with extreme accuracy the position of the scanning unit within the examined organ, in order not to degrade the localization precision that is typical for the (Coherent) Laser Radar technique.
  • the continuous data flow coming from the scanning head is arranged in parts separated by the scanning end mark, corresponding to the positions 1, p, p+1, 2p, 2p+1,... , np, with n and p integers.
  • the scanning optics for example polygonal as described above
  • the scanning optics carrying out the scanning by vertically moving while the cylinder rotates.
  • For each scanned pixel there is a data record containing physical parameters, as for example: scanning instant t measured delay/distance r, intensity /, speed vector v, colour values RGB (always detected through the laser radar scanning optics).
  • Intensity / is a fundamental parameter for endoscopy of body cavities.
  • vascular configuration or "pattern” in particular recognising the topological feature that better adapts to body examinations: the vascular configuration or "pattern”.
  • Intensity / of the reflected light and its frequency spectrum in particular due to green light radiance, allow identifying the vascular pattern that is univocal for every living being having it, in particular for human beings.
  • a signal from an optical device for example a charge coupled one or CCD, suitably fixable to or integratable into the scanning head.
  • the data pre-processing module first analyses together / and r in order to identify the marks of end vertical scanning as defined above, in figure pixels 1, p, p+1, 2p, 2p+1, ... , np, np+1,.... At the same time, azimuth angular values for each pixel are accessed.
  • Overlap of slices is essential in the method according to the invention. By using the same data, it allows a continuous test of the scanning head position to be performed and, hence, as it will be seen, the scanning head position within the body cavity to be reconstructed with sufficient approximation. Division in slices is made in order to permit the scanning head to be considered, with sufficient approximation, as still with respect to a data set (exactly the slice). It has to be here noted that two slices may be certainly logically overlapping, but they may "physically" form an angle between them. In fact, while (continually) passing from one slice to the other, a notable scanning head movement could have occurred, whereby distances to the body cavity wall could have change. But this will be corrected in the method according to the invention.
  • Figure 4 shows an image of a body particular with four slices (of which only the two top ones show the points corresponding to the pixels) both vertically and horizontally overlapping by two rows of pixels.
  • step 1 relative spatial coordinates />, ⁇ r and ⁇ r (respectively: position, elevation and azimuth) for each pixel of a complete frame are recovered (or detected if the analysis is continually made during endoscopic examination). Such relative coordinates, together with the cavity detected inner surface, are shown in figure 6.
  • step 2 centroids of successive scanning sections are calculated. Scanning sections are the rows of the pixel array of figure 3, or, as it will be seen in the slice scanning module, the pixels of iso-level with respect to the reference cylinder axis.
  • the reference cylinder is practically approximated with what detected in the initial length, since there is not yet a reference cylinder.
  • the reference cylinder axis for the scanned cavity initial length is calculated. Connection of the centroids is a three dimensional skew arc, the linearization (linear fitting) of which gives the reference cylinder axis.
  • the mean radius R c ⁇ of the reference cylinder is calculated (the radius is orthogonal to the axis defined in the preceding step).
  • Figure 7 show centroid calculation and successive linearization, as well as the mean radius calculation.
  • step 5 coordinates of the initial point x c u, y , z c ⁇ -u end point Xd-f, Yc , Zc and median point x c1 , y c ⁇ , z c ⁇ of the axis of the reference cylinder initial length are calculated.
  • step 6 it is assumed as new reference the triad having origin equal to x c ⁇ , y c1 , z c ⁇ (thus setting these coordinates as null) and attitude ⁇ c7, ⁇ d, ⁇ c ⁇ (Euler angles) coinciding with that of the reference cylinder (the reference attitude is arbitrarily chosen along the axis and these angles are set equal to zero).
  • step 7 the scanning head position with respect to the newly established reference system is calculated.
  • the first position and the first attitude of the scanning head (r p ⁇ -7 , ⁇ p ⁇ - ⁇ , ⁇ p ⁇ - ⁇ , ⁇ P ⁇ - ⁇ , ⁇ p ⁇ - ⁇ , ⁇ pi-i) with respect to the new reference are calculated.
  • Figure 8 shows both the reference cylinder (in short dashes) and the detected surface (in continuous line). Also, the scanning head position, with the angles related to its attitude with respect to the established reference, is shown.
  • step 8 flow passes to the pixels and the "absolute" coordinates (with relation to the chosen reference) r a - ⁇ , ⁇ a ⁇ and ⁇ a1 of each detected point (pixel) are calculated, as vector sum of position and attitude of the scanning head r p ⁇ - ⁇ , ⁇ p ⁇ - ⁇ , ⁇ p ⁇ - ⁇ , ⁇ ⁇ - ⁇ , ⁇ P ⁇ - ⁇ , ⁇ pi.-i with the relative coordinates r n ⁇ r , cp r .
  • FIG. 11 shows the slice scanning module, wherein the following steps are performed.
  • the spatial relative coordinates r r , ⁇ r , ⁇ r for a whole slice are detected (or extracted from a memory buffer).
  • step 11 the spatial coordinates related to the scanning head position (r p1 . ⁇ , ⁇ p ⁇ - ⁇ , ⁇ p ⁇ - ⁇ , 0 P ⁇ - ⁇ , ⁇ P ⁇ - ⁇ , ⁇ pi-i) with respect to the reference cylinder already determined in the initial scanning module (x c ⁇ , y ⁇ , z c ⁇ , ⁇ c7 -,
  • ⁇ d, ⁇ d, Rci are vectorially summed, obtaining the absolute coordinates r ⁇ 7l ⁇ a ⁇ , ⁇ a ⁇ for each point.
  • figure 13 it is shown how the slice appears to be positioned taking the scanning head position in the preceding frame. It is clear that the slice appears to be not correctly positioned.
  • the points of coordinates r a1 , ⁇ a ⁇ and ⁇ p a ⁇ are projected onto the same reference cylinder above. Practically, for the new slice, the reference cylinder already found with the described procedure related to the first frame is still used.
  • pr ov (broken linear coordinate and angle related to the axis and orientation of the reference cylinder) are then calculated, and the records corresponding to each pixel are associated.
  • Each record is related to a time t and comprises the value of intensity / (which is function of the haemoglobin content, hence of the vascular pattern) and it is associated to the reference and to the absolute positions.
  • intensity / which is function of the haemoglobin content, hence of the vascular pattern
  • the comparison may be either “ineffective” or “effective”: a matching is effective when there is an overlap of new and old slices, while it is ineffective when there is no overlap; the overlap is evaluated through a correlation index according to standard techniques.
  • the matching serves for checking whether the scanning head has moved, and in the positive the overlapping points are taken and the slice is placed again re-calculating the scanning head position.
  • ineffective matching may occur when the slice is tilted with respect to the frame or when the whole frame is not overlapping or contiguous to the previously detected/handled one.
  • a step 14 the slice is "set aside” (however it remains stored in a suitable cache/database) until the completion of the frame, and flow returns to step 10 for acquiring a new slice. Instead, if it was the last slice in the frame, then, in a step 20, flow returns to the initial scanning block.
  • step 15 three or more corresponding not aligned points are selected in the two patterns (of reference and of the slice) and their respective cylindrical coordinates (s c . ⁇ f and > c-f7r - with s c-P r o v and ⁇ c . pro v) are compared.
  • the relative position vectors of the points of the slice to be referred are then added to the absolute position vectors of the points of the reference pattern (r a7 -, ⁇ a1 , ⁇ a1 ), obtaining updated position and attitude of the scanning head (r p ⁇ . 2 , ⁇ p1-2, ⁇ Pp1-2, ⁇ P 1-2, ⁇ P 1-2, ⁇ p1-2)-
  • the position and the attitude of the scanning head are thus calculated with respect to the present reference cylinder (i.e. to the present part of the surface union of the successive reference cylinders).
  • the handled slice is added to the cavity overall map, obtaining at this point one frame (the starting one) plus one slice. If flow then returns to the not projected coordinates, there is for example a situation as the one of figure 15. In the case when, for the slice scanning module, it is the last slice of the frame, or the same slice has been already set aside once, matching is reached as above. In case of ineffective matching, flow starts again from the initial scanning block and the frame is memorised according to a provisional reference cylinder unrelated to the previous one. This is done in order to re-use it later when the scanning head will acquire intermediate frames, partially overlapping this analysed one.
  • three or more corresponding not aligned points are selected in the two pattern (of reference and of the slice), of which the respective cylindrical coordinates (s c -n f and ⁇ c . ⁇ f with Sc-prov and ⁇ c - P rov) are already known.
  • the relative position vectors of the points of the slice to be referred (r r , ⁇ r , ⁇ r ) are then added to the absolute position vectors of the points of the reference pattern (r a1 , ⁇ a ⁇ , ⁇ a ⁇ ), obtaining updated position and attitude of the scanning head (r p1 . n , ⁇ p ⁇ .
  • the union of the centroids is taken, which is a three dimensional skew arc, and it is linearised obtaining the axis of the new length of the reference cylinder characterised by the start and end coordinates (x c2 -/, y C 2-u and x c , y c , z c ) and by the attitude ( ⁇ c2 ,
  • the mean radius R c ⁇ of the length of the reference cylinder is calculated.
  • step 25 the coordinates of the median and end points of the axis of the second length of the reference cylinder (respectively x c2 , y c2 , Zc2, Xc2-t, y C 2-f and z c2-r -) are calculated.
  • step 26 triad having origin equal to x c2 , y c ⁇ and z c2 and attitude coinciding with those of the reference cylinder ⁇ C 2, ⁇ c ⁇ , ⁇ c 2 - is assumed as reference (storing it in a suitable cache/database).
  • step 27 the already memorised points of the last frame are again projected onto the corrected reference cylinder (x c2 , y c2 , z c2 , ⁇ C2 , ⁇ c ⁇ , ⁇ c 2 and R c ).
  • Records referred to the cylindrical coordinates s c - ⁇ f and cp c - ⁇ f are obtained.
  • Each record is related to a time t and comprises the value of intensity / (that is function of the haemoglobin content, hence of the vascular pattern) and it is associated to the reference and to the absolute positions.
  • a successive step 28 it is calculated the transformation of the recorded position and attitude of the scanning head into position and attitude in the new reference (from: r p1 . n , ⁇ p ⁇ . n , ⁇ p ⁇ - n , ⁇ p ⁇ - n , ⁇ P ⁇ -n, ⁇ P ⁇ -n to: r P 2-1, ⁇ P 2-1, ⁇ P P 2-1, ⁇ P 2-1, ⁇ P 2-1, ⁇ p 2-l)-
  • the subscript "p2" stands for second position, since it is the position with respect to the second frame.
  • four reconstructions of the scanning head are displayed in two positions corresponding to two slices, showing the last described steps.
  • the fact that the scanning head position may be tracked slice by slice is important since it may also used for referring images coming from a possible CCD. Flow now passes to the slice scanning module, until the last frame is reached.
  • the last frame may also be a frame that has been skipped, or even the lastly detected frame when the endoscope is extracted from the examined cavity.
  • clinical procedure may provide two successive scanning: the first one at the insertion of the endoscope, typically slower, and the second one, faster, during extraction of the same.
  • the map matching technique may be already used starting from the first scanning and in addition it will be possible to recognise and couple the present reconstruction to the "historical" one, enabling one of the system fundamental functions: discovering new tissue degenerations and monitoring the same along time.
  • the detected speed v is available, which is exploited for simplifying some of the aforesaid steps.
  • Speed detection may be for example obtained by means of a Coherent Laser Radar, while it is not obtainable through a simple Laser Radar.
  • Figure 19 shows the flow diagram of the initial scanning module of the method according to the invention in case of use of the speed information.
  • step 1 it is not required that all the points of a frame are acquired, rather only one slice is acquired.
  • the scanning head starting position is assumed as reference zero point, instead of the median point of the cylinder as in the case of the preceding embodiment.
  • step I bis all the speeds of the acquired slice are vectorially summed. Assuming, as it is reasonable to be, that the slice is still during acquisition, the result of this vector sum is actually the scanning head speed. From this speed, by using the value of the slice acquisition time, flow will pass, in step 1ter, to the scanning head position with respect to the successive slice.
  • step I bis it will be possible to calculate the scanning head correct position and hence the absolute coordinates of the detected points. It is clear that this step will give a null result for the first slice, since there is no preceding slice from which the scanning head speed could be extracted.
  • matching may occur between both slice and frame (hence with a bottom overlap zone in the slice) and successive slices (hence with a side overlap zone in the slice). If matching is effective, in step 15-bis the scanning head position is corrected by using the same procedure of the previously described embodiment, possibly integrated with the composition of the speed vectors (for example with a mean between the two determinations). If matching is ineffective, the slice is not set aside into the cache, as in figure 11 , but the scanning head position is corrected in step 15-ter with the described speed method and flow then proceeds to the correction of the projected pattern and to the correction of the absolute position of the points.
  • the present invention thus consists in a new system of recording three dimensional images, capable to detect from realities and to digitally memorise the localization and the spatial distribution of a dense cloud of points of the inner walls of organs under endoscopic examination, so as to allow carrying out a customised three dimensional model of the scanned anatomical cavity as a whole, and the next 3D display.
  • Digital images corresponding to the same tissues, simultaneously detected with other desired techniques, may be overlapped onto such three dimensional model.
  • the expected advantages of the use of the 3D endoscope are connected to the information increase obtained in passing from the detected representation of hollow organs to their topographical representation (reference), as well as to their delicate spatial and volumetric measurement. The exact spatial knowledge of whatever anatomical point and the consequent memorization offer multiple advantages.
  • Availability of a digital 3D geo-referred representation of an organ will allow the doctor to have a customised map of the organ, to carefully examine it by navigating within the 3D reconstruction with interactive modes and hence to univocally identify the localization of a potentially pathological formation, and he will be able to measure its area and volume and to monitor along time its evolution and growth.
  • the same techniques of automatic recognition of changes with respect to data present on file - also called "change detection" techniques - will be usable, and through them the system will immediately point out the parts which have changed, letting the doctor analyse in detail the concerned parts, so also increasing the examination speed and, hence, reducing the patient discomfort, thanks to memory storing of the whole 3D reconstruction.
  • Such system opens the door to a semi-automatic remote diagnostic, by supplying a technological platform to a new generation of intelligent instruments, capable to know their own position and the one of the surrounding environment.
  • the conceived system uses capabilities of the Coherent Laser Radar technology illustrated above for fast measuring position and speed of the surrounding "objects" in a new implementation such as to permit a faithful topographical, spatial and volumetric representation of the observed reality.
  • the applications of the endoscopic apparatus and the method according to the invention may be for example: Endoscopy, Rectal- colonoscopy, Oesophagus-Gastro-Duodenoscopy, Bronchoscopy, Otoscopy, Rhinoscopy, Laryngoscopy, exploratory Laparoscopy, Mapping and volumetric detection of body surface, as Moles, substance Escape (burns, injuries, or else), body Volume.
  • the method and the 3D endoscope according to the present invention is not necessarily proposed for replacing the present-day used techniques, but also for supplying a modular platform capable to integrate images detected through the present-day and future techniques with congruent spatial information, precisely locating the area under analysis in a way repeatable along time.

Abstract

The invention refers to a method for 3D-reconstruction of, particularly endoscopic, images having: aim direction of a scanning signal, distance between the scanning head and detected point, intensity 1 of the signal reflected by the point, and characterised in that it comprises the following steps: A, subdividing the set of points detected along time in a plurality of sub-sets ordered in time; B, for each sub-set, calculating a reference surface approximating the same, and a reference system, and projecting the sub-set points onto such surface; C, for each sub-set, determining zone of overlap with one or more preceding sub-sets, on the basis of the comparison of the intensities 1 for a plurality of points; D, positioning the reference surfaces so that the overlap zones coincide; E, returning to the coordinates of the detected points of each sub-set with respect to its own reference system. The invention further concerns an endoscopic apparatus using the method according to the invention.

Description

METHOD FOR 3D-RECONSTRUCTION OF IMAGES ACQUIRED BY MEANS OF LASER RADAR, and ENDOSCOPIC APPARATUS USING SUCH METHOD The present invention refers to a method for 3D-reconstruction of images acquired by means of laser radar, and endoscopic apparatus using such method. More in particular, the method according to the invention allows a three dimensional map of the inner surface of examined cavities, in particular body cavities, to be produced, on the basis of a relative distance, detected through a scanning head comprising a laser radar scanning optics, between the scanning head inserted into the cavity and the inner surface of this, as well as some physical parameters identifying the same surface. The endoscopic apparatus according to the invention uses such method. In the last years, technological progress has enabled a great development of image diagnostic and it has caused an important improvement to sanitary treatment quality. Design of equipments allowing human body to be observed in section, as Computerized Tomography and Magnetic Resonance, has represented a change in Radiology up to then limited to the study of "shadows" of body structures projected on a photographic film. Digital technology and electronics and informatics development have allowed thinking of new solutions allowing three dimensional images to be obtained. These new methods are still at an initial and experimental stage. Tendency to new solutions just occurred when technics had taken some fundamental steps forward. One of these concerns the resolution of an endoscopic image, defined as the capacity to distinguish two very close objects or points, that nowadays is greatly enhanced. Human eye is capable to discriminate objects of 125-165 microns of diameter, while high resolution endoscopes allow discriminating objects of diameter ranging from 10 to 71 micron. In spite of that, today it is not yet possible to exactly know position and actual measurement of what is analysed. As a consequence of that, reliable and realistic three dimensional (3D) reconstructions of the scanned part cannot be obtained. There exist instruments capable to precisely measure relative distances between scanning head (more precisely scanning optics) and detected object, which however lack a sufficiently reliable determination of the position of the same scanning head. An instrument that is nowadays well known to be capable to precisely detect relative distances is the laser radar. The laser radar, lidar, ladar or optical radar, acts in a way similar to an ordinary radar, with the difference that it sends short pulses or light rays instead of radio waves (they are both electromagnetic waves, but radar waves present wavelengths from 10.000 to 100.000 times longer than the light ones). A receiving system picks up and processes the reflected light. The distance of the object (or "target") hit by the light ray is then calculated on the basis of the delay from emission to reception (also called "time of flight"). If the system is capable to record the reflected light frequency, possible frequency shifting with respect to the emitted light takes account of the target relative speed with respect to the scanning head: because of the known "Doppler effect", light reflected by a leaving object will shift towards longer wavelengths (effect also called as "red shift"), while light reflected by an approaching object will shift towards shorter wavelengths (or "blue shift" effect). Laser radars may be with "continuous wave" (CW) or pulsed, focalized or collimated. In case of close target, the focalized CW one is generally preferred, with lower peak power. Also, it may be possible to have separated sites for signal transmission and reception, or they may be coincident in a unique site (as generally preferred in scientific environment). Thus, a common laser radar records the intensity of picked up radiations and their delay with respect to transmission. Knowledge of delay (time of flight) and scanning optics tracking direction allows the target to be located with respect to the same scanning optics (and, hence, to the scanning head). Moreover, it has been recently introduced the "Coherent (Heterodyne) Laser Radar" also capable to record the phase of the same reflected radiation. From the phase, it is possible to obtain information on the mean frequency, the frequency spectrum and the photon polarization: from the mean frequency, it is possible to obtain the relative speed between scanning head and target. Through this technique, a large increase of sensitivity in conditions of low signal-to-noise ratio (SNR) is obtained, when a further "Local Oscillator (LO) Laser" is added to the laser radar. However, this entails a higher sensitivity of optical elements (composing the laser radar) to alignment and phase disturbances. From the above, it comes out that there are presently instruments which allow detecting the distance and the relative speed between scanning head and target, as well as the classical physical parameters RGB and intensity of the reflected radiation. When it is desired to carry out 3D survey of the inner walls of a hollow body with a scanning head the position and attitude of which is not known, a certain number of problems will be present, first of all the aforementioned one of determining the scanning head position. In fact, it has to be noted that, for example in endoscopy, the scanning head is not always still, and sometimes it may move stepping, and rigid body rotations and advances are hence added to its intrinsic scanning rotation (for instance, rotation round its own axis). Thus, it will be not enough to precisely detect the wall position with respect to the scanning head, but it will be necessary to simultaneously track the rigid body movements with the same precision. To this end, suitable location and navigation techniques will have to be used. Presently, different location and navigation techniques exist. One of these is the navigation technique called as "dead reckoning" (developed in the sailing field starting from the "deduced reckoning"). It is a mathematical procedure for determining the present position of a vehicle, which estimates in successive steps the new position by calculating the advance from the preceding position starting from direction and speed measured in the elapsed time interval. Normally, this technique is used along with others, because it ensures a good short-term accuracy, but it tends to progressively increase the position error up to being no more reliable for long paths, as for example the ones necessary in colonoscopy. Also, in case of colonoscopy, it is to be noted that air is used for enlarging colon, and it is clear that the final diameter of this may vary from one examination to another even for the same patient, so introducing a further source of error. Another technique is the one of determining and detecting topological features being a univocal reference. In the case when the "3D map" of a previous survey is available, the navigation method called as "map matching" may be adopted, that exploit the comparison of the topological features of the "3D map" created during an examination with the ones of the memory stored "historical map". At the present stage of perfection, these techniques are from time to time applied separately or in limited combination. Actually, the result is effective only in some specific cases. In cases of general interest, that is in cases of examination of whatever cavity with the scanning head moving even abruptly within it, the Applicant does not know effective solutions. This scientific and market lack is also found in lacking of patent documents disclosing a general solution to the aforesaid problems. By way of example, we will discuss some in the following, in which limited solutions are disclosed. Document WO97/32182 concerns a "method and apparatus for performing optical measurements using a fiber optic imaging guidewire, catheter or endoscope". The apparatus and the method perform the coherent optical tomography three dimensional scanning of body cavities, through rotational and longitudinal scanning of cavity walls. Document US6133989 concerns a "3D imaging laser radar". Thanks to ramp voltage switching, singly and for each pixel of the scanned object, the reflected pulse arrival time is recorded. This time is in relation with the third dimension of the object. The sensor receiving the return pulse is made of units arranged on a two dimensional surface. A computer reconstructs the 3D images starting from two dimensional data and arrival time (3rd dimension). Document US5877851 concerns a "scannerless ladar architecture employing focal plane detector arrays and FM-CW ranging theory". A laser transmitter is amplitude modulated for illuminating the whole field of view. The return signal is mixed by means of an electro- optical modulator placed just in front of the focal plane detector array. The array detects and integrates the mixed light signals, on different fields of view. Numerous image frames are recorded along the FM ("Frequency Modulation") period. Fourier transform taken on all the pixels establishes the detected object distance and gives the object 3D image in the field of view. Document WO00/55642 concerns an optical radar applied to three dimensional imaging. A modular light source sends a plurality of light signals on the object to be detected in three dimensional way. The response light signals are deviated by an optical system so that modularly aggregated corresponding sensors receive them. Document JP10262973 concerns an endoscope. It comprises a probe sending and receiving ultrasounds while it simultaneously rotates and moves forward and backward. The probe is placed at the endoscope end having an imaging optical device. The ultrasonic images are stored in a particular memory, and an image processor reconstructs the three dimensional image. Image processing is carried out depending on the reciprocal distance of the optical detector and the probe. The solutions of more generic interest disclosed in the following patent documents have to be noted. Document US5249046 concerns a "method and apparatus for three dimensional range resolving imaging". The apparatus uses Fresnel- Risley prisms and an arrangement of part of the frequencies emitted by the light source in times determined with respect to those of reflection, for avoiding a too extended sensor array when the two dimensional surface to be scanned is too extended. Document US6414746 concerns a "3-D imaging multiple target laser radar". The laser radar is used for producing 3D images of far objects - placed behind reflecting or absorbing penetrable barriers, as masks or darkening smoke. Document WO01/04576 concerns a "method for operating a laser scanner". The scanner laser is used for creating maps of building structures which will be then subject to subsequent processing, including comparison with previous maps. Document EP0680271 describes an endoscope capable to give three dimensional images through stereoscopy. To this end, two different optical channels are provided, along with related improved interaction optics. Thanks to this arrangement, image reconstruction from the ' observer is in this way made easier. All these proposals are as many specific solutions which do not provide a general method and apparatus for three dimensional reconstruction of scanned images, as necessary for example in endoscopy, in particular of colo-rectal, or in laparoscopy. The present endoscopic techniques, though on the one hand they offer analytic capabilities more and more detailed and they provide virtual three dimensional reconstructions obtained by the association of two or more diagnostic techniques, still nowadays they are not capable to provide an accurate three dimensional localization of the scanned anatomical parts. Consequently, still nowadays the classical reference points are used for the topographical description of possible anomalies identified during an endoscopic examination. It is therefore an object of the present invention to provide a method for 3D-reconstruction of images acquired through an electromagnetic or electro-optical device, in particular laser radar. It is also an object of the present invention to provide the apparatuses and instruments necessary for execution of the method object of the invention. It is also an object of the present invention to provide an endoscopic apparatus using the method object of the invention. It is subject matter of the present invention a method for 3D- reconstruction of surface images of objects, in particular endoscopic images, acquired by an apparatus comprising a detection electro-magnetic or electro-optical device, a scanning head connected to the detection device, the detection device detecting a set of points and being capable to supply a set of information per detected point comprising the aim direction of a scanning signal, the distance between the scanning head and the point, and the intensity / of the signal reflected by said point, the method being characterised in that it comprises the following steps: A. subdividing the set of points detected along time in a plurality of sub-sets ordered in time; B. for each sub-set of points according to step A, calculating a reference surface approximating the sub-set, and a reference system, and projecting the sub-set points onto the reference surface along with calculating the coordinates of the points due to projection; C. for each sub-set of points, determining possible zone of overlap between the sub-set and one or more sub-sets preceding in time, determination being based on the comparison of the intensities / for one or more points; D. in the case when overlap zones are found in step C, positioning the reference surfaces of overlapping sub-sets so that the overlap zones coincide; E. returning to the coordinates of the detected points of each sub-set with respect to its own reference system, starting from the point coordinates obtained with the projection of step B. According to the invention, step B may comprise sub-step: B.1 calculating a reference surface, choosing a type of surface from a predetermined set of surface types and adjusting it on the surface formed by the sub-set of points. Still according to the invention, step B may comprise sub-steps: B.2 choosing a reference system and a reference axis not lying on such surface and passing through the origin of the reference system, B.3 calculating the coordinates of the scanning head and of the detected points according to the new reference system and projecting the sub-set points onto the reference surface. Advantageously according to the invention, the set of reference axes of the sub-sets form a broken linear coordinate. According to the invention, in the case when the scanning head comprises a scanning optics apt to scan along two orthogonal directions and the scanning optics rotation frequencies are known, the aim direction of the scanning signal is advantageously obtained starting from the time coordinate of the detected points. Preferably according to the invention, the projection of step B.3 is done orthogonally to the reference surface. Still, according to the invention, it is preferable that the surface types of step B at least partially correspond to surfaces of bodies of revolution. Preferably according to the invention, the sub-sets of step A have size such that, in step B, the reconstructed surface is from time to time approximated by said surface type, the axis of the body of revolution being said reference axis. Preferably according to the invention, step C comprises determining points of each sub-set belonging to a predetermined topological feature of the examined object, and comparing the intensities / of the points of said topological feature. Still preferably according to the invention, the topological feature is the surface layer vascular system of a body cavity. Preferably according to the invention, the intensities / are related to the response of an optical signal in the green band. Still preferably according to the invention, the topological feature is made of crypts of surface biological tissue of an intestinal cavity. According to the invention, step C may comprise determining overlap of said zones by using a correlation index. Preferably according to the invention, the reference axis of step B.2 is obtained as straight line fitting the centroids of scanning sections corresponding to the points orthogonal to a given axis. Still preferably according to the invention, step B comprises sub-step: B.4 for each sub-set, positioning the arbitrary triad origin into the median point of the reference axis. Furthermore according to the invention, it is preferable that step B.3 comprises sub-step: B.3.1 determining position and attitude of the scanning head with respect to the reference system. Preferably according to the invention, step B.3 comprises sub- step: B.3.2 determining the position of the detected points with respect to the reference system, as sum of the detected related positions with the scanning head position with respect to the reference system. Still preferably according to the invention, step B.3 comprises sub-step: B.3.3 orthogonally projecting the points of the sub-set under consideration onto the related reference surface. Preferably according to the invention, in step A the sub-sets of detected points are further subdivided in a plurality of point portions ordered in time, still more preferably portions successive in the time order are at least two by two overlapping. Still preferably according to the invention, the overlap of portions is of at least 1 point along at least one of the two scanning orthogonal directions. According to the invention, step C may be performed by taking the portions of a sub-set and comparing each portion of this sub-set with one or more different sub-sets. According to the invention, each portion may be compared with the one preceding in the time order. Preferably according to the invention, the three dimensional reconstruction is made by processing the sub-sets starting from a first sub- set and proceeding with the successive sub-sets according to time order. Preferably according to the invention, in step B, for the first handled sub-set, the reference axis is obtained as straight line fitting the centroids of the scanning sections corresponding to orthogonal scanning directions of the scanning head scanning optics. Still preferably according to the invention, step C comprises sub-step: C.1 set as initial reference system, axis and surface of a sub-set processed after the first one, the corresponding reference system, axis and surface determined according to step B for the preceding sub-set. According to the invention, instead of step B.3.3, it may be performed step: B.3.4 projecting the portion under consideration onto the reference surface of the preceding sub-set. According to the invention, step C may comprise sub-step: C.2 comparing the projected portion under consideration with the preceding projected sub-set or a plurality of preceding projected sub-sets. According to the invention, step D may comprise sub-step: D.1 positioning the projected portion under consideration by using the zone of overlap with the preceding projected sub-set or with a plurality of preceding projected sub-sets. Advantageously according to the invention, step D comprises sub-step: D.2 calculating the scanning head position due to positioning of the handled portion, with respect to the reference system of the preceding sub-set. Advantageously according to the invention, step D comprises sub-step: D.3 calculating the position of the points of the portion in the new position, with respect to the reference system of the preceding sub-set. According to the invention, once all the portions of the sub-set under consideration have been handled, the reference surface and the reference axis may be calculated according to step B for the sub-set. Preferably according to the invention, the detection device is a laser radar. According to the invention, the detection device may also supply the relative speed vector of the acquired points, along the aim direction of the scanning optics. Preferably according to the invention, the detection device is a coherent laser radar. According to the invention, step B for the first sub-set under consideration may be performed by considering the sub-set portion by portion, the initial reference system and axis being corresponding to the scanning head position related to the first portion and to the axis of the same head, the scanning head position being considered as fixed for each portion. According to the invention, for each portion the speed vector of sum of all the speed vectors for each point may be calculated. Advantageously according to the invention, the scanning head position in the portion successive to the one under consideration is calculated by using said sum speed vector. According to the invention, in step C, in case of non effective comparison, the position of the scanning head and of the handled sub-set points may be determined by using said sum speed. According to the invention, after step E, the obtained point cloud may be converted into a continuous surface through a smoothing algorithm. According to the invention, said continuous surface may be displayed by means of a three dimensional display module. It is further specific subject matter of the present invention a computer program characterised in that it comprises code means adapted to execute, when running on a computer, the method according to what described above. It is still specific subject matter of the present invention a memory medium, readable by a computer, storing a program, characterised in that the program is the computer program according to what described above. Still, it is specific subject matter of the present invention an endoscopic apparatus, comprising a scanning head, a flexible unit, a detection electro-magnetic or electro-optical device, a processing unit, characterised in that the processing unit processes the detected data by using the method according to the invention. The present invention will be now described, by way of illustration and not by way of limitation, by particularly referring to the drawings of the enclosed figures, in which: figure 1 shows a continuous data flow coming from a scanning head; figure 2 shows the arrangement of the continuous data in subsets or "frames" and portions of sub-set or "slices" according to the invention; figure 3 shows the two slices of figure 2, separately; figure 4 shows overlapping slices related to a body particular; figure 5 shows the flow diagram of the initial scanning module in a first embodiment of the method for reconstruction of three dimensional images according to the invention; figure 6 shows a scanned surface, with coordinates relative to the scanning head; figure 7 shows the calculation of the centroids of the surface of figure 6, i.e. the first frame; figure 8 shows the reference cylinder overlapping the first frame, wherein the scanning head position is pointed out; figure 9 shows composition of the relative coordinates of a point with those of the scanning head with respect to with respect to the reference cylinder, for obtaining the absolute coordinates of the same point; figure 10 shows a particular of the projection of the points of the surface of the first frame onto the reference cylinder; figure 11 shows the flow diagram of the "slice" scanning module in the first embodiment of the method for reconstruction of three dimensional images according to the invention; figure 12 shows the scanning head with respect to a slice detected by it; figure 13 shows the scanning head and the slice of figure 12, oriented with respect to the reference cylinder of the previous frame; figure 14 shows the projection of the slice of figures 12 and 13 onto the reference cylinder at the top, and the overlap of the same slice with the previous frame at the bottom; figure 15 shows the slice of figures from 12 to 14 with respect to the present position of the scanning head; figure 16 shows the flow diagram of the reference correction module in the first embodiment of the method for reconstruction of three dimensional images according to the invention; figure 17 shows the centroids of the new frame and the calculation of the new reference cylinder; figure 18 shows two overlapping slices in the various processing steps of the method according to the invention; figure 19 shows the flow diagram of the initial scanning module in a second embodiment of the method for reconstruction of three dimensional images according to the invention; figure 20 shows the flow diagram of the "slice" scanning module in the second embodiment of the method for reconstruction of three dimensional images according to the invention. The method according to the present invention is applicable to any examination of even partial cavities. However, it is particularly intended for endoscopic examination of animal or human body cavities. In fact, in this case the method solves problems otherwise unsolvable nowadays except at the cost of making the examination complex and more dangerous. Consequently, in the following, we will refer only to endoscopy of body cavities, in particular la recto-colonoscopy and laparoscopy. In case of 3D recto-colonoscopy, it is deemed that the most proper topological features for univocal recognition of a tissue portion correspond to the vascular map of the examined tissues (similarly to human eye retina scanning, the configuration or "pattern" formed by the vascular network is stable and univocal from point to point). Its acquisition is normally possible by projecting a (near) infrared or visible low intensity light beam and picking up the reflected image by means of a device similar to a retinascope placed at a distance of three centimetres at most. To this end, the method and the apparatus according to the invention use the already known laser radar, for example the one disclosed in patent US 5,621 ,514 or in patent US 6,469,778. An endoscopic apparatus for recto-colonoscopy or laparoscopy according to the invention is made of a unit comprising a laser radar, a processing unit, a flexible unit that has to be inserted into the cavity, comprising a fibre optics, and a scanning unit or head comprising at least a laser radar scanning optics and possible other sensors (for example an optical charge coupled device or CCD). The scanning unit or head, preferably in the shape of a small cylinder, will be housed at the end of a flexible body very similar to those of the present endoscopes and it will comprise one or more sensors or scanning optics (such for example camera, spectrometer, microscope, mirror system). Housings for the passage of fibre optics and electric and mechanical cables necessary to sensor or scanning optics operation will be provided within such unit. Through the fibre optics, radiations reflected by the examined cavity surfaces are routed to the processing unit, which will be so capable to measure the picked up radiations intensity and the delay with respect to the transmission. The delay will give a distance sum of the whole optical path (the known part of which comprises the whole length of the connecting fibre optics). In case of a recto-colonoscopy, an endoscope length of about 4-5 metres and a distance between the scanning head and the colon walls ranging from 0 to 3 cm for an overall optical path of about 8-10 metres may be assumed: present Coherent Laser Radar commercial technologies already ensure an error less than one tenth of millimetre in case of measurements of distances of about two metres. Detection of the position of a moving object from an also moving scanning head is possible only when the typical frequencies of the two motions are decoupled. In the present application example, referring to live tissues motions, great peristalsis (3-4 movements per day lasting about 30 seconds from oesophagus to rectum) and micro-peristalsis (3-4 movements per second however nullified when the walls are stretched by air) have to be considered, while, referring to the scanning head motion, oscillations round the temporary fulcrum have to be taken in account. Thus, since only oscillation frequencies of the scanning head, within surfaces which may be assumed as fixed, have to be considered, scanning parameters will be adjusted so as to ensure the requested frequency decoupling, or it may more simply be calculate the time interval within which the scanning head and the surrounding surfaces may be considered as reciprocally "still" and, at the expiry of the interval, the navigation techniques described in the following have to be adopted for referring the detected points and integrating the resulting surface (slice or frame) with the previous ones. The scanning head rotates round its own axis for carrying out the scanning along a direction. However, it has to be noted that the scanning head cylinder is a moving rigid body, within the body cavity under examination, and hence there may be an additional rotation, which may be positively added to or subtracted from the first one. In other words, if the axis of the cylinder were still, the rotation of this round its own axis would cause that the scanned angle would be of 360 degrees. Instead, if, besides the revolution rotation, the rotation due to its rigid body movement is also added, then scanned angle could be smaller or larger (i.e. with an overlap). This problem of rotations is not at all related to the particular embodiment of the scanning head just introduced, since it is similarly present for any kind of scanning head. As it will be seen, the method according to the invention is apt to overcome also this difficulty. Another rotation in the scanning head may be that of the scanning optics, preferably rotating or moving with axis orthogonal to the rotation of the same cylinder. With this rotation, the tissue is scanned along a direction parallel to the scanning head axis. A vertical 360 degrees scanning of the cavity from the scanning optics (i.e., when the cavity is approximated to a right cylindrical pipe, along its axis) has not to necessarily correspond to the whole rotation of the cylinder. It is rather preferable to arrange the scanning head so as to have references or marks of scanning start and end. The reason for this choice are substantially two: first, when the axis direction is reached, too far parts are detected whereby the reconstruction of the gradually scanned zone becomes hard; secondly, without data operating as cyclic reference it would be hard to suitably rearrange the various pixels, as specified below. Moreover, this is done in order to avoid an excessive radiance of the examined cavity walls, and also to have a scanned point density ranging within well defined and sufficiently restricted limits (since distances do not vary excessively). The laser beam coming from the endoscope axis will be so capable to be aimed by the scanning optics so that tissues are scanned along the direction parallel to the axis of the same cylinder. The simultaneous rotation of the same cylinder will enable the scanning of a helicoidal band of the surrounding surfaces. Actually, in some cases it could be useful to watch also along the direction of the cylinder axis, for example in laparoscopy. Still, scanning start and end references could be eliminated through a rocking mirror or galvanometric elements scanning optics capable to provide the detection angle with sufficient precision. The scanning head optics continually detects data (data of intervals beyond the aforesaid references will be automatically eliminated), obviously with respect to its reference system that is herein defined as "relative". Hence, from the relative point of view, the data processing unit, to which the scanning head is connected, receives a continuous data flow. With regard to the scanning head, it is still to be noted that it could comprise a charge coupled device (CCD) watching for example along the direction of the cylinder axis. From the analytical point of view, the processing unit sends the laser pulse through the fibre optics at a time t and receives from the same means the response of intensity / at a time t + τ. For example, according to the Coherent Laser Radar technique, the signal is subjected to an optical and electronic processing, up to obtaining the delay r and the frequency spectrum of the detected elementary area: taking the whole optical path into account, the measured distance from the scanning unit is obtained from the first one, while the mean value of the frequency spectrum gives the frequency shift and hence the leaving/approaching speed v of the same elementary area with respect to the scanning unit. The processing unit hence receives a data string and associates the angles of elevation θ and azimuth φ of the scanning mirror with respect to the scanning unit reference axes, as well as the angles of elevation θs and azimuth φs of the upper scanning optics with respect to the same reference axes, with the detection time t Hence, the process described above gives, for each laser pulse sent at the time t, the values /, r, v, θ, φ, θs, φs related to the elementary area or "pixel" reached by the pulse. The process is repeated for each pixel, but in order to be able to represent the detected surfaces as a spatial continuum, it is necessary to know with high accuracy the position and orientation of the scanning unit at each time interval t with respect to an independent reference system, wherein the surfaces may be represented as a whole. Hence, the detection from the scanning head will correspond to a continuous data flow that will arrive at the external processing unit and will be processed by this. Processing will have, among others, the task of calculating and tracking with extreme accuracy the position of the scanning unit within the examined organ, in order not to degrade the localization precision that is typical for the (Coherent) Laser Radar technique. With reference to figure 1 , the continuous data flow coming from the scanning head is arranged in parts separated by the scanning end mark, corresponding to the positions 1, p, p+1, 2p, 2p+1,... , np, with n and p integers. Hence, there are p-2 pixels for each vertical scanning, the scanning optics (for example polygonal as described above) carrying out the scanning by vertically moving while the cylinder rotates. For each scanned pixel, there is a data record containing physical parameters, as for example: scanning instant t measured delay/distance r, intensity /, speed vector v, colour values RGB (always detected through the laser radar scanning optics). Intensity / is a fundamental parameter for endoscopy of body cavities. In fact, as already observed, presently available techniques do not allow examining large cavities, because of the progressive increase of the position error, while a topological approach is adopted according to the invention, in particular recognising the topological feature that better adapts to body examinations: the vascular configuration or "pattern". Intensity / of the reflected light and its frequency spectrum, in particular due to green light radiance, allow identifying the vascular pattern that is univocal for every living being having it, in particular for human beings. In addition, it may be provided a signal from an optical device, for example a charge coupled one or CCD, suitably fixable to or integratable into the scanning head. In the following, two embodiments of the method according to the invention will be described, wherein the aforesaid reference surfaces used for the reconstruction are all cylindrical. This is purely a descriptive choice suggested by the complexity of the method, but it should be understood that any other reference surface is advantageously usable. The data pre-processing module first analyses together / and r in order to identify the marks of end vertical scanning as defined above, in figure pixels 1, p, p+1, 2p, 2p+1, ... , np, np+1,.... At the same time, azimuth angular values for each pixel are accessed. Starting from these, it will be possible to assign a relative elevation angle (with respect to the plane perpendicular to the scanning head axis) θr and, from their specific position within the scanning sequence as said, the corresponding azimuth angle (relative to the conventional angle "0") φΛto each intermediate record. By differentiating the thicknesses of the edge 105 of figure 2, i.e. by giving a differentiated relief surface, the marks of end scanning will be differentiated with azimuth angle variation so as to respond with univocal and known distance values. Colour differentiation may be also added to thickness differentiation, so that there is a response differentiated as to intensity. In such way, an array will be obtained containing values function of the angular coordinates. As shown in figure 2, at the end of a complete scanning of the two angles (a "complete turn" of the external cylinder corresponds, without taking the aforesaid additional rigid body rotation into account, to a rotation of 360, i.e. 0 < φr < 2π, while for each complete vertical scanning the elevation angle covers a sector: θmιn < θr < θmax), it will be obtained a "frame" of pixels, for example of 118 horizontal pixels by 68 vertical pixels. The frames are logically subdivided in "slices" of n-(p-2) pixels, partially overlapping for q pixels. For example, the overlapping zone may be of 1-2 columns of pixels (in figure 3, pixels correspond to two columns). Overlap of slices is essential in the method according to the invention. By using the same data, it allows a continuous test of the scanning head position to be performed and, hence, as it will be seen, the scanning head position within the body cavity to be reconstructed with sufficient approximation. Division in slices is made in order to permit the scanning head to be considered, with sufficient approximation, as still with respect to a data set (exactly the slice). It has to be here noted that two slices may be certainly logically overlapping, but they may "physically" form an angle between them. In fact, while (continually) passing from one slice to the other, a notable scanning head movement could have occurred, whereby distances to the body cavity wall could have change. But this will be corrected in the method according to the invention. Figure 4 shows an image of a body particular with four slices (of which only the two top ones show the points corresponding to the pixels) both vertically and horizontally overlapping by two rows of pixels. Once the data flow is subdivided as described (it has to be noted that, in general, this may be continually done), and the first scanned frame is obtained along with its slices, the initial scanning module follows, the flow diagram of which is shown in figure 5. In such flow diagram, both main operations and secondary ones, related to data memorising and recovering, are represented.
However, in the following, only the main ones will be discussed, since the others are immediately understandable for the skilled in the art. The same will be done for the next flow diagrams. The initial scanning module operations to be performed are the following. In step 1 , relative spatial coordinates />, θr and φr (respectively: position, elevation and azimuth) for each pixel of a complete frame are recovered (or detected if the analysis is continually made during endoscopic examination). Such relative coordinates, together with the cavity detected inner surface, are shown in figure 6. In step 2, centroids of successive scanning sections are calculated. Scanning sections are the rows of the pixel array of figure 3, or, as it will be seen in the slice scanning module, the pixels of iso-level with respect to the reference cylinder axis. At the beginning, taking the array rows as described in figure 2, the reference cylinder is practically approximated with what detected in the initial length, since there is not yet a reference cylinder. In step 3, the reference cylinder axis for the scanned cavity initial length is calculated. Connection of the centroids is a three dimensional skew arc, the linearization (linear fitting) of which gives the reference cylinder axis. In step 4, the mean radius Rcι of the reference cylinder is calculated (the radius is orthogonal to the axis defined in the preceding step). Figure 7 show centroid calculation and successive linearization, as well as the mean radius calculation. In step 5, coordinates of the initial point xcu, y , zcι-u end point Xd-f, Yc , Zc and median point xc1, ycι, zcι of the axis of the reference cylinder initial length are calculated. In step 6, it is assumed as new reference the triad having origin equal to xcι, yc1, zcι (thus setting these coordinates as null) and attitude Θc7, Φd, Ψcι (Euler angles) coinciding with that of the reference cylinder (the reference attitude is arbitrarily chosen along the axis and these angles are set equal to zero). In step 7, the scanning head position with respect to the newly established reference system is calculated. The first position and the first attitude of the scanning head (rpϊ-7, θpι-ι, φpι-ι, ΘPι-ι, Φpι-ι, Ψpi-i) with respect to the new reference are calculated. Figure 8 shows both the reference cylinder (in short dashes) and the detected surface (in continuous line). Also, the scanning head position, with the angles related to its attitude with respect to the established reference, is shown. At this point, in successive step 8, flow passes to the pixels and the "absolute" coordinates (with relation to the chosen reference) ra-ι, θaι and φa1 of each detected point (pixel) are calculated, as vector sum of position and attitude of the scanning head rpι-ι, θpι-ι, φpι-ι, Θ ι-ι, ΦPι-ι, Ψpi.-i with the relative coordinates rn θr, cpr. In figure 9 it is seen how the absolute coordinates ("a") of the pixels result from the vector sum of the scanning head coordinates ("s") with respect to the reference cylinder with the relative coordinates ("r") of the detected pixel. In step 9, points of coordinates raι, θaι, and <paι are orthogonally projected onto the reference cylinder (xC7, ycι, zcι, ΘC7, Φd, Ψd and radius Rd), so that the distance of all the points from this is obviously equal to the mean radius Rc1. In figure 10, an example of this orthogonal projection is shown. Following the projection, resolution (pixel density) results different from the starting one. It is then possible to use algorithms of interpolation obtaining a set of points wherever homogeneously dense. The records referred to the cylindrical coordinates sc-πf and φcf
("broken" linear coordinate and angle related to axis and orientation of the reference cylinder, the curvilinear coordinate being the coordinate of the broken line resulting from the linearizations made for the various frames) are then obtained. Each record is related to a tempo t and comprises the value of intensity / (that gives an account of the haemoglobin content, thus of the vascular pattern) and it is univocally associated to the reference and to the absolute positions. Figure 11 shows the slice scanning module, wherein the following steps are performed. During the first step, i.e. step 10 of the overall process, the spatial relative coordinates rr, θr, φr for a whole slice are detected (or extracted from a memory buffer). It has to be here noted that, differently from the first module, flow proceeds slice by slice instead of handling a new frame as a whole. This greatly simplifies computations and permits a better approximation in determining the scanning head position. In slice by slice handling, the scanning head is assumed as fixed during acquisition of a same slice, an example of detected slice being shown in figure 12. In step 11 , the spatial coordinates related to the scanning head position (rp1.ι, θpι-ι, φpι-ι, 0Pι-ι, ΦPι-ι, Ψpi-i) with respect to the reference cylinder already determined in the initial scanning module (xcι, y ι, zcι, Θc7-,
Φd, Ψd, Rci) are vectorially summed, obtaining the absolute coordinates rβ7l θaι, φaι for each point. In figure 13 it is shown how the slice appears to be positioned taking the scanning head position in the preceding frame. It is clear that the slice appears to be not correctly positioned. In step 12, the points of coordinates ra1, θaι and <paι are projected onto the same reference cylinder above. Practically, for the new slice, the reference cylinder already found with the described procedure related to the first frame is still used. The cylindrical coordinates sc. rov and φc.prov (broken linear coordinate and angle related to the axis and orientation of the reference cylinder) are then calculated, and the records corresponding to each pixel are associated. Each record is related to a time t and comprises the value of intensity / (which is function of the haemoglobin content, hence of the vascular pattern) and it is associated to the reference and to the absolute positions. In figure 14, at the top, the projection procedure in this case may be observed. At the bottom, the overlap zone of old and new slices are pointed out on the preceding frame cylinder once re-positioned the new one. At this point, in step 13, the vascular patterns of the obtained slice are compared with those of the already projected cylinder portion (starting from the closest estimated point). After the comparison, flow will return to the "absolute" coordinates. The comparison, or "matching", may be either "ineffective" or "effective": a matching is effective when there is an overlap of new and old slices, while it is ineffective when there is no overlap; the overlap is evaluated through a correlation index according to standard techniques. The matching serves for checking whether the scanning head has moved, and in the positive the overlapping points are taken and the slice is placed again re-calculating the scanning head position. The case of ineffective matching may occur when the slice is tilted with respect to the frame or when the whole frame is not overlapping or contiguous to the previously detected/handled one. In case of ineffective matching, in a step 14 the slice is "set aside" (however it remains stored in a suitable cache/database) until the completion of the frame, and flow returns to step 10 for acquiring a new slice. Instead, if it was the last slice in the frame, then, in a step 20, flow returns to the initial scanning block. In case of effective matching, in step 15, three or more corresponding not aligned points are selected in the two patterns (of reference and of the slice) and their respective cylindrical coordinates (scf and >c-f7r- with sc-Prov and φc.prov) are compared. The relative position vectors of the points of the slice to be referred (r θr, φr) are then added to the absolute position vectors of the points of the reference pattern (ra7-, θa1, φa1), obtaining updated position and attitude of the scanning head (rpι.2, θp1-2, <Pp1-2, ΘP1-2, ΦP1-2, Ψp1-2)- The position and the attitude of the scanning head are thus calculated with respect to the present reference cylinder (i.e. to the present part of the surface union of the successive reference cylinders). Flow simultaneously proceeds to the integration of the just handled pattern (pattern cache), i.e. the handled slice is added to the cavity overall map, obtaining at this point one frame (the starting one) plus one slice. If flow then returns to the not projected coordinates, there is for example a situation as the one of figure 15. In the case when, for the slice scanning module, it is the last slice of the frame, or the same slice has been already set aside once, matching is reached as above. In case of ineffective matching, flow starts again from the initial scanning block and the frame is memorised according to a provisional reference cylinder unrelated to the previous one. This is done in order to re-use it later when the scanning head will acquire intermediate frames, partially overlapping this analysed one. In case of effective matching, three or more corresponding not aligned points are selected in the two pattern (of reference and of the slice), of which the respective cylindrical coordinates (sc-nf and φcf with Sc-prov and φc-Prov) are already known. The relative position vectors of the points of the slice to be referred (rr, θr, φr) are then added to the absolute position vectors of the points of the reference pattern (ra1, θaι, φaι), obtaining updated position and attitude of the scanning head (rp1.n, θpι.n, φPι-π, Qpi-n, ΦPι-n, ΨPι-n). Being now the last slice of the frame, flow passes to the reference correction module, the flow diagram of which is shown in figure 16. This module re-calculates the new reference cylinder and then the new axis and median point, the new attitude of the scanning head. In the first step of this module, i.e. in step 22 of the overall process, the centroid of the sections of the just completed frame are calculated, starting from the already calculated absolute position of the point. Calculation of the centroids is made for the scanning sections of iso- level with respect to the axis of the preceding reference cylinder. Starting from this, the union of the centroids is taken, which is a three dimensional skew arc, and it is linearised obtaining the axis of the new length of the reference cylinder characterised by the start and end coordinates (xc2-/, yC2-u
Figure imgf000024_0001
and xc , yc , zc ) and by the attitude (Θc2, In step 24, the mean radius Rc∑ of the length of the reference cylinder is calculated. These last two steps are shown in figure 17, wherein it may be seen the cavity inner surface forming as union of successive frames. In step 25, the coordinates of the median and end points of the axis of the second length of the reference cylinder (respectively xc2, yc2, Zc2, Xc2-t, yC2-f and zc2-r-) are calculated. In step 26, triad having origin equal to xc2, yc∑ and zc2 and attitude coinciding with those of the reference cylinder ΘC2, Φc∑, Ψc2- is assumed as reference (storing it in a suitable cache/database). During step 27, the already memorised points of the last frame are again projected onto the corrected reference cylinder (xc2, yc2, zc2, ΘC2, Φc∑, Ψc2 and Rc ). Records referred to the cylindrical coordinates sc-πf and cpcf (curvilinear coordinate and angle related to axis and orientation of the reference cylinder) are obtained. Each record is related to a time t and comprises the value of intensity / (that is function of the haemoglobin content, hence of the vascular pattern) and it is associated to the reference and to the absolute positions. In a successive step 28, it is calculated the transformation of the recorded position and attitude of the scanning head into position and attitude in the new reference (from: rp1.n, θpι.n, φpι-n, Θpι-n, ΦPι-n, ΨPι-n to: rP2-1, θP2-1, <PP2-1, ΘP2-1, ΦP2-1, Ψp2-l)- The subscript "p2" stands for second position, since it is the position with respect to the second frame. In figure 18 four reconstructions of the scanning head are displayed in two positions corresponding to two slices, showing the last described steps. From right to left, there are the scanning head when still and the distance that differs, then, by way of illustration, the non-overlap zone is temporarily eliminated. Subsequently, the overlapping zones overlap calculating the positions of the scanning head that rigidly follows them (hence the surfaces are now coincident and the scanning head positions differ), and data are finally collected (the non-overlap zone is again displayed) with the scanning head in two different positions. This has the advantage of tracking the scanning head movement and of constructing slice by slice the surface union of the frames, which corresponds to the scanned surface. The reproduction precision depends on the frame size, and it may be hence chosen in function of the available computation capacity. The method shown in figure 18 may be also inversely performed. The fact that the scanning head position may be tracked slice by slice is important since it may also used for referring images coming from a possible CCD. Flow now passes to the slice scanning module, until the last frame is reached. The last frame may also be a frame that has been skipped, or even the lastly detected frame when the endoscope is extracted from the examined cavity. In this regard, it has to be observed that clinical procedure may provide two successive scanning: the first one at the insertion of the endoscope, typically slower, and the second one, faster, during extraction of the same. In both cases, it will be taken account of quantities of detected points useful for processing frames partially overlapping one among them, so as to exploit the datum of intensity / in order to recognise the vascular pattern of each frame and to apply through it the topological approach technique to the overlapping part, consequently obtaining a unique and coherent 3D map. Moreover, at the moment of the extraction of the 3D endoscope, scanning repetition in the opposite direction will ensure, through the application of map matching technique, the completion of detection of the tissue portions missed at the first passage because of the not complete stretch of the colon inner walls, or because they were kept hidden by the thin framework of the scanning unit or because the excessive tilt and/or speed of the scanning head. Also, I case of a patient already subject in the past to the same type of examination, the map matching technique may be already used starting from the first scanning and in addition it will be possible to recognise and couple the present reconstruction to the "historical" one, enabling one of the system fundamental functions: discovering new tissue degenerations and monitoring the same along time. In a further embodiment of the method according to the invention, the detected speed v is available, which is exploited for simplifying some of the aforesaid steps. Speed detection may be for example obtained by means of a Coherent Laser Radar, while it is not obtainable through a simple Laser Radar. Figure 19 shows the flow diagram of the initial scanning module of the method according to the invention in case of use of the speed information. With respect to the diagram of figure 5, referring to the initial scanning module as well, some differences have to be noted. First of all, in step 1 , it is not required that all the points of a frame are acquired, rather only one slice is acquired. The scanning head starting position is assumed as reference zero point, instead of the median point of the cylinder as in the case of the preceding embodiment. At this point, in step I bis, all the speeds of the acquired slice are vectorially summed. Assuming, as it is reasonable to be, that the slice is still during acquisition, the result of this vector sum is actually the scanning head speed. From this speed, by using the value of the slice acquisition time, flow will pass, in step 1ter, to the scanning head position with respect to the successive slice. Proceeding hen with the successive slice, during step I bis it will be possible to calculate the scanning head correct position and hence the absolute coordinates of the detected points. It is clear that this step will give a null result for the first slice, since there is no preceding slice from which the scanning head speed could be extracted. Once the aforesaid handling of all the slices of a frame is terminated, flow passes to step 2, wherein the centroids of the scanning sections orthogonal to the axis calculated for the preceding frame are calculated. Successive steps 3 to 9 are identical to those of the other described embodiment (see figure 5). Turning now to the slice scanning module, the flow diagram of which is that of figure 20, it is observed that steps 10 to 13 remain unchanged with respect to those of the other described embodiment (see figure 11). In both the described embodiments, matching may occur between both slice and frame (hence with a bottom overlap zone in the slice) and successive slices (hence with a side overlap zone in the slice). If matching is effective, in step 15-bis the scanning head position is corrected by using the same procedure of the previously described embodiment, possibly integrated with the composition of the speed vectors (for example with a mean between the two determinations). If matching is ineffective, the slice is not set aside into the cache, as in figure 11 , but the scanning head position is corrected in step 15-ter with the described speed method and flow then proceeds to the correction of the projected pattern and to the correction of the absolute position of the points. In the case when it is the last acquired slice of the frame under handling, flow passes to the reference correction module, that is identical to that already described referring to figure 16; in the case when it is not the last slice of the frame, flow begins again the loop by handling the successive slice. The just described method gives a body cavity image that, although it is not referred with respect to an external reference system, for example that of the laboratory, is internally referred, i.e. the various frames are referred with desired approximation one with respect to the other. Other approaches try to refer the scanning head from the outside of the body, through radiations, but in any case they cannot obtain the colon inner map except by performing a procedure permitting inner detection. On the other hand, from a medical point of view, it is important to watch the cavity surface and to know where it is, without .paying attention that all the portions of such surface are correctly and reciprocally proportioned, but taking care only that the medical reference points are reproduced with sufficient precision (for example the intestine curves are correctly reproduced with the method according to the invention). It has to be also noted that usual approaches use high power radiations, which could cause damages to the examined tissues, while the method according to the invention use low power, hence harmless, pulsed lasers. At the end of the described method, the processed pixels are sent to a 3D display module which displays them on a screen by interpolating them. The present invention thus consists in a new system of recording three dimensional images, capable to detect from realities and to digitally memorise the localization and the spatial distribution of a dense cloud of points of the inner walls of organs under endoscopic examination, so as to allow carrying out a customised three dimensional model of the scanned anatomical cavity as a whole, and the next 3D display. Digital images corresponding to the same tissues, simultaneously detected with other desired techniques, may be overlapped onto such three dimensional model. Essentially, the expected advantages of the use of the 3D endoscope are connected to the information increase obtained in passing from the detected representation of hollow organs to their topographical representation (reference), as well as to their delicate spatial and volumetric measurement. The exact spatial knowledge of whatever anatomical point and the consequent memorization offer multiple advantages. Availability of a digital 3D geo-referred representation of an organ will allow the doctor to have a customised map of the organ, to carefully examine it by navigating within the 3D reconstruction with interactive modes and hence to univocally identify the localization of a potentially pathological formation, and he will be able to measure its area and volume and to monitor along time its evolution and growth. The same techniques of automatic recognition of changes with respect to data present on file - also called "change detection" techniques - will be usable, and through them the system will immediately point out the parts which have changed, letting the doctor analyse in detail the concerned parts, so also increasing the examination speed and, hence, reducing the patient discomfort, thanks to memory storing of the whole 3D reconstruction. In prospect, such system opens the door to a semi-automatic remote diagnostic, by supplying a technological platform to a new generation of intelligent instruments, capable to know their own position and the one of the surrounding environment. The conceived system uses capabilities of the Coherent Laser Radar technology illustrated above for fast measuring position and speed of the surrounding "objects" in a new implementation such as to permit a faithful topographical, spatial and volumetric representation of the observed reality. The applications of the endoscopic apparatus and the method according to the invention may be for example: Endoscopy, Rectal- colonoscopy, Oesophagus-Gastro-Duodenoscopy, Bronchoscopy, Otoscopy, Rhinoscopy, Laryngoscopy, exploratory Laparoscopy, Mapping and volumetric detection of body surface, as Moles, substance Escape (burns, injuries, or else), body Volume. The method and the 3D endoscope according to the present invention is not necessarily proposed for replacing the present-day used techniques, but also for supplying a modular platform capable to integrate images detected through the present-day and future techniques with congruent spatial information, precisely locating the area under analysis in a way repeatable along time. In such way, it is possible to exponentially increase available information quantity and readability of the same: let's just think about which information increase is obtained in the passage from a plane display of an aerial photography - even if having an optimum definition - to its 3D representation thanks to wrapping of the same on an ground elevation model faithful to the reality. The method according to the invention may be integrated with a
"beacon" (transmitter of echo or radio pulses trackable by receivers placed outside the body). However, an hybrid solution would be obtained, needing both external and internal detection, hence a not immediately usable and in any case expensive technique. The preferred embodiments have been above described and some modifications of this invention have been suggested, but it should be understood that those skilled in the art can make variations and changes, without so departing from the related scope of protection, as defined by the following claims.

Claims

1. Method for 3D-reconstruction of surface images of objects, in particular endoscopic images, acquired by an apparatus comprising a detection electro-magnetic or electro-optical device, a scanning head connected to the detection device, the detection device detecting a set of points and being capable to supply a set of information per detected point comprising the aim direction of a scanning signal, the distance between the scanning head and the point, and the intensity / of the signal reflected by said point, the method being characterised in that it comprises the following steps: A. subdividing the set of points detected along time in a plurality of sub-sets ordered in time; B. for each sub-set of points according to step A, calculating a reference surface approximating the sub-set, and a reference system, and projecting the sub-set points onto the reference surface along with calculating the coordinates of the points due to projection; C. for each sub-set of points, determining possible zone of overlap between the sub-set and one or more sub-sets preceding in time, determination being based on the comparison of the intensities / for one or more points; D. in the case when overlap zones are found in step C, positioning the reference surfaces of overlapping sub-sets so that the overlap zones coincide; E. returning to the coordinates of the detected points of each sub-set with respect to its own reference system, starting from the point coordinates obtained with the projection of step B. 2. Method according to claim 1 , characterised in that step B comprises sub-step: B.1 calculating a reference surface, choosing a type of surface from a predetermined set of surface types and adjusting it on the surface formed by the sub-set of points. 3. Method according to claim 1 or 2, characterised in that step B comprises sub-steps: B.
2 choosing a reference system and a reference axis not lying on such surface and passing through the origin of the reference system, B.
3 calculating the coordinates of the scanning head and of the detected points according to the new reference system and projecting the sub-set points onto the reference surface.
4. Method according to claim 3, characterised in that the set of reference axes of the sub-sets form a broken linear coordinate.
5. Method according to any one of claims 1 to 4, characterised in that, in the case when the scanning head comprises a scanning optics apt to scan along two orthogonal directions and the scanning optics rotation frequencies are known, the aim direction of the scanning signal is obtained starting from the time coordinate of the detected points.
6. Method according to any one of claims 1 to 5, characterised in that the projection of step B.3 is done orthogonally to the reference surface.
7. Method according to any one of claims 2 to 6, when depending on claim 2, characterised in that the surface types of step B at least partially correspond to surfaces of bodies of revolution.
8. Method according to claim 7, characterised in that the subsets of step A have size such that, in step B, the reconstructed surface is from time to time approximated by said surface type, the axis of the body of revolution being said reference axis.
9. Method according to any one of claims 1 to 8, characterised in that step C comprises determining points of each sub-set belonging to a predetermined topological feature of the examined object, and comparing the intensities / of the points of said topological feature.
10. Method according to claim 9, characterised in that the topological feature is the surface layer vascular system of a body cavity.
11. Method according to claim 10, characterised in that the intensities / are related to the response of an optical signal in the green band.
12. Method according to claim 9, characterised in that the topological feature is made of crypts of surface biological tissue of an intestinal cavity.
13. Method according to any one of preceding claims, characterised in that step C comprises determining overlap of said zones by using a correlation index.
14. Method according to any one of preceding claims, characterised in that the reference axis of step B.2 is obtained as straight line fitting the centroids of scanning sections corresponding to the points orthogonal to a given axis.
15. Method according to any one of preceding claims, when depending on claim 3, characterised in that step B comprises sub-step: B.4 for each sub-set, positioning the arbitrary triad origin into the median point of the reference axis.
16. Method according to any one of preceding claims, when depending on claim 3, characterised in that step B.3 comprises sub-step: B.3.1 determining position and attitude of the scanning head with respect to the reference system.
17. Method according to claim 16, characterised in that step B.3 comprises sub-step: B.3.2 determining the position of the detected points with respect to the reference system, as sum of the detected related positions with the scanning head position with respect to the reference system.
18. Method according to claim 17, characterised in that step B.3 comprises sub-step: B.3.3 orthogonally projecting the points of the sub-set under consideration onto the related reference surface.
19. Method according to any one of preceding claims, characterised in that in step A the sub-sets of detected points are further subdivided in a plurality of point portions ordered in time.
20. Method according to claim 19, characterised in that portions successive in the time order are at least two by two overlapping.
21. Method according to claim 20, when depending on claim 5, characterised in that the overlap of portions is of at least 1 point along at least one of the two scanning orthogonal directions.
22. Method according to claim 19 or 20, characterised in that step C is performed by taking the portions of a sub-set and comparing each portion of this sub-set with one or more different sub-sets.
23. Method according to claim 22, characterised in that each portion is compared with the one preceding in the time order.
24. Method according to any one of preceding claims, characterised in that the three dimensional reconstruction is made by processing the sub-sets starting from a first sub-set and proceeding with the successive sub-sets according to time order.
25. Method according to any one of preceding claims, when depending on claims 5 and 14, characterised in that in step B, for the first handled sub-set, the reference axis is obtained as straight line fitting the centroids of the scanning sections corresponding to orthogonal scanning directions of the scanning head scanning optics.
26. Method according to claim 25, characterised in that step C comprises sub-step: C.1 set as initial reference system, axis and surface of a sub-set processed after the first one, the corresponding reference system, axis and surface determined according to step B for the preceding sub-set.
27. Method according to claim 17 and any one of claims 23 to 26, characterised in that it is performed the step: B.3.4 projecting the portion under consideration onto the reference surface of the preceding sub-set.
28. Method according to claim 27, characterised in that step C comprises sub-step: C.2 comparing the projected portion under consideration with the preceding projected sub-set or a plurality of preceding projected sub-sets.
29. Method according to claim 28, characterised in that step D comprises sub-step: D.1 positioning the projected portion under consideration by using the zone of overlap with the preceding projected sub-set or with a plurality of preceding projected sub-sets.
30. Method according to claim 29, characterised in that step D comprises sub-step: D.2 calculating the scanning head position due to positioning of the handled portion, with respect to the reference system of the preceding sub-set.
31. Method according to claim 30, characterised in that step D comprises sub-step: D.3 calculating the position of the points of the portion in the new position, with respect to the reference system of the preceding sub-set.
32. Method according to any one of claims 27 to 31 , characterised in that, once all the portions of the sub-set under consideration have been handled, the reference surface and the reference axis are calculated according to step B for the sub-set.
33. Method according to any one of preceding claims, characterised in that the detection device is a laser radar.
34. Method according to any one of preceding claims, characterised in that the detection device also supplies the relative speed vector of the acquired points, along the direction of the scanning optics.
35. Method according to claim 34, characterised in that the detection device is a coherent laser radar.
36. Method according to claim 34 or 35, characterised in that step B for the first sub-set under consideration is performed by considering the sub-set portion by portion, the initial reference system and axis being corresponding to the scanning head position related to the first portion and to the axis of the same head, the scanning head position being considered as fixed for each portion.
37. Method according to claim 36, characterised in that for each portion the speed vector of sum of all the speed vectors for each point is calculated.
38. Method according to claim 37, characterised in that the scanning head position in the portion successive to the one under consideration is calculated by using said sum speed vector.
39. Method according to claim 38, when depending on claim 13, characterised in that in step C, in case of non effective comparison, the position of the scanning head and of the handled sub-set points are determined by using said sum speed.
40. Method according to any one of preceding claims, characterised in that, after step E, the obtained point cloud is converted into a continuous surface through a smoothing algorithm.
41. Method according to claim 40, characterised in that said continuous surface is displayed by means of a three dimensional display module.
42. Computer program characterised in that it comprises code means adapted to execute, when running on a computer, the method according to any one of claims 1 to 41.
43. Memory medium, readable by a computer, storing a program, characterised in that the program is the computer program according to claim 42.
44. Endoscopic apparatus, comprising a scanning head, a flexible unit, a detection electro-magnetic or electro-optical device, a processing unit, characterised in that the processing unit processes the detected data by using the method according to any one of claims 1 to 43.
PCT/IT2004/000379 2003-07-23 2004-07-09 Method for 3d-reconstruction of images acquired by means of laser radar, and endoscopic apparatus using such method WO2005008283A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ITRM2003A000360 2003-07-23
IT000360A ITRM20030360A1 (en) 2003-07-23 2003-07-23 PROCEDURE OF THREE-DIMENSIONAL RECONSTRUCTION OF IMAGES

Publications (2)

Publication Number Publication Date
WO2005008283A1 true WO2005008283A1 (en) 2005-01-27
WO2005008283B1 WO2005008283B1 (en) 2005-03-24

Family

ID=29765930

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IT2004/000379 WO2005008283A1 (en) 2003-07-23 2004-07-09 Method for 3d-reconstruction of images acquired by means of laser radar, and endoscopic apparatus using such method

Country Status (2)

Country Link
IT (1) ITRM20030360A1 (en)
WO (1) WO2005008283A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2124196A1 (en) * 2008-05-20 2009-11-25 Oticon A/S Apparatus and method for representing a scanned surface
CN109499010A (en) * 2018-12-21 2019-03-22 苏州雷泰医疗科技有限公司 Based on infrared and radiotherapy auxiliary system and its method of visible light three-dimensional reconstruction
CN113256795A (en) * 2021-05-31 2021-08-13 中国科学院长春光学精密机械与物理研究所 Endoscopic three-dimensional detection method
CN113925441A (en) * 2021-12-17 2022-01-14 极限人工智能有限公司 Imaging method and imaging system based on endoscope

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5459605A (en) * 1992-12-17 1995-10-17 Paul S. Kempf 3-D endoscope apparatus
US6057909A (en) * 1995-06-22 2000-05-02 3Dv Systems Ltd. Optical ranging camera

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5459605A (en) * 1992-12-17 1995-10-17 Paul S. Kempf 3-D endoscope apparatus
US6057909A (en) * 1995-06-22 2000-05-02 3Dv Systems Ltd. Optical ranging camera

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2124196A1 (en) * 2008-05-20 2009-11-25 Oticon A/S Apparatus and method for representing a scanned surface
US8314794B2 (en) 2008-05-20 2012-11-20 Oticon A/S Apparatus and method for representing a scanned surface
CN109499010A (en) * 2018-12-21 2019-03-22 苏州雷泰医疗科技有限公司 Based on infrared and radiotherapy auxiliary system and its method of visible light three-dimensional reconstruction
CN113256795A (en) * 2021-05-31 2021-08-13 中国科学院长春光学精密机械与物理研究所 Endoscopic three-dimensional detection method
CN113256795B (en) * 2021-05-31 2023-10-03 中国科学院长春光学精密机械与物理研究所 Endoscopic three-dimensional detection method
CN113925441A (en) * 2021-12-17 2022-01-14 极限人工智能有限公司 Imaging method and imaging system based on endoscope
CN113925441B (en) * 2021-12-17 2022-05-03 极限人工智能有限公司 Imaging method and imaging system based on endoscope

Also Published As

Publication number Publication date
ITRM20030360A0 (en) 2003-07-23
ITRM20030360A1 (en) 2005-01-24
WO2005008283B1 (en) 2005-03-24

Similar Documents

Publication Publication Date Title
JP2812670B2 (en) 3D ultrasonic diagnostic image processing device
US7794388B2 (en) Method and apparatus for generating at least one section of a virtual 3D model of a body interior
US7889905B2 (en) Fast 3D-2D image registration method with application to continuously guided endoscopy
US8116848B2 (en) Method and apparatus for volumetric image navigation
US20130345509A1 (en) System and method for endoscopic measurement and mapping of internal organs, tumors and other objects
US8559685B2 (en) Robust and accurate freehand 3D ultrasound
JP2014217745A (en) Ultrasonic diagnostic apparatus and method of controlling the same
EP1219259A1 (en) System for locating relative positions of objects
Lathrop et al. Minimally invasive holographic surface scanning for soft-tissue image registration
JP2001526927A (en) Method and apparatus for calibrating a position sensor on a scanning transducer
JP2007528500A (en) Methods and systems for tomographic imaging using fluorescent proteins
US6396940B1 (en) Optical correlator based automated pathologic region of interest selector for integrated 3D ultrasound and digital mammography
CN101669831A (en) Reference image display method
JPH04317641A (en) Ultrasonic visualizing system
EP3578126B1 (en) Surgical navigation system
CN101002681A (en) Ultrasonic probe track display device and method, ultrasonic wave diagnosis device and method
CN101264002A (en) Three-dimensional endoscope measuring device and method based on optical grating projection
AU2019202779B1 (en) System and method for constructing virtual radial ultrasound images from CT data and performing a surgical navigation procedure using virtual ultrasound images
WO2014139032A1 (en) Data display and processing algorithms for 3d imaging systems
CN101422378A (en) Ultrasound diagnostic device
JP2001286474A (en) Dynamic measurement of subject&#39;s parameter
CN113974830A (en) Surgical navigation system for ultrasonically guiding thyroid tumor thermal ablation
JP2010088699A (en) Medical image processing system
Long et al. A marching cubes algorithm: application for three-dimensional surface reconstruction based on endoscope and optical fiber
WO2005008283A1 (en) Method for 3d-reconstruction of images acquired by means of laser radar, and endoscopic apparatus using such method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
B Later publication of amended claims

Effective date: 20050209

122 Ep: pct application non-entry in european phase