WO2004055545A1

WO2004055545A1 - Method for image conversion for bitmap display

Info

Publication number: WO2004055545A1
Application number: PCT/EP2003/050766
Authority: WO
Inventors: François Maurice
Original assignee: Thales Ultrasonics Sas
Priority date: 2002-12-17
Filing date: 2003-10-29
Publication date: 2004-07-01
Also published as: AU2003286195A1; FR2848673A1; FR2848673B1

Abstract

The invention concerns an image conversion method applicable in particular to ultrasound echography, which essentially consists, prior to the shots, in preparing a data table and index tables by ordering the bitmap points into visible quadrilaterals as received in the sequence of shots, then, during shots, in preparing a data structure including, for each visible quadrilateral, the values of its four peaks, and after each shot, in calculating the values of all the points of the bitmap which can be displayed.

Description

IMAGE CONVERSION PROCESS FOR PRESENTATION UNDER

BITMAP FORM

The present invention relates to a method for converting images for presentation in bitmap form.

The acquisition of images in fields such as medical ultrasound, radars, lidars or sonars is generally carried out by “shots” of emission beams, their echoes being collected in the direction of the emission beams . These echoes are reflected, on reception, by luminance values of points arranged, after data processing, on the different directions of the beams, which are generally parallel or radial. However, the commonly used display devices have a “bitmap” display and are either television type or pixel type scanning screens with a matrix layout (liquid crystal screens). The pixels of these display devices generally have a higher density than that of the echoes and a different arrangement. It is therefore necessary, to constitute a “bitmap” image, to convert the various points collected, or more exactly their respective luminances, into points with a matrix layout, and to interpolate between the points collected in order to be able to present a “bitmap” image without gaps.

The subject of the present invention is a method of converting images acquired using parallel and / or radial “shots” into bitmap type images, a method which allows processing as quickly as possible.

The method according to the invention consists, consists, before a sequence of shots, and for a given bitmap display format, in determining for each point of the bitmap, if it belongs to an analysis polygon, each polygon d analysis being defined by at least two consecutive points of a shot and by the two points of the same rank of the next shot, and if so, to chain it to the last point of the bitmap found in this polygon, to determine its position in the polygon, to reorganize the data thus collected according to the order of arrival of the points in the order of the shots, to locate the last point of each non-empty polygon, to associate with each of the empty polygons a determined point of the bitmap, then , during the shootings, to produce for each point received an operand comprising the luminances of at least four of the vertices of the polygon whose point in question is the last vertex received which identifies it, to arrange these operands in the order of reception of the corresponding points, and, at the end of each shot, to calculate the luminance of each point contained in each of the polygons taken in the order of reception of the vertices identifying them, this calculation being made according to an appropriate interpolation law from the luminances of at least the vertices of the polygon containing the point in question, the passage from one polygon to the next being controlled when the last point contained in this polygon is detected, to memorize the luminosities of the points thus calculated and to produce the bitmap image by reading the luminosities of the points thus memorized.

Advantageously, the polygons are quadrilaterals. According to another characteristic of the invention, during the processing of a set of points of the bitmap, the loading into cache memory of the data necessary for processing a following set of points of the bitmap is forced.

According to yet another characteristic of the invention, the coefficients of the interpolation law are stored in a table addressed by the position of the points in the polygon examined.

According to yet another characteristic of the invention, a data table is made up containing the Cartesian addresses of each point of the bitmap and its relative position inside the corresponding polygon. According to yet another characteristic of the invention, said data table is organized in ascending order of shots and of acquired polygons and that it contains information signaling the last point of the polygon in question.

According to yet another characteristic of the invention, a table is created containing the luminance operands of the vertices of each polygon, in the order of acquisition of these polygons, before calculating the luminances of the points of the bitmap.

According to yet another characteristic of the invention, a table is created giving, for each shot, the indices of the first and last polygons visible in the bitmap.

According to yet another characteristic of the invention, a global table is created giving, for each shot, the addresses of the first entry and of the last entry of the bitmap data table for a given bitmap. According to yet another characteristic of the invention, the bitmap point associated with each of the empty polygons is the same for all these polygons.

According to yet another characteristic of the invention, the point produced for each of the empty polygons is a point on the edge of the bitmap, which is whitened or blackened at the end of the calculation of the bitmap according to the background color of the bitmap.

According to yet another characteristic of the invention, the method is applied to at least one of the following fields: ultrasound medical imaging, acoustic camera, radar, NMR apparatus.

The present invention will be better understood on reading the detailed description of an embodiment, taken by way of nonlimiting example and illustrated by the appended drawing, in which:

• Figure 1 is a simplified diagram relating to a method of converting images of the prior art; and

• Figure 2 is a simplified diagram for explaining the method of the invention.

The invention is described below with reference to the conversion of medical ultrasound images, but it is understood that it is not limited to this single application, and that it can be implemented for the conversion of images supplied by various apparatuses operating on the principle of "shots", and scanning a portion of space, whether these shots are parallel to each other and / or radial, such as for example radars, sonars, NMR,. .. There is shown in Figure 1, in a simplified manner, two consecutive and intangible lines of fire 1, 2 of an ultrasound probe.

These two lines are radial (divergent), but the explanations below remain valid if these lines are parallel to each other. The rank of line 1, in the firing order of the different lines is i, and that of line 2 is i + 1. On each of these two lines, the consecutive echoes are represented by points: for line 1, they are references Pj-1, Pj, Pj + 1, Pj + 2, ... and ... Qj-1, Qj, Qj + 1, Qj + 2 ... for line 2, the indices ... j-1, j, j + 1, ... corresponding to the same shooting distances from one line to the next. On the other hand, three consecutive and immaterial lines 3, 4, 5 are shown which are parallel to each other and on which the points are regularly aligned. spaced from a bitmap (raster) image as displayed on a CRT monitor screen or a liquid crystal or plasma screen. In the drawing, which is shown relatively far from the probe from which the firing lines start, the density of the bitmap points is higher than that of the P and Q points of lines 1 and 2, but it is understood that in the vicinity of the probe, the reverse may be true. In FIG. 1, the lines 3, 4, 5 pass between the points Pj and Pj + 1 on the one hand, and between the points Qj and Qj + 1, and ten of the points of these three lines are included in the quadrilateral 6 whose four vertices are Pj, Pj + 1, Qj, Qj + 1. Let A be one of these ten points, located on line 4. This point, like all the other points of the bitmap, has Cartesian coordinates x and y, and its luminance is noted Lx, y because it depends on its position relative at the "luminous" points Pj, Pj + 1, Qj and Qj + 1, which are the only luminous points in the immediate vicinity of A. We could also take into account the points not immediately adjacent to A and located on lines 1 and 2 and / or on the previous and next lines, but, in general, this complicates the calculation without significantly improving the bitmap image.

As is well known, the luminance Lxy of point A is calculated as follows. Let Lij, Li, j + 1, Li + 1, j and Li + 1, j + 1, be the respective luminances of the points Pj, Pj + 1, Qj, Qj + 1 and a, b, c, d of the dependent coefficients relative distances from point A to said four vertices of quadrilateral 6. We then have:

Lxy = aLij + bLij + 1 + cLi + 1j + dLi + 1j + 1 Such a linear combination of four luminances is not the only possible way to calculate Lxy, which can be calculated according to other types of combination (nonlinear , with more than four points, ...).

From this data, the conversion process performs the following steps E1 to E5:

E1 - As soon as the brightness of a point in the bitmap has been calculated, it determines the coordinates (x, y) of the next point to be displayed.

E2 - It determines the analysis quadrilateral to which this next point belongs. E3 - It calculates the interpolation coefficients (a, b, c, d) relative to this next point. E4 - It calculates the luminance of this next point from the above-mentioned interpolation coefficients. E5 - It stores this luminance value in the memory defining the bitmap. In general, each shot and the resulting luminance data take place in sequence, and it may be advantageous, immediately after making two successive shots, to calculate the luminances of all the points of the bitmap located between the lines (lines of shot) relating to these two shots. Two different approaches are known for implementing the conversion process recalled above.

According to the first approach, steps E1 to E5 are carried out in real time, as the luminance data is acquired. Patent US-A-5 957846, for example, describes an algorithm allowing step E1 to be carried out by addressing a bitmap by polar exploration (rotation of a scanning line around a fixed point located at the intersection of the lines shooting). This process requires little storage memory, but requires a lot of calculations.

According to a second approach, which applies when the points of the bitmap have a fixed position from one image frame to the next with respect to the analysis quadrilaterals, the steps E1 to E3 are calculated once and for all. We then produce an array of data, stored in memory. This table contains the successive coordinates (x, y) of the points of the bitmap, their quadrilaterals of belonging, their interpolation coefficients. The data in this table are then called in real time, as successive shots are taken for the calculations in step E4. This process is technically feasible since the memories of current processors are sufficiently large (typically, the memory capacity required is a few megabytes). Currently, the second approach tends to supplant the first because it is faster and requires less computing means than the first, the current most efficient processors being sufficient to perform the necessary calculations in real time. However, the implementation of this second approach is relatively complex and cannot be done on common personal computers, especially if one uses ultrasonic probes with a large number of piezoelectric elements. Indeed, the number of points included in an analysis quadrilateral is variable (from zero to the entire bitmap), depending on the zoom factor and the location of the quadrilateral. In addition, some quadrilaterals may not contain any bitmap points. This results in a complexity in the logic of the unfolding and connection of the algorithm, which spends a lot of time looking for the next non-empty quadrilateral, this time spent "unproductive", being too large compared to the time taken to calculate the luminance points in the bitmap. Another consequence is that it is difficult to anticipate the precise course of the process. This is particularly troublesome when you want to use a personal computer processor to perform the conversion. Indeed, such a processor has a cache memory much faster than the central memory (large capacity in general), but this cache memory is of limited capacity, and does not allow to store the entire data table as as defined by the known method.

The present invention makes it possible to use the precharging instructions in cache memory to store therein in advance the data of the data table which will be requested in the near and foreseeable future. It is therefore no longer dependent on the use of the central memory whose access times are too slow. This is made possible thanks to a very simple program sequence making it possible to identify with certainty the moments when we must preload in cache the data to be used later. The method of the invention organizes the data table relating to the points of the bitmap and their associated data quadrilateral by quadrilateral, these quadrilaterals being classified in the order of their formation, that is to say in the order of reception of their last vertex (lower right vertex): for example in the case of FIG. 1, quadrilateral 6 (whose last vertex received is Qj + 1) comes immediately after quadrilateral 6A (whose last vertex received is Qj) , and it is immediately followed by quadrilateral 6B (whose last vertex received is Qj + 2).

The only expected branching therefore occurs when we have processed the last bitmap point of a quadrilateral and we have to move on to the next point to be processed (branching consists in answering the question: "the point coming after the one who has just been treated is it located in the next quadrilateral? "). The problem is that there can occur cases where the next quadrilateral containing at least one point of the bitmap is not the next, but the second or the third neighbor. This kind of situation arises in particular when the quadrilaterals are those of an area close to the probe. In such an area, the density of the analysis points (echoes received by the probe) is greater than the density of the points in the bitmap. There is therefore not, on average, one bitmap point per analysis quadrilateral.

The method of the invention makes it possible to resolve this difficulty by virtue of a characteristic explained below with reference to FIG. 2. In this FIG. 2, several successive divergent lines of fire 7 to 13 have been represented (originating from a probe shown) and the outline 14 of a bitmap image in the desired viewing position of this image with respect to these lines of fire. In the example shown in this figure, the image is zoomed and the bitmap does not represent the entire acquired image. The bitmap intersects, for example, firing lines 8 to 12. To simplify the drawing, analysis points have only been represented for lines 9 and 10. The analysis points are, in general, referenced Pi j , the index i being the sequence number of the shot and the index j the sequence number of the point on the firing line considered. On line 9 (i = 9), the five analysis points contained in contour 14 are, respectively, P9j, P9J + 1, ... P9J + 4 (j is the rank of the first point of line 9 visible in outline 14). On line 10 (i = 10), the five analysis points contained in contour 14 are, respectively: P10J, P10, j + 1, ... P10, + 4. The analysis quadrilaterals are identified for example by the coordinates of their upper left vertex. Thus, the quadrilaterals between lines 9 and 10 and comprising points visible in bitmap 14 are the quadrilaterals referenced by points P9, j-1 to P9, j + 4. The rule adopted by the invention is that if the first quadrilateral comprising points visible in the bitmap is Pij and if the last quadrilateral comprising points visible in the bitmap is (Pi + nj + n) then all the intermediate quadrilaterals will be assumed to contain each at least one visible point, for which a luminance will be calculated and displayed. In the case where one of these intermediate quadrilaterals does not include any point of the bitmap, we associate with it a unique dummy point, common to all the empty quadrilaterals, for example the first point of the bitmap (the first one at the top, on the left). No calculation is made for the quadrilaterals prior to Pij and those posterior to Pi + n j + n. Thus, rather than wasting time making a connection, the method of the invention consists in systematically traversing all the quadrilaterals from Pij to Pi + nj + n and performing for each of them at least one luminance calculation. If a quadrilateral is empty or if it contains only one point of the bitmap, it is considered to contain a single point (dummy or real) and this point is also considered as the last point of the quadrilateral, which is indicated by a special bit (described below). If a quadrilateral has several points, the last of them (in the direction of scanning the bitmap) is also indicated by a special bit. This special bit controls the incrementation of the reading of the data table (luminance table of the vertices of the quadrilaterals) to pass to the reading of the data relating to the next quadrilateral (empty or not) at the end of the calculations relating to the last point of the quadrilateral Analyzing. This avoids the loss of time caused by the search for the next non-empty quadrilateral.

We will now describe in detail the progress of the image conversion process according to the present invention.

1) - Preparation of the data table and index tables

This phase begins as soon as you have made a choice of image geometry or zoom factor and want to start the analysis.

It is not necessary to optimize it, because we typically have a few tenths of a second for this phase without this creating discomfort for the user of the ultrasound system.

This phase takes place as follows. For each point

(of rectangular coordinates x, y) of the bitmap, we determine if it belongs to an analysis quadrilateral. If yes, we chain it to the last point found in this quadrilateral, if this last point exists. We also determine the position of this point in the quadrilateral.

In the present example, this position is called

“POS” and its value consists of the fractional part of i (firing index), on 5 bits and the fractional part of j (index of the point analyzed on the firing line), on 3 bits. The parameters i and j are the polar coordinates of the treated points, obtained by transformation of the Cartesian coordinates of the points of the bitmap (these Cartesian coordinates are x and y). The above chaining is used to arrange the data in ascending order of their quadrilaterals of belonging. The table has one entry per bitmap point located in one of these quadrilaterals. Each entry in the table (on 32 bits for example) therefore contains the following data (relating to a single point):

> the address (in x, y) of the bitmap point considered, linearly coded (for example on 23 bits);

> its position (“POS”) in its quadrilateral (for example on 8 bits, as specified above); and

> a flag which is validated if the point in question is the last point of the quadrilateral (for example on 1 bit).

When the analyzed quadrilateral has no visible point in the bitmap, an entry is generated with a null value as an address in the bitmap (upper left point), a zero value for the position in the examined quadrilateral and 1 for the bit flag.

At the end of this preparation phase, there is also an index table indicating, for each shot, the first and last entries in the data table relating to this shot, as well as the serial numbers of the first and of the last quadrilateral visible for this shot.

- During the shots

In order to accelerate the final calculation, which comprises four multiplications and three additions to obtain the luminance of each point, the invention provides for using the instructions with multiple operands available with the INTEL processors (instructions of the SIMD or MMX type). Among these instructions, there is in particular an operation which carries out at one time four multiplications and two additions. Knowing that in general a quadrilateral can contain several points of the bitmap (except the special cases in the vicinity of the probe cited above), it is advantageous to generate each time that we receive a new data of shooting a multiple operand which concatenates the four values of luminance of vertices of the quadrilateral whose last point received is the bottom right vertex. These data are in limited volume, and can therefore all reside in the cache memory of the processor for the subsequent calculation of the luminance of the bitmap points. According to an exemplary embodiment, the four luminance values (La, Lb, Le, Ld) are each defined on 16 bits and concatenated into a 64-bit operand, and on the other hand, the four coefficients (a, b, c , d), also defined on 16 bits, are concatenated, in the same order, into a 64-bit operand. Then, La, Lb, Le and Ld are respectively multiplied by a, b, c, d, and a first addition consists in simultaneously adding on the one hand aLa + bLb, which gives a partial result E, and on the other hand cLc + dLd, which gives another partial result F, these two partial results being concatenated into a 64-bit word, and finally performing a second addition E + F.

- At the end of each shot.

At the end of each shot, the conversion operation is carried out, reduced to its simplest expression, namely a calculation loop covering all of the entries in the data table relating to the shot in question, this table being formed during the preparation phase. This loop performs the calculations mentioned above, namely the multiplication of the luminances of the four vertices of the quadrilateral being analyzed by the interpolation coefficients of the bitmap point considered. The luminances were acquired during the shots and stored in cache memory and the coefficients are stored in a constant array of multiple operands, also residing in cache memory. This table of coefficients is indexed by the value (“POS”) of the entry considered in the data table. Using the SIMD instructions of the aforementioned processor, all it takes is a multiplication operation - accumulation followed by an addition, and a offset normalization (scaling). The luminance of the bitmap point analyzed is thus obtained directly, which it suffices to store in the bitmap memory at its address, which is provided by the input considered in the data table. If the flag associated with the current entry of the data table is set (i.e. if it corresponds to the last point included in the analyzed quadrilateral), the index of the luminance data table must be incremented formed during the shots. This index then points to the luminances of the next quadrilateral. These two operations form the very compact body of the bitmap image calculation program. It can be seen that, whatever the result of the second operation linked to the state of the flag, the calculation program consumes exactly one entry in the data table per path of the loop. We can therefore accurately predict the moment when we will need data from the data table, and use, if it exists, the preload instruction in the processor cache memory. For example, in the case where this processor is a Pentium III ™, and from the digital example cited above, a line of cache memory represents 256 bits, and the optimum is therefore to carry out a preload (called "prefetch" ) of the cache memory every 8 calculated points.

When the entire bitmap has been calculated, you must blacken (or whiten, depending on the choice of background color) the upper left point of the bitmap which holds luminance information linked to the last quadrilateral located in the area visible, but does not itself contain any point of the bitmap.

Claims

1. Method for converting images acquired according to parallel and / or divergent shots into a bitmap type image, characterized in that it consists, before a sequence of shots, and for a given bitmap display format, to be determined for each point of the bitmap, if it belongs to an analysis polygon, each analysis polygon being defined by at least two consecutive points of a shot and by the two points of the same rank of the next shot, and if so , to chain it to the last point of the bitmap found in this polygon, to determine its position in the polygon, to reorganize the data thus collected according to the order of arrival of the points in the order of shots, to locate the last point of each non-empty polygon, to associate with each of the empty polygons a determined point of the bitmap, then, during the shots, to produce for each point received an operand comprising the luminances of at least four of the vertices of the polygon whose point in question is there th last vertex received which identifies it, put these operands in the order of reception of the corresponding points, and, at the end of each shot, calculate the luminance of each point contained in each of the polygons taken in the order of reception of the vertices identifying them, this calculation being made according to an appropriate interpolation law from the luminances of at least the vertices of the polygon containing the point in question, the passage from one polygon to the next being controlled upon detection of the last point contained in this polygon, to memorize the luminosities of the points thus calculated and to produce the bitmap image by reading the luminosities of the points thus memorized.

2. Method according to claim 1, characterized in that the polygon is a quadrilateral.

3. Method according to claim 1 or 2, characterized in that, during the processing of a set of points of the bitmap, the loading in cache memory of the data necessary for the processing of a following set of points of the bitmap.

4. Method according to one of the preceding claims, characterized in that the coefficients of the interpolation law are stored in a table addressed by the position of the points in the polygon examined.

5. Method according to one of the preceding claims, characterized in that one constitutes a data table containing the Cartesian addresses of each point of the bitmap and its relative position inside the corresponding polygon.

6. Method according to claim 5, characterized in that said data table is organized in ascending order of shots and acquired polygons and that it contains information signaling the last point of the polygon in question.

7. Method according to one of the preceding claims, characterized in that a table is created containing the luminance operands of the vertices of each polygon, in the order of acquisition of these polygons, before calculating the luminances of the bitmap points.

8. Method according to one of the preceding claims, characterized in that a table is created giving, for each shot, the indices of the first and last polygons visible in the bitmap.

9. Method according to one of claims 5 to 8, characterized in that one constitutes a global table giving, for each shot, the addresses of the first entry and of the last entry of the bitmap data table for a bitmap given.

10. Method according to one of the preceding claims, characterized in that the bitmap point associated with each of the empty polygons is the same for all these polygons.

11. Method according to one of the preceding claims, characterized in that the point produced for each of the empty polygons is a point on the edge of the bitmap, which is whitened or blackened at the end of the calculation of the bitmap according to the color of the bitmap background.

12. Method according to one of the preceding claims, characterized in that it is applied to at least one of the following fields: ultrasound medical imaging, acoustic camera, radar, NMR apparatus.