CN100429551C - Composing method for large full-scene depth picture under microscope - Google Patents

Abstract

The present invention relates to a composing method for a large full-scene depth picture under a microscope. The method comprises: sequential images with different depth pictures in the same region of microscopic objects are firstly used under the microscope; computer image treating technology is used for reconstructing a full-scene reconstructing depth picture which can reflect information within all focusing ranges and background information in the region; then, the full-scene reconstructing depth picture of an adjacent region is spliced; finally a large full-scene depth picture which can reflect the whole information of microscopic object surfaces and backgrounds is obtained. The present invention has the advantages that the defects of the prior art are overcome, and the complete information of the microscopic object surface under the microscope is comprehensively reflected.

Description

The dark significantly joining method of picture of microscopically panorama

Technical field

The present invention relates to a kind of image split-joint method of three-dimensional surface, particularly relates to utilize ordinary optical microscope to carry out the dark significantly joining method of picture of microscopically panorama of three-dimensional splicing by continuous shot object video or sequence chart.

Technical background

Because the micro objective depth of field is limited, and along with the increase depth of field of enlargement factor can corresponding reducing.Fig. 1, Fig. 2 are the schematic diagrams of microscope imaging.Can be clear that by Fig. 1 the picture plane of object might not overlap with the sensitive surface of image collecting device, the image that obtains like this may be the picture that blurs; See by Fig. 1, object is owing to itself there is certain degree of depth, and the microscope depth of focus is limited, the part that only is distributed in the depth of field could be known imaging, the object of other positions becomes fuzzy picture, general object under test is difficult to whole focusing below microscope, so can't obtain the image of reflection surface information comprehensively at a certain observation position.At present, there has been the scholar to use this situation to propose to use at same position both at home and abroad by take a sequence of pictures obtains this position by reconstruct the dark picture of the panorama that can reflect all in-focus information at vertical direction dollying head at this situation.

On the other hand because microscope is the instrument of observation micro-object, its field range is very limited, can only observe a little zone at one time, just can't from microscope, directly obtain its overall picture for bigger object or bigger regional the present invention.At this situation, general method is to adopt computing machine that the little picture of adjacent area is spliced significantly picture, so just can observe the overall picture of object in computing machine or document printing.But this joining method is not considered the problem of the object depth of field, just might to take place above the microscope only be that clear and out-of-focus place all is the situation of bluring in the place of field depth inner focusing if be applied directly to, if each shift position all adjusts the telescope to one's eyes when gathering and focuses on again, will because of because the focusing, the picture of same position can change in the process of twice focusing, causes splicing accurately putting in place.Even stitching position does not have mistake, also can cause splicing picture and also can produce distortion because of the lap of two width of cloth pictures is inconsistent.The main at present joining method that uses all is subject image to be gathered the method that is spliced then at same surface level, can not get the information of body surface out-focus position like this, because the depth of field of microscope under the situation that high multiple is observed is more narrow, so the phenomenon of drop-out is even more serious when high multiple is observed.

Summary of the invention

For the restriction on the function that overcomes ordinary optical microscope, overcome in the prior art by the defective in the big picture approach of splicing collection micro-object, the purpose of this invention is to provide a kind of in conjunction with computer technology utilization image processing method, carry out the body surface signal reconstruct by video or sequence image to the different depth of field of the continuous shooting under the same area, obtain not having the dark picture of panorama of depth of field restriction, and then the dark picture of the panorama of adjacent area spliced, obtaining reflecting the dark picture of significantly panorama of object under test surface overall picture, is a kind of dark significantly joining method of picture of panorama that microscopically obtains reflection complete object surface information that is applied in.

To achieve these goals, the technical solution used in the present invention is: the sequence image that at first utilizes the different depth of field of micro-object the same area under simple microscope, the image processing techniques that uses a computer reconstitutes the very grave composition sheet of panorama that can reflect all focusing range internal informations of this zone and background information, then the very grave composition sheet of the panorama of adjacent area is spliced, the panorama that finally obtains reflecting micro-object surface and background full detail is picture deeply significantly.

The invention has the beneficial effects as follows: overcome the deficiency in the conventional images splicing, reflected the complete information on microscopically micro-object surface comprehensively.

Description of drawings

Fig. 1 is the formation synoptic diagram of out-of-focus image.

Fig. 2 is the imaging scope synoptic diagram with object of certain altitude.

Fig. 3 moves three direction synoptic diagram for microscope of the present invention and camera lens thereof.

Fig. 4 is the system works flow process figure of the embodiment of the invention 1.

Fig. 5 is the vertical direction restructing algorithm process flow diagram of the embodiment of the invention 1.

Fig. 6 increases part Boolean algebra algorithm synoptic diagram for the present invention looks for novelty, dark part is newly-increased part.

Fig. 7 is the sharpness division methods synoptic diagram that the present invention is based on thresholding.

Fig. 8 is the processing flow chart of the embodiment of the invention 2.

Fig. 9 is the processing flow chart of the embodiment of the invention 3.

Figure 10 is 7 of the present invention single-point focus function appraisal curve pie graph after level and smooth.

Figure 11 is the processing flow chart of the embodiment of the invention 4.

Figure 12 splices information drawing for the adjacent figure of the present invention.

Figure 13 is the adjacent picture joining method of the present invention figure.

Among the figure: 1-object, 2-lens, 3-image capture device sensitive surface, 4-have the object of certain altitude as plane, 5-optical axis, 6-.

Embodiment

The present invention is further described below in conjunction with accompanying drawing and example.

The present invention at first utilizes the sequence image of the different depth of field of micro-object the same area under simple microscope, the image processing techniques that uses a computer reconstitutes the very grave composition sheet of panorama that can reflect all focusing range internal informations of this zone and background information, then the very grave composition sheet of the panorama of adjacent area is spliced, the panorama that finally obtains reflecting micro-object surface and background full detail is picture deeply significantly.

Embodiment 1:

The present invention uses ordinary optical microscope and computing machine and image capture device (camera, image pick-up card etc.).Carry out Flame Image Process (seeing accompanying drawing 4) on computers at the sequence image that microscopically is collected by camera and capture card.The basic procedure of this method is the magnitude range of at first determining to gather picture, the vertical direction moving range, microscope enlargement factor or the like initialization condition, the initiating task flow process begins to gather picture, each width of cloth picture judges that at first it is at vertical direction or moves in the horizontal direction, if vertical direction moves, just the sequence of pictures of the different depth of field that vertical direction is collected above is reconstructed, and reconstruct becomes the dark reconstructed picture of panorama.If the dark picture of panorama that just moves in the horizontal direction the continuous zone of vertical direction reconstruct splices, its concrete steps are:

The first step: to the reconstructing method (see accompanying drawing 5) of vertical direction direction use based on the very grave composition sheet of the panorama of sharpness thresholding, picture to the different depth of field under the same area is reconstructed, and obtains reflecting the very grave composition sheet of panorama of this zone all surfaces information and background information.

At first use gradient operator that picture is carried out convolution, if certain any convolution value is then thought it in focusing range greater than thresholding, with its reservation and zone bit is set gets this in the pixel of this two field picture pixel as reconstructed image.If all do not have picture greater than thresholding to this all sequences image, can think that then it is a background, the pixel of getting last piece image or ad-hoc location image is as the value of reconstructed image at this point.

The gradient operator computing formula is seen formula (1): gradient operator has the single order operator, second order operator and other operators, and the operator masterplate can select 2*2 to the 100*100 scope, can select in different occasions.

The single order operator:

$\▿f(x,y)=\left|\frac{\∂f(x,y)}{\∂x}\right|+\left|\frac{\∂f(x,y)}{\∂y}\right|---\left(1\right)$

The computing method that keep pixel:

$F(i,j)=\underset{x-i-N}{\overset{i-N}{\mathrm{\Σ}}}\underset{y=j-N}{\overset{i+N}{\mathrm{\Σ}}}\▿f(x,y)$ If $\▿f(x,y)>T---\left(2\right)$

The second order operator:

${\▿}^{2}f(x,y)=\left|\frac{{\∂}^{2}f(x,y)}{{\∂}^2{x}^2}\right|+\left|\frac{{\∂}^{2}f(x,y)}{{\∂}^2{y}^2}\right|---\left(3\right)$

The computing method that keep pixel:

$F(i,j)=\underset{x-i-N}{\overset{i-N}{\mathrm{\Σ}}}\underset{y=j-N}{\overset{i+N}{\mathrm{\Σ}}}{\▿}^{2}f(x,y)$ If ${\▿}^{2}f(x,y)>T---\left(4\right)$

By formula 1-4 can obtain present frame in focusing range have a few, use then current depth of field binary map mark present frame in focusing range have a few; The binary map of being had a few in field depth of using accumulative total part binary map mark to store; The cromogram of being had a few in field depth that uses accumulation partial colour figure storage to store; Use newly-increased part binary map storage present frame than in field depth, increasing the some binary map in the past newly; Use newly-increased partial colour figure to store the cromogram of present frame than former newly-increased point in field depth.These pictures all are changed to 0 in initialization.

The restructing algorithm flow process is with reference to Fig. 5, at first be to find out the point that all can focus in shooting process, find these points through constantly image sequence being carried out the sharpness analysis, by Boolean calculation these points are reconfigured to above the accumulation partial colour figure (with reference to Fig. 5 subroutine A) then.

Used three kinds of Boolean calculations in algorithm, inclusive-OR operation, AND operation and looking for novelty increase the Boolean calculation of part.

Inclusive-OR operation: used the inclusive-OR operation of binary map and binary map and the inclusive-OR operation of binary map and cromogram here, the expression formula of inclusive-OR operation is AYB.

AND operation: used the AND operation of binary map and binary map and the AND operation of binary map and cromogram here, the expression formula of AND operation is AI B

The Boolean calculation formula of newly-increased part is seen formula (5), is mainly used in to extract the some (see figure 6) of present frame A than the additional that part of newly-increased image of accumulation part B.

$\{x:(x\∉\mathrm{AI\; B})I(x\∈B)\}---\left(5\right)$

Find after the point of all focusing, also the part that did not focus in sequence to be joined as a setting in the last reconstructed picture, with reference to Fig. 5 subroutine B, just can obtain the very grave composition sheet of panorama that all focusings of this zone of complete reflection are enclosed internal information and background information.

Second the step: with the very grave composition sheet of adjacent panorama horizontally-spliced be deeply significantly picture of panorama

The microscopical visual field is smaller, can't once collect wide-field picture, simultaneously need could form an integral body to the image mosaic to of various piece piece for the general objective image.So splicing is another key.Must satisfy-W＜w＜W the splicing of picture as shown in figure 14, w is two secondary picture width displacements, and W is the picture width;-W＜h＜H, h are two secondary picture height displacement, and H is the picture height.If do not satisfy above-mentioned condition, picture can't splice.

The different depth of field pictures in a zone have been reconstituted the picture that does not have depth of field restriction in first step the present invention, following work is exactly that little picture with these reconstruct splices in the horizontal direction and becomes the big picture that can reflect integral image.The present invention takes in implementation process, after the vertical direction in two zones that link to each other goes depth of field reconstruct to finish, just this two width of cloth picture is spliced, then next piece adjacent areas is carried out vertical direction reconstruct, the dark synthesising picture of the panorama of next piece adjacent area and front are spliced good picture again and splice, flow process is seen Figure 15.

In order to realize the splicing of two adjacent figure, the present invention utilizes image matching technology, so-called images match be meant two two width of cloth images that collect from same scenery spatially face aim at, to determine the process of relative translation between this two width of cloth figure.There is overlapping adjacent image for two width of cloth,, in another width of cloth image, seeks match point then if the overlay region of piece image is chosen a pocket and made reference map therein.Just can determine the translation relation of the two, also just can realize the splicing of the two by changes in coordinates.The present invention here chooses gray scale relevant matches strategy as joining method.

If reference map T (size of T is K*L) overlays with reference to figure S (size of S is M*N) and goes up translation, the search figure under reference map T covers is called subgraph, uses convolutional calculation S, T, at an i, the convolution above the j, determining the position of convolution value maximum in search area, is exactly stitching position.

$R(i,j)=\underset{m=1}{\overset{K}{\mathrm{\Σ}}}\underset{n=1}{\overset{L}{\mathrm{\Σ}}}{S}_{i,j}(m,n)*T(m,n)---\left(6\right)$

[x，y]＝{i，j|max(R(i，j))} (7)

The related coefficient of formula (6) and subgraph S ', (i is j) 0 for R, value between 1, and only S (i, j) with T (i, j) value is a maximal value 1 when congruence, according to similarity principle R (i, j) value is big more, and both are similar more, satisfies R (i so the present invention gets here, j) get that peaked (i j) is correct matched position.

Just can splice after finding matched position two width of cloth figure, the subject matter of splicing is the processing at the imbricate part, processing to lap has several different methods desirable, but because the influence of illumination or other factors, two width of cloth picture laps are not necessarily identical to cause splicing unsmooth, and partial information perhaps runs off.The part that can get last width of cloth figure is as lap figure, and the part that also can distinguish a back width of cloth figure is as the lap image, and these two kinds of methods all can be cast out the information of whole laps of certain width of cloth figure.Two images laps can also be carried out mean value computation in addition, keep the partial information of two width of cloth images simultaneously, but this method can cause the fuzzy of lap, can lose the high-frequency information of lap when serious.Here in order to utilize the information of front and back two width of cloth figure as far as possible, and do not lose pictorial information, the present invention adopts and gets the lap mid point as cut-point, picture in the mid point front keeps the lap of last width of cloth figure, the lap (seeing Figure 15) of the back width of cloth figure of the reservation of mid point back so just can constitute a complete splicing picture.

Embodiment 2:

In embodiment 1 first step since not all in focal range the institute a bit can get big in threshold value, some zone is because factors such as illumination, smoother on every side, in the calculating of sharpness, just there is not point like this greater than threshold value, computing machine just thinks that it is a background, be used as net result and got unsharp point, be exactly because have the part noise in the image acquisition on the other hand, make the gradient operator result of calculation of the point in focusing range not greater than threshold value, computing machine can think that this shop is the point of focusing range and remaining of mistake like this.

In general, the point in the zone of similar position has similar focus characteristics, so can revise above-mentioned two kinds of errors by two-value morphology by the factor of reference surrounding pixel point.Expanding corrosion, the morphologic processing of two-value such as connection to being divided the binary map that obtains by threshold value by gradient operator.Remove acnode, leak in the binary map.Can make to focus on and differentiate more accurate (seeing accompanying drawing 8).

Corrosion: set A is corroded by set B, and it is defined as

$x:B+x\⋐A---\left(8\right)$

Expand: set A is expanded by set B, and it is defined as

x：(-B+x)∩A≠φ (9)

At first the binary map of being come out by Threshold Segmentation in the front is corroded, and removes the noise acnode, it is expanded again, and removes the leak in focusing range, and then corrodes the size that reverts to original focusing range.Use between masterplate and corrosion or the desirable 1-20 of expansion number of times, big or small desirable 2 * 2-100 * 100 of masterplate, the masterplate shape can be square, rectangle, rhombus, circle and other irregular figures.

Embodiment 3

Though embodiment 2 improves the first step of embodiment 1, but still may start a leak or noise, and the masterplate that will expand at picture design corrosion, in operation can be cumbersome, so introduce the reconstructing method of the full depth map of seeking high-resolution point here.

The algorithm flow of reconstructing method with reference to the accompanying drawings 9, at first captured full sequence picture is preserved, by the time after vertical direction moves and finishes, each width of cloth image is adopted gradient operator (formula 10,12) carry out convolution algorithm, and will the convolution value record of each point with the corresponding position of initial point on.After convolutional calculation finishes, get the some generation of the convolution value on all sequences focusing clear figure curve (Figure 12), the curve that directly calculates as we can see from the figure has a lot of burrs and noise, so need carry out curve smoothing processing (Fig. 8) to it.Because the motion of microscope camera lens is that straight line fortune is fixed slowly, so picture is by defocusing focusing, defocus by focusing on again, focus on articulation curve and be from low to high from high to low, highest point is the center of its focusing, so the present invention gets the position at maximal value place in the sequence, think to focus on the most at this frame in the moving process that the pixel that keeps this point is as the very grave composition sheet of panorama wherein (formula 10-13).

Gradient operator has single order operator (formula 10), and second order operator (formula 12) and other operators can be selected in different occasions.

The single order operator:

$\▿f(x,y)=\left|\frac{\∂f(x,y)}{\∂x}\right|+\left|\frac{\∂f(x,y)}{\∂y}\right|---\left(10\right)$

The computing method that keep pixel:

$F(i,j)=\underset{x-i-N}{\overset{i-N}{\mathrm{\Σ}}}\underset{y=j-N}{\overset{i+N}{\mathrm{\Σ}}}\mathrm{Mf}(x,y)$ If $\mathrm{Mf}(x,y)=\max (\▿f(x,y,z))---\left(11\right)$

The second order operator:

${\&dtri;}^{2}f(x,y,z)=\left|\frac{{\&PartialD;}^{2}f(x,y,z)}{{\&PartialD;}^2{x}^2}\right|+\left|\frac{{\&PartialD;}^{2}f(x,y,z)}{{\&PartialD;}^2{\mathrm{<figure-callout\; id="2"\; label="y"\; filenames="C20051001892100132.png"\; state="\{\{state\}\}">y</figure-callout>}}^2}\right|---\left(12\right)$

The computing method that keep pixel:

$F(i,j)=\underset{x-i-N}{\overset{i-N}{\mathrm{\&Sigma;}}}\underset{y=j-N}{\overset{i+N}{\mathrm{\&Sigma;}}}\mathrm{Mf}(x,y)$ If $\mathrm{Mf}(x,y)=\max \left({\&dtri;}^{2}f(x,y,z)\right)---\left(13\right)$

Embodiment 4:

In embodiment 3, independent articulation curve on sequence has been carried out smoothly, eliminated burr and the noise above the curve, for further contemplate the object chart as neighbor spatially face also be adjacent, similar focus characteristics is arranged, the present invention uses intermediate value or the F (i of mean filter operator to calculating at formula (10) (12), j) the elevation information image carries out convolution algorithm, the filtering singular point, further eliminate error, finally obtain elevation information figure accurately, be mapped to highly corresponding with it sequence image pixel then, be converted into the very grave composition sheet of single width panorama, algorithm flow is with reference to accompanying drawing 11.So just can make the focusing location more accurate, shortcoming is that calculated amount is big, and request memory is big, and arithmetic speed is slow slightly.But in actual mechanical process, microscope will move horizontally after vertical direction moves, and can be chosen at this moment picture is carried out vertical direction reconstruct, can not influence works speed.

Claims (5)

1, the dark significantly joining method of picture of a kind of microscopically panorama, this method is at first utilized the sequence image of the different depth of field of micro-object the same area under simple microscope, the image processing techniques that uses a computer reconstitutes the very grave composition sheet of panorama that can reflect all focusing range internal informations of this zone and background information, then the very grave composition sheet of the panorama of adjacent area is spliced, the panorama that finally obtains reflecting micro-object surface and background full detail is picture deeply significantly, its concrete grammar is: the magnitude range of at first determining to gather picture, the vertical direction moving range, microscope enlargement factor initialization condition, the initiating task flow process begins to gather picture, each width of cloth picture judges that at first it is at vertical direction or moves in the horizontal direction, if vertical direction moves, the just sequence of pictures of the different depth of field that vertical direction is collected above splicing, splicing becomes the very grave composition sheet of this zone panorama; If move in the horizontal direction, just will just splice the good adjacent with it very grave composition sheet of panorama of the very grave composition sheet of panorama and carry out the level splicing.

2, the dark significantly joining method of picture of microscopically panorama as claimed in claim 1, it is characterized in that: the step of this method is:

The first step: to the reconstructing method of vertical direction use based on the very grave composition sheet of the panorama of sharpness thresholding, the sequence of pictures that is in the different depth of field under the same area is reconstructed, obtains to reflect the very grave composition sheet of panorama of this zone all surfaces information and background information;

Second step: the very grave composition sheet of the panorama of adjacent area level is spliced into deeply significantly picture of panorama.

3, the dark significantly joining method of picture of microscopically panorama as claimed in claim 2 is characterized in that: at first the binary map that is split by the sharpness thresholding is corroded, remove the noise acnode; Again it is expanded, remove the leak in focusing range, and then corrode the size that reverts to original focusing range, use between masterplate and corrosion or the desirable 1-20 of expansion number of times, big or small desirable 2 * 2-100 * 100 of masterplate, the masterplate shape can be square, rectangle, rhombus, circle and other irregular figures.

4, the dark significantly joining method of picture of microscopically panorama as claimed in claim 1, it is characterized in that: at first captured full sequence picture is preserved, by the time after vertical direction moves and finishes, adopt gradient operator to carry out convolution algorithm to each width of cloth image in the sequence, and will the convolution value record of each point with the corresponding position of initial point on, after all images convolutional calculation finishes, getting a bit, the convolution value on all sequences generates the focusing articulation curve, directly draw the curve that comes out a lot of burrs and noise are arranged, it is carried out the curve smoothing processing, get the position at maximal value place in the sequence, think to focus on the most at this frame in the moving process, keep the corresponding pixel in this position as the very grave composition sheet of panorama wherein.

5, the dark significantly joining method of picture of microscopically panorama as claimed in claim 4, it is characterized in that: use intermediate value or mean filter operator that the elevation information image is carried out convolution algorithm, the filtering singular point, further eliminate error, finally obtain elevation information figure accurately, be mapped to then and highly corresponding sequence image pixel, be converted into the very grave composition sheet of single width panorama.

Images