WO2005048171A1

WO2005048171A1 - Method and apparatus for imaging through glossy surfaces and/or through transparent materials

Info

Publication number: WO2005048171A1
Application number: PCT/NO2004/000331
Authority: WO
Inventors: Kjell KRÅKENES; Karl H. Haugholt; Jon Tschudi; Ib-Rune Johansen; Henrik Schumann-Olsen; Arne Sommerfelt
Original assignee: Cargoscan As
Priority date: 2003-11-14
Filing date: 2004-11-02
Publication date: 2005-05-26
Also published as: NO20035094L; NO320064B1; NO20035094D0

Abstract

Method and system for providing an image of an object moving with a known speed relative to a camera, especially objects being covered by a specular reflecting surface, comprising the use of a linear sensor camera adapted to sample a linear image with a chosen resolution of the object at a chosen sampling rate, the object passing in a direction essentially perpendicular to the image line, the method comprising illuminating the object from at least two directions in a sequence synchronized with said sampling rate, so as to obtain different illumination direction in adjacent samples.

Description

Method and apparatus for imaging through glossy surfaces and/or through transparent materials

The present invention relates to a system for imaging of glossy surfaces and for imaging through transparent materials without having the image quality severely degraded by specular reflections from the surface or from transparent overlay such as plastic wrapping, which may cover the surface to be imaged. More specifically the present invention relates to the imaging of objects on a conveyor belt with a line scan camera with dedicated illumination, for the purpose of: 1) automatic and/or manual detection and decoding of symbology printed on the objects, 2) inspection of size, shape and other geometrical characteristics of the objects and 3) inspection of surface characteristics of the objects in order to characterize and classify the objects based on these properties.

A major problem in such applications is specular reflections from the surfaces, and several solutions for reducing the effect of such reflections have been demonstrated. One method is using polarized illumination and a polarization filter on the receiver optics which is rotated 90 degrees relative to the polarization of the illuminating light. This method is based on the assumption that the specular reflected light is polarized whereas the diffusely reflected light is unpolarized. Since the polarizer on the receiver optics is rotated 90° relative to the polarization of the illuminating light, the polarizer on the receiver optics will stop the specular reflected light. This methods suffers from two major problems. One problem is that 50% of the diffusely reflected light is co linear with the specular reflected light, and is thereby also stopped by the polarizer. If in addition, one uses an unpolarized light source, one also needs a polarizer on the outgoing light from the light source. This means that 75% of the light energy from the light source is wasted. In real life, polarizers are not ideal, and in practice one may end up loosing 90% or more of the light. This method also suffers from the fact that transparent overlays may be birefringent causing a rotation of the specular reflected light. If such polarization rotation occurs, the specular reflected light will in general have an unpredictable, elliptical state of polarization, and thereby cannot be eliminated with a linear polarizing filter. Another method is to set up the illumination at an angle such that the specular reflection will not reach the entrance pupil of the camera objective. In the applications in mind for the present invention, the surface of the objects to be imaged maybe irregular, and one, therefore, need to set up the illumination at a high angle to the camera. The illumination must then also be out of plane of the field of view of the camera. This again leads to the fact that the illumination must be spread out into a wide angle in order to cover the entire height range of the objects under inspections. Such "flood lightening" of the objects and their vicinity is viewed as undesirable in many of the application in mind for the present invention.

It is thus an object of this invention to provide an efficient and simple way to eliminate reflections from surfaces when scanning e.g. for text or machine readable codes, or for controlling objects being at least partially covered by reflecting surfaces.

This problem is solved in the present invention by setting up the illumination on each side of the camera, in the same plane as defined by the linear sensor of the camera, or nearly in. With this arrangement, specular reflections may occur for parts of the field of view from each of the light sources. However, by setting up the light sources on each side of the camera correctly, one is guaranteed that the parts in the field of view of the camera which experiences specular reflections from the light source on one side, will not experience specular reflections form the light source on the opposite side. Then, by alternatingly taking images with illumination from the left and right light sources, and by using a computer to analyze the images to identify and discard or reject the affected sections from each image, one ends up with a synthetic total image of the entire width of the parcel or object without any pixels being affected by specular reflections, thus comprising only good pixels, h order to avoid loosing resolution, the image is over sampled twice along the direction of travel of the conveyor belt. After the combination of two consecutive lines, the synthetic image has the same resolution transversally and longitudinally to the conveyor belt.

The illumination system is primarily based on known solutions, such as described in US 5063460 and US 5777314, or the specialized version of the latter US 6628445, both describing the use of light sources being coplanar with a linear sensor with corresponding optical system for scanning an object or surface. In both these publications a cylindrical lens element is used to focus the light toward the line which is scanned by the linear sensor. In US 5063460 the light sources are two single lamps positioned on each side of the camera, while US 5777314 describes the use of two or more light emitting diodes (LEDs), being positioned symmetrically on each side of the camera in a number of different configurations e.g. a linear array of LEDs on each side of the sensor and parallel with the sensor and cylindrical lens, so as to provide a smooth illumination stripe over the line which is sampled by the camera. Neither of these solutions are, however, suitable for imaging where the object is covered by a glossy surface, as they will give rise to two sets of specular reflection making any information behind the glossy surface impossible to read.

More specific the objects of this invention are obtained as described in the accompanying claims.

In order to extend the range of object heights that can be measured, the present invention can be equipped with two light sources on each side, mounted a different heights, and controlled by a priori knowledge of the height of the object to be measured.

The present invention is presented in more details in the following drawings, which illustrates the invention by way of examples:

Fig. 1 illustrates the side view of one embodiment of the invention mounted above a conveyor belt. Fig. 2. shows the front view of the setup shown in figure 1.

Fig. 3 shows an illustration of the rays of light from the light sources as they hit the object and are reflected off the surface and into the camera objective. Fig. 4a and b gives an illustration of the pixel grid of the detector projected into object space. Pixels affected by specular reflections from the two light sources are marked as black. Fig. 5a and b shows a raw image and processed image, respectively, generated by a system based on the present invention. Fig. 6. shows the side view an illustration of an alternative embodiment of the invention extended to cover a larger range of object heights.

Fig. 7 illustrates the front view of the invention in the case illustrated in fig. 6.

Fig. 8 illustrates the invention by way of a block diagram, including a number of external components and functions that are required in order to obtain the intended functionality of the preferred embodiment of invention.

Figure 1 shows the present invention 1 mounted above a conveyor belt 3 for imaging of objects travelling on the belt. A linear camera 2 is directed in a direction 4 perpendicular to the belt 3 so as to obtain a sequence of linear images of the object. Preferably the camera 2 is coupled to a computer means for combining these images to a two dimensional representation of the object. By synchronizing the imaging frequency of the camera 2 to the speed of the conveyer belt 3, correct proportions of the object may be obtained in the image. Because of the later signal processing the sampling rate is preferably chosen so as to provide twice the spatial resolution in the direction of movement relative to the spatial resolution across the direction of movement.

The scanning frequency may be made dependent on the speed of the object, e.g. by monitoring the speed of the conveyer belt and adjusting the sampling frequency according to the required resolution of the image of the object. Thus disturbances in the velocity and even stopping of the conveyer belt will not affect the resulting image. A simplified version could be not to measure the speed, but to maintain a predetermined speed corresponding to the scanning frequency and to provide a warning signal stopping the sampling if the belt stops.

According to the preferred embodiment of the invention, two light sources 5,6 (see figure 2) illuminating the conveyer belt and thus the objects passing thereon. As the camera 2 employs a linear array sensor, the light sources 5, 6 are preferably mounted in plane, or nearly in plane with the field of view of the linear sensor, and projects a beam of light which overlaps the field of view of the sensor. This beam of light is highly concentrated in the longitudinal direction of the conveyor belt. The benefits of this are as follows: 1) It concentrates the available light into the field of view of the camera only, and thereby keeps the required optical and electrical power to a minimum, with the result of reducing the cost of the light sources, and reducing the power consumption and excessive heating of the equipment and its surroundings. Thus e.g. light emitting diodes maybe used for illumination purposes. 2) It keeps the illumination of the surroundings to a minimum, which is considered beneficial to people and optical and electronic equipment operating in the vicinity of the present invention. Also, reflections from the environment which also may disturbed the measurements are avoided.

As mentioned above this may be achieved with illumination means based on the solutions described in US 5063460 and US 5777314, wherein the light sources on the sides of the camera illuminates the passing object sequentially and not simultaneously, as is suggested in these publications.

Fig. 2 shows the front view of the invention 1 in the same setup as in Figure 1. The beams of light from the two light sources 5, 6 are shown with dotted lines, and as can be seen, each light source illuminates the entire width of the conveyor belt.

Fig. 3 shows the rays from the two light sources 5, 6 as they are reflected off the object and into the camera objective. The object in this illustration has a perfect flat surface, and in such cases, specular reflection may represent a problem to the image quality at only on specific points on the top surface. Simple geometrical analysis shows that with two separate light sources 5, 6, the points 7,8 on the object surface that may give rise to such specular reflections into the camera objective does not coincide. In this illustration, the two light sources are located on each side of the camera, and simple geometrical analysis shows that in such cases, the points 7,8 on the surface that may give rise to specular reflections are located on each side of the centre line of the camera. It should be noted that in the application in mind for the present invention, the objects may have uneven top surfaces, with local variation of the surface normal. The consequences of this is that instead of there being one point on the surface that may cause specular reflection into the camera, there will be a larger area around this point which may give rise to specular reflections into the camera objective. Because of this, the two light sources should be located as far apart as possible in order for the two regions, which may give rise to specular reflections, not to overlap. The type of reflections will depend to some degree on the reflecting surface, so that a certain margin should be applied to the angle between the light sources and the camera.

Figs. 4a and 4b illustrate the alternating illumination of the object where the even numbered lines are captured with the right light source 6 on and the left source off (see Fig. 4a), and the odd number lines are captured with the left source on and the right off (see Fig. 4b). Fig. 4 also shows the pixel grid in object space, with the pixels affected by specular reflection shown in black. As can be seen from this figure, and which is also shown in Fig. 3, the affected pixels does not coincide.

This is exploited in the present invention by first rejecting the pixel(s) which are affected by specular reflections, and then combining the remaining good sections from both of the two half images. Since the pixels which are affected in one half image, are unaffected in the other half image, it is always possible to generate a new complete super line consisting of the only good pixels from the two half images.

The combination of lines from the two half images into a new super line can be done in several ways, with varying degree of sophistications. In its simplest form, one can analyse two consecutive lines, e.g. line number N and N+l, pixel by pixel and for each pair of pixels select the darkest pixel. In this way the pixels affected by specular reflections will automatically be deselected because they are always brighter than their counterparts from the other half image which are not affected by specular reflections. This method has the benefit of being very simple, and requires little processing time. A disadvantage with this method is that since the darkest pixel is always chosen, the contrast in the resulting super image maybe lower than in the original half image lines.

Another method is to define a pixel brightness threshold, above which every pixel is assumed to be degraded by specular reflection and therefore should be discarded. The remaining pixels are then assumed to be good, and are used in the construction of the new super image. The combination of the remaining good pixels can be done either by selecting the darkest or the brightest pixel in each pixel pair, by calculating the average value of these pixels, or by doing some other combining of neighbouring pixels with varying degree of sophistication to enhance the contrast in the final image.

Fig. 5 shows the image of an address label on a parcel generated with a camera system and illumination based on the present invention. Fig. 5a shows the unprocessed image consisting of both the odd and even numbered lines, and Fig. 5b shows the same image after processing where the pixels affected by specular reflections are discarded. As is evident from figure 5a several parts of the image would have been unreadable because of specular reflections, especially the graphic pattern in the lower right part of the image which would have been almost completely hidden by the reflections if only the right light source had been active. This may be seen as every second pixel line in the image is white.

The image in Fig. 5b is generated by, for each pair of image lines, selecting the darkest pixel in each pair. When comparing the images in figure 5a and 5b it is also evident that the image in figure 5 a is distorted. Thus is due to the over sampling in the direction of movement. In order to obtain sufficient resolution in the direction of movement the spatial sampling rate must be twice the required sampling rate in the finished image and thus usually twice the resolution in the direction of the sensor line in the camera, i the processed image the resolution in the direction of movement is half of what was sampled, so that the processed image obtains correct proportions.

Fig. 6 shows a setup with light sources at two levels for extending the range of heights that can be covered, e.g. when the system is used for reading codes on objects having a large range of sizes. In this setup, the two lower light sources 9 and 10 are illuminated when the object to be measured is lower than a set threshold, whereas the two upper light sources 11 and 12 are illuminated when the object height is higher than the same threshold. The height at which this threshold should be set will depend on the type of application, and on the exact geometry of the setup. It should be noted that in order to make this setup possible within a reasonable compact configuration, the light sources and the camera field of view will have to be moved out of plane as illustrated in the side view of the same configuration shown in Fig. 7. The reason for this is that, if all were in plane, the lower light sources will obstruct the light from the upper light sources, and also the camera field of view. Therefore, in order to realize this configuration, the lower light sources must be moved out of plane of the camera field of view. How much out of plane depends on specific parameters of the actual configuration such as height above conveyor belt, height range of objects to be covered, width of conveyor belt, image line exposure time and several others. It may also be necessary to move the upper light source out of plane of the camera field of view as shown in Fig. 7.

Fig. 8 shows a block diagram of the present invention 1 integrated into a system in which it is intended to be used. This system consists of a linear array sensor camera 2, a means of focusing the camera 15 based on the height information of the object given by an external height sensing device 18, illumination sources 9, 10, 11,12, a means 19 for providing conveyor belt speed information, e.g. by measuring the speed or by indicating whether the belt is moving or not, a means 17 for controlling and synchronizing the light sources and the exposure of the image sensor to the movement of the conveyor belt, a means 16 for reading, analyzing and processing the image data from the camera 2 and sending the processed information to an external unit 20 for further image recognition and/or displaying the processed image.

The exact nature of the different parts of the system illustrated in figure 8, such as processing equipment, optical wavelength ranges and conveyer belt technology, is not relevant to the invention and may be chosen depending on the available technology as well as local requirement, e.g. from hostile environments, the main aspect of the invention being related to measurements on objects passing a camera. The camera used according to the invention is required to be able to sampled linear images, and thus should preferably comprise a linear imaging sensor. Cameras comprising two dimensional sensor may, however, also be used by extracting image information from a sensor line in the matrix. It should also be noted that even thougli the preferred system comprises a conveyer belt moving the objects other solutions maybe contemplated, e.g. moving the camera system and lamps relative to the objects, or changing the angle of the field of view of the camera and illumination means, as suggested in US 5063460.

Also, the invention is primarily described in relation to handling of packages and parcels, but it is clear that other uses may also be contemplated. For example reflecting objects in production lines which are to be inspected before marketing.

Claims

C l a i m s

1. Method for providing an image of an object moving with a known speed relative to a camera, especially objects being covered by a specular reflecting surface, comprising the use of a linear sensor camera adapted to sample a linear image with a chosen resolution of the object at a chosen sampling rate, the object passing in a direction essentially perpendicular to the image line, the method comprising illuminating the object from at least two directions in a sequence synchronized with said sampling rate, so as to obtain different illumination direction in adjacent samples.

2. Method according to claim 1, wherein the object is illuminated by two lamps being positioned on each side of the camera with a chosen angle between them.

3. Method according to claim 1, wherein the illumination is provided by lamps adapted to focus light toward the line being scanned by the camera.

4. Method according to claim 1 wherein the sampling rate and speed of the object are adjusted to obtain partially overlapping samples in subsequent images.

5. Method according to claim 4, wherein the sampling rate and speed is adjusted so as to obtain twice the resolutions along the axis of movement as along the image line.

6. Method according to claim 1, comprising a step of analyzing each sampled linear image for detecting specular reflections, e.g. when by detecting pixels reading intensities above a predefined threshold.

7. Method according to claim 6, comprising a step of rejecting individual pixels exceeding said threshold.

8. Method according to claim 6, comprising the steps of defining pairs of adjacent linear images and thus a set of pairs of adjacent pixels in the direction of movement, rejecting pixels above said threshold in each linear image, and combining the linear images by calculating the mean value of pairs of valid pixels and in pairs comprising one rejected pixel applying the value of the valid pixel, thus resulting in a new linear image with only good pixels.

9. Method according to claim 1, comprising the step of defining pairs of adjacent linear images and thus a set of pairs of adjacent pixels in the direction of movement, and rejecting the pixels representing the highest reflected intensity in each pair, thus resulting in a new linear image with only good pixels.

10. System for scanning objects moving with a known speed relative to a camera, comprising a linear sensor camera for sampling a linear image of a part of the object at a chosen rate, the image line being perpendicular to the direction of movement, at least two lighting means for illuminating the part of the object being sampled by the camera from at least two different angles, and control means coupled to said lighting means and to the camera for activating the lighting means in a sequence synchronized with the sampling, so that two adjacent image samples are illuminated from different directions.

11. System according to claim 10, comprising two lighting means positioned on each side of the camera.

12. System according to claim 10, wherein the lighting means are adapted to focus light toward the line being scanned by the camera.

13. System according to claim 10, wherein the sampling rate and speed of the object are chosen to obtain partially overlapping samples in subsequent images.

14. System according to claim 13, wherein the sampling rate and speed is chosen so as to provide twice the resolution along the axis of movement as along the linear image.

15. Method according to claim 10, comprising analyzing means for analyzing each sampled linear image for detecting specular reflections, e.g. when by detecting pixels reading intensities above a predefined threshold.

16. System according to claim 15, wherein said analyzing means are adapted to defining pairs of adjacent linear images and thus a set of pairs of adjacent pixels in the direction of movement, rejecting pixels above said threshold in each linear image, and combining the linear images by calculating the mean value of pairs of valid pixels and in pairs comprising one rejected pixel applying the value of the valid pixel, thus resulting in a new linear image with only good pixels.

17. System according to claim 15, wherein said analyzing means are adapted to define pairs of adjacent linear image and thus a set of pairs of adjacent pixels in the direction of movement, said analyzing means also being adapted to rejecting pixels above a chosen threshold in each linear image, and combining the linear images by calculating the mean value of pairs of valid pixels and in pairs comprising one rejected pixel applying the value of the valid pixel thus resulting in a new linear image with only good pixels.

18. System according to claim 10, comprising analyzing means for analyzing each sampled linear image wherein said analyzing means are adapted to define pairs of adjacent linear image and thus a set of pairs of adjacent pixels in the direction of movement, said analyzing means also being adapted to rejecting the pixels representing the highest reflected intensity in each pair, thus resulting in a new linear image with only good pixels.