WO2013102858A1

WO2013102858A1 - Apparatus for and method of measuring spectral information of an object or material

Info

Publication number: WO2013102858A1
Application number: PCT/IB2013/050035
Authority: WO
Inventors: Udayan Kanade
Original assignee: Udayan Kanade
Priority date: 2012-01-02
Filing date: 2013-01-02
Publication date: 2013-07-11

Abstract

An apparatus and method of measuring spectral information regarding an object or material is disclosed. The spectral information is gathered by shining light onto an object, and detecting light reflected, scattered and transmitted by it in various directions using light detection patches. Each light detection patch has a distinct wavelength sensitivity characteristic. In an embodiment, the above procedure is applied as the object under consideration passes under the apparatus. Also disclosed are methods of deriving useful information about said object.

Description

[Title established by the ISA under Rule 37.2] APPARATUS FOR AND METHOD OF MEASURING SPECTRAL INFORMATION OF AN OBJECT OR MATERIAL

This application claims priority from provisional patent application 10/MUM/2012 titled "Technology for Detecting Missing Components" filed in Mumbai, India on 2nd January 2012.

Technical Field

This invention relates to detection, analysis, measurement and classification of objects and materials. More particularly, this invention relates to detection, analysis, measurement and classification of objects and materials using detection of light spectra.

Background Art

Spectrum detection generally involves complex optics such as light collimation optics and spectrum separation optics, and costly and slow detection. The speed, cost and complexity are deterrents to using present spectrum detection techniques in a wide variety of applications.

Summary

The present invention detects various light spectrum related features of objects and materials. For any given object, the appearance of the object is completely defined by the knowledge of what wavelength falling where in which direction, given rise to light of which wavelengths emanating from where in which directions. This is a huge space of information. For example, appearance under any lighting conditions, in any environment, and using any kind of optics may be derived from this huge space of information.

In the present invention, a small set of information regarding an object, derivable from this huge space of information, is detected using direct detection methods. This small set of information shall be termed to be a compressed spectrum of the object. This small set of information allows us to derive important information about the object under consideration without having the entire space of appearance related information.

More particularly, the present invention involves shining a light onto an object and detecting light reflected, scattered and transmitted by the object in various directions using light detection patches. Each light detection patch may have a distinct wavelength sensitivity characteristic. In an embodiment, the above procedure is applied as the object under consideration passes under the apparatus of the present invention, thus producing different “views”, i.e. readings at various times as the object moves under this apparatus. Also disclosed are methods of deriving various useful information about said object.

DETECTING MISSING FOOD ITEMS

A particular application of the present invention is the detection of missing food items, for food packaging industry. In a typical setup, A conveyor system is conveying trays of food at a very high speed. Dispensers dispense food items in the trays. Sometimes, one or more food items are missing and these missing food items need to be automatically detected. The food packaging may be some food packaging other than trays.

According to an embodiment, the detection of food items is carried out by measuring light bounced off the food as it passes light detectors. Different food items have different light spectra, i.e. they reflect different wavelengths of light to a different degree. The light detectors are made to be sensitive to different spectra, and using the detected light, the presence of a component is detected.

The above and other preferred features, including various details of implementation and combination of elements are more particularly described with reference to the accompanying drawings and pointed out in the claims. It will be understood that the particular methods and systems described herein are shown by way of illustration only and not as limitations. As will be understood by those skilled in the art, the principles and features described herein may be employed in various and numerous embodiments without departing from the scope of the invention.

Bried Description of Drawings

The accompanying drawings, which are included as part of the present specification, illustrate the presently preferred embodiment and together with the general description given above and the detailed description of the preferred embodiment given below serve to explain and teach the principles of the present invention.

Figure 1 depicts an apparatus for measuring a compressed spectrum of a material or object, according to an embodiment.

Figure 2A depicts an apparatus for measuring a compressed spectrum of a material or object, according to an embodiment .

Figure 2B depicts an apparatus for measuring a compressed spectrum of a material or object, as viewed from the top, according to an embodiment .

Figure 3A depicts an apparatus for measuring a compressed spectrum of a material or object, according to an embodiment .

Figure 3B depicts an apparatus for measuring a compressed spectrum of a material or object, as viewed from the top, according to an embodiment .

Figure 3C depicts an apparatus for measuring a compressed spectrum of a material or object, as viewed from a side, according to an embodiment .

Figure 4 depicts a light detection patch, according to an embodiment.

Figure 5 depicts an apparatus for measuring a compressed spectrum of a material or object, according to an embodiment .

Figure 6A shows an apparatus measuring a compressed transmission spectrum of material, according to an embodiment .

Figure 6B shows an apparatus measuring a compressed transmission spectrum of material, according to an embodiment .

Figure 7 shows an apparatus measuring a compressed transmission spectrum of a material, according to an embodiment .

Figure 8 shows an apparatus measuring a compressed spectrum of a material as it falls under gravity, according to an embodiment.

Figure 9 shows a portable apparatus for measuring a compressed spectrum, according to an embodiment.

Figure 10 shows an apparatus for measuring compressed spectrum of a fluid, according to an embodiment .

Figure 11 depicts an apparatus for measuring transmission compressed spectrum of an object for an elongated light path, according to an embodiment .

Figure 12 depicts an apparatus for measuring transmission compressed spectrum of a fluid, according to an embodiment .

Figure 13 depicts an apparatus for measuring transmission compressed spectrum of a fluid, according to an embodiment .

Figure 14 depicts an apparatus for measuring transmission compressed spectrum of a fluid, according to an embodiment .

Figure 15 depicts an apparatus for measuring transmission compressed spectrum of a fluid, according to an embodiment .

Description

Figure 1 depicts an apparatus 199 for measuring a compressed spectrum of a material or object, according to an embodiment . An object 101 is being conveyed by a conveyor belt 102, or other means of movement. The object 101 is to be analyzed by the analysis method of the present invention. The object 101 enters an enclosure 106. The enclosure 106 shields the object 101 from light when a spectral measurement is being performed on the object 101.

In an embodiment, the object 101 is a tray of food being inspected for quality. The object 101 may be any other object or material.

Figure 2A depicts an apparatus 299 for measuring a compressed spectrum of a material or object, according to an embodiment . In one wall of the enclosure 206, a light source 204 is present. In the opposite wall, a light detector 205, along with optics 203 (such as a tube or lens) which preferentially sensitizes the light detector 205 to the light source 204 is present. Optics (such as a tube or lens) may be provided for the light source 204 too, so that the light from the light source 204 does not pollute the cavity of the enclosure 206.

Figure 2B depicts an apparatus 299 for measuring a compressed spectrum of a material or object, as viewed from the top, according to an embodiment . In one wall of the enclosure, a light source 204 is present. In the opposite wall, a light detector 205 along with optics 203 (such as a tube or lens) which preferentially sensitizes the light detector 205 to the light source 204 is present. An object 201 is being conveyed by a conveyor belt 202, or other means of movement. When the object 201 cuts the light path between light source 204 and light detector 205, the light detector registers a drop in detected light, and thus, the precise position of object 201 is known. Now, using the knowledge of the speed of conveyor 202, the precise position of object 201 is known for all instants after it cuts this light path. The light source 204 may be a LASER, so that position can be very precisely detected.

Figure 3A depicts an apparatus 399 for measuring a compressed spectrum of a material or object, according to an embodiment . An enclosure 306 has a light source 303 in it. The light source 303 may be any light source such as an incandescent lamp, LED, halogen lamp, etc. The light source 303 may be a light source generating light of a particular spectrum – for example neon lamps will produce a very particular spectrum. The apparatus may have multiple light sources like 303, of the same or different spectra placed at different locations. In an embodiment, the material or object being detected is possibly photoluminescent in part, and at least one light source 303 produces light of a spectrum that will excite the photoluminescence and at least one light detection patch can detect light of a spectrum created by photoluminescence.

As an object enters the enclosure, it is shielded from stray light, and the light falling on it is primarily light from the light source 303. This light is reflected, scattered in various directions and transmitted by the object that is to be measured. This reflected, scattered and transmitted light falls on light detection patches 305. Each light detection patch is capable of producing a reading at every moment of operation.

In an embodiment, the light detection patches 305 are positioned on the top of the object to be measured. Light detection patches may also be positioned to the sides of the object to be measured. Light detection patches may also be positioned to be under the object to be measured, but for this scheme to be useful, the conveyor or transport mechanism should be transparent.

In an embodiment, the light source 303 and light detection patches 305 are embedded in or very close to the walls of the enclosure 306. In another embodiment, the light source 303 and light detection patches 305 are separated from the space in which the measured objects sits or travels by a sheath of transparent material, such as glass or plastic.

Figure 3B depicts an apparatus 399 for measuring a compressed spectrum of a material or object, as viewed from the top, according to an embodiment . An object 301 is traveling on a conveyor (or other conveyance mechanism) in a light enclosure. At various positions of the object 301, it reflects, scatters or transmits light from a light source 303.

Readings from various light detection patches 305 are obtained at various precise positions of the object 301. The precise position of the object 301 may be known either by (a) having a very fixed position in the object conveying system for each object; (b) having a position detector (such as one based on cutting the path of light and known speed of the object 301, as disclosed above) or (c) by registering the time series obtained from the light detection patches 305 against an expected time series for the object 301. Light from a particular light detection patch may be detected for multiple positions of the object 301.

These readings will be within a particular range for objects which are acceptable according to set quality norms, and the readings will change for objects which are unacceptable. Thus, this method can be used as an inspection and quality assurance method.

In an embodiment, the light detection patch produces electrical output, which is converted using an analog-to-digital converter. In an embodiment, there are fewer analog-to-digital converters than there are light detection patches. Each analog-to-digital converter is responsible for converting the output of multiple light detection patches. An analog multiplexer circuit is used to choose the correct light detection patch to convert at the correct moment.

Figure 3C depicts an apparatus 399 for measuring a compressed spectrum of a material or object, as viewed from a side, according to an embodiment . An object 301 is traveling on a conveyor (or other conveyance mechanism) in a light enclosure. At various positions of the object 301, it reflects, scatters or transmits light from a light source. This light falls on light detection patches 305. In an embodiment, there is no light focusing optics between the object 301 and the light detection patches 305. The light reflected, scattered or transmitted by the object 301 falls directly on the light detection patches 305.

A light detection patch is an instrument sensitive to light. The sensitivity to light can be different for different wavelengths of light. The light detection patch can be sensitive to visible as well as invisible light such as infrared, ultraviolet, etc. The wavelength dependent sensitivity to light may be different for each light detection patch. Let the wavelength dependent sensitivity of light detection patch i to light of wavelength λ be σ _i(λ). If light of spectral intensity I _i(λ) falls on the light detection patch i, then the reading (value) output by the light detection patch is

r _i = ∫I _i(λ) σ _i(λ) dλ = <I _i,σ _i>..............(1)

Figure 4 depicts a light detection patch 499, according to an embodiment. Light detection patch 499 comprises a photodetector 409 and a frequency sensitive light filter 410. The photodetector 409 converts light falling on it into some detectable phenomenon such as voltage, current or charge. Such instruments are well known in the art. E.g. a photodiode, phototransistor, charge-coupled device, etc. In an embodiment, a photodetector of a relatively large area (3-5 mm) is used such as a photodiode of the said dimensions. When a large area photodetector is used, light can be detected accurately and very fast, almost instantaeneously. When readings can be taken instantaneously, the objects being detected can pass along without being slowed down, thus simplifying the design of the instrument, and improving detection throughput.

A frequency sensitive light filter 410 allows light to pass through it, but different frequencies (i.e. different wavelengths) of light suffer different transmissivities (and thus, different attenuations). If the basic sensitivity of the photodetector 409 is s _i(λ), and the transmissivity of frequency sensitive light filter 410 is t _i(λ), then the wavelength dependent sensitivity of light detection patch 499 is σ _i(λ) = s _i(λ)t _i(λ).

Even with the same photodetector, the frequency sensitive light filter 410 may be changed to achieve a light detection patch having a different sensitivity to various frequencies. The frequency sensitive light filter 410 may be a “color filter”, also known as “light gel”, which is usually a sheet with a particular chemical composition that absorbs certain frequencies more than others. The frequency sensitive light filter 410 may also be a tuned resonance filter such as a dichroic filter, a multi-layer filter, a thin-film interference filter, etc. The frequency sensitive light filter 410 may also be a combination of more than one filters.

Different photodetectors 409 may also be used for various light detection patches, e.g. some more sensitive to infrared, some more sensitive to visible, some more sensitive to ultraviolet, and so forth.

Figure 5 depicts an apparatus 599 for measuring a compressed spectrum of a material or object, according to an embodiment . An object 501 has light from light source 503 falling on it. Light reflected, scattered or transmitted by the object 501 is then detected by light detection patches 505. The light detection patches 505 are shielded from the light source 503, so that light from the light source 503 does not directly fall on the light detection patches, but falls on the light detection patches 505 only after being reflected, scattered or transmitted by the object 501. This shielding may be carried out, for example, by sinking the light source 503 into a hole in the ceiling of the enclosure, while the light detection patches 505 are at the level of the ceiling of the enclosure.

Figure 6A shows an apparatus 699 measuring a compressed transmission spectrum of material, according to an embodiment . Material 601 is passing on a conveyor belt 602. Alternatively, it may be falling through air or other gaseous or fluid material or vacuum. A light source 603 shines light on material 601, as it passes in front of the light source. The light from light source 603 passes through the material 601 and falls on multiple light detection patches 604. In this way, spectrum of light transmitted through the object can be detected. Similarly, light scattered in generally a forward direction may be detected.

Figure 6B shows an apparatus 699 measuring a compressed transmission spectrum of material, according to an embodiment . Material 601 is passing on a conveyor belt 602. Alternatively, it may be falling through air or other gaseous or fluid material or vacuum. A light source 603 shines light on material 601, as it passes in front of the light source. The light from light source 603 passes through the material 601 and falls on multiple light detection patches 604. In this way, spectrum of light transmitted through the object can be detected. Similarly, light scattered in generally a forward direction may be detected. In an embodiment, there are also present backward light detection patches 605 which detect light scattered or reflected back from the material 601.

Figure 7 shows an apparatus 799 measuring a compressed transmission spectrum of a material, according to an embodiment . A transparent conveyor 702 is conveying material 701. A light source 703 shines light on material 701, and multiple light detection patches 704 detect light passed by their respective filters. Similarly, there may also be present backward light detection patches 705.

Figure 8 shows an apparatus 899 measuring a compressed spectrum of a material as it falls under gravity, according to an embodiment. A stream of material 801 is continuously falling through air or other gas, fluid or vacuum. The material 801 may comprise a stream of solid particles, a liquid stream or a mixture of the above. In an embodiment, this stream of material 801 emanates from a conveyor 802. Alternatively, it may emanate from other dispensing mechanism such as a nozzle, a funnel, a hopper, etc. In an embodiment, the stream of material is finally carried away by another conveyor 806. It may similarly be conveyed away by other means.

As the stream of material 801 falls under gravity (or other forces), a light source 803 shines light onto it. Forward light detection patches 804 and backward light detection patches 805 detect light transmitted and reflected from the material.

Figure 9 shows a portable apparatus 999 for measuring a compressed spectrum, according to an embodiment. The portable apparatus may be in the shape of an easy to operate instrument such as a gun or a portable drill. One end of the portable apparatus 999 has a light source 903 and reflected light detection patches 905. A button 907 (possibly in the shape and location of a trigger) or other kinds of input signals the device to take a reading. A display 906 may show results. Alternatively, results may be communicated to another location, such as a separate computer or database.

The user brings this device close to an object or material to be measured, and then activates the button 907. The light source then shines light on the object or material. Light reflected and scattered back from the object or material shines upon the light detection patches 905.

Figure 10 shows an apparatus 1099 for measuring compressed spectrum of a fluid 1001, according to an embodiment . Fluid 1001 is flowing through a pipe 1002. The fluid 1001 may comprise liquids, gases or a mixture of the two, and may further have dissolved or suspended solids. A light source 1003 shines light onto the fluid 1001. Light detection patches 1004 detect light reflected, scattered and/or transmitted by the fluid 1001.

In an embodiment, the pipe 1002 is transparent in a section, and the light source 1003 and light detection patches 1004 are attached to the outside of this transparent section. In another embodiment, the pipe 1002 need not be transparent, and light source 1003 and light detection patches 1004 are present on the inside of the pipe 1002.

Figure 11 depicts an apparatus 1199 for measuring transmission compressed spectrum of an object 1101 for an elongated light path, according to an embodiment . Object 1101 is an elongated object such as a tube, cylinder or prism. Light source 1103 shines light on one end of elongated object 1101. As the light travels through the object 1101, it is guided (by total internal reflection, or by reflectors at the sides) to stay within the object 1101, while acquiring spectral properties. At the other end of the elongated object 1101 are light detection patches 1104, which detect spectral properties.

Light travels a long path in the object 1101. In this way, materials present in a very slight concentration may be accurately detected. For example, if a certain impurity could be present in a few parts per million, this will not change the usual appearance of the object, but for light which takes a long path through the object, the difference will be observable. Furthermore, the concentration level could also be accurately predicted.

The object 1101 may be a tube filled with a fluid whose properties are to be measured.

Figure 12 depicts an apparatus 1299 for measuring transmission compressed spectrum of a fluid, according to an embodiment . Fluid 1201 is flowing through a pipe 1202. Pipe 1202 has a large section that is straight, and the ends of this section are transparent to light. In an embodiment, the entire straight section is transparent, and light will be guided by total internal reflection. In another embodiment, the straight section has a reflector coating, and light will be guided by reflection. A light source 1203 injects light into one end of the straight section of pipe 1202. Light is guided through the straight section, primarily through the fluid 1201. Light exits the other end of the straight section of pipe 1202, and falls on light detection patches 1204.

In this way spectral properties of the fluid 1201 may be measured as it is flowing. Very small concentrations of materials can be measured since light takes a long path through the fluid 1201.

Figure 13 depicts an apparatus 1399 for measuring transmission compressed spectrum of a fluid, according to an embodiment . A fluid 1301 is flowing through a pipe 1302. Fluid enters the pipe 1302 through an inlet and exits through an outlet. In an embodiment, the inlet and outlet are not at the ends of the pipe 1302. A light source 1303 injects light into one end of pipe 1302, and light is guided by reflection or total internal reflection till the other end. Light detection patches 1304 detect light that exits the other end of the pipe 1302.

Figure 14 depicts an apparatus 1499 for measuring transmission compressed spectrum of a fluid, according to an embodiment . A fluid 1401 is present in a tank 1402. Light source 1403 injects a focused beam of light into the tank 1402. Reflectors are placed in the tank such that this beam is reflected multiple times inside the tank before coming out of the tank. The focused beam optionally passes through a diffuser, and then falls on light detection patches 1404.

In this way, a light beam travels a long distance in a tank before being spectrally analyzed, allowing detection of very small concentrations of material.

Figure 15 depicts an apparatus 1599 for measuring transmission compressed spectrum of a fluid, according to an embodiment . A fluid 1501 is present in a tank 1502. Light source 1503 injects a beam of light into the tank 1502. The tank 1502 has diffuse reflective walls, and the light beam is bounced all around the tank before exiting some optical exit hole in the tank. After exiting the optical hole, this light falls on light detection patches 1504 or light detection patches 1505.

Though some light may hit the exit hole immediately, a large amount of light makes a large number of bounces in tank 1502 before exiting, thus increasing the path length of light, and allowing detection of small concentrations of material in the fluid 1501. In an embodiment, only of the light detection patches 1505 (near the light source 1503), and the light detection patches 1504 (near an independent optical exit hole), are present. If the light detection patches 1504 are not present, the corresponding optical exit hole need not be present either.

In embodiments corresponding to Figures 11, 12, 13, 14 and 15, the group of light detection patches may be replaced by a spectrum detector based on well known principles of spectroscopy.

Mathematics of Detecting Missing Food Items

Using the apparatus of the present invention, specifically embodiments where objects pass under a light source and light detection patches, missing food components can be detected very accurately, and using very less computational power. The most basic idea is to compare the readings received with what would be expected of a “perfect tray of food”. This simple idea becomes slightly complicated because many food items will have a range of possible spectra, any of which is a correct spectrum. Solving this problem requires some mathematical treatment, which we present below. We get very robust and computationally light algorithms to detect missing food components.

A tray of food items (or other geometrical configuration) has multiple food components in it. A food component j outputs a spectral intensity I _ij ( λ) towards light detection patch i. (For the sake of this mathematical treatment, we treat two readings at different positions by the same patch to be two different patches.) The food component may itself be a mixture of food constituents, and this spectral intensity I _ij(λ) will thus vary according to the exact constituents on the surface of the food component. For example, a salad may have nuts and fruits, and the number of nuts of a particular kind that appear on top of the salad will vary slightly per sample. Let μ _jk be the fraction of food component j dominated by food constituent k. The spectral intensity I _ij(λ) output by food component j towards light detection patch i is thus

I _ij (λ) = Σ_k μ _jk I _ijk(λ) ............................. (2)

where I _ijk(λ) is the hypothetical spectral intensity output by food component j towards light detection patch i, if food component j had been completely dominated by food consituent k. Since the food component j is a mixture of its food consituents, we get that, for each food component j,

Σ_k μ _jk = 1 .......................... (3)

Apart from the restriction (3) above, each μ _jk should conform to at least

0 ≤ μ _jk ≤ 1 ........................ (4)

Some μ _jk may have stronger constraints on them, such as

a _jk ≤ μ _jk ≤ b _jk ........................ (5)

Restriction (4) obviously becomes a special case of restriction (5), with a _jk = 0 and b _jk = 1.

Now, the light that falls on light detection patch i is the sum of lights from all food components j

I _i(λ) = Σ_j I _ij(λ) ....................... (6)

Note that, there may also be light falling on it reflected by the tray itself, or from other parts of the enclosure or conveyor belt. Though we will try to minimize such light by choosing dark trays, dark paint inside the enclosure and for the conveyor belt, we may not be completely successful. In this case, consider all of these extraneous sources of reflected light as a special “food component” j = 0. j = 0 is, in reality, not a food component at all, but all the static non-food (or background) components that may reflect light. This component j = 0 has only one constituent k = 0, giving μ ₀₀ = 1, (a ₀₀ = b ₀₀ = 1) and I _i00(λ) set to the spectral intensity of the non-food (or background) components reflected in direction of light detection patch i.

From equations (1), (2) and (6), we get

r _i = Σ_j<I _ij , σ _i> = Σ_j Σ_k μ _jk <I_ijk, σ _i> ....................... (7)

Defining

α _ijk = <I _ijk , σ _i> .............................. (8)

we get the set of equations

r _i = Σ_j Σ_k α _ijk μ _jk ........................ (9)

Now, the coefficients α _ijk depend on the actual food constituents, and can be evaluated experimentally for each food component. The reading r _i is the actual reading acquired by light detection patch i. The unknowns are μ _jkthe i.e. how much of the kth constituent of the jth food component is present. The equations (9) become linear equations in the unknowns μ _jk. There are further linear constraints (3) and inequality constraints (5). The question is to find μ _jk that satisfy all these equations and constraints together. If such μ _jk can be found, then we have found a valid reason for all the readings r _ithat we have. Else, we have not, and that particular tray fails our test!

Methods of Solution. There are many ways of solving (3), (5) and (9) together. (A) Most basically, it may be regarded as a linear program (involving only feasibility, no optimization). Since this is a linear program in low number of dimensions, it can be solved very fast.

(B) If the a _jk and corresponding b _jk are all close together, i.e. if the appearance of every food component is almost fixed, then simpler, heuristic algorithms may be used. In this case, simplistically, we may assume the inequalities (5) to also be equalities. The problem becomes an overconstrained one, since there are now more equalities than unknowns. This problem can be solved using the Moore-Penrose matrix pseudoinverse, a well known method of solving overconstrained problems. The solution will not precisely satisfy all constraints, or in fact any constraint. If the error incurred in each equation is small, then we accept the tray, else we reject it.

Note that since all the matrix elements are fixed, and do not change per tray, the pseudoinverse may be computed in advance. Thus, all that has to be performed live is a low dimensional matrix multiplication, which is computationally very cheap.

(C) This also inspires an algorithm which may be used for larger tolerances in the inequalities (5). We initially replace an inequality of the type (5) by the equality

μ _jk = (a _jk + b _jk) / 2 .............................. (10)

(3), (9) and (10) is solved together using an overconstrained solver. In this solution, we allow the errors in the equations (10) to be larger than the errors in equations (3) and (9), by increasing the “weight” attached to each of the equations (3) and (9), and decreasing the “weight” attached to equation (10). If the solutions so attained are within or close to the tolerances specified in (5), then we have successfully solved the original system of equations, and our tray passes, else fails. Just as in (B), the pseudoinverse may be precomputed, leaving just one matrix multiplication to perform live.

(D) As a version of the same idea, we may consider (3) and (9) to be hard constraints, and convert the equations (10) to an optimization function. I.e. the problem becomes to minimize

Σ_j Σ_k(μ _jk - (a _jk + b _jk) / 2) ² ............................ (11)

subject to linear equality constraints (3) and (9). This is a very simple problem to solve using linear algebra. The linear equality constraints (3) and (9) together form an affine space of solutions. The optimization function (11) then asks us to find the closest point μ _jkto the point γ _jk = (a _jk + b _jk) / 2. This may be achieved by projecting γ _jkonto this affine space, which is numerically a pseudo inversion problem, and just as above, may be manipulated such that only a low dimensional matrix multiplication needs to be performed in real time.

(E) Another version of the same idea is to initially solve the equations (3) and (9) together as an overconstrained system. This is possible if the number of equations in (3) and (9) is equal to or more than the number of unknowns μ _jk. Once such μ _jkare found, we test whether they satisfy (5).

Designing the Color Filters. The color filter spectral transmissivities t _i(λ) are hidden in the coefficients α _ijk of the present system of equations. For the faster heuristic methods to work robustly, and to give maximum testing in a minimum number of readings, the color filters should be chosen such that the system of equations (9) is well conditioned. This may be done computationally as follows.

Assume that we have a large library of color filter spectral transmissivities t _i(λ). Many manufacturers of color filters (such as Rosco) publish this as data sheets. Very highly wavelength selective filters can be created using dichroic filters (see Wikipedia on this topic). Further assume that we have the spectral intensities I _ijk of all constituents k of all food components j in direction of all light detection patches i. This data may easily generated by spectrometry, and needs to be generated only during the design stage. Also assuming that we know the photodetector spectral sensitivity s. Then the α _ijkmay be evaluated using equation (8) and numerical integration. (See equation (1) for the exact formula of the integral that needs to be evaluated.) These may be evaluated for all possible filters, even before knowing which filters we are choosing. The color filters need to be chosen, such that the α _ijkdo not form a ill-conditioned system of linear equations (9). Adding a color filter adds a row to the α _ijkmatrix. Ideally, each new row should be chosen to be orthogonal or nearly orthogonal to the rows before, and yet not filled with zeros or very low numerical values (i.e. do not choose wavelengths not reflected by anyone!). This means that, after adding each row, check whether the condition number of the matrix remains good, such that it may be easily inverted. All of this needs to be performed only once while designing the detection methodology for a particular food combination, not on the actual line.

A Machine Learning Solution. Imagine the μ _jks as forming a space, with the r _is as measurable features in this space, as given by equation (9). The classification problem is whether this tray is correctly filled with food or not. This may be solved using classical supervised learning algorithms such as support vector machines, nearest neighbor, etc. In fact, the methods (A) to (E) given above may be considered to be problem-specific classifiers in this sense. Choosing a good set of features is very important in classifier design, and the above section “Designing the Color Filters” explains exactly how to do that.

The advantages of this method over an imaging method are obvious when seen from a machine learning point of view. Firstly, our “feature vector extraction”, i.e. finding the r _is happens in physics, not computationally! Feature design is actually color filter design. This is what will save tremendous amounts of processing effort over an image processing solution. Secondly, imaging can use only predefined RGB color filters of a camera, and thus a large amount of spectral information is lost immediately.

Another paradigm through which the present idea may be viewed is as follows: spectral analysis is the detection of material using either transmission or reflection spectrum. Spectral analysis is a well known and respected means of detecting materials. The present technology may be thought of as “compressed sensing” of the spectrum, where the entire detail of the spectrum is not found, but some readings known to be very relevant to the detection problem are taken.

How to Find Which Component is Missing. The above framework may be adapted to exactly identify which component is missing, and by how much. Initially, assume that we “allow” food components to go missing. We do this by allowing the food component to have any appearance between “none” and the actual spectral appearance of the food component. Note that the “none” appearance is not a black appearance, but whatever is the spectral component of the tray that will now become visible because of the lack of food in that place.

Recall that equation (2) expressed the spectral intensity I _ijof a food component based on the spectral intensities of food constituents I _ijk and the fractions μ _jk. The fractions μ _jkexpressed what fraction of component j was dominated by constituent k. Now we will consider the absence of this particular food component to also be a special “food constituent” for that food component, the constituent k = 0. I.e., for each light detection patch i and food component j, I _ij0specifies the spectral intensity at patch i caused by the tray due to absence of the food component j. Now, μ _j0 = 1 will indicate that food component j is missing whereas μ _j0 = 0 will indicate that food component j is present. An intermediate value of μ _j0 will indicate that the food component is partially missing, i.e. the tray is partially covered and partially empty.

Since we are “allowing” food components to go missing:

0 ≤ μ _j0≤ 1 ........................ (12)

For all other food constituents, their fractions will now have constraints based on their relative presence compared to the other food constituents. I.e., for each k ≥ 1, the equation (5) is replaced by

a _jk ≤ ( μ _jk/ (Σ _k'=1 ^kj μ _jk') ) ≤ b _jk ........................... (13)

where kj is the number of constituents in food component j. Using equation (3), the above is rewritten as

a _jk (1 - μ _j0) ≤ μ _jk ≤ b _jk(1 - μ _j0) ............................ (14)

These linear constraints replace the earlier linear constraints (5). The equations (3), (9) and (14) now form the group of equations that need to be solved together. The methods (A), (B) and (E) as specified above are still directly applicable. The methods (C) and (D) are applicable with very minor modifications. Basically, in deference to using (14) instead of (5), these methods will have their a _jks replaced by a _jk(1 - μ _j0) and b _jks by b _jk(1-μ _j0).

Once we solve the system using one of these methods, we will get the μs. We look at all the μ _j0s. We want these to be close to zero. If any of these numbers is rather large, then that particular food component is missing to some degree. If this number is close to one, then that particular food component is completely missing. Also, we remember that the solution method itself may fail, which is also considered a rejection of a particular tray. For example, if a food component spills onto the tray, or onto other food components, or if some unwarranted substance exists on the tray, or if the tray is deformed in some way, the solution method itself will fail. In other words, the μ _j0test is over and above the original test given in the methods above.

Please note that even though finally, we want μ _j0 = 0, initially we allowed it to range between 0 and 1 in our formulation, so that we will get a number in this range, based on which we can find the missing component. If, on the other hand, we had mandated μ _j0 = 0 in our formulation, this would have been very similar to the formulation before, which will detect failure but not detect the reason for failure.

Detecting Food Amounts

The section “How to Find Which Component is Missing” above has given one means of detecting food amounts. But this is only able to detect an amount of “coverage” of food components. If the food component covers the tray entirely, but has a wrong level, then this is not detectable to the previous method. This is because such a difference is not very observable in the spectra sensed by the light detection patches. See light detection patch 1605 of Figure 16A and Figure 16B.

On the other hand, a light detection patch which is not on top of the food tray but to one side can detect the level of food (see light detection patch 1604 of Figure 16A and Figure 16B). If the level is low, the food will get “hidden” behind a tray wall. If there is no tray wall to hide behind in a certain implementation, then a special wall can be put up specifically for level detection. (It need not move with the tray.) Thus, using sideways patches, a better “feature set” is achieved if food components are going to be present at a wrong level, and this needs to be detected.

Detecting Food Shapes. Instead of a general illuminating lamp, we use a light source that illuminates the tray with a line of bright light. As the tray passes under this line, the line scans across the tray. The detectors continuously detect as the tray goes under the scanning line. Using this data, an estimate of the shape of each food glob may be made, and this shape may be converted computationally to the volume of food dispensed.

Detector Design

The section on “Designing the Color Filters” mathematically describes how to create detection methodologies that will optimally detect features relevant to the problem. Detectors may be designed by varying three things (I) Color filters, (II) Position and (III) Timing. Varying all of these gives a huge number of possible detection strategies, out of which a few will be chosen that optimally detect the relevant features.

(I) Color filters. Foods have very different reflection spectra, and foods of different reflection spectra may be distinguished by using appropriate color filters. This has been explained in detail before.

(II) Position. The position of the light detection patch will change what the light detection patch detects. Nearer items are sensed more than items far apart. Foods having similar spectra but at different locations (horizontally) on the tray can be distinguished using this light detection patch position.

(III) Timing. The same light detection patch, with the same color filter can produce output at various times. The exact moment at which the reading is captured will determine the exact relative position of the light source, the food tray and the detector. In fact, a single detector can be used to take multiple readings, or even a continuum of readings. Detection timing can be used to distinguish between different locations (vertically) on the tray.

Other Detection and Classification Problems

Even though the mathematics above has been developed with reference to detection of missing food items, the same mathematics and procedure is useful for many detection and classification problems. Consider an object or material made up of a multitude of material components j. Each material component may further be made of a multitude of material constituents k. The material components and material constituents may be treated exactly like food components and food constituents have been treated above.

Pure Classification Problem

Consider the problem of classifying material. For example, a particular geology application could be classifying the type of rock passing under the detector at the present moment. The same technology could be used to classify farm product, for example. (Separating kinds of produce, or separating raw from ripe, good from bad, etc.) We imagine each possible type of rock to be a material component, each material component comprising a multitude of material constituents.

The main difference from the mathematics developed above is, we do not expect the equation (3) to hold for all material components j, but for a particular material component j. In fact, we expect the summation in (3) to be zero for all the non-present types of rock, and one for the present type of rock. One way to solve this, is to solve the altered (3) (with one one and all else zero), (5) and (9) together. Since we do not know beforehand which type of rock is actually present, we do not know how exactly to alter (3). One simple solution is to alter (3) in all possible ways, and choose the way which produces a positive result. In other words, we may try to solve (5) and (9) together, but assuming the μjk are non-zero for only one type of rock. If a feasible solution is reached, it flags the detection of that particular type of rock.

Pure Detection Problem

Consider the problem of detecting presence and level of material in a mixture. For example, detecting pollutants in air or water, detecting constituents in a reacting mixture, etc. This may be treated to be a single material component, multiple material constituent problem. All the methods developed above are applicable. The final μ _jks indicate the presence of each constituent.

Uses in Food and Packaging Industry

This technology can be used to detect missing or partial food components on a food tray. It can also be used for food quality control: since changes in food quality (due to aging or because of changes in preparation) usually change the food’s color or appearance (reflection spectrum), this can be detected by the present technology. If some food is stacked vertically, i.e. one over the other, one check needs to be performed for each stacking. For horizontally separated food items, all items can be checked in a single pass. Even then, it may be beneficial to check after each food component is added so that rejects are identified early on and other foods are not introduced to an already wasted tray. (Furthermore, the tray with a missing food component can easily be introduced back into the line if other components down the line have not been added into it.)

This technology can be approved for us in food processing very easily because there is no contact between the food and the detectors. The light source and detectors may be hermetically sealed behind glass windows. The detectors can be changed from outside very easily to adapt the technology to a new product. The detectors can also be chosen to be able to detect a large set of required foods, so that they do not have to be changed at all, or a very few among them need to be changed to adapt to a new product. There are no moving parts apart from the conveyor belt itself, giving robustness and ease of sealing.

Other Uses

This technology can also be used for industrial inspection of many manufactured items. This is a cheap alternative to vision systems etc., and thus can be replicated on the manufacturing line in many places with minimal changes in the algorithm necessary. This way, many rejections may be identified early and amount of wasted material and effort reduced.

The present invention may be used in chemical manufacturing or material manufacturing industry. Embodiments that apply to flowing fluids or fluids in a tank are directly applicable to the chemical industry. The present invention can be used to accurately determine chemical composition, and from that calculate rate of reaction, yield, impurities, etc.

The present invention may be used to measure properties of farm produce, either at the farm itself, or while accepting farm produce in the food processing industry. Thus, detecting ripeness, type, presence of foreign contaminants, presence of fungus, etc. can be cost effectively performed with the present technology. Hand held, gravity based and conveyor based embodiments are applicable in this area.

The present invention may be used to measure constitution of rock, soil, etc. for geology, geological surveys and in applications such as mining and oil and gas. In geological exploration and in mining, the hand held embodiment of the present invention is applicable. In prospecting for shaft mines and oil&gas, various properties of rock can be classified as rock cuttings exit the hole being drilled. For example, the type of rock, mineral content, oil content, porosity, etc. may be revealed. For oil and gas, the rock may be washed, and the wash liquid (which could have hydrocarbon content) and the clean rock may be analyzed separately. The rock may be ground to finer samples before analysis.

The present invention may be used to continuously monitor the constitution of fluid flowing in a pipe. For example, the hydrocarbon percentages of an oil and/or gas well can change throughout it's lifetime. The present invention can be used to accurately track these changes, and to get the exact composition of the currently flowing oil and gas mixture.

The present invention may be used to monitor environmental impurities, such as impurities in air and in water. Presence of poisonous substances in water, pollutants in water and in air, pesticides and harmful chemicals, etc. can be detected. Composition of fresh water and sea water may be analyzed.

The present invention may be used in dermatological and medical applications. Various parameters of skin and body may be more accurately predicted using the present invention.

An apparatus and method of measuring spectral information of an object or material is disclosed. It is understood that the embodiments described herein are for the purpose of elucidation and should not be considered limiting the subject matter of the present patent. Various modifications, uses, substitutions, recombinations, improvements, methods of productions without departing from the scope or spirit of the present invention would be evident to a person skilled in the art.

Claims

A system of sensing a first material, comprising:

at least one light source, and

a plurality of light detection patches, wherein

the light source shines a first light on the first material,

the first material emits light directly into the plurality of light detection patches, and

the plurality of light detection patches collectively bear more than one wavelength sensitivity characteristics.
The system of claim 1 wherein the light emitted by the first material is deflected first light.
The system of claim 2 wherein deflected first light comprises reflected first light.
The system of claim 2 wherein deflected first light comprises scattered first light.
The system of claim 1 wherein the light emitted by the first material is transmitted first light.
The system of claim 1 further comprising means of conveyance for the first material.
The system of claim 1 wherein the a light detection patch of the plurality of light detection patches comprises a photodetector and a frequency sensitive light filter.
A method, comprising:

shining light on a first material,

detecting light emitted by the first material using a plurality of light detection patches, wherein

detecting light using a light detection patch of the plurality of light detection patches comprises detecting light emitted towards a particular area with a particular wavelength sensitivity.