WO1992017287A2 - Device to uniquely recognize travelling containers - Google Patents

Abstract

The present invention relates to a method and device for identifying individual linearly constrained containers based upon visual information, automatically (without human intervention). In accordance with the present invention, the identification is performed without requiring special equipping or marking of the containers.

Description

DEVICE TO UNIQUELY RECOGNIZE TRAVELLING CONTAINERS

Background

A major economic factor in the transportation industry is logistics: not only is it important to make efficient plans for moving passengers and cargo from place to place, but also to know where the elements of the transportation fleet are so that plans can be updated if necessary, so that interconnections between carriers or modes of transportation can be made, and so that shippers, receivers and carriers can determine the location, of. a particular shipment or vehicle for their own planning and billing purposes.

Transportation can be quite complex. For example, if a container of some commodity is to be moved from an origination point to a destination point, it may travel by automated guided vehicle (AGV)T or conveyor belt to truck; by truck to a local rail yard; from there by a series of trains, often owned and operated by multiple carriers through multiple classification yards, and interchange points; then by another train to a barge, seaport, or airport terminal; then by barge, ship or airplane to a second terminal; from there by rail to a local rail yard; by truck to the destination point; and finally by internal conveyor system to the location where it is emptied. Other possible modes of conveyance include container on truck chassis, enclosed pneumatic tube and external conveyor system.

At some portion of each step of its journey, the container may be thought of as a "linearly- constrained container", hereinafter referred to as an "LCC"), by which is meant a container which remains stationary or is constrained to travel for at least some portion of its route along a predetermined (although not necessarily permanently fixed) path (hereinafter referred to as a "channel") , and includes (but is not limited to) such common vehicles as railway cars including locomotives (constrained by railroad tracks) , motor vehicles and tractor trailers

(constrained by roads) , marine vessels (when operating in defined shipping channels and thereby constrained) , airplanes (when operating within predetermined ground paths) and containers on conveyor belts (constrained by the conveyor path) or AGVs (operating within predefined paths) . Such an LCC may contain cargo as in the example above or may carry passengers, as would a passenger train, airplane, cruise ship or bus. An

LCC can move along a path individually or a train of individual LCCs can move along the path as a unit.

Effective logistic planning requires that the location of each LCC be known at certain points of its journey (hereinafter referred to as "check points") . In the above example, when a particular truck arrives at the local rail yard, it is necessary to know of its arrival so that steps can be initiated to transfer it or its cargo to the train. Likewise, when a train arrives at the switching yard, it is necessary to know that so that the container of the commodity can be switched to a train heading for the correct destination. Furthermore, if there is a delay in any portion of the trip, it may be necessary to reschedule connections. Thus, it is critical to know when the LCC has arrived at any switching point, and it is important to know of the LCCs progress at certain interim stages and when standing order has changed. Several methods of acquiring such location information have been used.

When there are few LCCs in the transportation system or little value is placed on the speed or accuracy with which location information is gathered and processed, human visual observation is sufficient. For example, during the early 1800's when

SUBSTITUTESHEET a stagecoach (an LCC when traveling on a predetermined road as, for example, in a town) arrived in a small town once a week, all that was necessary was for "local merchants to look out their windows. It was sufficient for the manager of a way station, where horses were to be changed and passengers fed, to know the day on which the stagecoach was due—the approaching cloud of dust gave enough advance warning of its pending arrival. As the size and complexity of the transportation system grew, efforts were made to organize the increasing volume of information necessary to manage the transportation fleet. For example, there are now millions of rail cars in service in the United States. Modern freight cars (LCCs constrained by rails) are typically marked with a four letter initial identifying ownership and a six digit number uniquely identifying the particular car. Until the early 1980's there had been no reliable improvement over direct visual observation of freight cars at classification yards or shipper terminals and manual entry of data by human operators. The drawback should be clear: it is almost impossible to avoid clerical errors when manually entering a large number of six digit combinations, and the time involved to do so introduces a delay which makes a real-time picture of the state of the transportation fleet impossible. The opportunity for error and delay increases each time the LCC passes through a check point.

Partially automated solutions have been made possible by advances in the technologies of data processing, optical character recognition and radio frequency transmission and reception although, as will be seen, all have drawbacks.

One advance was simply to allow the clerk checking cars to do so remotely. In 1985, CSX Rail Transport began installing closed-circuit television

SUBSTITUTESHEET systems at its major terminals to eliminate the prevailing method of a clerk standing and checking cars as the train passed, or "walking the train" once it had arrived. Suitable systems are manufactured by CCTC International and Video Masters. The improvement of digitizing processes allowed for the transmission of video information by modem to a remote location where the information could be restored to an analog image. Cameras could then theoretically be placed at any location. While this advance provided greater flexibility and improved the comfort of the clerk, it did not eliminate the risk of human error or the delays in getting information.

An early effort at automation involved the use of barcoding. The barcode technique is widely used in supermarket checkout counters, where readers scan barcode markings on packages that are passed over a reading window and the markings are interpreted so as to identify the class of goods in the package. Barcode interpretation is illustrated in a number of issued U.S. Patents, including U.S. Patent Number 4,554,446, entitled "Supermarket Inventory Control System and Method", issued to Murphy, et al. which is incorporated by reference. This technique, however, requires specially designed markings on the package.

The Automatic Car Identification (ACI) system was an attempt to identify rail cars by placing color barcodes on the sides of the cars and attempting to read them using barcode scanners. While theoretically similar to the supermarket system, the vast difference in environments in which the two systems operate introduced significant operational difficulties. The supermarket environment is well-controlled and the package may be placed in close proximity to the scanner. Railroad cars are exposed to rain, snow, mud and abrasion which can obscure or

SUBSTITUTESHEET distort the barcodes, and the sensors must be placed a greater distance from the car. Moreover, railroad cars pass the scanner at considerably higher speed than do supermarket packages and there is no opportunity to back up a railroad car for rescanning. In addition, while the Association of American Railroads ("AAR"), a rail industry trade association which is empowered to define industry standards, adopted standards for how the bar code labels should be affixed to the car, in practice these standards were not always followed. This caused problems in reading the labels reliably. Furthermore, while all of the above factors helped to cause the ultimate failure of this system, a further problem was that all the cars to be identified required modification (the affixing of the bar code label) . Real benefits from the system could not be realized until the vast majority of the car fleet was labelled. Because of logistical difficulties, the entire fleet never was labelled with bar codes.

Another approach currently being considered is "Automatic Equipment Identification", or "AEI". It utilizes radio frequency transponder tags ("RF tags") rather than bar-coded labels. Such tags are placed on the vehicles in the fleet, and tag readers are placed in locations where the information as to fleet status is to be gathered. One manufacturer of such tags is Amtech. Voluntary AEI standards for technology, formatting, and content of .AEI tags were approved by the AAR on October 12, 1990. A number of railroads have begun to equip portions of their fleets with RF tags and the AAR's AEI Committee is currently considering mandatory tagging. The American Trucking Association has developed comparable automatic vehicle identification standards. Some initial tests have indicated that with the technology of such RF tags, a

SUBSTITUTE T reading accuracy of up to $99.99% may be achievable. Of course, that accuracy is only achievable for vehicles equipped with working RF tags. However, only a tiny fraction of the combined fleet of LCCs is so equipped, and the material expense (excluding labor) of adding transponders to the entire fleet would be substantial (anywhere from $30 to $125 per LCC) . Even if economies of scale were to bring fleet-wide tagging within the reach of large carriers (such as railroads and national trucking fleets) , the cost for individual vehicle owners (such as independent truckers or owners of private rail cars) would be high. Unless the entire national fleet were equipped with AEI tags, every system would have to provide for manual recognition. A system provided for Burlington Northern by Union Switch and Signal is designed to identify the act that a car does not have a working RF tag, thereby permitting determination of the location of tagged cars within a train, but does not address the problem of identifying untagged cars automatically. Even were the entire fleet so equipped, there would be substantial maintenance costs and delays due to downtime for tagging of cars and repair or replacement of tags and tag readers. CSX Transportation, Inc. has experimented with a hybrid of AEI and remote visual identification, allowing an operator to view an image of a railroad car remotely and a list of expected cars in order to attempt to identify cars which do not have an operative RF tag. In concept, an AEI reader reads the tags on a passing train and a video camera scans the same train. When a car is detected but does not transmit a recognized RF identifier, a video image of the car is transmitted for manual car identification. The preferred solution, however, is a passive system to recognize LCCs without requiring any special markings or devices to be placed on the fleet solely for the purpose of aiding recognition by a machine system.

Another approach considered by CSX is to construct a machine to read the numbers which may be printed on the side of an LCC which uniquely identify that LCC, incorporating currently available Optical Character Recognition ("OCR") techniques and equipment. This method breaks down, however, when the printed characters on the side of an LCC are obscured or degraded either because of lighting and imaging conditions or because of physical degradation or human error (such as the wrong number being painted on the side) . Therefore, a machine which only relies on machine-based OCR is not robust (viz. , reliable across all LCCs commonly encountered) . Furthermore, it is not obvious how to construct the imaging equipment (cameras and lighting) so as to enhance the capability that exists for an OCR component. The present invention can recognize a car even if it has the wrong number or the number is degraded beyond recognition.

Generalized pattern or object recognition by computer (being a general case of the special case of optical character recognition) , is well known in the art. See, for example, U.S. Patent Number 4,244,650 entitled "Automatic Optical Inspection and Sorting", issued to Garfunkel, et al. Patent number 4,611,347, entitled "Video Recognition System", granted to Netravali, et al., hereby incorporated by reference, pertains to an object recognition system and, more particularly, to an apparatus and a method for screening an image for reference patterns and selecting the reference pattern which most closely matches the image. Object or pattern recognition is finding wide applications in industry. The two main

SUBSTITUTE SHEET techniques utilized for object classification or recognition are template matching and recognition by features. In template matching, the objective is to find the best embedding of a template subimage in an observed image, over transformations such as translation. In practice, one approach is to store a set of possible views (or other image descriptors) so that any sensed image is sufficiently close to one member of the dense set of views. This approach suffers from at least two problems for many real applications. First, the set of views becomes too large for storage and efficient retrieval. Secondly, template matching (particularly matching of an entire image) is very time consuming for a large template library unless it is done in special purpose hardware.

Recognition by features, on the other hand, may have less accuracy of recognition, especially if simple features are used. Accuracy can be improved by including a larger or more sophisticated set of features, but this increases complexity and therefore computation time.

Object recognition may be performed using any distinctive characteristic of the object. One such characteristic useful in many cases is the silhouette of the object. If the silhouette of an unknown object can be matched to the silhouette of a known object, the unknown object can be considered to be of the same class as the known object. Since silhouettes are equivalent to two-dimensional patterns, well-known pattern-matching techniques may be used.

An important element of the silhouette technique is edge-detection, which is described in

U.S. Patent No. 4,969,202 issued to Groezinger, which is incorporated herein by reference. Once constructed, known and unknown silhouettes are deemed to be matched if the contents of a predetermined percentage of corresponding pixels are the same. This technique, however, though computationally simple, performs poorly if the silhouettes being compared are not in proper alignment registration with each other.

The technique also performs poorly if the image of the unknown object is a different size than that of the known object (as might be the case if the images were recorded by devices different distances from the object). Patent number 4,901,362, entitled "Method of Recognizing Patterns", granted to Terzian, (and hereby incorporated by reference) offers a solution by normalizing the silhouette of an unknown object for comparison with a similarly normalized reference silhouette, thereby improving performance when the silhouettes are not precisely registered or differ in size. The current invention requires fewer computations and is therefore faster.

.Another technique, which involves comparing one-dimensional Fourier transforms of a known image silhouette with the transform of an unknown image silhouette, operates well even if the known and unknown silhouettes are oriented differently. However, such a technique is fragile in the face of image degradation and can only identify classes of LCCs and is not practical for identifying individual LCCs.

All techniques which require storage of and rapid access to digitized complete images require the handling of staggering amounts of data. For example, representing one two-dimensional view of an object the size of a railroad car or a tractor trailer would require 9 megabytes if the object were digitized to an array 1000 pixels by 3000 pixels, using 8-bit representation for each of the three primary colors.

TE SHEET In summary, existing methods fall into two classes: human recognition and machine recognition. The human recognition method is prone to error and is expensive to implement. The machine recognition methods provided to date require modification of the fleet by the addition of special-purpose markings or devices fixed to the cars and the creation of an associative directory of existing LCCs (for example, a list of all RF tags and their associated cars) or an impractically large amount of computations and data storage.

The invention presented here overcomes those drawbacks while providing a method which requires no markings or additions to the LCCs.

Summary of the Invention

It is an object of this invention to provide a device which identifies individual linearly constrained containers ("LCCs"), based upon visual information, automatically (without human intervention) and as well as or better than human verification.

It is a further object of this invention to provide a device which can identify LCCs without requiring special equipping or marking of the LCCS.

It is a further object of this invention to provide a device which can identify LCCs marked with unique symbols without requiring that such symbols be legible. It is a further object of this invention to provide a device which can scan LCCs in a fashion which is suitable for machine-based LCC identification from visually available information.

It is a further object of this invention to provide a device which can visually scan and verify the identity of all the LCCs in a train of LCCs moving at various, non-constant speeds as the train passes a data acquisition site.

It is a further object of this invention to provide a device for identifying LCCs which is indifferent to weather conditions and highly robust against the effects of surface contamination.

It is a further object of this invention to provide a device for identifying LCCs which can track progressive changes in the appearance of the cars. It is a further object of this invention to provide a device which can visually scan and store the scans of all the LCCs in a moving train of LCCs as the train passes a site.

It is a further object of this invention to provide a device which can visually scan and transmit the LCC images of a train of LCCs moving at arbitrary speed as the train passes a site.

It is a further object of this invention to provide a device which can identify an LCC from an image of any view of the LCC without having to detect or otherwise sense a reference side (left or right side indifference) .

It is a further object of this invention to provide a device for identifying LCCs and developing, autonomously and automatically, high speed LCC identification methods.

It is a further object of this invention to provide a means whereby a human operator may associate a unique identifier with a given image, but which will then autonomously and automatically develop high speed

LCC identification methods.

It is a further object of this invention to provide a device for identifying LCCs which can start identifying an LCC after one previous view (per side) or zero previous views (on a side basis) of that LCC.

SUBSTITUTE SHE- It is a further object of this invention to provide a device for identifying LCCs by any external feature, including, but not limited to, examination of damage or surface defects or irregularities on the LCC.

It is a further object of this invention to provide a device suitable for illuminating LCCs for obtaining images of the LCCs which are improved for Optical Character Recognition and LCC identification. It is a further object of this invention to provide a device which can display the images of LCCs in a form improved for human recognition.

It is a further object of this invention to provide a device which can synchronize the action of a camera with a moving train of LCCs in a fashion which simplifies machine identification and precision image alignment.

It is a further object of this invention to provide a device which can determine visually the condition of an LCC in order to signal the need for repair, cleaning or repainting.

It is a further object of this invention to provide a device which may reduce the risk of unsafe conditions by checking conditions of LCC connections and equipment visually.

It is a further object of this invention to provide a device which can detect abnormal conditions, such as whether LCC doors are open or closed and the amount of closure. It is a further object of this invention to provide an improved database management system suitable for machine identification at the LCC rate of passage of large numbers of LCCs.

In its general form, the invention comprises a plurality of identification stations (#110) , each having access to a common LCC recognition image database ( 120) via access means (which may include physical presence of an embodiment of the data on site) (#130) , and reporting means (#140) for reporting the results of identification to an LCC location database (#150) which may be on site or centrally located.

Each identification station comprises a channel (#210) along which are located one or more sensing devices (#220) responsive to the presence of a linearly constrained container ("LCC", #230) within the channel, ambient light restriction means (#240) , lighting means (which may include natural light) (#250) illuminating the LCC, optical path shielding means (#260) , image acquisition means (#280) positioned to receive image data from the LCC as it passes in front of a background means (#270) and passing said data to the processing and control means (#290) , which is connected by communications means (#130,140) to LCC location database (#150) and LCC recognition image database (#120) .

Each processing and control means (#290) comprises multiple image buffer and processing means ("buffer/processors") (#310) which may independently accept or run tests on images received from the image acquisition means (#280) under the direction of the system control means(#320). The system control means monitors the sensing devices (#220) so as to direct the image acquisition means (#280) and the image buffer and processing means to capture LCC images one or more at a time, and then directs each of the buffer processing means holding images what test to perform on their respective images based on information from the car recognition data base and from the car location data base. As the system control means determines if it can identify each LCC, it will either update the car location database or, upon failure,

SUBS IT instruct the buffer/processor (#310) holding the image of an unknown LCC to pass it to image compression and storage means (#330) for future identification.

The general operation of the device is illustrated in the flow chart in figure 4. When a new car image has been captured in one of the buffer/processor means (#310) , the system control means (#320) directs and executes activities as described in Figure 4. It is important to note that there is a timeout interrupt in the system that forces the identification decision tree process to halt immediately if so much time has elapsed since the image was captured that the buffer/processor involved must be freed in time to accept the next image assigned to it.

It will be appreciated by those skilled in the art that slower travelling LCCs or faster computing means would require fewer buffer/processors, and the number shown in Figure #3 is for illustrative purposes only.

Comparison and updating are accomplished using a computer equipped with the appropriate special processing components and software described in the flow chart in Figure 4.

Brief Description of Drawings

Figure 1 is a block diagram of the system.

Figure 2 is a block diagram of a typical LCC identification station. Figure 3 is a block diagram of a typical processor control system.

Figure 4 is a flow-chart of software to control a general purpose computer in order to identify an LCC and respond to the presence of an unidentified LCC. Figure 5 is a diagram of the geometric constraints of the image acquisition means.

Detailed Description of the Invention The method of identifying linearly-constrained containers (LCCs) and maintaining an up-to-date database presented by this invention has numerous applications. For the purpose of describing the invention, the example of identifying individual cars of a conventional railroad train is used. The specific cars and the order in which they are connected is referred to as a "consist". One skilled in the art of identifying LCCs will recognize that the source of LCCs is illustrative only and that the present invention operates regardless of the type of LCC being recognized, provided only that:

(a) they meet the definition of an LCC; and

(b) each LCC has a unique identification code and at least one unique identifying visual feature, which may or may not include an identification code; and

(c) each LCC has known reference points, for example wheel center (for wheeled LCCs) or edge (for wheeled or non-wheeled LCCs) and which remains in alignment within specific tolerances characteristic for the LCC.

Furthermore, containerized cargo may be shipped in the same container in intermodal transport, travelling part of its route by railroad and other parts by ship, truck or airplane. Provided only that the container's appearance does not undergo any radical changes, not only will the present invention

SUBSTITUTE operate in substantially the same manner, but the same database described below may be used as well.

There are two separate and distinct requirements for the scanning system. one is recognition of an LCC that is part of the existing database of LCCs and the other is registering a new

LCC view into the database (note that registering a new view does not necessarily mean that a new LCC has been identified—it may, for example, be a second side of an LCC whose first side is already in the database or it may be that the appearance of an already recognized side has changed significantly since it was last scanned) . Separating the methods of car identification search and verification from car reading and learning greatly simplifies the design and reduces cost.

The recognition of a consist of railroad cars would proceed as follows.

The train is anticipated, either by advanced sensors or scheduling, so that artificial lighting has time to be activated if necessary. As the train approaches, sensors detect direction and speed travel. The measurement of speed is among the more precise measurements required from the non-imaging sensors since the image acquisition operation depends on speed measurements for estimating and controlling horizontal (orthogonal to the line scan line) resolution. Failure to achieve good continuous speed measurements can result in a great increase in computations necessary for LCC identification or can leave the identification in question. The most precision generally required is a speed measurement resulting in consistent images that only vary within one pixel over the length of the LCC; however it is generally accepted that the constraint can apply to an error of one pixel across the longest horizontal strike region (where "strike" is defined below and a "region" refers to a collection of horizontal strikes which are referenced by the same set of position-seeking strikes) . Also, horizontal registration (alignment) is achieved by a wheel detector at the bottom of the scan line object plane. Horizontal registration could also be achieved through image processing of regular features such as vertical side panel ribs, leading or trailing edges, wheel axles, or lines of rivets. A light beam passes diagonally across the track, from top to bottom, very close to the coupler so that the beam is occluded whenever there is a car in front of the scan line and is completed when there is not. The image is acquired using a high resolution scanner-type camera. Just before the beginning of the car crosses the scanning line, the detector light beam triggers the system to begin saving the image. Just after the end of the car crosses the scanning line, the detector light beam triggers the system to stop saving the image. Due to the way the light detector beams gate the optical scanner, the system produces a frame size that is just slightly larger than an individual car. It has been determined by experiment that resolutions of 1/8" per pixel are sufficient to discriminate between cars, although the required resolution will depend on the size and speed of the LCC being scanned. The scanning data may variously include (a) 8 bits each of red, green and blue color data, (b) 8 bits of gray scale data, or (c) few bits of either color or gray scale data dependent on the variety of appearances available for LCCS. For example, in railroad car recognition, color information of at least 6 bits in two chroma is necessary. It will be appreciated that imaging at less spatial or color intensity resolution, or in black and white, are proper subsets of the described

SUBSTITUT imaging method. Furthermore, it will be appreciated that selecting less resolution can dramatically reduce the bandwidth loading on the device and therefore the construction and operation costs. Lighting must either be controlled or the deleterious effects of not controlling it compensated. The control of light must be for both light intensity and chromatic characteristics. The light intensity is generally set at a minimum of 12 EV incident at the design LCC plane which is approximately the value for shaded sunlight. The required chromatic characteristics are generally determined by the particular color camera characteristics. In addition, if highly specular surfaces are common the lighting structure must be carefully structured.

The lighting towers contain many bulbs

(e.g., many fluorescent tubes) for robustness. If one tube dims or goes out, overall imaging and car identification, particularly that based on normalized computations, will remain robust.

Color cameras require white balance compensation and calibration against a standard in order to perform uniformly. It is desirable to place a white card at the principal image plane in order to assist the white balance computation, but this technique requires manual intervention which is not desirable. Art alternative technique is to compute white balance against the background plate (#270) adjusting for overall intensity. .Another alternative is to position small white bars across the viewing slit. Another alternative is to have a standard reference LCC, although this would not be practical in many situations. Another alternative is to locate the lights from the opposite side camera assemblies in such a way as to allow the lights from the opposite unit to act as the background (#270) . The technique has the advantage of enhancing the silhouette of the car and thereby improving the accuracy of the position-seeking strikes, of inspecting the condition of the lights, and of better enabling inspection for hanging cables in the undercarriage and other possibly hazardous conditions.

An LCC identification device may be housed or unhoused. A housed device completely encloses the LCC at the time of imaging. .An unhoused device shades the LCC at the time of imaging and controls the chromatic characteristics of the surfaces off indirect light (preferably white or black for color imaging) . The geometric constraints on imaging which permit improved imaging are illustrated and described in Figure 5. The housed device adds a closed, positive dry airflow system for the optical image path to the point of the source lighting plane. Positive airflow is desirable to keep the optical image path free of dust, debris, and fog. The housed device also adds complete, optically black, shading which seeks to prevent even highly acute sources of sidelighting from sunlight or primary indirect reflections. Allowance in the housed device must be made for strong air pressure changes as a high speed LCC enters and leaves the housed device. A housed device reduces the complexity of the camera lens and the calibration procedures and most importantly has a secondary effect of creating a device which is largely immune to environmental conditions. One example of an unhoused system for railroad controlled lighting has been built under a highway overpass and further baffled from direct sunlight by placing a black wall (baffle) along the track on both sides of the scan line to reduce the intrusion of natural light. Treatment of the grounds surrounding the unhoused device with black paint or tar will improve results. Nine feet from the center of the track, between the train and the line scan baffle, there are two columns of high output reflective fluorescent tubes to illuminate the object plane. More or less baffling may be required depending on the nature of ambient light and the means of compensation. A uniform background for the image is achieved by placing a solid colored surface on the opposite side of the track from the scanning system so that the scanner sees the colored surface in the opening between cars and other openings passes the scanner. This facilitates easy discrimination of car from background if the surface is chosen to be a color that does not appear on cars. An optical scanner head is positioned far enough away to minimize perspective occlusion and geometric distortion, and the simplification of camera lens and color element design, while being close to reduce cost. In the housed design, a hung float mirror may be used to fold the optical image path and thereby move it along the linear path of the LCC. In the case of railroad cars, the optical scanner head is positioned approximately 40 feet from the near side of the passing rail cars with a lens angle of acceptance that makes the object plane about 16 feet above the top of the rail so as to contain an entire car image. As indicated in Figure 5, it is necessary to keep the light sources as close to the LCC surface as possible to minimize light costs. The optical head produces 24 bits of data for each of a maximum of 2048 pixels in the line element (i.e. - 8 bits per color per pixel) . A line scan scanner is preferable to the more common video array cameras because of lighting control. To gain lighting control over a large region (e.g., an entire railroad car) requires a great deal of energy. The line scan camera allows lights (typically tubes or strings of flood or spot lamps along the scanned line) to be in near proximity to the surface to be imaged without obscuring any of the surface to be imaged. The cost to construct and maintain high identification or display quality LCC imaging for either housed or unhoused embodiments is greatly reduced by the use of the line scanning technique and is a distinct improvement over the prior art.

Suitable line scan scanner camera assemblies are available from Tangent Engineering Corp. of Denver, Colorado. These can be synchronized for the red, green, and blue inputs required. The output of the CCD line scan in each case is connected to a video processor which transforms the analog signal representative of the pixels in each scan line to a digital signal representative of a pixel as viewed by each CCD in each scan line. The red, green, and blue digital images are transferred to and stored in an image buffer. In applications where the side of the LCC cannot be guaranteed to be constant (as in railroad applications) , it is necessary to consider scanner assemblies on either side of the linear path of the LCC and to consider image acquisition adjustments to the direction of the motion. These simple adjustments will result in images, which, when viewed on a display device, do not appear left-right reversed and which give the same view of a car's side whether that side appears to one scanner assembly or the other or whether it appears going in one direction or the other along the linear path.

Since a railroad car may pass the scanner in less than one second after the previous car, additional image buffers, may be used so that while one is acquiring an image, an image processor can be running identification processes on a previously acquired image. Further, additional image buffers may be used to store a compressed car image to be used if recognition fails, and a new car must be entered into the database or the image must be presented to a person for ID verification. An active frame buffer with necessary I/O may be designed around a high speed CPU chip with built in DMA control for very high bandwidth operation such as the Intel i860 superscaler microprocessor chip. The active buffer should have 64 MB of DRAM configured for 32 bit wide access. Such a buffer system is capable of testing 10,0000 meaningful image and/or non-image features per second.

The image features comprise three classes,

(a) registration or position-seeking, (b) identity confirmation, and (c) inspection. It is necessary to know the exact position of the LCC image within a frame in order to minimize the computations associated with the position-seeking features. For example, this may be done by sensing either wheel position or edge position.

While LCC identification search and verification might conceivably be accomplished by global transformation on the database image of the subject LCC, this procedure would be too time consuming. Instead, the image is mapped directly into a data buffer, unaltered, and then subjected to feature extraction on only a relatively few image pixels. The selection of these pixels is accomplished by definition of a primitive operation, called a strike, which can be optimized for speed and performance at low cost and used in all facets including search, verification, reading, and learning. Such a strike can be only of the vertical and horizontal types if image resolution is sufficient to allow for safe strike positioning, i.e, that the result of offsetting a strike by a few pixels does not

rs; appreciably affect an outcome. It will be appreciated that the use of the terms "vertical" and "horizontal" herein is solely for convenience, and that the only requirement is that strikes be such as to permit the recovery of orthogonal data. If image resolution is low, the strikes must operate as region operators.

A strike is defined as a simple image feature test. An example is a single row of 128 pixels intersecting the object of interest along either a vertical or horizontal line or a minimum of two lines which enable the retrieval of orthogonal registration. A registration or position seeking strike is one that will perform a moving correlation in a large window (e.g., 256 pixels) with a single peak. The result of this operation is an estimate of the position of the LCC in the image. An identification strike will perform a moving correlation in a small window (e.g., 140 pixels) for which a single maximum value is inspected. An inspection strike will perform a moving correlation to confirm the condition of a region on an identified LCC.

These strikes may operate variously with unnormalized correlations and moments, normalized correlations, and normalized correlations on the absolute values of the derivatives of the image data. The conditions for using unnormalized correlations are for inspection strikes and others which may demand "blankness" in the image. The conditions for normalized correlations are for computing identification strike indices based on strikes of different lengths. The conditions for normalized correlations on absolute values of the first derivatives of the image data (in the directions of the strike) involve operations with cameras with automatic apertures and imaging in ambient (relatively uncontrolled) lighting conditions.

Strikes apply to linear combinations of red, green and blue image data. A function may be encoded with every strike to provide a linear weighting for the pixel values. Thus, for example, gray scale image data can be made equivalent to the simple sum of red, green, and blue values.

Strikes are powerful discriminators if used as described. Because these discriminators are so simple, they are computationally inexpensive to process in real time. Using a large number of strikes provides, either a strong match or strong mismatch in the car verification process. A search algorithm for finding cars in the image database may be used. Typically, there is a significant amount of a priori information concerning the consist - - given the date and time, a limited number of consists are expected. For example, it would be logical to first compare the current image against the expected image from advance consist data. Failing that, it would be logical to search images of other cars in the expected consist. Statistical data on the types of errors made in consist lists could be used to further refine the order of search through the rest of the image universe.

The database management system may also use caching strategies which seek to anticipate the information requirements of the system. Since consist data is available some time before the arrival of a train of LCCs, this time is employed by the system in establishing a recognition network for the LCCs in that consist which will minimize unnecessary tests. So, for example, if all LCCs are black, tests which confirm blue or yellow ones are only loaded after failure of all the more probable alternatives. The verification process will proceed as follows. First, vertical and horizontal strikes are used to check the vertical and horizontal registration of the car. Next, a large number of predefined strikes that are strong discriminators for the expected car are used for identification verification. The values of these strikes are stored in a periodically updated central database of known car identifiers. Each car will have a set of image operators associated with it that distinguish it from similar cars in its class. The image operators take advantage of features such as silhouettes, ID numbers, logos, placards, and rust patches that distinguish one car from another. Such features can be reduced to a set of related unidimensional weighted correlations with some normalization. If previous identification of the car is not verified, the image data may be analyzed and identifying 'strikes' defined and entered into the database before the image is removed from the buffer, or the image may be compressed and stored for later analysis.

An individual LCC or an individual LCC class (such as "hoppers" and "box" for railway cars, for example) is identified normally by an assigned structured set or complex of strikes. The goal of the identification process is to identify the individual LCC. It would be impractical for the machine to test each complex of strikes in sequence for over a million candidate LCCs. Individual LCC classes are used to select sets of candidate LCCs and reject other sets thereby reducing the number of strike complexes which have to be tested.

The process for selecting candidate LCCs,or candidate classes of LCCs may optionally incorporate consist data. If consist data is not present for a train, the process-selects candidate LCCs by the identification of the class membership of the target LCC. By successive refinement, a specific LCC is identified. If consist data is present or if some LCCs have already been identified, the process can shortcut the successive refinement procedure. The shortcut is enabled by maintaining a candidate LCC (or LCC class) list. Consist data can load the head of this list with specific LCC candidates which may be confirmed without any successive refinement tests. This list is called the candidate list and may include strike complexes for individual LCCs or classes of LCCs.

The candidate list is actively manipulated according to (a) the confirmation status of the consist data (viz., what LCCs have been identified) , including the selection of alternative consists if two trains may be reversed in their entirety, (b) the cost of a sequential search as against the cost of LCC class search, and (c) what LCC class information, if any, is available by other means (such as predominant color, number of axles, axle spacing, box car door detection, weight) .

While high image resolution such as 1/8" per pixel is not commonly necessary for directly reading a railroad car identification code, such high resolution improves recognition based on degradation and other fine car features (when the car identification code has been damaged, for example) .

Furthermore such high resolution simplifies feature extraction in that image region transformations and operations - do not have to be used. With the design described herein, the bandwidth requirements on the system are high only for loading, compressing, and uncompressing images. If the LCCs are expected to have highly readable identification codes printed on their sides, the choice would then revert to much

SUBSTITUTE SHEE^" lower image resolution and direct OCR algorithms which are nonetheless improved by the apparatus for image collection and image handling described herein.

An image is saved for later analysis if the system encounters an imminent time-out condition or if the system is instructed explicitly to store the image for servicing and calibration purposes. Images are saved in compressed form using methods which can control the amount of data which is saved even at the expense of losing some image resolution. Thus, in the presence of a time-out condition, there is a predictable amount of time necessary to store each image. This is also the case for image transmission and display where it is desirable to control the amount of data. A variety of image compression methods fit this requirement including the well known JPEG method. Even compression methods which do not generally guarantee known compression ratios can be given this property by compressing small squares and, if failing, compressing when an allowed data value is exceeded.

The harder task is identification of an unknown car. For example, the North American rail system fleet alone comprises more than one million cars; thus, it is important to narrow the range of possibilities before applying car-specific operators. Two general methods allow the quick and robust determination of the broad class of an unknown car. The first method is to determine the number and spacing pattern of the car axles. This data is obtainable from currently deployed magnetic wheel detectors. The second method uses color histograms. For example, the system should never confuse a white with a blue car. Further, it is possible to distinguish blue cars with orange lettering/logos from blue cars with white lettering/logos. The use of

SUBSTITUTE SHEET histograms also provides the ability to tune the code to give less weight to colors that correspond to short lived obscurations such as snow or dust.

Axle spacing or the use of color histograms alone can give only broad car class information, but the combination of the two has much greater discriminatory power.

The device is capable of learning to identify new LCCs by various processes including the reading of the identification code on a view of an LCC followed by the derivation of a set of strike complexes which can be employed to later confirm recognition.-.of_thatJiGC.. Thg- rjocess_which_derives a._ set of strike complexes is different for position-seeking, verification, and inspection strikes. For position-seeking strikes the objectives are to (a) discover strikes which represent unambiguous edges against which precise image coordinates can be derived, and (b) discover several, including orthogonal, position seeking strikes for locales in which verification and inspection strikes are to be used. A locale is defined by the amount of image distortion present in the worst case set up which includes but is not limited to error in estimating speed (so-called horizontal image distortion) . The method here is to categorize the type of LCC (e.g., hopper, tanker, box, in the case of the railroad) and then use rules specific to that type for seeking register. Types are categorized by precoded type categorization strike complexes. The method for deriving verification strikes involves developing strikes for areas which are determined to be identification code areas, strikes defined around centroids of natural growth areas (e.g., spots), piping configurations, placard placements, and the like. Finally the method for inspection strikes is again sensitive to LCC type and is both rule driven (e.g., inspect the undercarriage for hanging debris, inspect for an open door on a box car) and specially input (e.g., inspect a particular area for irregularity) .

Since strike placement is rule driven, the same LCC or a like LCC will yield similar strike placements. This feature enables the rapid evaluation of the effectiveness of a particular strike in discriminating the LCC in question, if two views of the same LCC are learned, the system will recognize that the same LCC is involved and the lack of discrimination between similarly placed strikes is employed not to prune the strikes, but to make database access more efficient by reducing storage requirements.

The addition of a car to the database would proceed as follows. The image is analyzed for determination of car type. This is accomplished by analysis of features common to all cars of a given type and unique to such type. A non-image example would be wheel spacing, which may be measured optically or by sensors located in the railroad track.

Based on car type, the image is further analyzed for areas that should be strong discriminators and information sufficient to identify the car in the form of 'strikes' is stored in the database.

Next, the identifying information must be associated with a unique identifier. Railroad cars may be registered with the American Association of Railroads under a unique identifier, which would be the most likely identifier to be used if the object is to produce a fleet-wide database, although the choice of identifier is not constrained by the present invention. The identifier is normally lettered on each side of each car and may be read into the database by optical character recognition techniques with or without verification by a human operator, or may be entered by a remote operator directly based upon an image provided by the scanning system that is presented to the operator. As noted above, many railroads have experimented with RF tags; in such case, the identifier may be transmitted automatically by that or similar verification devices. Any particular car may produce several recognition matrices, varying with weather, side of car, damage, graffiti or other factors. These will all be associated with the same car and will be applied to the recognition problem by computer algorithm. It will be appreciated that the less uniform the cars are, the easier the recognition problem. Thus, unlike RF tagging systems where adverse conditions such as corrosion create an installation or maintenance problem, with the present invention adverse conditions may affect the cars in ways which actually make recognition easier.

As a car's image changes with time, the recognition matrices associated with it can be updated each time the car passes an identification station. The database may also be updated to delete data images which no longer provide high correlations.

Other modifications of the present invention are possible in light of the above description which should not be construed as placing limitations on the invention other than those expressly set forth in the claims which follow.

Claims

WHAT IS CLAIMED IS; 1. Apparatus to recognize any uniquely identifiable container which is constrained to travel for a period of time sufficient for identification within a previously determined path without having to modify said containers solely for the purpose of identification comprising: means for visually scanning a side of said uniquely identifiable container; sensor means for determining a location on the uniquely identifiable container to facilitate analysis of scanned image; means for performing object recognition on identifying features of said uniquely identifiable container; means for determining whether said identifying features correspond to features of a known container; and means for determining said container's location at the time of recognition.

2. Apparatus for recognizing unique features of a container and locating the container automatically without having to modify the container for identification comprising: means for visually scanning said container to obtain image data; means for performing object recognition on said image data of said container; means for determining whether said image data includes previously scanned identifying features which uniquely identify the container; and means for identifying said container's location at the time of recognition.

3. Apparatus as in Claim 2 further comprising: means for automatically entering a learning mode when said image data does not correspond to any previously scanned features which uniquely identify a known container; and means for registering and storing at least portions of said image data as a new container.

4. Apparatus as in Claim 3 wherein the means for determining further comprises: means for automatically analyzing the image data to identify unique features of the container.

5. Process for automatically identifying containers by recognizing unique features of each container, the process comprising the steps of: measuring velocity of a container by detecting at least one reference feature on the container; scanning the container to obtain image data of the container, said step of scanning further including the steps of: adjusting a scan line rate in response to said measured velocity; and storing scan line data representing an image of said container in memory, a scan line number being associated with each stored scan line; detecting at least one predetermined reference point on said container; correlating said predetermined reference point to one of said scan line numbers; and creating a fingerprint image for the container by analyzing the stored scan line data relative to said predetermined reference point and establishing a unique collection of sampled portions of features representing intensity versus displacement variations across a surface of the container.

6. Process according to claim 5, wherein the step of measuring the velocity further includes a step of: detecting at least one wheel attached to the container using a magnetic proximity detector.

7. Process according to claim 5, wherein the step of detecting at least one predetermined reference point on said container further includes a step of: detecting an exact center of at least one wheel attached to the container using a magnetic proximity detector.

8. Process according to claim 7, wherein scan line data can reproduce a full color image of the container, said step of correlating further including a step of: relating said one scan line number to at least one wheel tangent point identified by the magnetic proximity detector.

9. Process according to claim 5, further comprising a step of: associating the fingerprint image for the container with an identification number of the container.

10. Process according to claim 9, wherein said step of associating further includes the step of: keying in the container identification number for registration in memory with the fingerprint image.

11. Process according to claim 9, wherein said step of associating further includes a step of: using optical character recognition to identify the container identification number.

12. Process for automatically recognizing containers comprising the steps of: scanning a container to obtain image data of the container; comparing the container image data with a set of previously stored container fingerprints, each fingerprint uniquely identifying a single container; determining whether said container image data matches any of said set of previously stored fingerprints; indicating whether a match exists between said image data and one of said stored fingerprints; and storing the image data in memory if no match exists between the image data and said set of previously stored fingerprints.

13. Process according to claim 12, wherein said step of comparing further includes a step of: establishing said set of previously stored fingerprints using consist data.

14. Process according to claim 13, wherein the consist data includes at least one of predominant color, number of axles, axle spacing and weight.

15. Process according to claim 12, wherein said step of comparing further includes a step of: establishing said set of previously stored fingerprints using a complex of strikes.

16. Process according to claim 12, wherein said step of determining further includes a step of: using only a portion of each stored fingerprint to determine whether the container image data matches any of the stored fingerprints.

17. Process according to claim 12, wherein said set of previously stored container fingerprints includes fingerprints of all previously scanned containers.

18. Process according to claim 12, further including a step of: updating features of a matched, previously stored fingerprint after a match has been indicated to exist between said image data and said matched, previously stored fingerprint.

19. Process according to claim 12, further comprising a step of: representing each of said fingerprints in memory as a collection of strikes, each of said strikes corresponding to a random location on a surface of said container.

20. Apparatus for automatically identifying containers by recognizing unique features of each container, the apparatus comprising: means for measuring velocity of a container by detecting at least one reference feature on the container; means for scanning the container to obtain image data of the container, said scanning means further including: means for adjusting a scan line rate in response to said measured velocity; and means for storing scan line data representing an image of said container in memory, a scan line number being associated with each stored scan line; means for detecting at least one predetermined reference point on said container; means for correlating said predetermined reference point to one of said scan line numbers and for creating a fingerprint image of the container by analyzing the stored scan line data relative to said predetermined reference point to establish a unique collection of sampled portions of features representing intensity versus displacement variations across a surface of the container.