CA2190596C - Method for collecting and processing visual and spatial position information - Google Patents

Abstract

Visual (130) and spatial (120) position information is collected and processed (132) for the formation of a geographic information database. In addition, the information collected and processed can be used to update the United States Census Bureau TIGER/Line files (134) to be more accurate files (146). Information collected and processed can also be used to accurately determine the spatial position of an object (144) seen in the collected visual information. The method includes the high speed collection and correlation of video images and spatial position information.

Description

WO 95/32483 l~ uv~.r~7~
21qO596. - -METHOD FOR COLLECTING AND PROCESSING VISUAL AND
SPATIAL POSITION INFORMATION
B~CKGROUND OF T~F I~EN~ON
The present invention relates to a method of using integrated 5 video and spatial position information to maintain and expand i"r.,l",~,;l, located in a geographic information system. The method irlcludes the high speed collection of video and spatial position data as well as fast and efficient storage of the data. Such a method is of particular use in updating the United States Census Bureau TlGER/LinelM files to improve the accuracy of coordinates contained therein, add missing street and address information, and add vehicle routing data. The method may also be used to create files similar to TlGER/Line files in areas where such files do not already exist.
The TlGER/Line files are a computer readable v~v~ ;C
database for the United States. The original sources of the database include U.S. geological survey maps, the 1980 U.S. Census Bureau DIME files, and various other sources. While these geographic files are usable as a base for a geographic i~lruJ~ iOII system, they are not re~omm~n-l~d for detailed applications which require a high degree of positional accuracy. This is because the positional accuracy of the information contained in the file is limited by the scale of the U.S. Geological Survey maps from which the files are created.
In addition, information in the TIGER/Line files is only as complete and accurate as the source (ln~lm~ntc used by the U.S. Census 25 Bureau and errors often occur in the comrilz~ti~n and translation of information from source documents. Further, the il~ru~ a~;u~l is only as current as the year in which the source (~ lm~ntc were compiled and translated.

WO95/32483 r~ l~U., '.'~
.
21 CJrJ596 Because the information contained in a geographic ill~UI IllaliUII
system such as the TlGER/Line files must be continually updated to reflect the most recent changes in the inform~tion contained therein, it is desirable to provide a method for collecting and processing video and spatial position information in a manner that allows updated attribute information to be tied to previous~y obtained spatial positional information.
One way in which to acquire accurate spatial position information is through the use of the Global Positioning System (GPS). The Global Pù~iLiù--i--g System is a network of satellites which constantly transmits its location at known, precise times. Since the orbital positiûn of each satellite is known, a GPS receiver on the surface of the eanh can accurately measure the time it takes for the transmitted signal tO reach the receiver's location on earth. This IllC~l~ul ~ provides the distance between the satellite and the receiver. To obtain three (lim-~nci~n:~l pO~iLiOllillg, the GPS receiver takes this IlI~.a~UI~,III~,II~ from each of at least four orbiting satellites. Using geometric principles, the ill~UI Ill~liUII gathered by the GPS receiver can be used to calculate the distance of each satellite with respect to the receiver and thus calculate an accurate spatial position on eanh.
While the Global Positioning System can provide accurate spatial position information, problems arise when the GPS receiver is not able to receive information from at least four orbiting satellites. This occurs when the GPS receiver is obstructed by geographic features such as canyons or bluffs as well as by man-made structures. Problems also arise when a signal transmitted by an orbiting satellite is reflected from a nearby object, giving the GPS receiver one or more incorrect readings from that satellite.
A further problem with the use of the Global ru~iLi~ g System to provide spatial position information is that spatial position readings are often provided by the GPS receiver at a relatively slow rate.

WO 95132483 l'~ .,S,'t -7~'~

SUMMARY OF ~IE INVF.NTlON
The present invention provides a method for collecting and processing video and spatia~ position information for the formation of a ~ngr~rh~ r~ u~ database~ Thedatabasecanbefurtherprocessedto S create additional databases which can be used for a variety of ~U,UIi~ iU
such as to provide access to video images of streets and their :~UllUUlldill~.
ln addition, the information collected and processed according to the method of the present invention can be used to update the United States Census Bureau TlGER/Line computer map files to improve the accuracy 10 of the files and add il~Olll~iOIl.
The present invention includes a collection system cu...,u.i~i..g video cameras, a GPS receiver, an inertial navigation system (rNS) and a control computer. The collection system is mounted in a moving vehicle such as a van. The GPS receiver provides satellite-based spatial position information and precise time values to the control computer while the INS
provides kinematic-based spatial position information to the control computer. At the same time, a video tape recorder controlled by the control computer records interleaved video images from the video cameras onto a single video tape.
Each frame of the video tape has a time code associated therewith. This time code is recorded by the control computer along with the current spatial position information provided by the GPS receiver and the INS at the time that frame is recorded. In this way, each recorded video image can be correlated with the spatial position of the camera at the time 25 the image was recorded.
A GPS base receiver is located at a known position and provides absolute spatial position information and absolute time values from the same GPS satellites as those used by the GPS receiver in the collection system. This position information is used to remove location errors in the WO 95/32483 1 ~
.
2 1 ~5qb recorded GPS spatial position illru. IlldLiU~l by a process known as differential processing. ., A centerline offset process is used to determine the true centerline of each street from which data was collected. The C.,.lt~ s are used to create street segments in a process called street s~", 1~ and each street segment is associated with a sequence of video images. The r~l~ti-)nchir between each segment and associated video images is stored in a database. Additional attribute infnrm~tir~n databases can be created for address location~ vehicle routing and point location ~rpli~lti(?nc among lû others.
Because the method of the present invention records the spatial position at the time each video image is recorded, the video images can be later reviewed as many times as is necessary to validate the accuracy and comrl~t~n~c~ of the attributes shown therein. In addition, because the ~5 INS provides six--lim~n~i~n~l spatial position inforrnation, the present invention allows the location of a point to be accurately determined from any two video images containing the point.
BRTF.~ DESCRIPTION OF THE DRAWINGS
Figure 1 is a perspective view of a vehicle in which the 2û collection system of the present invention can be located.
Figure 2 is a schematic diagram showing the collection system of the presenl inventi(?n.
Figure 3 is a schematic diagram showing a portion of the image collection system of the present invention.
Figure 4 is a schematic diagram showing a portion of the image collection system of the present invention.
Figure 5 shows the six--l;",~ l relative position of the vehicle provided hy the inertial navigation system.

2~ 90596 Figure 6 is a schematic diagram showing data inputs and outputs of the collection system of the present invention.
Figure 7 shows a series of steps which are followed in the present invention.
Figure 8 shows the manner in which a portion of a street centerline is determined.
Figure 9 shows a street segment created according to the method of the present invention.
Figure 10 shows a portion of a street segment created according to the method of the present invention.
Figure 11 shows the ~ ;.",~ , between a series of recorded video images and a street segment.
Figure 12 jc a schematic diagram showing a number of ways in which updated street segments can be used.
DETAII Fn DF.~CRIPTlOlsl OF THF PFFF~ Fn FMROD~MF~TS
In the method of present invention, accurate visual and positional information is collected in a rapid and efficient manner by a collection system 10. The information is then stored for post-collection processing, which places the data in a more easily usable form. After processing, variouc extraction processes are used to create a variety of geographic/informational databases. This is accomplished in part through the creation of street segments and the creation of indirect relationships between the segments and recorded video images. Generally, the collection system 10 is located on a moving platform such as a moving vehicle 11, as shown in Figure 1.
The col~ection system 10 for collecting video and positional information is shown in Figure 2 and includes a land reference mapping computer 12, a data ~gl~ici1ion control computer 16, a video tape recorder 20, an image control system 22 including a digital matrix router 23, a GPS
, .

WO 95132483 1 ~
.
21 qO5q6 receiver 24 and an inertial navigation system (INS) 28. In addition, the collection system 10 includes a first computer monitor 30, a second computer monitor 32, a keyboard 36, a selecting switch 38 and a trackball 40. Each of these ~u~ are located in the moving vehicle 11. A plurality of 5 batteries located in the vehicle 11 provide power to the collection system equipment even when the engine is not running. AC power is provided to the equipment through the use of a power inverter. The collection system 10 further includes a GPS base receiver 42 which is located at a known, fixed location separate from the rest of the collection system 10.
The control computer is linked with the video tape recorder 20, the digital matrix router 23, the GPS receiver 24, the INS 28 and the mapping computer 12. The keyboard 36 is selectably linked to both the control computer 16 and the mapping computer 12 through the selecting switch 38. In addition, the trackball 40 is electrically connected to the 15 mapping computer 12. The first computer monitor 30 is linked to the mapping computer 12 while the second computer monitor 32 is linked to the control computer 16.
The data acquisition control computer 16 operates a set of nested asyl,~l..ullous interrupt processing routines to provide real time 20 control of all data ~ licition equipment. The control computer 16 controls the signal routing of all data collected as well as the recording of the data by the video tape recorder 20. A video equipment controller 44 such as a Video Logic V-LAN'T~ controller is located in the control computer 16 and provides control of the video tape recorder 20 while it is recording. The 25 video equipment controller 44 is linked with the video tape recorder 20 by a RS-422 serial ~( mml-nir~tionc link.
Both the GPS receiver 24 and the INS 28 are linked to the control computer 16 by RS-232 communications links and provide spatial position information to the computer 16. The GPS receiver 24 ~ Tonn~fi~lly Wo95/32483 P( IIIJ,~

acquires, tracks and selects the satel~ites to be used for position and time ti~nc and provides location readings to the control computer 16 at a rate of d~ "d",ately three times per second. The INS 28 provides the control computer 16 with relative positional locations at the rate of thirty locations per second. The positional i"r.".,.A~ ,. or data received by the control computer 16 is transferred to a hard drive for p~ . .l storage.
Both the mapping computer 12 and the control computer 16 include a backup tape to provide for backup of the collected data.
The land reference mapping computer 12 provides real time tracking of the location of the moving vehicle 11. This is :~r~omrlich~l through an RS-232 communications link between the mapping computer 12 and the control computer 16, allowing positional il.roll.ldLioll to be transmitted to the mapping computer 12. A street map is displayed on the first computer monitor 30 shov ing both the current and previous locations of the vehicle 11. The map is initially based on current U.S. Census Bureau TlGER/Line files. The mapping computer 12 thus acts as a navigation aid for routing the moving vehicle 11 during the data collection process.
Figures 3 and 4 each show a portion of the image control system 22 of the present invention, which is not completely shown in a single figure for ease of illustration. Figure 3 shows the portion of the image control system 22 in which the video signals are routed. This portion of the image control system ~2 comprises the digital matrix router 23, ten video cameras 50A, 50B, 50C. 50D, 50E, 50F, 50G, 50H, 50I and 50J (collectively referred to as 50), four video monitors 54A, 54B, 54C and 54D ~collectively referred to as 54), a four-view generator 62, a time base corrector 64, a vectorscope 68 and ten video camera control units 70A, 70B, 70C, 70D, 70E, 70F, 70G, 70H, 701 and 70J (collectively referred to as 70). Each of the video cameras 50 is mounted in the moving vehicle 11 at a known poSition WO 95132483 P_l/U,,. '/C -~?
21 90~96 relative to the GPS receiver 24 and oriented to capture a desired view. The orientation of the video cameras 50 are as follows:
CAMERA VIEW
SOA Driver's Front S SOB Boulevard Front SOC Stereo Front SOD Right Side - Short Focal Length SOE Right Side - Long Focal Length SOF Driver's Rear 50G Boulevard Rear SOH Stereo Rear 50~ Left Side - Short Focal Length SOJ Left Side - Long Focal Length ~Video cameras SOB and SOG are located on remote control pan/tilt units and can be adjusted through the use of two joystick controls while cameras SOB, SOE. SOG and 501 have remote zoom and focus controls.
Each video camera 50 is electrically connected to a l,UllC~IJOlldillg camera control unil 70 which is used to provide normal camera head controls. Each 20 camera contro~ unit 70 is electrically connected to one of a plurality of inputs 74 of the digital matrix router 23 and provides a signal to the router 23. The distance between the GPS receiver 24 and each video camera 50 WO 95~32483 P~ J.,. 5.

'~ is measured so that the spatial position of each camera 50 can be determined from the collected spatial position information.
The digitaJ matrix router 23 is linked to the data ~ ";~
control computer 16 which controls the operation of the router 23. The 5 router 23 provides at a plurality of router outputs 78 any desired combination of signals received at its inputs 74 as instructed by software located in the control computer 16. Four router outputs 78 are electrically connected to the four-view generator 62. Each of three router outputs 78 are electrically connected to first, second and third video monitors 54A, 54B
]0 and 54C~ respectively, through the vectorscope 68. The two remaining router outputs 78 are electrically connected to the video tape recorder 20 and a fourth video monitor 54D respectively. Both the time base corrector 64 and a first output 80 of the video tape recorder 20 are electrically connected to router inputs 74.
The four-view generator 62 has four inputs 82, allowing signals from four of the video cameras 50 to be input cimlllt~n~ol~cly. The generator 62 reduces the ima~e represented by each signal to one-fourth of its original size and then combines the reduced images to form a single video image by placing each of the reduced images into one of the four 20 corners of an output image. This output image is referred to as a four-view image and is provided at an output 84 of the four-view generator 62, which is electrically connected to an input 86 of the time base corrector 64. The time base c~rrector 64 holds the four-view image in a buffer until the image is selected by the digital matrix router 23. This buffer is necessary because 25 four-view images created by the four-view generator 62 require a~ u~ cLely one-tenth of a second to become sufficiently stable to be accurately recorded by the video tape recorder 20.
In addition to sending signals to the four-view generator 62 and to the videû tape recorder 20, the digital matrix router 23 routes images W0 95/3_483 l _ l / U ~ ?
21qO5~6 from the video cameras 50 and the time base corrector 64 to the four video monitors 54. The order and duration with which each of the images is routed to the monitors 54 is determined by software in the control computer 16 which then instructs the router 23 which images to provide. The images 5 provided by the router 23 to the first, second and third video monitors 54A, 54B and 54C are routed through the vectorscope 68. The vectorscope 68 monitors the signals from the video sources provided by the router 23 and provides those signals to the first, second and third video monitors 54A, 54B
and 54C for normal video IllUllilUlill~. The VC~,Lul~,O~C 68 is used as a 10 diagnostic sou~rce and provides diagnostic signals from two diagnostic outputs 90 to channerB of both the first video monitor 54A and the second video monitor 54B.
The video tape recorder 20 comprises a Sony PVW-2800 BetaCam SP Video Tape Recorder and is controlled by the data 2~ itinn control cornputer 16. A second output 94 of the video tape recorder 20 is electrically connected to a second input 96 of the fourth video monitor 54D
and provides to channel B of the monitor 54D a signal containing the video images being recorded. The video tape recorder 20 records images provided from the digital matrb~ router 23 on Sony Betacam SP video tapes which are 20 generally 90 minutes in length. Each tape includes three video ~ulllpol~e~ll tracks, two audiû tracks, and a Society of Motion Picture and Television Engineers (SMPTE) time code track.
The portion of the image co~lection system 22 that controls the precise tim;ng of all video equipment is shown in Figure 4. A standard 25 video black reference timing generator 100 is used to ~I~ ul~iL~ the tirning of the image control system 22. The timing generator 100 has a first generator output ~02 which is electrically connected to the input of a first six-channel distribution amplifier 104 and a second generator output 106 which is electrically connected to the input of a second six-channel , . ,, .. _ . _ . , . .. , . ,, ... _ .. ,,, . . ,, . ,, .. , _ _ _ , , WO 95/32483 r~ . '.T '7~'~

amplifier 108. Both the first amplifier 104 and the second amplifier 108 amplify a black reference timing signa~ generated by the timing generator 100 and provide the timing signal at each of six first and second amplifier outputs 110 and 112, respectively.
The first distribution amplifier 104 provides the timing signal to the video tape recorder 20, the digital matrix router 23, the four view generator 62, the vectorscope 68 and the time base corrector 64. The second distribution amplifier 108 provides the timing signal to the video eql-irmPnt controller 44, the second camera control unit 70B, the fourth camera control unit 70D, the sixth camera control unit 70F, the eighth camera control unit 70H and the tenth camera control unit 70J. The first, third, fifth, seventh and ninth camera control units 70A, 70C, 70E, 70G and 701 are linked to the second, fourth, sixth, eighth and tenth camera control units 70B, 70D, 70F, 70H and 70J, respectively, to allow the timing signal to be provided to the remaining camera control units. Electrical circuitry with each video device ~y~ a National Television Standards ~'.ommitt~e (NTSC) video blanking interval with the black reference timing signal.
Before recording on the video tape, the cable ~fmnPt~fir~nc to the video tape recorder 20 are modified to provide a video black reference signal to the video tape recorder 20, which creates a series of individual frames on a blank tape. A time code generator built into the video tape recorder 20 provides an accurate marking of each of these video frames with a SMPTE time code. The SMl'lk time codes are written on the videotape in a continuous stream of non-drop frame time codes in an hours:minutes:seconds:frame format which comprises thirty frames (0-29) per second, 60 seconds per minute and 60 minutes per hour. The marking of the video tape permits video images to be accurately recorded to the video frames during the data collection process.

WO 95/32483 r._l/u~ 7 The control computer 16 can send a set of instructions to the digital matrix router 23 to perform a desired ..~ ."""~ sequence of predetermined routings called a cycle. Instructions sent to the router 23 are timed by the video e~ui~ controller 44. Instructions are sent by the 5 control computer 16 to the router 23 before the NTSC video blanking interval of the desired video frame SMPTE time code and router switching commands are queued with the router 23. When the desired NTSC video blanking interval of the black reference signal is detected by the router 23, the router inputs 74 are connected to the router outputs 78 as specified by 10 the commands from the control computer 16. The fr~nn~ction~ between the router inputs and outputs 74 and 78 are completed before the end of the NTSC video blanking interva] to provide seamless switching as often as once per blanking interval.
Each of the video cameras 50 is capable of obtaining images 15 at a rate of 30 frames per second. In addition, the digital matrix router 23 is capable of altering the routing of the signals received at its inputs 74 thirty times a second. The video images provided at the router outputs 78 can thus be altered as frequently as thirty times per second.
Within the instructions sent to the router 23 is contained a 20 sequence of the video cameras 50 from which video images are to be recorded by the videotape recorder 20. In this way, images from each of the video cameras 50 can be recorded in a desired order by a single video recording device. A typical 18-frame recording cycle is given as follows:

WO 95/32483 A ~ 7 FRAME C~MER~ VIEW FR~IE CAMERA
VIEW
4-View 5 9 14 4-View 64-View 15 7 t~1() 17 5 9 5 18 4-View g ~

The ~iven recording cycle is repeated c~mtinllol1cly during the data collection process, resulting in the recording of a plurality of interleaved video images from multiple video image sources onto a recording medium.
.. . .. . . . .. ... , .. ... _ .. .. _ .. .... .. _ . .. . . _ _ _ W0 9~/32483 P~,l/ll,, ~7~7 As can be seen, not only can the view from any video camera S0 be recorded, but so can any desired four-view image. The occasional recording of the same camera view for two consecutive frames proYides sequential side views which can be useful in terrestrial phuLu~ulull~LIy.
The recording cycle shown above can be dynamically configured during recording. The computer keyboard 36 permits up to ten preset recording c~Ycles. When a new recording cycle is selected, the control computer 16 begins the new cycle immi~riizt~ly after completion of the then current recording c~Ycle. The precise timing described above permits a change in the recording cycle, as well as a change in the signals provided to the video monitors 54, to occur within a single NTSC blanking interval.
During the data collection process~ which is organized into a series of "runs", the video cameras 50 continually send image signals to the digital matrix router 23 through the camera control units 70. Pursuant to instructions from the control computer 16, the router 23 provides to the Yideo tape recorder 20 the signal from a selected one of the image sources, which include the ten video cameras 50 and the four-view generator 62. The images provided to the video tape recorder 20 are recorded at the rate of thirty per second in a predetermined order comprising a cycle. In addition, audio comments from both the driver and navigator of the moving vehicle 11 are recorded by the video tape recorder 20.
While the video tape recorder 20 is recording video images, the GPS receiver 24 acquires three ~iml~ncinn~l fixed location data from the NASA NAVSTAR Satellite Network at selected time interYals and at selected locations. This fLxed location data is provided to the control computer 16 along with the Greenwich Mean Time (GMT) ~ . d by the satellites. The control computer records the location data from each positional reading provided by the GPS receiver 24, the data being given in terms of latitude, longitude and elevation. The video equipment controller . _ .. . . . ... . ...... . . . . .

WO 95/32483 1~ c 7~?

44 provides the control computer 16 with the SMPTE time code of the frame on which the video tape recorder 20 is recording for each positional reading of the GPS receiver that it receives and records. The SMPTE time code is then recorded by the control computer 16 with the ~:o~ pull.lillg set of GPS location data.
The inertial navigation system 28 comprises a set of rotation and acceleration sensors and continually detertnines the relative position of the vehicle 11 in terms of latitude, longitude, eleYation, pitch, roll and yaw as shown in Figure 5. The INS 28 can thus provides spatial position with six l 0 degrees of freedom relative to previous positional readings taken by the GPS
receiver 24. The INS provides this six-dimPn~;~ n~l relative position data at the rate of thirty times per second and the data is recorded by the control computer 16. Preferably, the INS 28 utilizes high quality laser ~/IU:~,U~
which provide extremely accurate rotational/attitude dala. The INS 28 can also include an odometer coupled to an undriven wheel of the vehicle 11, which reduces the need to stop for re-alignment of the INS 28.
The video ~Ui~ lL controller 44 provides the control computer 16 with the SMPTE time code of the frame on which the video tape recorder 20 is recording for each positional reading of the INS 28 that it receives and records. The SMPTE time code is then recorded by the control computer ]6 with the corresponding set of INS position data. The recording of GPS and INS spatial position data along with the ~:UI I e:~,UOlUIill~
SMPTE time code allows the position data to be interpolated by SMPTE
time code during post-collection processing.
By using the inertial navigation system 28, the collection system 10 of the present invention is able to collect accurate positional readings under many ~il . .", ~ in which the GPS receiver 24 is unable to obtain accurate position data. This often occurs when the GPS receiver cannot obtain enough satellite signals to calculate a position and when the WO95/32483 1 I/lJ r.~

signals from one or more of the sate~lites is reflected off of a building or other surface, If the vehicle must be stopped for an extended period of time, or if a previously recorded street must be driven again to reach a desired 5 location, a pause command is given by the navigator to stop the recording of both video images and positional infnrm~tinn. When it is desired to begin recording data, a resume command is given. The control computer 16 records the SMPTE time code of the last frame on which the video tape recorder 20 has recorded after the issuance of each pause command and the 10 first frame on which recording begins aher each resume command.
The flow of data in the collection system during a data collection run is shown in Figure 6. The collected data includes a video tape 114, GPS and INS spatial position data 115 and a record of each pause and resume command given 116. The video tape 114 contains all of the 15 recorded video images as well as all audio illrullll~iull 118 from the driver and navigator. The video tape 114 is converted to a digital ~u.~ d format and stored in a digital video file for easy random access during post-collection processing. A correlation table is created to maintain the relationship of each digital video frame SMPTE time code with the SMPTE
20 time code on the original video tape. The GPS and INS position data 115 provided from the GPS receiver 24 and the INS 28 is recorded to a digital file. The recorded six-dimensional spatial position data is indexed by SMPTE time code, allowing it to be correlated with each recorded video image or frame. Each set of position data also includes the ~:oll~,~Julldi 25 GMT, which identifies when the infnrm~rion was collected and allows the SMPTE time codes to be indexed by GMT. The record of each pause and resume command is indexed by SMPTE time code.
During each run, the GPS base receiver 42, which is not shown in Figure 6, collects three--limf~ncinn~l position data from the NASA
.. .. . . ..

WO 95/32483 r 21 9~596 NAVSTAR Satellite Network at the same intervals as the GPS receiver 24 in the vehicle 11. Each set of position data is recorded in a base station file along with a corresponding GMT provided by the GPS satellites. The collection of position data by the GPS base receiver 42 must begin prior to 5 the beginning of each run and must be ~-mtin~ lc Lfiluu6l~uu~ each run. In addition, the GPS base receiver 42 must be positioned so that there are no obstructions between the receiver 42 and the GPS satellites.
Figure 7 shows a sequence of steps that are performed during the method of the present invention. As described above, both visual and 10 spatial position data is collected by the data collection system 10, as shownby bûx 120, and spatial position data is collected using the GPS base receiver 42, as shown by box 122. The spatial position data from the GPS receiver 24 and the GPS base receiver 42 is then differentially post-processed as shown by box 124. The differentially processed GPS data and the INS
15 spatial position data is correlated as shown by box 126, errors in the INS
position data are corrected, and the GPS and INS position data is combined as shown by box 128.
Next, the offset between the spatial position of the vehicle 11 and the street centerline is calculated as shown by box 132 using the images 20 on the collected video tape 130. Street centerline points are then plotted on a computer-aided design display system as shown by box 136. Street segment nodes from a TlGER/Line file 134 are updated as shown by box 138 and the updated street segment nodes allow new street segments to be created as shown by hox ~40. Shape points are then calculated for the street 25 segments as shown by box 142 and a video im~gc ~_6~ If l'~ is establisbed as shown by box 144. This provides a complete data file 146 containing accurate, updated street segments having a known 1. l~lin"~l,;l, wjth the recorded video images. Each of these steps is more ~ollll,l.,tf ly described below.

W0 95132483 r ~ ,, r. ~ 7 The United States D~ L~ l of Defense int-~ntit)n~lly introduces errors into the NAVSTAR satellite inform~tion provided by each satellite, which results in position data inaccuracies. This error is removed prior to processing the collected illrul llldtiull by differential processing of the 5 position data acquired by the known position GPS base receiver 42 with that of the vehicle-based GPS receiver 24. The position data recorded by the GPS base receiver 42 is indexed by GMT time code and is correlated with the position data recorded by the GPS receiver 24 through the use of the GMT time codes. Differential processing yields a corrected three-10 flim~nciQnul spatial position of the GPS receiver 24 in terms of latitude,longitude and elevation for each position at which a GPS reading was taken.
Once an absolute position for each GPS reading is determined, the recorded relative position data from the INS 28 can be converted into absolute spatial position data in terms of latitude, longitude and elevation.
15 These absolute positions are then combined with the recorded pitch~ roll and yaw of the vehicle to form a completed data file of accurate six-~lim-n~inm~l spatial positions which can be referenced by date, SMPTE time code, GMT
and run. During this processing, the INS position data is corrected when di~ dl~cics exist between the position data provided by the INS 28 and 20 that provided by the GPS receiver 24. When this occurs, the position data provided by the INS 28 is corrected by prorating the error found in speed and direction at the discrepant point back through the records to the previous GPS positional reading.
After the data collection and post-processing is c- mrlet~rl the 25 information is ready for visual data correlation and extraction. This extraction process is performed entirely after both the data collection process and the data post-processing. By separating the collection, post-processing and extraction processes, a greater amount of data can be captured and a greater aml)unt of information extracted than if they were WO 95132483 1 ~ 5 ~

- performed .~imlllt~nPoucly. The information on which the extraction process is performed includes the video images recorded by the video tape recorder 20, the six-l1imf n~;~)n~ absolule spatial positions of the vehicle 11, and the record of pause and resume comm~n-lc A file containing the absolute spatial positior~s of the vehicle 11 along with the ollc~l~olldillg SMPI`E time codes is loaded into a computer-aided design (CAD) display system which is not shown. The display system plots the X and Y ~;uuldill~lLcs of each spatial position recorded in the file as a point. The spacing between the points varies based on the velocity of the vehicle 11 and the visual trail of plotted points definesthe true path of the vehicle 11. Generally, the vehicle 11 travels to one side of the center of the street. Further, the spatial position data is collected relative to a single point within the vehicle 11. Therefore, the collected spatial position data does not relate directly to the true cenler of the street and the offset of the vehicle 11 from the actual center of the street must be calculated to determine the true center of the street.
To determine the centerline of the street, an operator combines the display of plolted points representing the true spatial position of the vehicle 11 with a display of either the front or rear facing video images. The front or rear facing video image that ~ullc~o~lJ~ to each plotted point can be recalled through a random access procedure through the use of the known SMPTE time code for each point. The operator selects the first plotted point in the segment and displays the Cu~lc~uull iillgvideo image in a digital video computer display window. The two views are calibrated to place the center of the video image directly in line with the lineof travei of the vehicle 11. A line representing an imaginary projection from the video image source is displayed on the video display as a graphical cursor. This cursor can be moved left or right by a control interface such as a computer mouse or trackball. The distance between the cursor and the .... . . . . .. _ . . .. , .. ... .. _ WO 95/32483 r~l~u~ S'C~
21 9~596 graphical image vertical center is measured in terms of the number of pixels.
The distance in pixels is translated to a distance in feet, using known information about the video source mounting and focal length. This distance in feet is the distance that the true vehicle position is offset from the center5 of the street and can be used to calculate the spatial position of the center of the street that corresponds to the vehicle position.
The centerline ~ L. . ",;",.l inn process is shown in more detail in Figure 8. A video image 150 taken from a camera lens 152 of one of the cameras 50 is shown as being suspended above a street 154 on which the ~0 centerline must be determined for purposes of illustration. A known vector D extends perpendicular~y from the camera lens 152 to the surface of the street 154 and is defined in lerms of a camera coordinate system that has its origin at the camera lens 152. Using a pointing device, a pixel 156 on the centerline in the video image 150 is chosen by an operator and a vector U
lS is defined in coordinates from the camera coordinate system as extending from the camera lens 152 to the pixel 156. A multiplier L is then found for the vector U such that L'U extends to a point 158 in the plane of the surface of the street 1~4. The multiplier L is determined using the following equation:
- 2 ~-T,_ IIDI
UD
Once the multiplier L is found, the vector L~U is determined.
25 The vector L~U, which is defined in camera coordinates, is then converted to a global spatial position. This is done through a rotation of L"U to account for the rotational di~elc~ s of roll, pitch and yaw between the camera coordinate system and the global spatial position and a translation of LIU to account for the offset differences of latitude, longitude and W0 9513A83 r~ ~ ?
2 1 ~0596 elevation between the camera coordinate system and the global spatial position. The converted vector is then used to determine the spatia~ position of the point 158 on the centerline.
When there is no marked centerline in the street, a pointing 5 device with two additiona~ markers can be used. The additional markers are equally spaced on either side of the centerline marker as equal distances forward of the vehicle 11. The width of the space between the two markers can be controlled to allow an operator to track the edges of the street. A
computer ~ om~ti~lly determines the point on the video image 150 of the 10 street to which the vector U from the camera lens 152 should extend.
When performed for each plotted point in a street segment, the centerline d~e~ lio~- process generates a set of latitude, longitude and elevation ~ool ;iil,~Les It~ s~llLil-g the true street centerline. Once thisprocess has been repeated fot each point, a street centerline for the segment 15 can be displayed on the CAD system. This process further takes into account changes in the location of the vehicle 11 relative to the street centerline as the vehicle 11 travels down the street, including those caused by traffic and obstructions. This centerline det~l",i,l~iol~ or offset calculaîion process is performed until a street centerline location has been 20 calculated for each collected spatial position of the vehicle 11.
A street segment is a set of ordered spatial positions, each of which is connected to an adjacent position, l~llts~llLillg an ~lwdllldLiu-l of a street centerline. Generally, street segments are deflned in terms of two such spatial positions, known as a begin node and an end node, which 25 represent the ends of the segment. To help further define curved street segments, a set of intermediate positions known as shape points are usually chosen between the begin and end nodes. Street segments that are straight generally do not re4uire shape points. Street segments are initially defined from U.S. Census Bureau two dilll~ iOIl~l TlGER/Line files and each , W0 95/32483 F~ J~ C'~-?~?

segment is uniquely identified based on a TlGER/Line file i~ ntifi~tit)n code called a TIGER/Line Identifier (TLID).
Figures 9 and 10 illustrate the process of updating a street segment. Referring to Figure 9, a plurality of plotted points 190 is shown 5 .~.,es~l.Li.l~ a set of spatial positions collected on a single street segment.
A plurality of centerline position points 194, each of which is calculated from one of the plotted points 190 using the centerline dc~ iull process described above, comprise a centerline 198. A TlGER/Line street segment 204 having a begin node 208 arld an end node 210 is also shown in Figure 9, along with an updated street segment 212 having a new begin node 216, a new end node 220 and a plurality of shape points 224.
After the completion of the centerline d~ Lion process, the street segment d~ U~illldLillg the calculated centerline position points 194 is retrieved from a U.S. Census Bureau TlGER/Line file and is displayed on the CAD on top of the calculated street centerline 198.
Because the TlGER/Line street segments are generally not very accurate ~lu~illla~ions of newly calculated street centerlines, the TlGER/Line street segment 204 must be updated. This is done by relocating the begin node 208 and the end node 210 of the TlGER/Line street segment 204 and defining the shape points 224 to be used in the updated street segment 212.
Figure 10 shows the relocation of the TlGER/Line segment begin node 20~. Two points 228 and 232 from the centerline 198 define a first vector which is m~th~m~tically projected toward a second vector defined by points 236 and 240 from an intersecting street centerline 244. The m~th~m~ti~l intersection of these two vectors is determined and defines the latitude and longitude of the new begin node 216 for the updated street segment 212. An elevation value of the new begin node 216 is i--~ t~,d from a vertical intersection of the two vectors. The end node 210 of the TlGER/Line street segment 204 is relocated in the same manner as the _ _ _ _ _ .. .. . . .. . _ .. .. ... .. . .. . . . ... .... ..... . . . .

WO 95/32483 P~ .'C -~?
2 1 90~95 begin node 208, thus defining the new end node 220 for the updated street segment 212 in terms of latitude, longitude and elevation. After the begin and end nodes 208 and 210 of the TlGER/Line segment 204 have been updated, the node location attributes in the TlGER/Line file are updated.
5 Because each node acts as both Ihe begin node and end node for several street segments, a number of segments are updated with the relocation of each node.
Referring to Figure 9, once the begin and end nodes 208 and 210 of the TlGER/Line street segment 204 have been relocated, the centerline position points 194 between the new begin node 216 and the new end node 220 are statistically evaluated based on a deflection and distance formula to select the shape points 224 that will be used to graphically represent the updated street segment 212. The use of a few shape points æ4 rather than the large number of centerline position points 194 facilitates 15 the efficient storage of the graphical database. A graphical display of the redefined street segment 212 having five shape points 224 is shown in Figure 9.
After the segment updating process has been completed. the segment database record is updated with reference to the new shape points 20 224 by means of the standard TLID. This provides backward compatibility with all other TlGER/Line information. This process is repeated for all segments that were driven in the run. In addition to updating the segment location information, the run identifier and SMPTE time code for the new begin and end nodes 216 and æo of the updated segment 212 is added to 25 the segment database. This additional information provides the reference necessary to provide access to the digital video file. Through direct random access to the video file, a new digital video file is created for each view captured in a street segment. The run identifier and beginning SMPTE time Wogsl32483 r~,.J c code provide a way to uniquely identify each group of recorded video images for each street segment.
Once the physical street segment topology is defined, the video image sequences are associated with each street segment. Each video S sequence is defined by a vehicle run ifl~ntifi~ti~n number as well as a beginning and ending video frame SMPTE time code. By identifying each video sequence in this manner, a unique set of video images can be recalled for each segment. A video source sequence number is also stored to further define individual video sources within each interleaved video segment. The 10 video source sequence number is the position in the video recording cycle of the beginning video image for the segment. This information, along with the video recording sequence noted in the run database, allows the video source to be determined for any SMPTE time code within the segment video range.
A video image to segment matching process is used to match each street segment wilh at least one video image. Video image position information is used to find the video images which are geographically closest to both the begin node and the end node of a segment. First, a street segment having a known identifier is selected by an operator. The operator 20 then selects a cenlerline position point within the segment. Using the SMPTE time code of the selected centerline position point. and working s~ nti~lly in both directions. each centerline position point at which a video image was recorded is compared to the begin node and the end node until the closest geographical match is found for each node. The SMPrE
25 time codes for these two video frames are recorded as part of the video-segment rel~tion~hir The lower of the two recorded time codes is compared to thepause and resume recor(ls from the original vehicle run to determine the subset of video frames in which the segment was recorded. From the start wo 95/32483 P~

of this subset, Ihe known recording cycle is used to calculate the video source sequence number for the segment. If the lower of the two recorded time codes is associated with the begin node, then the video images associated with the segment were taken in the direction of the street 5 segment. If the lower of the two recorded time codes is associated with the end node, then the video images associated with the segment were taken in a direction opposite to the direction of the street segment. The video-segment direction rPl~tir)nchip is recorded as part of the video-segment relationship.
After the video-segment rel~ti~nchir) is formed, the video sequences and street segments can be used for a variety of applications. In one such application. random digital video access techniques are used to reassemble the video sequence for a segment into a plurality of smaller video image sets containing sequential video images from a single video 15 source. Each such image set is identified by run. video source and beginning SMPTE time code. A cross reference table is ~,OII~Ll u~..d to reference each image set with its associated street segment identifier. By reducing the large set of recorded digital video images into smaller sequences that are associated with street segments, maximum flexibilitv in data storage is 20 provided. Through this street segment database. Iarge amounts of digital vide() image sets can he manipulated and using current indexing methodology, up to 10l~ street segments can be managed.
The integration of video images with a geographic information syslem (GIS) requires multiple access methods for effective use in a GIS
~5 environment. To meet this requirement, both a set of segment-video and video-segment rPl~tir~nchirc is established and used to form a dalabase. In addition, a number of secondar rPl:-tionchirs are çst~hlichPd from these video-segment and segment-video rPl~tionchirs to permit a wide variey of GIS applications.

WO 95/32483 r~ v. .r -Figure 11 helps illust}ate the creation of a video-segment rr l:~tionchir and shows a street segment 250 having a begin node 252 and an end node 254. A first intermediate shape point 256 and a second intermediate shape point 258 are shown and are used to help define the segment 250 and are directly re~ated to the begin and end nodes 252 and 254. A series of video images are shown as points Vl-V7 and each has a known position defined in terms of latitude, longitude, elevation, roll, pitch and yaw. Video image Vl is direct]y related to the end node 254 and video image V7 is directly related to the hegin node 252 through the matching ]() process descr~be~i ahove. The remaining video images are indirectly relatedto the begin node 252, the end node 254 and the shape points 256 and 258.
The indirect rel~tinnchir between the remaining video images and the shape points 256 and 258 is shown in the following example. The video image V3 can be defined in terms of the segment 250 as the ]5 perpendicular inlerseClion belween Ihe end node 254 and the second shape point ~S~. This point of intersection can be described in absolute latitude, longilude and elevation or it may be expressed relative to the end node 254 as Ihe negalive dislance from Ihe poinl of inlerseclion lo the end node 254.
The relalive position of video image V3 may also be expressed as the posilive dislance from the begin node 252 to the point of illt._l~C~,liOII whichis equal IO the distance from the begin node 'S~ to Ihe firsl shape point 256 plus Ihe distance from Ihe firsl shape point 756 to the second shape point 25~ plus Ihe distance from the second shape point 258 to Ihe poinl of inlersecliOn.
:Because Ihe rPl~tinnchir between the video points V2-V6 and the begin node 252, end node 254 and shape poinls 256 and 258 is an in/l~p~ndent one. a new relalionship can easily be ct~hlich~ d if c~lhs~r~ nt video and posilional data were co~lecled for Ihe same segmenl. The new reiationship may be est~hlishedv belween any new video points and the _ W095/32483 1~ J",JI,-7~7 existing shape points 256 and 258 by establishing a new video to begin node 252 and video to end node 254 r~l~ti~lnchir and r~lr~ tinr~ the point of intPrSerti-)n as described above. In other words, once the street segment, including the begin node, end node and shape poinls, has been established, 5 it does not need to be updated and sllhc~qu~nt video images can be related to the segment in the same manner as were video images Vl-V7.
By creating a direct r~lAti~nchip between each node and the c~osest video image, a segment-video r~lz ti~n~hip is also created. As a result of this relationship, an ordered set of video images are associated with anv ~0 given segment. This ailows easy access to a set of video images and related attribute information for any given segment. Therefore, a segment position point described as a position relative to a begin node or an end node can be expressed as an absolute position in terms of video image positions. It is therefore possible to determine the closest video image to any given segment 15 position, providing a user with the abilit~v to examine a visual ~ cs~ iol~
as well as related attribute il,r.,.,lldLi-)ll of a given position. As with the video-segment rPlAtionchir the video image positions are in~p~n~irnt from the segment nodes and intermediate shape points. Therefore, the segment positions can be corre~aled in the same manner with video and positional 20 information collected during ~llhs~g~l~nt runs.
A number of applications in which the updated street segment and video-segment relationship data i4~ can be used are shown in Figure IZ.
The applications include the creation and update of address ranges 282 and the integration of address attribute information 284, the creation and 25 ~ nce of street networl; topologies 286, the collection of vehicle routing information 288, the creation and m~int~nAnre of boundary polygon topologies and attributes ~90 and 292. the accurate location of point features and their attributes 294 and _96, and the integration of street segment wo g~/32483 21 9059~ ~
-~8 -attributes wjth other GIS systems 298. These applications are discussed in more detail below.
Attributes shown in a video image have an extended indirect r~l~ticmchip with segment positions. For example, a street address has an 5 indirecl address-segment r~ rionchir. By taking this r~ ionchir in combination with the indirect segment-video rt l~tionchir~ an address-video r~l~tionchir can also be ~ct~hiichfrl With such a rel~tinnchir, an address seen in a video image can be compared to a calculated street address If the two addresses are not the same, the visually observed address can be used 10 to correct the model hy which the address was calculated.
Another example of the use of an extended r~ tinnchir is in vehicle routing where the efficient collection of vehicle routing il~r~ c.Lioll is greatly enhanced by the video-segment data if l,, ~ "~ s Vehicle routing information is divided into the two categories of node attribute i,lr~,l.,ld,iul., 15 which defines the r~l~tionchir between two segments through a node, and s~gment attribute information, which defines the characteristics of Ihe segments. Bv coupliny these two sets of data with GIS based routing soflware~ ~ccurale vehicle naviyation or routing can be accomplished.
Node attrihute information is collected through a visual ~0 examination of the recorded video images for each intersection. Each passage through an intPrS~ction is stored as a movement from a specified segment, through a specified node, to another specified segment. Using segment-video data relationships, an operator recalls the video images from any location around a node. The recalled video images are then visually inspected to delermine the existence of a segment to segment connection as well as any traffic restrictions or controls. An indexed node attribute database is created to store all of this information for later reference.
In the collection of segment attribute illrulllldLioll~ the video images for each segment are visual~y reviewed and evaluated for speed WO 9!i/32483 . ~ 7~
~ 1 q35q6 - limits, street type rl~ccifi~tion, one way status, weight or height r~ctricti~
and othe} traffic controls. Each of these attributes is stored in a segment attribute database as a begin/end range with a left or right orientation. For example, a short no parking zone could be described as an attribute from a 5 relative location X on the segment to the end of the segment and on the appropriate side of the street.
Extended r~l~tinnchirs can also be used to determine relative point positions which are points that are referenced to an attribute which is related to ~ segment. To determine a relative point position. a set of ]0 attribules must alread~ exist and be linked to a segment b~ an attribute-segment relationship. The atlribute to which the point is referenced can be visually verified by viewing the closest video image. Another type of point position is direct video position, which is the direct ~ccn~ tinn of attribute data with the video frame identifier of the video image on which the 1~ attribute is found.
The ahsolute position of a point can be found through the use of terrestrial photogrammetr~ n photo~-d~ Lly, multiple images containing a single chosen reference point are used to triangulate the position of the point relative to the cameras taking the images. This relative ~0 position is then used in combination with the known position of the cameras when the image was recorded to determine the absolute position of the chosen point. Once the ahsolute position of the point is determined, related attribute information can he visuall!~ determined and recorded. To help ensure the accuracy of the photogrammetric process, camera characteristics 25 such as lens calibration information as well as specific camera mounting and orientation illrOIll-~llioll must be taken into account.
The present invention allows the photogrammetric process to be performed using anv two images containing the chosen reference point.
This is possihle because the six-dimensional spatial position of each camera .. .... .... .. . .. _ . . _ ... , ....... _ .. _ . . .

WO 95/324X3 1[ ~ '7~7 is known and recorded for each video image laken. Therefore, the distance between where ~he two images were taken is known along with the flri~nt~tinn~ of the cameras from which the images were taken. The present invention thus allows the position of a chosen reference point to be S determined through the use of images taken from cameras that are not in the same plane as well as through the use of multiple images from one camera. The performance of the photogrammetry process using images from a variety of cameras is aided through the use of interleaved image storage on the video tape. Further, the performance of the phuLo~ c~y process 10 using images~from a variety of cameras allows positiorls to be calculated after the data collection process is ~lmrl~t~l The ability to perform the photu~ ..,LIiC process using images from an~ two cameras is a significant ;~ IU~ IL over the traditional stereo photogrammetric approach. Such a traditional approach 15 must ~omr~n~t~ for a relatively short baseline between two cameras by using high resolution digital cameras, which requires ~ oldilldlily large amounts of data to be stored. Further, when computing the position of a point or an ohject which i~ a great distance from the cameras, as must often he done in the stereo photogrammetric approach. the potential for error in 70 calculating the position of the ohject is greatl~ increased. The present system, hy allowing the use of any two images in which an object is found, allows the position of the object to be computed using images in which the object is relatively close to the camera.
With the present invention, an accurate address database can 1S he created and an existing databa~e such as a TlGER/Line file can be updated. In addition. a dataha~e can he created using i"rul",atio" collected and processed by the method of the present invention which allows the retrieval of video images associated with a street address.

w0 95/32483 2 1 9 0 S q 6 ~ 'T

All segments created through the process described above are referenced by a TlGER/Line irl~ntifir~tion number. This ~ g method provides a unique identifier for al~ street segments in the United States. Further~ new streets can be added to the database in compliance with the TlGER/Line identification syslem. Thus street segments created through the update process described above can be directly related with a TLlD. This rel~tion~hir allows references to be made to all the data elements contained in the TlGER/Line file~ allowing for total cnmr~tihility of the database provided by the present invention with this national I 0 standard.
Each TLID contains an overall address range for the street segment il represents which is known as a nominal address range. In addition each TLID supports both a left address range and a right address range as well as multiple sub-ranges. Each TLID is l~rLl~l.ccd to the 15 coordinates of a node at each end of the street segment it represents. The coordinates of the node are used as a beginning point for the address range within the street segment. Therefore the beginning address in each nominal address ran-ge is a.ss~me~ to he located in the middle of a street intersection.For any specific street tddress. only ~ne p~ssihle position may exisl in the 20 dalabase.
Address parsing is possible using the database provided by the present invention. Address parsing converts a user supplied address to a standard address within the database by matching it to a real address range in the database. This address range is referenced to a segment TLID. a ~5 relative TLID ~ffsel fr~m one of the seyment nodes and a side of the street.
The side of the street on which an address is located is known as the parity of the address.
~ ddress ranges are stored internally as an offset from the beginning of each street segmer~t. For instance the beginning segment node wo 9s/32483 ~ 7~7 is considered to have a position offset of zero while the ending segment node is referred to as having an offset of one. The coordinate position of an address is based on the offset which is determined by taking the total segment length as defined by the shape points and prorating the distance 5 along the segment. For each address~ there wil] therefore be a pair of coordinates and a parity for the true address position.
A video image can be recalled for any coordinate pair through the use of a coordinate-video image matching system. The video information is independent from the spatial position information and video 10 frame calculations rely on the segment-video r~l~ti-~n~hir discussed above.
This process allows the retrieval of the nearest video image to a coordinate pair.
An address range can be construcled after the street segment update process. The nominal address ranges are acquired from the 1~ TlGER/Line file for each street segment. The video for a given segment is displayed and a determination is made whether to insert an additional address range. Address matching is a..v~ lisll.,d through the use of the information found in the TlGER/Line files prior to the address range construction. This information includes the identifiers for both the begin 20 node and the end node ~f each segment, which contain Ihe actual coordinates of the end points of the segment. Any shape points that are connected tt) Ihe hegin ~nd end nodes to define the segment would also be accessed. Based on the node and shape points, the length of the segment is calculaled. The information found in the TIGER/Line files also includes 2~ the beginning and ending address ranges that define the nominal address range for each segment.
The address range construction process begins by viewing the video for a segment As an address is observed, the video frame on which it is observed is converted to image coordinates. The actual offset from the _ _ .. .. ....... ....

WO 95/32483 1 ~

, -33 -begin node is then determined through the video-segment r~l~ti- n~hir. The ,. operator indicates whether the address is an exception to the ~ td range value. If the address is not an exception, the parity of the address is ~lltonn~tit ~lly recorded based on the camera view being observed. The raw S information is then tabulated and includes for each address: the offset from the begin node, whether the address is an ~ntirir~t~d range value or an exception~ the predicted parit,v, and the actual parity.
A nominal range within a segment may be split b,v inserting additional range or exception points. This most frequently occurs when ne~
10 streets are added. If the new street is perpendicular to an existing street, then the existing street is split and two new TLlDs are created for the split street using the TLID offset generated by the new node created by intersection with Ihe ne~i street. The operator will then set new nominal address ranges for the two new street segments and the old segment address 1~ ranges will be reassi~ned to the proper new segment TLID. A TLID for the new street will he created and address ranges will be processed similar to the initial TLID processing.
While the collection system 10 was described as having a single GPS receiver 24. the presenl invention ma,v he operated with two or more ~U GPS recei~ers or with ~ GPS receiver having multiple antennas. Such a configuration might he desirahle to provide a more frequent update of the vehicle position if a lower quality g,vroscope is used in the lNS 28. The update could then be used to more frequently correct any errors in the readings provided by the INS 28. Further, the collection system 10 can 25 utilize a GPS receiver that is capable of providing its spatial position withsix degree~ of freedom. Spatial position information can also be provided by other devices such as odometers and pressure gauges and can be correlated with the spatial position data provided b,v the GPS receiver 24 and the INS 28.

wo 95/32483 1'~
.
2 1 ~0596 The position of each camera was described as being known through the use of the recorded spatia] positions of the vehicle 11 However, it is also possible to determine the position of a camera by using any three images of the same object captured at GPS ~,ouldilldles not in a 5 straight line. The information provided by three such images would be sufficient to determine the six-(limPnci(ln~l position of the camera, thus allowing photogrammetric analysis to be performed.
While the collection system 10 was described as having only one video tape recorder ~0, the system 10 can be configured to operale two 10 or more video tape recorders. Further, the collection system 10 can be located on or in any moving platform, including a platform capable of being carried by a single person.
Although the present invention has been described with reference to preferred embodiments, workers skilled in the art will recognize 1~ that changes mav be made in form and detail without departing from the spirit and scope of Ihe invention.

Claims (16)

I CLAIM:

1. A method for collection of visual and spatial position information from a moving platform, the method comprising:
obtaining spatial position data relating to spatial position and orientation of the moving platform;
obtaining video images from a plurality of video cameras carried by the moving platform at known positions and orientations with respect to the moving platform;
recording non-coplanar video images obtained at different times from the video cameras;
associating with each recorded video image an instantaneous spatial position and orientation of the video camera which produced the video image at the time that the video image was obtained, based upon the spatial position and orientation of the moving platform and the known position and orientation of the video camera with respect to the moving platform; and determining location of an object shown in at least two of the recorded neon-coplanar video images obtained at different times based upon a location of the object in each of those recorded images and the associated spatial positions and orientations of the video cameras which produced those video images.

2. The method of claim 1 wherein the video images are recorded in a sequential cycle.

3. The method of claim 1 wherein the spatial position data includes data obtained from a receiver located on the moving platform and capable of receiving data from Global Positioning System satellites.

4. The method of claim 3 wherein the spatial position data further includes relative spatial position data obtained from an inertial navigation system located on the moving platform.

5. The method of claim 1 wherein operation of the video cameras is synchronized.

6. The method of claim 1 and further comprising:
associating with each recorded video image the time at which that video image was obtained.

7. The method of claim 1 wherein the plurality of video cameras are positioned and oriented to capture video images in front, behind, and on each side of the moving platform.

8. The method of claim 1 and further comprising:
replacing selected data in a geographic information system database, based upon the recorded video images and associated spatial position and orientation of the video cameras which produced the video images.

9. The method of claim 8 wherein the database includes data relating to spatial positions of a plurality of street segments.

10. The method of claim 9 wherein replacing selected data includes replacing data relating to a beginning and an end of a selected street segment.

11. The method of claim 1 wherein the moving platform is moved along streets, and wherein the method further comprises:
matching street segments to at least one video image based upon the spatial position and orientation of the video camera associated with that video image.

12. An apparatus for collection of visual and spatial position information, the apparatus comprising:
a moving platform;

means located on the moving platform for obtaining spatial position data relating to spatial position and orientation of the moving platform;
a plurality of video cameras located on the moving platform, the video cameras being positioned at known locations and orientations with respect to the moving platform and being oriented to capture images of fields of view in front, behind, on a left side and on a right side of the moving platform;
means for recording non-coplanar video images obtained at different times from the video cameras;
means for associating each recorded video image with an instantaneous spatial position and orientation of the video camera which produced the recorded video image at the time that the video image was obtained, based upon the spatial position data relating to the spatial position and orientation of the moving platform and the known position and orientation of the video cameras with respect to the moving platform; and means for deriving spatial position information relating to an object appearing in at least two of the recorded images obtained at different times from the video cameras from (a) a location of the object in each of the recorded images, and (b) the instantaneous spatial position and orientation of the video camera which produced each recorded video image at the time that video image was obtained.

13. The apparatus of 12 wherein the means for obtaining spatial position data includes a receiver capable of receiving data from Global Positioning System satellites.

14. The apparatus of 13 wherein the means for obtaining spatial position data further includes an inertial navigation system.

15. The apparatus of 14 and further comprising:
synchronizing means for synchronizing operation of the video cameras.

16. A method for collection of visual and spatial position information from a moving platform, the method comprising:
obtaining spatial position data relating to spatial position and orientation of the moving platform;
obtaining video images from a first video camera carried by the moving platform at a known position and orientation with respect to the moving platform to capture images of a first field of view in front of the moving platform;
obtaining video images from a second video camera carried by the moving platform at a second known position and orientation with respect to the moving platform to capture images of a second field of view behind the moving platform;
obtaining video images from a third video camera carried by the moving platform at a known position and orientation with respect to the moving platform to capture images of a third field of view on a left side of the moving platform;
obtaining video images from a fourth video camera carried by the moving platform at a known position and orientation with respect to the moving platform to capture images of a fourth field of view on a right side of the moving platform;
recording a sequence of non-coplanar video images obtained at different times from the first, second, third and fourth video cameras;
associating with each recorded video image an instantaneous spatial position and orientation of the video camera which produced that video image at the time that the video image was obtained, based upon the spatial position and orientation of the moving platform and the known position and orientation of that video camera with respect to the moving platform; and determining location of an object shown in at least two of the recorded non-coplanar video images obtained from different ones of the first, second, third, and fourth video cameras.