US20080033606A1 - Driving-operation assist and recording medium - Google Patents

Driving-operation assist and recording medium Download PDF

Info

Publication number
US20080033606A1
US20080033606A1 US11/846,027 US84602707A US2008033606A1 US 20080033606 A1 US20080033606 A1 US 20080033606A1 US 84602707 A US84602707 A US 84602707A US 2008033606 A1 US2008033606 A1 US 2008033606A1
Authority
US
United States
Prior art keywords
vehicle
assumed
image
movement
driving
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/846,027
Inventor
Shusaku Okamoto
Hirofumi Ishii
Masamichi Nakagawa
Kunio Nobori
Atsushi Morimura
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Holdings Corp
Original Assignee
Matsushita Electric Industrial Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Matsushita Electric Industrial Co Ltd filed Critical Matsushita Electric Industrial Co Ltd
Priority to US11/846,027 priority Critical patent/US20080033606A1/en
Publication of US20080033606A1 publication Critical patent/US20080033606A1/en
Priority to US13/848,800 priority patent/US9272731B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B62LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62DMOTOR VEHICLES; TRAILERS
    • B62D15/00Steering not otherwise provided for
    • B62D15/02Steering position indicators ; Steering position determination; Steering aids
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B62LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62DMOTOR VEHICLES; TRAILERS
    • B62D15/00Steering not otherwise provided for
    • B62D15/02Steering position indicators ; Steering position determination; Steering aids
    • B62D15/027Parking aids, e.g. instruction means
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B62LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62DMOTOR VEHICLES; TRAILERS
    • B62D15/00Steering not otherwise provided for
    • B62D15/02Steering position indicators ; Steering position determination; Steering aids
    • B62D15/027Parking aids, e.g. instruction means
    • B62D15/028Guided parking by providing commands to the driver, e.g. acoustically or optically
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05Geographic models
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/102Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using 360 degree surveillance camera system
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/302Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/303Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/304Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using merged images, e.g. merging camera image with stored images
    • B60R2300/305Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using merged images, e.g. merging camera image with stored images merging camera image with lines or icons
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/307Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing virtually distinguishing relevant parts of a scene from the background of the scene
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/60Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
    • B60R2300/607Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective from a bird's eye viewpoint
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/806Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for aiding parking
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8086Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for vehicle path indication

Definitions

  • the present invention relates to a driving-operation assist for assisting driving operations of a vehicle and a recording medium for storing a program for making a computer execute all or some of functions of each means of the driving-operation assist.
  • a conventional general driving-operation assist estimates a moving trace of a vehicle corresponding to a steering angle of a steering wheel for backward movement of the vehicle by a steering sensor for detecting the steering angle of the steering wheel.
  • a vehicle moves backward, an image of a rear or side-rear field of view photographed by a camera is displayed and moreover, when the steering wheel is operated, a moving trace of the vehicle estimated correspondingly to the steering angle of the steering wheel is superimposed on the image of the rear or side-rear field of view.
  • driving operations by a driver will be performed as described below. That is, a driver moves a vehicle to a place where the vehicle can be probably parked while fixing the steering wheel of the vehicle.
  • the driver finds a steering angle capable of moving the vehicle to a space for parking the vehicle without operating any steering wheel while confirming a vehicle-moving trace estimated by operating the steering wheel. Then, by moving the vehicle backward toward a parking space while keeping the steering angle, parking is theoretically completed.
  • a driver cannot intuitively find a place where a vehicle can be easily moved to a parking space at a glance while confirming an image obtained by synthesizing the parking space, a state around the space, and a route for guiding the vehicle to the parking space.
  • the present invention is made to solve the above problems of an conventional driving-operation assist and its object is to provide a driving-operation assist by which a driver can directly confirm the relation between movements of a vehicle according to a predetermined series of driving operations and the circumferential state by displaying movements of the vehicle when the driver performs the predetermined series of driving operations together with the circumferential state and the load of the driver can be reduced.
  • circumferential-state imaging means for imaging a circumferential state of a vehicle with a camera and generating a circumferential-state image and/or storing the generated circumferential-state image
  • synthetic-image generating means for generating a synthetic image by superimposing on the circumferential-state image, an assumed-movement pattern which is the video data showing movement of the vehicle in case of performing a predetermined series of driving operations for the vehicle;
  • Another aspect of the present invention is a recording medium characterized by storing a program for making a computer execute all or some of functions of each means of the above driving-operation assist.
  • FIG. 1 is a block diagram showing a configuration of a driving-operation assist of a first embodiment of the present invention
  • FIG. 2 shows a top view and an elevation view of a vehicle provided with cameras of an imaging section 101 of a driving-operation assist of the first embodiment of the present invention
  • FIG. 3 is an elevation view showing a viewpoint of a virtual camera of the driving-operation assist of the first embodiment of the present invention
  • FIG. 4 is an illustration showing a circumferential-state image viewed from the virtual camera of the driving-operation assist of the first embodiment of the present invention
  • FIGS. 5 ( a ) and 5 ( b ) are illustrations showing assumed-movement data of the driving-operation assist of the first embodiment of the present invention
  • FIGS. 6 ( a ) and 6 ( b ) are illustrations showing assumed-movement data of the driving-operation assist of the first embodiment of the present invention
  • FIG. 7 is an illustration showing a synthetic image of the driving-operation assist of the first embodiment of the present invention.
  • FIG. 8 is an illustration showing a synthetic image of the driving-operation assist of the first embodiment of the present invention.
  • FIG. 9 is an illustration showing movement of a vehicle at the time of performing parallel parking to the left side
  • FIGS. 10 ( a ) to 10 ( c ) are illustrations showing synthetic images of the driving-operation assist of the first embodiment of the present invention.
  • FIG. 11 shows illustrations of variation of assumed-movement patterns stored in assumed-movement-pattern storing means 108 of the driving-operation assist of the first embodiment of the present invention
  • FIG. 12 is an illustration showing a modification of the synthetic image of the driving-operation assist of the first embodiment of the present invention.
  • FIG. 13 is a block diagram showing a configuration of a driving-operation assist of a second embodiment of the present invention.
  • FIGS. 14 ( a ) to 14 ( c ) are illustrations showing synthetic images of the driving-operation assist of the second embodiment of the present invention.
  • FIG. 15 is a block diagram showing a configuration of a driving-operation assist of a third embodiment of the present invention.
  • FIGS. 16 ( a ) to 16 ( c ) are illustrations showing synthetic images of the driving-operation assist of the third embodiment of the present invention.
  • FIG. 17 is a block diagram showing a configuration of a driving-operation assist of a fourth embodiment of the present invention.
  • FIG. 18 is a block diagram showing a configuration of a driving-operation assist of a fifth embodiment of the present invention.
  • FIGS. 19 ( a ) to 19 ( c ) are illustrations showing synthetic images of the driving-operation assist of the fifth embodiment of the present invention.
  • FIG. 20 is a block diagram showing a configuration of a driving-operation assist of a sixth embodiment of the present invention.
  • FIGS. 21 ( a ) to 21 ( c ) are illustrations showing synthetic images of the driving-operation assist of the sixth embodiment of the present invention.
  • FIG. 22 is a block diagram of a configuration of a modification of the driving-operation assist of the sixth embodiment of the present invention.
  • FIG. 23 is a block diagram showing a configuration of a driving-operation assist of a seventh embodiment of the present invention.
  • FIG. 24 is a conceptual view showing a mapping table stored in a mapping table 2302 of the driving-operation assist of the seventh embodiment of the present invention.
  • FIG. 25 is a block diagram showing a configuration of a driving-operation assist of an eighth embodiment of the present invention.
  • FIGS. 26 ( a ) to 26 ( c ) are illustrations showing synthetic images of the driving-operation assist of the eighth embodiment of the present invention.
  • FIG. 27 is a block diagram showing a configuration of a driving-operation assist of a ninth embodiment of the present invention.
  • FIGS. 28 ( a ) to 28 ( c ) are illustrations showing synthetic images of the driving-operation assist of the ninth embodiment of the present invention.
  • FIG. 29 is a block diagram showing a configuration of a driving-operation assist of a tenth embodiment of the present invention.
  • FIGS. 30 ( a ) and 30 ( b ) are illustrations showing synthetic images of the driving-operation assist of the tenth embodiment of the present invention.
  • FIG. 31 is a block diagram showing a configuration of a driving-operation assist of an eleventh embodiment of the present invention.
  • FIGS. 32 ( a ) and 32 ( b ) are illustrations showing synthetic images of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 33 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 34 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 35 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 36 is a graph for explaining a contact-hazard evaluation function of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 37 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 38 is an illustration [sic; expression] showing a contact-hazard evaluation function of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 39 shows examples of assumed movement data of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 40 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 41 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention.
  • FIG. 42 is a block diagram showing a configuration of a driving-operation assist of a twelfth embodiment of the present invention.
  • FIG. 43 is an illustration showing a synthetic image of the driving-operation assist of the twelfth embodiment of the present invention.
  • FIG. 44 is a block diagram showing a configuration of the driving-operation assist of the twelfth embodiment of the present invention.
  • FIG. 45 is an illustration showing a synthetic image of the driving-operation assist of the twelfth embodiment of the present invention.
  • FIG. 46 is an illustration showing a synthetic image of the driving-operation assist of the twelfth embodiment of the present invention.
  • FIG. 1 is a block diagram showing a configuration of the driving-operation-assist of this embodiment.
  • the driving-operation assist of this embodiment is mainly used to assist driving operations for garaging or parallel parking and the like.
  • the driving-operation assist of this embodiment is configured by an imaging section 101 comprising N cameras (camera 1 to camera N), a camera parameter table 103 for storing camera parameters which are characteristics of the cameras, space configuring means 104 for generating space data relating each pixel configuring an image output from each camera to a point in a three-dimensional space in accordance with a camera parameter, viewpoint converting means 106 for generating an image viewed from a predetermined viewpoint as a circumferential-state image by referring to the space data, a space-data buffer 105 for temporarily storing the space data, assumed-movement-pattern storing means 108 for storing assumed-movement data including assumed-movement patterns, superimposing means 102 for superimposing the assumed-movement pattern on the circumferential-state image and generating a synthetic image, and displaying means 107 for displaying a synthetic image.
  • an imaging section 101 comprising N cameras (camera 1 to camera N), a camera parameter table 103 for storing camera parameters which are characteristics of the cameras, space configuring means
  • An assembly including the imaging section 101 , camera parameter table 103 , space reconfiguring means 104 , and viewpoint converting means 106 corresponds to circumferential-state imaging means of the present invention and the superimposing means 102 corresponds to synthetic-image generating means of the present invention.
  • FIG. 2 shows a top view and an elevation view of a vehicle provided with cameras of the imaging section 101 .
  • the six cameras 201 to 206 are arranged so that a part of each camera overlaps with a part of the imaging range of other camera and no dead angle is produced flatly.
  • the camera parameter table 103 stores camera parameters of each of the above cameras (parameters showing camera characteristics such as setting position, setting angle, lens-strain correction value, and focal distance of camera).
  • the space reconfiguring means 104 generates space data relating each pixel configuring an image output from each camera in accordance with the camera parameters to a point in a three-dimensional space on the basis of a vehicle.
  • the space-data buffer 105 temporarily stores the above space data and the viewpoint converting means 106 generates an image viewed from an optional viewpoint such as a viewpoint of a virtual camera 301 shown in FIG. 3 as a circumferential-state image by referring to space data and thereby synthesizing pixels.
  • FIG. 4 shows a circumferential-state image viewed from the viewpoint of the virtual camera 301 shown in FIG. 3 .
  • This example shows a case of performing parallel parking, in which two vehicles currently parked are shown on the circumferential-state image as an obstacle 401 and an obstacle 402 .
  • the assumed-movement-pattern storing means 108 stores an assumed-movement pattern which is video data showing movement of a vehicle to which a typical vehicle driving operation is applied and time-series data showing the relation between vehicle moving distance (moving distance due to rotation of tire) and steering angle of steering wheel (rotation angle of steering wheel) as assumed-movement data of the present invention.
  • FIGS. 5 ( a ) and 6 ( a ) respectively show an assumed-movement pattern which is video data corresponding to a driving operation according to the time-series data shown in FIGS.
  • a driver selects one of the assumed-movement patterns stored in the assumed-movement-pattern storing means 108 by pattern selecting means (not illustrated).
  • the superimposing means 102 superimposes a selected assumed-movement pattern (for example, FIG. 5 ( a )) on a circumferential-state image generated by the viewpoint converting means 106 (for example, FIG. 4 ) to synthesize them and generate a synthetic image of the present invention, and the displaying means 107 displays the synthetic image.
  • the operation end position 502 becomes an operation end position when starting a driving operation corresponding to the assumed-movement pattern from the current position, that is, a parking position.
  • FIG. 7 shows a synthetic image obtained by synthesizing the assumed-movement patterns shown in FIG. 5
  • FIG. 8 shows a synthetic image obtained by synthesizing the assumed-movement patterns shown in FIG. 6 .
  • the driver can perform parallel parking to the left side (garaging to the right side) by moving the vehicle to a start position 701 ( 801 ) where the obstacles 401 and 402 ( 803 and 804 ) do not interfere with a parking position 702 ( 802 ), the tire trace 503 ( 603 ), and the start position 701 ( 801 ) and starting a series of driving operations according to time-series data from the position so as to park the vehicle at the parking position 702 ( 802 ).
  • a detailed procedure for moving the vehicle to the start position 701 ( 801 ) will be described below by using a case of performing parallel parking to the left side as an example and referring to FIG. 9 and FIGS. 10 ( a ) to 10 ( c ).
  • FIG. 9 is an illustration showing movement of a vehicle when performing parallel parking to the left side.
  • a driver of the vehicle to park the vehicle at a target parking position 902 , a driver of the vehicle must move the vehicle located at a current position 901 to a target start position 903 by assuming the operation start position 501 when adjusting the operation end position 502 of the assumed-movement pattern ( FIG. 5 ( a )) for performing parallel parking to the left side to the target parking position 902 as the target start position 903 .
  • the relative positional relation between the operation end position 502 and the operation start position 501 in FIG. 5 ( a ) corresponds to a case of performing a driving operation in accordance with the time-series data in FIG. 5 ( b ).
  • fine adjustment can be made in accordance with fine adjustment of a currently-operated steering wheel.
  • the driver When the driving-operation assist of this embodiment is not used, the driver must move the vehicle to the target start position 903 by assuming the obstacles 401 and 402 and the target parking position 902 in accordance with a scene which can be confirmed from the inside of the vehicle through direct viewing or a mirror. In this case, there is a problem that an operation for the driver to assume the obstacles 401 and 402 and the target parking position 902 in accordance with the scene which can be confirmed from the inside of the vehicle through direct viewing or a mirror requires skill. Moreover, even when a size of a vehicle or a position of a mirror is changed, there is a problem that a driver cannot easily correspond to the change.
  • the assumed-movement pattern shown in FIG. 5 ( a ) is superimposed on the circumferential-state image viewed from a viewpoint of a virtual camera as if being imaged from a position just above the vehicle as shown in FIG. 4 and thereby, the synthetic image shown in FIG. 7 is generated and displayed for a driver.
  • the operation end position 502 when assuming the current position 901 as the operation start position 501 is displayed as a parking position 1001 corresponding to the current position together with tire traces by displaying adjusting the current position 901 of the vehicle to the operation start position 501 in FIG. 5 ( a ) as shown in FIGS. 10 ( a ) to 10 ( c ).
  • a parking operation is started from the position when the synthetic image 3 in FIG. 10 ( c ) is displayed on the displaying means 107 , a tire trace does not overlap with the obstacle 401 or 402 and the driver can confirm at a glance that the parking position 1001 is a position suitable for parking. Therefore, it is possible to confirm that a parking operation can be started from the position.
  • the driver can directly confirm the positional relation between them at a glance.
  • the driver can confirm a place suitable for start of a parking operation at a glance, easily move a vehicle to the place, and start a parking operation. Therefore, it is possible to park the vehicle at a purposed position more safely and accurately.
  • Assumed-movement-pattern-operation start and end positions and a tire trace are intrinsic to each vehicle. For example, they are greatly different in small vehicle and large vehicle. This can be settled by storing an assumed-movement pattern in the assumed-movement-pattern storing means 108 in FIG. 1 every vehicle. Therefore, a driver can perform driving operations while confirming the relation between an assumed-movement pattern corresponding to the vehicle and a circumferential obstacle even if the vehicle is changed to another one.
  • the camera parameter table 103 in FIG. 1 stores camera parameters of each camera every vehicle and an image shown to a driver is not directly influenced. Therefore, the driver can perform driving operations while confirming the relation between an assumed-movement pattern corresponding to a vehicle displayed in a synthetic image almost similarly displayed before vehicles are changed and a circumferential obstacle even if vehicles are changed and thereby, camera positions are changed.
  • a driver can perform a driving operation while confirming the relation between an assumed-movement pattern corresponding to a vehicle and a circumferential obstacle or the like, almost similarly to a state before vehicles are changed even if vehicles are changed and thereby, it is possible to greatly decrease a load of a driver's skill for change of vehicles.
  • FIG. 11 shows a variation of assumed-movement patterns stored in the assumed-movement-pattern storing means 108 in FIG. 1 .
  • the patterns are right and left parallel-parking assumed-movement patterns 1101 and 1102 and right and left garaging assumed-movement patterns 1103 and 1104 .
  • a driver selects any one of these patterns by pattern selecting means (not illustrated).
  • Areas to be displayed as synthetic images are also determined as shown by outer frames of the assumed-movement patterns 1101 to 1104 in FIG. 11 correspondingly to the above four assumed-movement patterns. That is, an operation start position is assumed as a current vehicle position and a rectangular area including a tire trace and an operation end position is assumed as a synthetic-image area.
  • assumed-movement patterns of the present invention are described as video data showing an operation start position (assumed-movement start area of the present invention), an operation end position (assumed-movement end area of the present invention), and tire traces (image data showing vehicle tire traces of the present invention).
  • other pattern is also permitted, for example, it can be that traces (video data showing a vehicle moving area of the present invention) in which a projection of a vehicle moves are included instead of or together with tire traces.
  • assumed-movement patterns of the present invention serve as video data showing movement of a vehicle when assuming that a predetermined series of driving operations are previously applied to the vehicle.
  • a circumferential-state image of the present invention is obtained by synthesizing video data imaged by the imaging section 101 in real time.
  • other pattern is also permitted, for example, it can be to store the data produced in accordance with already-imaged video data in the space-data buffer 105 and use the data.
  • FIG. 13 is a block diagram showing a configuration of the driving-operation assist of the second embodiment of the present invention.
  • the driving-operation assist of this embodiment is also mainly used to assist driving operations such as garaging and parallel parking. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that start detecting means 1301 , integrating means 1302 , and space converting means 1303 are included.
  • the start detecting means 1301 to which a gear signal showing forward movement or backward movement and a steering-wheel-turning-angle signal showing a front-wheel steering angle are input judges that a driving operation (parking operation) corresponding to an assumed-movement pattern stored in the assumed-movement-pattern storing means 108 is started when a gear signal shows a backward-movement state and a front-wheel steering angle according to a steering-wheel-turning-angle signal reaches a certain value or more, which corresponds to operation start detecting means of the present invention.
  • the integrating means 1302 integrates the steering-wheel rotation angle and the number of revolutions of rear wheels and computes a spatial movement change of a vehicle up to the current point of time after a driving operation (parking operation) is started, which corresponds to moving-position computing means of the present invention.
  • the space converting means 1303 moves an assumed-movement pattern in accordance with the spatial movement change and an assembly of the superimposing means 102 and space converting means 1303 corresponds to synthetic-image generating means of the present invention.
  • a procedure until a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 is the same as the procedure described for the first embodiment.
  • a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started is described below by using a case of performing parallel parking to the left side as an example.
  • the start detecting means 1301 judges that a driving operation (parking operation) corresponding to parallel parking to the left side is started, communicates to the integrating means 1302 that the driving operation (parking operation) is started, and thereafter inputs the steering-wheel-rotation-angle signal and the number-of-revolution-of-rear-wheel signal to the integrating means 1302 .
  • the integrating means 1302 integrates the input steering-wheel-rotation-angle signal and number-of-revolution-of-rear-wheel signal after start of the driving operation and computes the positional relation between a current vehicle position 1402 and a parking-operation start position 1401 as shown in FIG. 14 ( a ).
  • the space converting means 1303 moves an assumed-movement pattern 1403 corresponding to parallel parking to the left side so that the operation start position ( 501 in FIG. 5 ) coincides with the parking-operation start position 1401 as shown in FIG. 14 ( b ).
  • the space converting means 1303 spatially fixes the assumed-movement pattern 1403 at a position when the parking operation starts after start of the driving operation.
  • the superimposing means 102 synthesizes the assumed-movement pattern 1403 spatially fixed at the parking-operation start position and the current vehicle position 1402 by superimposing the pattern 1403 and the position 1402 on a circumferential-state image to generate a synthetic image of the present invention as shown in FIG. 14 ( c ).
  • the displaying means 107 displays the synthetic image. Because circumferential-state images such as the obstacles 401 and 402 are naturally fixed in a space, the positional relation between the circumferential-state images and the assumed-movement pattern 1403 is fixed in the synthetic image.
  • a synthetic image is an image viewed from a viewpoint fixed on to a vehicle, only the current vehicle position 1402 is fixed on the screen when the vehicle moves and circumferential-state images and the assumed-movement pattern 1403 are displayed while they relatively move in FIG. 14 ( c ).
  • a circumferential-state image viewed from a viewpoint of a virtual camera moves in accordance with movement of an actual vehicle and the superimposed and synthesized assumed-movement pattern 1403 also moves in accordance with movement of a vehicle computed by the integrating means 1302 . Therefore, the circumferential-state image and the assumed-movement pattern 1403 show coincident movement. Because a driver only has to operate a steering wheel along a tire trace of an assumed-movement pattern displayed at each point of time, a simpler and safer vehicle operation is realized.
  • FIG. 15 is a block diagram showing a configuration of the driving-operation assist of the third embodiment of the present invention.
  • the driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that start detecting means 1501 and image tracking means 1502 , and space converting means 1503 are included.
  • the start detecting means 1501 has the same functions as the start detecting means 1301 in FIG. 13 described for the second embodiment except that a steering-wheel-turning-angle signal or a number-of-revolution-of-rear-wheel signal is not output to any other unit.
  • the image tracking means 1502 stores positional information of the whole or a part (e.g. operation end position) of video data for assumed-movement patterns on the basis of the whole or a part (e.g. obstacle) of video data for circumferential-state images on a synthetic image when a driving operation (parking operation) is started and corresponds to positional-information storing means of the present invention.
  • the space converting means 1503 moves an assumed-movement pattern in accordance with the positional information and an assembly configured by the superimposing means 102 and space converting means 1503 corresponds to synthetic-image generating means of the present invention.
  • a procedure until a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 is the same as that described for the first embodiment.
  • a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started is described below by using a case of performing parallel parking to the left side as an example.
  • the start detecting means 1501 judges that a driving operation (parking operation) corresponding to parallel parking to the left side is started and communicates to the image tracking means 1502 that the driving operation (parking operation) is started.
  • the image tracking means 1502 When the image tracking means 1502 receives the information showing that the driving operation (parking operation) is started, there is obtained the video data for an end-position circumferential image 1603 including a part of the obstacle 402 which is a part of the video data for a circumferential-state image and a parking-operation end position 1602 on the then synthetic image ( FIG. 16 ( a )) through the space-data buffer 105 , and the video data is stored.
  • the means 1502 finds the portion concerned of the obstacle 402 from a circumferential-state image at each point of time (obtained through the space-data buffer 105 ), adjusts the portion concerned of the obstacle 402 in the end-position circumferential image 1603 to the portion concerned of the obstacle 402 in a circumferential-state image, and thereby determines the positional relation between the parking-operation end position 1602 and the circumferential-state image at that point of time. That is, the image tracking means 1502 tracks the positional relation between the parking-operation end position 1602 and the circumferential-state image at each point of time.
  • the space converting means 1503 moves an assumed-movement pattern corresponding to parallel parking to the left side so that the operation end position ( 502 in FIG. 5 ) coincides with the parking-operation end position 1602 .
  • the space converting means 1503 spatially fixes the assumed-movement pattern to a position at start of the parking operation.
  • the superimposing means 102 superimposes the assumed-movement pattern 1605 spatially-fixed to the position at start of the parking operation and the current vehicle position 1604 on a circumferential-state image and synthesizes them to generate a synthetic image of the present invention.
  • the displaying means 107 displays the synthetic image. Because circumferential-state images of the obstacles 401 and 402 or the like, are naturally fixed in a space, the positional relation between the circumferential-state image and the assumed-movement pattern 1403 is fixed in the synthetic image.
  • the synthetic image is an image viewed from a viewpoint fixed on a vehicle, if the vehicle moves, only the current vehicle position 1604 is fixed on the screen and the circumferential-state image and the assumed-movement pattern 1605 are displayed while they relatively move. That is, as for the driving-operation assist of this embodiment, if a procedure is executed under the same condition as the case of the driving-operation assist of the second embodiment, the synthetic image shown in FIG. 16 ( c ) becomes same as the synthetic image shown in FIG. 14 ( c ).
  • FIG. 17 is a block diagram showing a configuration of the driving-operation assist of the fourth embodiment of the present invention.
  • the driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that start inputting means 1701 , driving controlling means 1702 , integrating means 1703 , and space converting means 1704 are included.
  • the start inputting means 1701 is used for a driver to input and designate start of an actual driving operation (parking operation) corresponding to an assumed-movement pattern, which corresponds to operation start detecting means of the present invention.
  • the driving controlling means 1702 automatically controls operations of a vehicle by controlling a steering-wheel rotation angle and the number of revolutions of rear wheels in accordance with time-series data (e.g. FIG. 5 ( b )) corresponding to an assumed-movement pattern when a driving-operation start instruction is input, which corresponds to operation controlling means of the present invention.
  • time-series data e.g. FIG. 5 ( b )
  • the integrating means 1703 integrates a steering-wheel rotation angle and the number of revolutions of rear wheels and computes a spatial movement change of a vehicle up to the current point of time after a driving operation (parking operation) is started, which corresponds to moving-position computing means of the present invention. That is, the integrating means 1703 has the same function as the integrating means 1302 in FIG. 13 described for the second embodiment.
  • the space converting means 1704 moves an assumed-movement pattern in accordance with the spatial movement change and an assembly constituted of the superimposing means 102 and space converting means 1704 corresponds to synthetic-image generating means of the present invention. That is, the space converting means 1704 has the same functions as the space converting means 1303 in FIG. 13 described for the second embodiment.
  • a procedure until a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 is the same as that described for the first embodiment.
  • a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started is described below by using a case of performing parallel parking to the left side as an example.
  • a driver locates a vehicle at a position suitable for start of a parking operation while viewing a synthetic image displayed on the displaying means 107 before starting the parking operation and then, inputs an instruction for start of the parking operation to the start inputting means 1701 .
  • the start inputting means 1701 communicates that the driving-operation start instruction is input to the driving controlling means 1702 and integrating means 1703 through the assumed-movement-pattern storing means 108 .
  • the driving controlling means 1702 When receiving a parking-operation start instruction, automatically controls driving of a vehicle by generating a steering-wheel-rotation-angle control signal and a number-of-revolution-of-rear-wheel control signal in accordance with the time-series data ( FIG. 5 ( b )) corresponding to an assumed-movement pattern and controlling a steering-wheel control system and a rear-wheel control system.
  • the integrating means 1703 integrates a steering-wheel-operation-angle signal and a number-of-revolution-of-rear-wheel signal to compute the positional relation between the current vehicle position 1402 and the parking-operation start position 1401 as shown in FIG. 14 ( a ).
  • the space converting means 1704 moves the assumed-movement pattern 1403 corresponding to parallel parking to the left side so that the operation start position ( 501 in FIG. 5 ) and the parking-operation start position 1401 coincide with each other in accordance with the computed positional relation. In other words, after start of a driving operation, the space converting means 1704 spatially fixes the assumed-movement pattern 1403 to the parking-operation start position.
  • a subsequent procedure in which the superimposing means 102 generates a synthetic image and the displaying means 107 displays the synthetic image is the same as that described for the second embodiment.
  • the driving control means 1702 stops the vehicle in accordance with the time-series data and thereby, the parking operation is completed.
  • FIG. 18 is a block diagram showing a configuration of the driving-operation assist of the fifth embodiment of the present invention.
  • the driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • the configuration of the driving-operation assist of this embodiment is different from that of the first embodiment in that trace correcting means 1801 is included.
  • the trace correcting means 1801 corrects an assumed-movement pattern and time-series data in accordance with a driving-operation start position and an operation end position input from a driver, which corresponds to pattern correcting means of the present invention.
  • a procedure until a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 is the same as that described for the first embodiment.
  • a procedure until an assumed-movement pattern stored in the assumed-movement-pattern storing means 108 is displayed on a synthetic image by making the operation start position of the pattern coincide with the current vehicle position in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • a procedure until a driver corrects an assumed-movement pattern and time-series data by using the trace correcting means 1801 and the assumed-movement pattern and time-series data are displayed on a synthetic image after the assumed-movement pattern is displayed on the synthetic image is described below by using a case of performing garaging to the left side as an example.
  • FIG. 19 ( a ) a case is assumed in which as a result of moving a vehicle to a current position 1901 where an assumed-movement-pattern-operation end position coincides with a target parking position 1902 in order to perform a garaging operation by assuming the target parking position 1902 as an operation end position so that a driver does not contact with an obstacle 1904 or 1905 , it is found that a tire trace 1903 of the assumed-movement pattern interferes with the obstacle 1905 .
  • the assumed-movement-pattern storing means 108 stores an assumed-movement pattern for a garaging operation to another left side, it is possible to study whether a parking operation is smoothly performed by selecting the assumed-movement pattern stored in the means 108 by pattern selecting means (not illustrated). However, when the means 108 does not store the assumed-movement pattern or when another assumed-movement pattern also interferes with an obstacle, a driver corrects the assumed-movement pattern.
  • the driver moves a pattern showing a vehicle present at the current position 1901 of a synthetic image ( FIG. 19 ( a )) displayed on the displaying means 107 to a new operation start position 1906 by inputting of a numerical value, a pointer, or other means as shown in FIG. 19 ( b ).
  • the trace correcting means 1801 obtains a new tire trace 1907 (refer to FIG. 19 ( c )) for a vehicle to move from the new operation start position 1906 to the target parking position 1902 and generates new assumed-movement patterns and time-series data corresponding to the new assumed-movement patterns.
  • the superimposing means 102 adjusts an operation start position of the new assumed-movement pattern to the current position 1901 of a vehicle and generates a synthetic image as shown in FIG. 19 ( d ) and the displaying means 107 displays the synthetic image.
  • a driver can park a vehicle to the target parking position 1902 by moving the vehicle so as to adjust an operation end position 1908 of the new assumed-movement pattern to the target parking position 1902 and then, starting a driving operation (parking operation) in accordance with the new assumed-movement pattern.
  • an assumed-movement pattern to be renewedly stored or additionally stored in the assumed-movement-pattern storing means 108 is automatically obtained in accordance with positions of a vehicle at start and end of driving input by a driver.
  • it is also permitted to perform an actual driving operation, sample time-series data at that time for the steering angle of a steering wheel, the number of revolution of wheels and the like, and generate and store an assumed-movement pattern in accordance with the time-series data.
  • FIG. 20 is a block diagram showing a configuration of the driving-operation assist of the sixth embodiment of the present invention.
  • the driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that CG-image synthesizing means 2001 is included.
  • the CG-image synthesizing means 2001 stores three-dimensional data corresponding to the assumed-movement patterns stored in the assumed-movement-pattern storing means 108 and generates a three-dimensional (or two-dimensional) image adjusted to a viewpoint of a circumferential-state image, which corresponds to a part of the function of assumed-movement-pattern storing means of the present invention and a part of the function of synthetic-image generating means of the present invention.
  • the viewpoint converting means 106 can change positions of a viewpoint automatically or in accordance with an input from a driver.
  • the assumed-movement-pattern storing means 108 stores positions of a plurality of virtual poles 2001 [sic; 2101 ] arranged on the tire traces 503 together with stored assumed-movement patterns (operation start position 501 , operation end position 502 , and tire traces 503 ).
  • the CG-image synthesizing means 2001 previously generates and stores three-dimensional-data (refer to FIG. 21 ( b )) corresponding to the assumed-movement patterns in accordance with the data for the assumed-movement patterns and the virtual poles 2001 [sic; 2101 ].
  • a procedure until a circumferential-state image of the present invention is generated in accordance with the video data imaged by the imaging section 101 is the same as that described for the first embodiment.
  • a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started is described below by using a case of performing parallel parking to the left side as an example.
  • a driver Before starting an actual driving operation, a driver changes a viewpoint of a circumferential-state image used by the viewpoint converting means 106 to a viewpoint for viewing the rear from the back of the vehicle concerned from a position just above the vehicle concerned. Or, when the viewpoint converting means 106 detects that an actual driving operation corresponding to an assumed-movement pattern is started, the viewpoint is automatically changed to the latter viewpoint.
  • Specific means for detecting that an actual driving operation is started includes, for examples, means same as the start detecting means 1301 described for the second embodiment.
  • a circumferential-state-image which is an output from the viewpoint converting means 106 becomes the image shown in FIG. 21 ( c ).
  • the CG-image synthesizing means 2001 generates a CG image viewed from a viewpoint same as the viewpoint used by the viewpoint converting means 106 by adjusting the current position of the vehicle concerned to the operation start position 501 . In this case, the CG image becomes the image shown in FIG. 21 ( d ).
  • the superimposing means 102 superimposes the CG image on a circumferential-state image and synthesizes them to generate a synthetic image of the present invention as shown in FIG. 21 ( e ).
  • the displaying means 107 displays the synthetic image. Because the synthetic image is an image viewed from a viewpoint fixed on a vehicle, the whole image is displayed while moving relatively when the vehicle moves in case of FIG. 21 ( e ).
  • a driver can determine a parking start position while viewing the displayed image and confirming the relation between a virtual pole or an operation end position and an actual obstacle at a glance, the driver can perform a safe and secure driving operation.
  • the CG-image synthesizing means 2001 generates a CG image from a viewpoint same as the viewpoint used by the viewpoint converting means 106 in real time. However, it is also permitted to use a configuration of previously generating a CG image viewed from a predetermined viewpoint position every assumed-movement pattern and storing these CG images.
  • a circumferential-state image viewed from a virtual camera is generated similarly to the case of the first embodiment.
  • the CG-image synthesizing means 2001 generates a CG image by obtaining the data for a viewpoint of a vehicle-mounted camera 2201 from the camera parameter table 103 .
  • FIG. 23 is a block diagram showing a configuration of the driving-operation assist of the seventh embodiment of the present invention.
  • the driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that mapping means 2301 and a mapping table 2302 are included.
  • the mapping means 2301 fast performs the processing for converting an image input from each camera of the imaging section 101 into an image viewed from an optional viewpoint.
  • the mapping table 2302 stores the data used for the mapping means 2301 to perform conversion.
  • FIG. 24 is a conceptual view showing are example of a mapping table stored in the mapping table 2302 .
  • the mapping table is configured by cells equal to the number of pixels of a screen (that is, a synthetic image generated by the superimposing means 102 ) displayed by the displaying means 107 . That is, the mapping table is configured so that the number of horizontal pixels on a display screen becomes equal to the number of columns of the table and the number of vertical pixels on the display screen becomes equal to the number of rows of the table.
  • each cell has a camera number and pixel coordinates of an image photographed by each camera as data.
  • the top-left cell (cell in which the data (1, 10, 10) is entered) in FIG. 24 shows the top left, that is, the portion of (0,0) on the display screen and the mapping means 2301 performs the processing of displaying the data for the pixel (10,10) of an image photographed by a camera No. 1 on a display screen (0,0)′′ in accordance with the data content (1,10,10) stored in the cell.
  • the mapping means 2301 performs the processing of displaying the data for the pixel (10,10) of an image photographed by a camera No. 1 on a display screen (0,0)′′ in accordance with the data content (1,10,10) stored in the cell.
  • the viewpoint converting means 106 uses a plurality of viewpoints by changing them, it is necessary that the table shown in FIG. 24 is stored in the mapping table 2302 every viewpoint.
  • FIG. 25 is a block diagram showing a configuration of the driving-operation assist of the eighth embodiment of the present invention.
  • the driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that final-position inputting means 2501 , start-position determining means 2502 , and space fixing means 2503 are included.
  • the final-position inputting means 2501 inputs a target end position of a driving operation by a pointer. It is also permitted to input the target end position by inputting a numerical value or through other means.
  • the start-position determining means 2502 obtains a start position of a driving operation corresponding to a target end position input by the final-position inputting means 2501 in accordance with an assumed-movement pattern corresponding to the driving operation concerned.
  • the space fixing means 2503 makes the target end position coincide with the operation end position and thereafter, spatially fixes an assumed-movement pattern corresponding to the driving operation, which has the functions of the integrating means 1302 and space converting means 1303 in FIG. 13 (in FIG. 25 , a rear-wheel-rotation-signal input and a steering-wheel-turning-angle signal input are not illustrated).
  • the means 2503 has the functions of the image tracking means 1502 and space converting means 1503 in FIG. 15 . In this case, however, it is necessary to receive an input of space data from the space buffer 105 [sic; space data buffer 105 ] similarly to the case of the image tracking means 1502 in FIG. 15 .
  • An assembly configured by the superimposing means 102 and space fixing means 2503 corresponds to synthetic-image generating means of the present invention.
  • a procedure until a circumferential-state image of the present invention is generated in accordance with the video data imaged by the imaging section 101 is the same as that described for the first embodiment.
  • a procedure until an assumed-movement pattern stored in the assumed-movement-pattern storing means 108 is displayed on a synthetic image by adjusting an operation start position of the pattern to a current position of a vehicle in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • a procedure until a driver inputs a target end position of a driving operation by using the final-position inputting means 2501 and an assumed-movement pattern including a driving-operation start position corresponding to the target end position is displayed on the synthetic image after an assumed-movement pattern is displayed on a synthetic image will be described below by using a case of performing garaging to the left side as an example.
  • FIG. 26 ( a ) a case is assumed in which, as a result of driver's displaying a synthetic image of the present invention on the displaying means 107 to park a vehicle between the obstacles 401 and 402 so as not to contact them, the parking position 1001 which is the operation end position of the assumed-movement pattern 1403 using the current position 901 of a vehicle as an operation start position overlaps with the obstacle 402 .
  • the driver moves the parking position 1001 to a target position 2602 by using a pointer 2601 displayed on the screen of the displaying means 107 .
  • the assumed-movement pattern 1403 moves together with the parking position 1001 . Therefore, the operation start position of the assumed-movement pattern 1403 is displayed as a start position 2603 for starting a parking operation.
  • the current position 901 of the vehicle is still displayed on the screen of the displaying means 107 .
  • the driver only has to move the vehicle to the start position 2603 while viewing the screen.
  • the assumed-movement pattern 1403 is fixed in a space by the space fixing means 2503 , the relative positional relation between the assumed-movement pattern 1403 and the obstacles 401 and 402 is not changed.
  • a start-position guiding means to the driving-operation assist of this embodiment, which computes a relative positional relation with the current position 901 when the start position 2603 is determined, obtains time-series data for a steering-wheel rotation angle and the number of revolutions of rear wheels necessary to guide a vehicle from the current position 901 up to the start position 2603 , generates a steering-wheel-rotation-angle control signal and a number-of-revolution-of-rear-wheel control signal in accordance with the time-series data, controls a steering-wheel system and a rear-wheel driving system, and thereby automatically controls driving of the vehicle and automatically guides the vehicle from the current position 901 up to the start position 2603 .
  • a start-position guiding means to the driving-operation assist of this embodiment, which computes a relative positional relation with the current position 901 when the start position 2603 is determined, obtains time-series data for a steering-wheel rotation angle and the number of revolutions of rear wheels necessary to guide a vehicle from the current position 90
  • FIG. 27 is a block diagram showing a configuration of the driving-operation assist of the ninth embodiment of the present invention.
  • the driving-operation assist of this embodiment is different from that of the second embodiment only in that outputs of the start detecting means 1301 and the integrating means 1302 are input to the viewpoint converting means 106 and the viewpoint converting means 106 changes viewpoints of a virtual camera in accordance with the outputs.
  • a procedure until a circumferential-state image of the present invention is generated in accordance with the video data imaged by the imaging section 101 is the same as that described for the first embodiment.
  • a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started will be described below by using a case of performing parallel parking to the left side as an example.
  • a viewpoint position of a virtual camera is fixed to a position just above a vehicle as described for the first embodiment by referring to FIG. 10 , the current vehicle position 901 and the assumed-movement pattern 1403 using the position 901 as an operation start position is fixed on a screen as shown in FIG. 28 ( a ), and circumferential-state images such as the obstacles 401 , 402 or the like, are displayed while relatively moving on the screen in accordance with movement of the vehicle.
  • a driver starts a driving operation corresponding to parallel parking to the left side.
  • a gear signal shows a backward-movement state and a front-wheel steering angle according to a steering-wheel-turning-angle signal reaches a certain value or more. Therefore, the start detecting means 1301 judges that the driving operation (parking operation) corresponding to the parallel parking to the left side is started and communicates that the driving operation (parking operation) is started to the integrating means 1302 and viewpoint converting means 106 .
  • the integrating means 1302 integrates input steering-wheel-rotation-angle signal and number-of-revolution-of-rear-wheel signal after start of the driving operation to compute the positional relation between the current vehicle position 1402 and the parking-operation start position 1401 as shown in FIG. 14 ( a ).
  • the space converting means 1303 moves the assumed-movement pattern 1403 corresponding to the parallel parking to the left side so that the operation start position ( 501 in FIG. 5 ) of the pattern 1403 coincides with the parking-operation start position 1401 as shown in FIG. 14 ( b ).
  • the space converting means 1303 spatially fixes the assumed-movement pattern 1403 to the position at start of the parking operation after start of the driving operation.
  • the viewpoint converting means 106 When the viewpoint converting means 106 receives that the driving operation (parking operation) is started, it fixes the then viewpoint position of the virtual camera to a space (ground). That is, after start of the driving operation, the circumferential-state images (obstacles 401 , 402 or the like) are fixed on the screen.
  • the superimposing means 102 After start of the driving operation, the superimposing means 102 superimposes the assumed-movement pattern 1403 spatially fixed to the position at start of the parking operation and the current vehicle position 1402 on the circumferential-state images and synthesizes them to generate a synthetic image of the present invention as shown in FIG. 28 ( c ). Because the viewpoint of the synthetic image is obtained by spatially fixing the viewpoint position of the virtual camera at start of the parking operation similarly to viewpoints of circumferential-state images. Therefore, the superimposing means 102 generates the synthetic image by computing the positional relation computed by the integrating means 1302 inversely to a viewpoint. That is, in this synthetic image ( FIG.
  • the circumferential-state images (obstacles 401 , 402 or the like) and the assumed-movement pattern 1403 are fixed on the screen and the current vehicle position 1402 is displayed while relatively moving on the screen in accordance with actual movement of the vehicle.
  • a driver can confirm a vehicle moving state to a circumferential state of a parking space at a glance.
  • a movement assumed in accordance with an assumed-movement pattern includes a wide range or is complex and a range larger than the field of view of a camera of the imaging section 101 occurs after a driving operation is started, it is also permitted to display the range by using the data stored in the space-data buffer 105 .
  • FIG. 29 is a block diagram showing a configuration of the driving-operation assist of the tenth embodiment of the present invention.
  • the driving-operation assist of this embodiment is different from that of the first embodiment only in that two cases are assumed in which a vehicle moves backward when turning a steering wheel up to the maximum angle clockwise and counterclockwise to tire rotation as default driving patterns stored in the assumed-movement-pattern storing means 108 as shown in FIG. 30 ( b ), two circumscribed area traces 604 in a space through which the whole of the vehicle passes in stead of the tire traces 603 caused by assumed movements of the two cases are synthesized by the superimposing means 102 at the same time as shown in FIG. 30 ( a ), and the synthesized result is displayed on the displaying means 107 .
  • two simplest cases are assumed in which a vehicle moves backward when turning a steering wheel up to the maximum angle clockwise and counterclockwise as shown in FIG. 30 ( b ) as defaults of the assumed-movement storing pattern 108 [sic; assumed-movement pattern storing means]. Therefore, when the vehicle can be parked in accordance with the defaults of the assumed-movement storing pattern 108 [sic; assumed-movement pattern-storing means], advantages can be obtained that it becomes unnecessary to change and select a plurality of assumed-movement storing patterns and an operating load of a driver can be reduced.
  • the circumscribed area traces 60 in a space through which the whole of the vehicle passes instead of the tire traces 603 are synthesized by the superimposing means 102 , and the synthesized result is displayed on the displaying means 107 , and thereby an advantage can be obtained that a driver can easily and more accurately confirm whether a portion of a vehicle to be protruded beyond a tire trace such as the right front portion of the vehicle when parking the vehicle while turning left-backward contacts with an obstacle.
  • FIG. 31 is a block diagram showing a configuration of the driving-operation assist of the eleventh embodiment of the present invention.
  • the driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging and parallel parking. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first and fifth embodiments and a component provided with the same symbol as the case of the first and fifth embodiments has the same function as the case of the first and fifth embodiments unless otherwise explained. Moreover, it is assumed that each modified example described for the first and fifth embodiments can be applied to this embodiment by applying the same modification unless otherwise described.
  • the configuration of the driving-operation assist of this embodiment is different from those of the first and fifth embodiments in that the obstacle inputting means 3101 and assumed-pattern correcting means 3102 shown in FIG. 31 are included and the assumed-movement-pattern storing means 108 includes a movement pattern for changing backward and forward movements of a vehicle while the vehicle moves as shown in FIG. 32 ( a ).
  • a steering-wheel angle corresponding to tire rotation is stored in the assumed-movement-pattern storing means 108 in FIG. 31 as time-series data for assumed-movement patterns as shown in FIG. 32 ( b ).
  • tire rotation of 0 to 0.8 shows backward movement of a vehicle at which backward movement is changed to forward movement.
  • the position of the vehicle is present at a backward- and forward-movement changing position 3201 shown in FIG. 32 ( a ).
  • the vehicle moves forward to a tire angle of 0.8 to 0.6.
  • forward movement is changed to backward movement again such as a tire angle of 0.6 to 1.4.
  • a procedure until a circumferential-state image of the present invention is generated in accordance with the video data imaged by the imaging section 101 is the same as that described for the first embodiment.
  • a procedure until an assumed-movement pattern stored in the assumed-movement-pattern storing means 108 is displayed on a synthetic image of the present invention by adjusting an operation start position of the pattern to the current position of a vehicle in a procedure until the superimposing means 102 generates the synthetic image and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • a procedure until a driver corrects an assumed-movement pattern and time-series data by using the trace correcting means 1801 and obstacle inputting means 3101 after the assumed-movement pattern is displayed on a synthetic image and the corrected assumed-movement pattern and time-series data are displayed on the synthetic image is described below by using a case of performing garaging to the right side as an example.
  • a case is assumed in which a driver moves a vehicle to the current position 1901 where an operation end position of an assumed-movement pattern coincides with the target parking position 1902 in order to perform a garaging operation using the target parking position 1902 as an operation end position so that the vehicle does not contact with an obstacle (a 3204 , b 3205 , or c 3206 ) and thereby, it is found that the circumscribed area 604 of the assumed-movement pattern may contact with the obstacle a, b, or c.
  • the assumed-movement-pattern storing means 108 stores an assumed-movement pattern for another garaging operation to the right side, it is permitted to select the assumed-movement pattern by pattern selecting means (not illustrated) and study if a parking operation can be smoothly performed.
  • pattern selecting means not illustrated
  • the driver corrects the assumed-movement pattern.
  • the driver assumes a pattern showing the vehicle present at the current position 3200 of the synthetic image ( FIG. 33 ) shown on the displaying means 107 in FIG. 31 as the parking-operation start position 1901 , assumes an area in the image in which the obstacle a, b, or c is present as an obstacle designation rectangle: a 3207 as shown in FIG. 34 by using the obstacle inputting means 3101 in FIG. 31 or as an obstacle designation area: 3210 (refer to FIG. 36 ) by using an obstacle designation circle: b 3208 , and sets and inputs the obstacle designation rectangle or the obstacle designation area through numerical-value input, a pointer, or other means. Moreover, when it is necessary to correct the target parking position, the driver performs movement input through numerical-value input, a pointer, or other means.
  • the trace correcting means 1801 sets a contact-hazard area 3209 in an area having a circumference of 60 cm including the obstacle designation area: 3210 as shown in FIG. 35 .
  • a contact-hazard evaluation function H 3210 as shown in FIG. 36 is provided for the area.
  • the function is configured by synthesizing three quadratic functions which sharply increase as approaching for 10 cm or less from the obstacle designation area: 3210 , slowly decrease as separating up to 10 cm or more, and become 0 for 60 cm or more.
  • a trace-contact-hazard evaluation function H′′ 3213 is obtained from the total of the contact-hazard evaluation function H 3210 at the position of the trace evaluation point 3212 .
  • the trace-contact-hazard evaluation function H′′ 3213 becomes a function of N items (tire rotation tm and tire angle km) in the table in FIG. 32 ( b ) as shown in FIG. 37 . Therefore, by successively correcting the diagram (tire rotation tm and tire angle km) through the partial differential method, it is possible to obtain values (tire rotation tm and tire angle km) for minimizing the trace-contact-hazard evaluation function H′′ 3213 .
  • the corrected assumed-movement pattern is a movement pattern for minimizing the trace-contact-hazard evaluation function H′′ 3213 , the assumed-movement pattern generates a trace having a larger margin from an obstacle as shown in FIG. 40 and thereby, a vehicle can be parked more safely.
  • the superimposing means 102 generates a synthetic-image by adjusting the operation start position 1901 of a corrected assumed-movement pattern 3214 to a current vehicle position 3200 and the displaying means 107 displays the synthetic image.
  • the driver when a driver starts a driving operation (parking operation) of a vehicle in accordance with the new corrected assumed-movement pattern 3214 , the driver can park the vehicle at the target parking position 1902 in accordance with a movement pattern having a larger margin from an obstacle.
  • this embodiment is described above by assuming that an assumed-movement pattern to be renewedly-stored or additionally-stored in the assumed-movement-pattern storing means 108 can be automatically obtained in accordance with a position of a vehicle at start or end of a movement input from a driver. However, it is also permitted to perform an actual driving operation and thereby, sample time-series data for the then steering angle of a steering wheel and the number of revolutions of wheels, generate an assumed-movement pattern in accordance with the time-series data, and store the pattern.
  • FIGS. 42 to 46 a twelfth embodiment of the present invention will be described below by referring to FIGS. 42 to 46 .
  • FIG. 42 is different from FIG. 31 in that movement-pattern selection trace correcting means 3301 is used instead of the trace correcting means 3101 .
  • a driver designates and inputs the target parking position 1902 on an image displayed on a display unit by assuming a current vehicle position as a parking-operation start position.
  • the movement-pattern selection trace correcting means 3301 extracts two assumed-movement patterns for parking a vehicle right backward as shown in FIG. 44 in accordance with a rough positional relation of the target parking position 1902 to the parking-operation start position out of a plurality of assumed-movement patterns stored in the assumed-movement-pattern storing means 108 .
  • the two assumed-movement patterns for respectively minimizing the trace-contact-hazard evaluation function H′′ 3213 are corrected similarly to the case of the eleventh embodiment.
  • the two minimized trace-contact-hazard evaluation functions H′′ 3213 with each other and selecting a smaller one of the two functions H′′ 3213 , it is possible to select a safer assumed-movement pattern.
  • a simpler assumed-movement pattern is selected by previously raising the priority of a function H′′ 3213 allowing a simpler driving operation.
  • a driver can automatically select an optimum assumed-movement pattern only by inputting a target parking position and an obstacle area and thereby realize optimum parking through a safer and simpler driving operation.
  • circumferential-state imaging means of the present invention mainly generates an image viewed from a viewpoint of a virtual camera by using a plurality of vehicle-mounted cameras. However, it is also permitted to use one camera set to the ceiling of a roof-provided parking lot. In short, it is possible to use any type of means as circumferential-state imaging means of the present invention as long as the means generates a circumferential-state image by imaging a circumferential state of a vehicle with a camera and/or stores the generated circumferential-state image.
  • the driving-operation assist includes circumferential-state imaging means for imaging a circumferential state of a vehicle with a camera and thereby generating a circumferential-state image and/or storing the circumferential-state image, synthetic-image generating means for superimposing on the circumferential-state image, an assumed-movement pattern which is video data showing movement of the vehicle in the case of performing a predetermined series of driving operations for the vehicle and generating a synthetic image, and displaying means for displaying the synthetic image.
  • the driving-operation assist includes circumferential-state imaging means for imaging a circumferential state of a vehicle with a camera and thereby generating a circumferential-state image and/or storing the circumferential-state image, synthetic-image generating means for superimposing on the circumferential-state image, an assumed-movement pattern which is video data showing movement of the vehicle in the case of performing a predetermined series of driving operations for the vehicle and generating a synthetic image, and displaying means for displaying the synthetic image.
  • the present invention provides a driving-operation assist by which a driver can directly confirm the relation between vehicle movement according to a predetermined series of driving operations and a circumferential state by displaying the vehicle movement when the driver performs the predetermined series of driving operations together with the circumferential sate and a driver's load can be reduced.
  • a driver can confirm a position for starting a driving operation such as garaging or parallel parking, a place for finally stopping a vehicle, and a positional relation with an obstacle such as other vehicle by the display image at a glance by using a driving-operation assist of the present invention. Therefore, it is expected that a driver's operation load is reduced and safety is improved. Moreover, by introducing an automatic-driving technique, it is possible to automatically perform every operation including garaging only by driver's moving a vehicle up to a parking-operation start position while viewing the movement trace data.
  • the present invention provides a recording medium for storing a program for making a computer execute all or some of functions of each means of a driving-operation assist of the present invention.

Abstract

A vehicle-operation assist includes a circumferential-state imager for imaging a circumferential state of a vehicle with a camera and generating a circumferential-state image; a synthetic-image generator for generating a synthetic image by superimposing on the circumferential-state image, an assumed-movement pattern of the vehicle performing a predetermined series of driving operations; and a display for displaying the synthetic image. The circumferential-state imager has at least one camera and a camera parameter table for storing characteristics of the camera and generating the circumferential-state image on the basis of the camera characteristics.

Description

  • This application is a continuation of U.S. patent application Ser. No. 09/581,004, filed Jul. 17, 2000, which is a U.S. National Phase Application of PCT International Application PCT/JP1999/05509, filed Oct. 6, 1999, the entire disclosure of which is incorporated herein by reference.
  • TECHNICAL FIELD
  • The present invention relates to a driving-operation assist for assisting driving operations of a vehicle and a recording medium for storing a program for making a computer execute all or some of functions of each means of the driving-operation assist.
  • BACKGROUND ART
  • A conventional general driving-operation assist estimates a moving trace of a vehicle corresponding to a steering angle of a steering wheel for backward movement of the vehicle by a steering sensor for detecting the steering angle of the steering wheel. When a vehicle moves backward, an image of a rear or side-rear field of view photographed by a camera is displayed and moreover, when the steering wheel is operated, a moving trace of the vehicle estimated correspondingly to the steering angle of the steering wheel is superimposed on the image of the rear or side-rear field of view. According to the assist, driving operations by a driver will be performed as described below. That is, a driver moves a vehicle to a place where the vehicle can be probably parked while fixing the steering wheel of the vehicle. Then, at the place, the driver finds a steering angle capable of moving the vehicle to a space for parking the vehicle without operating any steering wheel while confirming a vehicle-moving trace estimated by operating the steering wheel. Then, by moving the vehicle backward toward a parking space while keeping the steering angle, parking is theoretically completed.
  • A conventional example of the above driving-operation assist is disclosed in the official gazette of Japanese Patent Laid-Open No. 1-14700.
  • However, to park a vehicle by using the above assist, it is necessary to find a place where the vehicle can be moved to a parking space and then determine a steering angle at which the steering wheel is fixed. To master these operations, skill is necessary. Moreover, when the size of a vehicle to be driven is changed, a sense differs. Therefore, the driving know-how accumulated during skill is not greatly useful.
  • Incidentally, to park a vehicle, it is generally difficult to complete parking operations while keeping a steering angle of a steering wheel constant from start of the parking operations except a case in which there is no obstacle nearby. For example, to perform parallel parking, a driver first moves a vehicle backward by turning the steering wheel in a proper direction while moving the vehicle to a place for parking the vehicle from a parking-operation start position and then, moves the vehicle to a target place by inversely turning the steering wheel when the vehicle properly moves backward. That is, in case of taking parallel parking as an example, it is difficult to park a vehicle while keeping a steering angle of a steering wheel constant.
  • Moreover, as for a conventional assist, if a driver slightly moves the steering wheel of a vehicle, a moving route of the vehicle re-estimated by the slight steering-angle change is displayed. Therefore, the driver may be confused.
  • That is, as for a conventional driving-operation assist, a driver cannot intuitively find a place where a vehicle can be easily moved to a parking space at a glance while confirming an image obtained by synthesizing the parking space, a state around the space, and a route for guiding the vehicle to the parking space.
  • DISCLOSURE OF THE INVENTION
  • The present invention is made to solve the above problems of an conventional driving-operation assist and its object is to provide a driving-operation assist by which a driver can directly confirm the relation between movements of a vehicle according to a predetermined series of driving operations and the circumferential state by displaying movements of the vehicle when the driver performs the predetermined series of driving operations together with the circumferential state and the load of the driver can be reduced.
  • Furthermore, it is another object of the present invention to provide a recording medium for storing a program for making a computer execute all or some of functions of each means of the above driving-operation assist.
  • To solve the above described problems the present invention provides a driving-operation assist characterized by comprising:
  • circumferential-state imaging means for imaging a circumferential state of a vehicle with a camera and generating a circumferential-state image and/or storing the generated circumferential-state image;
  • synthetic-image generating means for generating a synthetic image by superimposing on the circumferential-state image, an assumed-movement pattern which is the video data showing movement of the vehicle in case of performing a predetermined series of driving operations for the vehicle; and
  • displaying means for displaying the synthetic image.
  • Another aspect of the present invention is a recording medium characterized by storing a program for making a computer execute all or some of functions of each means of the above driving-operation assist.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a block diagram showing a configuration of a driving-operation assist of a first embodiment of the present invention;
  • FIG. 2 shows a top view and an elevation view of a vehicle provided with cameras of an imaging section 101 of a driving-operation assist of the first embodiment of the present invention;
  • FIG. 3 is an elevation view showing a viewpoint of a virtual camera of the driving-operation assist of the first embodiment of the present invention;
  • FIG. 4 is an illustration showing a circumferential-state image viewed from the virtual camera of the driving-operation assist of the first embodiment of the present invention;
  • FIGS. 5(a) and 5(b) are illustrations showing assumed-movement data of the driving-operation assist of the first embodiment of the present invention;
  • FIGS. 6(a) and 6(b) are illustrations showing assumed-movement data of the driving-operation assist of the first embodiment of the present invention;
  • FIG. 7 is an illustration showing a synthetic image of the driving-operation assist of the first embodiment of the present invention;
  • FIG. 8 is an illustration showing a synthetic image of the driving-operation assist of the first embodiment of the present invention;
  • FIG. 9 is an illustration showing movement of a vehicle at the time of performing parallel parking to the left side;
  • FIGS. 10(a) to 10(c) are illustrations showing synthetic images of the driving-operation assist of the first embodiment of the present invention;
  • FIG. 11 shows illustrations of variation of assumed-movement patterns stored in assumed-movement-pattern storing means 108 of the driving-operation assist of the first embodiment of the present invention;
  • FIG. 12 is an illustration showing a modification of the synthetic image of the driving-operation assist of the first embodiment of the present invention;
  • FIG. 13 is a block diagram showing a configuration of a driving-operation assist of a second embodiment of the present invention;
  • FIGS. 14(a) to 14(c) are illustrations showing synthetic images of the driving-operation assist of the second embodiment of the present invention;
  • FIG. 15 is a block diagram showing a configuration of a driving-operation assist of a third embodiment of the present invention;
  • FIGS. 16(a) to 16(c) are illustrations showing synthetic images of the driving-operation assist of the third embodiment of the present invention;
  • FIG. 17 is a block diagram showing a configuration of a driving-operation assist of a fourth embodiment of the present invention;
  • FIG. 18 is a block diagram showing a configuration of a driving-operation assist of a fifth embodiment of the present invention;
  • FIGS. 19(a) to 19(c) are illustrations showing synthetic images of the driving-operation assist of the fifth embodiment of the present invention;
  • FIG. 20 is a block diagram showing a configuration of a driving-operation assist of a sixth embodiment of the present invention;
  • FIGS. 21(a) to 21(c) are illustrations showing synthetic images of the driving-operation assist of the sixth embodiment of the present invention;
  • FIG. 22 is a block diagram of a configuration of a modification of the driving-operation assist of the sixth embodiment of the present invention;
  • FIG. 23 is a block diagram showing a configuration of a driving-operation assist of a seventh embodiment of the present invention;
  • FIG. 24 is a conceptual view showing a mapping table stored in a mapping table 2302 of the driving-operation assist of the seventh embodiment of the present invention;
  • FIG. 25 is a block diagram showing a configuration of a driving-operation assist of an eighth embodiment of the present invention;
  • FIGS. 26(a) to 26(c) are illustrations showing synthetic images of the driving-operation assist of the eighth embodiment of the present invention;
  • FIG. 27 is a block diagram showing a configuration of a driving-operation assist of a ninth embodiment of the present invention;
  • FIGS. 28(a) to 28(c) are illustrations showing synthetic images of the driving-operation assist of the ninth embodiment of the present invention;
  • FIG. 29 is a block diagram showing a configuration of a driving-operation assist of a tenth embodiment of the present invention;
  • FIGS. 30(a) and 30(b) are illustrations showing synthetic images of the driving-operation assist of the tenth embodiment of the present invention;
  • FIG. 31 is a block diagram showing a configuration of a driving-operation assist of an eleventh embodiment of the present invention;
  • FIGS. 32(a) and 32(b) are illustrations showing synthetic images of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 33 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 34 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 35 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 36 is a graph for explaining a contact-hazard evaluation function of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 37 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 38 is an illustration [sic; expression] showing a contact-hazard evaluation function of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 39 shows examples of assumed movement data of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 40 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 41 is an illustration showing a synthetic image of the driving-operation assist of the eleventh embodiment of the present invention;
  • FIG. 42 is a block diagram showing a configuration of a driving-operation assist of a twelfth embodiment of the present invention;
  • FIG. 43 is an illustration showing a synthetic image of the driving-operation assist of the twelfth embodiment of the present invention;
  • FIG. 44 is a block diagram showing a configuration of the driving-operation assist of the twelfth embodiment of the present invention;
  • FIG. 45 is an illustration showing a synthetic image of the driving-operation assist of the twelfth embodiment of the present invention; and
  • FIG. 46 is an illustration showing a synthetic image of the driving-operation assist of the twelfth embodiment of the present invention.
  • DESCRIPTION OF SYMBOLS
    • 101 Imaging section
    • 102 Superimposing means
    • 103 Camera parameter table
    • 104 Space reconfiguring means
    • 105 Space-data buffer
    • 106 Viewpoint converting means
    • 107 Displaying means
    • 108 Assumed-movement-pattern storing means
    • 1301, 1501 Start detecting means
    • 1302, 1703 Integrating means
    • 1303, 1503, 1704 Space converting means
    • 1502 Image tracking means
    • 1701 Start inputting means
    • 1702 Driving controlling means
    • 1801 Trace correcting means
    • 2001 CG-image synthesizing means
    • 2201 Camera
    • 2301 Mapping means
    • 2302 Mapping table
    • 2501 Final-position inputting means
    • 2502 Start-position determining means
    • 2503 Space fixing means
    BEST MODE FOR CARRYING OUT THE INVENTION
  • Embodiments of the present invention will be described below by referring to the accompanying drawing.
  • First Embodiment
  • First, a first embodiment of the present invention will be described by referring to the drawings.
  • FIG. 1 is a block diagram showing a configuration of the driving-operation-assist of this embodiment. The driving-operation assist of this embodiment is mainly used to assist driving operations for garaging or parallel parking and the like.
  • As shown in FIG. 1, the driving-operation assist of this embodiment is configured by an imaging section 101 comprising N cameras (camera 1 to camera N), a camera parameter table 103 for storing camera parameters which are characteristics of the cameras, space configuring means 104 for generating space data relating each pixel configuring an image output from each camera to a point in a three-dimensional space in accordance with a camera parameter, viewpoint converting means 106 for generating an image viewed from a predetermined viewpoint as a circumferential-state image by referring to the space data, a space-data buffer 105 for temporarily storing the space data, assumed-movement-pattern storing means 108 for storing assumed-movement data including assumed-movement patterns, superimposing means 102 for superimposing the assumed-movement pattern on the circumferential-state image and generating a synthetic image, and displaying means 107 for displaying a synthetic image.
  • An assembly including the imaging section 101, camera parameter table 103, space reconfiguring means 104, and viewpoint converting means 106 corresponds to circumferential-state imaging means of the present invention and the superimposing means 102 corresponds to synthetic-image generating means of the present invention.
  • First, a detailed configuration of the imaging section 101 and a procedure in which a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 will be described below.
  • FIG. 2 shows a top view and an elevation view of a vehicle provided with cameras of the imaging section 101. In this example, six cameras 201 to 206 are arranged on the roof portion of a vehicle as N=6. The six cameras 201 to 206 are arranged so that a part of each camera overlaps with a part of the imaging range of other camera and no dead angle is produced flatly.
  • The camera parameter table 103 stores camera parameters of each of the above cameras (parameters showing camera characteristics such as setting position, setting angle, lens-strain correction value, and focal distance of camera). The space reconfiguring means 104 generates space data relating each pixel configuring an image output from each camera in accordance with the camera parameters to a point in a three-dimensional space on the basis of a vehicle. The space-data buffer 105 temporarily stores the above space data and the viewpoint converting means 106 generates an image viewed from an optional viewpoint such as a viewpoint of a virtual camera 301 shown in FIG. 3 as a circumferential-state image by referring to space data and thereby synthesizing pixels.
  • FIG. 4 shows a circumferential-state image viewed from the viewpoint of the virtual camera 301 shown in FIG. 3. This example shows a case of performing parallel parking, in which two vehicles currently parked are shown on the circumferential-state image as an obstacle 401 and an obstacle 402.
  • Then, a procedure will be described in which the superimposing means 102 generates a synthetic image of the present invention and finally, the displaying means 107 displays the synthetic image.
  • The assumed-movement-pattern storing means 108 stores an assumed-movement pattern which is video data showing movement of a vehicle to which a typical vehicle driving operation is applied and time-series data showing the relation between vehicle moving distance (moving distance due to rotation of tire) and steering angle of steering wheel (rotation angle of steering wheel) as assumed-movement data of the present invention.
  • Among stored assumed-movement data, the assumed-movement data for performing the driving operation of parallel parking to the left side is shown in FIG. 5 and the assumed-movement data for performing the driving operation for garaging to the right side is shown in FIG. 6. FIGS. 5(a) and 6(a) respectively show an assumed-movement pattern which is video data corresponding to a driving operation according to the time-series data shown in FIGS. 5(b) and 6(b) and showing operation start positions 501 and 601 (corresponding to an assumed-movement start area of the present invention), operation end positions 502 and 602 (corresponding to an assumed-movement end area of the present invention), and tire traces 503 and 603 (corresponding to video data showing vehicle tire traces of the present invention).
  • First, a driver selects one of the assumed-movement patterns stored in the assumed-movement-pattern storing means 108 by pattern selecting means (not illustrated). The superimposing means 102 superimposes a selected assumed-movement pattern (for example, FIG. 5(a)) on a circumferential-state image generated by the viewpoint converting means 106 (for example, FIG. 4) to synthesize them and generate a synthetic image of the present invention, and the displaying means 107 displays the synthetic image. In this case, for example, by adjusting the operation start position 501 in FIG. 5(a) to the current position of the vehicle, the operation end position 502 becomes an operation end position when starting a driving operation corresponding to the assumed-movement pattern from the current position, that is, a parking position.
  • FIG. 7 shows a synthetic image obtained by synthesizing the assumed-movement patterns shown in FIG. 5 and FIG. 8 shows a synthetic image obtained by synthesizing the assumed-movement patterns shown in FIG. 6.
  • That is, in FIG. 7 (FIG. 8), the driver can perform parallel parking to the left side (garaging to the right side) by moving the vehicle to a start position 701 (801) where the obstacles 401 and 402 (803 and 804) do not interfere with a parking position 702 (802), the tire trace 503 (603), and the start position 701 (801) and starting a series of driving operations according to time-series data from the position so as to park the vehicle at the parking position 702 (802).
  • A detailed procedure for moving the vehicle to the start position 701 (801) will be described below by using a case of performing parallel parking to the left side as an example and referring to FIG. 9 and FIGS. 10(a) to 10(c).
  • FIG. 9 is an illustration showing movement of a vehicle when performing parallel parking to the left side. As shown in FIG. 9, to park the vehicle at a target parking position 902, a driver of the vehicle must move the vehicle located at a current position 901 to a target start position 903 by assuming the operation start position 501 when adjusting the operation end position 502 of the assumed-movement pattern (FIG. 5(a)) for performing parallel parking to the left side to the target parking position 902 as the target start position 903.
  • The relative positional relation between the operation end position 502 and the operation start position 501 in FIG. 5(a) corresponds to a case of performing a driving operation in accordance with the time-series data in FIG. 5(b). As for an actual driving operation, fine adjustment can be made in accordance with fine adjustment of a currently-operated steering wheel.
  • When the driving-operation assist of this embodiment is not used, the driver must move the vehicle to the target start position 903 by assuming the obstacles 401 and 402 and the target parking position 902 in accordance with a scene which can be confirmed from the inside of the vehicle through direct viewing or a mirror. In this case, there is a problem that an operation for the driver to assume the obstacles 401 and 402 and the target parking position 902 in accordance with the scene which can be confirmed from the inside of the vehicle through direct viewing or a mirror requires skill. Moreover, even when a size of a vehicle or a position of a mirror is changed, there is a problem that a driver cannot easily correspond to the change.
  • However, when the driving-operation assist of this embodiment is used, the assumed-movement pattern shown in FIG. 5(a) is superimposed on the circumferential-state image viewed from a viewpoint of a virtual camera as if being imaged from a position just above the vehicle as shown in FIG. 4 and thereby, the synthetic image shown in FIG. 7 is generated and displayed for a driver.
  • Therefore, at the time of performing a driving operation for moving the vehicle to the target start position 903 in FIG. 9, the operation end position 502 when assuming the current position 901 as the operation start position 501 is displayed as a parking position 1001 corresponding to the current position together with tire traces by displaying adjusting the current position 901 of the vehicle to the operation start position 501 in FIG. 5(a) as shown in FIGS. 10(a) to 10(c). When the vehicle is located at the current position 901 where the parking position 1001 coincides with the target parking position 902, movement to the target start position 903 is completed.
  • That is, if a parking operation is started from the position when the synthetic image 1 in FIG. 10(a) is displayed on the displaying means 107, the parking position 1001 overlaps with the obstacle 402. Therefore, a driver can confirm at a glance that the parking operation must be started from a position where the vehicle is further moved forward (upward in FIG. 10(a)).
  • Moreover, if a parking operation is started from the position when the synthetic image 2 in FIG. 10(b) is displayed on the displaying means 107, a tire trace overlaps with the obstacle 401. Therefore, the driver can confirm at a glance that the parking operation must be started from a position where the vehicle is further moved backward (downward in FIG. 10(a)).
  • Furthermore, if a parking operation is started from the position when the synthetic image 3 in FIG. 10(c) is displayed on the displaying means 107, a tire trace does not overlap with the obstacle 401 or 402 and the driver can confirm at a glance that the parking position 1001 is a position suitable for parking. Therefore, it is possible to confirm that a parking operation can be started from the position.
  • Thus, by virtually synthesizing images showing the positional relation between a vehicle viewed from a position just above the vehicle, circumferential obstacles, parking end position, and tire traces and showing a synthesized image to a driver, the driver can directly confirm the positional relation between them at a glance. As a result, the driver can confirm a place suitable for start of a parking operation at a glance, easily move a vehicle to the place, and start a parking operation. Therefore, it is possible to park the vehicle at a purposed position more safely and accurately.
  • Assumed-movement-pattern-operation start and end positions and a tire trace are intrinsic to each vehicle. For example, they are greatly different in small vehicle and large vehicle. This can be settled by storing an assumed-movement pattern in the assumed-movement-pattern storing means 108 in FIG. 1 every vehicle. Therefore, a driver can perform driving operations while confirming the relation between an assumed-movement pattern corresponding to the vehicle and a circumferential obstacle even if the vehicle is changed to another one.
  • Moreover, when vehicles are changed, it is considered that positions of and the number of vehicle-mounted cameras shown in FIG. 2 are also changed. However, this can also be settled by the fact that the camera parameter table 103 in FIG. 1 stores camera parameters of each camera every vehicle and an image shown to a driver is not directly influenced. Therefore, the driver can perform driving operations while confirming the relation between an assumed-movement pattern corresponding to a vehicle displayed in a synthetic image almost similarly displayed before vehicles are changed and a circumferential obstacle even if vehicles are changed and thereby, camera positions are changed.
  • As described above, according to this embodiment, it is possible to directly confirm a driving vehicle, an obstacle, and a purposed position at a glance in a driving operation such as parallel parking conventionally requiring a considerable skill for a driver, perform more safe and accurate driving operation, and greatly decrease an operation load of the driver.
  • Moreover, a driver can perform a driving operation while confirming the relation between an assumed-movement pattern corresponding to a vehicle and a circumferential obstacle or the like, almost similarly to a state before vehicles are changed even if vehicles are changed and thereby, it is possible to greatly decrease a load of a driver's skill for change of vehicles.
  • FIG. 11 shows a variation of assumed-movement patterns stored in the assumed-movement-pattern storing means 108 in FIG. 1. The patterns are right and left parallel-parking assumed- movement patterns 1101 and 1102 and right and left garaging assumed- movement patterns 1103 and 1104. As described above, a driver selects any one of these patterns by pattern selecting means (not illustrated). Areas to be displayed as synthetic images are also determined as shown by outer frames of the assumed-movement patterns 1101 to 1104 in FIG. 11 correspondingly to the above four assumed-movement patterns. That is, an operation start position is assumed as a current vehicle position and a rectangular area including a tire trace and an operation end position is assumed as a synthetic-image area.
  • Though a vehicle is not imaged from a vehicle-mounted camera in general, it is permitted in this case to hold CG data for a vehicle and actual-vehicle data and display these data in a synthetic image by superimposing them on each other similarly to trace data.
  • For this embodiment, assumed-movement patterns of the present invention are described as video data showing an operation start position (assumed-movement start area of the present invention), an operation end position (assumed-movement end area of the present invention), and tire traces (image data showing vehicle tire traces of the present invention). However, other pattern is also permitted, for example, it can be that traces (video data showing a vehicle moving area of the present invention) in which a projection of a vehicle moves are included instead of or together with tire traces. In short, it is only necessary that assumed-movement patterns of the present invention serve as video data showing movement of a vehicle when assuming that a predetermined series of driving operations are previously applied to the vehicle.
  • Moreover, as described above, if directly displaying a tire trace and/or a vehicle moving area, there is no margin for contact between the vehicle and an obstacle or the like. Therefore, as shown in FIG. 12, it is also permitted to display a margin line 1201 set-outward by a predetermined value (e.g. 50 cm) from a tire trace or the outer margin of a vehicle moving area.
  • Furthermore, for this embodiment, it is described that a circumferential-state image of the present invention is obtained by synthesizing video data imaged by the imaging section 101 in real time. However, at a place where the same driving operation is frequently performed and circumferential state is hardly fluctuated, other pattern is also permitted, for example, it can be to store the data produced in accordance with already-imaged video data in the space-data buffer 105 and use the data.
  • Second Embodiment
  • Then, a second embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 13 is a block diagram showing a configuration of the driving-operation assist of the second embodiment of the present invention. The driving-operation assist of this embodiment is also mainly used to assist driving operations such as garaging and parallel parking. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • As shown in FIG. 13, the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that start detecting means 1301, integrating means 1302, and space converting means 1303 are included.
  • The start detecting means 1301 to which a gear signal showing forward movement or backward movement and a steering-wheel-turning-angle signal showing a front-wheel steering angle are input judges that a driving operation (parking operation) corresponding to an assumed-movement pattern stored in the assumed-movement-pattern storing means 108 is started when a gear signal shows a backward-movement state and a front-wheel steering angle according to a steering-wheel-turning-angle signal reaches a certain value or more, which corresponds to operation start detecting means of the present invention.
  • The integrating means 1302 integrates the steering-wheel rotation angle and the number of revolutions of rear wheels and computes a spatial movement change of a vehicle up to the current point of time after a driving operation (parking operation) is started, which corresponds to moving-position computing means of the present invention.
  • The space converting means 1303 moves an assumed-movement pattern in accordance with the spatial movement change and an assembly of the superimposing means 102 and space converting means 1303 corresponds to synthetic-image generating means of the present invention.
  • In this embodiment, a procedure until a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 is the same as the procedure described for the first embodiment. Moreover, a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • A procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started is described below by using a case of performing parallel parking to the left side as an example.
  • When a driver starts a driving operation corresponding to parallel parking to the left side, a gear signal shows a backward-movement state and a front-wheel steering angle according to a steering-wheel-turning-angle signal reaches a certain value or more. Therefore, the start detecting means 1301 judges that a driving operation (parking operation) corresponding to parallel parking to the left side is started, communicates to the integrating means 1302 that the driving operation (parking operation) is started, and thereafter inputs the steering-wheel-rotation-angle signal and the number-of-revolution-of-rear-wheel signal to the integrating means 1302.
  • The integrating means 1302 integrates the input steering-wheel-rotation-angle signal and number-of-revolution-of-rear-wheel signal after start of the driving operation and computes the positional relation between a current vehicle position 1402 and a parking-operation start position 1401 as shown in FIG. 14(a).
  • In accordance with the computed positional relation, the space converting means 1303 moves an assumed-movement pattern 1403 corresponding to parallel parking to the left side so that the operation start position (501 in FIG. 5) coincides with the parking-operation start position 1401 as shown in FIG. 14(b). In other words, the space converting means 1303 spatially fixes the assumed-movement pattern 1403 at a position when the parking operation starts after start of the driving operation.
  • After start of a driving operation, the superimposing means 102 synthesizes the assumed-movement pattern 1403 spatially fixed at the parking-operation start position and the current vehicle position 1402 by superimposing the pattern 1403 and the position 1402 on a circumferential-state image to generate a synthetic image of the present invention as shown in FIG. 14(c). The displaying means 107 displays the synthetic image. Because circumferential-state images such as the obstacles 401 and 402 are naturally fixed in a space, the positional relation between the circumferential-state images and the assumed-movement pattern 1403 is fixed in the synthetic image. Moreover, because a synthetic image is an image viewed from a viewpoint fixed on to a vehicle, only the current vehicle position 1402 is fixed on the screen when the vehicle moves and circumferential-state images and the assumed-movement pattern 1403 are displayed while they relatively move in FIG. 14(c).
  • That is, according to this embodiment, a circumferential-state image viewed from a viewpoint of a virtual camera moves in accordance with movement of an actual vehicle and the superimposed and synthesized assumed-movement pattern 1403 also moves in accordance with movement of a vehicle computed by the integrating means 1302. Therefore, the circumferential-state image and the assumed-movement pattern 1403 show coincident movement. Because a driver only has to operate a steering wheel along a tire trace of an assumed-movement pattern displayed at each point of time, a simpler and safer vehicle operation is realized.
  • Third Embodiment
  • Then, a third embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 15 is a block diagram showing a configuration of the driving-operation assist of the third embodiment of the present invention. The driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • As shown in FIG. 15, the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that start detecting means 1501 and image tracking means 1502, and space converting means 1503 are included.
  • The start detecting means 1501 has the same functions as the start detecting means 1301 in FIG. 13 described for the second embodiment except that a steering-wheel-turning-angle signal or a number-of-revolution-of-rear-wheel signal is not output to any other unit.
  • The image tracking means 1502 stores positional information of the whole or a part (e.g. operation end position) of video data for assumed-movement patterns on the basis of the whole or a part (e.g. obstacle) of video data for circumferential-state images on a synthetic image when a driving operation (parking operation) is started and corresponds to positional-information storing means of the present invention.
  • The space converting means 1503 moves an assumed-movement pattern in accordance with the positional information and an assembly configured by the superimposing means 102 and space converting means 1503 corresponds to synthetic-image generating means of the present invention.
  • In this embodiment, a procedure until a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 is the same as that described for the first embodiment. Moreover, a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • A procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started is described below by using a case of performing parallel parking to the left side as an example.
  • When a driver starts a driving operation corresponding to parallel parking to the left side, a gear signal shows a backward-movement state and a front-wheel steering angle according to a steering-wheel-turning-angle signal reaches a certain value or more. Therefore, the start detecting means 1501 judges that a driving operation (parking operation) corresponding to parallel parking to the left side is started and communicates to the image tracking means 1502 that the driving operation (parking operation) is started.
  • When the image tracking means 1502 receives the information showing that the driving operation (parking operation) is started, there is obtained the video data for an end-position circumferential image 1603 including a part of the obstacle 402 which is a part of the video data for a circumferential-state image and a parking-operation end position 1602 on the then synthetic image (FIG. 16(a)) through the space-data buffer 105, and the video data is stored. After start of the driving operation, the means 1502 finds the portion concerned of the obstacle 402 from a circumferential-state image at each point of time (obtained through the space-data buffer 105), adjusts the portion concerned of the obstacle 402 in the end-position circumferential image 1603 to the portion concerned of the obstacle 402 in a circumferential-state image, and thereby determines the positional relation between the parking-operation end position 1602 and the circumferential-state image at that point of time. That is, the image tracking means 1502 tracks the positional relation between the parking-operation end position 1602 and the circumferential-state image at each point of time.
  • According to the positional relation, the space converting means 1503 moves an assumed-movement pattern corresponding to parallel parking to the left side so that the operation end position (502 in FIG. 5) coincides with the parking-operation end position 1602. In other words, after start of the driving operation, the space converting means 1503 spatially fixes the assumed-movement pattern to a position at start of the parking operation.
  • As shown in FIG. 16(c), after start of the driving operation, the superimposing means 102 superimposes the assumed-movement pattern 1605 spatially-fixed to the position at start of the parking operation and the current vehicle position 1604 on a circumferential-state image and synthesizes them to generate a synthetic image of the present invention. The displaying means 107 displays the synthetic image. Because circumferential-state images of the obstacles 401 and 402 or the like, are naturally fixed in a space, the positional relation between the circumferential-state image and the assumed-movement pattern 1403 is fixed in the synthetic image.
  • Moreover, in FIG. 16(c), because the synthetic image is an image viewed from a viewpoint fixed on a vehicle, if the vehicle moves, only the current vehicle position 1604 is fixed on the screen and the circumferential-state image and the assumed-movement pattern 1605 are displayed while they relatively move. That is, as for the driving-operation assist of this embodiment, if a procedure is executed under the same condition as the case of the driving-operation assist of the second embodiment, the synthetic image shown in FIG. 16(c) becomes same as the synthetic image shown in FIG. 14(c).
  • According to this embodiment, because a circumferential-state image viewed from a viewpoint of a virtual camera moves in accordance with movement of an actual vehicle and the superimposed and synthesized assumed-movement pattern 1605 also moves in accordance with the movement of the vehicle. Therefore, both show coincident movement. Because a driver only has to operate a steering wheel along trace data for an assumed-movement pattern displayed at each point of time, a simpler and safer vehicle operation is realized.
  • Fourth Embodiment
  • Then, a fourth embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 17 is a block diagram showing a configuration of the driving-operation assist of the fourth embodiment of the present invention. The driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • As shown in FIG. 17, the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that start inputting means 1701, driving controlling means 1702, integrating means 1703, and space converting means 1704 are included.
  • The start inputting means 1701 is used for a driver to input and designate start of an actual driving operation (parking operation) corresponding to an assumed-movement pattern, which corresponds to operation start detecting means of the present invention.
  • The driving controlling means 1702 automatically controls operations of a vehicle by controlling a steering-wheel rotation angle and the number of revolutions of rear wheels in accordance with time-series data (e.g. FIG. 5(b)) corresponding to an assumed-movement pattern when a driving-operation start instruction is input, which corresponds to operation controlling means of the present invention.
  • The integrating means 1703 integrates a steering-wheel rotation angle and the number of revolutions of rear wheels and computes a spatial movement change of a vehicle up to the current point of time after a driving operation (parking operation) is started, which corresponds to moving-position computing means of the present invention. That is, the integrating means 1703 has the same function as the integrating means 1302 in FIG. 13 described for the second embodiment.
  • The space converting means 1704 moves an assumed-movement pattern in accordance with the spatial movement change and an assembly constituted of the superimposing means 102 and space converting means 1704 corresponds to synthetic-image generating means of the present invention. That is, the space converting means 1704 has the same functions as the space converting means 1303 in FIG. 13 described for the second embodiment.
  • In this embodiment, a procedure until a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 is the same as that described for the first embodiment. Moreover, a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • A procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started is described below by using a case of performing parallel parking to the left side as an example.
  • A driver locates a vehicle at a position suitable for start of a parking operation while viewing a synthetic image displayed on the displaying means 107 before starting the parking operation and then, inputs an instruction for start of the parking operation to the start inputting means 1701. The start inputting means 1701 communicates that the driving-operation start instruction is input to the driving controlling means 1702 and integrating means 1703 through the assumed-movement-pattern storing means 108.
  • When receiving a parking-operation start instruction, the driving controlling means 1702 automatically controls driving of a vehicle by generating a steering-wheel-rotation-angle control signal and a number-of-revolution-of-rear-wheel control signal in accordance with the time-series data (FIG. 5(b)) corresponding to an assumed-movement pattern and controlling a steering-wheel control system and a rear-wheel control system.
  • In case of receiving a parking-operation start instruction, the integrating means 1703 integrates a steering-wheel-operation-angle signal and a number-of-revolution-of-rear-wheel signal to compute the positional relation between the current vehicle position 1402 and the parking-operation start position 1401 as shown in FIG. 14(a).
  • As shown in FIG. 14(b), the space converting means 1704 moves the assumed-movement pattern 1403 corresponding to parallel parking to the left side so that the operation start position (501 in FIG. 5) and the parking-operation start position 1401 coincide with each other in accordance with the computed positional relation. In other words, after start of a driving operation, the space converting means 1704 spatially fixes the assumed-movement pattern 1403 to the parking-operation start position.
  • A subsequent procedure in which the superimposing means 102 generates a synthetic image and the displaying means 107 displays the synthetic image is the same as that described for the second embodiment. When the vehicle reaches a predetermined parking position, the driving control means 1702 stops the vehicle in accordance with the time-series data and thereby, the parking operation is completed.
  • According to this embodiment, it is possible to obtain an advantage that a steering-wheel operation, and the like are automatically performed in addition to the advantages described for the second embodiment after start of a driving operation is designated. Because a driver only has to confirm that a steering-wheel operation is automatically generated in accordance with the trace data for a displayed assumed-movement pattern at each point of time and monitor that a new obstacle appears, a simpler and safer vehicle operation is realized.
  • Fifth Embodiment
  • Then, a fifth embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 18 is a block diagram showing a configuration of the driving-operation assist of the fifth embodiment of the present invention. The driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • As shown in FIG. 18, the configuration of the driving-operation assist of this embodiment is different from that of the first embodiment in that trace correcting means 1801 is included.
  • The trace correcting means 1801 corrects an assumed-movement pattern and time-series data in accordance with a driving-operation start position and an operation end position input from a driver, which corresponds to pattern correcting means of the present invention.
  • In this embodiment, a procedure until a circumferential-state image of the present invention is generated in accordance with video data imaged by the imaging section 101 is the same as that described for the first embodiment. Moreover, a procedure until an assumed-movement pattern stored in the assumed-movement-pattern storing means 108 is displayed on a synthetic image by making the operation start position of the pattern coincide with the current vehicle position in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • A procedure until a driver corrects an assumed-movement pattern and time-series data by using the trace correcting means 1801 and the assumed-movement pattern and time-series data are displayed on a synthetic image after the assumed-movement pattern is displayed on the synthetic image is described below by using a case of performing garaging to the left side as an example.
  • As shown in FIG. 19(a), a case is assumed in which as a result of moving a vehicle to a current position 1901 where an assumed-movement-pattern-operation end position coincides with a target parking position 1902 in order to perform a garaging operation by assuming the target parking position 1902 as an operation end position so that a driver does not contact with an obstacle 1904 or 1905, it is found that a tire trace 1903 of the assumed-movement pattern interferes with the obstacle 1905.
  • When the assumed-movement-pattern storing means 108 stores an assumed-movement pattern for a garaging operation to another left side, it is possible to study whether a parking operation is smoothly performed by selecting the assumed-movement pattern stored in the means 108 by pattern selecting means (not illustrated). However, when the means 108 does not store the assumed-movement pattern or when another assumed-movement pattern also interferes with an obstacle, a driver corrects the assumed-movement pattern.
  • First, the driver moves a pattern showing a vehicle present at the current position 1901 of a synthetic image (FIG. 19(a)) displayed on the displaying means 107 to a new operation start position 1906 by inputting of a numerical value, a pointer, or other means as shown in FIG. 19(b).
  • When the new operation start position 1906 is determined, the trace correcting means 1801 obtains a new tire trace 1907 (refer to FIG. 19(c)) for a vehicle to move from the new operation start position 1906 to the target parking position 1902 and generates new assumed-movement patterns and time-series data corresponding to the new assumed-movement patterns.
  • The superimposing means 102 adjusts an operation start position of the new assumed-movement pattern to the current position 1901 of a vehicle and generates a synthetic image as shown in FIG. 19(d) and the displaying means 107 displays the synthetic image.
  • Therefore, a driver can park a vehicle to the target parking position 1902 by moving the vehicle so as to adjust an operation end position 1908 of the new assumed-movement pattern to the target parking position 1902 and then, starting a driving operation (parking operation) in accordance with the new assumed-movement pattern.
  • It is permitted to store the generated new assumed-movement pattern and time-series data in the assumed-movement-pattern storing means 108 after updating an original assumed-movement pattern or to additionally store the new assumed-movement pattern and time-series data in the assumed-movement pattern storing means 108 without updating the original assumed-movement pattern. Moreover, it is permitted not to store them as temporary matters assumed-movement pattern and time-series data in the means 108. Furthermore, it is permitted that a driver selects renewed storage, additional storage, or non-storage each time.
  • Moreover, for this embodiment, it is described that an assumed-movement pattern to be renewedly stored or additionally stored in the assumed-movement-pattern storing means 108 is automatically obtained in accordance with positions of a vehicle at start and end of driving input by a driver. However, it is also permitted to perform an actual driving operation, sample time-series data at that time for the steering angle of a steering wheel, the number of revolution of wheels and the like, and generate and store an assumed-movement pattern in accordance with the time-series data.
  • According to this embodiment, it is possible to realize an extensible driving-operation assist compared to the driving-operation assist of the first embodiment.
  • Sixth Embodiment
  • Then, a sixth embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 20 is a block diagram showing a configuration of the driving-operation assist of the sixth embodiment of the present invention. The driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • As shown in FIG. 20, the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that CG-image synthesizing means 2001 is included.
  • The CG-image synthesizing means 2001 stores three-dimensional data corresponding to the assumed-movement patterns stored in the assumed-movement-pattern storing means 108 and generates a three-dimensional (or two-dimensional) image adjusted to a viewpoint of a circumferential-state image, which corresponds to a part of the function of assumed-movement-pattern storing means of the present invention and a part of the function of synthetic-image generating means of the present invention.
  • It is assumed that the viewpoint converting means 106 can change positions of a viewpoint automatically or in accordance with an input from a driver.
  • Moreover, as shown in FIG. 21(a), the assumed-movement-pattern storing means 108 stores positions of a plurality of virtual poles 2001 [sic; 2101] arranged on the tire traces 503 together with stored assumed-movement patterns (operation start position 501, operation end position 502, and tire traces 503). The CG-image synthesizing means 2001 previously generates and stores three-dimensional-data (refer to FIG. 21(b)) corresponding to the assumed-movement patterns in accordance with the data for the assumed-movement patterns and the virtual poles 2001 [sic; 2101].
  • In this embodiment, a procedure until a circumferential-state image of the present invention is generated in accordance with the video data imaged by the imaging section 101 is the same as that described for the first embodiment. Moreover, a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • A procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started is described below by using a case of performing parallel parking to the left side as an example.
  • Before starting an actual driving operation, a driver changes a viewpoint of a circumferential-state image used by the viewpoint converting means 106 to a viewpoint for viewing the rear from the back of the vehicle concerned from a position just above the vehicle concerned. Or, when the viewpoint converting means 106 detects that an actual driving operation corresponding to an assumed-movement pattern is started, the viewpoint is automatically changed to the latter viewpoint. Specific means for detecting that an actual driving operation is started includes, for examples, means same as the start detecting means 1301 described for the second embodiment.
  • Because viewpoints are changed, a circumferential-state-image which is an output from the viewpoint converting means 106 becomes the image shown in FIG. 21(c). The CG-image synthesizing means 2001 generates a CG image viewed from a viewpoint same as the viewpoint used by the viewpoint converting means 106 by adjusting the current position of the vehicle concerned to the operation start position 501. In this case, the CG image becomes the image shown in FIG. 21(d).
  • The superimposing means 102 superimposes the CG image on a circumferential-state image and synthesizes them to generate a synthetic image of the present invention as shown in FIG. 21(e). The displaying means 107 displays the synthetic image. Because the synthetic image is an image viewed from a viewpoint fixed on a vehicle, the whole image is displayed while moving relatively when the vehicle moves in case of FIG. 21(e).
  • In this embodiment, because a driver can determine a parking start position while viewing the displayed image and confirming the relation between a virtual pole or an operation end position and an actual obstacle at a glance, the driver can perform a safe and secure driving operation.
  • In the above explanation, the CG-image synthesizing means 2001 generates a CG image from a viewpoint same as the viewpoint used by the viewpoint converting means 106 in real time. However, it is also permitted to use a configuration of previously generating a CG image viewed from a predetermined viewpoint position every assumed-movement pattern and storing these CG images.
  • Moreover, in this embodiment, it is described that a circumferential-state image viewed from a virtual camera is generated similarly to the case of the first embodiment. However, when purposing only an image viewing the rear from the back of a vehicle without changing any viewpoint, it is also permitted to directly use an image imaged by a camera set to the viewpoint position concerned as a circumferential-state image. A configuration of a driving-operation assist in this case is shown by the block diagram in FIG. 22. That is, the CG-image synthesizing means 2001 generates a CG image by obtaining the data for a viewpoint of a vehicle-mounted camera 2201 from the camera parameter table 103.
  • Seventh Embodiment
  • Then, a seventh embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 23 is a block diagram showing a configuration of the driving-operation assist of the seventh embodiment of the present invention. The driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • As shown in FIG. 23, the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that mapping means 2301 and a mapping table 2302 are included.
  • The mapping means 2301 fast performs the processing for converting an image input from each camera of the imaging section 101 into an image viewed from an optional viewpoint.
  • The mapping table 2302 stores the data used for the mapping means 2301 to perform conversion.
  • FIG. 24 is a conceptual view showing are example of a mapping table stored in the mapping table 2302. The mapping table is configured by cells equal to the number of pixels of a screen (that is, a synthetic image generated by the superimposing means 102) displayed by the displaying means 107. That is, the mapping table is configured so that the number of horizontal pixels on a display screen becomes equal to the number of columns of the table and the number of vertical pixels on the display screen becomes equal to the number of rows of the table. Moreover, each cell has a camera number and pixel coordinates of an image photographed by each camera as data.
  • For example, the top-left cell (cell in which the data (1, 10, 10) is entered) in FIG. 24 shows the top left, that is, the portion of (0,0) on the display screen and the mapping means 2301 performs the processing of displaying the data for the pixel (10,10) of an image photographed by a camera No. 1 on a display screen (0,0)″ in accordance with the data content (1,10,10) stored in the cell. By using the table, it is unnecessary to perform the computation for deciding the data for a pixel of a camera with which each pixel on a display screen should be replaced whether to replace the data for a pixel of a replaced one. Therefore, it is possible to realize to accelerate the processing.
  • When the viewpoint converting means 106 uses a plurality of viewpoints by changing them, it is necessary that the table shown in FIG. 24 is stored in the mapping table 2302 every viewpoint.
  • Eighth Embodiment
  • Then, an eighth embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 25 is a block diagram showing a configuration of the driving-operation assist of the eighth embodiment of the present invention. The driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging, parallel parking and the like. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment and a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • As shown in FIG. 25, the configuration of the driving-operation assist of this embodiment is different from that of the driving-operation assist of the first embodiment in that final-position inputting means 2501, start-position determining means 2502, and space fixing means 2503 are included.
  • The final-position inputting means 2501 inputs a target end position of a driving operation by a pointer. It is also permitted to input the target end position by inputting a numerical value or through other means.
  • The start-position determining means 2502 obtains a start position of a driving operation corresponding to a target end position input by the final-position inputting means 2501 in accordance with an assumed-movement pattern corresponding to the driving operation concerned.
  • The space fixing means 2503 makes the target end position coincide with the operation end position and thereafter, spatially fixes an assumed-movement pattern corresponding to the driving operation, which has the functions of the integrating means 1302 and space converting means 1303 in FIG. 13 (in FIG. 25, a rear-wheel-rotation-signal input and a steering-wheel-turning-angle signal input are not illustrated).
  • Moreover, it is permitted that the means 2503 has the functions of the image tracking means 1502 and space converting means 1503 in FIG. 15. In this case, however, it is necessary to receive an input of space data from the space buffer 105 [sic; space data buffer 105] similarly to the case of the image tracking means 1502 in FIG. 15. An assembly configured by the superimposing means 102 and space fixing means 2503 corresponds to synthetic-image generating means of the present invention.
  • In this embodiment, a procedure until a circumferential-state image of the present invention is generated in accordance with the video data imaged by the imaging section 101 is the same as that described for the first embodiment. Moreover, a procedure until an assumed-movement pattern stored in the assumed-movement-pattern storing means 108 is displayed on a synthetic image by adjusting an operation start position of the pattern to a current position of a vehicle in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • A procedure until a driver inputs a target end position of a driving operation by using the final-position inputting means 2501 and an assumed-movement pattern including a driving-operation start position corresponding to the target end position is displayed on the synthetic image after an assumed-movement pattern is displayed on a synthetic image will be described below by using a case of performing garaging to the left side as an example.
  • As shown in FIG. 26(a), a case is assumed in which, as a result of driver's displaying a synthetic image of the present invention on the displaying means 107 to park a vehicle between the obstacles 401 and 402 so as not to contact them, the parking position 1001 which is the operation end position of the assumed-movement pattern 1403 using the current position 901 of a vehicle as an operation start position overlaps with the obstacle 402.
  • The driver moves the parking position 1001 to a target position 2602 by using a pointer 2601 displayed on the screen of the displaying means 107. In this case, as shown in FIG. 26(b), the assumed-movement pattern 1403 moves together with the parking position 1001. Therefore, the operation start position of the assumed-movement pattern 1403 is displayed as a start position 2603 for starting a parking operation.
  • As shown in FIG. 26(c), even after the above moving operation is completed, the current position 901 of the vehicle is still displayed on the screen of the displaying means 107. The driver only has to move the vehicle to the start position 2603 while viewing the screen. In this case, because the assumed-movement pattern 1403 is fixed in a space by the space fixing means 2503, the relative positional relation between the assumed-movement pattern 1403 and the obstacles 401 and 402 is not changed.
  • According to this embodiment, because a driving-operation start position can be efficiently obtained in addition to the advantages described for the first embodiment, it is possible to decrease the time required by start of the operation.
  • Moreover, it is permitted to add a start-position guiding means to the driving-operation assist of this embodiment, which computes a relative positional relation with the current position 901 when the start position 2603 is determined, obtains time-series data for a steering-wheel rotation angle and the number of revolutions of rear wheels necessary to guide a vehicle from the current position 901 up to the start position 2603, generates a steering-wheel-rotation-angle control signal and a number-of-revolution-of-rear-wheel control signal in accordance with the time-series data, controls a steering-wheel system and a rear-wheel driving system, and thereby automatically controls driving of the vehicle and automatically guides the vehicle from the current position 901 up to the start position 2603. Thereby, because the vehicle is guided up to a start position without operations by the driver, simpler and safer vehicle operations are realized.
  • Ninth Embodiment
  • Then, a ninth embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 27 is a block diagram showing a configuration of the driving-operation assist of the ninth embodiment of the present invention. The driving-operation assist of this embodiment is different from that of the second embodiment only in that outputs of the start detecting means 1301 and the integrating means 1302 are input to the viewpoint converting means 106 and the viewpoint converting means 106 changes viewpoints of a virtual camera in accordance with the outputs.
  • Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the second embodiment and a component provided with the same symbol as the case of the second embodiment has the same function as the case of the second embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the second embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • In this embodiment, a procedure until a circumferential-state image of the present invention is generated in accordance with the video data imaged by the imaging section 101 is the same as that described for the first embodiment. Moreover, a procedure until an actual driving operation corresponding to an assumed-movement pattern is started in a procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • A procedure until the superimposing means 102 generates a synthetic image of the present invention and the displaying means 107 displays the synthetic image after an actual driving operation corresponding to an assumed-movement pattern is started will be described below by using a case of performing parallel parking to the left side as an example.
  • Before an actual driving operation is started, a viewpoint position of a virtual camera is fixed to a position just above a vehicle as described for the first embodiment by referring to FIG. 10, the current vehicle position 901 and the assumed-movement pattern 1403 using the position 901 as an operation start position is fixed on a screen as shown in FIG. 28(a), and circumferential-state images such as the obstacles 401, 402 or the like, are displayed while relatively moving on the screen in accordance with movement of the vehicle.
  • As shown in FIG. 28(b), when the current vehicle position 901 comes to a position corresponding to the target parking position 902, a driver starts a driving operation corresponding to parallel parking to the left side. When the driving operation is started, a gear signal shows a backward-movement state and a front-wheel steering angle according to a steering-wheel-turning-angle signal reaches a certain value or more. Therefore, the start detecting means 1301 judges that the driving operation (parking operation) corresponding to the parallel parking to the left side is started and communicates that the driving operation (parking operation) is started to the integrating means 1302 and viewpoint converting means 106.
  • The integrating means 1302 integrates input steering-wheel-rotation-angle signal and number-of-revolution-of-rear-wheel signal after start of the driving operation to compute the positional relation between the current vehicle position 1402 and the parking-operation start position 1401 as shown in FIG. 14(a).
  • In accordance with the computed positional relation, the space converting means 1303 moves the assumed-movement pattern 1403 corresponding to the parallel parking to the left side so that the operation start position (501 in FIG. 5) of the pattern 1403 coincides with the parking-operation start position 1401 as shown in FIG. 14(b). In other words, the space converting means 1303 spatially fixes the assumed-movement pattern 1403 to the position at start of the parking operation after start of the driving operation.
  • When the viewpoint converting means 106 receives that the driving operation (parking operation) is started, it fixes the then viewpoint position of the virtual camera to a space (ground). That is, after start of the driving operation, the circumferential-state images ( obstacles 401, 402 or the like) are fixed on the screen.
  • After start of the driving operation, the superimposing means 102 superimposes the assumed-movement pattern 1403 spatially fixed to the position at start of the parking operation and the current vehicle position 1402 on the circumferential-state images and synthesizes them to generate a synthetic image of the present invention as shown in FIG. 28(c). Because the viewpoint of the synthetic image is obtained by spatially fixing the viewpoint position of the virtual camera at start of the parking operation similarly to viewpoints of circumferential-state images. Therefore, the superimposing means 102 generates the synthetic image by computing the positional relation computed by the integrating means 1302 inversely to a viewpoint. That is, in this synthetic image (FIG. 28(c)), the circumferential-state images ( obstacles 401, 402 or the like) and the assumed-movement pattern 1403 are fixed on the screen and the current vehicle position 1402 is displayed while relatively moving on the screen in accordance with actual movement of the vehicle.
  • According to this embodiment, because a viewpoint after start of a parking operation is fixed in a space, a driver can confirm a vehicle moving state to a circumferential state of a parking space at a glance.
  • When a movement assumed in accordance with an assumed-movement pattern includes a wide range or is complex and a range larger than the field of view of a camera of the imaging section 101 occurs after a driving operation is started, it is also permitted to display the range by using the data stored in the space-data buffer 105.
  • Tenth Embodiment
  • Then, a tenth embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 29 is a block diagram showing a configuration of the driving-operation assist of the tenth embodiment of the present invention. The driving-operation assist of this embodiment is different from that of the first embodiment only in that two cases are assumed in which a vehicle moves backward when turning a steering wheel up to the maximum angle clockwise and counterclockwise to tire rotation as default driving patterns stored in the assumed-movement-pattern storing means 108 as shown in FIG. 30(b), two circumscribed area traces 604 in a space through which the whole of the vehicle passes in stead of the tire traces 603 caused by assumed movements of the two cases are synthesized by the superimposing means 102 at the same time as shown in FIG. 30(a), and the synthesized result is displayed on the displaying means 107.
  • Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first embodiment, a component provided with the same symbol as the case of the first embodiment has the same function as the case of the first embodiment unless otherwise explained. Moreover, it is assumed that each modified example described for the first embodiment can be applied to this embodiment by applying the same modification unless otherwise described.
  • According to this embodiment, two simplest cases are assumed in which a vehicle moves backward when turning a steering wheel up to the maximum angle clockwise and counterclockwise as shown in FIG. 30(b) as defaults of the assumed-movement storing pattern 108 [sic; assumed-movement pattern storing means]. Therefore, when the vehicle can be parked in accordance with the defaults of the assumed-movement storing pattern 108 [sic; assumed-movement pattern-storing means], advantages can be obtained that it becomes unnecessary to change and select a plurality of assumed-movement storing patterns and an operating load of a driver can be reduced.
  • Moreover, it becomes unnecessary to change and select two left and right assumed-movement storing patterns by simultaneously synthesizing circumscribed area traces 604 produced due to assumed movements of the above described two cases by the superimposing means 102 and by displaying the synthesized result on the displaying means 107, and consequently there can be obtained such an effect that an operating load of the driver can be reduced.
  • Furthermore, as shown in FIG. 30(a), the circumscribed area traces 60 in a space through which the whole of the vehicle passes instead of the tire traces 603 are synthesized by the superimposing means 102, and the synthesized result is displayed on the displaying means 107, and thereby an advantage can be obtained that a driver can easily and more accurately confirm whether a portion of a vehicle to be protruded beyond a tire trace such as the right front portion of the vehicle when parking the vehicle while turning left-backward contacts with an obstacle.
  • Eleventh Embodiment
  • Then, an eleventh embodiment of the present invention will be described below by referring to the accompanying drawings.
  • FIG. 31 is a block diagram showing a configuration of the driving-operation assist of the eleventh embodiment of the present invention. The driving-operation assist of this embodiment is also mainly used to assist driving operations for garaging and parallel parking. Therefore, in this embodiment, it is assumed that a component with no explanation is the same as the case of the first and fifth embodiments and a component provided with the same symbol as the case of the first and fifth embodiments has the same function as the case of the first and fifth embodiments unless otherwise explained. Moreover, it is assumed that each modified example described for the first and fifth embodiments can be applied to this embodiment by applying the same modification unless otherwise described.
  • The configuration of the driving-operation assist of this embodiment is different from those of the first and fifth embodiments in that the obstacle inputting means 3101 and assumed-pattern correcting means 3102 shown in FIG. 31 are included and the assumed-movement-pattern storing means 108 includes a movement pattern for changing backward and forward movements of a vehicle while the vehicle moves as shown in FIG. 32(a).
  • In this case, a steering-wheel angle corresponding to tire rotation is stored in the assumed-movement-pattern storing means 108 in FIG. 31 as time-series data for assumed-movement patterns as shown in FIG. 32(b). As shown in FIG. 32(b), tire rotation of 0 to 0.8 shows backward movement of a vehicle at which backward movement is changed to forward movement. In this case, the position of the vehicle is present at a backward- and forward-movement changing position 3201 shown in FIG. 32(a). Then, the vehicle moves forward to a tire angle of 0.8 to 0.6. When the vehicle reaches the forward- and backward-movement changing position 3202 shown in FIG. 32(a), forward movement is changed to backward movement again such as a tire angle of 0.6 to 1.4.
  • Thus, by including a movement pattern for changing backward and forward movements during movement, it is possible to control a position and direction of a vehicle even if there is only a small spatial margin for an obstacle as shown in FIG. 32(a).
  • In this embodiment, a procedure until a circumferential-state image of the present invention is generated in accordance with the video data imaged by the imaging section 101 is the same as that described for the first embodiment. A procedure until an assumed-movement pattern stored in the assumed-movement-pattern storing means 108 is displayed on a synthetic image of the present invention by adjusting an operation start position of the pattern to the current position of a vehicle in a procedure until the superimposing means 102 generates the synthetic image and the displaying means 107 displays the synthetic image is the same as that described for the first embodiment.
  • A procedure until a driver corrects an assumed-movement pattern and time-series data by using the trace correcting means 1801 and obstacle inputting means 3101 after the assumed-movement pattern is displayed on a synthetic image and the corrected assumed-movement pattern and time-series data are displayed on the synthetic image is described below by using a case of performing garaging to the right side as an example.
  • As shown in FIG. 33, a case is assumed in which a driver moves a vehicle to the current position 1901 where an operation end position of an assumed-movement pattern coincides with the target parking position 1902 in order to perform a garaging operation using the target parking position 1902 as an operation end position so that the vehicle does not contact with an obstacle (a 3204, b 3205, or c 3206) and thereby, it is found that the circumscribed area 604 of the assumed-movement pattern may contact with the obstacle a, b, or c.
  • When the assumed-movement-pattern storing means 108 stores an assumed-movement pattern for another garaging operation to the right side, it is permitted to select the assumed-movement pattern by pattern selecting means (not illustrated) and study if a parking operation can be smoothly performed. However, when the means 108 does not store the assumed-movement pattern or another assumed-movement pattern also interferes with an obstacle or the like, the driver corrects the assumed-movement pattern.
  • First, the driver assumes a pattern showing the vehicle present at the current position 3200 of the synthetic image (FIG. 33) shown on the displaying means 107 in FIG. 31 as the parking-operation start position 1901, assumes an area in the image in which the obstacle a, b, or c is present as an obstacle designation rectangle: a 3207 as shown in FIG. 34 by using the obstacle inputting means 3101 in FIG. 31 or as an obstacle designation area: 3210 (refer to FIG. 36) by using an obstacle designation circle: b 3208, and sets and inputs the obstacle designation rectangle or the obstacle designation area through numerical-value input, a pointer, or other means. Moreover, when it is necessary to correct the target parking position, the driver performs movement input through numerical-value input, a pointer, or other means.
  • When an obstacle area is input, the trace correcting means 1801 sets a contact-hazard area 3209 in an area having a circumference of 60 cm including the obstacle designation area: 3210 as shown in FIG. 35. Moreover, a contact-hazard evaluation function H3210 as shown in FIG. 36 is provided for the area. The function is configured by synthesizing three quadratic functions which sharply increase as approaching for 10 cm or less from the obstacle designation area: 3210, slowly decrease as separating up to 10 cm or more, and become 0 for 60 cm or more.
  • Moreover, as shown in FIG. 37, an evaluation point 3211 (xi,yi):(i=1-6) is set to six points around a certain area as shown in FIG. 37 and a trace evaluation point 3212 (xi,yi)n:(n=1-N) about N items (tire rotation tm and tire angle km):(m=1-N) in the table shown in FIG. 32(b) is computed.
  • As shown by the expression in FIG. 38, a trace-contact-hazard evaluation function H″ 3213 is obtained from the total of the contact-hazard evaluation function H3210 at the position of the trace evaluation point 3212.
  • The trace-contact-hazard evaluation function H″ 3213 becomes a function of N items (tire rotation tm and tire angle km) in the table in FIG. 32(b) as shown in FIG. 37. Therefore, by successively correcting the diagram (tire rotation tm and tire angle km) through the partial differential method, it is possible to obtain values (tire rotation tm and tire angle km) for minimizing the trace-contact-hazard evaluation function H″ 3213.
  • Thereby, as shown in FIG. 39, it is possible to correct an assumed-movement pattern from (tire rotation tm and tire angle km) first shown in FIG. 32(b) to (tire rotation tm and tire angle km) for minimizing the trace-contact-hazard evaluation function H″ 3213.
  • When even one point of 10 cm or less is found in the obstacle designation area: 3210 from the contact-hazard evaluation function H about each point of trace evaluation points 3212 (xi,yi)n in (tire rotation tm and tire angle km) for minimizing the trace-contact-hazard evaluation function H″ 3213, a warning Careful” is issued to a driver. When even one point of 0 cm or less is found in the obstacle designation area: 3210, a warning Parking not possible” is issued to a driver.
  • Because the corrected assumed-movement pattern is a movement pattern for minimizing the trace-contact-hazard evaluation function H″ 3213, the assumed-movement pattern generates a trace having a larger margin from an obstacle as shown in FIG. 40 and thereby, a vehicle can be parked more safely.
  • As shown in FIG. 41, the superimposing means 102 generates a synthetic-image by adjusting the operation start position 1901 of a corrected assumed-movement pattern 3214 to a current vehicle position 3200 and the displaying means 107 displays the synthetic image.
  • Therefore, when a driver starts a driving operation (parking operation) of a vehicle in accordance with the new corrected assumed-movement pattern 3214, the driver can park the vehicle at the target parking position 1902 in accordance with a movement pattern having a larger margin from an obstacle.
  • It is permitted to store generated new assumed-movement pattern and time-series data in the assumed-movement-pattern storing means 108 by updating an original assumed-movement pattern or to additionally store the new assumed-movement pattern in the means 108 without changing the original assumed-movement pattern. Moreover, it is permitted to temporarily use them without storing them. Furthermore, it is permitted that a driver selects renewed storage, additional storage, or non-storage each time.
  • Furthermore, this embodiment is described above by assuming that an assumed-movement pattern to be renewedly-stored or additionally-stored in the assumed-movement-pattern storing means 108 can be automatically obtained in accordance with a position of a vehicle at start or end of a movement input from a driver. However, it is also permitted to perform an actual driving operation and thereby, sample time-series data for the then steering angle of a steering wheel and the number of revolutions of wheels, generate an assumed-movement pattern in accordance with the time-series data, and store the pattern.
  • Twelfth Embodiment
  • Then, a twelfth embodiment of the present invention will be described below by referring to FIGS. 42 to 46.
  • In the eleventh embodiment, it is assumed that an assumed-movement pattern shown in FIG. 32(a) is corrected in accordance with an obstacle area input from a driver. However, it is also permitted to correct two assumed-movement patterns or more and select a preferable one. FIG. 42 is different from FIG. 31 in that movement-pattern selection trace correcting means 3301 is used instead of the trace correcting means 3101.
  • First, operations of the twelfth embodiment are described. As shown in FIG. 43, a driver designates and inputs the target parking position 1902 on an image displayed on a display unit by assuming a current vehicle position as a parking-operation start position. The movement-pattern selection trace correcting means 3301 extracts two assumed-movement patterns for parking a vehicle right backward as shown in FIG. 44 in accordance with a rough positional relation of the target parking position 1902 to the parking-operation start position out of a plurality of assumed-movement patterns stored in the assumed-movement-pattern storing means 108.
  • Moreover, as shown in FIGS. 45 and 46, when the driver inputs an obstacle area on the image displayed on the display unit, the two assumed-movement patterns for respectively minimizing the trace-contact-hazard evaluation function H″ 3213 are corrected similarly to the case of the eleventh embodiment. In this case, by comparing the two minimized trace-contact-hazard evaluation functions H″ 3213 with each other and selecting a smaller one of the two functions H″ 3213, it is possible to select a safer assumed-movement pattern. Moreover, when the two trace-contact-hazard evaluation functions H″ 3213 have the same value, a simpler assumed-movement pattern is selected by previously raising the priority of a function H″ 3213 allowing a simpler driving operation.
  • Therefore, when a spatial margin from an obstacle is small, a safer movement pattern having a cutback is automatically selected as shown in FIG. 45. However, when a spatial margin from an obstacle is large, a simpler movement pattern having no cutback is automatically selected as shown in FIG. 446 [sic; FIG. 46].
  • According to the above configuration, a driver can automatically select an optimum assumed-movement pattern only by inputting a target parking position and an obstacle area and thereby realize optimum parking through a safer and simpler driving operation.
  • According to this embodiment, it is possible to realize an extensible driving-operation assist compared to the driving-operation assist of the first embodiment.
  • The above first to twelfth embodiments are described by assuming that circumferential-state imaging means of the present invention mainly generates an image viewed from a viewpoint of a virtual camera by using a plurality of vehicle-mounted cameras. However, it is also permitted to use one camera set to the ceiling of a roof-provided parking lot. In short, it is possible to use any type of means as circumferential-state imaging means of the present invention as long as the means generates a circumferential-state image by imaging a circumferential state of a vehicle with a camera and/or stores the generated circumferential-state image.
  • Moreover, the above first to twelfth embodiments are described by assuming that driving-operation assist of the embodiments are respectively provided with assumed-movement-pattern storing means of the present invention. However, it is also permitted to use any type of means as assumed-movement-pattern storing means of the present invention as long as an assumed-movement pattern can be input to the means each time. In short, it is permitted to use any type of driving-operation assist as driving-operation assist of the present invention as long as the driving-operation assist includes circumferential-state imaging means for imaging a circumferential state of a vehicle with a camera and thereby generating a circumferential-state image and/or storing the circumferential-state image, synthetic-image generating means for superimposing on the circumferential-state image, an assumed-movement pattern which is video data showing movement of the vehicle in the case of performing a predetermined series of driving operations for the vehicle and generating a synthetic image, and displaying means for displaying the synthetic image. Thereby, because at least a circumferential-state image and an assumed-movement pattern are superimposed on each other and displayed on a synthetic screen, a driver can confirm the relation between the circumferential-state image and the assumed-movement pattern at a glance.
  • INDUSTRIAL APPLICABILITY
  • As apparent from the above description, the present invention provides a driving-operation assist by which a driver can directly confirm the relation between vehicle movement according to a predetermined series of driving operations and a circumferential state by displaying the vehicle movement when the driver performs the predetermined series of driving operations together with the circumferential sate and a driver's load can be reduced.
  • That is, a driver can confirm a position for starting a driving operation such as garaging or parallel parking, a place for finally stopping a vehicle, and a positional relation with an obstacle such as other vehicle by the display image at a glance by using a driving-operation assist of the present invention. Therefore, it is expected that a driver's operation load is reduced and safety is improved. Moreover, by introducing an automatic-driving technique, it is possible to automatically perform every operation including garaging only by driver's moving a vehicle up to a parking-operation start position while viewing the movement trace data.
  • Furthermore, the present invention provides a recording medium for storing a program for making a computer execute all or some of functions of each means of a driving-operation assist of the present invention.

Claims (13)

1. A vehicle operation assist for a vehicle mounted with a camera, comprising:
a circumferential state imaging unit operable to generate a circumferential state image using an output image from the camera;
a synthetic-image generating unit operable to generate a synthetic image by using the circumferential-state image, said synthetic image including an assumed-movement pattern showing movement of the vehicle in case of performing a predetermined driving operation, said assumed-movement pattern including (i) a trace between a start position and an end position of the vehicle, and (ii) a synthetic image of the vehicle at the end position in case of performing the predetermined driving operation; and
a display unit operable to display the synthetic image.
2. The vehicle operation assist according to claim 1, wherein
the assumed-movement pattern further includes a synthetic image of the vehicle at the start position in case of performing the predetermined driving operation.
3. The vehicle-operation assist according to claim 2, wherein
the synthetic image further includes a synthetic image of the vehicle at the current position.
4. The vehicle-operation assist according to claim 1, wherein
the synthetic image of the vehicle shows a circumscribed area of the vehicle.
5. The vehicle operation assist according to claim 1, wherein
the synthetic image of the vehicle is an actual-vehicle image.
6. The vehicle operation assist according to claim 1, wherein
the circumferential-state image is a virtual viewpoint image in which the vehicle and its surroundings are viewed from a virtual viewpoint, said virtual viewpoint being different from a viewpoint of the camera.
7. The vehicle-operation assist according to claim 8, wherein
the predetermined viewpoint is a point fixed to the three-dimensional space or the vehicle, and the circumferential-state imaging unit changes a predetermined viewpoint automatically or through an input from a driver.
8. The vehicle-operation assist according to claim 1, wherein
the assumed-movement pattern includes virtual poles arranged on the outer edge of the vehicle movement area.
9. The vehicle-operation assist according to claim 3, wherein
the synthetic-image generating unit superimposes the synthetic image of the vehicle at the start position in case of performing the predetermined driving operation on a position same as the current-position of the vehicle.
10. The vehicle-operation assist according to claim 1, further comprising
a pattern selecting unit selects one of the assumed-movement patterns through an input from a driver.
11. The vehicle-operation assist according to claim 1, wherein
the trace included in assumed-movement pattern, showing a circumscribed area where the vehicle passes through in case of performing the predetermined driving operation.
12. The vehicle-operation assist according to claim 1, wherein
the synthetic-image generating unit generates a synthetic image by superimposing two or more of the assumed-movement patterns on the circumferential-state image.
13. The vehicle-operation assist according to claim 1, wherein
the assumed-movement pattern includes a change from backward movement to forward movement or from forward movement to backward movement in the predetermined driving operation.
US11/846,027 1998-10-08 2007-08-28 Driving-operation assist and recording medium Abandoned US20080033606A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US11/846,027 US20080033606A1 (en) 1998-10-08 2007-08-28 Driving-operation assist and recording medium
US13/848,800 US9272731B2 (en) 1998-10-08 2013-03-22 Driving-operation assist and recording medium

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
JP28624698 1998-10-08
JPJP10-286246 1998-10-08
JPJP10-349107 1998-12-08
JP34910798 1998-12-08
PCT/JP1999/005509 WO2000020257A1 (en) 1998-10-08 1999-10-06 Driving assisting device and recording medium
US58100400A 2000-07-17 2000-07-17
US11/846,027 US20080033606A1 (en) 1998-10-08 2007-08-28 Driving-operation assist and recording medium

Related Parent Applications (3)

Application Number Title Priority Date Filing Date
US09/581,004 Continuation US7277123B1 (en) 1998-10-08 1999-10-06 Driving-operation assist and recording medium
PCT/JP1999/005509 Continuation WO2000020257A1 (en) 1998-10-08 1999-10-06 Driving assisting device and recording medium
US58100400A Continuation 1998-10-08 2000-07-17

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US13/848,800 Continuation US9272731B2 (en) 1998-10-08 2013-03-22 Driving-operation assist and recording medium

Publications (1)

Publication Number Publication Date
US20080033606A1 true US20080033606A1 (en) 2008-02-07

Family

ID=26556225

Family Applications (5)

Application Number Title Priority Date Filing Date
US09/581,004 Expired - Lifetime US7277123B1 (en) 1998-10-08 1999-10-06 Driving-operation assist and recording medium
US11/846,085 Expired - Fee Related US8111287B2 (en) 1998-10-08 2007-08-28 Driving-operation assist and recording medium
US11/846,027 Abandoned US20080033606A1 (en) 1998-10-08 2007-08-28 Driving-operation assist and recording medium
US11/846,048 Expired - Fee Related US8077202B2 (en) 1998-10-08 2007-08-28 Driving-operation assist and recording medium
US13/848,800 Expired - Lifetime US9272731B2 (en) 1998-10-08 2013-03-22 Driving-operation assist and recording medium

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US09/581,004 Expired - Lifetime US7277123B1 (en) 1998-10-08 1999-10-06 Driving-operation assist and recording medium
US11/846,085 Expired - Fee Related US8111287B2 (en) 1998-10-08 2007-08-28 Driving-operation assist and recording medium

Family Applications After (2)

Application Number Title Priority Date Filing Date
US11/846,048 Expired - Fee Related US8077202B2 (en) 1998-10-08 2007-08-28 Driving-operation assist and recording medium
US13/848,800 Expired - Lifetime US9272731B2 (en) 1998-10-08 2013-03-22 Driving-operation assist and recording medium

Country Status (5)

Country Link
US (5) US7277123B1 (en)
EP (1) EP1038734B1 (en)
KR (1) KR100568396B1 (en)
CN (1) CN1132750C (en)
WO (1) WO2000020257A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050236201A1 (en) * 2002-12-05 2005-10-27 Bayerische Motoren Werke Aktiengesellschaft Method for steering a vehicle that is to be backed into a parking space
US20110082613A1 (en) * 2007-02-28 2011-04-07 Moritz Oetiker Semiautomatic parking machine
US20110095910A1 (en) * 2008-06-10 2011-04-28 Nissan Motor Co., Ltd. Parking assistance system and parking assistance method
US20120249342A1 (en) * 2011-03-31 2012-10-04 Koehrsen Craig L Machine display system
US20130144492A1 (en) * 2010-06-09 2013-06-06 Teruhisa Takano Parking mode selection apparatus and method
US20130231863A1 (en) * 1998-10-08 2013-09-05 Panasonic Corporation Driving-operation assist and recording medium

Families Citing this family (75)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3297040B1 (en) * 2001-04-24 2002-07-02 松下電器産業株式会社 Image composing and displaying method of vehicle-mounted camera and apparatus therefor
JP3947375B2 (en) 2001-08-24 2007-07-18 アイシン精機株式会社 Parking assistance device
JP4316960B2 (en) * 2003-08-22 2009-08-19 株式会社半導体エネルギー研究所 apparatus
DE102004009924A1 (en) * 2004-02-23 2005-09-01 Valeo Schalter Und Sensoren Gmbh Picture processing method for motor vehicle, involves processing picture based on determination of real position of obstacle in vicinity of vehicle and determination of position from picture provided by camera device
US20060178787A1 (en) * 2005-02-09 2006-08-10 Mccall Clark E Rear obstacle avoidance system for vehicle
WO2006087993A1 (en) 2005-02-15 2006-08-24 Matsushita Electric Industrial Co., Ltd. Periphery supervising device, and periphery supervising method
DE102005018408A1 (en) * 2005-04-20 2006-10-26 Valeo Schalter Und Sensoren Gmbh Method and device for evaluating distance measuring data of a distance measuring system of a motor vehicle
EP1916846B1 (en) * 2005-08-02 2016-09-14 Nissan Motor Company Limited Device and method for monitoring vehicle surroundings
FR2891934B1 (en) * 2005-10-12 2008-01-18 Valeo Electronique Sys Liaison DEVICE FOR PROCESSING VIDEO DATA FOR A MOTOR VEHICLE
JP2007176324A (en) * 2005-12-28 2007-07-12 Aisin Seiki Co Ltd Parking assist device
JP4812510B2 (en) * 2006-05-17 2011-11-09 アルパイン株式会社 Vehicle peripheral image generation apparatus and photometric adjustment method for imaging apparatus
JP4818816B2 (en) 2006-06-05 2011-11-16 富士通株式会社 Parking assistance program and parking assistance device
US20090128630A1 (en) * 2006-07-06 2009-05-21 Nissan Motor Co., Ltd. Vehicle image display system and image display method
JP4595976B2 (en) * 2007-08-28 2010-12-08 株式会社デンソー Video processing apparatus and camera
JP5053776B2 (en) * 2007-09-14 2012-10-17 株式会社デンソー Vehicular visibility support system, in-vehicle device, and information distribution device
KR100871044B1 (en) * 2007-09-28 2008-11-27 한국오므론전장주식회사 Automatic osd line generating method for parking assist system
JP2009101718A (en) * 2007-10-19 2009-05-14 Toyota Industries Corp Image display device and image display method
JP5090126B2 (en) * 2007-10-23 2012-12-05 アルパイン株式会社 In-vehicle imaging device
JP5132249B2 (en) * 2007-10-23 2013-01-30 アルパイン株式会社 In-vehicle imaging device
JP5057936B2 (en) * 2007-11-09 2012-10-24 アルパイン株式会社 Bird's-eye image generation apparatus and method
JP4900232B2 (en) 2007-12-26 2012-03-21 日産自動車株式会社 Vehicle parking assist device and video display method
JP4902575B2 (en) * 2008-02-27 2012-03-21 日立オートモティブシステムズ株式会社 Road sign recognition device and road sign recognition method
DE102008027779A1 (en) * 2008-06-11 2009-12-17 Valeo Schalter Und Sensoren Gmbh Method for assisting a driver of a vehicle when parking in a parking space
EP2230629A3 (en) * 2008-07-16 2012-11-21 Verint Systems Inc. A system and method for capturing, storing, analyzing and displaying data relating to the movements of objects
JP4661917B2 (en) * 2008-07-25 2011-03-30 日産自動車株式会社 Parking assistance device and parking assistance method
JP4840427B2 (en) * 2008-07-29 2011-12-21 日産自動車株式会社 Vehicle control device
JP2010128794A (en) * 2008-11-27 2010-06-10 Aisin Seiki Co Ltd Surrounding recognition assisting device for vehicle
DE102009024083A1 (en) * 2009-06-05 2010-12-09 Valeo Schalter Und Sensoren Gmbh Method for carrying out an at least semi-autonomous parking operation of a vehicle and parking assistance system for a vehicle
US8174375B2 (en) * 2009-06-30 2012-05-08 The Hong Kong Polytechnic University Detection system for assisting a driver when driving a vehicle using a plurality of image capturing devices
CN102055956B (en) 2009-11-02 2017-05-10 通用汽车环球科技运作公司 Vehicle-mounted three-dimensional video system and method for monitoring vehicle surrounding environment by using same
US9204108B2 (en) * 2009-12-07 2015-12-01 Clarion Co., Ltd. Vehicle periphery monitoring system
DE102010023162A1 (en) * 2010-06-09 2011-12-15 Valeo Schalter Und Sensoren Gmbh A method for assisting a driver of a motor vehicle when parking in a parking space, Fahrerassistzeinrichtung and motor vehicle
WO2011158609A1 (en) * 2010-06-15 2011-12-22 アイシン精機株式会社 Drive support device
JP5444139B2 (en) * 2010-06-29 2014-03-19 クラリオン株式会社 Image calibration method and apparatus
DE102010034139A1 (en) * 2010-08-12 2012-02-16 Valeo Schalter Und Sensoren Gmbh Method for supporting a parking process of a motor vehicle, driver assistance system and motor vehicle
DE102010034127A1 (en) 2010-08-12 2012-02-16 Valeo Schalter Und Sensoren Gmbh Method for displaying images on a display device in a motor vehicle, driver assistance system and motor vehicle
DE102010034142A1 (en) 2010-08-12 2012-02-16 Valeo Schalter Und Sensoren Gmbh A method of assisting a driver in driving a motor vehicle and driver assistance system
JP5454934B2 (en) * 2010-09-21 2014-03-26 アイシン精機株式会社 Driving assistance device
DE102010048185B4 (en) * 2010-10-13 2021-10-28 Wirtgen Gmbh Self-propelled construction machine
CN102463989A (en) * 2010-11-18 2012-05-23 江彦宏 Assistant driving system based on video radar
TWI421624B (en) * 2011-04-01 2014-01-01 Ind Tech Res Inst Adaptive surrounding view monitoring apparatus and method thereof
JP2012253543A (en) * 2011-06-02 2012-12-20 Seiko Epson Corp Display device, control method of display device, and program
DE102011080930A1 (en) * 2011-08-12 2013-02-14 Robert Bosch Gmbh Method and device for assisting a driver of a motor vehicle in a driving maneuver
KR20130021988A (en) * 2011-08-24 2013-03-06 현대모비스 주식회사 Device for overlaying image obtained by camera for vehicle and method thereof
EP2581268B2 (en) * 2011-10-13 2019-09-11 Harman Becker Automotive Systems GmbH Method of controlling an optical output device for displaying a vehicle surround view and vehicle surround view system
CN103987582B (en) * 2011-12-15 2017-02-22 松下知识产权经营株式会社 Drive assistance device
JP5941292B2 (en) * 2012-02-10 2016-06-29 矢崎総業株式会社 Vehicle display device
JP5888087B2 (en) * 2012-04-25 2016-03-16 ソニー株式会社 Driving support image generation device, driving support image generation method, in-vehicle camera, and device operation support image generation device
DE102012008858A1 (en) * 2012-04-28 2012-11-08 Daimler Ag Method for performing autonomous parking process of motor vehicle e.g. passenger car, involves storing target position and/or last driven trajectory of vehicle in suitable device prior to start of autonomous vehicle parking operation
JP5814187B2 (en) * 2012-06-07 2015-11-17 日立建機株式会社 Display device for self-propelled industrial machine
US20140057237A1 (en) * 2012-08-27 2014-02-27 Stephen Chen Method for parking a vehicle by using a parking assistant system
JP6143469B2 (en) * 2013-01-17 2017-06-07 キヤノン株式会社 Information processing apparatus, information processing method, and program
US9167214B2 (en) * 2013-01-18 2015-10-20 Caterpillar Inc. Image processing system using unified images
JP6001792B2 (en) * 2013-09-09 2016-10-05 三菱電機株式会社 Driving support device and driving support method
TWI552907B (en) * 2013-10-30 2016-10-11 緯創資通股份有限公司 Auxiliary system and method for driving safety
US20160110650A1 (en) * 2013-11-29 2016-04-21 Ims Solutions, Inc. Advanced context-based driver scoring
US20150156391A1 (en) * 2013-12-04 2015-06-04 Chung-Shan Institute Of Science And Technology, Armaments Bureau, M.N.D Vehicle image correction system and method thereof
US9598012B2 (en) * 2014-03-11 2017-03-21 Toyota Motor Engineering & Manufacturing North America, Inc. Surroundings monitoring system for a vehicle
JP6274936B2 (en) * 2014-03-25 2018-02-07 ダイハツ工業株式会社 Driving assistance device
US9592826B2 (en) * 2015-02-13 2017-03-14 Ford Global Technologies, Llc System and method for parallel parking a vehicle
KR101860610B1 (en) * 2015-08-20 2018-07-02 엘지전자 주식회사 Display Apparatus and Vehicle Having The Same
CN105824592A (en) * 2016-03-07 2016-08-03 乐卡汽车智能科技(北京)有限公司 Reversing trajectory display method and device
JP6723820B2 (en) * 2016-05-18 2020-07-15 株式会社デンソーテン Image generation apparatus, image display system, and image display method
CN105799596A (en) * 2016-05-20 2016-07-27 广州市晶华精密光学股份有限公司 Intelligent automobile rear-view system and image display method
CN105933680A (en) * 2016-07-07 2016-09-07 广州勘帝德电子科技有限公司 CAN_BUS free intelligent vehicle dynamic track line rearview image system
CN107776489B (en) * 2016-08-26 2020-07-10 比亚迪股份有限公司 Vehicle and display method and display system of panoramic image of vehicle
KR101949438B1 (en) * 2016-10-05 2019-02-19 엘지전자 주식회사 Display apparatus for vehicle and vehicle having the same
US10162360B2 (en) * 2016-12-01 2018-12-25 GM Global Technology Operations LLC Vehicle environment imaging systems and methods
DE102017218921A1 (en) 2017-10-24 2019-04-25 Bayerische Motoren Werke Aktiengesellschaft A method, apparatus, computer program and computer program product for operating a display unit of a vehicle
JP6984373B2 (en) * 2017-12-07 2021-12-17 トヨタ自動車株式会社 Parking support device
CN108332716A (en) * 2018-02-07 2018-07-27 徐州艾特卡电子科技有限公司 A kind of autonomous driving vehicle context aware systems
CN110211256A (en) * 2018-02-28 2019-09-06 上海博泰悦臻电子设备制造有限公司 Appearing method is answered in aobvious device and vehicle driving fact again for vehicle and its vehicle driving fact
DE102018208513A1 (en) * 2018-05-29 2019-12-05 Continental Automotive Gmbh Camera monitor system for a motor vehicle and use of a mirror replacement system for a motor vehicle
US10497232B1 (en) * 2019-03-01 2019-12-03 Motorola Solutions, Inc. System and method for dynamic vehicular threat detection perimeter modification for an exited vehicular occupant
CN113496626B (en) * 2020-03-19 2023-06-02 广州汽车集团股份有限公司 Vehicle collision early warning method and device and automobile

Citations (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4214266A (en) * 1978-06-19 1980-07-22 Myers Charles H Rear viewing system for vehicles
US4926346A (en) * 1985-12-27 1990-05-15 Aisin-Warner Kabushiki Kaisha Road image input system for vehicle control
US4931930A (en) * 1988-04-19 1990-06-05 Industrial Technology Research Institute Automatic parking device for automobile
US5172315A (en) * 1988-08-10 1992-12-15 Honda Giken Kogyo Kabushiki Kaisha Automatic travelling apparatus and method
US5410346A (en) * 1992-03-23 1995-04-25 Fuji Jukogyo Kabushiki Kaisha System for monitoring condition outside vehicle using imaged picture by a plurality of television cameras
US5424952A (en) * 1993-03-26 1995-06-13 Mitsubishi Denki Kabushiki Kaisha Vehicle-surroundings monitoring apparatus
US5517412A (en) * 1993-09-17 1996-05-14 Honda Giken Kogyo Kabushiki Kaisha Self-navigating vehicle equipped with lane boundary recognition system
US5530420A (en) * 1993-12-27 1996-06-25 Fuji Jukogyo Kabushiki Kaisha Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof
US5535144A (en) * 1993-03-24 1996-07-09 Fuji Jukogyo Kabushiki Kaisha Distance detection method and system using a stereoscopical imaging apparatus
US5587938A (en) * 1993-09-29 1996-12-24 Robert Bosch Gmbh Method and device for maneuvering a motor vehicle out of a parking space
US5638116A (en) * 1993-09-08 1997-06-10 Sumitomo Electric Industries, Ltd. Object recognition apparatus and method
US5647019A (en) * 1992-05-29 1997-07-08 Fuji Electric Co., Ltd. Method of identifying a position of object in camera image
US5692061A (en) * 1994-02-23 1997-11-25 Matsushita Electric Works, Ltd. Method of utilizing a two-dimensional image for detecting the position, posture, and shape of a three-dimensional objective
US5757287A (en) * 1992-04-24 1998-05-26 Hitachi, Ltd. Object recognition system and abnormality detection system using image processing
US5796991A (en) * 1994-05-16 1998-08-18 Fujitsu Limited Image synthesis and display apparatus and simulation system using same
US5883739A (en) * 1993-10-04 1999-03-16 Honda Giken Kogyo Kabushiki Kaisha Information display device for vehicle
US5915033A (en) * 1996-09-26 1999-06-22 Fuji Electric Co., Ltd. Method for correcting detected distance values
US5929784A (en) * 1994-02-17 1999-07-27 Fuji Electric Co., Ltd. Device for determining distance between vehicles
US5949331A (en) * 1993-02-26 1999-09-07 Donnelly Corporation Display enhancements for vehicle vision system
US5969969A (en) * 1992-09-30 1999-10-19 Hitachi, Ltd. Vehicle driving support system which is responsive to environmental conditions
US5994701A (en) * 1996-10-15 1999-11-30 Nippon Avonics Co., Ltd. Infrared sensor device with temperature correction function
US6119068A (en) * 1996-12-27 2000-09-12 Kannonji; Michihiro Rear-end collision alarming device and method linked to speed control device of a vehicle
US6169572B1 (en) * 1996-06-06 2001-01-02 Fuji Jukogyo Kabushiki Kaisha Object recognizing apparatus for vehicle and the method thereof
US6192146B1 (en) * 1983-03-08 2001-02-20 Canon Kabushiki Kaisha Image processing system
US6192145B1 (en) * 1996-02-12 2001-02-20 Sarnoff Corporation Method and apparatus for three-dimensional scene processing using parallax geometry of pairs of points
US6396535B1 (en) * 1999-02-16 2002-05-28 Mitsubishi Electric Research Laboratories, Inc. Situation awareness system
US6476855B1 (en) * 1998-05-25 2002-11-05 Nissan Motor Co., Ltd. Surrounding monitor apparatus for a vehicle
US6583403B1 (en) * 1999-10-26 2003-06-24 Honda Giken Kogyo Kabushiki Kaisha Object detecting device, and travel safety system for vehicle
US6950120B1 (en) * 1997-12-03 2005-09-27 Canon Kabushiki Kaisha Camera layout for acquiring images used in panoramic synthesis
US7277123B1 (en) * 1998-10-08 2007-10-02 Matsushita Electric Industrial Co., Ltd. Driving-operation assist and recording medium

Family Cites Families (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5257867A (en) 1975-11-06 1977-05-12 Nippon Soken Inspecting and indicating device
JPS61113532A (en) 1985-09-05 1986-05-31 Nippon Denso Co Ltd Maintenance indicating device
JPS6414700A (en) 1987-07-08 1989-01-18 Aisin Aw Co Device for displaying prospective track of vehicle
JPH0441117Y2 (en) 1987-07-18 1992-09-28
JP2792566B2 (en) 1989-05-24 1998-09-03 マツダ株式会社 Travel control device for mobile vehicles
JPH0399952A (en) 1989-09-12 1991-04-25 Nissan Motor Co Ltd Surrounding situation monitor for vehicle
JPH03166534A (en) 1989-11-25 1991-07-18 Seikosha Co Ltd Range-finding device for camera
JPH05310078A (en) 1992-05-07 1993-11-22 Clarion Co Ltd Vehicle safety checking device and camera used therein
JP2943552B2 (en) 1993-02-12 1999-08-30 トヨタ自動車株式会社 Parking assistance device
JP3381351B2 (en) 1993-12-24 2003-02-24 日産自動車株式会社 Ambient situation display device for vehicles
JP3475507B2 (en) 1994-08-08 2003-12-08 日産自動車株式会社 Ambient monitoring device for vehicles
JPH0896118A (en) 1994-09-28 1996-04-12 Nissan Motor Co Ltd Circumferential state display device for vehicle
JP3478432B2 (en) 1995-03-02 2003-12-15 矢崎総業株式会社 Vehicle periphery monitoring device
JP3503840B2 (en) 1995-04-06 2004-03-08 矢崎総業株式会社 Vehicle periphery monitoring device
JPH09305796A (en) 1996-05-16 1997-11-28 Canon Inc Image information processor
JP3328478B2 (en) 1995-10-18 2002-09-24 日本電信電話株式会社 Camera system
JP3293441B2 (en) 1996-01-09 2002-06-17 トヨタ自動車株式会社 Imaging device
DE19611718A1 (en) * 1996-03-25 1997-10-02 Trw Repa Gmbh Method for controlling activation of a vehicle occupant restraint system, control system and vehicle occupant restraint system
JP3600378B2 (en) 1996-07-24 2004-12-15 本田技研工業株式会社 Vehicle external recognition device
JP3625622B2 (en) 1996-08-30 2005-03-02 三洋電機株式会社 Stereo model creation device, stereo model creation method, and medium on which a stereo model creation program is recorded
JPH10164566A (en) 1996-11-28 1998-06-19 Aiphone Co Ltd Multi-ceiling camera device
JPH10244891A (en) * 1997-03-07 1998-09-14 Nissan Motor Co Ltd Parking auxiliary device
JPH10257482A (en) * 1997-03-13 1998-09-25 Nissan Motor Co Ltd Vehicle surrounding condition display device
JP3284917B2 (en) * 1997-03-17 2002-05-27 三菱自動車工業株式会社 Perimeter recognition device for vehicles
JPH10264841A (en) 1997-03-25 1998-10-06 Nissan Motor Co Ltd Parking guiding device
EP2410742A1 (en) * 1999-04-16 2012-01-25 Panasonic Corporation Image processing apparatus and monitoring system
GB2364192A (en) * 2000-06-26 2002-01-16 Inview Systems Ltd Creation of a panoramic rear-view image for display in a vehicle
EP1916846B1 (en) * 2005-08-02 2016-09-14 Nissan Motor Company Limited Device and method for monitoring vehicle surroundings
JP4812510B2 (en) * 2006-05-17 2011-11-09 アルパイン株式会社 Vehicle peripheral image generation apparatus and photometric adjustment method for imaging apparatus
JP5053776B2 (en) * 2007-09-14 2012-10-17 株式会社デンソー Vehicular visibility support system, in-vehicle device, and information distribution device
JP5090126B2 (en) * 2007-10-23 2012-12-05 アルパイン株式会社 In-vehicle imaging device
JP5132249B2 (en) * 2007-10-23 2013-01-30 アルパイン株式会社 In-vehicle imaging device
JP4900326B2 (en) * 2008-06-10 2012-03-21 日産自動車株式会社 Parking assistance device and parking assistance method
JP4840427B2 (en) * 2008-07-29 2011-12-21 日産自動車株式会社 Vehicle control device
US8174375B2 (en) * 2009-06-30 2012-05-08 The Hong Kong Polytechnic University Detection system for assisting a driver when driving a vehicle using a plurality of image capturing devices

Patent Citations (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4214266A (en) * 1978-06-19 1980-07-22 Myers Charles H Rear viewing system for vehicles
US6192146B1 (en) * 1983-03-08 2001-02-20 Canon Kabushiki Kaisha Image processing system
US4926346A (en) * 1985-12-27 1990-05-15 Aisin-Warner Kabushiki Kaisha Road image input system for vehicle control
US4931930A (en) * 1988-04-19 1990-06-05 Industrial Technology Research Institute Automatic parking device for automobile
US5172315A (en) * 1988-08-10 1992-12-15 Honda Giken Kogyo Kabushiki Kaisha Automatic travelling apparatus and method
US5410346A (en) * 1992-03-23 1995-04-25 Fuji Jukogyo Kabushiki Kaisha System for monitoring condition outside vehicle using imaged picture by a plurality of television cameras
US5757287A (en) * 1992-04-24 1998-05-26 Hitachi, Ltd. Object recognition system and abnormality detection system using image processing
US5647019A (en) * 1992-05-29 1997-07-08 Fuji Electric Co., Ltd. Method of identifying a position of object in camera image
US5969969A (en) * 1992-09-30 1999-10-19 Hitachi, Ltd. Vehicle driving support system which is responsive to environmental conditions
US5949331A (en) * 1993-02-26 1999-09-07 Donnelly Corporation Display enhancements for vehicle vision system
US5535144A (en) * 1993-03-24 1996-07-09 Fuji Jukogyo Kabushiki Kaisha Distance detection method and system using a stereoscopical imaging apparatus
US5424952A (en) * 1993-03-26 1995-06-13 Mitsubishi Denki Kabushiki Kaisha Vehicle-surroundings monitoring apparatus
US5638116A (en) * 1993-09-08 1997-06-10 Sumitomo Electric Industries, Ltd. Object recognition apparatus and method
US5517412A (en) * 1993-09-17 1996-05-14 Honda Giken Kogyo Kabushiki Kaisha Self-navigating vehicle equipped with lane boundary recognition system
US5587938A (en) * 1993-09-29 1996-12-24 Robert Bosch Gmbh Method and device for maneuvering a motor vehicle out of a parking space
US5883739A (en) * 1993-10-04 1999-03-16 Honda Giken Kogyo Kabushiki Kaisha Information display device for vehicle
US5530420A (en) * 1993-12-27 1996-06-25 Fuji Jukogyo Kabushiki Kaisha Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof
US5929784A (en) * 1994-02-17 1999-07-27 Fuji Electric Co., Ltd. Device for determining distance between vehicles
US5692061A (en) * 1994-02-23 1997-11-25 Matsushita Electric Works, Ltd. Method of utilizing a two-dimensional image for detecting the position, posture, and shape of a three-dimensional objective
US5796991A (en) * 1994-05-16 1998-08-18 Fujitsu Limited Image synthesis and display apparatus and simulation system using same
US6192145B1 (en) * 1996-02-12 2001-02-20 Sarnoff Corporation Method and apparatus for three-dimensional scene processing using parallax geometry of pairs of points
US6169572B1 (en) * 1996-06-06 2001-01-02 Fuji Jukogyo Kabushiki Kaisha Object recognizing apparatus for vehicle and the method thereof
US5915033A (en) * 1996-09-26 1999-06-22 Fuji Electric Co., Ltd. Method for correcting detected distance values
US5994701A (en) * 1996-10-15 1999-11-30 Nippon Avonics Co., Ltd. Infrared sensor device with temperature correction function
US6119068A (en) * 1996-12-27 2000-09-12 Kannonji; Michihiro Rear-end collision alarming device and method linked to speed control device of a vehicle
US6950120B1 (en) * 1997-12-03 2005-09-27 Canon Kabushiki Kaisha Camera layout for acquiring images used in panoramic synthesis
US6476855B1 (en) * 1998-05-25 2002-11-05 Nissan Motor Co., Ltd. Surrounding monitor apparatus for a vehicle
US7277123B1 (en) * 1998-10-08 2007-10-02 Matsushita Electric Industrial Co., Ltd. Driving-operation assist and recording medium
US6396535B1 (en) * 1999-02-16 2002-05-28 Mitsubishi Electric Research Laboratories, Inc. Situation awareness system
US6583403B1 (en) * 1999-10-26 2003-06-24 Honda Giken Kogyo Kabushiki Kaisha Object detecting device, and travel safety system for vehicle

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130231863A1 (en) * 1998-10-08 2013-09-05 Panasonic Corporation Driving-operation assist and recording medium
US9272731B2 (en) * 1998-10-08 2016-03-01 Panasonic Intellectual Property Corporation Of America Driving-operation assist and recording medium
US20050236201A1 (en) * 2002-12-05 2005-10-27 Bayerische Motoren Werke Aktiengesellschaft Method for steering a vehicle that is to be backed into a parking space
US7469765B2 (en) * 2002-12-05 2008-12-30 Bayerische Motoren Werke Aktiengesellschaft Method for steering a vehicle that is to be backed into a parking space
US20110082613A1 (en) * 2007-02-28 2011-04-07 Moritz Oetiker Semiautomatic parking machine
US8645015B2 (en) * 2007-02-28 2014-02-04 Cobra Automotive Technologies Spa Semiautomatic parking machine
US20110095910A1 (en) * 2008-06-10 2011-04-28 Nissan Motor Co., Ltd. Parking assistance system and parking assistance method
US8803707B2 (en) 2008-06-10 2014-08-12 Nissan Motor Co., Ltd. Parking assistance system and parking assistance method
US20130144492A1 (en) * 2010-06-09 2013-06-06 Teruhisa Takano Parking mode selection apparatus and method
US9457842B2 (en) * 2010-06-09 2016-10-04 Nissan Motor Co., Ltd. Parking mode selection apparatus and method using the steering wheel
US20120249342A1 (en) * 2011-03-31 2012-10-04 Koehrsen Craig L Machine display system

Also Published As

Publication number Publication date
EP1038734A4 (en) 2005-10-12
US9272731B2 (en) 2016-03-01
EP1038734A1 (en) 2000-09-27
US7277123B1 (en) 2007-10-02
CN1287532A (en) 2001-03-14
KR100568396B1 (en) 2006-04-05
US8077202B2 (en) 2011-12-13
US20130231863A1 (en) 2013-09-05
CN1132750C (en) 2003-12-31
US20070299572A1 (en) 2007-12-27
KR20010032902A (en) 2001-04-25
US8111287B2 (en) 2012-02-07
WO2000020257A1 (en) 2000-04-13
EP1038734B1 (en) 2019-05-15
US20070299584A1 (en) 2007-12-27

Similar Documents

Publication Publication Date Title
US9272731B2 (en) Driving-operation assist and recording medium
JP3445197B2 (en) Driving operation assist device
JP4914458B2 (en) Vehicle periphery display device
US7012548B2 (en) Driving operation assisting method and system
US7212653B2 (en) Image processing system for vehicle
JP2001218197A (en) Device for displaying periphery of vehicle
US20010030688A1 (en) Steering assist device
JP4904997B2 (en) Parking support method and parking support device
US20030045973A1 (en) Motor vehicle parking support unit and method thereof
JP2006298115A (en) Driving-support method and driving-support device
KR101510655B1 (en) Around image generating method and apparatus
JP2003244688A (en) Image processing system for vehicle
JP2012514558A (en) Peripheral image generation method and apparatus
JP2013154730A (en) Apparatus and method for processing image, and parking support system
JP2006327498A (en) Parking support method and parking support device
JP2004240480A (en) Operation support device
US6463363B1 (en) Back monitoring apparatus for vehicle
JP7426174B2 (en) Vehicle surrounding image display system and vehicle surrounding image display method
WO2019034916A1 (en) System and method for presentation and control of virtual camera image for a vehicle
JP3988551B2 (en) Vehicle perimeter monitoring device
CN113327201A (en) Image processing apparatus and image processing method
JP4059309B2 (en) In-vehicle camera image display control method and apparatus
JP2007096496A (en) Vehicle periphery display system
JP4561512B2 (en) Parking support method and parking support device
US20230406410A1 (en) Method for displaying an environment of a vehicle having a coupled trailer, computer program, computing device and vehicle

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION