WO2001030079A1 - Camera with peripheral vision - Google Patents
Camera with peripheral vision Download PDFInfo
- Publication number
- WO2001030079A1 WO2001030079A1 PCT/EP2000/009770 EP0009770W WO0130079A1 WO 2001030079 A1 WO2001030079 A1 WO 2001030079A1 EP 0009770 W EP0009770 W EP 0009770W WO 0130079 A1 WO0130079 A1 WO 0130079A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- camera
- view
- field
- hemispherical
- assembly
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
Definitions
- This invention relates to a camera viewing system with peripheral vision.
- Camera viewing systems are now used for a variety of purposes including surveillance, security, video conferencing, etc., and, ideally, allow for simultaneous remote viewing, close viewing, and wide-angle and include systems in which a camera converts the optical image into an electrical signal, the image signals are digitized electronically, the digitized images are transformed by image processors controlled by a microcomputer into two or three dimensional images which are stored in an output image buffer or system, and the stored images are in turn scanned out by display electronics to a video display device for viewing.
- This limited field of view may be expanded by tilting and panning as is well known in the art but such a system suffers from drawbacks including the significant amount of time required to make a full rotation to view the expanded view making the system unsuitable for real-time applications.
- Omni-directional cameras can capture a hemispherical field of view but the captured images have limited resolution and a blind spot on top where the camera is mounted.
- Systems which employ multiple cameras with a corresponding reflector or with multiple reflectors pointing in different directions, may also be employed to achieve a large field of view. In practice, however, it is difficult to design such a configuration in which the centers of projection of all the imaging systems coincide so that a true omnidirectional system may be achieved.
- a single planar mirror does not enhance the field of view of a traditional imaging system.
- Multiple planar mirrors (each pointing in a different direction) can be used with a single imaging system. However, for any 2 given imaging system, each planar mirror produces a different center of projection. Multiple planar mirrors therefore require the use of multiple imaging systems (multiple lens and multiple image detectors).
- a fish-eye lens can also be used to obtain a hemispherical field of view. Such fish-eye lens cameras have a short focal length and can expand the field of view to be as large as a hemisphere but are larger and more complex than conventional lenses and often distort the center field of view (as well as the periphery).
- An object of the invention is to provide camera viewing systems with a panoramic or hemispherical field of view with peripheral vision that efficiently capture and display visual information within a substantially hemispheric or entire field of view containing particularly peripheral content, and that permit electronic manipulation and display of the image while minimizing distortion effects.
- a computer vision system for producing a substantially hemispheric or panoramic image comprising a camera viewing system adapted to provide a substantially panoramic or hemispherical image wherein the camera viewing system comprises at least one dual opposing camera assembly in which at least two cameras or camera assemblies are mounted in back-to-back relationship, i.e.
- At least one said camera or camera assembly providing at least a hemispherical field of view and at least one said camera or camera assembly providing a direct forward field of view that is less than hemispherical to collectively display a substantially hemispherical or entire field of view, and provide at least one image signal representative of said image; and further comprising an image signal processing apparatus coupled to receive said at least one image signal to convert said image signal into image signal data; means to convert said image signal data to form a video image, either analog or digital; and a display device to permit the viewing of the video data.
- a camera viewing system which comprises: at least one dual opposing camera assembly having a first camera effective to provide a direct forward field of view that is less than hemispherical, and a second camera effective to provide a hemispherical field of view that includes areas of a field of view that is peripheral to the field of view of said first camera, said first and second cameras being mounted in back- to-back relationship one to another.
- a computer vision system which comprises a camera viewing system in which a first camera or camera and lens assembly is effective to provide a direct forward field of view that is less than hemispherical, a second camera or camera and lens assembly which has the same optical axis as the first camera or camera and lens assembly is arranged in back-to-back relationship therewith; a mirror is arranged to reflect light from a hemispherical field of view, said second camera or camera and lens assembly and said mirror being effective to provide a reflected hemispherical field of view that includes areas of a field of view that is peripheral to the field of view of said first camera or camera and lens assembly.
- the second camera or camera and lens assembly is arranged to receive light that is reflected from the mirror, i.e. the second camera or camera and lens assembly "looks off the mirror.
- the mirror is oriented to a line which is the common optical axis of the back-to-back mounted first and second camera or camera and lens assemblies.
- Cameras and camera and lens assemblies (hereafter collectively referred to as camera assembly or camera assemblies) suitable for use in this invention may be any optical device that is capable of receiving a focussed image from any source (preferably from a lens) and transforming that image into an electronic signal or into hard copy storage such as photographic film.
- Any video source and image capturing device that captures an image and converts it into memory, electronic or otherwise, can serve as the first camera assembly herein.
- the image can be stored onto an assortment of media, e.g., photographic film, electronic storage, or other conventional storage means. Electronic storage is preferred because of the ease of electronic manipulation thereof and the invention is further described herein in terms of electronic media.
- the image processing apparatus will include an image scanner or other capture-and-conversion method to change the image into electronic format so that electronic manipulation may be performed.
- the system may also include means for digitizing an incoming video image signal, transforming a portion of the video image, if necessary, and producing one or more output images for viewing in a teleconferencing environment.
- Input and output buffers can be constructed using commercially available video random access memory chips.
- a control interface can be accomplished using microcontrollers and transform processors, image filtering and display by means well known in the art. Transmission may be via telephone lines, with the system including signal compression and decompression units, or wireless transmission with appropriate broadcasting apparatus may be utilized if desired.
- the display device may be a monitor, television, virtual reality helmet or other virtual reality display system or a projection display.
- the display device is a monitor, which may be conventional or a specially designed modular LCD monitor; it may be a single-view or multi-view monitor; and most preferably is a split screen monitor.
- the monitor may also comprise a picture-in-picture feature, if desired.
- the split-screen monitor is particularly preferred when used with the imaging systems of the invention since such a monitor enables the user to zoom in on the speaker ("X" in the figures) while retaining the peripheral scene ("y” or “yy” and “z” or “zz” in the figures) and while maintaining a high resolution of both areas of the substantially panoraic or hemispheric scene in real-time.
- Each camera assembly is schematically illustrated as comprising a lens and an imaging device such as a CCD imager.
- a conventional camera with zoom feature has been used with particular success as the first camera assembly.
- various types of cameras for wide-angle viewing such as NTSC, and HDTV-type cameras may be used.
- the camera assembly can be mounted and supported in conventional manner.
- the second camera assembly will be any device that captures or senses an image or receives light within a hemispherical field of view and may be a conventional camcorder or an omnidirectional video camera which provides a digital video signal output.
- each sensing unit will be an assembly commonly referred to as an omnicam and comprised of an off-the-shelf video camera, an off-the-shelf lens, and a curved mirror which provides a single effective center of projection for the complete sensing unit. This allows the construction of distortion-free images for any user selected portion of the acquired omnidirectional image.
- the mirror may be any suitable reflector of any suitable configuration that is capable of receiving light from a hemispherical field of view and reflecting the same to be received by the second camera assembly.
- Suitable examples of mirrors and of suitable shapes therefor are preferably a substantially paraboloid-shaped reflector as described in WO 97/43854 positioned to orthographically reflect an image of a substantially hemispheric scene.
- Other mirrors or reflectors suitable for use include a conical reflecting surface and/or hyperboloidal mirrors.
- the omnicam sensing unit is capable of imaging a complete hemispherical field of view.
- the curved mirror used to accomplish this may be hyperbolic when the imaging projection used is perspective, and parabolic when the imaging projection used is orthographic.
- the omnicam will include a telecentric lens to achieve orthographic projection and a parabolic mirror to achieve a hemispherical field of view. This configuration offers calibration advantages over one that uses a hyperbolic mirror.
- the second camera assembly is an omni-directional camera and lens assembly commonly designated an omnicam, which covers nearly a half sphere as the peripheral viewing device
- said first camera assembly is a camera with pan, tilt and zoom features or a CCD camera and lens assembly mounted in such a manner that it picks up the area that the omnicam cannot see, and arranged in back-to-back relationship on the same axis whereby the first camera assembly looks forward, and the second camera assembly looks at the hemisphere.
- a standard camera is mounted in back- to-back relationship with an omnicam camera and related components to produce a computer vision system that has an undistorted center view and a peripheral area view.
- Such a system may focus on one object or subject, while providing image data on peripheral objects or subjects.
- the undistorted center view and the peripheral area view are aligned such that parallax error between the two views is minimized and preferably is eliminated.
- the camera assemblies are preferably mounted on a pan, and tilt "PT" drive mechanism.
- the camera viewing system is eminently suitable for use in video conferencing and security systems and is substantially free of the disadvantage of prior systems discussed above.
- Figure 1 is a side view of an exemplary embodiment of an omni-directional imaging apparatus of this invention
- Figure 2 is a perspective view of a split-screen monitor according to one embodiment of the invention
- Figure 3 is a perspective view of a split-screen monitor according to a second embodiment of the invention.
- Figure 4 is a block diagram of a computer vision system according to an embodiment of the invention.
- a computer vision system (100) in which a camera viewing system (110) has a first camera assembly comprising a camera (111) with a lens (108), the camera assembly being effective to provide a direct forward field of view that is less than hemispherical and that provides an image that at least encompasses object x, and a second camera assembly comprising a camera (112) with a lens (109) and mirror (113), the camera assembly and mirror being effective to provide a hemispherical field of view that includes areas of a field of view that is hemispherical and provides an image that includes at least objects y or yy and z or zz in a field of view that is peripheral to the field of view of said first camera assembly (111,108), said first and second camera assemblies (108,109,111,112) being mounted in back-to-back relationship one to another.
- the second camera assembly (112,109) receives the reflected image of the hemispherical scene from the mirror (113), which is mounted to reflect a substantially hemispherical scene.
- the cameras (111) and (112) generate a signal which may be sent through a cable or telephone line (120) to a computer (125), all as is well known in the art.
- the stored electronic image data may then be accessed by a transform processor and can be electronically manipulated by panning, up/down, zooming, etc.
- the second camera assembly and mirror are preferably mounted on a suitable support (114) which may be a thin wire or transparent column or tubing effective to support the camera without interfering with the image. Additionally, if desired, the entire assembly may be mounted on a PT drive mechanism.
- a first camera (111), and lens (108), preferably with a zoom feature for magnification, is mounted on a pan- tilt base mechanism (116) and is effective to provide a direct forward and close-up field of view that looks forward at object X, and an omnicam second camera (112) and lens (109) looks at the hemisphere and is effective to receive from a mirror (113) a reflected substantially panoramic or hemispherical field of view that includes objects x, yy, and zz.
- the cameras (111,112), lenses (108,109), and mirror (113) are mounted, on a suitable camera support base (114), if desired, in back-to-back relationship along a common optical axis (115).
- the cameras (111,112) generate a signal which is sent through a cable or telephone line (120) to computer (125), to provide a close-up image of the selected participant or participants and a field of view that includes the peripheral environment that exists at the time of imaging.
- the computer 125 is programmed to allow the user to view any desired portion of the hemispherical scene, to zoom in on a selected portion, or to pan the scene in any desired manner.
- it is necessary and customary to zoom in to the person speaking (x) while excluding the neighboring participant (y or z) or participants (yy,zz) adjacent to the speaker from the field of vision. As a result, the context of the other persons participating in the conference is lost.
- a split-screen monitor 130 may display images of central and peripheral areas of the panoramic or hemispherical scene of high resolution.
- the view of x (the speaker), which is data passed from the first camera assembly with zoom lens is shown in the upper % to l / ⁇ of the screen, and the peripheral data content of the scene, yy and zz (other members of the teleconference or background) is shown in the lower V ⁇ to V-i of the screen.
- FIG 4 illustrates a block diagram of one embodiment of a system that incorporates one embodiment of an image signal processing apparatus for controlling data produced by the cameras of a computer viewing system described in Figs. 1 to 3 and is similar to a control system illustrated in EPO 740,177 A2.
- cameras 111, 112 obtain views of an area as described herein above.
- the image signal or output signal of the cameras is passed through analog to digital converters (131, 132), if necessary.
- the output of the cameras can be thought of as a stream of pixels and the output from the converters can be thought of as data representative of the pixels from the cameras.
- the output of the converters is passed through MUX (134) which allows the pixel data from each converter to reach the memory (135).
- Controller (136) cycles the lines of MUX so that all outputs of the A D converters are stored in the memory (135). It is also possible to eliminate MUX and store the output of each A/D converter in a separate memory. It is also possible to eliminate the A/D converters when using digital cameras or sensors. Controller (136) is implemented using a microprocessor which provides control signals to counters that control the switching of MUX and counters used to provide addressing to memory (135). As the pixel information is read from memory (135), all of the information is passed over telecommunications network (137) to telecommunications bridge (138).
- the information from memory is provided to bridge via modem (139); however, the data may be passed to bridge (138) without use of modem (139) if a digital connection is made between the memory and the bridge.
- the bridge then distributes all of the data received from memory to each user in communication with the bridge. If the bridge provides analog link to users, a modem should be used at each user port. If the bridge provides digital link with each user a modem is not required.
- the pixel data is passed to video memory (140) which then passes the data to display (126-131) under control of user input device (125), which is preferably a computer keyboard although a mouse or joystick may also be employed.
Abstract
Description
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP00969431A EP1142332A1 (en) | 1999-10-19 | 2000-10-04 | Camera with peripheral vision |
JP2001531309A JP2003512783A (en) | 1999-10-19 | 2000-10-04 | Camera with peripheral vision |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US42134999A | 1999-10-19 | 1999-10-19 | |
US09/421,349 | 1999-10-19 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2001030079A1 true WO2001030079A1 (en) | 2001-04-26 |
Family
ID=23670145
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2000/009770 WO2001030079A1 (en) | 1999-10-19 | 2000-10-04 | Camera with peripheral vision |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP1142332A1 (en) |
JP (1) | JP2003512783A (en) |
WO (1) | WO2001030079A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2006017402A2 (en) * | 2004-08-06 | 2006-02-16 | Ipix Corporation | Surveillance system and method |
WO2010015970A1 (en) * | 2008-08-04 | 2010-02-11 | Koninklijke Philips Electronics N.V. | Communication device with peripheral viewing means |
US9677840B2 (en) | 2014-03-14 | 2017-06-13 | Lineweight Llc | Augmented reality simulator |
US10324290B2 (en) | 2015-12-17 | 2019-06-18 | New Skully, Inc. | Situational awareness systems and methods |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2600308C1 (en) * | 2015-11-03 | 2016-10-20 | Вячеслав Михайлович Смелков | Device of computer system for panoramic television surveillance |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4549208A (en) * | 1982-12-22 | 1985-10-22 | Hitachi, Ltd. | Picture processing apparatus |
US5027202A (en) * | 1989-08-26 | 1991-06-25 | Messerschmitt-Bolkow-Blohm Gmbh | Picture transmission system of optical wave guide guided missiles |
WO1998047291A2 (en) * | 1997-04-16 | 1998-10-22 | Isight Ltd. | Video teleconferencing |
US5883663A (en) * | 1996-12-02 | 1999-03-16 | Siwko; Robert P. | Multiple image camera for measuring the alignment of objects in different planes |
WO1999045511A1 (en) * | 1998-03-04 | 1999-09-10 | The Trustees Of Columbia University In The City Of New York | A combined wide angle and narrow angle imaging system and method for surveillance and monitoring |
-
2000
- 2000-10-04 EP EP00969431A patent/EP1142332A1/en not_active Withdrawn
- 2000-10-04 JP JP2001531309A patent/JP2003512783A/en not_active Withdrawn
- 2000-10-04 WO PCT/EP2000/009770 patent/WO2001030079A1/en not_active Application Discontinuation
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4549208A (en) * | 1982-12-22 | 1985-10-22 | Hitachi, Ltd. | Picture processing apparatus |
US5027202A (en) * | 1989-08-26 | 1991-06-25 | Messerschmitt-Bolkow-Blohm Gmbh | Picture transmission system of optical wave guide guided missiles |
US5883663A (en) * | 1996-12-02 | 1999-03-16 | Siwko; Robert P. | Multiple image camera for measuring the alignment of objects in different planes |
WO1998047291A2 (en) * | 1997-04-16 | 1998-10-22 | Isight Ltd. | Video teleconferencing |
WO1999045511A1 (en) * | 1998-03-04 | 1999-09-10 | The Trustees Of Columbia University In The City Of New York | A combined wide angle and narrow angle imaging system and method for surveillance and monitoring |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2006017402A2 (en) * | 2004-08-06 | 2006-02-16 | Ipix Corporation | Surveillance system and method |
WO2006017402A3 (en) * | 2004-08-06 | 2006-12-28 | Ipix Corp | Surveillance system and method |
WO2010015970A1 (en) * | 2008-08-04 | 2010-02-11 | Koninklijke Philips Electronics N.V. | Communication device with peripheral viewing means |
US9677840B2 (en) | 2014-03-14 | 2017-06-13 | Lineweight Llc | Augmented reality simulator |
US10324290B2 (en) | 2015-12-17 | 2019-06-18 | New Skully, Inc. | Situational awareness systems and methods |
Also Published As
Publication number | Publication date |
---|---|
JP2003512783A (en) | 2003-04-02 |
EP1142332A1 (en) | 2001-10-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US5686957A (en) | Teleconferencing imaging system with automatic camera steering | |
Nalwa | A true omnidirectional viewer | |
US6118474A (en) | Omnidirectional imaging apparatus | |
EP0897636B1 (en) | Omnidirectional imaging apparatus | |
US6977676B1 (en) | Camera control system | |
US6426774B1 (en) | Panoramic camera | |
JP3862895B2 (en) | Panorama imager | |
US7224382B2 (en) | Immersive imaging system | |
US7667730B2 (en) | Composite surveillance camera system | |
US7593041B2 (en) | System and method for a software steerable web camera with multiple image subset capture | |
US20100045773A1 (en) | Panoramic adapter system and method with spherical field-of-view coverage | |
US6628897B2 (en) | Camera system | |
KR102087450B1 (en) | A System and Method for Processing a Very Wide Angle Image | |
US20040263611A1 (en) | Omni-directional camera design for video conferencing | |
WO2005013001A2 (en) | Panoramic video system with real-time distortion-free imaging | |
JP2000089395A (en) | Compact type high resolution panorama screen display system | |
WO1998047291A2 (en) | Video teleconferencing | |
WO2005002201A2 (en) | Visual teleconferencing apparatus | |
EP1142332A1 (en) | Camera with peripheral vision | |
JP4448001B2 (en) | Imaging device | |
JP2006025340A (en) | Wide angle imaging apparatus, imaging system, and control method thereof | |
JP4199940B2 (en) | Imaging apparatus and imaging system | |
JP2859417B2 (en) | Display / imaging device | |
CA2251879C (en) | Omnidirectional imaging apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): JP |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2000969431 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref country code: JP Ref document number: 2001 531309 Kind code of ref document: A Format of ref document f/p: F |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWP | Wipo information: published in national office |
Ref document number: 2000969431 Country of ref document: EP |
|
WWW | Wipo information: withdrawn in national office |
Ref document number: 2000969431 Country of ref document: EP |