WO1993014595A1 - Method for and apparatus for increasing the dynamic range of optical sensors - Google Patents
Method for and apparatus for increasing the dynamic range of optical sensors Download PDFInfo
- Publication number
- WO1993014595A1 WO1993014595A1 PCT/US1993/000512 US9300512W WO9314595A1 WO 1993014595 A1 WO1993014595 A1 WO 1993014595A1 US 9300512 W US9300512 W US 9300512W WO 9314595 A1 WO9314595 A1 WO 9314595A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- time
- pulses
- optical sensor
- reset
- pulse
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
- H04N23/71—Circuitry for evaluating the brightness variation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/70—Circuitry for compensating brightness variation in the scene
Definitions
- the present invention relates to a method and apparatus for increasing the dynamic range of optical sensors, e.g., for receiving viewable light or infrared thermal radiation.
- the invention is particularly useful for increasing the dynamic range of video cameras or other optical sensors including a matrix or array of photo-sensitive elements and is therefore described below particularly with respect to such applications.
- a known problem that exists today in optical sensors is the limited dynamic range of the photo-sensitive elements.
- the photo-sensitive elements commonly have a dynamic range of two to three orders of magnitude, while the range of illumination levels that they may be exposed to may be as high as eight or more orders of magnitude.
- the optical sensor is to sense a person in a darkened room standing near a lit window, the details outside the window which are at a high illumination level cannot be perceived together with the details of the person's face which are at a much lower illumination level. Under these conditions, the optical sensors encounter such problems as "blooming" .
- the present invention provides a novel method and apparatus for increasing the dynamic range of an optical sensor to overcome the above problems .
- a method of increasing the dynamic range of an optical sensor by controlling the integration time during which the optical sensor integrates the light received thereon and produces an electrical output corresponding thereto comprising: generating a series of regular reset pulses separated by prefixed time intervals of equal duration; generating a series of sample pulses each at a predetermined time interval after the generation of a regular reset pulse, which predetermined time interval represents a full integration time; generating a series of reset enable pulses at time intervals, between pulses, of shorter duration than that of the regular reset pulses; generating a control signal whenever a predetermined characteristic of the electrical output of the optical sensor exceeds a reference value; generating a conditional reset pulse whenever there is coincidence between a reset enable pulse and a control signal; starting the integration of the optical sensor at the time of generation of either a regular reset pulse or a conditional reset pulse; terminating the integration of the
- the effective integration time is determined by counting the number of conditional reset pulses between the generation of a regular reset pulse and a sample pulse.
- the reset enable pulses are generated at prefixed, but unequal time periods.
- the time intervals to the next sample pulse decrease according to a downgoing geometric series .
- the invention is particularly useful with respect to optical sensors, such as video cameras, which include a rectangular matrix or other array of photo-sensitive elements. When so used, it is preferable that the integration times of a block of photo-sensitive elements be controlled simultaneously by the control signal, although it is conceivable that each photo-sensitive elements can be individually controlled.
- such a method can increase the dynamic range of an optical sensor in a real time manner so as to substantially reduce the above-mentioned "blooming" problem.
- Fig. 1 illustrates one form of optical sensor whose dynamic range may be increased in accordance with the present invention
- Fig. 2 illustrates a conventional (prior art) circuitry for normally controlling the integration time of each photo-sensitive element in a matrix-type optical sensor
- Fig. 3 is a timing diagram illustrating the conventional (prior art) manner of controlling integration times according to Fig. 2;
- Fig. 4 illustrates one form of control circuit in accordance with the present invention which may be used for controlling the integration time of the optical sensor illustrated in Fig. 1;
- Figs. 4a, 4b and 4c illustrate examples of preferred ways of deriving the control signal in the control circuit of Fig. 4;
- Fig. 5 is a timing diagram illustrating one example for operating the control signals of Fig. 4;
- Fig. 6 schematically illustrates an example of a circuit for producing an output corresponding to the increased dynamic range of the optical sensor
- Fig. 7 illustrates one form of digital circuit which may be used for determining the effective integration time in the novel method
- Fig. 8 illustrates one form of analog circuit that may be used for determining the effective integration time in the novel method.
- an optical sensor generally designated 10, including a rectangular matrix of photo-sensitive elements 11, 12, 13, etc., as may be found in a conventional video camera.
- Each of the photo-sensitive elements 11-13 outputs an electrical signal corresponding to the amount of light received thereby in a predetermined time period, called the integration time.
- Figs. 2 and 3 illustrate the manner of determining the integration time in a conventional system.
- each photo-sensitive element, therein designated 21 is controlled by a transistor 23 receiving a series of regular reset pulses RST via line 22, separated by predetermined time intervals of equal duration.
- a second transistor 24 is supplied with a series of sample pulses SMPL at a predetermined interval after the generation of a regular reset pulse RST.
- the normal integration time in a conventional video camera is the time from the end of a reset pulse RST to be the beginning of the sample pulse SMPL. This is called the full integration time of the photo-sensitive elements.
- the circuit illustrated in Fig. 2 may include a storage element 26. This storage element, however, can be omitted when the readout is directly to the video line via readout circuit 27, for example.
- the integration time is the same for all the photo-sensitive elements. As pointed out earlier, however, when the illumination is high at a certain part of the picture, the integration time would be too long as the sensor would enter into saturation. On the other hand, in other parts of the picture where the illumination is weak, the integration time would be too low, producing a low output voltage such that the background noise might mask the signal.
- the integration time of the photo-sensitive elements are not necessarily the full integration time as described above and as illustrated in Figs. 2 and 3, but rather may be shortenend, to produce a shorter effective integration time, according to the amount of light received by the photo ⁇ sensitive elements at that particular instance.
- the photo-sensitive elements are arranged in blocks, as shown at 15, each block including a plurality of such elements.
- Each block further includes a control circuit, schematically shown at 17, for controlling the effective integration time for the respective block.
- Fig. 1 illustrates each block as comprising a plurality of photo-sensitive elements, which is generally preferred, it is contemplated that a block may consist of a single photo-sensitive element so that the photo-sensitive elements are controlled individually rather than in blocks.
- the effective integration time for each block of sensors is determined and is used for computing the outputs of the photo-sensitive elements as appearing in the output line 18, Fig. 1.
- the electrical output appearing on output line 18 is subsequently multiplied by a scaling factor, e.g., as illustrated in Fig. 6 to be described below, so that it will more closely correspond to the amount of light actually received by the photo-sensitive elements in the respective frame. In this manner, the dynamic range of the photo-sensitive elements is effectively increased.
- a control circuit 17 (Fig. 1) that may be used for increasing the dynamic range of the optical sensor is illustrated in Fig. 4.
- Such a circuit also includes, as in the conventional system, means for generating a series of regular reset pulses RST separated by prefixed time intervals of equal duration, and a series of sample pulses SMPL, each at a predetermined time interval after the generation of a regular reset pulse RST.
- a full integration time is equal to the time from the end of the reset pulse RST to the beginning of the sample pulse SMPL.
- the control circuit illustrated in Fig. 4 may produce an "effective" integration time which is shorter than the full integration time defined by the regular reset pulses RST and the sample pulses SMPL.
- the circuit illustrated in Fig. 4 further include means for generating a control signal CS corresponding to a predetermined characteristic of the optical sensor.
- This control signal CS may be derived from one of the photo-sensitive elements within the respective block 15, e.g., the centermost element as shown in Fig. 4a.
- Another alternative is to have the control signal derived from the weighted average illumination level of photo ⁇ sensitive elements in the respective block as shown in Fig. 4b.
- a further alternative is to derive the control signal CS in the manner illustrated in Fig. 4c, namely from a weighted average, produced by circuit 20, in the illumination levels of the surrounding area, e.g., blocks 15a-15d, including the controlled block 15.
- This control signal CS is applied via lead 32, together with a reference signal Vref applied via lead 33, to a comparator 34, which produces an output whenever the control signal CS exceeds the reference signal Vref.
- the circuit illustrated in Fig. 4 further includes means for generating a series of reset enable pulses RSTE at time intervals, between pulses, of shorter duration than that of the regular reset pulses RST.
- Fig. 5 illustrates a preferred example of the timing of the reset enable pulses RSTE as compared to the regular reset pulses RST and the sample pulses SMPL. In the example illustrated in Fig.
- the reset enable pulses RSTE are generated at prefixed, but unequal, time periods decreasing according to the following downgoing geometric series: T/X, T/X , T/X , T/X n wherein T is the full integration time (as determined by the ending by the regular reset pulse RST at the beginning of the sample pulse SMPL), and "X " is a preselected constant, for example, "X" may equal "2".
- the reset enable pulses RSTE are applied via lead 35 to a transistor 39, such that when an output pulse is produced from the comparator 34 in coincidence with a reset enable pulse RSTE, the transistor 39 produces a conditional reset pulse CRST.
- the so produced conditional reset pulse CRST is applied via lead 36 to an OR-gate 38 together with the regular reset pulse RST applied via lead 37, so that gate 38 produces an output on line 38A upon the occurrence of either pulse.
- OR-gate 38 The output from OR-gate 38 is applied to a transistor 40 for each photo-sensitive element in the block 15 to start the integration time of the respective photo ⁇ sensitive elements.
- the sample pulses SMPL are applied via lead 41 to another transistor 42 to terminate the integration time for the respective photo-sensitive elements.
- the voltages produced by the photo-sensitive elements during this integration time are read out via readout circuit 43 to the output line 18. It will thus be seen that for each photo-sensitive element in the respective block 15, a conditional reset pulse CRST will be generated whenever the characteristic (e.g., illumination level) represented by control signal CS exceeds a reference (represented by a signal Vref) during the time of the generation of a reset enable pulse RSTE.
- a conditional reset pulse CRST will be generated whenever the characteristic (e.g., illumination level) represented by control signal CS exceeds a reference (represented by a signal Vref) during the time of the generation of a reset enable pulse RSTE.
- the integration time will restart upon the generation of either a conditional reset pulse CRST or a regular reset pulse RST.
- the full integration time will be provided by the regular reset pulses RST; however, for those photo ⁇ sensitive elements which receive relatively high levels of illumination, the integration time will be shortened by the conditional reset pulses CRST so that those elements do not go into saturation.
- the electrical output of the photo-sensitive elements is then multiplied by a scaling factor representing the ratio of the full integration time to the effective — o —
- Fig. 6 The latter operation is schematically shown in Fig. 6, wherein it will be seen that the video output from the photo-sensitive elements in the matrix are applied via output line 18 to an analog-to-digital converter 50 and converted to digital values before being applied to a multiplier 52.
- the multiplier 52 also receives, via input lead 54, the effective integration time for the respective photo-sensitive elements, determines the scaling factor, i.e., the ratio of the full integration time to the effective integration time, and multiplies the value outputted by the photo-sensitive elements by this scaling factor. For example, if the effective integration time was one-fourth the full integration time, the scaling factor would be "4", and the value outputted by the respective photo-sensitive elements would therefore be multiplied by "4". Thus, the output on line 56 more closely corresponds to the amount of illumination actually received by the respective photo-sensitive elements.
- the effective integration time may be determined by counting the number of conditional reset pulses CRST.
- Fig. 7 illustrates a digital circuit that may be used for this purpose, including a counter having flip-flops 71 , 72 constituting a two-bit counter 75, and two further flip-flops 73, 74 constituting a two-bit register 76, for storing the number of conditional reset pulses CRST.
- Fig. 8 illustrates an analog circuit that may be used for this purpose, including a capacitor 81 which is charged each time a regular reset pulse RST is generated, and is discharged each time a conditional reset pulse CRST is generated, such that the charge in capactitor 81 will be proportional to the number of conditional resets. This value is stored in the capacitor 82.
- the effective integration time can be determined by a heuristic algorithm or other means, within or outside the chip, instead of by counting the conditional reset pulses. While the illustrated examples are based on MOS technology, the method and apparatus can also be used in other semiconductor technologies, e.g., CCD and CID. Many other variations, modifications and applications of the invention will be apparent.
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
IL10062092A IL100620A (en) | 1992-01-09 | 1992-01-09 | Method and apparatus for increasing the dynamic range of optical sensors |
IL100620 | 1992-01-09 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO1993014595A1 true WO1993014595A1 (en) | 1993-07-22 |
Family
ID=11063251
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1993/000512 WO1993014595A1 (en) | 1992-01-09 | 1993-01-06 | Method for and apparatus for increasing the dynamic range of optical sensors |
Country Status (3)
Country | Link |
---|---|
AU (1) | AU3587493A (en) |
IL (1) | IL100620A (en) |
WO (1) | WO1993014595A1 (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5565915A (en) * | 1993-06-15 | 1996-10-15 | Matsushita Electric Industrial Co., Ltd. | Solid-state image taking apparatus including photodiode and circuit for converting output signal of the photodiode into signal which varies with time at variation rate depending on intensity of light applied to the photodiode |
EP0885456A1 (en) * | 1996-11-11 | 1998-12-23 | Mordechai Shefer | Increasing the dynamic range of image sensors |
EP0912043A2 (en) * | 1997-10-27 | 1999-04-28 | Texas Instruments Incorporated | Improvements in or relating to image sensing devices |
WO2000024190A1 (en) * | 1998-10-19 | 2000-04-27 | Ben-Gurion University Of The Negev | Optical imager using a method for adaptive real-time expanding of the dynamic range |
WO2000079784A1 (en) * | 1999-06-04 | 2000-12-28 | The Trustees Of Columbia University In The City Of New York | Apparatus and method for high dynamic range imaging using spatially varying exposures |
WO2003017643A1 (en) * | 2001-08-07 | 2003-02-27 | Hrl Laboratories, Llc | Image signal conversion with extended precision and dynamic range |
US7142240B1 (en) * | 2000-07-17 | 2006-11-28 | Psion Teklogix Systems, Inc. | Active pixel sensor array reset |
US7701499B2 (en) | 2004-12-07 | 2010-04-20 | Bright Imaging Ltd. | Method and apparatus for imaging of scenes having large intensity variance |
US8610789B1 (en) | 2000-02-23 | 2013-12-17 | The Trustees Of Columbia University In The City Of New York | Method and apparatus for obtaining high dynamic range images |
US9100514B2 (en) | 2009-10-28 | 2015-08-04 | The Trustees Of Columbia University In The City Of New York | Methods and systems for coded rolling shutter |
EP2761657B1 (en) * | 2011-09-29 | 2019-10-02 | Eye-Tech S.r.l. | Electro-optical sensor with variable light dynamics |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4984002A (en) * | 1989-01-12 | 1991-01-08 | Sony Corporation | Video camera shutter control by charge accumulation time of CCD images |
US4985775A (en) * | 1985-12-26 | 1991-01-15 | Fuji Photo Film Co., Ltd. | Electronic still camera system |
US5117292A (en) * | 1989-09-13 | 1992-05-26 | Kabushiki Kaisha Toshiba | Solid-state camera having changeable sensitivity |
US5166779A (en) * | 1991-09-18 | 1992-11-24 | Videotek, Inc. | Subcarrier to horizontal phase measurement and display for multiple video signals |
-
1992
- 1992-01-09 IL IL10062092A patent/IL100620A/en not_active IP Right Cessation
-
1993
- 1993-01-06 WO PCT/US1993/000512 patent/WO1993014595A1/en active Application Filing
- 1993-01-06 AU AU35874/93A patent/AU3587493A/en not_active Abandoned
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4985775A (en) * | 1985-12-26 | 1991-01-15 | Fuji Photo Film Co., Ltd. | Electronic still camera system |
US4984002A (en) * | 1989-01-12 | 1991-01-08 | Sony Corporation | Video camera shutter control by charge accumulation time of CCD images |
US5117292A (en) * | 1989-09-13 | 1992-05-26 | Kabushiki Kaisha Toshiba | Solid-state camera having changeable sensitivity |
US5166779A (en) * | 1991-09-18 | 1992-11-24 | Videotek, Inc. | Subcarrier to horizontal phase measurement and display for multiple video signals |
Cited By (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5565915A (en) * | 1993-06-15 | 1996-10-15 | Matsushita Electric Industrial Co., Ltd. | Solid-state image taking apparatus including photodiode and circuit for converting output signal of the photodiode into signal which varies with time at variation rate depending on intensity of light applied to the photodiode |
EP0885456A1 (en) * | 1996-11-11 | 1998-12-23 | Mordechai Shefer | Increasing the dynamic range of image sensors |
EP0885456A4 (en) * | 1996-11-11 | 1998-12-23 | ||
EP0912043A2 (en) * | 1997-10-27 | 1999-04-28 | Texas Instruments Incorporated | Improvements in or relating to image sensing devices |
EP0912043A3 (en) * | 1997-10-27 | 2001-02-07 | Texas Instruments Incorporated | Improvements in or relating to image sensing devices |
US6831689B2 (en) | 1998-10-19 | 2004-12-14 | Orly Yadid-Pecht | Optical imager using a method for adaptive real-time expanding of the dynamic range |
WO2000024190A1 (en) * | 1998-10-19 | 2000-04-27 | Ben-Gurion University Of The Negev | Optical imager using a method for adaptive real-time expanding of the dynamic range |
US8934029B2 (en) | 1999-06-04 | 2015-01-13 | The Trustees Of Columbia University In The City Of New York | Apparatus and method for high dynamic range imaging using spatially varying exposures |
US7924321B2 (en) | 1999-06-04 | 2011-04-12 | The Trustees Of Columbia University In The City Of New York | Apparatus and method for high dynamic range imaging using spatially varying exposures |
WO2000079784A1 (en) * | 1999-06-04 | 2000-12-28 | The Trustees Of Columbia University In The City Of New York | Apparatus and method for high dynamic range imaging using spatially varying exposures |
US9363447B2 (en) | 1999-06-04 | 2016-06-07 | The Trustees Of Columbia University In The City Of New York | Apparatus and method for high dynamic range imaging using spatially varying exposures |
US8610789B1 (en) | 2000-02-23 | 2013-12-17 | The Trustees Of Columbia University In The City Of New York | Method and apparatus for obtaining high dynamic range images |
US7142240B1 (en) * | 2000-07-17 | 2006-11-28 | Psion Teklogix Systems, Inc. | Active pixel sensor array reset |
US7929034B2 (en) | 2000-07-17 | 2011-04-19 | Harusaki Technologies, Llc | Active pixel sensor array reset |
US6720899B2 (en) | 2001-08-07 | 2004-04-13 | Hrl Laboratories, Llc | Extended precision pixel |
WO2003017643A1 (en) * | 2001-08-07 | 2003-02-27 | Hrl Laboratories, Llc | Image signal conversion with extended precision and dynamic range |
US7701499B2 (en) | 2004-12-07 | 2010-04-20 | Bright Imaging Ltd. | Method and apparatus for imaging of scenes having large intensity variance |
US9100514B2 (en) | 2009-10-28 | 2015-08-04 | The Trustees Of Columbia University In The City Of New York | Methods and systems for coded rolling shutter |
US9736425B2 (en) | 2009-10-28 | 2017-08-15 | Sony Corporation | Methods and systems for coded rolling shutter |
EP2761657B1 (en) * | 2011-09-29 | 2019-10-02 | Eye-Tech S.r.l. | Electro-optical sensor with variable light dynamics |
Also Published As
Publication number | Publication date |
---|---|
IL100620A (en) | 1994-11-11 |
AU3587493A (en) | 1993-08-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6831689B2 (en) | Optical imager using a method for adaptive real-time expanding of the dynamic range | |
US8477205B2 (en) | Electronic camera that synthesizes two images taken under different exposures | |
EP0102030B1 (en) | Still image recording apparatus | |
US4635126A (en) | Image pick-up system | |
KR100247936B1 (en) | Readout method and apparatus in moving picture camera system | |
US4597014A (en) | Solid-state image pickup device | |
EP0133586A1 (en) | Apparatus and method for producing a still image video signal using solid-state imaging device | |
JPH04170175A (en) | Driver for solid-state image pickup element | |
JP2002314873A (en) | Circuit for imaging device and signal processing method for imaging | |
WO1993014595A1 (en) | Method for and apparatus for increasing the dynamic range of optical sensors | |
JPH0250584A (en) | Dynamic range enlarging system | |
KR100430411B1 (en) | Programmable incremental a/d converter for digital camera and image processing | |
US5512950A (en) | Solid state imager device and driving method thereof | |
KR100341978B1 (en) | Video Processing System and Video Image Processing Method | |
JP2893687B2 (en) | Focus detection device | |
JPH06197286A (en) | Image pickup unit | |
JPH0556343A (en) | Electronic image pickup device | |
EP0712236B1 (en) | Timing signal generator for a solid state imaging device having an electronic shutter | |
JP2548705B2 (en) | Imaging device | |
KR100248226B1 (en) | Method and apparatus of brightness control for image scanner | |
JPH10173988A (en) | Electronic still camera | |
JP2519202B2 (en) | Electronic imager | |
KR0183808B1 (en) | Electronic shutter speed control system for ccd camera | |
JPH0522653A (en) | Exposure control method for image pickup device | |
JPH05207219A (en) | Picture reader |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AT AU BB BG BR CA CH DE DK ES FI GB HU JP KP KR LK LU MG MN MW NL NO PL RO RU SD SE US |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN ML MR SN TD TG |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
ENP | Entry into the national phase |
Ref country code: US Ref document number: 1994 256383 Date of ref document: 19940708 Kind code of ref document: A Format of ref document f/p: F |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
122 | Ep: pct application non-entry in european phase | ||
NENP | Non-entry into the national phase |
Ref country code: CA |