US20090102953A1 - Performance digital image sensing - Google Patents

Performance digital image sensing Download PDF

Info

Publication number
US20090102953A1
US20090102953A1 US12/339,543 US33954308A US2009102953A1 US 20090102953 A1 US20090102953 A1 US 20090102953A1 US 33954308 A US33954308 A US 33954308A US 2009102953 A1 US2009102953 A1 US 2009102953A1
Authority
US
United States
Prior art keywords
sensing elements
machine
light
series
sensing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/339,543
Inventor
William Bornstein
Anthony Cappa Spielberg
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to US12/339,543 priority Critical patent/US20090102953A1/en
Publication of US20090102953A1 publication Critical patent/US20090102953A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/48Increasing resolution by shifting the sensor relative to the scene
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • H04N23/84Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/843Demosaicing, e.g. interpolating colour pixel values
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/68Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects
    • H04N25/683Noise processing, e.g. detecting, correcting, reducing or removing noise applied to defects by defect estimation performed on the scene signal, e.g. real time or on the fly detection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2209/00Details of colour television systems
    • H04N2209/04Picture signal generators
    • H04N2209/041Picture signal generators using solid-state devices
    • H04N2209/042Picture signal generators using solid-state devices having a single pick-up sensor
    • H04N2209/045Picture signal generators using solid-state devices having a single pick-up sensor using mosaic colour filter
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/11Arrangement of colour filter arrays [CFA]; Filter mosaics
    • H04N25/13Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
    • H04N25/134Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements

Definitions

  • the present invention is in the field of color sensing. More particularly, the present invention relates to methods and arrangements to capture a digital image.
  • a digital image sensor is an integral part of several electronic systems including computers, cell phones, digital photography systems, robotics vision systems, security cameras, medical instruments, color matching applications and other color photosensitive devices.
  • One digital image sensor such as a TCS230 Programmable Color Light-to-Frequency Converter manufactured by TAOS, Inc., typically includes sensing elements, which may be sensitive to a broad range of frequencies of light. Some systems include lenses that are added and positioned on top of each sensing element to collect light and to make the sensing elements more sensitive.
  • Adding color filters to light sensors on an image sensor to specifically be color sensing elements may capture the brightness of the light that passes through and provide color sensing for red, green, and blue, for example.
  • color sensing elements typically record only the brightness of the light that matches its filter and passes through it while other colors are blocked. For example, a pixel with a red filter senses the brightness of only the red light that strikes it.
  • the color sensing elements appear in a photodiode grid consisting of groups of individual color sensing elements, each checkered throughout the grid on the same optical plane.
  • a group may include individual color sensing elements, for example, a red sensing element, a green sensing element, a blue sensing element, and sometimes a clear sensing element with no filter for intensity information. All of the photodiodes of the same color are typically connected in parallel.
  • the color selected for use during operation is dynamically selectable via two programming pins. If the output is digital, the output for individual color sensing elements is typically a square wave whose frequency is directly proportional to the intensity of the selected color.
  • Defect sensitivity is introduced in the fabrication of the sensing array in the conventional approach. Spatial error occurs because readings are taken at different points than the actual point interpolated; inference error takes place because each reading is not a direct observation.
  • interpolation calculations involve a great deal of two-dimensional computation to be performed on each image in real time after each exposure through mathematical algorithms used to determine an appropriate color to assign to pixels in an image. In particular, algorithms typically interpolate visual colors at various points on a grid using a checkering of red, green, and blue sensing elements. Such interpolation computation requires sophisticated processing capability to be built into the imaging device. These interpolations translate into increased hardware cost, increased energy consumption, and slower cycle time for the image capturing device.
  • An artifact is a distortion of the image that degrades image quality, for example, stair steps on a diagonal line.
  • One solution to defect sensitivity is to use firmware to ignore the information coming from a single sensing element of an array of sensing elements that has been found to be defective after its manufacture, and replace that information with additional interpolation from nearby elements. This solution, however, degrades image quality.
  • the X3 sensor does not use a conventional checkerboard array of elements sensitive to a single color
  • use of the X3 sensor requires redesigning the camera system to accommodate the X3 sensor; such redesign increases system design costs and lengthens product development cycles.
  • the X3 sensor outputs raw sensor data, requiring additional processing outside the camera, which is time-consuming and inconvenient.
  • One embodiment provides a device to capture a digital image.
  • the device contemplates a series of sensing elements in one collecting location to collect multiple color data readings to determine a color value for a pixel in an image and reflective optics to redirect light to the series of sensing elements successively within one exposure.
  • An additional embodiment provides a machine-accessible medium to capture a digital image.
  • the medium includes instructions, which when executed by a machine, cause said machine to perform operations, that generally include taking multiple color data readings with a series of sensing elements in one collecting location during a single exposure.
  • the operations may include directing light successively to the sensing elements of the series of sensing elements within one exposure via reflective optics.
  • the operations may include determining a sensing element of the series of sensing elements is defective.
  • the operations may include redirecting light to align a non-defective sensing element of the series of sensing elements with the collecting location.
  • the non-defective sensing element and the defective sensing element may be associated with the same color.
  • the operations may include associating the collecting location with a pixel position in an image to be portrayed, and determining a color value for the pixel position in the image based on the multiple color data readings.
  • a further embodiment provides a system to capture a digital image.
  • the system contemplates a plurality of sensing elements, including light sensors covered with colored filters to receive light and to generate filtered readings of data of the light.
  • the system also includes a motor coupled to the plurality of light sensors, wherein the motor is configured to move the plurality of light sensors, and moving logic coupled with the motor to move the plurality of light sensors in succession into the single collecting location to take the filtered readings.
  • the system includes a memory to associate the filtered readings from a collecting location with a pixel position in an image and to store the association into a location in memory and a calculator to calculate a color value based upon the filtered readings from the plurality of light sensors.
  • FIG. 1 depicts an embodiment of a system including a lens, software, and an image sensor having sensing elements to capture a representation of a visual image;
  • FIG. 2 depicts sensing elements capturing data from light falling on an image sensor
  • FIG. 3A depicts patterns of movement of an image sensor during data captures of an image
  • FIG. 3B depicts examples of conversions of sensing element readings to pixel information in an output image
  • FIG. 4 depicts a block diagram of an embodiment of internal software and hardware of an imaging system including an image sensor, sensing elements, motor, processor, memory, and I/O to capture a representation of a visual image;
  • FIG. 5 depicts a block diagram of an embodiment of a CMOS chip implementation of an image sensor
  • FIG. 6 depicts an example of a flow chart illustrating a method to capture a representation of a visual image.
  • hardware and/or software may implement logic to move a sensing array of photosensitive elements during one exposure to allow multiple sensing elements to receive the same light falling at each pixel position in an image.
  • the sensing array may be read several times.
  • information from a first color sensing element for example red
  • the sensor is then moved so that a second color sensing element, for example blue, is positioned in the collecting location that the first color sensing element occupied during the first read.
  • a second color sensing element for example blue
  • many embodiments provide methods to recover from defective sensing elements.
  • the motion of the sensor can be adjusted to redirect the light intended for a defective sensing element to another nearby non-defective sensing element. Thus, no loss of image quality accompanies that defective sensing element.
  • two-dimensional interpolation may be eliminated to calculate the appropriate color to be assigned to that location. Eliminating real-time complex interpolation may reduce imaging hardware system cost, improve system power efficiency, and improve system cycle time.
  • FIG. 1 depicts an embodiment of a system ( 100 ) to capture a representation of a visual image.
  • the system ( 100 ) captures color for the visual image during one exposure by taking three distinct readings with a physical difference in placement between an image sensor ( 108 ) and a lens ( 102 ), typically between readings.
  • the system ( 100 ) is a digital camera including a lens ( 102 ) to focus.
  • the image sensor ( 108 ) of the system ( 100 ) provides capability to capture colors of an image.
  • the image sensor ( 108 ) may be mounted, for example, with a complimentary metal oxide semiconductor (CMOS) chip included on a laminate substrate.
  • CMOS complimentary metal oxide semiconductor
  • the image sensor ( 108 ) includes sensing elements to take readings and record color images, wherein the elements are typically in fixed locations on the image sensor ( 108 ).
  • a motor ( 110 ) may be mounted on the image sensor ( 108 ).
  • the motor ( 110 ) is typically capable of moving the image sensor ( 108 ) both laterally and vertically.
  • this motor ( 110 ) is a piezoelectric motor, such as a Piezoelectric Micropositioning Motor Model PDA130, manufactured by EDO Corporation. Piezoelectric motors are based on quartz crystals generating an electric field when stressed along a primary axis. The piezoelectric quartz crystals exhibit a relationship between mechanical strain and voltage across their surfaces.
  • a piezoelectric crystal when compressed or pulled, a piezoelectric crystal may build up alternate charges on opposite faces, thus acting like a capacitor with an applied voltage. Piezoelectricity current can then be generated between the faces. On the other hand, when subjected to an external voltage, the crystal will expand or contract accordingly. This process is functionally analogous to magnetization of material shapes containing iron. Once poled, the material exhibits useful piezoelectric properties. Piezoelectric motors use this poled ceramic shape to create motion with the use of periodic electric fields, which in some embodiments are sinusoidal.
  • the system ( 100 ) also includes equipment to support the functionality of the image sensor ( 108 ).
  • the system ( 100 ) includes one or more filters ( 106 ), for example, an infrared absorption filter to remove infrared light that is not contributing to the physical image and possibly degrading quality of readings taken by the image sensor ( 108 ).
  • the system ( 100 ) also includes a shutter mechanism ( 104 ). When the shutter ( 104 ) is open, the system ( 100 ) may receive light to the image sensor ( 108 ).
  • the shutter ( 104 ) may not be a physical means, but software where the photosensitive elements on the image sensor ( 108 ) may receive light only at intervals that are controlled by software.
  • the camera ( 100 ) may also include firmware ( 114 ) on a circuit board ( 112 ) that includes software ( 116 ), for example, to control the shutter opening and closing and to manipulate the sensor to capture images.
  • Sensing elements are depicted capturing color data readings from light ( 202 ) falling on an image sensor in an image position ( 204 ) in FIG. 2 , in which the same light rays ( 202 ) hit multiple different sensing elements at sequential time periods, for example Time 0 , Time 1 , and Time 2 , within a single exposure.
  • Captured color data readings may include brightness, hue, saturation, intensity, or other data describing light.
  • Rays of light ( 202 ) are falling in one collecting location of an image sensor position ( 204 ) at the beginning of a capture time.
  • the image sensor is moved by, for example, the width of one sensing element until another sensing element is receiving the light ( 202 ) to capture further data in a new image sensor position ( 206 ).
  • the image sensor position ( 204 ) is moved another length, for example, a width of a photosensitive element until an additional sensing element is receiving the light ( 202 ) to take another data reading in another image sensor position ( 206 ).
  • FIG. 2 includes additional sensor positions for data captures to other sensing elements included on the image sensor. Patterns of movement of the image sensor are further discussed in FIG. 3 .
  • reflective optics may be used. Instead of moving a series of optical sensing elements, adapted to collect multiple color data readings to calculate a color value for a pixel in an image, some reflective optics redirect light to the series of optical sensing elements successively within one exposure.
  • the series of optical sensing elements comprises a first element sensitive to red light, a second element sensitive to blue light, and a third element sensitive to green light.
  • the reflective optics may include a digital light processor including a digital micromirror device and control circuitry.
  • the reflective optics may include one or more mirrors to scan light to the series of sensing elements in conjunction with, for example, a digital light processor.
  • each sensing element of the image sensor ( 300 ) captures different information; for example, red, blue, or green.
  • the sensing elements may each include a light sensor to detect luminance and a color filter.
  • sensing elements featuring a red sensing element, blue sensing element, or green sensing element may incorporate a light sensor and a red filter, blue filter, or green filter to capture and indicate an amount of red light, blue light, or green light, respectively.
  • sensing elements may be sensitive to cyan, orange, yellow, magenta, or other colors.
  • the image sensor ( 300 ) moves in a pattern ( 302 , 304 ) between data captures by sensing elements.
  • FIG. 3 illustrates an L-shaped pattern ( 302 ).
  • a blue sensing element captures data first; then, a red sensing element captures data; and lastly, a green sensing element captures data.
  • Another pattern ( 304 ), shaped like a backwards P is also shown to be used in other embodiments.
  • data is captured sequentially by a green sensing element, blue sensing element, and then a red sensing element.
  • sensing elements move between various collecting locations during an exposure.
  • the sensing elements at the edge of the image sensor also move in and out of collecting locations and do not necessarily capture the same number of data readings.
  • the patterns ( 304 , 306 ) of the image sensor do not necessarily provide the same number of multiple captures for every edge sensing element as for interior sensing elements of the image sensor.
  • One embodiment accounts for this difference with extra rows and columns to form an outline of sensing elements around the image sensor. Within the outline, sensing elements do not necessarily equally contribute to multiple data readings to supply pixels in an image and may be ignored from determining the pixels.
  • An additional embodiment to account for the possible difference of numbers of edge and interior sensing elements' captures includes additional captures, for example, in the case of aiming for three data readings per sensing element, instead of three captures, six captures may occur: three in a pattern, then an additional one on the edge moved away from last, and an additional two on the edge moved away from first.
  • Another embodiment to account for the possible difference of numbers of edge and interior sensing elements' captures includes interpolating the color values of the edge sensing elements from the interior sensing elements.
  • edge sensing elements with one value may produce pixels that are mere duplicates of pixels produced by the nearest interior sensing element.
  • the image sensor ( 300 ) provides a solution for situations in which a sensing element is defective.
  • a sensing element is found to be defective by one of several ways.
  • the image sensor ( 300 ) may perform a self-test to individually address each sensing element to simulate a burst of light.
  • a sensing element may be labeled defective if it fails to respond.
  • a system ( 100 ) such as in FIG. 1 might take a test exposure simulating no light entering the lens. Any specks of color that diminish a pure black image would indicate those associated sensing elements are defective.
  • a pure white image is simulated and any sensing elements detracting from the image are defective.
  • the image sensor ( 300 ) runs a test run and queries the user to respond that all of the sensing elements are functioning, by showing, for example, a red lattice, a blue lattice, and then a green lattice.
  • the image sensor ( 300 ) may alter its movement of data collection. For example, if the green sensing element of the pattern ( 302 ) is defective, the pattern ( 302 ) may be altered so that the image sensor ( 300 ) moves in the linear pattern from the blue sensing element to the red sensing element of the pattern ( 302 ) and continuing in the same direction to include the next sequential green sensing element instead. In this way, the image captured does not lack any information that it would have included if the original green sensing element had been fully functional.
  • the pattern ( 304 ) could be diverted to the red sensing element on the right after the blue sensing element.
  • software such as software ( 116 ) in FIG. 1 , may add additional data captures to accommodate completing the gathering of multiple data readings per sensing element, for example four captures.
  • FIG. 3B depicts examples of conversions of sensing element readings to pixel information in an output image.
  • FIG. 3B includes sensing elements ( 350 , 352 , 354 , 360 , 362 , 364 , 370 , 372 , 374 , 380 , 382 , 384 ) to capture multiple data readings in accordance with the present invention.
  • the sensing elements 350 , 352 , 354 , 360 , 362 , 364 , 370 , 372 , 374 , 380 , 382 , 384 ) capture three color readings, an amount each of red, blue, and green.
  • each element 350 , 352 , 354 , 360 , 362 , 364 , 370 , 372 , 374 , 380 , 382 , 384 ) may be directly associated with one or more pixel positions without further interpolation calculation.
  • groups of three sensing elements may be associated, and their data readings combined to provide several readings to interpolate a color value for resulting pixels ( 355 ), ( 365 ), ( 375 ), ( 385 ), for example.
  • three elements are associated with one pixel position; however, any number of elements may be associated with any number of pixel positions.
  • FIG. 4 depicts a block diagram of an embodiment of internal software and hardware of a camera ( 100 ) to capture a representation of a visual image.
  • light ( 402 ) enters through the lens ( 102 ) and falls on the image sensor ( 108 ) which includes an array of sensing elements ( 404 ).
  • these sensing elements ( 404 ) include different types of sensing elements that are checkered within a two-dimensional array.
  • a motor ( 110 ) is also included to move the image sensor ( 108 ) between captures.
  • Camera ( 100 ) also includes a processor ( 408 ), memory ( 412 ), input/output (I/O) ( 416 ), flash card ( 418 ), and a system bus ( 406 ) to connect the camera's components.
  • the processor ( 408 ) includes color calculating logic ( 410 ) within it to process data captures and to convert data captures from the sensor into pixel information.
  • the memory ( 412 ) includes instructions ( 414 ) to move the image sensor ( 108 ) with the motor ( 110 ) between multiple color readings. These instructions provide functionality for the motor ( 110 ) to move the image sensor ( 108 ) so sensing elements ( 404 ) line up in the same collecting location for data readings within a single exposure.
  • the I/O ( 416 ) manages receiving instructions from a user, importing an image from a flash card ( 418 ), capturing data from the image sensor ( 108 ) to transfer to the processor ( 408 ), and outputting a picture output ( 420 ) on an output pin or pins, and storing an image on the flash card ( 418 ).
  • the internals of the camera ( 100 ) may be implemented with CMOS logic, for example, to provide the ability for the camera to function, turn on, expose the shutter, convert data captures from the sensor into pixel information, or write a resulting image file out to a compact flash card.
  • the image sensor may be implemented on a separate charged coupled device (CCD) integrated circuit chip.
  • the internals of the camera ( 100 ) and the image sensor ( 108 ) are implemented on separate CMOS chips within the same camera ( 100 ) or other imaging system.
  • FIG. 5 a block diagram depicts an embodiment of an image sensor ( 500 ) implemented as a CMOS chip.
  • the CMOS sensor chip ( 500 ) includes memory ( 506 ) which is typically a rectangular array of temporary storage RAM.
  • a photosensitive array ( 502 ) of sensing elements is also included as well as some control circuitry ( 504 ) which may be amplifiers or register addressing circuitry or both around the outside of the photosensitive array ( 502 ) to provide an ability for information to be read out of the array ( 502 ).
  • the CMOS sensor chip ( 500 ) also includes a portion whose function is I/O ( 510 ) to provide access to chip information.
  • the CMOS sensor chip ( 500 ) may include, hardwired into its circuit, certain logic functions ( 508 ) to repeatedly perform.
  • the logic ( 508 ) may contain logic to move the motor ( 110 ) or send data captured by the photosensitive array ( 502 ) to I/O ( 510 ).
  • an algorithm to combine data captured by the photosensitive array ( 502 ) into a pixel array may be hardwired into the circuitry of this CMOS sensor chip ( 500 ).
  • the algorithm is soft-wired in flash ROM (not shown) on the CMOS sensor chip ( 500 ).
  • the algorithm is not stored on this CMOS sensor chip, but stored externally in a magnetic medium, flash memory, ROM, or the like, elsewhere in the camera ( 100 ).
  • the algorithm may be loaded into the CMOS sensor chip ( 500 ) when it is powered up.
  • Method ( 600 ) begins with element 602 , during which an imaging system, such as a camera, determines whether any of its sensing elements are defective. If any sensing elements are defective, then the imaging system adjusts its software to redirect its image sensor motion to include a replacement non-defective element (element 604 ).
  • an imaging system such as a camera
  • color readings are taken at a collecting location with a sensing element of an array (element 606 ).
  • the collecting location refers to a static physical position to receive a designated light falling in that location.
  • the array is moved between readings by the width of one sensing element to allow a new sensing element to analyze the same light and thus receive the light in the collecting location (element 608 ).
  • the collecting location may be defined relative to the ray of light and refer to the location where optics focus rays of light on the image sensor.
  • the reflective optics is able to redirect light to sensing elements successively within one exposure.
  • a digital light processor including a digital micromirror device and control circuitry may redirect light to the sensing elements successively within one exposure. For as many elements and readings from the collecting position are chosen, element 610 follows back to element 606 .
  • the captured information is used to create image information.
  • Mathematical algorithms may be performed to convert the data readings to pixel information (element 612 ).
  • an image is outputted (element 614 ).
  • the pixel information may be stored to memory or sent out on a bus to another memory location or coupled device.
  • One embodiment of the invention is implemented as a program product for use with an image system such as, for example, as shown in FIG. 1 .
  • the program(s) of the program product defines functions of the embodiments (including the methods described herein) and can be contained on a variety of signal-bearing media.
  • Illustrative signal-bearing media include, but are not limited to: (i) information permanently stored on non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive); (ii) alterable information stored on writable storage media (e.g., floppy disks within a diskette drive or hard-disk drive); and (iii) information conveyed to a computer by a communications medium, such as through a computer or telephone network, including wireless communications. The latter embodiment specifically includes information downloaded from the Internet and other networks.
  • Such signal-bearing media when carrying computer-readable instructions that direct the functions of the present invention, indicate embodiments of the present invention.
  • routines executed to implement the embodiments of the invention may be part of an operating system or a specific application, component, program, module, object, or sequence of instructions.
  • the computer program of the present invention typically is comprised of a multitude of instructions that will be translated by the native computer into a machine-readable format and hence executable instructions.
  • programs are comprised of variables and data structures that either reside locally to the program or are found in memory or on storage devices.
  • various programs described hereinafter may be identified based upon the application for which they are implemented in a specific embodiment of the invention. However, it should be appreciated that any particular program nomenclature that follows is used merely for convenience, and thus the invention should not be limited to use solely in any specific application identified and/or implied by such nomenclature.

Abstract

Systems and media to capture a digital image are disclosed. Embodiments include hardware and/or software for taking multiple color data readings with a series of sensing elements in one collecting location during a single exposure, associating the collecting location with a pixel, and calculating a color value for the pixel based on the multiple color data readings. The hardware or software may also direct light successively to the sensing elements of the series of sensing elements within one exposure via reflective optics, and may determine that a sensing element of the series of sensing elements is defective. The hardware or software may redirect light to align a non-defective sensing element of the series of sensing elements with the collecting location. The non-defective sensing element and the defective sensing element may be associated with the same color.

Description

    CROSS-REFERENCES TO RELATED APPLICATIONS
  • Pursuant to 35 USC § 120, this continuation application claims priority to and benefit of U.S. patent application Ser. No. 10/755,888, entitled “PERFORMANCE DIGITAL IMAGE SENSING”, attorney docket number AUS920030988US1(4031), filed on Jan. 13, 2004, the disclosure of which is incorporated herein in its entirety for all purposes.
  • BACKGROUND
  • The present invention is in the field of color sensing. More particularly, the present invention relates to methods and arrangements to capture a digital image.
  • A digital image sensor is an integral part of several electronic systems including computers, cell phones, digital photography systems, robotics vision systems, security cameras, medical instruments, color matching applications and other color photosensitive devices. One digital image sensor, such as a TCS230 Programmable Color Light-to-Frequency Converter manufactured by TAOS, Inc., typically includes sensing elements, which may be sensitive to a broad range of frequencies of light. Some systems include lenses that are added and positioned on top of each sensing element to collect light and to make the sensing elements more sensitive.
  • Adding color filters to light sensors on an image sensor to specifically be color sensing elements may capture the brightness of the light that passes through and provide color sensing for red, green, and blue, for example. Thus, with color filters in place, color sensing elements typically record only the brightness of the light that matches its filter and passes through it while other colors are blocked. For example, a pixel with a red filter senses the brightness of only the red light that strikes it.
  • The color sensing elements appear in a photodiode grid consisting of groups of individual color sensing elements, each checkered throughout the grid on the same optical plane. For example, a group may include individual color sensing elements, for example, a red sensing element, a green sensing element, a blue sensing element, and sometimes a clear sensing element with no filter for intensity information. All of the photodiodes of the same color are typically connected in parallel. With the TCS230, the color selected for use during operation is dynamically selectable via two programming pins. If the output is digital, the output for individual color sensing elements is typically a square wave whose frequency is directly proportional to the intensity of the selected color.
  • As common as digital image sensors are to electronic systems, problems of defect sensitivity, spatial error, inference error and interpolation calculations make these benefits difficult to realize. Defect sensitivity is introduced in the fabrication of the sensing array in the conventional approach. Spatial error occurs because readings are taken at different points than the actual point interpolated; inference error takes place because each reading is not a direct observation. And, interpolation calculations involve a great deal of two-dimensional computation to be performed on each image in real time after each exposure through mathematical algorithms used to determine an appropriate color to assign to pixels in an image. In particular, algorithms typically interpolate visual colors at various points on a grid using a checkering of red, green, and blue sensing elements. Such interpolation computation requires sophisticated processing capability to be built into the imaging device. These interpolations translate into increased hardware cost, increased energy consumption, and slower cycle time for the image capturing device.
  • Unfortunately, this interpolation typically also introduces artifacts into the image resulting from the mathematical interpolation. An artifact is a distortion of the image that degrades image quality, for example, stair steps on a diagonal line.
  • One solution to defect sensitivity is to use firmware to ignore the information coming from a single sensing element of an array of sensing elements that has been found to be defective after its manufacture, and replace that information with additional interpolation from nearby elements. This solution, however, degrades image quality.
  • One solution to spatial error, inference error and interpolation has been making digital image sensors with a technology called X3, where the sensing elements are stacked vertically to read light at the same time. The colors, however, tend to appear undersaturated and cannot always be tuned to brilliance with software. In addition, vertically stacked sensing elements produce excessive noise, especially in shadows and red hues. The noise problem becomes even more severe at higher International Standards Organization (ISO) settings for photographic sensitivity; for example, ISO 400 is a well-known photographic sensitivity in the art of photography and typically indicates a digital emulation of a resulting traditional exposure based on a given shutter speed and aperture size. ISO 400 shots taken with X3 technology involved show multicolored noise that would ruin many prints. As the X3 sensor does not use a conventional checkerboard array of elements sensitive to a single color, use of the X3 sensor requires redesigning the camera system to accommodate the X3 sensor; such redesign increases system design costs and lengthens product development cycles. Furthermore, the X3 sensor outputs raw sensor data, requiring additional processing outside the camera, which is time-consuming and inconvenient.
  • Therefore, there is a need for methods and arrangements capable of capturing quality images with less spatial error, inference error and interpolation.
  • SUMMARY OF THE INVENTION
  • One embodiment provides a device to capture a digital image. The device contemplates a series of sensing elements in one collecting location to collect multiple color data readings to determine a color value for a pixel in an image and reflective optics to redirect light to the series of sensing elements successively within one exposure.
  • An additional embodiment provides a machine-accessible medium to capture a digital image. The medium includes instructions, which when executed by a machine, cause said machine to perform operations, that generally include taking multiple color data readings with a series of sensing elements in one collecting location during a single exposure. The operations may include directing light successively to the sensing elements of the series of sensing elements within one exposure via reflective optics. The operations may include determining a sensing element of the series of sensing elements is defective. The operations may include redirecting light to align a non-defective sensing element of the series of sensing elements with the collecting location. The non-defective sensing element and the defective sensing element may be associated with the same color. The operations may include associating the collecting location with a pixel position in an image to be portrayed, and determining a color value for the pixel position in the image based on the multiple color data readings.
  • A further embodiment provides a system to capture a digital image. The system contemplates a plurality of sensing elements, including light sensors covered with colored filters to receive light and to generate filtered readings of data of the light. The system also includes a motor coupled to the plurality of light sensors, wherein the motor is configured to move the plurality of light sensors, and moving logic coupled with the motor to move the plurality of light sensors in succession into the single collecting location to take the filtered readings. Further, the system includes a memory to associate the filtered readings from a collecting location with a pixel position in an image and to store the association into a location in memory and a calculator to calculate a color value based upon the filtered readings from the plurality of light sensors.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • An explanation of the invention will become apparent upon reading the following detailed description and upon reference to the accompanying drawings in which like references may indicate similar elements:
  • FIG. 1 depicts an embodiment of a system including a lens, software, and an image sensor having sensing elements to capture a representation of a visual image;
  • FIG. 2 depicts sensing elements capturing data from light falling on an image sensor;
  • FIG. 3A depicts patterns of movement of an image sensor during data captures of an image;
  • FIG. 3B depicts examples of conversions of sensing element readings to pixel information in an output image;
  • FIG. 4 depicts a block diagram of an embodiment of internal software and hardware of an imaging system including an image sensor, sensing elements, motor, processor, memory, and I/O to capture a representation of a visual image;
  • FIG. 5 depicts a block diagram of an embodiment of a CMOS chip implementation of an image sensor; and
  • FIG. 6 depicts an example of a flow chart illustrating a method to capture a representation of a visual image.
  • DETAILED DESCRIPTION OF EMBODIMENTS Introduction
  • The following is a detailed description of embodiments of the invention depicted in the accompanying drawings. The embodiments are in such detail as to clearly communicate the invention. However, the amount of detail offered is not intended to limit the anticipated variations of embodiments; but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present invention as defined by the appended claims. The detailed descriptions below are designed to make such embodiments obvious to a person of ordinary skill in the art.
  • Generally speaking, methods and arrangements are contemplated to capture a digital image. In many embodiments, hardware and/or software may implement logic to move a sensing array of photosensitive elements during one exposure to allow multiple sensing elements to receive the same light falling at each pixel position in an image. During one exposure, the sensing array may be read several times. During a first read, information from a first color sensing element, for example red, is captured. The sensor is then moved so that a second color sensing element, for example blue, is positioned in the collecting location that the first color sensing element occupied during the first read. Once the movement is complete, information from the second color sensing element is captured. Then, the captured information is used to determine a color value for a pixel. Determining a color value may include combining the captured information as component values of the pixel color value.
  • Additionally, many embodiments provide methods to recover from defective sensing elements. In one embodiment, the motion of the sensor can be adjusted to redirect the light intended for a defective sensing element to another nearby non-defective sensing element. Thus, no loss of image quality accompanies that defective sensing element.
  • Because the information at each location in the focal plane is being sensed directly by three sensing elements, two-dimensional interpolation may be eliminated to calculate the appropriate color to be assigned to that location. Eliminating real-time complex interpolation may reduce imaging hardware system cost, improve system power efficiency, and improve system cycle time.
  • DETAILED DESCRIPTION
  • Turning now to the drawings, FIG. 1 depicts an embodiment of a system (100) to capture a representation of a visual image. The system (100) captures color for the visual image during one exposure by taking three distinct readings with a physical difference in placement between an image sensor (108) and a lens (102), typically between readings. In one embodiment, the system (100) is a digital camera including a lens (102) to focus.
  • More particularly, the image sensor (108) of the system (100) provides capability to capture colors of an image. The image sensor (108) may be mounted, for example, with a complimentary metal oxide semiconductor (CMOS) chip included on a laminate substrate. The image sensor (108) includes sensing elements to take readings and record color images, wherein the elements are typically in fixed locations on the image sensor (108).
  • While capturing one image, the image sensor (108) is able to move multiple times and gather multiple readings by its sensing elements. To move the image sensor (108) quickly, especially within a single image exposure, a motor (110) may be mounted on the image sensor (108). The motor (110) is typically capable of moving the image sensor (108) both laterally and vertically. In one embodiment, this motor (110) is a piezoelectric motor, such as a Piezoelectric Micropositioning Motor Model PDA130, manufactured by EDO Corporation. Piezoelectric motors are based on quartz crystals generating an electric field when stressed along a primary axis. The piezoelectric quartz crystals exhibit a relationship between mechanical strain and voltage across their surfaces. Specifically, when compressed or pulled, a piezoelectric crystal may build up alternate charges on opposite faces, thus acting like a capacitor with an applied voltage. Piezoelectricity current can then be generated between the faces. On the other hand, when subjected to an external voltage, the crystal will expand or contract accordingly. This process is functionally analogous to magnetization of material shapes containing iron. Once poled, the material exhibits useful piezoelectric properties. Piezoelectric motors use this poled ceramic shape to create motion with the use of periodic electric fields, which in some embodiments are sinusoidal.
  • The system (100) also includes equipment to support the functionality of the image sensor (108). In one embodiment, the system (100) includes one or more filters (106), for example, an infrared absorption filter to remove infrared light that is not contributing to the physical image and possibly degrading quality of readings taken by the image sensor (108). In one embodiment, the system (100) also includes a shutter mechanism (104). When the shutter (104) is open, the system (100) may receive light to the image sensor (108). In other embodiments, the shutter (104) may not be a physical means, but software where the photosensitive elements on the image sensor (108) may receive light only at intervals that are controlled by software. The camera (100) may also include firmware (114) on a circuit board (112) that includes software (116), for example, to control the shutter opening and closing and to manipulate the sensor to capture images.
  • Sensing elements are depicted capturing color data readings from light (202) falling on an image sensor in an image position (204) in FIG. 2, in which the same light rays (202) hit multiple different sensing elements at sequential time periods, for example Time0, Time1, and Time2, within a single exposure. Captured color data readings may include brightness, hue, saturation, intensity, or other data describing light. Rays of light (202) are falling in one collecting location of an image sensor position (204) at the beginning of a capture time. Then, the image sensor is moved by, for example, the width of one sensing element until another sensing element is receiving the light (202) to capture further data in a new image sensor position (206). Next, the image sensor position (204) is moved another length, for example, a width of a photosensitive element until an additional sensing element is receiving the light (202) to take another data reading in another image sensor position (206). In further embodiments, FIG. 2 includes additional sensor positions for data captures to other sensing elements included on the image sensor. Patterns of movement of the image sensor are further discussed in FIG. 3.
  • As an alternative to the movement of the image sensor in FIG. 2, reflective optics may be used. Instead of moving a series of optical sensing elements, adapted to collect multiple color data readings to calculate a color value for a pixel in an image, some reflective optics redirect light to the series of optical sensing elements successively within one exposure. In one embodiment, the series of optical sensing elements comprises a first element sensitive to red light, a second element sensitive to blue light, and a third element sensitive to green light. Specifically, the reflective optics may include a digital light processor including a digital micromirror device and control circuitry. In further embodiments, the reflective optics may include one or more mirrors to scan light to the series of sensing elements in conjunction with, for example, a digital light processor.
  • Turning to FIG. 3, some patterns (302, 304) of movement of an image sensor (300) during data captures of an image, for example three captures, are illustrated. In one embodiment, each sensing element of the image sensor (300) captures different information; for example, red, blue, or green. The sensing elements may each include a light sensor to detect luminance and a color filter. Thus, sensing elements featuring a red sensing element, blue sensing element, or green sensing element may incorporate a light sensor and a red filter, blue filter, or green filter to capture and indicate an amount of red light, blue light, or green light, respectively. In further embodiments, sensing elements may be sensitive to cyan, orange, yellow, magenta, or other colors.
  • In one embodiment, to detect and record three different data captures of the same light by a red sensing element, blue sensing element, and green sensing element, the image sensor (300) moves in a pattern (302, 304) between data captures by sensing elements. For example, FIG. 3 illustrates an L-shaped pattern (302). In the pattern (302), a blue sensing element captures data first; then, a red sensing element captures data; and lastly, a green sensing element captures data. Another pattern (304), shaped like a backwards P is also shown to be used in other embodiments. In the pattern (304), data is captured sequentially by a green sensing element, blue sensing element, and then a red sensing element.
  • As the image sensor is moved between data captures, sensing elements move between various collecting locations during an exposure. The sensing elements at the edge of the image sensor also move in and out of collecting locations and do not necessarily capture the same number of data readings. Thus, the patterns (304, 306) of the image sensor do not necessarily provide the same number of multiple captures for every edge sensing element as for interior sensing elements of the image sensor. One embodiment accounts for this difference with extra rows and columns to form an outline of sensing elements around the image sensor. Within the outline, sensing elements do not necessarily equally contribute to multiple data readings to supply pixels in an image and may be ignored from determining the pixels. An additional embodiment to account for the possible difference of numbers of edge and interior sensing elements' captures includes additional captures, for example, in the case of aiming for three data readings per sensing element, instead of three captures, six captures may occur: three in a pattern, then an additional one on the edge moved away from last, and an additional two on the edge moved away from first. Another embodiment to account for the possible difference of numbers of edge and interior sensing elements' captures includes interpolating the color values of the edge sensing elements from the interior sensing elements. In yet another embodiment, edge sensing elements with one value may produce pixels that are mere duplicates of pixels produced by the nearest interior sensing element.
  • In several embodiments, the image sensor (300) provides a solution for situations in which a sensing element is defective. First, a sensing element is found to be defective by one of several ways. In one embodiment, the image sensor (300) may perform a self-test to individually address each sensing element to simulate a burst of light. A sensing element may be labeled defective if it fails to respond. In another embodiment, a system (100) such as in FIG. 1 might take a test exposure simulating no light entering the lens. Any specks of color that diminish a pure black image would indicate those associated sensing elements are defective. Next, a pure white image is simulated and any sensing elements detracting from the image are defective. In a further embodiment, the image sensor (300) runs a test run and queries the user to respond that all of the sensing elements are functioning, by showing, for example, a red lattice, a blue lattice, and then a green lattice.
  • With the knowledge that a sensing element is defective, the image sensor (300) may alter its movement of data collection. For example, if the green sensing element of the pattern (302) is defective, the pattern (302) may be altered so that the image sensor (300) moves in the linear pattern from the blue sensing element to the red sensing element of the pattern (302) and continuing in the same direction to include the next sequential green sensing element instead. In this way, the image captured does not lack any information that it would have included if the original green sensing element had been fully functional. As another example, if the image sensor (300) is following the pattern (304), and the red sensing element becomes defective, the pattern (304) could be diverted to the red sensing element on the right after the blue sensing element. In another embodiment, software such as software (116) in FIG. 1, may add additional data captures to accommodate completing the gathering of multiple data readings per sensing element, for example four captures.
  • FIG. 3B depicts examples of conversions of sensing element readings to pixel information in an output image. FIG. 3B includes sensing elements (350, 352, 354, 360, 362, 364, 370, 372, 374, 380, 382, 384) to capture multiple data readings in accordance with the present invention. In one embodiment, the sensing elements (350, 352, 354, 360, 362, 364, 370, 372, 374, 380, 382, 384) capture three color readings, an amount each of red, blue, and green. The red, green, and blue values for each pixel are determined at the time of capture; thus, in displaying an image, each element (350, 352, 354, 360, 362, 364, 370, 372, 374, 380, 382, 384) may be directly associated with one or more pixel positions without further interpolation calculation.
  • In another embodiment, in displaying an image, groups of three sensing elements (350, 352, 354), (360, 362, 364), (370, 372, 374), (380, 382, 384) may be associated, and their data readings combined to provide several readings to interpolate a color value for resulting pixels (355), (365), (375), (385), for example. In this example, three elements are associated with one pixel position; however, any number of elements may be associated with any number of pixel positions.
  • FIG. 4 depicts a block diagram of an embodiment of internal software and hardware of a camera (100) to capture a representation of a visual image. In this embodiment, light (402) enters through the lens (102) and falls on the image sensor (108) which includes an array of sensing elements (404). In some embodiments, these sensing elements (404) include different types of sensing elements that are checkered within a two-dimensional array. A motor (110) is also included to move the image sensor (108) between captures.
  • Camera (100) also includes a processor (408), memory (412), input/output (I/O) (416), flash card (418), and a system bus (406) to connect the camera's components. The processor (408) includes color calculating logic (410) within it to process data captures and to convert data captures from the sensor into pixel information. In one embodiment, the memory (412) includes instructions (414) to move the image sensor (108) with the motor (110) between multiple color readings. These instructions provide functionality for the motor (110) to move the image sensor (108) so sensing elements (404) line up in the same collecting location for data readings within a single exposure. Additionally, these instructions provide functionality for the image sensor (108) itself to capture multiple data readings per exposure. The I/O (416) manages receiving instructions from a user, importing an image from a flash card (418), capturing data from the image sensor (108) to transfer to the processor (408), and outputting a picture output (420) on an output pin or pins, and storing an image on the flash card (418).
  • In one embodiment, the internals of the camera (100) may be implemented with CMOS logic, for example, to provide the ability for the camera to function, turn on, expose the shutter, convert data captures from the sensor into pixel information, or write a resulting image file out to a compact flash card. In another embodiment, the image sensor may be implemented on a separate charged coupled device (CCD) integrated circuit chip. In a further embodiment, the internals of the camera (100) and the image sensor (108) are implemented on separate CMOS chips within the same camera (100) or other imaging system.
  • Turning to FIG. 5, a block diagram depicts an embodiment of an image sensor (500) implemented as a CMOS chip. The CMOS sensor chip (500) includes memory (506) which is typically a rectangular array of temporary storage RAM. A photosensitive array (502) of sensing elements is also included as well as some control circuitry (504) which may be amplifiers or register addressing circuitry or both around the outside of the photosensitive array (502) to provide an ability for information to be read out of the array (502). The CMOS sensor chip (500) also includes a portion whose function is I/O (510) to provide access to chip information. Additionally, the CMOS sensor chip (500) may include, hardwired into its circuit, certain logic functions (508) to repeatedly perform. For example, the logic (508) may contain logic to move the motor (110) or send data captured by the photosensitive array (502) to I/O (510). In one embodiment, an algorithm to combine data captured by the photosensitive array (502) into a pixel array may be hardwired into the circuitry of this CMOS sensor chip (500). In another embodiment, the algorithm is soft-wired in flash ROM (not shown) on the CMOS sensor chip (500). In a further embodiment, the algorithm is not stored on this CMOS sensor chip, but stored externally in a magnetic medium, flash memory, ROM, or the like, elsewhere in the camera (100). In this embodiment, the algorithm may be loaded into the CMOS sensor chip (500) when it is powered up.
  • Turning to FIG. 6, an example of a flow chart illustrates a method to capture a representation of a visual image. Method (600) begins with element 602, during which an imaging system, such as a camera, determines whether any of its sensing elements are defective. If any sensing elements are defective, then the imaging system adjusts its software to redirect its image sensor motion to include a replacement non-defective element (element 604).
  • After either no sensing elements are found defective or software is completed being adjusted to accommodate for defective sensing elements, color readings are taken at a collecting location with a sensing element of an array (element 606). In one embodiment, the collecting location refers to a static physical position to receive a designated light falling in that location. In some embodiments, the array is moved between readings by the width of one sensing element to allow a new sensing element to analyze the same light and thus receive the light in the collecting location (element 608). The collecting location may be defined relative to the ray of light and refer to the location where optics focus rays of light on the image sensor. In one embodiment, while the image sensor is static, the reflective optics is able to redirect light to sensing elements successively within one exposure. For example, a digital light processor including a digital micromirror device and control circuitry may redirect light to the sensing elements successively within one exposure. For as many elements and readings from the collecting position are chosen, element 610 follows back to element 606.
  • After the data readings of color and luminance are taken, the captured information is used to create image information. Mathematical algorithms may be performed to convert the data readings to pixel information (element 612). Then, an image is outputted (element 614). For example, the pixel information may be stored to memory or sent out on a bus to another memory location or coupled device.
  • One embodiment of the invention is implemented as a program product for use with an image system such as, for example, as shown in FIG. 1. The program(s) of the program product defines functions of the embodiments (including the methods described herein) and can be contained on a variety of signal-bearing media. Illustrative signal-bearing media include, but are not limited to: (i) information permanently stored on non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive); (ii) alterable information stored on writable storage media (e.g., floppy disks within a diskette drive or hard-disk drive); and (iii) information conveyed to a computer by a communications medium, such as through a computer or telephone network, including wireless communications. The latter embodiment specifically includes information downloaded from the Internet and other networks. Such signal-bearing media, when carrying computer-readable instructions that direct the functions of the present invention, indicate embodiments of the present invention.
  • In general, the routines executed to implement the embodiments of the invention, may be part of an operating system or a specific application, component, program, module, object, or sequence of instructions. The computer program of the present invention typically is comprised of a multitude of instructions that will be translated by the native computer into a machine-readable format and hence executable instructions. Also, programs are comprised of variables and data structures that either reside locally to the program or are found in memory or on storage devices. In addition, various programs described hereinafter may be identified based upon the application for which they are implemented in a specific embodiment of the invention. However, it should be appreciated that any particular program nomenclature that follows is used merely for convenience, and thus the invention should not be limited to use solely in any specific application identified and/or implied by such nomenclature.
  • It will be apparent to those skilled in the art having the benefit of this disclosure that the present invention contemplates methods and arrangements for improved performance digital image sensing. It is understood that the form of the invention shown and described in the detailed description and the drawings are to be taken merely as examples. It is intended that the following claims be interpreted broadly to embrace all the variations of the example embodiments disclosed.

Claims (12)

1-7. (canceled)
8. A machine-readable medium containing instructions, which when executed by a machine, cause said machine to perform operations, comprising:
taking multiple color data readings with a series of sensing elements in one collecting location during a single exposure, wherein the taking comprises directing light successively to the sensing elements of the series of sensing elements within one exposure, the directing via reflective optics;
determining a sensing element of the series of sensing elements is defective, wherein the sensing element is associated with a color;
redirecting light to align a non-defective sensing element of the series of sensing elements with the one collecting location, wherein the non-defective sensing element is associated with the color;
associating the one collecting location with a pixel position in an image to be portrayed; and
determining a color value for the pixel position in the image based on the multiple color data readings.
9. (canceled)
10. The machine-readable medium of claim 8, wherein the instructions, which when executed by a machine, cause said machine to perform the taking comprise instructions, which when executed by a machine, cause said machine to perform taking multiple color data readings with more than one sensing elements, wherein the more than one sensing elements comprise color filters selected from a group of color filters comprising red, green, blue, cyan, orange, yellow, magenta, or clear.
11-12. (canceled)
13. A device, comprising:
a series of sensing elements in one collecting location, adapted to collect multiple color data readings to determine a color value for a pixel in an image
reflective optics to redirect light to the series of sensing elements successively within one Exposure; and
logic to determine that a light sensor of the plurality of light sensors is defective, the light sensor associated with a color, wherein the reflective optics is to redirect light to align a non-defective sensing element of the series of sensing elements with the one collecting location, wherein the non-defective sensing element is associated with the color.
14. The device of claim 13, wherein the series of sensing elements comprises a first element sensitive to red light, a second element sensitive to blue light, and a third element sensitive to green light.
15. The device of claim 13, wherein the reflective optics comprise a digital micromirror device and control circuitry to redirect the light to the series of sensing elements successively within the one exposure.
16. The device of claim 13, wherein the reflective optics include a mirror to scan light to the series of sensing elements.
17-21. (canceled)
22. The machine-accessible medium of claim 8, wherein the instructions, which when executed by a machine, cause said machine to perform determining a color value comprise instructions, which when executed by a machine, cause said machine to perform calculating the color value with at least one of the multiple color data readings.
23. The machine-accessible medium of claim 8, wherein the instructions, which when executed by a machine, cause said machine to perform directing comprise instructions, which when executed by a machine, cause said machine to perform directing light to the series of sensing elements successively within one exposure, the directing via a digital micromirror and control circuitry.
US12/339,543 2004-01-13 2008-12-19 Performance digital image sensing Abandoned US20090102953A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/339,543 US20090102953A1 (en) 2004-01-13 2008-12-19 Performance digital image sensing

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/755,888 US7483596B2 (en) 2004-01-13 2004-01-13 Performance digital image sensing
US12/339,543 US20090102953A1 (en) 2004-01-13 2008-12-19 Performance digital image sensing

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10/755,888 Continuation US7483596B2 (en) 2004-01-13 2004-01-13 Performance digital image sensing

Publications (1)

Publication Number Publication Date
US20090102953A1 true US20090102953A1 (en) 2009-04-23

Family

ID=34739692

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/755,888 Active 2026-07-20 US7483596B2 (en) 2004-01-13 2004-01-13 Performance digital image sensing
US12/339,543 Abandoned US20090102953A1 (en) 2004-01-13 2008-12-19 Performance digital image sensing

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/755,888 Active 2026-07-20 US7483596B2 (en) 2004-01-13 2004-01-13 Performance digital image sensing

Country Status (1)

Country Link
US (2) US7483596B2 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100091124A1 (en) * 2008-10-14 2010-04-15 International Business Machines Corporation Photo Sensor Array Using Controlled Motion
US20120275521A1 (en) * 2010-08-02 2012-11-01 Bin Cui Representative Motion Flow Extraction for Effective Video Classification and Retrieval
US20130293703A1 (en) * 2011-10-03 2013-11-07 Panasonic Corporation Imaging apparatus, and system and distance measuring device using imaging apparatus

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7483596B2 (en) * 2004-01-13 2009-01-27 International Business Machines Corporation Performance digital image sensing
DE102010041569B4 (en) * 2010-09-28 2017-04-06 Leica Geosystems Ag Digital camera system, color filter element for digital camera system, method for determining deviations between the cameras of a digital camera system and image processing unit for digital camera system
WO2013099917A1 (en) * 2011-12-28 2013-07-04 富士フイルム株式会社 Imaging device
EP2955913A1 (en) * 2014-06-10 2015-12-16 Thomson Licensing Hybrid plenoptic camera
EP3413557B1 (en) * 2017-06-08 2021-03-03 IMEC vzw A method, an image sensor and a device for acquiring an image of an object

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US2236430A (en) * 1938-04-30 1941-03-25 Proctor & Schwartz Inc Rug drier
US20050134712A1 (en) * 2003-12-18 2005-06-23 Gruhlke Russell W. Color image sensor having imaging element array forming images on respective regions of sensor elements
US7099002B2 (en) * 2001-06-21 2006-08-29 Ricoh Company, Ltd. Defect detector and method of detecting defect
US7483596B2 (en) * 2004-01-13 2009-01-27 International Business Machines Corporation Performance digital image sensing

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5291293A (en) * 1992-06-01 1994-03-01 Eastman Kodak Company Electronic imaging device with defect correction
US6236430B1 (en) * 1995-11-10 2001-05-22 Techno Media Co., Ltd. Color still image sensing apparatus and method
DE19980650T1 (en) * 1998-03-17 2000-07-20 Matsushita Electric Ind Co Ltd Image reading device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US2236430A (en) * 1938-04-30 1941-03-25 Proctor & Schwartz Inc Rug drier
US7099002B2 (en) * 2001-06-21 2006-08-29 Ricoh Company, Ltd. Defect detector and method of detecting defect
US20050134712A1 (en) * 2003-12-18 2005-06-23 Gruhlke Russell W. Color image sensor having imaging element array forming images on respective regions of sensor elements
US7483596B2 (en) * 2004-01-13 2009-01-27 International Business Machines Corporation Performance digital image sensing

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100091124A1 (en) * 2008-10-14 2010-04-15 International Business Machines Corporation Photo Sensor Array Using Controlled Motion
US8125545B2 (en) * 2008-10-14 2012-02-28 International Business Machines Corporation Photo sensor array using controlled motion
US8373794B2 (en) 2008-10-14 2013-02-12 International Business Machines Corporation Photo sensor array using controlled motion
US20120275521A1 (en) * 2010-08-02 2012-11-01 Bin Cui Representative Motion Flow Extraction for Effective Video Classification and Retrieval
US8995531B2 (en) * 2010-08-02 2015-03-31 Peking University Representative motion flow extraction for effective video classification and retrieval
US9268794B2 (en) 2010-08-02 2016-02-23 Peking University Representative motion flow extraction for effective video classification and retrieval
US20130293703A1 (en) * 2011-10-03 2013-11-07 Panasonic Corporation Imaging apparatus, and system and distance measuring device using imaging apparatus
US8994870B2 (en) * 2011-10-03 2015-03-31 Panasonic Intellectual Property Management Co., Ltd. Imaging apparatus, and system and distance measuring device using imaging apparatus

Also Published As

Publication number Publication date
US7483596B2 (en) 2009-01-27
US20050151861A1 (en) 2005-07-14

Similar Documents

Publication Publication Date Title
US20090102953A1 (en) Performance digital image sensing
CN101371591B (en) Image sensor with improved light sensitivity
US7151560B2 (en) Method and apparatus for producing calibration data for a digital camera
EP2087725B1 (en) Improved light sensitivity in image sensors
JP4833085B2 (en) A technique for correcting image field data by extrapolation.
CN102742279B (en) Iteratively denoising color filter array images
CN102369721B (en) CFA image with synthetic panchromatic image
US7626619B2 (en) Digital camera
CN102461174B (en) Color filter array pattern having four-channels
US20120105688A1 (en) Image processing device, imaging device, and image processing method and program
US20050128509A1 (en) Image creating method and imaging device
CN101390381B (en) Blur detecting device, blur correcting device, imaging device, and blur detecting method
KR100399884B1 (en) Apparatus and method for defective pixel concealment of image sensor
CN101755463A (en) Multiple component readout of image sensor
WO2009025825A1 (en) Image sensor having a color filter array with panchromatic checkerboard pattern
US20050275904A1 (en) Image capturing apparatus and program
CN110278360A (en) Image processing method and device, electronic equipment, computer readable storage medium
CN111131798B (en) Image processing method, image processing apparatus, and imaging apparatus
CN103460703B (en) Color image capturing element and image capturing device
CN104641276B (en) Camera head and signal processing method
EP2502422A1 (en) Sparse color pixel array with pixel substitutes
TWI458342B (en) Camera device and image processing device
US9019405B2 (en) Method and apparatus for wavelength specific correction of distortion in digital images
US20030223649A1 (en) Method for down-scaling a digital image and a digital camera for processing images of different resolutions
CN110428391A (en) To remove the image interfusion method and device of ghost illusion

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE