US20070097232A1 - Programmable anti-aliasing systems and methods for cameras - Google Patents
Programmable anti-aliasing systems and methods for cameras Download PDFInfo
- Publication number
- US20070097232A1 US20070097232A1 US11/261,275 US26127505A US2007097232A1 US 20070097232 A1 US20070097232 A1 US 20070097232A1 US 26127505 A US26127505 A US 26127505A US 2007097232 A1 US2007097232 A1 US 2007097232A1
- Authority
- US
- United States
- Prior art keywords
- image
- aliasing
- blur
- camera
- existing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
- H04N23/685—Vibration or motion blur correction performed by mechanical compensation
- H04N23/687—Vibration or motion blur correction performed by mechanical compensation by shifting the lens or sensor position
Definitions
- the described subject matter relates to cameras in general and more particularly to programmable anti-aliasing systems and methods for cameras.
- digital cameras are widely commercially available, ranging both in price and in operation from sophisticated single lens reflex (SLR) cameras used by professional photographers to inexpensive “point-and-shoot” cameras that nearly anyone can use with relative ease.
- SLR single lens reflex
- digital cameras include image capture electronics that convert light (or photons) into electrical charge. The electrical charge accumulated on each photo-cell (or pixel) is read out and used to generate a digital image of the scene being photographed.
- the resolution of digital cameras is limited by the number of pixels. If the camera does not have enough pixels to represent smooth lines, individual lines may appear jagged (e.g., as “stair-steps”) and/or the image as a whole may appear grainy.
- the color of individual pixels in areas of high-contrast and/or fine detail may be erroneously represented as the primary colors registered by the individual pixels and appear as “color spots” in the photograph.
- Anti-aliasing filters shift the light being exposed on the pixels to slightly “blur” the image and thereby reduce aliasing, e.g., by making jagged lines appear smooth or colors blend together.
- the degree of blur cannot be controlled by the user, and therefore may not provide the image sharpness the user desires for certain photographs.
- introducing more blur with the anti-aliasing filter may blur the image too much if the camera is moving or shaking.
- An exemplary embodiment of programmable anti-aliasing for cameras may be implemented as a system.
- the system may comprise anti-aliasing logic for characterizing at least one image at the camera.
- the system may also comprise an optical controller operatively associated with the anti-aliasing logic.
- the optical controller introduces varying degrees of blur during image capture operations by the camera based on the characterized image.
- programmable anti-aliasing for cameras may be implemented as a method, comprising: characterizing at least one image for existing aliasing, characterizing existing blur in the at least one image, and introducing blur during image capture operations by the camera based on the aliasing and existing blur in the at least one image.
- FIG. 1 is a high-level diagram of an exemplary camera system which may implement programmable anti-aliasing.
- FIG. 2 is a functional block diagram of exemplary anti-aliasing logic which may be implemented for cameras.
- FIG. 3 is a three-dimensional plot of exemplary data which may be implemented for programmable anti-aliasing in cameras.
- FIG. 4 is a flowchart illustrating exemplary operations which may implement programmable anti-aliasing for cameras.
- systems and methods of programmable anti-aliasing for cameras may be implemented to reduce the effects of aliasing, e.g., the prominence of jagged edges or color spots in digital images.
- the systems and methods described herein implement different degrees of anti-aliasing, e.g., based on movement of the camera, camera settings, and/or user preferences.
- Exemplary systems may include anti-aliasing logic which characterizes existing blur using a pre-image and adjusts the amount of aliasing for a particular image. For example, anti-aliasing may be increased if the camera is being held relatively still, and reduced if the camera is “shaking” or otherwise moving.
- FIG. 1 is a high-level diagram of an exemplary camera system which may implement programmable anti-aliasing.
- Camera systems include digital still-photo and video cameras, now known or that may be later developed.
- Exemplary camera system 100 may be provided with logic for characterizing existing blur in a pre-image, and for adjusting the degree of anti-aliasing based on the existing blur and optionally other factors (e.g., camera settings and/or user preferences).
- Exemplary camera system 100 may include a lens 120 positioned in the camera system 100 to focus light 130 reflected from one or more objects 140 in a scene 145 onto an image sensor 150 when a shutter 155 is open (e.g., for image exposure).
- Exemplary lens 150 may be any suitable lens which focuses light 130 reflected from the scene 125 onto image sensor 150 .
- Exemplary image sensor 150 may be implemented as a plurality of photosensitive cells, each of which builds-up or accumulates an electrical charge in response to exposure to light. The accumulated electrical charge for any given pixel is proportional to the intensity and duration of the light exposure.
- Exemplary image sensor 150 may include, but is not limited to, a charge-coupled device (CCD), or a complementary metal oxide semiconductor (CMOS) sensor.
- CCD charge-coupled device
- CMOS complementary metal oxide semiconductor
- Camera system 100 may also include image processing logic 160 .
- the image processing logic 160 receives electrical signals from the image sensor 150 representative of the light 130 captured by the image sensor 150 to generate a digital image of the scene 145 .
- Shutters, image sensors, and image processing logic, such as those illustrated in FIG. 1 are well-understood in the camera and photography arts. These components may be readily provided for camera system 100 by those having ordinary skill in the art after becoming familiar with the teachings herein, and therefore further description is not necessary.
- Camera system 100 may also include anti-aliasing logic 170 .
- Anti-aliasing logic 170 may be operatively associated with the image processing logic 160 .
- anti-aliasing logic 170 receives one or more pre-image from the image processing logic, and uses the pre-image(s) to make a determination as to how much, if any, anti-aliasing to apply for an image based at least in part on existing blur and aliasing in the pre-images.
- pre-image Using a pre-image enables the existing blur and aliasing to be determined independent of any knowledge about the lens quality, zoom, and/or aperture settings.
- the pre-image(s) may be one or more of the pre-images already being implemented by the camera for focusing, auto-exposure, pre-flash calculations, and/or pre-image(s) specifically for the purpose of determining the amount of existing blur.
- anti-aliasing is not limited to use with pre-images. For example, images captured by the user may also be used.
- anti-aliasing logic 170 may also receive input from a camera settings module 180 .
- Camera settings module 180 may include factory-configured and/or user-configured settings for the camera system 100 .
- factory-configured settings may include pre-stored information about lens resolution (or blur) at different combinations of zoom settings, aperture settings, and focus settings.
- Other exemplary factors may include, but are not limited to, user preferences (e.g., the desired image sharpness), camera mode (e.g., flash-on), operational mode (e.g., focal length), and/or the scene being photographed (e.g., light levels).
- anti-aliasing logic 170 may issue a signal to an optical controller 175 .
- the signal “programs” the optical controller 175 such that the desired amount of blur is introduced during the image capture operations to achieve the desired anti-aliasing.
- optical controller 175 may cause one or more optical element to move or shift during the image capture operation, thereby blurring the image.
- the programmable optical controller 175 may cause the lens 120 , the image sensor 150 , and/or other optical element(s) to move or shift during at least part of the exposure.
- the amount and time of movement or shifting will depend at least to some extent on design considerations, such as, e.g., which of the optical element(s) is/are being moved or shifted.
- the amount of movement or shifting may be a fraction of a pixel or more for all or a portion of the exposure time, and is based on the amount of blur to be introduced.
- the camera system 100 shown and described above with reference to FIG. 1 is merely exemplary of a camera system which may implement programmable anti-aliasing.
- the systems and methods described herein are not intended to be limited only to use with the camera system 100 .
- Other embodiments of cameras which may implement programmable anti-aliasing are also contemplated.
- FIG. 2 is a functional block diagram of exemplary anti-aliasing logic 200 which may be implemented for cameras (e.g., as the anti-aliasing logic 170 for camera system 100 shown in FIG. 1 ).
- Anti-aliasing logic 200 may be implemented to determine the amount, if any, blur that should be introduced during exposure of an image in order to achieve the desired anti-aliasing.
- anti-aliasing logic 200 includes a analyzer 210 .
- Analyzer 210 receives one or more pre-image 220 (e.g., from the image processing logic 160 described above with reference to FIG. 1 ).
- Analyzer 210 may also receive other input 225 , such as, but not limited to, factory-configured and/or user-configured camera settings, a user identity, and other information about the camera (e.g., focal length) and/or scene being photographed (e.g., ambient light levels).
- the analyzer 210 analyzes the pre-image(s) 220 , and optionally some or all of the other input 225 , to characterize existing blur and aliasing.
- the analyzer 210 may be implemented as a comparator. For example, if desired blur is Bd, and the un-modified expected blur is less than Bd, then the anti-aliasing logic 200 signals the optical controller to introduce enough blur to achieve Bd. If on the other hand, expected blur is greater than Bd, the anti-aliasing logic 200 disables the optical controller from inducing more blur.
- the analyzer may be implemented with a table (see, e.g., exemplary data in FIG. 3 ), where the table input is one or more of zoom, aperture, focus or other camera settings, and information from analysis of the pre-images, and the corresponding entry in the table is used to signal the optical controller to induce blur.
- Techniques for characterizing expected blur from one or more images are well-understood. These techniques generally involve comparing at least one picture element (pixel) in a first image (or video frame) with at least one pixel in a second image (or video frame) to discern a change in the scene between the two frames. This process may be repeated for successive pairs of frames to characterize image blur relative to the background of the scene in approximately real time. As applied to still cameras, these techniques may be performed on pre-images, e.g., as obtained for the video preview mode, focus, auto-exposure, and/or specifically for anti-aliasing.
- an image with little or no high-frequency information may exhibit little (if any) aliasing.
- the pre-image includes many high frequency edges with high contrast, then the image may exhibit significant aliasing.
- the pre-image(s) 220 may also be characterized for specific types of aliasing. For example, if there are near horizontal lines, there may be aliasing in the vertical direction, and blur may be introduced for anti-aliasing (e.g., by moving one or more optical element in the vertical direction). Likewise, if there are near vertical lines, there may be aliasing in the horizontal direction, and blur may be introduced for anti-aliasing (e.g., by moving one or more optical element in the horizontal direction).
- the pre-image may also be analyzed for color aliasing. For example, if the pre-image includes color spots, blur may be introduced for color anti-aliasing.
- Analyzer 210 may also be implemented to determine how much blur (if any) should be introduced during the image capture operations for the desired anti-aliasing based on the characterized pre-image(s) 220 .
- analyzer 210 may access one or more blur functions 230 for making this determination.
- Blur functions 230 may be stored in a data store in long-term and/or short-term memory in the camera. An exemplary blur function is described in more detail below with reference to FIG. 3 .
- a blur injection module 240 issues one or more signals 250 to program the optical controller.
- the signal 250 may include the amount of blur to be introduced and/or how the blur should be introduced.
- the signal 250 may indicate which optical element(s) should be moved or shifted, to what degree the optical element(s) should be moved or shifted, and/or start/stop/duration time for moving or shifting the optical element(s).
- FIG. 3 is a three-dimensional plot 300 of exemplary data which may be implemented for programmable anti-aliasing in cameras.
- Plot 300 includes an exemplary blur function 310 , i.e., the blur 320 to be introduced during image capture operations based on existing blur 330 and existing aliasing 340 in the pre-image(s).
- the blur function 310 may be predetermined for any of a wide variety of different conditions (e.g., camera settings, ambient light in the scene, camera motion, etc.). In another embodiment, blur function 310 may be at least partly user-defined and/or adaptable, e.g., generated and/or modified based on actual use of the camera and/or user feedback.
- the blur 320 to be introduced increases (arrow 322 ) as existing blur 330 decreases (arrow 332 ) and as existing aliasing 340 increases (arrow 342 ).
- the blur 320 to be introduced decreases (arrow 324 ) as existing blur 330 increases (arrow 334 ) and as existing aliasing 340 decreases (arrow 344 ).
- the blur function 310 may be implemented as follows to determine the amount of blur to introduce.
- the pre-image(s) are characterized for existing blur and existing aliasing. The intersection of these values is illustrated in plot 300 at point 350 , and represents the blur 320 to introduce based on the blur function 310 .
- the blur 320 to be introduced may be expressed as data for moving one or more optical element, such as, e.g., which optical element(s) to move, the frequency, and/or timing information for the movement.
- plot 300 is provided only for purposes of illustration and is not intended to be limiting.
- the blur function 310 is not limited to a three-dimensional plot, and may instead be expressed as a mathematical equation (or other suitable model) and/or data in any suitable data structure(s).
- the example illustrated in FIG. 3 is based on analyzing image blur and aliasing in real-time, but it is noted that adaptive data analysis models may also be implemented which base the decision at least in part on other data (e.g., camera settings and/or user input).
- FIG. 4 is a flowchart illustrating exemplary operations which may implement programmable anti-aliasing for cameras.
- Operations 400 may be embodied as logic instructions on one or more computer-readable medium in the camera. When executed on a processor at the camera, the logic instructions implement the described operations.
- the components and connections depicted in the figures may be used for programmable anti-aliasing.
- the process starts in operation 410 .
- the process may start every time a user depresses the shutter button to the S1 position to obtain an image on the LCD prior to taking a picture of an image.
- the process may start after the image has been brought into focus.
- the process may start only if one or more predetermined criteria have been satisfied (e.g., the anti-aliasing mode is selected by the user).
- the anti-aliasing process may also be deactivated automatically or manually by the user so that the process does not start in operation 410 .
- the process may be automatically deactivated, e.g., based on motion detected during pre-exposure.
- an image is captured.
- a pre-image used to focus the camera on the scene may be used, or a separate image may be captured for the anti-aliasing process.
- the image is analyzed in operations 430 and 435 .
- the image is characterized for existing blur.
- blur may be due to intentional camera movement or unintentional camera movement (e.g., shaking by the user).
- the image is characterized for existing aliasing. Different degrees of aliasing, including little or no aliasing, may exist in the image. For example, aliasing may due to the focal length of the camera.
- one or more type of aliasing may be introduced, such as, e.g., jagged lines and/or color spots.
- a determination may be made to repeat operations 420 - 430 / 435 . Additional images may be obtained in operation 420 and used to characterize blur (operation 430 ) and/or aliasing (operation 435 ) over time (e.g., using statistical models). If a determination is made not to repeat, or to stop repeating, operations may proceed.
- the amount of blur is determined for the desired anti-aliasing.
- the amount of blur needed may be based on the degree of aliasing in the pre-image (e.g., as determined in operation 435 ), and may be independent of the amount of blur already present.
- the amount of blur to introduce during the image capture operations is determined.
- the amount of blur to introduce may be the same as, or less than, the amount of blur needed to achieve the desired anti-aliasing effect (e.g., as determined in operation 450 ).
- the amount of blur to introduce during the image capture operations may be adjusted based on the amount of existing blur (e.g., as determined in operation 430 ).
- the image is captured while introducing blur (e.g., as determined in operation 460 ).
- blur may be introduced by issuing a control signal to program an optical controller to move or shift one or more optical element in the camera during image exposure.
- the control signal may identify which optical element(s) should be moved or shifted, to what extent the optical element(s) should be moved or shifted, and/or timing information for moving or shifting the optical element(s).
- operations shown and described herein are provided to illustrate exemplary embodiments of programmable anti-aliasing in cameras. It is noted that the operations are not limited to the ordering shown. For example, operations 430 and 435 may be implemented one before the other and/or simultaneously. In another example, operations 450 and 460 may be executed in reverse order or even simultaneously.
- operations may return from operation 450 and/or operation 460 to operation 420 to capture another pre-image if the previously captured images were not sufficiently characterized in operation 430 and/or operation 435 .
- operations may terminate and/or restart at any point in time, e.g., if the user focuses the camera on a different scene, or if the characterizations in operations 430 / 435 have otherwise become invalid.
Abstract
Description
- The described subject matter relates to cameras in general and more particularly to programmable anti-aliasing systems and methods for cameras.
- Conventional film and more recently, digital cameras, are widely commercially available, ranging both in price and in operation from sophisticated single lens reflex (SLR) cameras used by professional photographers to inexpensive “point-and-shoot” cameras that nearly anyone can use with relative ease. Unlike conventional film cameras, however, digital cameras include image capture electronics that convert light (or photons) into electrical charge. The electrical charge accumulated on each photo-cell (or pixel) is read out and used to generate a digital image of the scene being photographed.
- Generally, the resolution of digital cameras is limited by the number of pixels. If the camera does not have enough pixels to represent smooth lines, individual lines may appear jagged (e.g., as “stair-steps”) and/or the image as a whole may appear grainy. In addition, the color of individual pixels in areas of high-contrast and/or fine detail may be erroneously represented as the primary colors registered by the individual pixels and appear as “color spots” in the photograph. These undesired effects are known in the digital photography arts as aliasing (and color aliasing).
- Cameras are also available with anti-aliasing filters. Anti-aliasing filters shift the light being exposed on the pixels to slightly “blur” the image and thereby reduce aliasing, e.g., by making jagged lines appear smooth or colors blend together. However, the degree of blur cannot be controlled by the user, and therefore may not provide the image sharpness the user desires for certain photographs. In addition, introducing more blur with the anti-aliasing filter may blur the image too much if the camera is moving or shaking.
- An exemplary embodiment of programmable anti-aliasing for cameras may be implemented as a system. The system may comprise anti-aliasing logic for characterizing at least one image at the camera. The system may also comprise an optical controller operatively associated with the anti-aliasing logic. The optical controller introduces varying degrees of blur during image capture operations by the camera based on the characterized image.
- In another exemplary embodiment, programmable anti-aliasing for cameras may be implemented as a method, comprising: characterizing at least one image for existing aliasing, characterizing existing blur in the at least one image, and introducing blur during image capture operations by the camera based on the aliasing and existing blur in the at least one image.
-
FIG. 1 is a high-level diagram of an exemplary camera system which may implement programmable anti-aliasing. -
FIG. 2 is a functional block diagram of exemplary anti-aliasing logic which may be implemented for cameras. -
FIG. 3 is a three-dimensional plot of exemplary data which may be implemented for programmable anti-aliasing in cameras. -
FIG. 4 is a flowchart illustrating exemplary operations which may implement programmable anti-aliasing for cameras. - Briefly, systems and methods of programmable anti-aliasing for cameras may be implemented to reduce the effects of aliasing, e.g., the prominence of jagged edges or color spots in digital images. The systems and methods described herein implement different degrees of anti-aliasing, e.g., based on movement of the camera, camera settings, and/or user preferences.
- Exemplary systems may include anti-aliasing logic which characterizes existing blur using a pre-image and adjusts the amount of aliasing for a particular image. For example, anti-aliasing may be increased if the camera is being held relatively still, and reduced if the camera is “shaking” or otherwise moving.
- Exemplary System
-
FIG. 1 is a high-level diagram of an exemplary camera system which may implement programmable anti-aliasing. Camera systems include digital still-photo and video cameras, now known or that may be later developed.Exemplary camera system 100 may be provided with logic for characterizing existing blur in a pre-image, and for adjusting the degree of anti-aliasing based on the existing blur and optionally other factors (e.g., camera settings and/or user preferences). -
Exemplary camera system 100 may include alens 120 positioned in thecamera system 100 to focuslight 130 reflected from one ormore objects 140 in ascene 145 onto animage sensor 150 when ashutter 155 is open (e.g., for image exposure).Exemplary lens 150 may be any suitable lens which focuseslight 130 reflected from the scene 125 ontoimage sensor 150. -
Exemplary image sensor 150 may be implemented as a plurality of photosensitive cells, each of which builds-up or accumulates an electrical charge in response to exposure to light. The accumulated electrical charge for any given pixel is proportional to the intensity and duration of the light exposure.Exemplary image sensor 150 may include, but is not limited to, a charge-coupled device (CCD), or a complementary metal oxide semiconductor (CMOS) sensor. -
Camera system 100 may also includeimage processing logic 160. In digital cameras, theimage processing logic 160 receives electrical signals from theimage sensor 150 representative of thelight 130 captured by theimage sensor 150 to generate a digital image of thescene 145. - Shutters, image sensors, and image processing logic, such as those illustrated in
FIG. 1 , are well-understood in the camera and photography arts. These components may be readily provided forcamera system 100 by those having ordinary skill in the art after becoming familiar with the teachings herein, and therefore further description is not necessary. -
Camera system 100 may also includeanti-aliasing logic 170.Anti-aliasing logic 170 may be operatively associated with theimage processing logic 160. During operation,anti-aliasing logic 170 receives one or more pre-image from the image processing logic, and uses the pre-image(s) to make a determination as to how much, if any, anti-aliasing to apply for an image based at least in part on existing blur and aliasing in the pre-images. - Using a pre-image enables the existing blur and aliasing to be determined independent of any knowledge about the lens quality, zoom, and/or aperture settings. The pre-image(s) may be one or more of the pre-images already being implemented by the camera for focusing, auto-exposure, pre-flash calculations, and/or pre-image(s) specifically for the purpose of determining the amount of existing blur. Although examples are discussed herein with reference to a pre-image, it is noted that anti-aliasing is not limited to use with pre-images. For example, images captured by the user may also be used.
- In addition to characterizing the pre-images, other factors may also be considered for determining the amount of anti-aliasing. For example,
anti-aliasing logic 170 may also receive input from acamera settings module 180.Camera settings module 180 may include factory-configured and/or user-configured settings for thecamera system 100. For example, factory-configured settings may include pre-stored information about lens resolution (or blur) at different combinations of zoom settings, aperture settings, and focus settings. Other exemplary factors may include, but are not limited to, user preferences (e.g., the desired image sharpness), camera mode (e.g., flash-on), operational mode (e.g., focal length), and/or the scene being photographed (e.g., light levels). - If a determination is made to implement anti-aliasing for the image being photographed,
anti-aliasing logic 170 may issue a signal to anoptical controller 175. The signal “programs” theoptical controller 175 such that the desired amount of blur is introduced during the image capture operations to achieve the desired anti-aliasing. In response,optical controller 175 may cause one or more optical element to move or shift during the image capture operation, thereby blurring the image. - By way of example, the programmable
optical controller 175 may cause thelens 120, theimage sensor 150, and/or other optical element(s) to move or shift during at least part of the exposure. The amount and time of movement or shifting will depend at least to some extent on design considerations, such as, e.g., which of the optical element(s) is/are being moved or shifted. For example, the amount of movement or shifting may be a fraction of a pixel or more for all or a portion of the exposure time, and is based on the amount of blur to be introduced. - Before continuing, it is noted that the
camera system 100 shown and described above with reference toFIG. 1 is merely exemplary of a camera system which may implement programmable anti-aliasing. The systems and methods described herein are not intended to be limited only to use with thecamera system 100. Other embodiments of cameras which may implement programmable anti-aliasing are also contemplated. -
FIG. 2 is a functional block diagram of exemplaryanti-aliasing logic 200 which may be implemented for cameras (e.g., as theanti-aliasing logic 170 forcamera system 100 shown inFIG. 1 ).Anti-aliasing logic 200 may be implemented to determine the amount, if any, blur that should be introduced during exposure of an image in order to achieve the desired anti-aliasing. - In an exemplary embodiment,
anti-aliasing logic 200 includes aanalyzer 210.Analyzer 210 receives one or more pre-image 220 (e.g., from theimage processing logic 160 described above with reference toFIG. 1 ).Analyzer 210 may also receiveother input 225, such as, but not limited to, factory-configured and/or user-configured camera settings, a user identity, and other information about the camera (e.g., focal length) and/or scene being photographed (e.g., ambient light levels). - The
analyzer 210 analyzes the pre-image(s) 220, and optionally some or all of theother input 225, to characterize existing blur and aliasing. In an exemplary embodiment, theanalyzer 210 may be implemented as a comparator. For example, if desired blur is Bd, and the un-modified expected blur is less than Bd, then theanti-aliasing logic 200 signals the optical controller to introduce enough blur to achieve Bd. If on the other hand, expected blur is greater than Bd, theanti-aliasing logic 200 disables the optical controller from inducing more blur. Alternatively, the analyzer may be implemented with a table (see, e.g., exemplary data inFIG. 3 ), where the table input is one or more of zoom, aperture, focus or other camera settings, and information from analysis of the pre-images, and the corresponding entry in the table is used to signal the optical controller to induce blur. - Techniques for characterizing expected blur from one or more images are well-understood. These techniques generally involve comparing at least one picture element (pixel) in a first image (or video frame) with at least one pixel in a second image (or video frame) to discern a change in the scene between the two frames. This process may be repeated for successive pairs of frames to characterize image blur relative to the background of the scene in approximately real time. As applied to still cameras, these techniques may be performed on pre-images, e.g., as obtained for the video preview mode, focus, auto-exposure, and/or specifically for anti-aliasing.
- The comparison of pixels may be implemented in a variety of ways. For example, the magnitude of the pixel-by-pixel difference in brightness (luminance) may be computed. Alternatively, a pixel-by-pixel correlation (multiplication) may be performed. If the pixels compared are in corresponding locations in the two digital preview frames, an indication may be inferred that there is blur between the frames.
- Techniques for characterizing an image for aliasing are also well-understood. For example, an image with little or no high-frequency information may exhibit little (if any) aliasing. On the other hand, if the pre-image includes many high frequency edges with high contrast, then the image may exhibit significant aliasing.
- The pre-image(s) 220 may also be characterized for specific types of aliasing. For example, if there are near horizontal lines, there may be aliasing in the vertical direction, and blur may be introduced for anti-aliasing (e.g., by moving one or more optical element in the vertical direction). Likewise, if there are near vertical lines, there may be aliasing in the horizontal direction, and blur may be introduced for anti-aliasing (e.g., by moving one or more optical element in the horizontal direction). The pre-image may also be analyzed for color aliasing. For example, if the pre-image includes color spots, blur may be introduced for color anti-aliasing.
-
Analyzer 210 may also be implemented to determine how much blur (if any) should be introduced during the image capture operations for the desired anti-aliasing based on the characterized pre-image(s) 220. In an exemplary embodiment,analyzer 210 may access one or more blur functions 230 for making this determination. Blur functions 230 may be stored in a data store in long-term and/or short-term memory in the camera. An exemplary blur function is described in more detail below with reference toFIG. 3 . - If the determination is made to introduce at least some blur during the image capture process, a
blur injection module 240 issues one ormore signals 250 to program the optical controller. Thesignal 250 may include the amount of blur to be introduced and/or how the blur should be introduced. For example, thesignal 250 may indicate which optical element(s) should be moved or shifted, to what degree the optical element(s) should be moved or shifted, and/or start/stop/duration time for moving or shifting the optical element(s). - Other exemplary implementations of analyzing an image for anti-aliasing operations are known in the art. For example, co-owned U.S. patent application Ser. No. 10/637,956 filed on Aug. 8, 2003 of Matherson, et al. (Published on Feb. 10, 2005 as U.S. patent application Publication No. 2005/0030409) discloses methods and apparatus for generating data representative of an image and selecting a blur profile for anti-aliasing, and is hereby incorporated by reference for all that it discloses.
-
FIG. 3 is a three-dimensional plot 300 of exemplary data which may be implemented for programmable anti-aliasing in cameras.Plot 300 includes anexemplary blur function 310, i.e., theblur 320 to be introduced during image capture operations based on existingblur 330 and existingaliasing 340 in the pre-image(s). - The
blur function 310 may be predetermined for any of a wide variety of different conditions (e.g., camera settings, ambient light in the scene, camera motion, etc.). In another embodiment,blur function 310 may be at least partly user-defined and/or adaptable, e.g., generated and/or modified based on actual use of the camera and/or user feedback. - Generally, however, it is observed from the
plot 300 that theblur 320 to be introduced increases (arrow 322) as existingblur 330 decreases (arrow 332) and as existingaliasing 340 increases (arrow 342). Likewise, theblur 320 to be introduced decreases (arrow 324) as existingblur 330 increases (arrow 334) and as existingaliasing 340 decreases (arrow 344). - During operation, the
blur function 310 may be implemented as follows to determine the amount of blur to introduce. The pre-image(s) are characterized for existing blur and existing aliasing. The intersection of these values is illustrated inplot 300 atpoint 350, and represents theblur 320 to introduce based on theblur function 310. Although illustrated graphically inFIG. 3 , it is noted that theblur 320 to be introduced may be expressed as data for moving one or more optical element, such as, e.g., which optical element(s) to move, the frequency, and/or timing information for the movement. - Before continuing, it is noted that
plot 300 is provided only for purposes of illustration and is not intended to be limiting. For example, theblur function 310 is not limited to a three-dimensional plot, and may instead be expressed as a mathematical equation (or other suitable model) and/or data in any suitable data structure(s). In addition, the example illustrated inFIG. 3 is based on analyzing image blur and aliasing in real-time, but it is noted that adaptive data analysis models may also be implemented which base the decision at least in part on other data (e.g., camera settings and/or user input). - Exemplary Operations
-
FIG. 4 is a flowchart illustrating exemplary operations which may implement programmable anti-aliasing for cameras.Operations 400 may be embodied as logic instructions on one or more computer-readable medium in the camera. When executed on a processor at the camera, the logic instructions implement the described operations. In an exemplary embodiment, the components and connections depicted in the figures may be used for programmable anti-aliasing. - The process starts in
operation 410. For example, the process may start every time a user depresses the shutter button to the S1 position to obtain an image on the LCD prior to taking a picture of an image. Alternatively, the process may start after the image has been brought into focus. In still another example, the process may start only if one or more predetermined criteria have been satisfied (e.g., the anti-aliasing mode is selected by the user). - It is noted that the anti-aliasing process may also be deactivated automatically or manually by the user so that the process does not start in
operation 410. For example, it may be desirable to deactivate anti-aliasing if the user is photographing a moving subject, or panning a scene. In an exemplary embodiment, the process may be automatically deactivated, e.g., based on motion detected during pre-exposure. - In
operation 420, an image is captured. For example, a pre-image used to focus the camera on the scene may be used, or a separate image may be captured for the anti-aliasing process. In any event, the image is analyzed inoperations - In
operation 430, the image is characterized for existing blur. For example, blur may be due to intentional camera movement or unintentional camera movement (e.g., shaking by the user). Inoperation 435, the image is characterized for existing aliasing. Different degrees of aliasing, including little or no aliasing, may exist in the image. For example, aliasing may due to the focal length of the camera. In addition, one or more type of aliasing may be introduced, such as, e.g., jagged lines and/or color spots. - In
operation 440, a determination may be made to repeat operations 420-430/435. Additional images may be obtained inoperation 420 and used to characterize blur (operation 430) and/or aliasing (operation 435) over time (e.g., using statistical models). If a determination is made not to repeat, or to stop repeating, operations may proceed. - In
operation 450, the amount of blur is determined for the desired anti-aliasing. The amount of blur needed may be based on the degree of aliasing in the pre-image (e.g., as determined in operation 435), and may be independent of the amount of blur already present. Inoperation 460, the amount of blur to introduce during the image capture operations is determined. The amount of blur to introduce may be the same as, or less than, the amount of blur needed to achieve the desired anti-aliasing effect (e.g., as determined in operation 450). For example, the amount of blur to introduce during the image capture operations may be adjusted based on the amount of existing blur (e.g., as determined in operation 430). - In
operation 470, the image is captured while introducing blur (e.g., as determined in operation 460). For example, blur may be introduced by issuing a control signal to program an optical controller to move or shift one or more optical element in the camera during image exposure. The control signal may identify which optical element(s) should be moved or shifted, to what extent the optical element(s) should be moved or shifted, and/or timing information for moving or shifting the optical element(s). - The operations shown and described herein are provided to illustrate exemplary embodiments of programmable anti-aliasing in cameras. It is noted that the operations are not limited to the ordering shown. For example,
operations operations - Still other operations may also be implemented. For example, operations may return from
operation 450 and/oroperation 460 tooperation 420 to capture another pre-image if the previously captured images were not sufficiently characterized inoperation 430 and/oroperation 435. In addition, operations may terminate and/or restart at any point in time, e.g., if the user focuses the camera on a different scene, or if the characterizations inoperations 430/435 have otherwise become invalid. - In addition to the specific embodiments explicitly set forth herein, other aspects and embodiments will be apparent to those skilled in the art from consideration of the specification disclosed herein. It is intended that the specification and illustrated embodiments be considered as examples only.
Claims (20)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/261,275 US7755678B2 (en) | 2005-10-28 | 2005-10-28 | Programmable anti-aliasing systems and methods for cameras |
TW095135984A TW200718186A (en) | 2005-10-28 | 2006-09-28 | Programmable anti-aliasing systems and methods for cameras |
JP2006294242A JP4385383B2 (en) | 2005-10-28 | 2006-10-30 | Programmable anti-aliasing system and method for a camera |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/261,275 US7755678B2 (en) | 2005-10-28 | 2005-10-28 | Programmable anti-aliasing systems and methods for cameras |
Publications (2)
Publication Number | Publication Date |
---|---|
US20070097232A1 true US20070097232A1 (en) | 2007-05-03 |
US7755678B2 US7755678B2 (en) | 2010-07-13 |
Family
ID=37995744
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/261,275 Expired - Fee Related US7755678B2 (en) | 2005-10-28 | 2005-10-28 | Programmable anti-aliasing systems and methods for cameras |
Country Status (3)
Country | Link |
---|---|
US (1) | US7755678B2 (en) |
JP (1) | JP4385383B2 (en) |
TW (1) | TW200718186A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130272609A1 (en) * | 2011-12-12 | 2013-10-17 | Intel Corporation | Scene segmentation using pre-capture image motion |
US8947521B1 (en) * | 2011-08-08 | 2015-02-03 | Kla-Tencor Corporation | Method for reducing aliasing in TDI based imaging |
US20170142347A1 (en) * | 2014-07-02 | 2017-05-18 | Sony Corporation | Image processing device, image processing method, and program |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2820007C (en) | 2010-09-09 | 2016-06-28 | John R. Kouns | Optical filter opacity control reducing temporal aliasing in motion picture capture |
US9274403B2 (en) * | 2013-03-13 | 2016-03-01 | Robert C. Weisgerber | Method for selectively imparting a cinematic appearance to motion pictures photographed and exhibited at high frame rates |
JP6422942B2 (en) | 2013-04-05 | 2018-11-14 | レッド.コム,エルエルシー | Optical filtering for cameras |
Citations (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6122447A (en) * | 1995-03-27 | 2000-09-19 | Canon Kabushiki Kaisha | Image blur prevention apparatus |
US6194695B1 (en) * | 1998-08-27 | 2001-02-27 | The United States Of America As Represented By The Secretary Of The Navy | Photoreceptor array for linear optical flow measurement |
US6297913B1 (en) * | 1998-01-05 | 2001-10-02 | Intel Corporation | Zoom lens system |
US6326998B1 (en) * | 1997-10-08 | 2001-12-04 | Eastman Kodak Company | Optical blur filter having a four-feature pattern |
US6486910B1 (en) * | 1997-03-12 | 2002-11-26 | Canon Kabushiki Kaisha | Photographing apparatus having the image blur preventing function |
US20030210251A1 (en) * | 2002-05-08 | 2003-11-13 | Brown Patrick R. | Arrangements for antialiasing coverage computation |
US20030214512A1 (en) * | 2002-05-14 | 2003-11-20 | Cheng Brett Anthony | Hardware method for sub-pixel anti-aliasing of text on an LCD display |
US20040113921A1 (en) * | 2002-12-16 | 2004-06-17 | Xerox Corporation | System and method of reducing aliasing in printed images |
US6801250B1 (en) * | 1999-09-10 | 2004-10-05 | Sony Corporation | Converting a multi-pixel image to a reduced-pixel image to provide an output image with improved image quality |
US20050030409A1 (en) * | 2003-08-08 | 2005-02-10 | Matherson Kevin J. | Method and apparatus for generating data representative of an image |
US6868190B1 (en) * | 2000-10-19 | 2005-03-15 | Eastman Kodak Company | Methods for automatically and semi-automatically transforming digital image data to provide a desired image look |
US20050093883A1 (en) * | 1999-01-28 | 2005-05-05 | Lewis Michael C. | Method and system for providing edge antialiasing |
US6900831B2 (en) * | 2000-01-20 | 2005-05-31 | Canon Kabushiki Kaisha | Image-sensing apparatus for selecting optical blur correction or electronic blur correction |
US6930708B1 (en) * | 1998-11-30 | 2005-08-16 | Ricoh Company, Ltd. | Apparatus and system for correction based upon detecting a camera shaking |
US20050248661A1 (en) * | 2004-05-10 | 2005-11-10 | Stanvely Donald J | Image-stabilization systems and methods |
US6987530B2 (en) * | 2001-05-29 | 2006-01-17 | Hewlett-Packard Development Company, L.P. | Method for reducing motion blur in a digital image |
US7167596B2 (en) * | 2001-01-26 | 2007-01-23 | Sony Computer Entertainment Inc. | Image processing method for generating three-dimensional images on a two-dimensional screen |
US7167600B2 (en) * | 2000-12-27 | 2007-01-23 | Sony Computer Entertainment Inc. | Drawing method for drawing image on two-dimensional screen |
US20070071346A1 (en) * | 2005-09-27 | 2007-03-29 | Fuji Photo Film Co., Ltd. | Method and apparatus for judging direction of blur and computer-readable recording medium storing a program therefor |
US7212215B2 (en) * | 2000-08-23 | 2007-05-01 | Sony Computer Entertainment Inc. | Apparatus and method for rendering an antialiased image |
US7253820B2 (en) * | 2000-11-15 | 2007-08-07 | Sony Corporation | Method and system for dynamically allocating a frame buffer for efficient anti-aliasing |
US20080037970A1 (en) * | 2006-08-11 | 2008-02-14 | Canon Kabushiki Kaisha | Camera body and camera system including the same |
US7385632B2 (en) * | 2004-01-30 | 2008-06-10 | Canon Kabushiki Kaisha | Lens, camera body and camera system |
US7408559B2 (en) * | 2005-12-01 | 2008-08-05 | Hewlett-Packard Development Company, L.P. | Upscaling of anti-aliased graphical elements |
US7460132B2 (en) * | 2005-04-28 | 2008-12-02 | Texas Instruments Incorporated | System and method for motion adaptive anti-aliasing |
US7515191B2 (en) * | 2003-08-12 | 2009-04-07 | Fujifilm Corporation | Digital camera and solid-state image pickup unit |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4656706B2 (en) | 2000-08-28 | 2011-03-23 | オリンパス株式会社 | Imaging device and focal length conversion device |
JP4409069B2 (en) | 2000-08-28 | 2010-02-03 | オリンパス株式会社 | Imaging apparatus and imaging condition calculation apparatus |
JP3601500B2 (en) | 2001-10-18 | 2004-12-15 | ミノルタ株式会社 | Digital camera |
-
2005
- 2005-10-28 US US11/261,275 patent/US7755678B2/en not_active Expired - Fee Related
-
2006
- 2006-09-28 TW TW095135984A patent/TW200718186A/en unknown
- 2006-10-30 JP JP2006294242A patent/JP4385383B2/en not_active Expired - Fee Related
Patent Citations (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6122447A (en) * | 1995-03-27 | 2000-09-19 | Canon Kabushiki Kaisha | Image blur prevention apparatus |
US6486910B1 (en) * | 1997-03-12 | 2002-11-26 | Canon Kabushiki Kaisha | Photographing apparatus having the image blur preventing function |
US6326998B1 (en) * | 1997-10-08 | 2001-12-04 | Eastman Kodak Company | Optical blur filter having a four-feature pattern |
US6297913B1 (en) * | 1998-01-05 | 2001-10-02 | Intel Corporation | Zoom lens system |
US6194695B1 (en) * | 1998-08-27 | 2001-02-27 | The United States Of America As Represented By The Secretary Of The Navy | Photoreceptor array for linear optical flow measurement |
US6930708B1 (en) * | 1998-11-30 | 2005-08-16 | Ricoh Company, Ltd. | Apparatus and system for correction based upon detecting a camera shaking |
US20050093883A1 (en) * | 1999-01-28 | 2005-05-05 | Lewis Michael C. | Method and system for providing edge antialiasing |
US6801250B1 (en) * | 1999-09-10 | 2004-10-05 | Sony Corporation | Converting a multi-pixel image to a reduced-pixel image to provide an output image with improved image quality |
US6900831B2 (en) * | 2000-01-20 | 2005-05-31 | Canon Kabushiki Kaisha | Image-sensing apparatus for selecting optical blur correction or electronic blur correction |
US7212215B2 (en) * | 2000-08-23 | 2007-05-01 | Sony Computer Entertainment Inc. | Apparatus and method for rendering an antialiased image |
US6868190B1 (en) * | 2000-10-19 | 2005-03-15 | Eastman Kodak Company | Methods for automatically and semi-automatically transforming digital image data to provide a desired image look |
US7253820B2 (en) * | 2000-11-15 | 2007-08-07 | Sony Corporation | Method and system for dynamically allocating a frame buffer for efficient anti-aliasing |
US7167600B2 (en) * | 2000-12-27 | 2007-01-23 | Sony Computer Entertainment Inc. | Drawing method for drawing image on two-dimensional screen |
US7167596B2 (en) * | 2001-01-26 | 2007-01-23 | Sony Computer Entertainment Inc. | Image processing method for generating three-dimensional images on a two-dimensional screen |
US6987530B2 (en) * | 2001-05-29 | 2006-01-17 | Hewlett-Packard Development Company, L.P. | Method for reducing motion blur in a digital image |
US20030210251A1 (en) * | 2002-05-08 | 2003-11-13 | Brown Patrick R. | Arrangements for antialiasing coverage computation |
US20030214512A1 (en) * | 2002-05-14 | 2003-11-20 | Cheng Brett Anthony | Hardware method for sub-pixel anti-aliasing of text on an LCD display |
US20040113921A1 (en) * | 2002-12-16 | 2004-06-17 | Xerox Corporation | System and method of reducing aliasing in printed images |
US20050030409A1 (en) * | 2003-08-08 | 2005-02-10 | Matherson Kevin J. | Method and apparatus for generating data representative of an image |
US7515191B2 (en) * | 2003-08-12 | 2009-04-07 | Fujifilm Corporation | Digital camera and solid-state image pickup unit |
US7385632B2 (en) * | 2004-01-30 | 2008-06-10 | Canon Kabushiki Kaisha | Lens, camera body and camera system |
US20050248661A1 (en) * | 2004-05-10 | 2005-11-10 | Stanvely Donald J | Image-stabilization systems and methods |
US7460132B2 (en) * | 2005-04-28 | 2008-12-02 | Texas Instruments Incorporated | System and method for motion adaptive anti-aliasing |
US20070071346A1 (en) * | 2005-09-27 | 2007-03-29 | Fuji Photo Film Co., Ltd. | Method and apparatus for judging direction of blur and computer-readable recording medium storing a program therefor |
US7408559B2 (en) * | 2005-12-01 | 2008-08-05 | Hewlett-Packard Development Company, L.P. | Upscaling of anti-aliased graphical elements |
US20080037970A1 (en) * | 2006-08-11 | 2008-02-14 | Canon Kabushiki Kaisha | Camera body and camera system including the same |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8947521B1 (en) * | 2011-08-08 | 2015-02-03 | Kla-Tencor Corporation | Method for reducing aliasing in TDI based imaging |
US9297769B1 (en) | 2011-08-08 | 2016-03-29 | Kla-Tencor Corporation | Method for reducing aliasing in TDI based imaging |
US20130272609A1 (en) * | 2011-12-12 | 2013-10-17 | Intel Corporation | Scene segmentation using pre-capture image motion |
US20170142347A1 (en) * | 2014-07-02 | 2017-05-18 | Sony Corporation | Image processing device, image processing method, and program |
US10148894B2 (en) * | 2014-07-02 | 2018-12-04 | Sony Corporation | Image processing device, image processing method, and program |
US11044419B2 (en) | 2014-07-02 | 2021-06-22 | Sony Corporation | Image processing device, imaging processing method, and program |
Also Published As
Publication number | Publication date |
---|---|
TW200718186A (en) | 2007-05-01 |
US7755678B2 (en) | 2010-07-13 |
JP2007124665A (en) | 2007-05-17 |
JP4385383B2 (en) | 2009-12-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8081220B2 (en) | Digital still camera and method of controlling image combination | |
KR101756839B1 (en) | Digital photographing apparatus and control method thereof | |
US7817186B2 (en) | Camera and image processing method for synthesizing plural images forming one image group to generate a synthesized image | |
US20080044170A1 (en) | Image Capturing System And Method Of Operating The Same | |
US20120120282A1 (en) | Reducing Temporal Aliasing | |
JP4551839B2 (en) | IMAGING DEVICE AND IMAGING DEVICE CONTROL METHOD | |
JP2007180631A (en) | Imaging apparatus and photographing method | |
US7705883B2 (en) | Systems and methods of anti-aliasing with image stabilizing subsystems for cameras | |
US7755678B2 (en) | Programmable anti-aliasing systems and methods for cameras | |
JP2016143035A (en) | Focus adjustment device and focus adjustment method | |
WO2015049990A1 (en) | Imaging device and imaging method | |
JP5618765B2 (en) | Imaging apparatus and control method thereof | |
JP4871664B2 (en) | IMAGING DEVICE AND IMAGING DEVICE CONTROL METHOD | |
JP2003158673A (en) | Image pickup device and method | |
JP2017009815A (en) | Focus detection device, focus detection method, and camera system | |
JP2006050139A (en) | Optical apparatus | |
JP2002252804A (en) | Electronic camera | |
JP4356585B2 (en) | Digital camera | |
JP2007139874A (en) | Imaging apparatus | |
JP2007060577A (en) | Imaging apparatus | |
JP3985448B2 (en) | Imaging apparatus, imaging method, and recording medium | |
JP2006148645A (en) | Image pickup device | |
JP2003107336A (en) | Image pickup device | |
JP2010166461A (en) | Imaging apparatus | |
JP6149345B2 (en) | Image capturing apparatus and image quality adjustment method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GORIS, ANDREW C.;STAVELY, DONALD J.;SIGNING DATES FROM 20051025 TO 20051027;REEL/FRAME:017167/0163 Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GORIS, ANDREW C.;STAVELY, DONALD J.;REEL/FRAME:017167/0163;SIGNING DATES FROM 20051025 TO 20051027 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.) |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.) |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20180713 |