WO2008089069A1 - Panoramic imaging techniques - Google Patents
Panoramic imaging techniques Download PDFInfo
- Publication number
- WO2008089069A1 WO2008089069A1 PCT/US2008/050869 US2008050869W WO2008089069A1 WO 2008089069 A1 WO2008089069 A1 WO 2008089069A1 US 2008050869 W US2008050869 W US 2008050869W WO 2008089069 A1 WO2008089069 A1 WO 2008089069A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- tile
- image
- candidate
- tiles
- goal
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
- H04N1/3876—Recombination of partial images to recreate the original image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
Definitions
- the present disclosure relates generally to image processing, and more specifically to techniques for forming a panoramic image.
- panoramic images cannot be created in a camera device or a camera phone device. Instead, when generating panoramic images from individual digital images having some common overlapping region, it is required to load the pictures to a personal computer (PC) where software for creating panoramic images can be used. This software is not suitable for use in a camera device having limited power, memory and processing capabilities.
- PC personal computer
- panoramic images are very time consuming and not user friendly. Oftentimes, the two images being used to create the panoramic image are rotationally offset. Thus, a panoramic image cannot be created or if one is created, it is visually unattractive.
- the panoramic image of FIG. 4 was created using a conventional process on the images FIGS. 3A and 3B. While the conventional process was able to merge the two images of FIGS. 3A and 3B, the seam is diagonal and in a bad place.
- a wireless device having a processor operative to stitch together a left image and a right image having overlapping regions is provided. The process also automatically corrects for any rotational differences between the left and right images before stitching to create a larger panoramic image.
- a memory coupled to the processor is included.
- a processor has a video processing unit operative to automatically correct for a rotational difference, above a predetermined limit, between first and second selected images having a common overlapping region.
- the video processing unit is operative to stitch the first image to the second image to create a larger panoramic image at a calculated stitching location when the rotational difference is below the predetermined limit.
- the video processing unit is operative to stitch the first image to a rotated second image at a rotated stitching location when the rotational difference is above the predetermined limit.
- a memory unit is coupled to the video processing unit
- a computer program product includes a machine- readable medium having instructions for causing a machine to find, automatically, an ideal stitching location in a second image to stitch a first image, the first and second images having a common overlapping region.
- the instructions also cause a machine to correct, automatically, the second image for rotational differences between the first and second images in relation to the ideal stitching location to form a new rotated ideal location.
- the instructions cause a machine to stitch, automatically, the first image to the second image to generate a larger panoramic image using the new rotated ideal location.
- FIGS. IA and IB illustrate a pair of left and right photographic images of a scene wherein a portion of the left and right photographic images overlap.
- FIG. 2 illustrates a wireless device.
- FIGS. 3A and 3B illustrate left and right photographic images of a group picture wherein a portion of the left and right photographic images overlap.
- FIG. 4 illustrates a single image representative of the left and right photographic images of FIGS. 3A and 3B using a conventional process.
- FIG. 5 illustrates a single image representative of the left and right photographic images of FIGS. 3 A and 3B using the process to create a panoramic image.
- FIGS. 6 A and 6B illustrate a flowchart of the process to create a panoramic image.
- FIG. 7 illustrates a flowchart of the process to find and mark the "most interesting" tile in an overlapping region of one of the photographic images of the left and right photographic images to be stitched.
- FIG. 8 illustrates the left photographic image of FIG. IA with an overlapping area L ove riap marked.
- FIG. 9 illustrates a flowchart of the process to downsize and find the tile or point t sta rt in the right photographic image of FIG. IB.
- FIG. 10 illustrates the right photographic image of FIG. IB with the tile or point t sta rt marked in a reduced overlapping region.
- FIGS. HA and HB illustrate the left and right photographic images of
- FIG. 12 illustrates two overlaid square tiles which are offset or rotated with respect to the other.
- FIG. 13 illustrates the array of R 0V eriap with a starting point indicated to begin a Modified A* Heuristic Search.
- FIG. 14 illustrates the array of R over i ap of FIG. 13 with a search using a first radius in the Modified A* Heuristic Search.
- FIG. 15 illustrates the array of Roveriap of FIG. 14 with an extension of the search using the first radius in the Modified A* Heuristic Search.
- FIG. 16 illustrates the array of R over i ap of FIG. 15 with a further extension of the search using the first radius which exceeds the OVERSHOOT threshold in the
- FIG. 17 illustrates the array of R ⁇ ,ve r ia p of FIG. 16 with a second search using a second radius in the Modified A* Heuristic Search.
- FIG. 18 illustrates the array of Rove rkp of FIG. 17 with a third search using a third radius in the Modified A* Heuristic Search.
- FIG. 19 illustrates the array of R ⁇ ap of FIG. 18 with an extension of the search using the third radius in the Modified A* Heuristic Search.
- FIG. 20 illustrates the array of Rove r ia p of FIG. 19 with a further extension of the search using the third radius in the Modified A* Heuristic Search.
- FIG. 21 illustrates the array of R over i ap of FIG. 20 with a fourth search using sub-tiling searching in the Modified A* Heuristic Search.
- FIG. 22 illustrates the graph of the cost function f(t) versus the order of expansion in the search using the Modified A* Heuristic Search.
- FIGS. 23A and 23B illustrate the left and right photographic images of
- FIG. 24 illustrates overlaid tiles of t en dPos of FIG. 23B at multiple rotations.
- FIG. 25A illustrates the left photographic image of FIG. 8 with t goal designated.
- FIG. 25B illustrates a rotated right photographic image of FIG. IB with tendPos and tmatch designated.
- FIG. 26A illustrates the t goa i tile.
- FIG. 26B illustrates the W tile.
- FIG. 26C illustrates the t en dp O s tile.
- FIG. 26D illustrates the t en dRot tile.
- FIG. 26E illustrates the t match tile.
- FIG. 27 illustrates a flowchart of the process for computing the alignment rotation for the right photographic image.
- FIG. 28 illustrates a flowchart of the process for stitching the left photographic to the right photographic image.
- FIG. 29 illustrates a general block diagram of a histogram array.
- FIG. 30 illustrates a panoramic image representative of the left and right photographic images of FIGS. IA and IB using the process to create a panoramic image.
- FIGS. 31A and 3 IB illustrate left and right photographic images of a scene at a restaurant wherein a portion of the left and right photographic images overlap and are rotationally offset from each other.
- FIG. 32 illustrates a panoramic image representative of the left and right photographic images of FIGS. 31A and 3 IB using the process to create a panoramic image.
- FIG. 33 illustrates a stitching module
- the techniques described herein may be used for wireless communication, computing, networking, personal electronics, etc.
- An exemplary use of the techniques for wireless communication is described below.
- FIG. 2 shows a block diagram of an embodiment of a wireless device 10 in a wireless communication system.
- the wireless device 10 may be a cellular or camera phone, a terminal, a handset, a personal digital assistant (PDA), or some other device.
- the wireless communication system may be a Code Division Multiple Access (CDMA) system, a Global System for Mobile Communications (GSM) system, or some other system.
- CDMA Code Division Multiple Access
- GSM Global System for Mobile Communications
- the wireless device 10 is capable of providing bi-directional communication via a receive path and a transmit path.
- signals transmitted by base stations are received by an antenna 12 and provided to a receiver (RCVR) 14.
- the receiver 14 conditions and digitizes the received signal and provides samples to a digital section 20 for further processing.
- a transmitter (TMTR) 16 receives data to be transmitted from the digital section 20, processes and conditions the data, and generates a modulated signal, which is transmitted via the antenna 12 to the base stations.
- the digital section 20 includes various processing, interface and memory units such as, for example, a modem processor 22, a video processor 24, a controller/ processor 26, a display processor 28, an ARM/DSP 32, a graphics processing unit (GPU) 34, an internal memory 36, and an external bus interface (EBI) 38.
- the modem processor 22 performs processing for data transmission and reception (e.g., encoding, modulation, demodulation, and decoding).
- the video processor 24 performs processing on video content (e.g., still images, moving videos, and moving texts) for video applications such as camcorder, video playback, and video conferencing.
- the controller/processor 26 may direct the operation of various processing and interface units within digital section 20.
- the display processor 28 performs processing to facilitate the display of videos, graphics, and texts on a display unit 30.
- ARM/DSP 32 may perform various types of processing for the wireless device 10.
- the graphics processing unit 34 performs graphics processing.
- the techniques described herein may be used for any of the processors in the digital section 20, e.g., the video processor 34.
- the internal memory 36 stores data and/or instructions for various units within the digital section 20.
- the EBI 38 facilitates the transfer of data between the digital section 20 (e.g., internal memory 36) and a main memory 40.
- the digital section 20 may be implemented with one or more DSPs, micro-processors, RISCs, etc.
- the digital section 20 may also be fabricated on one or more application specific integrated circuits (ASICs) or some other type of integrated circuits (ICs).
- ASICs application specific integrated circuits
- ICs integrated circuits
- the techniques described herein may be implemented in various hardware units.
- the techniques may be implemented in ASICs, DSPs, RISCs, ARMs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, and other electronic units.
- DSPDs digital signal processing devices
- PLDs programmable logic devices
- FPGAs field programmable gate arrays
- processors controllers, microcontrollers, microprocessors, and other electronic units.
- the wireless device 10 further comprises panoramic processing software that is stored in internal memory 36 or other computer-readable medium, for carrying out a process for stitching two photographic images together to create a single panoramic image, generally designated as 100, shown in FIGS. 6A-6B.
- the internal memory 36 may store still images, such as shown in FIGS. IA, IB, 3A, 3B and 31A-31B, photographic images or video clips therein which can be selected by the user for automatic processing using the panoramic processing software.
- the panoramic processing software upon execution is capable of seamlessly compositing together two photographic images taken of a scene from different perspectives, even different rotations, provided there is some common overlapping region.
- FIGS. 6A and 6B illustrate a flowchart of the process 100 to create a panoramic image.
- the process 100 stitches two photographic images (FIGS. IA and IB) together.
- the two photographic images (FIGS. IA and IB) are denoted, for illustrative purposes, as a left photographic image L in FIG. 8 and a right photographic image R in FIG. 10, given there exists a common overlapping region.
- the process 100 begins with step S 102 where the "most interesting" square tile is found and marked in a tight area of the common overlapping region of the left photographic image L.
- a square tile is a 128x128 pixel tile.
- the overlapping tight area in the left photographic image L is denoted as L ove riap.
- the process carried out in step S 102 is described in detail below in relation to FIG. 7.
- Step S 102 is followed by step S 104 where the right photographic image
- R is downsized in the 2 dimensions by 2X for a total of 4X (4 times) reduction, as best seen in FIG. 10.
- the common overlapping region in the right image is downsized by the same factor.
- the size of the right photographic image R is shown in the dotted line around the border.
- the reduced overlapping area R oVer i ap is generally the common overlapping region in the right image R and is an array of candidate square or pixel tiles. Each candidate square or pixel tile is essentially the same size as the squares in the area L ove riap.
- the reduced overlapping area Roveriap is shown as a 3x4 array 25 of candidate square tiles.
- a starting tile is found in the reduced overlapping area Roverkp having a similar (lowest) cost function f(t) derived from the histogram and sum of absolute difference (SAD) of the tile which most closely matches tile t goal .
- the square starting tile which most closely matches tile t goal is the tile denoted as 30.
- the center of the square tile 30 is marked as t star t.
- the center tstart is a "sweet spot” or ideal starting spot or tile for starting the Modified A* Heuristic Search at step S 106.
- a search is conducted to find a perfect or ideal stitching tile in the overlapping area R over i ap which is a match to the tile t goal .
- a circular footprint 40 (hereinafter "circular tile") within the boundaries of the square tile t goal is used to compare image hues in the images L and R in order to avoid pitfalls caused by rotation and difference in lighting between two photographic images.
- Step S 106 is followed by step S 108 where a determination is made whether a match is found during the Modified A* Heuristic Search. If the determination is "YES,” (meaning a match is found) step S 108 is followed by step SI lO where the amount of rotation (in degrees) or alignment rotation of the matching tile in the image R to match the tile t goal in the image L is computed. However, if the determination at step S 108 is "NO,” (meaning no match) then step S 108 returns to step S 106 to continue the Modified A* Heuristic Search.
- the phantom lines of step Si l l indicates the step is optional.
- Step Sl 12 is followed by step Sl 14.
- step Sl 14 a search is performed again to find a new location of the matching perfect or ideal stitching tile. However, the full square footprint of the ideal stitching tile is used. The search uses a SAD-only function.
- step Sl 16 where a determination is made whether a location is found. If the determination is "YES,” (meaning a location is found)
- step Sl 16 is followed by step Sl 18 where the left photographic image L is stitched together with the right photographic image R (if the alignment rotation is less than 3°) or the rotated photographic image R ro tated to create a larger, panoramic image (FIG. 30).
- step S 102 a flowchart of step S 102, the process to find and mark the "most interesting" square tile t goal , is shown.
- a predetermine tight area denoted as L ove riap in the left photographic image L which overlaps the right photographic image R is determined at step S 132 and divided into an array of square tiles 20 at step S 133.
- the predetermined tight area L over iap is size constrained.
- the width constraint of the area L ove riap is 1/5 of the left photographic image L beginning from the rightmost edge.
- the height constraint of the area L ove riap is 7/10 (0.7) of the height of the left photographic image.
- the area L over i ap extends .35 of the height in the image L in the up direction and .35 of the height in the image L in the down direction.
- the area L ove riap does not extend completely to the top and bottom edges of the left photographic image L and is generally centered about the median line or horizon.
- the constraints in width and height can be varied and are programmable.
- the area L over iap is divided into a plurality of square tiles, with each tile being the same pixel size.
- the size of the square tile is selected to be 128X128.
- the number of square tiles in the area L over i ap is eight (8), such tiles are arranged in the array 20 of square tiles.
- the array 20 of square tiles is 2x4.
- the number of tiles in or size of the array 20 is a function of the programmed parameters such as the tile size, width and height constraints, etc. and/or the overlap between the image L and R.
- step S 133 is followed by step S 134 where a search is conducted for a tile in the area L ove riap that is most unique such that: a) the histogram of the tile in the area L ove riap is most unique amongst neighbors and b) the histogram has the most even distribution of the Luma and Chroma (i.e. lots of color).
- step S 134 is followed by step S 136 where a determination is made whether the "most interesting" tile is found in comparison with all of its neighbors in the array 20. If the determination at step S136 is "YES,” then step S136 is followed by step S138 where the tile is marked or set to t goal .
- the tile t goal is shown marked in the first column, third row.
- the lines defining the array 20 are white and the tile t goal is black.
- the tile t goal is found so that it has a lot of detail and color.
- FIG. 8 illustrates the left photographic image of FIG. IA with an overlapping area L ove riap marked.
- the Chroma or color has a range between 32 to 256 and is programmable.
- Luma is the gray scale (brightness) of pixels.
- FIG. 9 illustrates a flowchart of the process of step S 104 to downsize and find the tile or point t star t in the right photographic image of FIG. IB.
- FIG. 10 illustrates the right photographic image of FIG. IB with the tile or point t start marked in a reduced overlapping region.
- the process of step S 104 begins with step S 152 where the right photographic image R is downsized two times in each dimension so it is 1 A of the original size or aspect ratio to form a temporary right image 22.
- Step S 152 is followed by step S 153 where the reduced overlapping area R oVer i ap (FIG. 13) in the temporary right image 22 is found.
- the reduced overlapping area R 0V eria P in the right image R is selected to be greater than (>) 1/2 of the width of the temporary right image 22.
- the width is selected to be 3/5 of the width of the temporary right image 22.
- the height is selected to be 100% of the temporary right image 22.
- the reduced overlapping area Roveriap extends to the right and bottom borders of the temporary right image 22.
- the overlapping area R 0V eriap is then divided into a plurality of candidate square tiles of equal size to form an array 25.
- the size of the candidate square tiles in the area Ro Ve riap is equal to the size of the square tiles in the area L ove riap.
- the array 25 has 12 square tiles arranged in a 3x4 array with all candidate square tiles having a corresponding overlapping or common image portion with the left photographic image L.
- Step S 153 is followed by step S 154 where a search for a tile in the area
- the cost function f(t) has two criteria to evaluate the candidate tile during the search which best looks like or matches t goa i-
- the two criteria include: (1) the histogram of the candidate tile which compares each candidate tile of the array 25 with the Luma of the tile t goal ; and (2) the SAD which best matches tile t goal .
- each candidate tile in the area Roveriap of the right photographic image R is compared to the tile t goal of the area L over iap in the left photographic image L.
- a pixel by pixel or point by point subtraction between the two tiles is performed.
- the result would be zero or black (meaning the two tiles are matched).
- the result of the subtraction would yield a bigger number (meaning the two tiles do not match).
- the SAD operation is similar to a minimization function to find which tiles of the candidate tiles in the array 25 have the most number of colors that match the tile t goal and which SAD is the best (the lowest SAD number).
- Step S 154 is followed by step S 156 where a determination is made whether the best tile in array 25 which matches the tile t goal (FIG. 26A) is found.
- the tile which matches the tile t goal is denoted as 30. If the determination is "YES,” step S 156 is followed by step S 158 where the center of the tile 30 is marked as t sta rt in the area R over i ap - However, if the determination is "NO,” the search in step S 154 is continued.
- FIGS. 8 and 10 one can readily see that the tile tgoai and the tile 30 having the t star t are relatively close.
- FIGS. HA and HB illustrate the left and right photographic images of
- FIGS. IA and IB with ct goal , ct start , and ct endPos designated.
- the Modified A* Heuristic Search is begun at the square tile t star t (FIG. 26B) using a circular tile therein denoted as ctstart in the area Roveriap.
- ctstart in the area Roveriap.
- the area Roveriap in FIG. 1 IB is the same area Roveriap in the downsized temporary image 22.
- the area Roveriap in FIG. HB is 256x256 with the circular tile ct start shown as a bold black circle.
- the circular tile found to be the ideal stitching tile denoted as ct en dPos in the area Roveriap at the end of the Modified A* Heuristic Search and is shown in as a black circle in the right image R.
- the Modified A* Heuristic Search calculates the cost function f(t) defined in equation Eq.(l) as
- f(t) wh a ii ⁇ histogram Diff a u(t,tgoal) + ws*locationDiff(t,tgoal) Eq.(l)
- the function histogramDiff a u(t,tgoal) compares each candidate circular tile's histogram array to the histogram array of ct goal ;
- the function locationDiff(t,tgoal) determines a SAD result between each candidate circular tile and the circular tile ct goal ;
- wh a ii is a weight between 0 and 1 ;
- ws is the weight on the SAD result between 0 and 1 ; and wh a ii plus ws equals 1.
- wh a u and ws are .5 so that the histogram result and the SAD result can be treated equally. Nevertheless, wh a u or ws may be weighted more than the other.
- the component wh a u ⁇ histogram Diff a u(t,tgoal) is made up of four (4) sub-functions described later.
- FIG. 29 illustrates a general block diagram of a histogram array 200.
- the histogram array 200 is populated with a set of numbers which corresponds to the color info of the tile under evaluation in the Modified A* Heuristic Search.
- the histogram array 200 contains values for Y (brightness or Luma) 202, Cr (red) 204 and Cb (blue) 206and a plurality of common Hues 208.
- the histogram array 200 has a first set of first values (e.g. 28 values) corresponding to the Luma data (Y) 202 in the tile; a second set of values (e.g. 28 values) corresponding to the Cb (how much blue) 206 in the tile; and a third set of values (e.g. 28 values) corresponding to the Cr (how much red) 204 in the tile.
- the Y, Cr and Cb values 202, 204 and 206 have a range of 0 to 255 which are scaled (e.g. by a scale factor of 3). More specifically, the histogram array 200 is populated with data corresponding to how many pixels in the tile have a Y, Cr and Cb of from 0-255.
- the Hues 208 may include red, green, blue, cyan, yellow, magenta, ..., black, gray and white in the HSV-like space.
- HSV-like space corresponds to the hue 208 A, saturation 208B and value 208C of the pixel.
- the Hue values 208 are computed for every pixel, the hue value being a color of a pixel. For example, red, pink, and dark red are the same hues of red. Green, dark and light green are hues of green.
- the Hue values 208 indicate whether a tile has any red in it and does not care about lighting. Thus, by adding Hue to the histogram function calculation, any variation due to lighting are essentially removed. More specifically, by adding the Hue values to the histogram function calculation eliminates the fact that tile t and tile t goal may be slightly different due to lighting.
- the histogram array 200 for the tile t goa i and the tile "t" under evaluation during the Modified A* Heuristic Search is computed. Then, the two histogram arrays for t goal and the tile "t” can be simply subtracted.
- the histogram array allows the pixels to be identified in terms of a given Y, given Cb and given Cr (e.g., if the image is black, the Y is zero, so all 128x128 pixels have a Y of zero).
- the histogram is arranged to show a spike in the histogram. The spike caused by slight lighting differences causes histogram peaks to misalign throwing a great match way off.
- each pixel is given Y, Cr and Cb values 202, 204 and 206.
- the Modified A* Heuristic Search uses circular tiles instead of square tile.
- the combination of Hue/Histogram/SAD for use in finding a perfect or ideal stitching tile t en dPos (FIG. 26C) or its circular counterpart ct en dPos reduces the effect of lighting variation between two images.
- FIG. 12 illustrates two overlaid square tiles which are offset or rotated with respect to the other.
- the top square tile is from the left photographic image L and the bottom square tile is from the right photographic image R.
- the overlaid perspective demonstrates the rotational offset between the tile of with the circular footprint 40 representing t goal in the left image, and a similar or identical tile in the right image.
- the Modified A* Heuristic Search looks at the circular center of the two tiles, generally, the same number of colors (i.e. hue, saturation, value) regardless of the fact that the two tiles are rotationally offset are found.
- rotationally offset is not a factor when counting the number of pixels of a certain color for two essentially identical tiles.
- FIG. 13 illustrates the array of R 0Ver i ap with a starting point indicated to begin a Modified A* Heuristic Search.
- the process of the Modified A* Heuristic Search will now be described in detail in relation to FIGS. 13-21.
- the search begins exploring neighboring tiles (set of candidate tiles) from the tile t star t or its circular counterpart ct start at a predetermined first radius.
- the explored tiles are ranked to find a lowest cost on the horizon (H) tile and the lowest cost ever found (LCEF) tile.
- the process of the Modified A* Heuristic Search keeps track of the H tiles and the LCEF tiles, denoted as E tiles. In some instances the H tile and the E tile may be the same.
- the Modified A* Heuristic Search explores additional candidates tiles in a new set using the same predetermined first radius from a current lowest cost on the horizon (H) tile to update the H tile for the new set and tracks a current LCEF tile for all candidate tiles explored thus far.
- the exploring continues at the predetermined first radius until the cost associated with any current H tile is greater than the cost associated with the current LCEF tile (tile E) multiplied by an OVERSHOOT value.
- the Modified A* Heuristic Search returns to the LCEF tile (tile E) and reduces the size of the radius.
- the Modified A* Heuristic Search then explores those neighboring (candidate tiles) in Roverkp from the current LCEF tile (tile E) at the reduced radius from the current LCEF tile.
- the search continues in a similar manner until the radius equal 0 (meaning there is only 1 pixel in any direction).
- subtiling exploration or sampling takes place, as best seen in FIG. 21, such that the cost function is evaluated based on pixels instead of tiles, because the circular tiles begin to overlap.
- the average photograph is 3 megapixels (i.e. 2048X1536).
- the 20x28 boxes in FIGS. 13-21 are only provided to cover an area sufficient to demonstrate the Modified A* Heuristic Search. As can be appreciated an area 2048X1536 is prohibitive.
- the dashed lines on the right of the R 0V eriap denote that the area continues.
- the OVERSHOOT can be best described as how much greater than the
- the OVERSHOOT may be set to 10%.
- the OVERSHOOT value would be 110%.
- OVERSHOOT is a number that is programmable.
- the area R 0V eriap is shown with a solid-line circle within a square denoted as CTI l.
- the circle CTI l is ct star t of FIG. HB determined to be the ideal starting spot in the area R over i ap to initiate the Modified A* Heuristic Search.
- the circular tile denoted as ct sta rt, is shown within a corresponding square in the area R 0V eriap.
- the square tiles the area Roveriap are 128x128 corresponding to a tile's width and the height (T width x Theight).
- the starting circular tile ct star t is also marked as the E tile and the H tile.
- the solid line of the starting circular tile ct start indicates that the tile is newly opened.
- Each tile has 128x128 pixels which are distributed over the "square area" or square shape of each tile.
- the right photographic image R will be rotated by a calculated amount. Therefore, the square shape of each tile is changed to a circular shape. Only those pixels within the circular shape or footprint defining the circular tile are evaluated. Hence, those pixels corresponding to the corners of a square tile are omitted.
- t goa i is 128x128 and ct goa i is the circular footprint used for evaluation of the tile t gOa i.
- FIG. 14 illustrates the array of Roveriap of FIG. 13 with a search using a first radius in the Modified A* Heuristic Search.
- the neighbors with a (first) radius SRl are shown.
- the radius SRl is 4.
- the neighbors create a plurality of candidate tiles to be evaluated during the first stage of the Modified A* Heuristic Search.
- the plurality of candidate tiles (8 neighbors) are labeled CT21, CT22, CT23, CT24, CT25, CT26, CT27 and CT28.
- the solid lines of these circles indicate the tiles are newly opened.
- the starting circular tile ct star t (CTl 1) is closed, as represented by the hatched circle (with diagonal lines) and a new tile (tile CT26) is labeled as both tile E and tile H (meaning found to be both the LCEF and the lowest cost on the horizon).
- FIG. 15 illustrates the array of Roveriap of FIG. 14 with an extension of the search using the first radius in the Modified A* Heuristic Search.
- the Modified A* Heuristic Search is continued or extended to explore additional candidate tiles with the radius SRl beginning from the tile labeled CT26 because the tile CT26 is the lowest cost on the horizon (H) tile.
- the newly opened neighbors are labeled CT31, CT32, CT33, CT34, CT35, and shown with a solid circle line.
- Those neighbors which were previously opened in FIG. 14 are shown as a dotted circle line and correspond to candidate tiles CT21, CT22, CT23, CT24, CT25, CT27 and CT28.
- the tile CT32 is determined to be the lowest cost on the horizon (H) tile.
- the tile CTl 1 has been closed.
- the tile CT26 is closed and remains the LCEF tile labeled E.
- the cost for the newly opened neighbors CT31, CT32, CT33, CT34, CT35 needs to be computed, because the cost for the previously opened tiles in FIG. 14 does not change.
- FIG. 16 illustrates the array of Roveriap of FIG. 15 with a further extension of the search using the first radius which exceeds the OVERSHOOT threshold in the Modified A* Heuristic Search.
- the exploration continues where a new candidate list of tiles is determined having the first radius from the new H tile CT32.
- there are 8 candidates from the new H tile CT32 but only 3 are new candidate tiles, labeled CT41, CT42 and CT43, and shown in a solid dotted line. All other previously opened tiles are shown in dotted lines.
- the LCEF tile (tile E) remains tile CT26.
- the candidate tile CT43 is determined to be the lowest cost on the horizon (H) tile and the tile CT32 is closed (represented by the hatched circle).
- the H tile CT43 has a cost that is greater than the cost associated with the current LCEF tile (tile E) multiplied by the OVERSHOOT value.
- the Modified A* Heuristic Search returns to the current LCEF tile (tile E) and the radius is reduced by a predetermined factor (e.g. a factor of 0.5).
- the current LCEF tile (tile E) is also closed.
- FIG. 17 illustrates the array of Roveriap of FIG. 16 with a second search using a second radius in the Modified A* Heuristic Search.
- the Modified A* Heuristic Search finds a new set of candidate tiles from the current LCEF tile (tile E) at the reduced current radius SR2 (e.g. radius of 2).
- the new set of candidate tiles are newly opened and shown in solid lines.
- This new set of candidate tiles are labeled CT51, CT52, CT53, CT54, CT55, CT56, CT57 and CT58.
- the cost associated with these candidate tiles is computed.
- the tile CT51 is determined to be the lowest cost on the horizon (H) tile.
- the H tile CT51 has a cost that is greater than the cost associated with the current LCEF tile multiplied by the OVERSHOOT value.
- the Modified A* Heuristic Search returns to the current LCEF tile (tile E) and the current radius further reduced by a predetermined factor (e.g. a factor of .5).
- FIG. 18 illustrates the array of Roveriap of FIG. 17 with a third search using a third radius in the Modified A* Heuristic Search.
- the Modified A* Heuristic Search finds a new set of candidate tiles from the current LCEF tile (tile E) at the reduced current radius (e.g. radius of 1).
- the new set of candidate tiles are newly opened and shown in solid lines.
- This new set of candidate tiles are labeled CT61, CT62, CT63, CT64, CT65, CT66, CT67 and CT68.
- the cost associated with each of these candidate tiles is computed.
- the tile CT61 is determined to be the lowest cost on the horizon (H) tile.
- FIG. 19 illustrates the array of Roveriap of FIG. 18 with an extension of the search using the third radius in the Modified A* Heuristic Search.
- the H tile CT61 is determined to be the new LCEF tile (tile E).
- the previous LCEF tile because closed and no longer the current LCEF tile, is shown as a hatched circle, but the "E" has been removed.
- a new set of candidate tiles at the current radius are determined to be candidate tiles CT71 and CT72.
- the lowest cost on the horizon (H) tile is tile CT72.
- the new current LCEF tile is determined to also be the tile CT72.
- a new set of candidate tiles CT81, CT82 and CT83 from the H tile CT72 are determined in FIG. 20.
- FIG. 20 illustrates the array of Roveriap of FIG. 19 with a further extension of the search using the third radius in the Modified A* Heuristic Search.
- the OVERSHOOT is evaluated using the new tiles CT81, CT82 and CT83 but, when evaluating for the OVERSHOOT, it is determined that the H tile is greater than the cost associated with the current LCEF tile multiplied by the OVERSHOOT value.
- the Modified A* Heuristic Search returns to the current LCEF tile (tile E) CT72 and the current radius is further reduced by a predetermined factor (e.g. a factor of .5). However, the radius is now less than one.
- subtiling exploration begins, as shown in FIG. 21.
- FIG. 21 illustrates the array of Roveriap of FIG. 20 with a fourth search using sub-tiling searching in the Modified A* Heuristic Search.
- the subtiling exploration is shown as overlapping circular tiles 50.
- the current and final LCEF tile (tile E) is CT72 and is labeled ct end pos in FIG. HB.
- the white circles correspond to at least the overlapping circular tiles 50 in FIG. 21.
- the current and final LCEF tile (tile E) is the perfect or ideal stitching spot to stitch the left photographic image to the right photographic image.
- the subtiling exploration would continue in a manner similar to that previously described for full circular tiles.
- the new H tile is a subtile in the top left corner from the current E tile and would be evaluated the same for the OVERSHOOT.
- Roveriap are compared with the circular tile ct goa i, on a pixel basis.
- the radius can be further reduced on a pixel basis until there is only one pixel left. However, it may be determined that shifting by one pixel, left, down, up, right, would not yield a better LCEF tile, at which time the search stops.
- FIG. 22 illustrates the graph of the cost function f(t) versus the order of expansion in the search using the Modified A* Heuristic Search.
- the cost function f(t) is calculated for the tiles A, B and C, denoted as points on a curve, with a first radius (e.g. radius of 4) and any other neighbors (candidate tiles) that fall in the curve with the points A, B and C.
- the evaluation of the cost function for other tiles is stopped because the cost function f(C) is greater than OVERSHOOT * f(B) where f(B) is the cost associated with the current LCEF tile.
- the search using the first radius is stopped.
- the radius is reduced in half and the Modified A* Heuristic Search continues.
- the cost function f(t) is calculated for the tiles B, D and E, denoted as points on a curve, and any other opened neighbors (candidate tiles) that fall in the curve with the points B, D and E.
- the evaluation of the cost function for other tiles is stopped because the cost function f(E) is greater than OVERSHOOT * f(D) where f(D) is the cost associated with the current the LCEF tile.
- the search using the second radius is stopped.
- the second radius is reduced in half and the Modified A* Heuristic Search continues.
- the cost function f(t) is calculated for the tiles D, F and G, denoted as point on a curve, and any other opened neighbors (candidate tiles) that fall in the curve with the points D, F and G.
- the evaluation of the cost function for other tiles is stopped because the cost function f(G) is greater than OVERSHOOT * f(F) where f(F) is the cost associated with the current LCEF tile.
- the tile F has the lowest cost.
- FIGS. 23A and 23B illustrate the left and right photographic images of
- FIG. 24 illustrates overlaid tiles of tendPos of FIG. 23B at multiple rotations.
- the tiles t goal , t endRot , and t endPos are shown as a square tile with a circular tile counterpart.
- FIG. 27 illustrates a flowchart of the process for computing the alignment rotation the alignment rotation at step SI lO (FIG. 6A) in the right image R.
- step S 172 a cost using a SAD-only function with the circular tile for tile t goa i is calculated at step S 172.
- step S 172 is followed by step S 174 where a determination is made whether an end of the rotation range is reached. If the determination is "NO,” then step S 174 is followed by step S 176 where the rotation (in degrees) is increased by a predetermined amount. Step S 176 loops back to step S 170. However, if the determination at step S 174 is "YES,” then step S 174 is followed by step S178. At step S178, the alignment rotation for the image R is computed.
- the rotation (in degrees) associated with the lowest SAD may be determined to be the alignment rotation. So that the ideal stitching spot or tile, denoted as t end p os in the area Roveriap best matches the tile t goal , the alignment rotation process when calculating the SAD uses the circular tile format to eliminate the corners in the SAD calculations.
- the circular tile ct en dpos is rotated in the range from -10 degrees to 10 degrees with 21 different rotations. Hence, the increment used in this embodiment is 1°. However, other increments may be used.
- FIG. 24 illustrates tile t goal overlaid over rotated tiles (shown in white) of FIG. 23B at different degrees.
- the final rotation of the tile t en dPos (shown in black) is labeled t en dRot (also shown in black).
- the alignment rotation is determined to be 7 degrees. This compensates or corrects for any rotational differences between the left and right images.
- the tile t en dRot is also shown in FIG. 26D.
- the tile tendPos is the final tile and can be set as the tile t matc h (FIG. 26E), described below. Thereafter, the stitching process can immediately begin if the best matching tile is tile tendPos which saves those additional computations associated with the rotation alignment process.
- the left image L is put on top of the (non-rotated) right image R with the tiles t goal and t matc h overlapping.
- the rotation alignment is greater or equal to 3°, the right image R preferable is rotated.
- FIG. 25A illustrates the left photographic image of FIG. 8 with t goal designated.
- FIG. 25B illustrates a rotated right photographic image of FIG. IB with tendPos and tmatch designated.
- the right image R is rotated by the amount of the alignment rotation and labeled R rotated -
- a SAD-only cost function using square tiles is used in a new Modified A* Heuristic Search.
- the tile t en dPos (FIG. 26C) at the new rotation can be labeled the current (rotated) LCEF tile from which to begin searching.
- the current (rotated) LCEF tile begins a new Modified A* Heuristic Search for a square tile tmatch (FIG. 26E) in the rotated right image R that best matches the tile t goal (FIG. 26A).
- the square tile t ma tch is shown as black.
- the tile t ma tch is the tile t en dPos-
- This tile t ma tch is the spot to stitch the left image L to the rotated right image R ro tated- [0107]
- FIG. 28 illustrates a flowchart of the process for stitching the left photographic to the right photographic image.
- step Sl 18 in FIG. 6B begins with step S 180 where the right image R or the rotated right image R ro tated is sent to the output.
- step S 180 is followed by step S 182 where the left image is overlaid on top of the right image R or the rotated right image R ro tated.
- step S 182 is followed by step S 184 where left image L is pasted at the location the tile t goal is stitched on the tile t match while blending the seam between the left and right images.
- alpha blending is used where the right 100 columns of left image L is blended smoothly with what is underneath (the right image R or the rotated right image R ro tated).
- step S 184 is followed by step S 186 (shown in phantom) where optionally the panoramic image is cropped, represented by the black rectangular line.
- the stitching module 300 includes an overlaying sub-module 302 for performing the functions of step S 182 and a blending sub-module 304 for performing the functions of step Sl 84.
- the stitching module 300 further includes a seam selection sub-module 306 which includes a skin tone detection sub-module 308 and an object detection sub-module 310. Before or during blending of step S 184, the skin tone detection and object detection by sub-modules 308 and 310, respectively, may be performed.
- a cropping sub-module 312 is provided to crop the panoramic image.
- the seam may be stitched in a variety of manners or patterns.
- a straight line can be used as the seam pattern to stitch the seam.
- the seam pattern may be a wavy line to minimize cutting or stitching through an important object in the scene detected by the object detection sub-module 310. Nevertheless, other seam patterns may be employed.
- the seam pattern is selected by the seam selection sub- module 306 and may be done automatically.
- a seam preferably would not fall on someone's face in the panoramic image, as shown in FIG. 4.
- a skin tone detection algorithm is employed by the skin tone detection sub-module 308 to prevent placing a seam on someone's face in the images.
- the pixel colors used in the searches can be used to detect skin or flesh tone.
- FIGS. 31A and 3 IB illustrate left and right photographic images of a scene at a restaurant wherein a portion of the left and right photographic images overlap. These images are also at different rotations.
- the resultant larger panoramic image representative of the left and right photographic images of FIGS. 31A and 3 IB using the process 100 is shown in FIG. 32.
- the seam falls between two people and not over their faces.
- the conventional process could not create a panoramic image using two images of FIGS. 31A and 3 IB at different rotations.
- the embodiment described herein provides rotation-independent image stitching by finding and correcting for rotational differences prior to stitching.
- FIG. 5 the resultant panoramic image using the exemplary embodiment described herein using the left and right photographic images of FIGS. 3A and 3B is shown.
- the seam does not cut through the face of a person as did the conventional process shown in FIG. 4.
- the embodiment described herein provides a wireless device 10 capable of seamlessly compositing together two images taken of a scene from different perspectives or rotations provided there is some common overlapping region.
- the wireless device 10 is also capable creating a panoramic image in the camera device with a relatively small increase in memory.
- the wireless device 10 is capable of optimizing the process 100 for creating a panoramic image by eliminating the rotation correction when the rotation between the two images is below a threshold or lower limit. Moreover, the process compensates for lighting difference between the two images.
- the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium or machine-readable medium.
- Computer-readable media or machine-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
- a storage media may be any available media that can be accessed by a computer.
- Such computer-readable media or machine-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium or machine-readable medium.
- Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer- readable media or machine-readable media.
- a camera device or other wireless device having a camera can be also used by a personal computer (PC), laptop, or other computing device.
- PC personal computer
- laptop or other computing device.
Abstract
Description
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2009545703A JP2010516003A (en) | 2007-01-12 | 2008-01-11 | Panorama image technology |
EP08727584A EP2103101A1 (en) | 2007-01-12 | 2008-01-11 | Panoramic imaging techniques |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/623,050 | 2007-01-12 | ||
US11/623,050 US8331725B2 (en) | 2007-01-12 | 2007-01-12 | Panoramic imaging techniques |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2008089069A1 true WO2008089069A1 (en) | 2008-07-24 |
WO2008089069A9 WO2008089069A9 (en) | 2008-10-09 |
Family
ID=39414959
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2008/050869 WO2008089069A1 (en) | 2007-01-12 | 2008-01-11 | Panoramic imaging techniques |
Country Status (7)
Country | Link |
---|---|
US (1) | US8331725B2 (en) |
EP (1) | EP2103101A1 (en) |
JP (1) | JP2010516003A (en) |
KR (1) | KR20090099580A (en) |
CN (1) | CN101578851A (en) |
TW (1) | TW200903345A (en) |
WO (1) | WO2008089069A1 (en) |
Families Citing this family (41)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB0708676D0 (en) * | 2007-05-04 | 2007-06-13 | Imec Inter Uni Micro Electr | A Method for real-time/on-line performing of multi view multimedia applications |
US8275215B2 (en) * | 2007-05-08 | 2012-09-25 | Arcsoft (Shanghai) Technology Company, Ltd | Merging images |
US20090192921A1 (en) * | 2008-01-24 | 2009-07-30 | Michael Alan Hicks | Methods and apparatus to survey a retail environment |
US9307165B2 (en) * | 2008-08-08 | 2016-04-05 | Qualcomm Technologies, Inc. | In-camera panorama image stitching assistance |
JP5219690B2 (en) * | 2008-08-21 | 2013-06-26 | キヤノン株式会社 | Image processing apparatus, image processing apparatus control method, and program |
US8554014B2 (en) * | 2008-08-28 | 2013-10-08 | Csr Technology Inc. | Robust fast panorama stitching in mobile phones or cameras |
US20100194851A1 (en) * | 2009-02-03 | 2010-08-05 | Aricent Inc. | Panorama image stitching |
US8605783B2 (en) * | 2009-05-22 | 2013-12-10 | Microsoft Corporation | Composite video generation |
US8947502B2 (en) | 2011-04-06 | 2015-02-03 | Qualcomm Technologies, Inc. | In camera implementation of selecting and stitching frames for panoramic imagery |
GB2473248A (en) * | 2009-09-04 | 2011-03-09 | Sony Corp | Determining image misalignment by comparing image characteristics at points along a line |
JP2011119974A (en) * | 2009-12-03 | 2011-06-16 | Sony Corp | Panoramic image synthesizer, panoramic image synthesis method, and program |
US20110141226A1 (en) * | 2009-12-11 | 2011-06-16 | Fotonation Ireland Limited | Panorama imaging based on a lo-res map |
US20110141229A1 (en) * | 2009-12-11 | 2011-06-16 | Fotonation Ireland Limited | Panorama imaging using super-resolution |
US20110141224A1 (en) * | 2009-12-11 | 2011-06-16 | Fotonation Ireland Limited | Panorama Imaging Using Lo-Res Images |
US10080006B2 (en) * | 2009-12-11 | 2018-09-18 | Fotonation Limited | Stereoscopic (3D) panorama creation on handheld device |
US8294748B2 (en) * | 2009-12-11 | 2012-10-23 | DigitalOptics Corporation Europe Limited | Panorama imaging using a blending map |
US20110141225A1 (en) * | 2009-12-11 | 2011-06-16 | Fotonation Ireland Limited | Panorama Imaging Based on Low-Res Images |
JP4763847B1 (en) * | 2010-08-30 | 2011-08-31 | 楽天株式会社 | Image conversion apparatus, image processing apparatus, and image processing system |
US9153004B2 (en) | 2010-08-30 | 2015-10-06 | Rakuten, Inc. | Product image interpolating device, method and non-transitory information recording medium |
WO2012075250A1 (en) * | 2010-12-01 | 2012-06-07 | Magna Electronics Inc. | System and method of establishing a multi-camera image using pixel remapping |
US8374428B2 (en) * | 2010-12-05 | 2013-02-12 | Microsoft Corporation | Color balancing for partially overlapping images |
KR101040532B1 (en) * | 2011-01-31 | 2011-06-16 | 삼성탈레스 주식회사 | Contrast enhancement apparatus and method for transmiting of infrared panoramic images |
JP2012191486A (en) * | 2011-03-11 | 2012-10-04 | Sony Corp | Image composing apparatus, image composing method, and program |
US9390530B2 (en) | 2011-05-27 | 2016-07-12 | Nokia Technologies Oy | Image stitching |
US10793067B2 (en) | 2011-07-26 | 2020-10-06 | Magna Electronics Inc. | Imaging system for vehicle |
US20130293671A1 (en) * | 2012-05-01 | 2013-11-07 | Tourwrist, Inc. | Systems and methods for stitching and sharing panoramas |
US9058655B2 (en) * | 2012-11-06 | 2015-06-16 | Apple Inc. | Region of interest based image registration |
US10573045B2 (en) * | 2012-12-19 | 2020-02-25 | Shutterfly, Llc | Generating an assembled group image from subject images |
CN105592792B (en) * | 2013-10-23 | 2018-08-17 | 麦克赛尔控股株式会社 | Surface state measures analytical information management system and surface state measures analysis approaches to IM |
US10368097B2 (en) * | 2014-01-07 | 2019-07-30 | Nokia Technologies Oy | Apparatus, a method and a computer program product for coding and decoding chroma components of texture pictures for sample prediction of depth pictures |
CN104463778B (en) * | 2014-11-06 | 2017-08-29 | 北京控制工程研究所 | A kind of Panoramagram generation method |
US10127463B2 (en) | 2014-11-21 | 2018-11-13 | Magna Electronics Inc. | Vehicle vision system with multiple cameras |
US20160173784A1 (en) * | 2014-12-16 | 2016-06-16 | Hyangjin LEE | Apparatus and method for providing of rotation image |
TWI552600B (en) * | 2014-12-25 | 2016-10-01 | 晶睿通訊股份有限公司 | Image calibrating method for stitching images and related camera and image processing system with image calibrating function |
WO2016203282A1 (en) | 2015-06-18 | 2016-12-22 | The Nielsen Company (Us), Llc | Methods and apparatus to capture photographs using mobile devices |
US10043237B2 (en) | 2015-08-12 | 2018-08-07 | Gopro, Inc. | Equatorial stitching of hemispherical images in a spherical image capture system |
US10796262B2 (en) | 2015-09-30 | 2020-10-06 | The Nielsen Company (Us), Llc | Interactive product auditing with a mobile device |
US10187590B2 (en) | 2015-10-27 | 2019-01-22 | Magna Electronics Inc. | Multi-camera vehicle vision system with image gap fill |
CN107563987A (en) * | 2016-07-01 | 2018-01-09 | 北京疯景科技有限公司 | Demarcate the method and device of imaging difference |
TWI661392B (en) * | 2017-12-27 | 2019-06-01 | 聚星電子股份有限公司 | Image stitching method and device |
KR102392631B1 (en) | 2020-06-10 | 2022-04-29 | 중앙대학교 산학협력단 | System for panoramic image generation and update of concrete structures or bridges using deep matching, a method for generating and updating panoramic images, and a program for generating and updating panoramic images |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1993012501A1 (en) * | 1991-12-10 | 1993-06-24 | Logitech, Inc. | Apparatus and methods for automerging images |
EP0913991A1 (en) * | 1992-10-08 | 1999-05-06 | Sharp Kabushiki Kaisha | Image processing apparatus |
US6456323B1 (en) * | 1999-12-31 | 2002-09-24 | Stmicroelectronics, Inc. | Color correction estimation for panoramic digital camera |
US20050089244A1 (en) * | 2003-10-22 | 2005-04-28 | Arcsoft, Inc. | Panoramic maker engine for a low profile system |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3066173B2 (en) | 1992-03-18 | 2000-07-17 | 三洋電機株式会社 | Pattern matching method |
US6549681B1 (en) * | 1995-09-26 | 2003-04-15 | Canon Kabushiki Kaisha | Image synthesization method |
JP3302236B2 (en) | 1995-09-26 | 2002-07-15 | キヤノン株式会社 | Panoramic image synthesizing apparatus and method for creating panoramic image |
JPH1083442A (en) | 1996-09-09 | 1998-03-31 | Canon Inc | Picture synthesis method |
US6714689B1 (en) * | 1995-09-29 | 2004-03-30 | Canon Kabushiki Kaisha | Image synthesizing method |
JP3696952B2 (en) | 1995-09-29 | 2005-09-21 | キヤノン株式会社 | Image composition apparatus and method |
JPH11167640A (en) | 1997-12-05 | 1999-06-22 | Tani Denki Kogyo Kk | Measuring method due to image recognition and recording medium |
US7015954B1 (en) | 1999-08-09 | 2006-03-21 | Fuji Xerox Co., Ltd. | Automatic video system using multiple cameras |
JP2002042125A (en) | 2000-07-26 | 2002-02-08 | Minolta Co Ltd | Image synthesizing device, image synthesizing method, and computer readable recording medium recording image synthesizing program |
JP4501481B2 (en) | 2004-03-22 | 2010-07-14 | セイコーエプソン株式会社 | Image correction method for multi-projection system |
US7447337B2 (en) * | 2004-10-25 | 2008-11-04 | Hewlett-Packard Development Company, L.P. | Video content understanding through real time video motion analysis |
CN100455266C (en) * | 2005-03-29 | 2009-01-28 | 深圳迈瑞生物医疗电子股份有限公司 | Broad image processing method |
US7813590B2 (en) * | 2005-05-13 | 2010-10-12 | Given Imaging Ltd. | System and method for displaying an in-vivo image stream |
US7551234B2 (en) * | 2005-07-28 | 2009-06-23 | Seiko Epson Corporation | Method and apparatus for estimating shot boundaries in a digital video sequence |
EP1793580B1 (en) * | 2005-12-05 | 2016-07-27 | Microsoft Technology Licensing, LLC | Camera for automatic image capture having plural capture modes with different capture triggers |
EP1965350A1 (en) * | 2005-12-21 | 2008-09-03 | Nikon Corporation | Image combining method, image combining program, image combining device, template extracting method, and template extracting program |
-
2007
- 2007-01-12 US US11/623,050 patent/US8331725B2/en not_active Expired - Fee Related
-
2008
- 2008-01-11 EP EP08727584A patent/EP2103101A1/en not_active Withdrawn
- 2008-01-11 CN CNA2008800019320A patent/CN101578851A/en active Pending
- 2008-01-11 KR KR1020097016800A patent/KR20090099580A/en not_active Application Discontinuation
- 2008-01-11 WO PCT/US2008/050869 patent/WO2008089069A1/en active Application Filing
- 2008-01-11 JP JP2009545703A patent/JP2010516003A/en active Pending
- 2008-01-11 TW TW097101190A patent/TW200903345A/en unknown
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1993012501A1 (en) * | 1991-12-10 | 1993-06-24 | Logitech, Inc. | Apparatus and methods for automerging images |
EP0913991A1 (en) * | 1992-10-08 | 1999-05-06 | Sharp Kabushiki Kaisha | Image processing apparatus |
US6456323B1 (en) * | 1999-12-31 | 2002-09-24 | Stmicroelectronics, Inc. | Color correction estimation for panoramic digital camera |
US20050089244A1 (en) * | 2003-10-22 | 2005-04-28 | Arcsoft, Inc. | Panoramic maker engine for a low profile system |
Also Published As
Publication number | Publication date |
---|---|
EP2103101A1 (en) | 2009-09-23 |
WO2008089069A9 (en) | 2008-10-09 |
CN101578851A (en) | 2009-11-11 |
JP2010516003A (en) | 2010-05-13 |
TW200903345A (en) | 2009-01-16 |
US8331725B2 (en) | 2012-12-11 |
US20080170803A1 (en) | 2008-07-17 |
KR20090099580A (en) | 2009-09-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8331725B2 (en) | Panoramic imaging techniques | |
US8003927B2 (en) | Image projection apparatus which projects an image corrected according to a projection surface | |
US7936374B2 (en) | System and method for camera calibration and images stitching | |
US8401337B2 (en) | Registration of separations | |
WO2018032783A1 (en) | Image processing method and apparatus | |
Mills et al. | Image stitching with dynamic elements | |
US20050169555A1 (en) | Image processing apparatus and method, and computer program | |
US20050008254A1 (en) | Image generation from plurality of images | |
US20130342740A1 (en) | Method, apparatus and computer program product for capturing video content | |
US10154195B2 (en) | Image joining apparatus, image pickup apparatus, image joining method, and image joining program | |
JP4956581B2 (en) | Imaging apparatus, white balance processing method of imaging apparatus, program, and recording medium | |
US20110075924A1 (en) | Color adjustment | |
CN107395991A (en) | Image combining method, device, computer-readable recording medium and computer equipment | |
US20110285745A1 (en) | Method and apparatus for touch screen assisted white balance | |
JP3543774B2 (en) | Image processing apparatus, image processing method, and recording medium | |
US7505069B2 (en) | Method and apparatus for maintaining consistent white balance in successive digital images | |
US20100079582A1 (en) | Method and System for Capturing and Using Automatic Focus Information | |
JP2005250778A (en) | Vertical direction decision of image | |
WO2002056252A2 (en) | Apparatus and method for boundary detection in vector sequences and edge detection in color image signals | |
EP2107798B1 (en) | Imaging apparatus, image processing apparatus, and exposure control method | |
JP2005275765A (en) | Image processor, image processing method, image processing program and recording medium recording the program | |
JP2014048932A (en) | Celestial body image processing device, celestial body image processing program and camera | |
US8588523B2 (en) | Method and apparatus for image signal processing | |
JP2003141526A (en) | Image processing apparatus | |
JP2003187253A (en) | Density gradient vector calculator, program and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200880001932.0 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 08727584 Country of ref document: EP Kind code of ref document: A1 |
|
DPE1 | Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 1130/MUMNP/2009 Country of ref document: IN |
|
ENP | Entry into the national phase |
Ref document number: 2009545703 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2008727584 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020097016800 Country of ref document: KR |