WO2011114317A1

WO2011114317A1 - Method and apparatus for providing seam selection using a reduced complexity dynamic programming technique

Info

Publication number: WO2011114317A1
Application number: PCT/IB2011/051154
Authority: WO
Inventors: Gururaj Gopal Putraya; Pranav Mishra; Krishna Annasagar Govindarao
Original assignee: Nokia Corporation
Priority date: 2010-03-18
Filing date: 2011-03-18
Publication date: 2011-09-22

Abstract

A method for providing seam selection using a reduced complexity dynamic programming technique may include applying a scaling factor to an image to generate a decimated image in which the image is m an un-decimated domain and the decimated image is in a decimated domain, defining a decimated domain seam in an overlap region between at least two images comprising the image in the decimated domain, and determining an un-decimated domain seam for the image based on the decimated domain seam A corresponding computer program product and apparatus are also provided.

Description

METHOD AND APPARATUS FOR PROVIDING SEAM SELECTION USING A REDUCED COMPLEXITY DYNAMIC PROGRAMMING TECHNIQUE

TECHNOLOGICAL FIELD

Embodiments of the present invention relate generally to image processing technology and, more particularly, relate to a method, apparatus and computer program product for providing seam selection using a reduced complexity dynamic programming technique.

BACKGROUND

The modern communications era has brought about a tremendous expansion of wireline and wireless networks. Computer networks, television networks, and telephony networks are experiencing an unprecedented technological expansion, fueled by consumer demand. Wireless and mobile networking technologies have addressed related consumer demands, while providing more flexibility and immediacy of information transfer.

Current and future networking technologies continue to facilitate ease of information transfer and convenience to users by expanding the capabilities of mobile electronic devices. Meanwhile, very capable mobile terminals are now frequently affordable by a great many people and are becoming extremely popular for their usefulness in relation to enhancing communication capabilities, providing entertainment, performing tasks, and consuming media. Due to the ubiquitous nature of mobile electronic devices, people of all ages and education levels are now utilizing mobile terminals to communicate with other individuals or contacts, receive services and/ or to share information, media and other content. Additionally, given recent advances in processing power, battery life, the availability of peripherals such as cameras for taking pictures or generating video content, mobile electronic devices are increasingly used by individuals for creating and consuming image related content.

[0001] Many images or content items including images may be initially sized for viewing on a display screen having a different size than the screen upon which rendering will ultimately occur. Additionally, in some cases it may be desirable to stitch or splice images together to create larger images (e.g., for a panoramic view). Accordingly, mechanisms have been developed to adjust image sizes to fit different sized screens and to enable stitching of images together. However, these mechanisms are typically very complex and can therefore be time consuming and require large amounts of processing power.

BRIEF SUMMARY

A method, apparatus and computer program product are therefore provided to reduce the complexity of performing image processing. In this regard, for example, some embodiments may reduce the complexity of dynamic programming used in connection with seam selection. Accordingly, image resizing or stitching may be accomplished with relatively fewer resources and in less time. In one example embodiment, a method of providing seam selection using a reduced complexity dynamic programming technique is provided. The method may include applying a scaling factor to an image to generate a decimated (or downsampled) image in which the image is in an un-decimated domain and the decimated image is in a decimated domain, defining a decimated domain optimal path (or seam) in an overlap region between at least two images comprising the image in the decimated domain, and determining an un-decimated domain optimal path (or seam) for the image based on the decimated domain optimal path (or seam).

In another example embodiment, a computer program product for providing seam selection using a reduced complexity dynamic programming technique is provided. The computer program product includes at least one computer-readable storage medium having computer-executable program code instructions stored therein. The computer-executable program code instructions may include program code instructions for applying a scaling factor to an image to generate a decimated (or downsampled) image in which the image is in an un-decimated domain and the decimated image is in a decimated domain, defining a decimated domain optimal path (or seam) in an overlap region between at least two images comprising the image in the decimated domain, and determining an un-decimated domain optimal path (or seam) for the image based on the decimated domain optimal path (or seam).

In another example embodiment, an apparatus for providing seam selection using a reduced complexity dynamic programming technique is provided. The apparatus may include at least one processor and at least one memory including computer program code. The at least one memory and the computer program code may be configured to, with the at least one processor, cause the apparatus to perform at least applying a scaling factor to an image to generate a decimated (or downsampled) image in which the image is in an un-decimated domain and the decimated image is in a decimated domain, defining a decimated domain optimal path (or seam) in an overlap region between at least two images comprising the image in the decimated domain, and determining an un-decimated domain optimal path (or seam) for the image based on the decimated domain optimal path (or seam).

Embodiments of the invention may provide a method, apparatus and computer program product for employment in either mobile or fixed environments in which image processing services are provided. As a result, for example, mobile terminal users or users of personal computers (PCs) or other fixed computing devices may enjoy an improved capability with respect to image processing.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S) Having thus described embodiments of the invention in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:

FIG. 1 is a schematic block diagram of a wireless communications system according to an example embodiment of the present invention; FIG. 2 illustrates a block diagram of an apparatus for reduced complexity in image processing according to an example embodiment of the present invention;

FIG. 3 (which includes FIGS. 3 A to 3C) illustrates an example of image processing according to an example embodiment of the present invention; and

FIG. 4 is a flowchart according to another example method for providing reduced complexity in image processing according to an example embodiment of the present invention.

DETAILED DESCRIPTION

Some embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the invention are shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, the terms "data," "content," "information" and similar terms may be used interchangeably to refer to data capable of being transmitted, received and/or stored in accordance with embodiments of the present invention. Additionally, the terms "decimated" and "downsampled" may also be used interchangeably to refer to the reduction of data by sampling or selecting a portion of the original data for inclusion in a decimated or downsampled version of the data. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments of the present invention.

Additionally, as used herein, the term 'circuitry' refers to (a) hardware-only circuit implementations (e.g., implementations in analog circuitry and/or digital circuitry); (b) combinations of circuits and computer program product(s) comprising software and/or firmware instructions stored on one or more computer readable memories that work together to cause an apparatus to perform one or more functions described herein; and (c) circuits, such as, for example, a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation even if the software or firmware is not physically present. This definition of 'circuitry' applies to all uses of this term herein, including in any claims. As a further example, as used herein, the term ' circuitry' also includes an implementation comprising one or more processors and/or portion(s) thereof and accompanying software and/or firmware. As another example, the term 'circuitry' as used herein also includes, for example, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, other network device, and/ or other computing device.

As defined herein a "computer-readable storage medium," which refers to a non-transitory, physical storage medium (e.g., volatile or non-volatile memory device), can be differentiated from a "computer-readable transmission medium," which refers to an electromagnetic signal. As indicated above, some embodiments of the present invention may provide reduced complexity in image processing. In this regard, an example embodiment may reduce the complexity of dynamic programming that may be used to stitch images together. Dynamic programming is a technique for simplifying complex problems by breaking the problems into smaller less complex problems. In the context of embodiments of the present invention, dynamic programming may be used for seam selection to enable location of seams for image processing faster and with less resource consumption. Thus, for example, if two images are being stitched together for the creation of a panoramic view. To accomplish the stitching, a so-called "optimal seam" may be determined. The optimal seam may be defined as a path that passes through the minimal error path in the overlapping region between the images to be stitched together. By finding the optimal seam and combining the images along the optimal seam, a more realistic looking combination of the images may be performed.

One way to find the optimal seam or path is to calculate an accumulated error in the overlapping region and then backtrace the minimum accumulated error after completion of the calculation of the accumulated error to determine the optimal seam. The time complexity of this operation may be defined as 0(M x N) ,where M x N defines the dimensions of the resulting image. Some example embodiments may provide a reduction to the time complexity of the operation above by first obtaining an optimal path or seam in a decimated (or downsampled) image. From the path obtained in the decimated image, a bounded region of the full resolution image may then be defined and the optimal path through the bounded region may then be found. The overall complexity of this example approach may be reduced to 0((M x N)/(Q x Q)) + 0(L x N), where Q is a scaling factor (or decimation factor) and L is the width of the bounded region. Where L is less than M, the overall complexity of the operation for seam selection may be reduced significantly. Alternatively, accumulated error may only be determined for pixels in a region offset from the optimal path determined from the decimated image. The overall complexity of this example approach may be reduced to 0((M x N)/(Q x Q)) + 0(2 x offset x N), where Q is the scaling factor and "offset" is the amount of offset from the path in the decimated image.

FIG. 1 illustrates a generic system diagram in which a device such as a mobile terminal 10, which may benefit from embodiments of the present invention, is shown in an example communication environment. As shown in FIG. 1, a system in accordance with an example embodiment of the present invention includes a first communication device (e.g., mobile terminal 10) and a second communication device 20 that are each capable of communication with a network 30. The second communication device 20 is provided as an example to illustrate potential multiplicity with respect to instances of other devices that may be included in the network 30 and that may practice example embodiments. The communications devices of the system may be able to communicate with network devices or with each other via the network 30. In some cases, the network devices with which the communication devices of the system communicate may include a service platform 40. In an example embodiment, the mobile terminal 10 (and/or the second communication device 20) is enabled to communicate with the service platform 40 to provide, request and/or receive information.

In some embodiments, not all systems that employ embodiments of the present invention may comprise all the devices illustrated and/ or described herein. For example, while an example embodiment will be described herein in which an image processing service is provided from a network device (e.g., the service platform 40) and accessed at the mobile terminal 10, some embodiments may exclude the service platform 40 and network 30 altogether and simply be practiced on a single device (e.g., the mobile terminal 10 or the second communication device 20) in a stand alone mode.

While several embodiments of the mobile terminal 10 may be illustrated and hereinafter described for purposes of example, other types of mobile terminals, such as portable digital assistants (PDAs), pagers, mobile televisions, mobile telephones, gaming devices, laptop computers, cameras, camera phones, video recorders, audio/video player, radio, GPS devices, navigation devices, or any combination of the aforementioned, and other types of voice and text communications systems, can readily employ embodiments of the present invention. Furthermore, devices that are not mobile may also readily employ embodiments of the present invention. As such, for example, the second communication device 20 may represent an example of a fixed electronic device that may employ an example embodiment. For example, the second communication device 20 may be a personal computer (PC) or other computing device having a display.

In an example embodiment, the network 30 includes a collection of various different nodes, devices or functions that are capable of communication with each other via corresponding wired and/or wireless interfaces. As such, the illustration of FIG. 1 should be understood to be an example of a broad view of certain elements of the system and not an all inclusive or detailed view of the system or the network 30. Although not necessary, in some embodiments, the network 30 may be capable of supporting communication in accordance with any one or more of a number of first-generation (1 G), second- generation (2G), 2.5G, third-generation (3G), 3.5G, 3.9G, fourth-generation (4G) mobile communication protocols, Long Term Evolution (LTE), and/ or the like.

One or more communication terminals such as the mobile terminal 10 and the second communication device 20 may be capable of communication with each other via the network 30 and each may include an antenna or antennas for transmitting signals to and for receiving signals from a base site, which could be, for example a base station that is a part of one or more cellular or mobile networks or an access point that may be coupled to a data network, such as a local area network (LAN), a metropolitan area network (MAN), and/or a wide area network (WAN), such as the Internet. In turn, other devices such as processing devices or elements (e.g., personal computers, server computers or the like) may be coupled to the mobile terminal 10 and the second communication device 20 via the network 30. By directly or indirectly connecting the mobile terminal 10, the second communication device 20 and other devices to the network 30, the mobile terminal 10 and the second communication device 20 may be enabled to communicate with the other devices (or each other), for example, according to numerous communication protocols including Hypertext Transfer Protocol (HTTP) and/or the like, to thereby carry out various communication or other functions of the mobile terminal 10 and the second communication device 20, respectively.

Furthermore, although not shown in FIG. 1 , the mobile terminal 10 and the second communication device 20 may communicate in accordance with, for example, radio frequency (RF), Bluetooth (BT), Infrared (IR) or any of a number of different wireline or wireless communication techniques, including LAN, wireless LAN (WLAN), Worldwide Interoperability for Microwave Access (WiMAX), WiFi, ultra-wide band (UWB), Wibree techniques and/or the like. As such, the mobile terminal 10 and the second communication device 20 may be enabled to communicate with the network 30 and each other by any of numerous different access mechanisms. For example, mobile access mechanisms such as wideband code division multiple access (W-CDMA), CDMA2000, global system for mobile communications (GSM), general packet radio service (GPRS) and/or the like may be supported as well as wireless access mechanisms such as WLAN, WiMAX, and/or the like and fixed access mechanisms such as digital subscriber line (DSL), cable modems, Ethernet and/or the like.

In an example embodiment, the service platform 40 may be a device or node such as a server or other processing element. The service platform 40 may have any number of functions or associations with various services. As such, for example, the service platform 40 may be a platform such as a dedicated server (or server bank) associated with a particular information source or service (e.g. , an image processing service), or the service platform 40 may be a backend server associated with one or more other functions or services. As such, the service platform 40 represents a potential host for a plurality of different services or information sources. In some embodiments, the functionality of the service platform 40 is provided by hardware and/or software components configured to operate in accordance with known techniques for the provision of information to users of communication devices. However, at least some of the functionality provided by the service platform 40 is information provided in accordance with example embodiments of the present invention.

FIG. 2 illustrates a schematic block diagram of an apparatus for providing seam selection using a reduced complexity dynamic programming technique according to an example embodiment of the present invention. An example embodiment will now be described with reference to FIG. 2, in which certain elements of an apparatus 50 for providing seam selection using a reduced complexity dynamic programming technique are displayed. The apparatus 50 of FIG. 2 may be employed, for example, on the service platform 40. However, the apparatus 50 may alternatively be embodied at a variety of other devices, both mobile and fixed (such as, for example, the mobile terminal 10 or the second communication device 20). In some cases, embodiments may be employed on either one or a combination of devices. Accordingly, some embodiments of the present invention may be embodied wholly at a single device (e.g., the service platform 40, the mobile terminal 10 or the second communication device 20), by a plurality of devices in a distributed fashion or by devices in a client server relationship (e.g., the mobile terminal 10 as the client and the service platform 40 as the server). Furthermore, it should be noted that the devices or elements described below may not be mandatory and thus some may be omitted in certain embodiments.

Referring now to FIG. 2, an apparatus for providing seam selection using a reduced complexity dynamic programming technique is provided. The apparatus 50 may include or otherwise be in communication with a processor 70, a user interface 72, a communication interface 74 and a memory device 76. The memory device 76 may include, for example, one or more volatile and/or non-volatile memories. In other words, for example, the memory device 76 may be an electronic storage device (e.g., a computer readable storage medium) comprising gates configured to store data (e.g., bits) that may be retrievable by a machine (e.g., a computing device). The memory device 76 may be configured to store information, data, applications, instructions or the like for enabling the apparatus to carry out various functions in accordance with exemplary embodiments of the present invention. For example, the memory device 76 could be configured to buffer input data for processing by the processor 70. Additionally or alternatively, the memory device 76 could be configured to store instructions for execution by the processor 70.

The processor 70 may be embodied in a number of different ways. For example, the processor 70 may be embodied as one or more of various processing means such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), a processing element with or without an accompanying DSP, or various other processing devices including integrated circuits such as, for example, an ASIC (application specific integrated circuit), an FPGA (field programmable gate array), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, processing circuitry, or the like. In an exemplary embodiment, the processor 70 may be configured to execute instructions stored in the memory device 76 or otherwise accessible to the processor 70. Alternatively or additionally, the processor 70 may be configured to execute hard coded functionality. As such, whether configured by hardware or software methods, or by a combination thereof, the processor 70 may represent an entity (e.g., physically embodied in circuitry) capable of performing operations according to embodiments of the present invention while configured accordingly. Thus, for example, when the processor 70 is embodied as an ASIC, FPGA or the like, the processor 70 may be specifically configured hardware for conducting the operations described herein. Alternatively, as another example, when the processor 70 is embodied as an executor of software instructions, the instructions may specifically configure the processor 70 to perform the algorithms and/or operations described herein when the instructions are executed. However, in some cases, the processor 70 may be a processor of a specific device (e.g., the mobile terminal 10 or a network device) adapted for employing embodiments of the present invention by further configuration of the processor 70 by instructions for performing the algorithms and/ or operations described herein. The processor 70 may include, among other things, a clock, an arithmetic logic unit (ALU) and logic gates configured to support operation of the processor 70.

Meanwhile, the communication interface 74 may be any means such as a device or circuitry embodied in either hardware, software, or a combination of hardware and software that is configured to receive and/or transmit data from to a network and or any other device or module in communication with the apparatus. In this regard, the communication interface 74 may include, for example, an antenna (or multiple antennas) and supporting hardware and/or software for enabling communications with a wireless communication network. In some environments, the communication interface 74 may alternatively or also support wired communication. As such, for example, the communication interface 74 may include a communication modem and/or other hardware/software for supporting communication via cable, digital subscriber line (DSL), universal serial bus (USB) or other mechanisms.

The user interface 72 may be in communication with the processor 70 to receive an indication of a user input at the user interface 72 and/or to provide an audible, visual, mechanical or other output to the user. As such, the user interface 72 may include, for example, a keyboard, a mouse, a joystick, a display, a touch screen, soft keys, a microphone, a speaker, or other input/output mechanisms. In an exemplary embodiment in which the apparatus is embodied as a server or some other network devices, the user interface 72 may be limited, or eliminated. However, in an embodiment in which the apparatus is embodied as a communication device (e.g., the mobile terminal 10), the user interface 72 may include, among other devices or elements, any or all of a speaker, a microphone, a display, and a keyboard or the like. In this regard, for example, the processor 70 may comprise user interface circuitry configured to control at least some functions of one or more elements of the user interface, such as, for example, a speaker, ringer, microphone, display, and/or the like. The processor 70 and/ or user interface circuitry comprising the processor 70 may be configured to control one or more functions of one or more elements of the user interface through computer program instructions (e.g., software and/or firmware) stored on a memory accessible to the processor 70 (e.g., memory device 76, and/ or the like).

In an example embodiment, the processor 70 may be embodied as, include or otherwise control a path determiner (or seam determiner) 80. As such, in some embodiments, the processor 70 may be said to cause, direct or control the execution or occurrence of the various functions attributed to the path determiner 80 as described herein. The path determiner 80 may be any means such as a device or circuitry operating in accordance with software or otherwise embodied in hardware or a combination of hardware and software (e.g., processor 70 operating under software control, the processor 70 embodied as an ASIC or FPGA specifically configured to perform the operations described herein, or a combination thereof) thereby configuring the device or circuitry to perform the corresponding functions of the path determiner 80 as described herein. Thus, in examples in which software is employed, a device or circuitry (e.g., the processor 70 in one example) executing the software forms the structure associated with such means. The path determiner 80 may be configured to initially decimate an image (e.g., an image comprising two images that are to be stitched together) by a predetermined factor (Q), such that there is no aliasing in the image. After decimating the image a seam or path (e.g., an optimal path) may be determined for the image in the decimated domain. As indicated above, the complexity of this operation may be defined as 0((M x N /(Q x Q)),where M x N defines the dimensions of the original image (e.g., resulting from the combination of the two images being stitched together). Thereafter, the path determiner 80 may be configured to determine an optimal path for the image (e.g., in the un-decimated domain) based on the optimal path determined in the decimated domain. The complexity for this operation will be less than the complexity of directly determining an optimal path or seam in a full resolution version of the image. Accordingly, an optimal path or seam for image stitching (or for other image resizing operations) may be determined in a manner that may be capable of determination relatively quickly and with less resource consumption.

The operation of determining the optimal path for the image in the un-decimated domain based on the optimal path determined in the decimated domain may be accomplished in different ways. As such, according to a first example, the optimal path determined in the decimated domain may be used to bound a region of pixels in which the optimal path in the un-decimated or actual resolution image is likely to fall based on the location of the optimal path in the decimated domain. In an example embodiment, it may be assumed that the bounded region has dimensions of L x N, where L is the width of the bounded image and L is less than M. The optimal path may then be determined within the bounded region rather than by examining the entirety of the image. The overall complexity of this example approach may be reduced to 0((M x N)/(Q x Q)) + 0(L x N).

In an alternative example, the optimal path determined in the decimated domain may be used as a basic start point, and a predetermined number of pixels (e.g., a pixel offset) around the region of the optimal path determined in the decimated domain may define an offset region (e.g., including the predetermined or offset number of pixels in both positive and negative directions from the decimated domain optimal path). An accumulated error may then be determined only for those pixels in the offset region. The complexity of this approach is 0((M x N)/(Q x Q)) + 0(2 x offset x N). In situations where a large offset is defined, there may be a relatively high likelihood of the actual optimal path for the un- decimated domain corresponding to the optimal path determined. Meanwhile, in situations where a small offset is defined, there may be further complexity reductions, but there may be less likelihood of the actual optimal path for the un-decimated domain corresponding to the optimal path determined. However, even with a small offset defined, the optimal path determined is likely to be a relatively good approximation. Hence, offset enables provision of a tradeoff between computations and accuracy.

FIG 3, which includes FIGS. 3A to 3C, provides illustrations of alternative mechanisms for determining the optimal path for the image in the un-decimated domain based on the optimal path determined in the decimated domain. In this regard, FIG. 3A shows an optimal path 90 indicated by darkened pixels within an 8 x 8 matrix of pixels defining the decimated image domain. In the context of this example, assume the absolute difference image has dimensions of 16 x 16 pixels (M=N=16) and the scaling factor or decimation factor (Q) is 2. Based on these parameters, the 8 x 8 decimated domain of FIG 3 A results. The optimal path in the decimated domain may be determined based on calculating the accumulated error for the pixels of the decimated domain and backtracing the minimum accumulated error.

FIG 3B is an illustration of an embodiment that utilizes the first example described above for determining the optimal path for the image in the un-decimated domain based on the optimal path determined in the decimated domain. In this regard, FIG 3B shows the un-decimated domain (or actual resolution) of the image and further indicates a bounded region 92 (having a width L) that is selected based on the optimal path 90 in the decimated domain. Accordingly, the bounded region 92 is selected to contain the pixels that correspond to the optimal path 90. The optimal path for the un-decimated region may then be determined as indicated above based on calculating accumulated errors over a smaller region (e.g., the bounded region 92) than would otherwise be performed.

FIG 3C is an illustration of an embodiment that utilizes the alternative example described above for determining the optimal path for the image in the un-decimated domain based on the optimal path determined in the decimated domain. In this regard, FIG. 3C shows the un-decimated domain of the image and further indicates an offset region 94 that is defined by a three pixel offset on each side of the pixels 90' that correspond to the optimal path 90. The offset region may then be used for calculating the accumulated error during dynamic programming as described above.

Accordingly, by employing either the first example or the alternative example, described above, the optimal path that is determined in the decimated domain may be used to reduce the complexity of calculations made to determine the optimal path for the un-decimated domain. Thus, an optimal seam or path for the image may be determined in less time and/or with less resource consumption.

FIG. 4 is a flowchart of a method and program product according to example embodiments of the invention. It will be understood that each block of the flowchart, and combinations of blocks in the flowchart, may be implemented by various means, such as hardware, firmware, processor, circuitry and/or other device associated with execution of software including one or more computer program instructions. For example, one or more of the procedures described above may be embodied by computer program instructions. In this regard, the computer program instructions which embody the procedures described above may be stored by a memory device of the mobile terminal or network device and executed by a processor in the mobile terminal or network device. As will be appreciated, any such computer program instructions may be loaded onto a computer or other programmable apparatus (e.g., hardware) to produce a machine, such that the instructions which execute on the computer or other programmable apparatus create means for implementing the functions specified in the flowchart block(s). These computer program instructions may also be stored in a computer-readable memory that may direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block(s). The computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operations to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus implement the functions specified in the flowchart block(s).

Accordingly, blocks of the flowchart support combinations of means for performing the specified functions, combinations of operations for performing the specified functions and program instruction means for performing the specified functions. It will also be understood that one or more blocks of the flowchart, and combinations of blocks in the flowchart, can be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.

In this regard, a method according to one embodiment of the invention, as shown in FIG. 4, may include applying a scaling factor to an image to generate a decimated image in which the image is in an un- decimated domain and the decimated image is in the decimated domain at operation 100. The method may further include defining a decimated domain optimal path (or seam) in an overlap region between at least two images comprising the image in the decimated domain at operation 110 and determining an un- decimated domain optimal path (or seam) for the image based on the decimated domain optimal path at operation 120. Notably, in some cases the un-decimated domain may include images of different resolutions so long as the images are of a higher resolution than the decimated image. In other words, the resolution of the original image and the final image need not be exactly the same as long as both resolutions are higher than the resolution of the decimated image.

In some embodiments, certain ones of the operations above may be modified or further amplified as described below. It should be appreciated that each of the modifications or amplifications below may be included with the operations above either alone or in combination with any others among the features described herein. In this regard, for example, determining the un-decimated domain optimal path may include calculating an accumulated error over a portion of the un-decimated image, and backtracing a minimal accumulated error over the portion. In some cases, determining the un-decimated domain optimal path may include determining a bounded region of pixels in the un-decimated domain based on the decimated domain optimal path and determining the un-decimated domain optimal path by analyzing only the bounded region. In such examples, analyzing only the bounded region may include calculating an accumulated error for the bounded region and backtracing the minimal accumulated error over the bounded region. As an alternative, determining the un-decimated domain optimal path may include defining an offset region including a predetermined number of pixels proximate to pixels in the un-decimated domain that correspond to the decimated domain optimal path, and calculating an accumulated error during dynamic programming for only the offset region. In some cases, defining the offset region may include defining a region including the predetermined number of pixels on each opposite side of the pixels proximate to pixels in the un-decimated domain that correspond to the decimated domain optimal path. In an example embodiment, defining the decimated domain optimal path may include calculating an accumulated error for the decimated image and backtracing the minimal accumulated error over the decimated image.

In an example embodiment, an apparatus for performing the method of FIG. 4 above may comprise a processor (e.g., the processor 70) configured to perform some or each of the operations (100- 120) described above. The processor may, for example, be configured to perform the operations (100- 120) by performing hardware implemented logical functions, executing stored instructions, or executing algorithms for performing each of the operations. Alternatively, the apparatus may comprise means for performing each of the operations described above. In this regard, according to an example embodiment, examples of means for performing operations 100-120 may comprise, for example, the processor 70, the path determiner 80, and/ or a device or circuit for executing instructions or executing an algorithm for processing information as described above.

Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe example embodiments in the context of certain example combinations of elements and/or functions, it should be appreciated that different combinations of elements and/or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and/or functions than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

We Claim:

1. A method comprising:

applying a scaling factor to an image to generate a decimated image, the image being in an un- decimated domain and the decimated image being in a decimated domain;

defining a decimated domain seam in an overlap region between at least two images comprising the image in the decimated domain; and

determining an un-decimated domain seam for the image based on the decimated domain seam.

2. The method of claim 1 , wherein determining the un-decimated domain seam comprises calculating an accumulated error over a portion of the image, and backtracing a minimal accumulated error over the portion.

3. The method of claim 1 , wherein determining the un-decimated domain seam comprises determining a bounded region of pixels in the un-decimated domain based on the decimated domain seam and determining the un-decimated domain seam by analyzing only the bounded region.

4. The method of claim 3 , wherein analyzing only the bounded region comprises c alculating an accumulated error for the bounded region and backtracing the minimal accumulated error over the bounded region.

5. The method of claim 1 , wherein determining the un-decimated domain seam comprises defining an offset region including a predetermined number of pixels proximate to pixels in the un- decimated domain that correspond to the decimated domain seam, and calculating an accumulated error during dynamic programming for only the offset region.

6. The method of claim 5 , wherein defining the offset region comprises defining a region including the predetermined number of pixels on each opposite side of the pixels proximate to pixels in the un-decimated domain that correspond to the decimated domain seam.

7. The method of claim 1 , wherein defining the decimated domain seam comprises calculating an accumulated error for the decimated image and backtracing the minimal accumulated error over the decimated image. S. A computer program product comprising program instructions to cause an apparatus to perform method as claimed in claims 1 to 7. An apparatus causing to perform the method as claimed in claims 1 to 7.

A method substantially as herein described and illustrated in the accompanying drawing An apparatus substantially as herein described and illustrated in the accompanying