US20050084157A1 - Method for deblocking and transcoding a media stream - Google Patents
Method for deblocking and transcoding a media stream Download PDFInfo
- Publication number
- US20050084157A1 US20050084157A1 US10/688,082 US68808203A US2005084157A1 US 20050084157 A1 US20050084157 A1 US 20050084157A1 US 68808203 A US68808203 A US 68808203A US 2005084157 A1 US2005084157 A1 US 2005084157A1
- Authority
- US
- United States
- Prior art keywords
- coefficient
- quantization
- step size
- coefficients
- quantization step
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
- H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
Definitions
- Embodiments of the present invention relate to the field of digital media. Specifically, embodiments of the present invention relate to a method and system for deblocking and transcoding a media stream.
- Rate reduction transcoding is typically achieved using either truncation based transcoding or by requantization-based transcoding.
- Selective coefficient truncating transcoding typically achieves a relatively small amount of bit rate reduction and is rarely used often in typical video transcoding applications.
- Requantization-based transcoding achieves rate reduction by requantization using coarser quantization steps. Since the transform-domain coefficients are coarsely quantized, blocking artifacts may be produced. In particular, blocking artifacts arise as a result of each block being compressed independently. Blocking artifacts may appear as edges or produce ringing between blocks.
- a method for deblocking and transcoding a media stream are described.
- a coefficient associated with a block of pixels of the media stream is received.
- a deblocking operation is performed on the coefficient to generate a second coefficient.
- Quantization is performed on the second coefficient to generate a transcoded coefficient.
- FIG. 1 illustrates a block diagram of transcoding system in accordance with an embodiment of the present invention.
- FIG. 2 illustrates a graph of exemplary quantization bins for two quantizers in accordance with an embodiment of the present invention.
- FIG. 3 illustrates a flow chart of a process for determining responsiveness of a coefficient of a media stream in accordance with an embodiment of the present invention.
- FIG. 4 illustrates a flow chart of a process for determining responsiveness of a coefficient of a media stream using a uniform scalar quantizer with rounding to nearest in accordance with an embodiment of the present invention.
- FIG. 5A illustrates a graph of an exemplary distribution of responsive and unresponsive coefficients using a uniform scalar quantizer with rounding to nearest in accordance with an embodiment of the present invention.
- FIG. 5B illustrates a graph of an exemplary distribution of responsive and unresponsive coefficients using a uniform scalar quantizer with rounding down in accordance with an embodiment of the present invention.
- FIG. 6 illustrates a block diagram of transcoding system for deblocking responsive coefficients in accordance with an embodiment of the present invention.
- FIG. 7 illustrates an exemplary quantization error table in accordance with an embodiment of the present invention.
- FIG. 8 illustrates a flow chart of a process for determining an optimal quantization step size in accordance with an embodiment of the present invention.
- aspects of the present invention may be implemented in a computer system that includes, in general, a processor for processing information and instructions, random access (volatile) memory (RAM) for storing information and instructions, read-only (non-volatile) memory (ROM) for storing static information and instructions, a data storage device such as a magnetic or optical disk and disk drive for storing information and instructions, an optional user output device such as a display device (e.g., a monitor) for displaying information to the computer user, an optional user input device including alphanumeric and function keys (e.g., a keyboard) for communicating information and command selections to the processor, and an optional user input device such as a cursor control device (e.g., a mouse) for communicating user input information and command selections to the processor.
- RAM random access
- ROM read-only
- a data storage device such as a magnetic or optical disk and disk drive
- an optional user output device such as a display device (e.g., a monitor) for displaying information to the computer user
- FIG. 1 illustrates a block diagram of transcoding system 100 in accordance with an embodiment of the present invention.
- Transcoding system 100 utilizes requantization for bit rate reduction transcoding and has deblocking capability.
- Transcoding system 100 comprises three modules: inverse quantizer 105 , deblocking mechanism 110 , and transcoder 115 . It should be appreciated that transcoding system 100 may be implemented within a computer system.
- Transcoding system 100 receives a coefficient associated with a block of pixels of a media stream.
- Embodiments of the present invention pertain to media stream for transmitting graphical data, such as video streams.
- video stream is used in the present application.
- the coefficient is a quantized coefficient C q .
- quantized coefficient C q is received from a compressed video stream.
- quantized coefficient C q is a discrete cosine transform (DCT) coefficient.
- DCT discrete cosine transform
- Inverse quantizer 105 is operable to perform an inverse quantization operation (Q 1 ⁇ 1 ) on quantized coefficient C q using quantization step size s 1 . Inverse quantizer 105 outputs dequantized coefficient C. It should be appreciated that the accuracy of dequantized coefficient C may not be known because information of the original coefficient may be unavailable. Since it is a lossy reconstruction, dequantized coefficient C may bring blocking artifacts to the reconstructed picture. It should also be appreciated that embodiments of the invention may not require inverse quantizer 105 where the coefficient received at transcoding system 100 is already dequantized.
- Deblocking mechanism 110 is operable to perform a deblocking operation on dequantized coefficient C.
- a DCT domain deblocking operation is performed to obtain a second coefficient C′.
- DCT domain deblocking operations modify DCT coefficients depending on the neighboring coefficients in the subbands. It should be appreciated that DCT domain deblocking operations are understood in the art. An example of a DCT domain deblocking operation is described in Triantafyllidis et al., “Blocking Artifact Reduction in Frequency Domain,” Proc. ICIP 2001, pp 269-272, 2001, which is incorporated herein by reference as background material.
- Transcoder 115 is operable to requantize second coefficient C′ using quantization step size s 2 , resulting in transcoded coefficient C q ′ (e.g., a re-quantized coefficient).
- quantization step size s 2 is produced by a rate control module of transcoder 115 that regulates the bit budget spent for each macroblock depending on a block complexity measure and current bit-consumption status.
- quantization step size s 2 is less than quantization step size s 1 . Therefore, transcoded coefficient C q ′ is less than or equal to quantized coefficient C q , possibly requiring a fewer number of bits to code transcoded coefficient C q ⁇ 1 Bit rate reduction is thus achieved.
- coefficients modified after a deblocking operation are still subject to transcoding (e.g., re-quantization) that may render the deblocking modification ineffective.
- some coefficients may be responsive to a deblocking operation while other coefficients may be unresponsive to a deblocking operation.
- Embodiments of the invention are directed towards determining whether a coefficient is responsive to deblocking in order to accelerate the deblocking operation in transcoding.
- FIG. 2 is an example illustrating that some coefficients may be responsive to a deblocking operation while other coefficients may be unresponsive to a deblocking operation.
- FIG. 2 illustrates a graph 200 of exemplary quantization bins for two quantizers in accordance with an embodiment of the present invention.
- quantizer Q 1 and quantizer Q 2 are both uniform scalar quantizers
- Graph 200 illustrates two different sets of bins generated with two different quantization steps.
- the original quantization step size for quantizer Q 1 is quantization step size s
- the transcoding step size for quantizer Q 2 is quantization step size s 2 .
- the reconstructed coefficient is represented by the middle value of bin a, as represented by the arrow.
- the adjusted coefficient can vary within the range of bin a. Consequently, the requantization result can fall either in bin A or bin B of quantizer Q 2 .
- a post-deblocking coefficient may end up in different bins of Q 2 , the coefficient is responsive to deblocking.
- the post-deblocking coefficient always falls into bin C regardless of how the deblocking process alters a′ since the whole range of a′ is within C.
- the input coefficient is unresponsive to deblocking. Therefore, for unresponsive coefficients, no deblocking operation need be applied on them, as deblocking is ineffective.
- C i L is 1 ⁇ s 1 /2 ⁇
- C i U is 1 ⁇ s 1 /2 ⁇ + s 1 ⁇ 1
- C j L js 2 /2 ⁇ s 2 /2 ⁇
- C j U js 2 ⁇ s 2 /2 ⁇ +s 2 1
- ⁇ i,j ⁇ int, C i L ⁇ C j L and C i U ⁇ C j U ⁇ (4) Plugging Equation (3) into the condition items in Equation (4) and noting that ⁇ s/2 ⁇ + ⁇ s/2 ⁇ s, a quantization theorem is derived.
- the magnitude of unresponsive coefficients i that belongs to C q satisfies: ( j ⁇ ⁇ s 2 - ⁇ s 2 2 ⁇ + ⁇ s 1 2 ⁇ ) / s 1 ⁇ i ⁇ ( j ⁇ ⁇ s 2 + ⁇ s 2 2 ⁇ - ⁇ s 1 2 ⁇ ) / s 1 ( 5 ) and the magnitude of the responsive coefficients i′ fall into the intervals defined by: ( j ⁇ ⁇ s 2 + ⁇ s 2 2 ⁇ - ⁇ s 1 2 ⁇ ) / s 1 ⁇ i ′ ⁇ ( j ⁇ ⁇ s 2 + ⁇ s 2 2 ⁇ + ⁇ s 1 2 ⁇ ) / s 1 ( 6 ) where j
- Equation (7) considers a round-down scheme with uniform scalar quantizer only. It should be appreciated that other scalar quantizers can be used. For example, a round-down scheme with dead zone quantizer (for quantization of inter pictures in MPEG) can be evaluated similarly.
- a coefficient responsiveness table for recording responsive or unresponsive coefficients for a range of coefficients can be obtained according to a quantization operation (e.g., quantization function ⁇ ).
- a quantization operation e.g., quantization function ⁇
- the range R [0,1024]. It should be appreciated that the range depends on media stream encoding, such as MPEG, H.26x, and JPEG.
- the quantization operation is a uniform scalar quantizer with rounding to nearest as shown in Equation (6).
- the quantization operation is a uniform scalar quantizer with rounding down as shown in Equation (7).
- L j the left side of Equations (6) and (7)
- U j the right side of Equations (6) and (7)
- Equations (6) and (7) can be used to generate a coefficient responsiveness table indicating the responsiveness for a coefficient.
- FIG. 3 illustrates a flow chart of a process 300 for determining responsiveness of a coefficient of a media stream in accordance with an embodiment of the present invention.
- process 300 is carried out by processors and electrical components (e.g., a computer system) under the control of computer readable and computer executable instructions.
- processors and electrical components e.g., a computer system
- FIG. 3 illustrates a flow chart of a process 300 for determining responsiveness of a coefficient of a media stream in accordance with an embodiment of the present invention.
- process 300 is carried out by processors and electrical components (e.g., a computer system) under the control of computer readable and computer executable instructions.
- specific steps are disclosed in process 300 , such steps are exemplary. That is, the embodiments of the present invention are well suited to performing various other steps or variations of the steps recited in FIG. 3 .
- a plurality of first coefficients and a plurality of second coefficients associated with plurality of blocks of pixels, a first quantization step size, a second quantization step size, and a quantization operation are received.
- the plurality of first coefficients are quantized coefficients and the plurality of second coefficients are transcoded coefficients.
- the quantization operation is a uniform scalar quantizer with rounding to nearest. In another embodiment, the quantization operation is a uniform scalar quantizer with rounding down.
- the quantization operation is performed for a first coefficient of the plurality of first coefficients and a second coefficient of the plurality of second coefficients.
- the quantization operation utilizes the first quantization step size, the second quantization step size and the second coefficient.
- the quantization operation determines whether the first coefficient falls within an interval defined by the first quantization step size, the second quantization step size and the second coefficient.
- the quantization operation defines a lower bound and an upper bound. Based on the first quantization step size, the second quantization step size and the second coefficient, the lower bound and upper bound are determined. If lower bound is less than the first coefficient and the upper bound is greater than the first coefficient, the first coefficient is indicated as responsive.
- step 340 it is determined whether more coefficients require processing by the quantization operation. In one embodiment, it is determined whether the quantization operation has been performed on every combination of first coefficients of the plurality of first coefficients and second coefficients of the plurality of second coefficients. If there are more coefficients that require processing by the quantization operation, process 300 returns to step 320 with the next combination of first coefficients and second coefficients. Alternatively, if there are no more coefficients that require processing by the quantization operation, process 300 proceeds to step 350 .
- a coefficient responsiveness table is generated comprising a distribution of responsiveness of the plurality of first coefficients for the first quantization step size.
- a first axis of the coefficient responsiveness table corresponds to the plurality of first coefficients and the second axis corresponds to the second quantization step size.
- the coefficient responsiveness table can be generated prior to the completion of step 340 . That is, the coefficient responsiveness table can be generated one coefficient at a time, such that once responsiveness of a first coefficient is determined at step 330 , the responsiveness is indicated in the coefficient responsiveness table.
- FIG. 4 illustrates a flow chart of a process 400 for determining responsiveness of a coefficient of a media stream using a uniform scalar quantizer with rounding to nearest in accordance with an embodiment of the present invention.
- process 400 is carried out by processors and electrical components (e.g., a computer system) under the control of computer readable and computer executable instructions.
- processors and electrical components e.g., a computer system
- steps are exemplary. That is, the embodiments of the present invention are well suited to performing various other steps or variations of the steps recited in FIG. 4 .
- all possible quantized coefficients are set as unresponsive.
- L j e.g., the right side of Equation (6)
- U j e.g., the left side of Equation (6)
- Equation (6) it is determined whether Equation (6) is satisfied for the quantized coefficient. In other words, it is determined whether the quantized coefficient falls within the interval of L j and U j . If the quantized coefficient falls within the interval of L j and U j , as shown at step 410 , the quantized coefficient is set as responsive, and process 400 proceeds to step 412 . Alternatively, if the quantized coefficient does not fall within the interval of L j and U j process 400 proceeds to step 412 .
- step 412 it is determined whether all values of quantized coefficients have been processed at step 408 . If all values of quantized coefficients have not been processed, as shown at step 414 , the next possible quantized coefficient is selected and process 400 proceeds to step 408 . Alternatively, if all values of quantized coefficients have been processed, process 400 proceeds to step 416 .
- step 416 it is determined whether all intervals of L j and U j have been processed at step 408 . If all intervals have not been processed, as shown at step 418 , the next possible transcoded coefficient is selected and process 400 proceeds to step 408 . Alternatively, if all intervals have been processed, process 400 proceeds to step 420 . At step 420 , process 400 ends.
- FIG. 5A illustrates a graph 500 of an exemplary distribution of responsive and unresponsive coefficients using a uniform scalar quantizer with rounding to nearest in accordance with an embodiment of the present invention.
- the vertical axis indicates the second quantization step size s 2 which ranges from 1-31 in the case of MPEG-2 video streams, and the horizontal axis indicates the quantized coefficient C q (e.g., first coefficient), which ranges from 1-1024 in the case of MPEG-2 video streams.
- graph 200 only illustrates the first 30 quantized coefficients.
- ‘o’ indicates a responsive coefficient and ‘x’ indicates an unresponsive coefficient.
- the second quantization step size is greater than the first quantization step size, it should be understood that the responsiveness is only determined for second quantization step size from four through thirty-one.
- graph 500 is generated for each first quantization step size. However, it should be appreciated that a three-dimensional table can be generated that includes all first quantization step sizes, second quantization step sizes, and quantized coefficients.
- FIG. 5B illustrates a graph 510 of an exemplary distribution of responsive and unresponsive coefficients using a uniform scalar quantizer with rounding down in accordance with an embodiment of the present invention.
- Graph 510 is generated in a manner similar to graph 500 of FIG. 5A , and is for an MPEG-2 video stream having a first quantization step size of three. Due to the different quantizer used, the responsiveness of coefficients is different than the quantizer used at graph 500 .
- second quantization step size s 2 is typically no smaller than first quantization step size s 1 .
- the transcoded coefficient is smaller than the input quantized coefficient.
- bit rate reduction transcoding for any given i that belongs to C q , after deblocking there are at most two possible requantization outputs. Using this information, a deblocking capable transcoder can be implemented that only performs a deblocking operation on responsive coefficients, thus facilitating efficient throughput of input coefficients.
- FIG. 6 illustrates a block diagram of transcoding system 600 for deblocking responsive coefficients in accordance with an embodiment of the present invention.
- Transcoding system 600 utilizes requantization for bit rate reduction transcoding and has deblocking capability.
- Transcoding system 600 comprises inverse quantizer 605 , coefficient responsiveness table 610 , responsiveness selector 620 , deblocking mechanism 610 , and transcoder 615 . It should be appreciated that transcoding system 600 may be implemented within a computer system.
- Transcoding system 600 receives a coefficient associated with a block of pixels of a media stream.
- the coefficient is a quantized coefficient C q .
- quantized coefficient C q is received from a compressed video stream.
- quantized coefficient C q is a discrete cosine transform (DCT) coefficient.
- Inverse quantizer 605 is operable to perform an inverse quantization operation (Q 1 ⁇ 1 ) on quantized coefficient C q using quantization step size s 1 .
- Inverse quantizer 605 outputs dequantized coefficient C.
- inverse quantizer operates in the same manner as inverse quantizer 105 of FIG. 1 .
- Transcoder 615 is operable to perform a transcoding (e.g., quantization) operation (Q 2 ) on dequantized coefficient C using quantization step size s 2 .
- Transcoder 615 outputs transcoded coefficient C q ′.
- transcoder 615 operates in the same manner as transcoder 115 of FIG. 1 .
- Responsiveness selector 630 performs a table-lookup operation using quantized coefficient C q as an input, as well as quantization step sizes s 1 and s 2 .
- the table lookup operation is based on coefficient responsiveness table 620 which is generated beforehand.
- coefficient responsiveness table 620 is generated as described at process 300 of FIG. 3 or process 400 of FIG. 4 .
- an input coefficient i is mapped to a corresponding C q ′ of coefficient responsiveness table 620 .
- Responsiveness selector 630 performs a table-lookup operation and the output of 630 is used to control a switch arrow 635 . If i that belongs to C q is unresponsive according to coefficient responsiveness table 620 , switch arrow 635 is connected to unresponsive node 640 , transcoded coefficient C q ′ is output. Alternatively, if i is responsive according to coefficient responsiveness table 620 , switch arrow 635 is connected to responsive node 645 . Subsequently, a deblocking process of deblocking mechanism 610 is applied on the transcoded coefficient C q ′ according to dequantized coefficient C. Transcoded coefficient C q ′ is modified by ⁇ 1 or zero, and is output as C q ′′. In one embodiment, deblocking mechanism 610 operates in the same manner as deblocking mechanism 110 of FIG. 1 . In one embodiment, a memory unit of transcoding system 600 is used to pass dequantized coefficient C to deblocking mechanism 610 .
- quantization step size s 1 is small, there is typically minimal blocking introduced into the video stream. However, if quantization step size s 2 is much greater than quantization step size s 1 , there may be substantial blocking artifacts introduced.
- the distribution of responsive coefficients may become sparse with increasing quantization step size s 2 . Noting that the interval between the lower and upper bounds in Equation (5) equals s 2 /s 1 ⁇ 1, the following corollary can be derived. In general, for a uniform scalar quantizer, given first quantization step size s 1 and second quantization step size s 2 , the number of unresponsive coefficients increases when s 2 /s 1 increases.
- a bigger or smaller quantization step size in the vicinity of quantization step size s 2 is selected, leading to possibly more responsive coefficients.
- a plot of quantization error is generated.
- the quantization error is plotted based on standard TM5 quantization procedure employed in the MPEG standard. Given the range of quantization step size s 2 from 1 to 31, and possible magnitudes of DCT coefficients from 0 to 2048, a quantization error table containing all possible quantization errors for each (s 2 , C q ) pair can be generated.
- FIG. 7 illustrates an exemplary quantization error table 700 in accordance with an embodiment of the present invention.
- quantization error table 700 shows the quantization error for DCT coefficients for values from 0 to 1028 using the MPEG standard. For simplicity, only the first 64 coefficients are shown. The quantizer function and the default intra quantizer matrix used in MPEG are considered here.
- the visualization of the table only shows the results for the (0,1), (1,0) and (1,1) coefficients (the value of these entries in the default quantizer matrix is 16).
- the troughs (darker cells) indicate smaller quantization error. For a vertical line (for example the vertical line for coefficient value at 40 ) in FIG. 7 , multiple troughs are encountered, which indicates that a higher quantization step size s 2 value does not necessarily increase the quantization error. On the contrary, if quantization step size s 2 is selected carefully, the quantization error may be maintained or even reduced.
- An optimal quantization step size s 2 can be selected to boost the number of responsive coefficients so that the deblocking mechanism (e.g., deblocking mechanism 110 of FIG. 1 or deblocking mechanism 610 of FIG. 6 ) can be more effective. Furthermore, the altered quantization step size maintains the bit rate and the quality of the coded pictures. Based on this, a deblocking-capable transcoder can be implemented with enhanced deblocking capability. The enhanced deblocking capability can be implemented in transcoding system 100 of FIG. 1 and transcoding system 600 of FIG. 6 by selecting an optimal second quantization step size s 2 .
- FIG. 8 illustrates a flow chart of a process 800 for determining an optimal quantization step size in accordance with an embodiment of the present invention.
- process 800 is carried out by processors and electrical components (e.g., a computer system) under the control of computer readable and computer executable instructions.
- processors and electrical components e.g., a computer system
- FIG. 8 illustrates a flow chart of a process 800 for determining an optimal quantization step size in accordance with an embodiment of the present invention.
- process 800 is carried out by processors and electrical components (e.g., a computer system) under the control of computer readable and computer executable instructions.
- specific steps are disclosed in process 800 , such steps are exemplary. That is, the embodiments of the present invention are well suited to performing various other steps or variations of the steps recited in FIG. 8 .
- an input quantization step size and a plurality of coefficients of a macroblock are received.
- an input quantization step size and a plurality of coefficients are received for a group of blocks (e.g., in the case of JPEG encoding).
- a magnitude distribution of non-zero coefficients of the plurality of coefficients is determined.
- a plurality of candidate quantization step sizes is determined based on a first table.
- the first table is a quantization error table.
- a quantization error for a range of quantization step sizes including the input quantization step size is determined.
- the range of quantization step sizes comprises all quantization step sizes between the input quantization step size plus three and the input quantization step size minus three.
- the range of quantization step sizes ranges from s 2 ⁇ 3 to s 2 +3.
- a plurality of candidate quantization step sizes from is selected form the range of quantization step sizes.
- the plurality of candidate quantization step sizes produce a quantization error not greater than quantization error associated with the input quantization step size.
- the optimal quantization step size is determined from said plurality of quantization step sizes based on a second table.
- the second table is a coefficient responsiveness table.
- the optimal quantization step size is selected as a quantization step size of plurality of quantization step sizes that has the highest number of responsive coefficients according to the coefficient responsiveness table.
- one quantization step size s 2 is applied to the quantization of 6 to 12 blocks in a macroblock.
- the distribution of non-zero coefficients is obtained considering all the blocks.
- Weighting factors can be assigned to coefficients at different frequencies based on their importance to the reconstruction quality. In one embodiment, using the lower frequency coefficients (e.g., coefficients in the upper left 2 ⁇ 2 in a block) can be implemented.
- Various embodiments of the present invention provide an analytical model for identifying transform coefficients that are responsive to deblocking in a video transcoding process.
- a method that enables a transcoder to avoid unnecessary deblocking operations in the transcoding process is provided.
- An optimal quantizer selection method is also provided to increase the number of responsive coefficients so that the deblocking capability of the transcoder is enhanced.
- Embodiments of the present invention a method for deblocking and transcoding a media stream, are thus described. While the present invention has been described in particular embodiments, it should be appreciated that the present invention should not be construed as limited by such embodiments, but rather construed according to the following claims.
Abstract
Description
- Embodiments of the present invention relate to the field of digital media. Specifically, embodiments of the present invention relate to a method and system for deblocking and transcoding a media stream.
- One of the important video transcoding applications is bit rate reduction transcoding. Rate reduction transcoding is typically achieved using either truncation based transcoding or by requantization-based transcoding. Selective coefficient truncating transcoding typically achieves a relatively small amount of bit rate reduction and is rarely used often in typical video transcoding applications.
- Requantization-based transcoding achieves rate reduction by requantization using coarser quantization steps. Since the transform-domain coefficients are coarsely quantized, blocking artifacts may be produced. In particular, blocking artifacts arise as a result of each block being compressed independently. Blocking artifacts may appear as edges or produce ringing between blocks.
- Numerous methods have been developed to reduce the blocking artifacts for decoded pixel blocks. Specifically, current deblocking applications typically decode the pixel blocks before deblocking and re-encode the pixel blocks after decoding. However, for transcoding applications, the decoding and re-encoding cycle may result in increased blocking artifacts, as the decoding and re-encoding may rely on information that is not be available. Furthermore, due to the higher compression provided by transcoding applications, blocking artifacts get worse as the compression is increased. For example, requantization-based transcoding applies the requantization on the transform-domain coefficients so that a costly inverse transformation is avoided. Currently, deblocking cannot be performed in conjunction with transcoding in the transform domain.
- Various embodiments of the present invention, a method for deblocking and transcoding a media stream, are described. A coefficient associated with a block of pixels of the media stream is received. A deblocking operation is performed on the coefficient to generate a second coefficient. Quantization is performed on the second coefficient to generate a transcoded coefficient.
- The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention:
-
FIG. 1 illustrates a block diagram of transcoding system in accordance with an embodiment of the present invention. -
FIG. 2 illustrates a graph of exemplary quantization bins for two quantizers in accordance with an embodiment of the present invention. -
FIG. 3 illustrates a flow chart of a process for determining responsiveness of a coefficient of a media stream in accordance with an embodiment of the present invention. -
FIG. 4 illustrates a flow chart of a process for determining responsiveness of a coefficient of a media stream using a uniform scalar quantizer with rounding to nearest in accordance with an embodiment of the present invention. -
FIG. 5A illustrates a graph of an exemplary distribution of responsive and unresponsive coefficients using a uniform scalar quantizer with rounding to nearest in accordance with an embodiment of the present invention. -
FIG. 5B illustrates a graph of an exemplary distribution of responsive and unresponsive coefficients using a uniform scalar quantizer with rounding down in accordance with an embodiment of the present invention. -
FIG. 6 illustrates a block diagram of transcoding system for deblocking responsive coefficients in accordance with an embodiment of the present invention. -
FIG. 7 illustrates an exemplary quantization error table in accordance with an embodiment of the present invention. -
FIG. 8 illustrates a flow chart of a process for determining an optimal quantization step size in accordance with an embodiment of the present invention. - Reference will now be made in detail to various embodiments of the invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with these embodiments, it will be understood that they are not intended to limit the invention to these embodiments. On the contrary, the invention is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope of the invention as defined by the appended claims. Furthermore, in the following description of the present invention, numerous specific details are set forth in order to provide a thorough understanding of the present invention. In other instances, well-known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the present invention.
- Aspects of the present invention may be implemented in a computer system that includes, in general, a processor for processing information and instructions, random access (volatile) memory (RAM) for storing information and instructions, read-only (non-volatile) memory (ROM) for storing static information and instructions, a data storage device such as a magnetic or optical disk and disk drive for storing information and instructions, an optional user output device such as a display device (e.g., a monitor) for displaying information to the computer user, an optional user input device including alphanumeric and function keys (e.g., a keyboard) for communicating information and command selections to the processor, and an optional user input device such as a cursor control device (e.g., a mouse) for communicating user input information and command selections to the processor.
-
FIG. 1 illustrates a block diagram oftranscoding system 100 in accordance with an embodiment of the present invention.Transcoding system 100 utilizes requantization for bit rate reduction transcoding and has deblocking capability.Transcoding system 100 comprises three modules:inverse quantizer 105,deblocking mechanism 110, andtranscoder 115. It should be appreciated thattranscoding system 100 may be implemented within a computer system. -
Transcoding system 100 receives a coefficient associated with a block of pixels of a media stream. Embodiments of the present invention pertain to media stream for transmitting graphical data, such as video streams. For brevity and clarity, the term video stream is used in the present application. In one embodiment, the coefficient is a quantized coefficient Cq. In one embodiment, quantized coefficient Cq is received from a compressed video stream. In one embodiment, quantized coefficient Cq is a discrete cosine transform (DCT) coefficient. -
Inverse quantizer 105 is operable to perform an inverse quantization operation (Q1 −1) on quantized coefficient Cq using quantization step size s1.Inverse quantizer 105 outputs dequantized coefficient C. It should be appreciated that the accuracy of dequantized coefficient C may not be known because information of the original coefficient may be unavailable. Since it is a lossy reconstruction, dequantized coefficient C may bring blocking artifacts to the reconstructed picture. It should also be appreciated that embodiments of the invention may not requireinverse quantizer 105 where the coefficient received attranscoding system 100 is already dequantized. -
Deblocking mechanism 110 is operable to perform a deblocking operation on dequantized coefficient C. In one embodiment, a DCT domain deblocking operation is performed to obtain a second coefficient C′. DCT domain deblocking operations modify DCT coefficients depending on the neighboring coefficients in the subbands. It should be appreciated that DCT domain deblocking operations are understood in the art. An example of a DCT domain deblocking operation is described in Triantafyllidis et al., “Blocking Artifact Reduction in Frequency Domain,” Proc. ICIP2001, pp 269-272, 2001, which is incorporated herein by reference as background material. -
Transcoder 115 is operable to requantize second coefficient C′ using quantization step size s2, resulting in transcoded coefficient Cq′ (e.g., a re-quantized coefficient). In one embodiment, quantization step size s2 is produced by a rate control module oftranscoder 115 that regulates the bit budget spent for each macroblock depending on a block complexity measure and current bit-consumption status. In one embodiment, quantization step size s2 is less than quantization step size s1. Therefore, transcoded coefficient Cq′ is less than or equal to quantized coefficient Cq, possibly requiring a fewer number of bits to code transcoded coefficient Cq −1 Bit rate reduction is thus achieved. - It should be understood that in video transcoding, coefficients modified after a deblocking operation are still subject to transcoding (e.g., re-quantization) that may render the deblocking modification ineffective. In particular, some coefficients may be responsive to a deblocking operation while other coefficients may be unresponsive to a deblocking operation.
- Embodiments of the invention are directed towards determining whether a coefficient is responsive to deblocking in order to accelerate the deblocking operation in transcoding.
FIG. 2 is an example illustrating that some coefficients may be responsive to a deblocking operation while other coefficients may be unresponsive to a deblocking operation. Specifically,FIG. 2 illustrates agraph 200 of exemplary quantization bins for two quantizers in accordance with an embodiment of the present invention. In one embodiment, quantizer Q1 and quantizer Q2 are both uniform scalar quantizers,Graph 200 illustrates two different sets of bins generated with two different quantization steps. The original quantization step size for quantizer Q1 is quantization step size s, and the transcoding step size for quantizer Q2 is quantization step size s2. - For illustrative purposes, if an input coefficient falls into bin a of quantizer Q1, the reconstructed coefficient is represented by the middle value of bin a, as represented by the arrow. After deblocking, the adjusted coefficient can vary within the range of bin a. Consequently, the requantization result can fall either in bin A or bin B of quantizer Q2. If a post-deblocking coefficient may end up in different bins of Q2, the coefficient is responsive to deblocking. Alternatively, if an input coefficient falls into bin a′, the post-deblocking coefficient always falls into bin C regardless of how the deblocking process alters a′ since the whole range of a′ is within C. The input coefficient is unresponsive to deblocking. Therefore, for unresponsive coefficients, no deblocking operation need be applied on them, as deblocking is ineffective.
- The value of the unresponsive coefficients can be derived as follows given the quantizer functions ƒ, quantization steps s1 and s2. Assuming a uniform scale quantizer with rounding to nearest, the inverse quantization function ƒ1 is considered. Given an input DCT coefficient (a quantized DCT coefficient) Cq=n, the reconstructed coefficient is Cn=ƒ1(Cq,s)=ns, where s is the quantization step size. The quantization lower bound of Cn is defined as:
C n L =ns−└s/2┘ (1)
and the upper bound is defined as:
C n U =ns−└s/2┘+s−1 (2)
Note that the round-to-nearest quantizing (values at 0.5 are rounded to the larger magnitude) leads to the round-down term in Equations (1) and (2). Let i that belongs to Cq, the set of input coefficients, and j that belongs to Cq′, the set of coefficients after requantization, Cq and Cq′ are subsets of integers, from Equations (1) and (2) results in:
C i L =is 1 −└s 1/2┘, C i U =is 1 −└s 1/2┘+s 1−1
C j L =js 2/2−└s 2/2┘, C j U =js 2 └s 2/2┘+s 21 (3)
The set of unresponsive coefficients is defined as:
U={i|∀i,j∈int, C i L≧Cj L and C i U ≦C j U} (4)
Plugging Equation (3) into the condition items in Equation (4) and noting that ┌s/2┐+└s/2┘=s, a quantization theorem is derived. Specifically, for a uniform scalar quantizer (round-to-nearest quantizing), given first quantizer step size s1 and requantizer step size s2, the magnitude of unresponsive coefficients i that belongs to Cq, satisfies:
and the magnitude of the responsive coefficients i′ fall into the intervals defined by:
where j belongs to Cq′. - Alternatively, if round-down instead of round-to-nearest is considered in quantizing, the intervals for responsive coefficients become:
Equation (7) considers a round-down scheme with uniform scalar quantizer only. It should be appreciated that other scalar quantizers can be used. For example, a round-down scheme with dead zone quantizer (for quantization of inter pictures in MPEG) can be evaluated similarly. - In general, given any (s1, s2) pair, a coefficient responsiveness table for recording responsive or unresponsive coefficients for a range of coefficients can be obtained according to a quantization operation (e.g., quantization function ƒ). For example, if the quantized coefficients are obtained from an MPEG-2 video stream, the range R=[0,1024]. It should be appreciated that the range depends on media stream encoding, such as MPEG, H.26x, and JPEG.
- In one embodiment, the quantization operation is a uniform scalar quantizer with rounding to nearest as shown in Equation (6). In another embodiment, the quantization operation is a uniform scalar quantizer with rounding down as shown in Equation (7). For ease of explanation, the left side of Equations (6) and (7) is denoted as Lj and right side is denoted as Uj. For both Equations (6) and (7), it can be proven that Lj−Uj=1. In other words, there is at most one responsive coefficient in each interval. Therefore, Equations (6) and (7) can be used to generate a coefficient responsiveness table indicating the responsiveness for a coefficient.
-
FIG. 3 illustrates a flow chart of aprocess 300 for determining responsiveness of a coefficient of a media stream in accordance with an embodiment of the present invention. In one embodiment,process 300 is carried out by processors and electrical components (e.g., a computer system) under the control of computer readable and computer executable instructions. Although specific steps are disclosed inprocess 300, such steps are exemplary. That is, the embodiments of the present invention are well suited to performing various other steps or variations of the steps recited inFIG. 3 . - At
step 310, a plurality of first coefficients and a plurality of second coefficients associated with plurality of blocks of pixels, a first quantization step size, a second quantization step size, and a quantization operation are received. In one embodiment, the plurality of first coefficients are quantized coefficients and the plurality of second coefficients are transcoded coefficients. In one embodiment, the quantization operation is a uniform scalar quantizer with rounding to nearest. In another embodiment, the quantization operation is a uniform scalar quantizer with rounding down. - At
step 320, the quantization operation is performed for a first coefficient of the plurality of first coefficients and a second coefficient of the plurality of second coefficients. The quantization operation utilizes the first quantization step size, the second quantization step size and the second coefficient. In one embodiment, the quantization operation determines whether the first coefficient falls within an interval defined by the first quantization step size, the second quantization step size and the second coefficient. - At
step 330, it is determined whether the first coefficient is responsive based on the quantization operation. In one embodiment, if the coefficient falls within an interval as defined by the quantization operation, the first coefficient is identified as being responsive. Alternatively, if the coefficient does not fall within the interval as defined by the quantization operation, the first coefficient is identified as being unresponsive. In one embodiment, the quantization operation defines a lower bound and an upper bound. Based on the first quantization step size, the second quantization step size and the second coefficient, the lower bound and upper bound are determined. If lower bound is less than the first coefficient and the upper bound is greater than the first coefficient, the first coefficient is indicated as responsive. - At
step 340, it is determined whether more coefficients require processing by the quantization operation. In one embodiment, it is determined whether the quantization operation has been performed on every combination of first coefficients of the plurality of first coefficients and second coefficients of the plurality of second coefficients. If there are more coefficients that require processing by the quantization operation,process 300 returns to step 320 with the next combination of first coefficients and second coefficients. Alternatively, if there are no more coefficients that require processing by the quantization operation,process 300 proceeds to step 350. - At
step 350, a coefficient responsiveness table is generated comprising a distribution of responsiveness of the plurality of first coefficients for the first quantization step size. A first axis of the coefficient responsiveness table corresponds to the plurality of first coefficients and the second axis corresponds to the second quantization step size. It should be appreciated that the coefficient responsiveness table can be generated prior to the completion ofstep 340. That is, the coefficient responsiveness table can be generated one coefficient at a time, such that once responsiveness of a first coefficient is determined atstep 330, the responsiveness is indicated in the coefficient responsiveness table. -
FIG. 4 illustrates a flow chart of aprocess 400 for determining responsiveness of a coefficient of a media stream using a uniform scalar quantizer with rounding to nearest in accordance with an embodiment of the present invention. In one embodiment,process 400 is carried out by processors and electrical components (e.g., a computer system) under the control of computer readable and computer executable instructions. Although specific steps are disclosed inprocess 400, such steps are exemplary. That is, the embodiments of the present invention are well suited to performing various other steps or variations of the steps recited inFIG. 4 . - At
step 402 ofFIG. 4 , all possible quantized coefficients are set as unresponsive. Atstep 404, Lj (e.g., the right side of Equation (6)) and Uj (e.g., the left side of Equation (6)) are determined starting from the first interval where a transcoded coefficient of a plurality of transcoded coefficients is set equal to one (e.g., j=1). Atstep 406, the first possible quantized coefficient of the plurality of quantized coefficients is set equal to one (e.g., i=1). - At
step 408, it is determined whether Equation (6) is satisfied for the quantized coefficient. In other words, it is determined whether the quantized coefficient falls within the interval of Lj and Uj. If the quantized coefficient falls within the interval of Lj and Uj, as shown atstep 410, the quantized coefficient is set as responsive, andprocess 400 proceeds to step 412. Alternatively, if the quantized coefficient does not fall within the interval of Lj and Uj process 400 proceeds to step 412. - At
step 412 it is determined whether all values of quantized coefficients have been processed atstep 408. If all values of quantized coefficients have not been processed, as shown atstep 414, the next possible quantized coefficient is selected andprocess 400 proceeds to step 408. Alternatively, if all values of quantized coefficients have been processed,process 400 proceeds to step 416. - At
step 416 it is determined whether all intervals of Lj and Uj have been processed atstep 408. If all intervals have not been processed, as shown atstep 418, the next possible transcoded coefficient is selected andprocess 400 proceeds to step 408. Alternatively, if all intervals have been processed,process 400 proceeds to step 420. Atstep 420,process 400 ends. -
FIG. 5A illustrates agraph 500 of an exemplary distribution of responsive and unresponsive coefficients using a uniform scalar quantizer with rounding to nearest in accordance with an embodiment of the present invention.Graph 500 shows the actual distribution of responsive and unresponsive coefficients for an MPEG-2 video stream with a first quantization step size of three (s1=3). The vertical axis indicates the second quantization step size s2 which ranges from 1-31 in the case of MPEG-2 video streams, and the horizontal axis indicates the quantized coefficient Cq (e.g., first coefficient), which ranges from 1-1024 in the case of MPEG-2 video streams. For simplicity,graph 200 only illustrates the first 30 quantized coefficients. As shown, ‘o’ indicates a responsive coefficient and ‘x’ indicates an unresponsive coefficient. Also, since the second quantization step size is greater than the first quantization step size, it should be understood that the responsiveness is only determined for second quantization step size from four through thirty-one. In one embodiment,graph 500 is generated for each first quantization step size. However, it should be appreciated that a three-dimensional table can be generated that includes all first quantization step sizes, second quantization step sizes, and quantized coefficients. -
FIG. 5B illustrates agraph 510 of an exemplary distribution of responsive and unresponsive coefficients using a uniform scalar quantizer with rounding down in accordance with an embodiment of the present invention.Graph 510 is generated in a manner similar to graph 500 ofFIG. 5A , and is for an MPEG-2 video stream having a first quantization step size of three. Due to the different quantizer used, the responsiveness of coefficients is different than the quantizer used atgraph 500. - It should be appreciated that second quantization step size s2 is typically no smaller than first quantization step size s1. Thus, the transcoded coefficient is smaller than the input quantized coefficient. In bit rate reduction transcoding, for any given i that belongs to Cq, after deblocking there are at most two possible requantization outputs. Using this information, a deblocking capable transcoder can be implemented that only performs a deblocking operation on responsive coefficients, thus facilitating efficient throughput of input coefficients.
-
FIG. 6 illustrates a block diagram oftranscoding system 600 for deblocking responsive coefficients in accordance with an embodiment of the present invention.Transcoding system 600 utilizes requantization for bit rate reduction transcoding and has deblocking capability.Transcoding system 600 comprisesinverse quantizer 605, coefficient responsiveness table 610,responsiveness selector 620,deblocking mechanism 610, andtranscoder 615. It should be appreciated thattranscoding system 600 may be implemented within a computer system. -
Transcoding system 600 receives a coefficient associated with a block of pixels of a media stream. In one embodiment, the coefficient is a quantized coefficient Cq. In one embodiment, quantized coefficient Cq is received from a compressed video stream. In one embodiment, quantized coefficient Cq is a discrete cosine transform (DCT) coefficient.Inverse quantizer 605 is operable to perform an inverse quantization operation (Q1 −1) on quantized coefficient Cq using quantization step size s1.Inverse quantizer 605 outputs dequantized coefficient C. In one embodiment, inverse quantizer operates in the same manner asinverse quantizer 105 ofFIG. 1 . - Dequantized coefficient C is transmitted to
transcoder 615.Transcoder 615 is operable to perform a transcoding (e.g., quantization) operation (Q2) on dequantized coefficient C using quantization step size s2.Transcoder 615 outputs transcoded coefficient Cq′. In one embodiment,transcoder 615 operates in the same manner astranscoder 115 ofFIG. 1 . -
Responsiveness selector 630 performs a table-lookup operation using quantized coefficient Cq as an input, as well as quantization step sizes s1 and s2. In one embodiment, the table lookup operation is based on coefficient responsiveness table 620 which is generated beforehand. In one embodiment, coefficient responsiveness table 620 is generated as described atprocess 300 ofFIG. 3 orprocess 400 ofFIG. 4 . Based on quantization step size s2 proposed by a rate control module, an input coefficient i is mapped to a corresponding Cq′ of coefficient responsiveness table 620. -
Responsiveness selector 630 performs a table-lookup operation and the output of 630 is used to control aswitch arrow 635. If i that belongs to Cq is unresponsive according to coefficient responsiveness table 620, switcharrow 635 is connected tounresponsive node 640, transcoded coefficient Cq′ is output. Alternatively, if i is responsive according to coefficient responsiveness table 620, switcharrow 635 is connected toresponsive node 645. Subsequently, a deblocking process ofdeblocking mechanism 610 is applied on the transcoded coefficient Cq′ according to dequantized coefficient C. Transcoded coefficient Cq′ is modified by ±1 or zero, and is output as Cq″. In one embodiment,deblocking mechanism 610 operates in the same manner asdeblocking mechanism 110 ofFIG. 1 . In one embodiment, a memory unit oftranscoding system 600 is used to pass dequantized coefficient C todeblocking mechanism 610. - If quantization step size s1 is small, there is typically minimal blocking introduced into the video stream. However, if quantization step size s2 is much greater than quantization step size s1, there may be substantial blocking artifacts introduced. The distribution of responsive coefficients may become sparse with increasing quantization step size s2. Noting that the interval between the lower and upper bounds in Equation (5) equals s2/s1−1, the following corollary can be derived. In general, for a uniform scalar quantizer, given first quantization step size s1 and second quantization step size s2, the number of unresponsive coefficients increases when s2/s1 increases. This corollary indicates that a deblocking operation has limited impact when requantization is much more coarser. In some special cases, no responsive coefficients are available at all. Note from the example shown in
FIG. 5A that when s2=9, no coefficients are responsive. - In general, for a uniform scalar quantizer, given first quantization step size s1 and second quantization step size s2, if
-
- where k=(1, 2, . . . ), there are no responsive coefficients.
- It is therefore important to avoid choosing second quantization step size s2 as in Equation (8) if a deblocking-capable transcoder is desired. From the example shown in
FIG. 5A , there are more responsive coefficients when second quantization step size s2=8 or 10. Therefore, by altering the selection of second quantization step size s2, the number of responsive coefficients may increase so that the deblocking can be more effective. - In one embodiment, a bigger or smaller quantization step size in the vicinity of quantization step size s2 is selected, leading to possibly more responsive coefficients. To select an appropriate quantization step size, a plot of quantization error is generated. In one embodiment, the quantization error is plotted based on standard TM5 quantization procedure employed in the MPEG standard. Given the range of quantization step size s2 from 1 to 31, and possible magnitudes of DCT coefficients from 0 to 2048, a quantization error table containing all possible quantization errors for each (s2, Cq) pair can be generated.
-
FIG. 7 illustrates an exemplary quantization error table 700 in accordance with an embodiment of the present invention. Specifically, quantization error table 700 shows the quantization error for DCT coefficients for values from 0 to 1028 using the MPEG standard. For simplicity, only the first 64 coefficients are shown. The quantizer function and the default intra quantizer matrix used in MPEG are considered here. The visualization of the table only shows the results for the (0,1), (1,0) and (1,1) coefficients (the value of these entries in the default quantizer matrix is 16). The troughs (darker cells) indicate smaller quantization error. For a vertical line (for example the vertical line for coefficient value at 40) inFIG. 7 , multiple troughs are encountered, which indicates that a higher quantization step size s2 value does not necessarily increase the quantization error. On the contrary, if quantization step size s2 is selected carefully, the quantization error may be maintained or even reduced. - The quantization property described above proves valuable for the deblocking-capable transcoder. An optimal quantization step size s2 can be selected to boost the number of responsive coefficients so that the deblocking mechanism (e.g.,
deblocking mechanism 110 ofFIG. 1 ordeblocking mechanism 610 ofFIG. 6 ) can be more effective. Furthermore, the altered quantization step size maintains the bit rate and the quality of the coded pictures. Based on this, a deblocking-capable transcoder can be implemented with enhanced deblocking capability. The enhanced deblocking capability can be implemented intranscoding system 100 ofFIG. 1 andtranscoding system 600 ofFIG. 6 by selecting an optimal second quantization step size s2. -
FIG. 8 illustrates a flow chart of aprocess 800 for determining an optimal quantization step size in accordance with an embodiment of the present invention. In one embodiment,process 800 is carried out by processors and electrical components (e.g., a computer system) under the control of computer readable and computer executable instructions. Although specific steps are disclosed inprocess 800, such steps are exemplary. That is, the embodiments of the present invention are well suited to performing various other steps or variations of the steps recited inFIG. 8 . - At
step 810, an input quantization step size and a plurality of coefficients of a macroblock are received. In one embodiment, an input quantization step size and a plurality of coefficients are received for a group of blocks (e.g., in the case of JPEG encoding). - At
step 820, a magnitude distribution of non-zero coefficients of the plurality of coefficients is determined. - At
step 830, a plurality of candidate quantization step sizes is determined based on a first table. In one embodiment, the first table is a quantization error table. - At
step 840, a quantization error for a range of quantization step sizes including the input quantization step size is determined. In one embodiment, the range of quantization step sizes comprises all quantization step sizes between the input quantization step size plus three and the input quantization step size minus three. In other words, for quantization step size s2, the range of quantization step sizes ranges from s2−3 to s2+3. - At
step 850, a plurality of candidate quantization step sizes from is selected form the range of quantization step sizes. In one embodiment, the plurality of candidate quantization step sizes produce a quantization error not greater than quantization error associated with the input quantization step size. - At
step 860, the optimal quantization step size is determined from said plurality of quantization step sizes based on a second table. In one embodiment, the second table is a coefficient responsiveness table. In one embodiment, the optimal quantization step size is selected as a quantization step size of plurality of quantization step sizes that has the highest number of responsive coefficients according to the coefficient responsiveness table. - It should be appreciated that in the case of MPEG, one quantization step size s2 is applied to the quantization of 6 to 12 blocks in a macroblock. The distribution of non-zero coefficients is obtained considering all the blocks. Weighting factors can be assigned to coefficients at different frequencies based on their importance to the reconstruction quality. In one embodiment, using the lower frequency coefficients (e.g., coefficients in the upper left 2×2 in a block) can be implemented.
- Various embodiments of the present invention provide an analytical model for identifying transform coefficients that are responsive to deblocking in a video transcoding process. A method that enables a transcoder to avoid unnecessary deblocking operations in the transcoding process is provided. An optimal quantizer selection method is also provided to increase the number of responsive coefficients so that the deblocking capability of the transcoder is enhanced.
- Embodiments of the present invention, a method for deblocking and transcoding a media stream, are thus described. While the present invention has been described in particular embodiments, it should be appreciated that the present invention should not be construed as limited by such embodiments, but rather construed according to the following claims.
Claims (26)
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/688,082 US7570818B2 (en) | 2003-10-17 | 2003-10-17 | Method for deblocking and transcoding a media stream |
DE102004029086A DE102004029086B4 (en) | 2003-10-17 | 2004-06-16 | Method for unblocking and transcoding a media stream |
JP2004301491A JP4105676B2 (en) | 2003-10-17 | 2004-10-15 | How to deblock and transcode media streams |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/688,082 US7570818B2 (en) | 2003-10-17 | 2003-10-17 | Method for deblocking and transcoding a media stream |
Publications (2)
Publication Number | Publication Date |
---|---|
US20050084157A1 true US20050084157A1 (en) | 2005-04-21 |
US7570818B2 US7570818B2 (en) | 2009-08-04 |
Family
ID=34521099
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/688,082 Expired - Fee Related US7570818B2 (en) | 2003-10-17 | 2003-10-17 | Method for deblocking and transcoding a media stream |
Country Status (3)
Country | Link |
---|---|
US (1) | US7570818B2 (en) |
JP (1) | JP4105676B2 (en) |
DE (1) | DE102004029086B4 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100008594A1 (en) * | 2005-08-12 | 2010-01-14 | Bialkowski Jens-Guenter | Method for correcting a quantized data value and associated device |
US20100086225A1 (en) * | 2007-01-22 | 2010-04-08 | Shoji Yachida | Image re-encoding device, image re-encoding method, and image encoding program |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090060027A1 (en) * | 2007-08-30 | 2009-03-05 | Tektronix, Inc. | Compressed Signal Subjective Quality Ratings Prediction |
JP7458312B2 (en) | 2020-12-18 | 2024-03-29 | 株式会社オカムラ | Oil supply control device, transmission unit and power system |
Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4751742A (en) * | 1985-05-07 | 1988-06-14 | Avelex | Priority coding of transform coefficients |
US4754492A (en) * | 1985-06-03 | 1988-06-28 | Picturetel Corporation | Method and system for adapting a digitized signal processing system for block processing with minimal blocking artifacts |
US5086488A (en) * | 1989-08-19 | 1992-02-04 | Mitsubishi Denki Kabushiki Kaisha | Transform coding apparatus |
US5274442A (en) * | 1991-10-22 | 1993-12-28 | Mitsubishi Denki Kabushiki Kaisha | Adaptive blocking image signal coding system |
US5432615A (en) * | 1993-06-21 | 1995-07-11 | Nec Corporation | Compression/decompression method for image data |
US5563718A (en) * | 1993-11-30 | 1996-10-08 | Polaroid Corporation | Image coding by use of discrete cosine transforms |
US5654759A (en) * | 1995-02-15 | 1997-08-05 | Hitachi America Ltd. | Methods and apparatus for reducing blockiness in decoded video |
US5694492A (en) * | 1994-04-30 | 1997-12-02 | Daewoo Electronics Co., Ltd | Post-processing method and apparatus for removing a blocking effect in a decoded image signal |
US5737451A (en) * | 1996-04-10 | 1998-04-07 | Eastman Kodak Company | Method and apparatus for suppressing blocking artifacts in block-transform coded images |
US6208688B1 (en) * | 1998-05-29 | 2001-03-27 | Korea Telecom | Method of selecting a requantization step size and controlling a bit-rate |
US6587508B1 (en) * | 1999-06-30 | 2003-07-01 | Media Glue Corporation | Apparatus, method and computer program product for transcoding a coded moving picture sequence |
US6987808B2 (en) * | 2001-01-24 | 2006-01-17 | Yamaha Corporation | Transcoding method and transcoding apparatus |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2001058167A1 (en) * | 2000-02-04 | 2001-08-09 | Koninklijke Philips Electronics N.V. | Quantization method for bit rate transcoding applications |
US20020186769A1 (en) * | 2001-06-12 | 2002-12-12 | O'brien Royal | System and method for transcoding |
-
2003
- 2003-10-17 US US10/688,082 patent/US7570818B2/en not_active Expired - Fee Related
-
2004
- 2004-06-16 DE DE102004029086A patent/DE102004029086B4/en not_active Expired - Fee Related
- 2004-10-15 JP JP2004301491A patent/JP4105676B2/en active Active
Patent Citations (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4751742A (en) * | 1985-05-07 | 1988-06-14 | Avelex | Priority coding of transform coefficients |
US4754492A (en) * | 1985-06-03 | 1988-06-28 | Picturetel Corporation | Method and system for adapting a digitized signal processing system for block processing with minimal blocking artifacts |
US5086488A (en) * | 1989-08-19 | 1992-02-04 | Mitsubishi Denki Kabushiki Kaisha | Transform coding apparatus |
US5274442A (en) * | 1991-10-22 | 1993-12-28 | Mitsubishi Denki Kabushiki Kaisha | Adaptive blocking image signal coding system |
US5432615A (en) * | 1993-06-21 | 1995-07-11 | Nec Corporation | Compression/decompression method for image data |
US5563718A (en) * | 1993-11-30 | 1996-10-08 | Polaroid Corporation | Image coding by use of discrete cosine transforms |
US5694492A (en) * | 1994-04-30 | 1997-12-02 | Daewoo Electronics Co., Ltd | Post-processing method and apparatus for removing a blocking effect in a decoded image signal |
US5654759A (en) * | 1995-02-15 | 1997-08-05 | Hitachi America Ltd. | Methods and apparatus for reducing blockiness in decoded video |
US5737451A (en) * | 1996-04-10 | 1998-04-07 | Eastman Kodak Company | Method and apparatus for suppressing blocking artifacts in block-transform coded images |
US6208688B1 (en) * | 1998-05-29 | 2001-03-27 | Korea Telecom | Method of selecting a requantization step size and controlling a bit-rate |
US6587508B1 (en) * | 1999-06-30 | 2003-07-01 | Media Glue Corporation | Apparatus, method and computer program product for transcoding a coded moving picture sequence |
US6987808B2 (en) * | 2001-01-24 | 2006-01-17 | Yamaha Corporation | Transcoding method and transcoding apparatus |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100008594A1 (en) * | 2005-08-12 | 2010-01-14 | Bialkowski Jens-Guenter | Method for correcting a quantized data value and associated device |
US8363718B2 (en) * | 2005-08-12 | 2013-01-29 | Siemens Aktiengesellschaft | Method for correcting a quantized data value and associated device |
US20100086225A1 (en) * | 2007-01-22 | 2010-04-08 | Shoji Yachida | Image re-encoding device, image re-encoding method, and image encoding program |
Also Published As
Publication number | Publication date |
---|---|
DE102004029086B4 (en) | 2009-11-19 |
JP2005167991A (en) | 2005-06-23 |
JP4105676B2 (en) | 2008-06-25 |
DE102004029086A1 (en) | 2005-05-25 |
US7570818B2 (en) | 2009-08-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20180343456A1 (en) | System and method for using pattern vectors for video and image coding and decoding | |
US8531321B1 (en) | Signal processing and inheritance in a tiered signal quality hierarchy | |
US9509988B2 (en) | Motion video encoding apparatus, motion video encoding method, motion video encoding computer program, motion video decoding apparatus, motion video decoding method, and motion video decoding computer program | |
US7664184B2 (en) | Interpolation image compression | |
US8126053B2 (en) | Image encoding/decoding method and apparatus | |
Wu | An algorithmic study on lossless image compression | |
EP2154894A1 (en) | Video coding with coding of the locations of significant coefficients in a block of coefficients | |
US20070053429A1 (en) | Color video codec method and system | |
US8634668B2 (en) | Method of compression of digital images using a fixed number of bits per block | |
US7251375B2 (en) | Tile boundary artifact removal for arbitrary wavelet filters | |
US9077960B2 (en) | Non-zero coefficient block pattern coding | |
US20020136295A1 (en) | Code quantity control apparatus, code quantity control method and picture information transformation method | |
US20110026819A1 (en) | Apparatus, method, and medium of encoding and decoding image data using sampling | |
JPH08256335A (en) | Apparatus and method for determining quantization parameter | |
JP3067628B2 (en) | Image coding device | |
CN107666472B (en) | Method and apparatus for hybrid digital-analog coding | |
US20090116550A1 (en) | Video compression system, method and computer program product using entropy prediction values | |
JPH08213915A (en) | Conversion coder | |
US6289128B1 (en) | Method and apparatus for sharing a least significant bit of encoded data between multiple numeric data values | |
US7570818B2 (en) | Method for deblocking and transcoding a media stream | |
KR101703330B1 (en) | Method and apparatus for re-encoding an image | |
EP1639831B1 (en) | Method for compression of data | |
EP1629675B1 (en) | Fixed bit rate, intraframe compression and decompression of video | |
EP1892965A2 (en) | Fixed bit rate, intraframe compression and decompression of video | |
Pan et al. | Complexity-scalable transform coding using variable complexity algorithms |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHEN, BO;REEL/FRAME:014421/0998 Effective date: 20031015 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20210804 |