CA2118880A1 - Jpeg/mpeg decoder-compatible optimized thresholding for image and video signal compression - Google Patents

Abstract

For encoding signals corresponding to still images or video sequences, respective standards known as JPEG and MPEG have been proposed. These standards are based on digital cosine transform (DCT) compression. For economy of transmission, DCT coefficients may be "thresholded" prior to transmission, by dropping the less significant DCT
coefficients. While maintaining JPEG or MPEG compatibility, threshold selection for the DCT coefficients of an image or video frame is based on optimizing for minimum distortion for a specified maximum target coding bit rate or, equivalently, for minimized coding bit rate for a specified maximum allowable distortion constraint. In the selection process, a dynamic programming method is used.

Description

-2118880 ~9530 .
JPEG/MPEG D~CODER-COMPATI~LE OPTIMI2ED T~RESHOLDING
FOR IMAG~ AND VID~O S~GNAL C0MPRESSION

Teçhnical Field The invention relates to data compression in digital i~ge and video signal processing.

Backaro~n~ of the Invention For digital coding o~ signals correspond~ng ~o still im~ges or vi~eo sequences, respective standards known as JPE~ nd ~PEG have been proposed; see "JPEG Te¢hnic~l Specification" Revision t~raft), Join~ Photographic Experts Group, ISO-I~C/~T~1/SC2/WG~, ~CITT SG VIX~, Augus~ lg90 and "Coded Repre~en~a~ion of Picture and Audio In~ormation", Test Model 1, ~raft Revision 1, Expex~s ~roup on ATM Video ~oding, IS~-IEC/JTCl/SC2~/W~ll, CCITT SG XV, May 19~2.
1~ The~e standards are based on digital cosine tran~form (~T) compression. For economy o~ transmission, ~CT coefficients may be "thresholded" prior to tr~ns~ission, by dropping the less ~ignificant ~CT coefficients. This invo}ves use of a suitAble crîterion for determining significance.
The following are cited ~s background in the fie~d o~ digital i~age and video technology:
U.S. Paten~ No. ~,001,559~ "Transform Coding Using Coef~icient Prediction Te~hni~ues", issued March lg, 1~ O
C. A. Gonzale~ et ~1. discloses an encoding-decoding 25 te~ni~ue inVotying subdividing an image into block~, tran~form coding of t~e im~ge bloc~s to produce ~ a~d AC
coefficients, and recovery of t~e image from the coefficients of mage ~locks.
U.S. Patent No. 5,14~,~80, IlImage Data Proce~sing Appara~us~', issued August 25, 1~92 to K. ~a3cagami et al.
di~closes ima~e data processing from ~e~or~es for storing transform ~oefficient~ and th~e~hold values. The ratios of transform coefficients to thresho}d values are quantized, a D~ component i~ subtracted, zero values ~e dete~ted and 211~88 0 29~30 -counted, and Huffman coding is applied.
~ he paper ~y D. L. ~cLaren e~ al., "Removal of Subjective ~edundancy from DCT-coded Images", IEE
Proceeding~-I, Vol. 138, pp. 345-349 (19~1) discloses thresholding of ima~e ~ans~orm coefficients, with each threshold chosen as large a~ posslb~e sub3e~t to the requirement thak no differen~e ~e percei~ed ~e~ween an ori~in~l ~nd a reco~structed image~
The papex by C.-T. Chen, ~TransfOrm Coding of Digital Images Using Variable Block Size ~CT with A~apti~e Threshol~lng and Quantiz~tion", Proceedings of the SPIE, Vo~. 134g, Applications of ~igital Image Pro~essing XIII, pp. 43-54 (1990~ disclo~e~ thresholding and g~ntization applied to blocks of an image segmentation. The segmentat~on is optimiz~d based on st~ti~tical, standard-deviation considerations.
~ he paper by C.-T. Chen et ~1., "A K-~h Order Adaptive rransform Coding ~lgorithm for Image Data Comp~e~sion", Proceedings of the SPIE, Vol~ 1153, Applications o~ Dig~tal Image Processing XII, pp. 7-18 (1989) di~clos~s transform coding ~f an imager with compre sion parameters adapted to image contents.
The paper ~y Y.-Q. 2hang, t'A Co~bined-Transform Coding (CTC) Sche~e for lmage Da~a Compression", IEEE
Transactions of Consumer Electronics, Vol. 37, pp. 972-~77 ~1991) discloses image data compression by a so-called com~ined-transfor~ coding technique in which image informa~ion is classi~ied into a most significant portion and a les8 signif t cant portion. Data compressi~n is applied 30 to the most sign~ficant por~ion as a whole. The less significant portion is se~mented, and data compression i5 applied to the indivldual ~egments~
The paper by Y. Du, 'IScene A~pti~e Vector Quantization", Third International Conference an ~mage Processing and its Applications, ~E~ ~onf. Publ. No. 307, pp. 571-~7~ ~19~ discloses image en~ding invoIving ~1 1 8880 ~9530 -discrete co~ine transform, classi~ication, edge detection, spec~ral weighting, thresholding, spe~trum pa~tition, ~ector quantization, address coding, and sc~ler quanti~ation.
The paper by A. Baskurt e~ al., "~ncodin~ the 5 Location of Spec~ral ~oe~ficients Using Quadtrees in Transform Image Compres~ion", ~EEE, ICASSP~89: 1~8~
International Conference on A¢oustics, Speech and Si~nal Processing, Vol. 3, pp. 184~-~845 ~1~89) dis~loses a transform image coding ~ethod applied to bloc~s, with ~Q thresholding based on statisti~al prope~ties of the transform coef~icien~s.
The pape~ by Y. B. Yu, "Variable Bloçk Size and Posi~ion ~ransform Codi~g", Proceedings, EUSIPC0-88, ~o~th ~uropean Signal Processing Conference, Vol. 2, pp. 1~1-1913 (ls88) discloses transfor~ ~oding of im~ge data with optlmized deter~ination of block boundaries by a dynamic programming technigue.
The paper by D. K. ~itrakos et al., "Digital Image Co~pression with Fidelity Cons~raints", 1983 International Electrical, Ele¢tronics Conference Proceeding~, IEEE, Vol. 2, pp. ~14-617 ~1983) discloses identification and explo~ation of structure in d~ta represen~ing an image for optimized choi~e of encodin~ par~meters by dynamic progxamming.
A still or video image ~rame m~y have 256x256 signal value~ or pixel~, f~r example. For encoding, the frame may ~e ~ubdi~ided ln~o 32x32 ~1blocks", e~h having 8x8 pixels. Each one of ~he blocks is di~itally ençoded i~to ~CT coefficients. Thre~holding is applied to the DCT
coe~ficien~, and the thresholded coefficients are transmitted in accordan¢e wi~h a standardized seguence of the blooks, corresponding to a zigzag pattern on the image.
~ith such standardi2ation, JPEG and MPEG decQder~ h~ve become popular, o that, desirab~y, any i~proved technique should remain compa~ible with these decoders.

,~ . . , 2t 1~88~ 29530 Summary of the Invention ~ nile maintain~ng JPEG and I~PEG compatibility, the DC~ c:oefficients c:)f an image or video frame ~re seleGted for rate-di~;tortion optimality. Thus, distortion is minimized ~or a speci~ied :slaximum target coding blt rate or, equivalently, coding bit rate is mini~nized for ~ specified maximu~ allowable distortion le~el. In a preferred selaction process, a dynamic program~ing method is used.

~rie~ Descriptio~ ~f the Drawin~
Fig. 1 is a bloc~ diagram of signal processing involving quan~ization and thre~holding in accordance with a - prefexxed embodi~en~ o~ the invention.
Fig. 2 is a graphi~ representation of ~CT
coefficients for a typical 8x8 image bl~ck of JPEG/MPEG, linearly ordexed according to the 2igzag sc~n order.
Fig. 3 is a block diagram of dynamic programming re~u~sion in accordance with a preferred embodiment of the invention, applied to an 8x8 image block.
Fig. 4 is a schematic which illustra~es dropping or pruning of DCT ~oefficients in accordance with a preferxed embodiment of the inven~ion.
Fig. 5 is ~ graphi~ representation of distortion as a function of bit rate ~or t~o diferent JPEG scales, for an ~ge as in Figs. 7a and 7b.
Fig. 6 is a graphic representation of distortion as a funct~on o~ bit rate for th~ee different QP levels in MPEG, for an intraframe coded test frame.
Fig~ 7a is an imag~ of a house, transmitted without ~hre~holding.
Fi~. 7b is an image of the same house, ~nsmi~ted with ~hre~holding in accordance with a preferred embodiment of the invention.
Fig. 8 is a graphic xepresentation of distortion as a function of bit rate for thre~holding in ac~ordance with a prefe~red e~bodiment of the inven~ion, as compared with thresholdin~ by retaining the largest ~efficients in a block.

Detailed De~criPtlon o~ Pre~erred Embodiments In preferred data compression, that subset o~ the DCT coefficients i~ retained which i~ most favorable in a rate-distortion (R-P) sense. At a marginal sacrifice of coded quali~y, a significant reduction in ~oding hit rate may then be realized, as fe~er coef~icient~ have to be transmitted. This appliec especially when deciding on the last non-zero coefficient which, in JPEG ~nd MPEG, is followed by ~n inexpensive end~of-bloc:k code.
In a prefer~ed method, a fast recursive dynam~
progr~ing (DP) technique ~ used. Star~ing from the highe~t-quality point a~ter quantization at a ~ixed scale in the case of JPE~, or at a f ixed QP-le~el in the case of MPEG, the entire thresholding R-~ curve can be swept over a continuous xang~ o~ tar~e~ bit rates or, e~uivalently, of target coding quali~ies by dropping insignificant coeff~c~en~s ~n ~n image or vide~ frame~ Thus, all points on the convex hull o~ the thresholding R-D curve can be found. The ~ethod takes advantage of the monotoni~ n~ture o~ bit ~ate versus zero-run-length coun~ preceding ~ non-zero coefficient inherent i" the Hufman tables of ~PEG
and MP~G.
The following no~ation is helpful ~or further description, and also appears in Fig. 1:
X denote~ a sign~l of intexest, corresponding, e~g., to an 8x8 ~loc~ of an image or video ~rame;
X denotes a ~CT-quantized version of X
co~responding ~o a fixed scale or l'anchorll level representing ~he maxi~um guality operating point;
X~ denotes a desired thresholded version of X ;
~ (x,y) denotes an ~pp~op~iate distortion metric between signal~ x and y, such as mean-squared er~or ~MSE) distortion, for example;

21188~0 29530 R(x) denotes the mini~um bit rate required for transmission of a ~ignal x; and Rbudq~t denotes the maximum target bi~ rate.
Sought is X~, having a quantized version which is the same as the quanti~ed ~ersion of X, namely X , and such that D~X,X-) is minimized subject to R~X-) 5 RbUd~et Thi~ ~onstr~ined optimiz~ion problem c~n be ~on~erted into a corresponding unconstrained problem with a "Lagrange m~ltiplier", A. The task then becomes minimization of the f~n~tion J(A) defined by ~(A) = D~X,X~) + A ~(X-) Ad~antageously, the optimal coeffic~ent search ~r the image can be pe~fo~med independently for each 8x8 image lS blQck for the fixed q~ality "slope" A, which trades off distortion fo~ ra~e. This is because, at R-~ optim~lity, all block must operate at ~he ~e slope point A on their R-D curves; ~ee the paper by Y. Shoham et al., "Efficient bi~ allocation for an arbitr~ry set of ~uantizers", IEEE
Tr~n~a~tions on Acou~., Speech, Signal Proc., Vol. 36 (1988), pp. ~445-1453.
~ n the present example of an ~x8 blo~k, opti-mization i5 over the coxresponding set of 64 c~e~f icients .
Thus, if T - {0, 1, 2, ..., 63}
is the ~et of DC~ coefficient indices of ~he 8x8 ~lo~k, o~dered in the ~t~nd~d zigzag s~an order, if S ~ T denotec any feasible ordered subset o~ T, and if D(S) an~ R~S) denote the distortion and the bi~ rate, respectively, ~ssociated with retainin~ the coeffici~nts in S t the task of finding Dmin = mins ~ T D(Sl subjec~ to RtS) ~ Rbudget is ~ompli~hed upon defining J(A) -- D(S) + A~R~S) repre5enting the Lagrangian ~ost of S associated with the 3~ quality fact~r A, and f~nd~ng Jmin(1) = minS J~A) = minS(D(S) + A~R(S) 211~8~0 ~ 29530 The desired, initially unknown optimal value A~
for ~ depend~ on the particular target budget or quali~y const~aint. This value c~n be obtained readily ~y a convex search using the bisection method:
Jmin ( A ) = maxl ~ o ( Jmin ( ~ ~ ~ A ~budg~3t ) For ~n eXposition of the bi~ection method, see, e.g., the paper by ~. Ramch~ndran et al., slBest Wavelet Packe~ Bases in a R~te-Di~ortion Sense", IEEE Trans~tions on Im~ge Processing, Vol. 2, pp. 16~-173 (19931.
Since, as described above, the optimal con~ex-hull ~olu~ion can be ~ound by finding the minimum-L~grangi~n~cost oper~ting poin~ , i . e ., on~ whic:h ~ini~izes -- D~lock + A Rl lock ~or ea~h ~lock of the sequenc:e independently, consideration 15 of a sin~le bt GCk iS suf~icient. In a preferred embodiment, the zig~ag scan ~hat is part of the JPEG and MPEG st~ndards is used to order the 2-~ DCT coe~icients. As an initia~ization, the ~J~ k~s are determined which are associated with the increment~l ~agrangian cost of going from coef~ici~nt j ~ixe~tly to coeffi~ient k, i.e., of dropping ~11 the intermedl~ry coefficients, for all non-zero val~ed (i,k)-coeff~cient pairs with j c k. ~Jj k = -Ek ~
A'Rj k represents the ~Inet gain" of including ~ k condi~ioned on the previous non-thresholded coefficient ~eing C^j. Ek represen~s ~he "goodness" or ~uality measure as calculated by the de~rease in sguared error caused by ret~inin~ C^k, and is glven ~y C~ - (Ck - C ~2, where Ck and C k ~re the unquanti7.e~ and quantized coefficient v~lue~, respecti~ely, and Rj k is the conditional bit rate in coding c~efficient 30 C k given th~t the previous non-2ero coefficien~ is C j, ~ e., R~ k i~ the conditional ~ost of ret~ining C ~. For illus~rat-on, see Fig. 2. C^O is the DC coefficien~.
The va~ues ~j k can be pre~tored from the ~PE~ and MPEG ~tandard Huffman coding tables; see, e.g., the a~ove-3S refe~enced JPE~ ~nd MPEG references. Only the run lengths need to be stored, not the actua~ Hu~fman coded bit ~ream, , . ~., 211~88Q 2g530 -~o that little ~emory is required. The op~imal opera~ing point, for a fixed value o~ A, can ~e ~ound in a recursive f~shion by finding Pirs~ the minimum ~agrangian cost J*~k) an~ ~hen the optimal pred2ce~0r coefficient, 3 "prede~esscr~k)"~ associated wi~h choosing coefficient C k as ~he last non-zero ~oefficient for a~ 2, ..., 63.
Then, start~ng from tha~ coeffi~ient k* which is th~ least costly ~o re~in ~ the la~ non-zero coeffi~ient, i.e., minimum J~(.), the opti~al se~ ~n be "backtracked" from the optimal predecessor chai~ cal¢ul~ed for all pxedecessors of k . See Fig. 3 for illustr~tion.
A more detailed step-~y-s~ep ~escrip~ion of the metho~ follows. The re~ur~ion begins with coef~icient 0.
The cost o~ dropping a~l AC coefficlents is stored a~ J*(O).
Then, ~he mini~um-cost ~lpat~l that ends in coefficien~ 1 is selected. There is no choice in this, as there is only one pa~h that ends in coefficient 1, namely dropping all coefficien~s from 2 to 63. This cos~ is sa~ed as J*~1), and th~ optimal predecessor to 1 is 0. Pro~eeding to coe~icient 2, the mos~ efficient recur~ive way of de~ermining the hest path that ends in 2 i~ to find the op~imal predecessor to 2, i.e. eithe~ 0 or 1. Since the optim~l ~osts asso~i~ted wi~h ending at o or 1 are known ~om J~(0) and J~(}), respectively, the task of finding the le~st cost~y path ending in 2 amounts to find~ng the minimum o~ J*() + ~o 2 twhere ~J0 ~ is the incremental cost of going from 0 to 2~, ~nd ~*~ J1 2. The smaller of ~hese t~o costs is saved as J~(2~, and the opti~al predecessor o~

2, i.e., the one among 0 or 1 which resulted in the smaller total cost leadlng to 2, is saved as predececsor~2).
Proce~ding ~imilarly to coefficient 3, the bes~ p~h ending in 3 must have a direct predecessor whi~h is eithex 0, 1 or 2. As the best cocts a~sociated with ending at all predece~so~ ~re known ~rom previous iterations and are stored ~ J~predecessor), and as the incremental cost of going ~rom ea~h predecessor ~o "3" is known from the 2I 18880 ~9530 Precompu~ed ~JpredeceBor~ 3 for all predecessors 0, 1 and 2, ~he be~t p~h e~ding in 3 i~ computed as the least o~ the total cos~s J~tprede~eSsOr) + ~Jpredece~or, 3 for all predeces~ors 0, 1 and ~. The lea~t cos~ is sa~ed a~ J~(3), the optimal predeces~or is saved as predeces~or(3), and the re~ursion continue~ to coefficient 4 ~nd so on un~il the last coefficien~ 63 is p~oces~ed. At this point, the optimal l~st non-zero coefficient k~ is the one with the sma~lest ~(k~ for k - 0, ~, ..., 63. Backtracking from now yields the optimal prede~e~or ~hain sequence st~r~ing fro~ predecessortk~) and going back to 0, ~ whi~h point the entire optimal set o~ coefficients to be xetained for the bloc~ is known, for the given A.
General~y, in finding the optimal prede~essor at a particular iteration k as described ~bove, all coefficien~s : j c k have to be considered as ¢~ndid~tes. However, for the particular ~se of monotoni~ity o~ R~, k in the zero-run~
length count (k-j-1~, which is the ca~e fo~ the default coding tables of JPEG and MPEG, ~ "fast pruning'~ me~hod ~an be u~ed to speed up the sea~ch. Thi~ results in d ~u~st~ntial decrease in somputational complexi~y, ~nd leadc to a fast optimal method. See Fig. 4 for illustration.
Qptimal dynamic programming is performed independently on e~ch one of the ~locks. The composite R-D
~5 point for the picked l is then ob~ined ~ the sum of the op~i~ally o~tained R-n p~ints for each block for th~t value of A. Finally, the optimal slope A* whi~h solve~ the de~ired budget or quality cons~r~int is found using a fast convex sear~h.
A preferred methsd is illustrated ~urther in terms of method steps listed below, in Phase I for a fixed operating slope A for a sinqle ~ypical 8x8 image blo~k.
Phase I is applied independently and pre~ex~ly in parallel to each signal blo~k t~ determine the optimal coefflcient sequence to be retained for th~t block. Included in Pha~e I, through Step 6, is a de~ermination of the optimal ' ~9530 -last coefficient to b~ re~ained as a non-zero coefficient.
T~i~ coeffi~ient has the index k~. Then, starting at Step 7, ~he entire optimal set of coefficien~s is produced as {optset}. In Ph~se II, the optimal operating slope A~
for the co~posite problem lg obtained.
A one-time fixed ~ost o~ gathering the require~
stati tics i~ incurred as ~escribed ~ove with reference ~o Fig. 2. ~his invol~es g~thering, for ~a~h ~T coefficien~
C^k, its thresholding distor~ion Ek and its conditional non-thresholdi~g coding cost R~ k ~on~i~ioned vn every pre~edingnon-zero coe~icient i c k.
In the method de~cri~ed ky steps below, E denotes ~he total unquanti~ed AC energy in the signal block, l.e., ~ - ~ 3 Ck2. Ek deno~es the thresholding distortion 1~ ass~ciated with coef~icient k, R~ k denotes the increment~l bit rate c~s~ of coding k af~ex }, ~Jj k denotes the in~remental ~ag~angian cos~ of including k af~er j, Jk~ is the ~inimum Lagrangian ~ost associated with having k as ~he }ast non-z:ero coefficien~, ~nd Sk is the set of all ca~.didate opt~ l predece~or coeffi~ients to k. See Fig. 3 ~or ~urther illus~rati~n.

Phasç I: Findin~ ~he Optimal Coe~f icient Set for a A-value.
Findir~q the Qp~im~l. L~s~ Coef~icient.
Step 1. Compute ~Ji ~ ~ -Ej ~ A-RL j for all non-ze~o co~fficient pairs ~, j wi~h j ~ i SteP 2, (~nitializa~ion) k~ ~ 0; k ~ O;
SO~ ~ {O~; ~O ~ E; prede~es~or(O) I nil 5teP 3 . }C ~ k~ f k - 63, go ~o S~ep 7;
otherwise c~ontinue with Step 4 Step ~. If E~ , se~ 5~ ~ Sk_l and go to Step 3;
othere,Jise continue wi~h Step 5 ~ÇP S~ Jk minieS(k-l) ¢~ + I`Ji,k);
if J ~ ~ J * ~chen k~k Step 6. S~c ~ {k}v{i¦i~Sk_l ~nd Ji CJk ~;
predece~sor(k) ~ min 1i~s~ ) (Ji + hJi,k) i - r 211~88~ 2~53 go to Step 3 Backtracking to Flnd the Optim~l Coe~ficient set~
~te~ 7. Initiali2e ~he set of ~ptim~l coe~ficients as optset~k~; set k~k*
Step 8 ~ If predecessor(~) ~ nil, then go $o S~ep 10; ~therwise, con~inUe with Step ~
Step 9. Ge~ the optimal predecessor to k and include its membership in the set ~opt~et~:
{optset}~{optse~ predeces~or(k)}; go to step ~tep 1~. DONE. The optimal solution of coefficients to ~e retained for the given A ~s ~he set ~op~set~.
An impo~t~nt operation tha~ ensurcs a fast : algorithm is the pruning act~on in Step 6. This step eliminates from candidacy fo~ predecessor to the next non-zero coe~ficient, all those pri~r coef~icientc whose ~owest cost of ~eta~ning as ~he last non~zero coefficient exceeds th~t of the current iteration's coefficien~. ~hus, if the current coefficient produces the lowest Cost so far, it is the only ~andi~ate ~or predece~so~ t~ the next non~zero coeffi~ient. This is due to the monotonic natu~e of the bit-rata versus zero-run-length Huffman ta~le~ for JP~G ~n~
~PE~, wher~ the co~t of coding a non-zero coeff~cient is monotonically non-decreasing in the length of ~he zero-~un preced}ng tha~ coef~icient.

Phase II : Finding the Optimal Operating Slope.
The optim~l value A~ which sol~es ~ desired budget ~ons~raint Rb~dget is ound using a convex-~earch bisection algorithm. Starting from a known ini~ial inte~al encompass~ng the de~ired operating slope, ~he sear~h intervals are made ~uccessi~ely smaller, exploiting the convex relationship o~ both glo~al rate and global dis~ortion with respect to the oper~ting slope A, until ~onvergence is achieved. ~f A~(l) and A~ti) are the lower and upper bounds to A* at itera~ion i, then the convexity -pr~pe~y is exploited in tightening either the upper bound or the lower bound at the ~i+~)th iteration to A(~ D/~R)(i~¦, where ~his ratio of the difference in distortion and rate ~so~iated with the slopes ~1(1) and Au~) of the i~th iteration provides a closer ~pproximation to A* than the one available at the i-th iteration.
In ~he ex~mplary method, Phase I~ may be implemented as an "outer loop't for ~lo~k-by-block processing. However, as mentioned abo~e, para~lel processing i5 p~e~erred. In either ~ase, Phase I is invoked for ea~h value of 1 put forth ~y Phas~ II, for convergin~
Yalues of A for a blo~. The value of A* ~ol~ves the b~dget con~rained problem to within a con~ex hull ~pproximation.
The method of Ph~e I ~an also be used independently, witho~t Phase I~, e.g., for "sweeping"
through value~ of 1. For example, Phase I can be used to quantify the benefits of adap~ive thresholding applied to ~he JPEG and MPE~ ~oding environments, a~ R-D curves are obtained by sweeping the Lagrange multiplier 1 through all po~tive ~alues ~or typical guan~ization scale~ of interest.
Fig. 5 ~hows the R-D cur~es for a typical image using JPEG for pre~thresholding quantiza~ion ~cales of 1.~
for cur~e ~a), and of 0.7 for c~r~e (b). Point X on curve ~a) is the unthre~holded "reference" obt~ined f~r a cale of 2~ 1Ø Fir~, with the bit rate fixed at the referen~e~s hit rate of 0.615 bits per pixel (bpp): ~or the finer scale of 0.7, the non-thresholded bi~ rate, corresponding to point Z, is grea~er than that of X. Wi~h opti~al ~hresholding, however, the ~it budget con~r~int imposed by cur~e (a) is satisfied, and an adaptive thresholding ga~n in terms of increa~ed SNR f~r the ~ame bit rate is re~lized. Thus, point Y has ~ O.7 dB gain at ~he s~me b~t rate over X.
Alternati~ely, ~it~ the PSN~ fixed according to that of X
(37.15 dB): point W has a compression advantage at t~e same ~5 PSN~ of approximately 15~.
Fig. 6 also ~hows optim~l thresholding ~-D ~urves, - lZ -.
.' .

`

211~0 for an intxa-f~am~ coded frame o~ the "mit" test sequence and using an MPEG in~r~-frame code ~ook. Curve ~a) ~orr_sponds to a QP le~el of 4~, curve ~b) to a ~iner quantizer ~P level of 40, and cur~e (~) to a still finer QP
le~el o~ 32. If the reference is fixe~ at poin~ X on curve (a) correspondin~ ~o a QP of 48, point Y ~an ~e reached hy l'ba~king off'l to the finer P-4~ ~nd thresholding optimally to poin~ Y at the s~me bit rate as X. For this example, with reference X, the thresholding gain ~t point Y is ~0 approxima~ely 0.52 dR ~t a ~ a~e o~ 0.377 bpp.
Al~erna~lvely, at the ~me MSE of 112.5, a 12~ reduction in ~it rate is ~ealized at point W of ~u~ve (b). If point ~ on curve (c) with QP-3~ were ~ho~en, ~ lesser coding qain of approximately 0.26 d~ would be realized over point X.
~n experiments, it w~s found that "ba¢king o~f" ~o a finer quantizatîon sc~le and thresho1ding optimally until th~ same reference bi~ r~t~ ~r PSNR is achieved a~ an unthreshol~ed coarser quantized ve~sion resulted in an appreci~le codin~ gain. HGwever, as illus~ra~ed by Fi~. 6, 20 there wa~ an optimal hack-off poin~ beyond which the performance star~ed ~o degrade. ThreshQlding after starting with a finer ~uan~iza~ion s~ale is inadvisa~le beyond point, as the gain of representing the non-thresholded coefficients with less disto~tion is no longer out~eighed by dropping entire coef~icient~, since for fine quan~iz~tion scales, the~e is not much distortion to begin with.
Coding result~ o~tained ~rom performinq optimal threshold~ng on ~ypi~al images and video sequence frames used in ~he image processing community revealed ~ coding 30 ~ain o~ about 0.5-~ dB, or abo~t l~-15~ bit rate compre~ion improvemen~ while retai~in~ complete decoder ~offlp~tibility.
Subiecti~ely, opti~al thre3holding appears a~ most benef ici~l in the c~se of low to medium bit rate coding, as illustra~ed ~y Figs. 7a and 7b. Fig. 7a shows as a non-3S thre~holded reference a standard "House~ image coded wi~h~PEG using a quantiz~tion scale of 3Ø ~he thresholded - ~3 -version using a scale of 2.0 is shown in Fig. 7b. The coding ~ain is O.S dB, an~ the subjec~ive ~u~lity is improved. An intuitive reason lies in that, for low bit rate applic~tions, it i~ preferable to represen~ ~he low frequency coefficien~s with maxi~um idelity while dropping the expensi~e high frequen~y coef~icients. This gi~es a ~oother but less no}sy image, which is best possible a~ low ~it rate~. T~us, adapti~e thresholding c~n take ~he place sf noise shaping or low pass fil~ering withou~ ~ny external pro~essing and without affe~ting th~ deccder.
~ he perfor~ance of ~he optimal ~lgorith~ was compared with a simple-minded heuristic metho~ tha~ retains ~or each bloc~ the k ~T coeffi~ients which ~re largest in magni~ude. Fiq. ~ shows that consider~ble gain can ~e }5 realized by using ~n optimal ~etho~t namely more than S dB
for bit rates below Q.6 ~pp. At 0.5~ bpp, the gain is approximately 5.6 db.
While ~he ab~ve is a description of the invention in preferred e~odiments, ~arious modi~ications, altexnatives and equivalents may be employed, only some of wh~ch have ~een described above. For ex~mple, preferred p~ccess~ng can be applied to image or video fra~es which may be par~itioned in other ways ~han into 8x8 blocks. Also, ins~e~d of simple mean-squared error, ano~her suitable distor~ion me~ric can ~e used, e.g., ~o take subjective measures into account. Thus, for example, activity-weighted me~n-squared error can be used. Yet other variations will be app~ren~ to those skilled in the art.

Claims (6)

1. A method for signal compression in image transmission, comprising:
transform coding to obtain an ordered set of transform coefficients for at least a portion of an image;
repeatedly, for different values of a parameter, selecting an ordered subset of the set of transform coefficients, selection of the subset being for minimized thresholding distortion for the value of the parameter, and repetition being terminated upon at least approximate reaching of rate-distortion optimality; and transmitting the selected transform coefficients over a communications channel.

2. The method of claim 1, wherein different values of the parameter are chosen in accordance with a bisection method.

3. The method of claim 1, wherein finding the subset of transform coefficients comprises (i) finding a last transform coefficient of the subset, and (ii) backtracking to find all transform coefficients preceding the last coefficient.

4. A system for signal compression in image transmission, comprising:
coding means for transform coding to obtain an ordered set of transform coefficients for at least a portion of an image;
optimization means for repeatedly, for different values of a parameter, selecting an ordered subset of the set of transform coefficients, selection of the subset being for minimized thresholding distortion for the value of the parameter, and repetition being terminated upon at least approximate reaching of rate-distortion optimality; and transmission means for transmitting the selected transform coefficients over a communications channel.

5. The system of claim 4, comprising means for choosing different values of the parameter in accordance with a bisection method.

6. The system of claim 4, comprising first means for finding a last transform coefficient of the subset, and second means for backtracking to find all transform coefficients preceding the last coefficient.