CA2096878C - Efficient frequency scalable video encoding with coefficient selection - Google Patents
Efficient frequency scalable video encoding with coefficient selectionInfo
- Publication number
- CA2096878C CA2096878C CA002096878A CA2096878A CA2096878C CA 2096878 C CA2096878 C CA 2096878C CA 002096878 A CA002096878 A CA 002096878A CA 2096878 A CA2096878 A CA 2096878A CA 2096878 C CA2096878 C CA 2096878C
- Authority
- CA
- Canada
- Prior art keywords
- coefficients
- image resolution
- resolution scale
- sequence
- sets
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/37—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability with arrangements for assigning different transmission priorities to video input data or to video coded data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
Abstract
An improved technique for efficient frequency scaling wherein the number of coefficients to be included in each sub-block is selectable, and a code indicating the number of coefficients within each layer is inserted in the bistream at the beginning of each encoded video sequence. This technique allows the original runs of zero coefficients in the highest resolution layer to remain intact by forming a sub-block for each scale from a selected number of coefficients along a continuous scan. These sub-blocks may be decoded in a standard fashion, with an inverse discrete cosine transform applied to square sub-blocks obtained by the appropriate zero padding of and/or discarding of excess coefficients from each of the scales. This invention further improves coding efficiency by allowing an implicit end of block signal to separate blocks, making it unnecessary transmit an explicit end of block signal in most cases.
Description
E~ICII ~ REQUE:NCY ~ALABLE ~ 8 7 8 VIDEO ENCODING VVlTH COEFFICIENT SELECTION
T~ Fi~ld The invention relates to the enco~ine of video signals, and rnore particularly, 5 enco~in~ video signals in an efflcient manncr which pe~mits images having a wide range of resol~ltiQns to be derived f~om an encoded video signal.
Back2 . ound Of The Invention Many arp!ir~q~ )n~ such as, multi-point video t~,leco.~.~..cing, the windowing of displays for ~ n~ video co~ ;r?~ion~ on ~-.cl,ronous transfer mode ("Al~I") 10 n~ Lwo.L~, and ~.~adc~ high~efinitiQn ~elevision can ~enefit if video at v. rious r~sohltion~
can be derived from an encoded bi~h'~n. The simplest method of achi~ving this is the eim~ q~ ni~lue in which multiple i~ n~ coded replicas of a video ~u~ e (each scaled to a difÇ~ ,nl res~l~lti~n) are ~imllltq.~ouely ~ l In this applvach, caling is ~.Çu~ ed prior to dle co:l.yl~;s~;on/decomples ,ion o~ each replica, and each resolllti-~n scale 15 is then illde~ p~ encoded and ~ceign~ a portis)n of the total available h ~ " '~
bandwi.l~ll. As a result of his independent enrcYlin~ and tr~n~ e;on~ the ~ it technique requires a wide bandwidth.
A mo~ efficient alte~ativo to siml-leP~tin~ is scalable video enc~lin~. ~l~ucn~iy scaling is a low-~ Yity method of scalabie video encoding in which 20 a single video signal is ~ to mul~ple 1~C''i~3 which dccode images of varying resr~l ~ir,ne from that signal (le~ .~ e upon the particular signal deco~ing scherne employe~
by each receiver. A specific en~ ing method to which L~u~,ney scaling can be easily appliod uses d;.s~etc cûsin~ r..~ (~ blwks of oliginal or ~li.;~ion error.pixels to de~ive blocks of l~u~ o~ coe~f1~ien~c Vanous subsets of these L~uell.;y domain 25 e~ rrjf ~ .t~ CaJI be uscd ~0 generate ~r~le,l. resolution scales for a given image. . Such enCo~3in~ may be ~ '-~ - d using a slightly ~yl;fir~ version OI ~e ellCI:)de.~ close~l in the TntAmqtifnq1 Standards O,~ nl~ ~{~ Draft 11172-2, "Coding ûf Moving Pictures and ~ccr - ~ Audio ~c~r Digital Storage Media at up to about 1.5 Mbit/s," Nov.
1991, and in "Yideo Coding Using the MPE(;-1 Co~ ion Stand~rd," A. P~ri, P~e~
30 of the Socie~,r for 1~ Display 92, May 1992, Boston.
~. . .
Thc tl~i ,fol.lJ coding p~ion of these enr21~,9~6co~ 7u~s thc DCI as a ~unction of the energy in ~he bloc~ it receives from an o~iginal image, oq from the dif~erence between the onginal . nd its p~li.iLion from a previously decoded image. ~ccllmii~g an 8x8 pi~e element ("pixel") basic bloc~ size, each 8x8 pixel bloclc of a picture is ~ 1 in~o an 8x8 5 bloc3~ of DCI- coPffi~ent~ These c~,ffirientc re then ~ and scanned so that they may be converted into a one ~ n..nc~o~ s~uenc~. An entropy coder within the video encoder ccJmp~sses the one ~ .r-~- onAl sc~ ce to a series of number pairs called run/levels. Within a runllevel the first number ~l~n~ the number of zero coeffiri~nt~
~etween the last two non-zero co~fficientc in the one ~ nc;on~ nce" and the second 10 number ne~se~ the c~ ~ value (i.e., c~ n~ n index) of the next non-zero coefficient in that se~., nne~ The number of zero c~ m~:entc is called the length of "mn of zeros". For , '~, the one flim.oncionql s~l"~ ~re of ~ 1 coeffirient~ 1 1 0 0 S O O
0 6 3 0 0 4 0 0 0 ... 0 ~esults in the run/level seqllpnre of (0, 1) (0, 1) (2, S) (3, 6) (Q 3) (2, 4) (EOB). EOB, or End-of-Block, in~ S that all of thc ~ Y~g ~ I coçffir~
15 are zeros. Prior to 1~ , shese pairs, inr~ ing dle (EOB), are encoded using ~ P~l~
length codes ("VLCs") col~ L;i~g ~f a ~ ce of binary di~ts o'olD;~ed from a code ta~lc which is optimi7ed to assign shorter codes to most frequent pairs occ~lrring in typical video images. This way, long runs of zeros (which are c~m~n in mo~ion c~
based image coding) can be effiri~ntly coded. When the ll - d signal is received, or 20 stored signal is ~ ie~, th~ VLCs are decoded and the run/level pai~s are l~o.~ ,d. The non-zero ~lu~l; ~I cc~ are then inve~ ~ , the zero cC~r~ çnt~ are ~cu.~ t~,d, and an inverse DCI ("IDCT") is pc.ro.ll~ on the entire block.
r~u~ ,y scaling allows fo~ effi~ient c-~co~ of video in a manner which f~i1it~tes the d~ alion of various res~ tiong at the decod~ ~y using different IDCT block 25 sizes. It can be shown that if a decoder applies the lD~ to an upper left con~er sub-block of every received blocl~ of c4~ rl ;. i.c ~ can generate lower rçsohltion images. For example, if a 2x2 su~block of an encoded 8x8 bloclc is ~ e~ image " ~ oi~ will be reduced by a ~actor of four bo~ veItically and 1~ ly (referred to as f-scale 2 ~uen-;~ scaling).
In this manner, tho r.~"~ sC~lo~ility can be ,~-men~ed on the decoder without any 30 .- ~1;rir ~;o ~ to the encoder. Ilu..~ , because of dle VLCs used ~or roding run/leYel pairs, dlere is no way to detect the end of a block (EOB~, oq the b~ginning of a new Uock, unless the entire block is de~ As a result, in order to recov~ a su~blocl~, the variable-length word decoder must operate as though a full ~esolution image were being recovered (this requires very fast, expensive circuu~:ry to ~ used to decode even low resolu~ion images).
-3- 20~6~78 Using unique maricer codes to separate su~blocks is not feasible because of Ihe high oYerhead cause~ by such codes.
An existing method ~or ob~ihlil.g a L~uen~iy scalable 1~ is to encode the cc~r~ from ~u~blocks separately in a layered S~ and m~lltipl-x them as slices S of various layers in the bi~ In this ~thod, the lowest .~ sQ~ ;v~ layer keeps the basic u;lu e of the bi~ ,h~, but fewer DCT coefflri~nte are inrh1d~ with each block (such as four COer~ tC for f-scale 2). The ~...~ ine cO~mripntc for each block in the slice are sent in slave slices which are s~,pa~at~i by inAe~.~ entifi~ e bit pattems called slave slice start codes. One problem with this ~ u,h is that every coded Uoclc in each 10 scale has an EOB ~c~;~ted with it to mark the last non-ze~o co~ffiriPnt intllldpd in that scale. As a result, instead of sending a single EOB for each code~ bloclc as in non-scalable coding, multiple EOBs are sent which d~.~s the t;rf;~;~"cr of the Ç~Je.i~ scalable coding.
Another problem with this soludon is the ~" erhead it i..ll~lu~xs ove~ and above15 the overhead caused by dle slave slice start codes. The a~ows in FIG. lA shows a typical zigzag scan of an 8x8 co~f~cienr bloc~ (l00), whe~o each of t~e nu~ ;,~nds to the loca~on of the s~-~qc: in the one ~ 1 sc~l"e~r~ derived by tho scsn operation.
FIG. lB shows the zig-zag S~pnnin~ patte~n applied to 2x2 (l0l), 4x4 (102), and 8x8 (103) coeffic;en~ sub-blocks used with f-scale 2, ~-scale 4, and f-scale 8 laye~s l~e~ ely. If 20 c~rr;~ 3 and 4 are ze~os, they will be coded as part of a single run of zer~s within the bi~ ,~ if the 8x8 block is scanne~ as a whole (FIG. lA). II~ ., in exisdng methods of ~i~u~,n~;y scalable bl~ck ~ ,e they must be coded s~ t~ly as they are each in a dil'fe-~nl c~ rr;~ sub-block. While dlere are only th~ee such b~eaks ~C~soc ~ d with an f-scale 2 sub-block (between the co~ rr~ ,t~ the solid-line ovals of FIG. lB), ~here are 25 seven ~~~cc' ~ ~ with an f-scale 4 su~bloek (between the c~f~ .n6 within the dotted-line ovals~. Each ~ealc in a run of zeros inl~uduc~l by the su~block divisions l~ads to increased coding o.,, ~ lhile this ove~ ,dd may be reduced by using separate VLC tables~in~l~l for e~ch f~u~ scale, these sepa~ate tables Ll~ ce ~ 1 co~lpl~ ily.
Summ~rY Ot The Invenffon The arv,e .. ,1;r~ cm~ are solved, in ~co~ ce with the princir1~s of the h~,ntiol~, by employing an ef~lcient ~i~u~,ncy scaling 1ec~ e whe~ein the number of coerrll :e~ to be it~cln~ in each su~block is se1ectn~.1e and a code i!.-lirE~ g ~e number 2asGs7g of cQ~ffi~n~ within each layer is inserted in the bi~ at the ~g;nni.le of each encoded video s~ This technique allows the oliginal runs of zero co~ffirtçnt~ in the bighest resc llltion layes to remain intact by fonning a su~block fo~ each scale from a selected number of co~rr~cie~ls along a cor~inl~ou~ scan. These su~blocks may be decode,d in a standard S fashion, with an IDCT applied to square sub-blocks obtained by thc a~p.;atc zeTo padding of and/or discardtng of excess cocr~ c from each of the scales. Sincc tbe o~iginal leveVrun st~ictics arc not changed by this method, the original VLC tables can be used to encode/decode all video layers derived from the sub-blocks. This invendon further Llplo7~s coding efr~ by ~llowing an implicit EOB to separate blocks, making it u~ ece~ y to 10 transmit an explicit EOB signal in most cases.
Briel~ D~ n O~ The Dra~nn~
In the drawing:
FIG. lA is an illu~ tion of a zigzag 3c- - ~ g of co~ rl~ wi~3in an 8x8 block;
PIG. lB is an illui,~a~ of a zig-zag ~ ~ Ig of co~r~~i wi~Ll 2x2, 4x4, and 8x8 sub-blocks;
FIG. 2A is an illustration of the co~ff1rient s~ ction t~ch~ of the invendon applied along a zig-zag scan to a p~rtion ~ a 4x4 su~block of an 8x8 block;
FIG. 2B is an illu.,ild~oll of the c~hrr~ Fel;Oi~ tP~hniqlle of the invention 20 applied along a zig-zag scan to a 4x4 su~block and a portion of an 8x8 block;FIG. 3 is a ~;rnp~ifi~d block diagram of an encoder for imrle...... i.~g îl~u.,.l~;y scaling with scan bascd cc~fficient ~e1P~ n in acoo~ * with a particular c.. ho~;.. i of the in._.~lio~, FIG. 4 is a ~ -lified block diagram of a decoder ~apable of decoding the 25 r~,5O~ layer .,~ .o~ g to the the lowest rl~u~ ,y scale from the ~ ~Lu output ~y the encoder of FIG. 3; - -FIG. 5 i., a ~iml~lified block diagram of a d~der capable of (~co.l;~g hvoresol ltic-n layers c~ ~7~ to two &~uency scales; and FIG. 6 is a simpli~led block diagram of arl inlrl; ...f,r.li.lion of the invention 30 whereîn each vides> layer is encoded via a d~Aicflted ~ ,., scanner, and va~iable-length coder.
-5~ 2~9~878 Det~iled D~cr~tion O~ The L7~.~tio.i FIG. 2A shows an example of ~he adaptive cQ~ffi~nt selection techrLique of the in~ uon applied along a zig-zag scan to a portion of a 4x4 sub-block of an 8x8 block.
The number of co~ri~ icnt~ to be included in each layer is sP~ , and the desired number S of coemeiçnt~ foq each layer is incllld~p~ at the be~ of each video s~ e In the particular example illustrated in FIG. 2A, the zigzag scan pattern is such that if the selected number of co~pffir;ente for the f-scale 4 layer is 10 or less (see coefficient group 201), the selected coPffici~ntc will cc,~ D~nd to a subset of the copffiri~p-nt~ usually cc...lAi~ed within a 4x4 su~block. For groupings of coefflriPnte larger than 10, some c~rllcif ~Ic from the f-scale 8 layer would be in~ dp~1 For ,Pyr 1~, aS shown in FIG 2B, a grouping of 14 eoP.ffie~ would inelude the corrr;~;e~ within groups 2û1 and 202, and cc~ rr;rie ~I number 11 within group 202 would usually be inclll~ed in an f-scalP, 8 layer Such "extra" cc~-rf;c~
may be ignored by decod~ ~ which decode only f-scale 4, as such decoders employ a 4x4 lDCI on the 4x4 upper left corner su~block. For L~ue~lc3~ scalable decod~ that may 15 decode images at higher resol~tion~, the extra c~,~ t~ are nccessal~ to decode layers c~ to the higher res- ln~ion~
The i,.~ s overhead as it does not require the inse.lion of EOBs in cvery coded su~block of every scale to partitiQn the coefflrien~ within each layer. Overhead is .ni~-i...;,~ by ~_ ,1c ~- ~,, the following coding gniARlines. If the 20 last co~ of a set of c~ rr~: nt~ in a given layer i~ non-zero, no EOB is necc~ as the decoder can always d~ end of a sub-block by cou ~ine the number of coR~ff1ri~ontc decoded (the number of co ~rr~ ;e .~. in each layer is inclllded at the beginning of each video s~.,e ~e). If the last ç~,rr..~ of a set Of c~.~r.. i "~ in a given layer is z~,ro, and falls within a run of zeros, the whole run inch)clin~ the first non-zero coefficient following it is 25 inch~ d in Ihat layer (provided it can be en~ocled/decoded by the lower layer VLC table).
The "extra" c<~;c - ~ irtrlu~ in the cu~ent ~solllt;on layer are lgnored by low reso!-ltion ~lccod~ , ~d us~ill ~ing cs,l~ onJillg higher 1~-L1I ~r layers by scalable or high reso~ nn d~.~. ~he inr~ on of the "extra" coeffls~;pnt~ simply reduces the run/level pairs in~ "ll~ in the ncxt higher resollltinn layers by one (co~ onil;,.~ to the one extra 30 run/level pair inrlu/lçcl in the lower ~ hltirn laye~). There is nv net increase in overhead.
Finally, if all co~rl;-;enl~ after a non-zero co~rlici~n~ 3~e zero, but the run/level event g~ n i ~cd cannot be encode~l/d~o~ed by the lower level VLC table, then an E~OB is used once.
-6- 2096~7~
FIG. 3 shows a simrlified block diagram of an enulder for ilnple~,ltillg f~.cncy scaling with c~fficient sele~tion The encoder is similar to those '~ 051~l in the Internalional Standards 01~ ;7 ~ Co~ ilt-~ Draft 11172-2, and in "Video Coding Using the MPEa-l C~ ,Djion Standard," but includes layer erld ~ccssor 301 and b;bll~
S layers buffcr 302 to c~ Ats splitting the c~rl; :~ ~ into layers. An image bloclc is input on line 303 and passes through dirr~ ,nc.l~g element 304 to ~ ro.ll- coder 30S. Within transform coder 305 a DCI is p~rv~ cd on the image signal, and the resulting c~ rl~ s are ~ i and scanned. The scanned co rr~ t~ are decodcd by local decoder 306 and passed to motion ectim~ n and co~ ~n~ circuitry 307. Motion e~ A~ion and ~ e 'ic~n circuitry 307 c~l~n~l~tes a p~ediction of the image bloclc input to line 303 based upon the bcst match of the block to blocks in a previously decodcd image rec~ived from local decoder 306. This estirnate is L~ enccd from the in~o7n;n~ image signal via Lfr~ ing element 304. Motion estim~tir~n and c~.n~ circuitry 3a7 also outputs modon v~cto~s which fonn a portion of the overhead I~ ti~ --O-' ' ' with the bloc~c being çm~
The output of Iransform cod~ 305 is also passed to entropy coder 3û8, where run/leYels ar~
CO~.~pu~13 and ~ .e-length coding is p~ru~ . All of thc a~ve dfs ~;b~l ~ ,c' g op~ations are well h~own in the art and ~ ~ in ~he two lefe.~ efeTred to earlie~.
Rwl/level data and a v~ ''- length encoded bits~eam are output by entropy code~ 308 to layer end ~, ~ce~so. 301. Layer end I,~css~r 301 counts the number of COerf;C f~ ' and routes the VLCs to co~ onrling layers acco~ g to the gnidelinp~s of the nli~n ~psçrih-p~l above. The be~nnitlg of each slice is mal~f d within the bi~ by a slice start code or a slave slice start u)de ((lepp~ upon the r~uellcy scale). Codes ~1lOC~t~ to the lowest reSolntion layer are di~ecdy sent to ~e n ~ ipl~Yf~ to be inserted in the ~ ~. Each code which belong to higher ~esoll~ti~- ~ layers ar~ stored bil~eall. layers buffer 302. At the l~.~ e of all but dle lowest r~ u~,ncr Laye,r slice, a slaYe slice start code is inserted in the bitstream by mlllti~ Yf~ ("MPX") 309. This code is ~ollowed by any coded data f~r this r.~q~ n4,~ scale cou~ f~ layers buffer 302. This orde~ing of f~eque scale slices.~or ea~h layer is l~:ir- t~ wldl the highest resohl~ion slave slice is i~~~ cd in the ~ h~,d~. The process is repeated ~or all s~ces in each picture. Multiplexer 3~9 outputs the encoded ~;12h~. and overhead codes to buffer 310. The overhead codes inclusle an in~iir~tion of the number of c~ffic;ent~ in each laye~, which is ~en~ ted by an i-ld~pC~d~"t pl~CeSS~JI (nc,t shown in FIG. 3) and inse~.ed by MPX 309 at the beginnin~ of each video s~l~,e-lce Buffer 310 ~ the ~it~ at the a~pl~liat~ rate to a decoder or a storagc device.
7 2~96878 PM. 4 is a cimrlifiP~3 block diagram of an ~rraneem~n~ for de~ n~ ~ lowe~t resol~lti~n laycr f~m the bi~LI~ ou~put by tbe encoder of FIG. 3. The ncod~d bi~ke~
is input to buffer 4Q1 via line 402. Buffer 402 passes the b;~h~ ~o sli~id~pntifi~ 403 which scans the b;~ e~ for slice start GOdeS and slave slice start codes. Slice identifier 403 S passes the portion of the bi~4~ between the slice-start-code and thc first slave-slice-start-code to ~Pm~ erh~ -le~ h decodcr ("DMPX/VL dcc~. ) 404.
As this decoder is only conce~lcd with thc lowest (and first) encoded layer of video this is the only portion of the bit~h~ which need b~ ~p~ DMPXIVL decoder 404 d~rm~ and decodes the received portion of the l~ib~. to obtain 'I~ DCI
10 cc~ rr;~ These c~ rr;~ wes;,ed by inverse q~ 405 and inverse scanner 406, which perfonn thc inverse furlcdon of the scanncr and ~lu~ of ~n.~f~m coder 305 (l;IG. 3). The cG~fr~ e then counted, and ~ cof rl;~-:r l~ of ~o arc inserted to form square bloclcs by cGe~;cienr count~ ,addw 4~7 (square blocks are requi~d for ye.r,.~lg an IDCI). Since the coem~;ent~ used witb sm~ller size IDCIs must be scaled, coçff1r;ent scaling is p~.. r.. ,~ on ~be block by c~ rf; ~ t scale~ 408 before an lD~X is p~- r.-- .~ ed by ID~ circuit 409. The vidco blocks ou~put by ~CI ci~uit 409 are passed ~hrough sIlmm;r~g element 410, whe~e a p ~;~i~l ~ge may be added to them, prior to output on line 411. As is shown in FIG. 4, the p,~ t~Jd image is ~L~ c~ by motion e~
pr~;c~I 412. Modon CQ-l~ ~r d ~.~ic~. 412 g ~t~5 this y._~l;ct~d image as a funstion of scaled motion vectors received from modon vector scaler 413, and the video blocks output on line 411. Scaling is needed because ~he modon vectors are derived for the highest res ~ tinn sca1e and their values must be adjusted for dle c~rent, lower resoI~ltion scale FIG. S shows a ~;mplifi~d block diagram of a decoder capable of decoflin~ two scalable resoIvtinn layers. The encoded l~ ,~ is input to buffer 501 via line 502. Buffer 501 passes thc bil:~h~ to slice~ 503 which scans ~e ~il;,hea~ or slice start codes and slave slice start codes. Upon d~t~ a slice start code, slice il1entifi~ 503 sends a control sign~ inG 504, which directs the output of c~ i cou~ adder 505 to sli-~ebuffel 506. IJp~n~l~ tb;~ a slave slice s~art code, slice i~l ~ 503 sends a control signal, via line 50~, which direc~s the ou~tput of co~fflri~n~ co~lL~t~ '~ - 505 to line 507. This insures that the low ~ laye~ coe~- will be passu~ to slice bu~fer 506 for stolage, so that thcy may bo ay~ Ldtely eO~ ~1 widl c~,fr~ om dle higher resolution slavcslice which is received a~ter the low reso~ ion layer.
The l~ ~ is passed from slice iden~ifi~ 503 to DMPX/YL decoder 508.
where it is dem~ and docoded to obtain ~ 7PA D~ coeffi~ nt~ These 2~96~7~
cC~r~ cnl~ ~ then y~s¢d by invcrse ~1~AfI~ d inYCrSC scanne~ S10. The conrfifr~ ted, and ~ d J c~n5~l coef~ ntc of Za'o aI~ inscrted to f~m squa~c Wock by co~ffi~nt eoul ~.t~ ~' SOS (square blocks arc required for ~ .f.. ing a~ ~. If this is a low ~esoll~tion laycr, c~effirient count~,.fpr lt' 505 passcs the c~r.'.. ~ to slicc buffer 506; o~ ,.w.se co~fficien~ pz ''- SOS outputs thc c~fr ~ on Line 507.
Slice buffer 506 allows the low resol.u~i~n c~rr~ to be held whilc thc high rssolntion coeffirient~ are decoded. The c~ rr,~ ;~f~l~ are passed from slice buffer 506 and/or line 507 to coeffir;en~ scaler Sl l and IDCT circuit 512 via l~;y~ n select switch 513. The position of resoln~ion selcot switch is controlled by an in(lfc~ p~CCSSol not shown in FIG. S.
Thc video blocks output by IDCI circuit 512 are passcd through s~mming element 514, where a ~ lietcd im~ge may be adfled to them, prio~ to output on line SlS. Motion CC~ J
predictor 516 4. ..~ t~ s ~is predicted image as a function of scaled motion vectors recdved from motion vector scaler 517, and the ~ideo blocks output on linc SlS.
In certain ~rFIir ~inn~, different VLC tables may haw to be employed for each S ~rr~.~n~ video layer. Since the possible mn lengths for the lowe~ res~ on layers a~e s}~r than that of the high re~~ layers, it is possible ~o use shorter VLC tables for ~e lowcr reso'~ltin~ layers. This may reduce the required cc,...~ t ~;ol~ esoul~;cs and memory required to fi~filir~ the video ~li'r,cY~ g Also, 1;1 certain aMl'~ a separate .lu~.ti~. may be utilized for each video scale ~ a11ow i~d~ 'Af ~ quality ~ 1jr- -- cc,~ .>~ g to ~it 20 rates of each layer.
FIG. 6 shows an encoder imrl - - which is similar in ope~ation to the encoder of FIG. 3, but whe~ein each layer in enco~ed via a ~e ~,c a~ ~ ~ 4 ~ i; . .. and vanable-leng~h coder. ~ thi9 partic~ )n~ image bloeks are input on line 601 toJirrw~.lci.lg element 602 and passed to DCI circuit 603 where a DCT is p~. r~ A All of 25 the c~rr;c;~n~ output by DCI circuit ~03 aro ~hen passed to th~ separate 605, and 606), ~ree separa~e scanners (61)7, 608, and 609), ~hree ~eparate run~level Colllput~
(610, 611, and 612), thsee separate layer end 1J1VeeSSI~13 (613, 614, and 615), and three separate va.-'l~ eoders (616, 617, and 618). The layer end p..,cesso.~ are ~ go that ~ layer erld ~ SS~ fr~lit~tine thc enro line of ~e lower laye~s 30 may pass ~( - to higher reso~ ;on layer end pl~cc,ss~l ~ as to dle actual number of co~rfi~ :e~ which need be in~ f d in each layer su~block. CG~ ~r.~ :~,. .t~ that actually belong in a highel~ resolvti-n layer, but are inc1vr1~d in a lower resolvtion layer so as not to break a run of zeros, are coded with the lower resolution laye~. The valiable-length coders for the l;al~i (617) and highest (618) rcsol~tion layers output l,i~ o buf~ers 619 and 9 20968~8 620, ~ .vly. Illis cnables thc output of thc ~ ble l~,ngth coders ~o be p~op~ly se~n~nc~ by MPX 621. MPX 621 ou~puts the s~ ed bibLi~ ~d l~v~head codes to buffer 622 for ~ c ";~ioil sr s~age. Motion c~ on and C~ ;On circuitry 623 c~lcula~P~ a prediction of the imagc bloclc input on line 624 'oased upon various control S pa~ , the signal from local decoder 625, and the image blocl~ sigr.al on line 601. This estim~te is added to the inromillE im~ge signal via ~L;fl~ncing element 602. Motion es~ima~ion and co~ ion circuitry 623 also outputs modon vectors which are i~CO~ t~
in the overhead h~l,.la~ion for the layer being encoded by MPX 621.
Since employing a single 4~ t;Di \~ and variable-length coder gives rise to the 10 F~ -l of b~.s,.alin~ run lengths as large as the en~rc bloc~ length in any re~olntion layer, using s_aller VLC ta'oles requires imrl~ .b .~ g the following n~A;~ coding gui~Plines~
For each re Iayer, in addition to ~he desired num'oer of c~ rl t~, the ~
allowed num'oer of c~ rl ;- o ~ ~, wnich inr~ the rwlllevel ~eeodi7~g e I~ahility, is s~ifi~d at the be~nning of the video s~u .ee If, as a result of a long run of zeros, tnis limi~ ~.
15 going to 'oe e Gc~cA, tne mn is cut and an EOB is sont. The ~ ning ~eros are ser~t with the next resol~ on layer. ~learly9 ;..~ of EOB's in su~block~9 and cutting zero runs will increase tlhe u.. hc 1 The increase '~e~os-n~s si~if ~ when r.~ el ~7OCo~ e c~pa~ility is limited. The actual overhead incroase is image ~ but c~ , ~ly smalle~ t'nan tne o.~ n~d ~l~lu~c~ by the previously hlown method of sending EOBs fo~ each coded 20 Uock of every scale. Ille sig~nal ou~tput by the en~ of FIG. 6 may ~ decoded by a decoder similar to that of FIG. S, but which employs a separate inverse ~ nli~ inverse scanner, and DMPX/VL decode~ f~r each individual res~ n layer.
Thc aboYe~ sc ;hcd ~ ~n provides a highly efficient ~u~,ncy scaling t~ u~ wherein thc numbe.r of cc~ t~ to be inc~ ~ in each su~block is sçle~t~
25 and a code i~ ;. ~;n,@ the number of co~rl';~ S~; wi~in each laye~ is inserted irl the l~it~
at the bG~ of e~ch encoded video ..~llten~-e, It will be 1l~ dersto~l that the par~cular m.~h~ -il,~ are only illustr~ive of the ~ s of thc present i"l,~,n~ , and that various modificad ns co~ be made by those skilled in the ar~ without depardng from ~he scope and SpiIit of the present i~v~.tl~, which is limited only by the claims that follûw.
30 Such ,.~l;r;t~;onC wollld include ent~ schemes involYing morc ~an three res~lnti~n layers, or schemes wherein, prior to cC~rr~i;f ~t ~ Q~ the co.ff~f~ e~ sent in lower re.solllti~ n layers are ~,b~l .J.~d from the co~fl;~ ;e~ being sent in highe~ ~eso~ ion layers.
T~ Fi~ld The invention relates to the enco~ine of video signals, and rnore particularly, 5 enco~in~ video signals in an efflcient manncr which pe~mits images having a wide range of resol~ltiQns to be derived f~om an encoded video signal.
Back2 . ound Of The Invention Many arp!ir~q~ )n~ such as, multi-point video t~,leco.~.~..cing, the windowing of displays for ~ n~ video co~ ;r?~ion~ on ~-.cl,ronous transfer mode ("Al~I") 10 n~ Lwo.L~, and ~.~adc~ high~efinitiQn ~elevision can ~enefit if video at v. rious r~sohltion~
can be derived from an encoded bi~h'~n. The simplest method of achi~ving this is the eim~ q~ ni~lue in which multiple i~ n~ coded replicas of a video ~u~ e (each scaled to a difÇ~ ,nl res~l~lti~n) are ~imllltq.~ouely ~ l In this applvach, caling is ~.Çu~ ed prior to dle co:l.yl~;s~;on/decomples ,ion o~ each replica, and each resolllti-~n scale 15 is then illde~ p~ encoded and ~ceign~ a portis)n of the total available h ~ " '~
bandwi.l~ll. As a result of his independent enrcYlin~ and tr~n~ e;on~ the ~ it technique requires a wide bandwidth.
A mo~ efficient alte~ativo to siml-leP~tin~ is scalable video enc~lin~. ~l~ucn~iy scaling is a low-~ Yity method of scalabie video encoding in which 20 a single video signal is ~ to mul~ple 1~C''i~3 which dccode images of varying resr~l ~ir,ne from that signal (le~ .~ e upon the particular signal deco~ing scherne employe~
by each receiver. A specific en~ ing method to which L~u~,ney scaling can be easily appliod uses d;.s~etc cûsin~ r..~ (~ blwks of oliginal or ~li.;~ion error.pixels to de~ive blocks of l~u~ o~ coe~f1~ien~c Vanous subsets of these L~uell.;y domain 25 e~ rrjf ~ .t~ CaJI be uscd ~0 generate ~r~le,l. resolution scales for a given image. . Such enCo~3in~ may be ~ '-~ - d using a slightly ~yl;fir~ version OI ~e ellCI:)de.~ close~l in the TntAmqtifnq1 Standards O,~ nl~ ~{~ Draft 11172-2, "Coding ûf Moving Pictures and ~ccr - ~ Audio ~c~r Digital Storage Media at up to about 1.5 Mbit/s," Nov.
1991, and in "Yideo Coding Using the MPE(;-1 Co~ ion Stand~rd," A. P~ri, P~e~
30 of the Socie~,r for 1~ Display 92, May 1992, Boston.
~. . .
Thc tl~i ,fol.lJ coding p~ion of these enr21~,9~6co~ 7u~s thc DCI as a ~unction of the energy in ~he bloc~ it receives from an o~iginal image, oq from the dif~erence between the onginal . nd its p~li.iLion from a previously decoded image. ~ccllmii~g an 8x8 pi~e element ("pixel") basic bloc~ size, each 8x8 pixel bloclc of a picture is ~ 1 in~o an 8x8 5 bloc3~ of DCI- coPffi~ent~ These c~,ffirientc re then ~ and scanned so that they may be converted into a one ~ n..nc~o~ s~uenc~. An entropy coder within the video encoder ccJmp~sses the one ~ .r-~- onAl sc~ ce to a series of number pairs called run/levels. Within a runllevel the first number ~l~n~ the number of zero coeffiri~nt~
~etween the last two non-zero co~fficientc in the one ~ nc;on~ nce" and the second 10 number ne~se~ the c~ ~ value (i.e., c~ n~ n index) of the next non-zero coefficient in that se~., nne~ The number of zero c~ m~:entc is called the length of "mn of zeros". For , '~, the one flim.oncionql s~l"~ ~re of ~ 1 coeffirient~ 1 1 0 0 S O O
0 6 3 0 0 4 0 0 0 ... 0 ~esults in the run/level seqllpnre of (0, 1) (0, 1) (2, S) (3, 6) (Q 3) (2, 4) (EOB). EOB, or End-of-Block, in~ S that all of thc ~ Y~g ~ I coçffir~
15 are zeros. Prior to 1~ , shese pairs, inr~ ing dle (EOB), are encoded using ~ P~l~
length codes ("VLCs") col~ L;i~g ~f a ~ ce of binary di~ts o'olD;~ed from a code ta~lc which is optimi7ed to assign shorter codes to most frequent pairs occ~lrring in typical video images. This way, long runs of zeros (which are c~m~n in mo~ion c~
based image coding) can be effiri~ntly coded. When the ll - d signal is received, or 20 stored signal is ~ ie~, th~ VLCs are decoded and the run/level pai~s are l~o.~ ,d. The non-zero ~lu~l; ~I cc~ are then inve~ ~ , the zero cC~r~ çnt~ are ~cu.~ t~,d, and an inverse DCI ("IDCT") is pc.ro.ll~ on the entire block.
r~u~ ,y scaling allows fo~ effi~ient c-~co~ of video in a manner which f~i1it~tes the d~ alion of various res~ tiong at the decod~ ~y using different IDCT block 25 sizes. It can be shown that if a decoder applies the lD~ to an upper left con~er sub-block of every received blocl~ of c4~ rl ;. i.c ~ can generate lower rçsohltion images. For example, if a 2x2 su~block of an encoded 8x8 bloclc is ~ e~ image " ~ oi~ will be reduced by a ~actor of four bo~ veItically and 1~ ly (referred to as f-scale 2 ~uen-;~ scaling).
In this manner, tho r.~"~ sC~lo~ility can be ,~-men~ed on the decoder without any 30 .- ~1;rir ~;o ~ to the encoder. Ilu..~ , because of dle VLCs used ~or roding run/leYel pairs, dlere is no way to detect the end of a block (EOB~, oq the b~ginning of a new Uock, unless the entire block is de~ As a result, in order to recov~ a su~blocl~, the variable-length word decoder must operate as though a full ~esolution image were being recovered (this requires very fast, expensive circuu~:ry to ~ used to decode even low resolu~ion images).
-3- 20~6~78 Using unique maricer codes to separate su~blocks is not feasible because of Ihe high oYerhead cause~ by such codes.
An existing method ~or ob~ihlil.g a L~uen~iy scalable 1~ is to encode the cc~r~ from ~u~blocks separately in a layered S~ and m~lltipl-x them as slices S of various layers in the bi~ In this ~thod, the lowest .~ sQ~ ;v~ layer keeps the basic u;lu e of the bi~ ,h~, but fewer DCT coefflri~nte are inrh1d~ with each block (such as four COer~ tC for f-scale 2). The ~...~ ine cO~mripntc for each block in the slice are sent in slave slices which are s~,pa~at~i by inAe~.~ entifi~ e bit pattems called slave slice start codes. One problem with this ~ u,h is that every coded Uoclc in each 10 scale has an EOB ~c~;~ted with it to mark the last non-ze~o co~ffiriPnt intllldpd in that scale. As a result, instead of sending a single EOB for each code~ bloclc as in non-scalable coding, multiple EOBs are sent which d~.~s the t;rf;~;~"cr of the Ç~Je.i~ scalable coding.
Another problem with this soludon is the ~" erhead it i..ll~lu~xs ove~ and above15 the overhead caused by dle slave slice start codes. The a~ows in FIG. lA shows a typical zigzag scan of an 8x8 co~f~cienr bloc~ (l00), whe~o each of t~e nu~ ;,~nds to the loca~on of the s~-~qc: in the one ~ 1 sc~l"e~r~ derived by tho scsn operation.
FIG. lB shows the zig-zag S~pnnin~ patte~n applied to 2x2 (l0l), 4x4 (102), and 8x8 (103) coeffic;en~ sub-blocks used with f-scale 2, ~-scale 4, and f-scale 8 laye~s l~e~ ely. If 20 c~rr;~ 3 and 4 are ze~os, they will be coded as part of a single run of zer~s within the bi~ ,~ if the 8x8 block is scanne~ as a whole (FIG. lA). II~ ., in exisdng methods of ~i~u~,n~;y scalable bl~ck ~ ,e they must be coded s~ t~ly as they are each in a dil'fe-~nl c~ rr;~ sub-block. While dlere are only th~ee such b~eaks ~C~soc ~ d with an f-scale 2 sub-block (between the co~ rr~ ,t~ the solid-line ovals of FIG. lB), ~here are 25 seven ~~~cc' ~ ~ with an f-scale 4 su~bloek (between the c~f~ .n6 within the dotted-line ovals~. Each ~ealc in a run of zeros inl~uduc~l by the su~block divisions l~ads to increased coding o.,, ~ lhile this ove~ ,dd may be reduced by using separate VLC tables~in~l~l for e~ch f~u~ scale, these sepa~ate tables Ll~ ce ~ 1 co~lpl~ ily.
Summ~rY Ot The Invenffon The arv,e .. ,1;r~ cm~ are solved, in ~co~ ce with the princir1~s of the h~,ntiol~, by employing an ef~lcient ~i~u~,ncy scaling 1ec~ e whe~ein the number of coerrll :e~ to be it~cln~ in each su~block is se1ectn~.1e and a code i!.-lirE~ g ~e number 2asGs7g of cQ~ffi~n~ within each layer is inserted in the bi~ at the ~g;nni.le of each encoded video s~ This technique allows the oliginal runs of zero co~ffirtçnt~ in the bighest resc llltion layes to remain intact by fonning a su~block fo~ each scale from a selected number of co~rr~cie~ls along a cor~inl~ou~ scan. These su~blocks may be decode,d in a standard S fashion, with an IDCT applied to square sub-blocks obtained by thc a~p.;atc zeTo padding of and/or discardtng of excess cocr~ c from each of the scales. Sincc tbe o~iginal leveVrun st~ictics arc not changed by this method, the original VLC tables can be used to encode/decode all video layers derived from the sub-blocks. This invendon further Llplo7~s coding efr~ by ~llowing an implicit EOB to separate blocks, making it u~ ece~ y to 10 transmit an explicit EOB signal in most cases.
Briel~ D~ n O~ The Dra~nn~
In the drawing:
FIG. lA is an illu~ tion of a zigzag 3c- - ~ g of co~ rl~ wi~3in an 8x8 block;
PIG. lB is an illui,~a~ of a zig-zag ~ ~ Ig of co~r~~i wi~Ll 2x2, 4x4, and 8x8 sub-blocks;
FIG. 2A is an illustration of the co~ff1rient s~ ction t~ch~ of the invendon applied along a zig-zag scan to a p~rtion ~ a 4x4 su~block of an 8x8 block;
FIG. 2B is an illu.,ild~oll of the c~hrr~ Fel;Oi~ tP~hniqlle of the invention 20 applied along a zig-zag scan to a 4x4 su~block and a portion of an 8x8 block;FIG. 3 is a ~;rnp~ifi~d block diagram of an encoder for imrle...... i.~g îl~u.,.l~;y scaling with scan bascd cc~fficient ~e1P~ n in acoo~ * with a particular c.. ho~;.. i of the in._.~lio~, FIG. 4 is a ~ -lified block diagram of a decoder ~apable of decoding the 25 r~,5O~ layer .,~ .o~ g to the the lowest rl~u~ ,y scale from the ~ ~Lu output ~y the encoder of FIG. 3; - -FIG. 5 i., a ~iml~lified block diagram of a d~der capable of (~co.l;~g hvoresol ltic-n layers c~ ~7~ to two &~uency scales; and FIG. 6 is a simpli~led block diagram of arl inlrl; ...f,r.li.lion of the invention 30 whereîn each vides> layer is encoded via a d~Aicflted ~ ,., scanner, and va~iable-length coder.
-5~ 2~9~878 Det~iled D~cr~tion O~ The L7~.~tio.i FIG. 2A shows an example of ~he adaptive cQ~ffi~nt selection techrLique of the in~ uon applied along a zig-zag scan to a portion of a 4x4 sub-block of an 8x8 block.
The number of co~ri~ icnt~ to be included in each layer is sP~ , and the desired number S of coemeiçnt~ foq each layer is incllld~p~ at the be~ of each video s~ e In the particular example illustrated in FIG. 2A, the zigzag scan pattern is such that if the selected number of co~pffir;ente for the f-scale 4 layer is 10 or less (see coefficient group 201), the selected coPffici~ntc will cc,~ D~nd to a subset of the copffiri~p-nt~ usually cc...lAi~ed within a 4x4 su~block. For groupings of coefflriPnte larger than 10, some c~rllcif ~Ic from the f-scale 8 layer would be in~ dp~1 For ,Pyr 1~, aS shown in FIG 2B, a grouping of 14 eoP.ffie~ would inelude the corrr;~;e~ within groups 2û1 and 202, and cc~ rr;rie ~I number 11 within group 202 would usually be inclll~ed in an f-scalP, 8 layer Such "extra" cc~-rf;c~
may be ignored by decod~ ~ which decode only f-scale 4, as such decoders employ a 4x4 lDCI on the 4x4 upper left corner su~block. For L~ue~lc3~ scalable decod~ that may 15 decode images at higher resol~tion~, the extra c~,~ t~ are nccessal~ to decode layers c~ to the higher res- ln~ion~
The i,.~ s overhead as it does not require the inse.lion of EOBs in cvery coded su~block of every scale to partitiQn the coefflrien~ within each layer. Overhead is .ni~-i...;,~ by ~_ ,1c ~- ~,, the following coding gniARlines. If the 20 last co~ of a set of c~ rr~: nt~ in a given layer i~ non-zero, no EOB is necc~ as the decoder can always d~ end of a sub-block by cou ~ine the number of coR~ff1ri~ontc decoded (the number of co ~rr~ ;e .~. in each layer is inclllded at the beginning of each video s~.,e ~e). If the last ç~,rr..~ of a set Of c~.~r.. i "~ in a given layer is z~,ro, and falls within a run of zeros, the whole run inch)clin~ the first non-zero coefficient following it is 25 inch~ d in Ihat layer (provided it can be en~ocled/decoded by the lower layer VLC table).
The "extra" c<~;c - ~ irtrlu~ in the cu~ent ~solllt;on layer are lgnored by low reso!-ltion ~lccod~ , ~d us~ill ~ing cs,l~ onJillg higher 1~-L1I ~r layers by scalable or high reso~ nn d~.~. ~he inr~ on of the "extra" coeffls~;pnt~ simply reduces the run/level pairs in~ "ll~ in the ncxt higher resollltinn layers by one (co~ onil;,.~ to the one extra 30 run/level pair inrlu/lçcl in the lower ~ hltirn laye~). There is nv net increase in overhead.
Finally, if all co~rl;-;enl~ after a non-zero co~rlici~n~ 3~e zero, but the run/level event g~ n i ~cd cannot be encode~l/d~o~ed by the lower level VLC table, then an E~OB is used once.
-6- 2096~7~
FIG. 3 shows a simrlified block diagram of an enulder for ilnple~,ltillg f~.cncy scaling with c~fficient sele~tion The encoder is similar to those '~ 051~l in the Internalional Standards 01~ ;7 ~ Co~ ilt-~ Draft 11172-2, and in "Video Coding Using the MPEa-l C~ ,Djion Standard," but includes layer erld ~ccssor 301 and b;bll~
S layers buffcr 302 to c~ Ats splitting the c~rl; :~ ~ into layers. An image bloclc is input on line 303 and passes through dirr~ ,nc.l~g element 304 to ~ ro.ll- coder 30S. Within transform coder 305 a DCI is p~rv~ cd on the image signal, and the resulting c~ rl~ s are ~ i and scanned. The scanned co rr~ t~ are decodcd by local decoder 306 and passed to motion ectim~ n and co~ ~n~ circuitry 307. Motion e~ A~ion and ~ e 'ic~n circuitry 307 c~l~n~l~tes a p~ediction of the image bloclc input to line 303 based upon the bcst match of the block to blocks in a previously decodcd image rec~ived from local decoder 306. This estirnate is L~ enccd from the in~o7n;n~ image signal via Lfr~ ing element 304. Motion estim~tir~n and c~.n~ circuitry 3a7 also outputs modon v~cto~s which fonn a portion of the overhead I~ ti~ --O-' ' ' with the bloc~c being çm~
The output of Iransform cod~ 305 is also passed to entropy coder 3û8, where run/leYels ar~
CO~.~pu~13 and ~ .e-length coding is p~ru~ . All of thc a~ve dfs ~;b~l ~ ,c' g op~ations are well h~own in the art and ~ ~ in ~he two lefe.~ efeTred to earlie~.
Rwl/level data and a v~ ''- length encoded bits~eam are output by entropy code~ 308 to layer end ~, ~ce~so. 301. Layer end I,~css~r 301 counts the number of COerf;C f~ ' and routes the VLCs to co~ onrling layers acco~ g to the gnidelinp~s of the nli~n ~psçrih-p~l above. The be~nnitlg of each slice is mal~f d within the bi~ by a slice start code or a slave slice start u)de ((lepp~ upon the r~uellcy scale). Codes ~1lOC~t~ to the lowest reSolntion layer are di~ecdy sent to ~e n ~ ipl~Yf~ to be inserted in the ~ ~. Each code which belong to higher ~esoll~ti~- ~ layers ar~ stored bil~eall. layers buffer 302. At the l~.~ e of all but dle lowest r~ u~,ncr Laye,r slice, a slaYe slice start code is inserted in the bitstream by mlllti~ Yf~ ("MPX") 309. This code is ~ollowed by any coded data f~r this r.~q~ n4,~ scale cou~ f~ layers buffer 302. This orde~ing of f~eque scale slices.~or ea~h layer is l~:ir- t~ wldl the highest resohl~ion slave slice is i~~~ cd in the ~ h~,d~. The process is repeated ~or all s~ces in each picture. Multiplexer 3~9 outputs the encoded ~;12h~. and overhead codes to buffer 310. The overhead codes inclusle an in~iir~tion of the number of c~ffic;ent~ in each laye~, which is ~en~ ted by an i-ld~pC~d~"t pl~CeSS~JI (nc,t shown in FIG. 3) and inse~.ed by MPX 309 at the beginnin~ of each video s~l~,e-lce Buffer 310 ~ the ~it~ at the a~pl~liat~ rate to a decoder or a storagc device.
7 2~96878 PM. 4 is a cimrlifiP~3 block diagram of an ~rraneem~n~ for de~ n~ ~ lowe~t resol~lti~n laycr f~m the bi~LI~ ou~put by tbe encoder of FIG. 3. The ncod~d bi~ke~
is input to buffer 4Q1 via line 402. Buffer 402 passes the b;~h~ ~o sli~id~pntifi~ 403 which scans the b;~ e~ for slice start GOdeS and slave slice start codes. Slice identifier 403 S passes the portion of the bi~4~ between the slice-start-code and thc first slave-slice-start-code to ~Pm~ erh~ -le~ h decodcr ("DMPX/VL dcc~. ) 404.
As this decoder is only conce~lcd with thc lowest (and first) encoded layer of video this is the only portion of the bit~h~ which need b~ ~p~ DMPXIVL decoder 404 d~rm~ and decodes the received portion of the l~ib~. to obtain 'I~ DCI
10 cc~ rr;~ These c~ rr;~ wes;,ed by inverse q~ 405 and inverse scanner 406, which perfonn thc inverse furlcdon of the scanncr and ~lu~ of ~n.~f~m coder 305 (l;IG. 3). The cG~fr~ e then counted, and ~ cof rl;~-:r l~ of ~o arc inserted to form square bloclcs by cGe~;cienr count~ ,addw 4~7 (square blocks are requi~d for ye.r,.~lg an IDCI). Since the coem~;ent~ used witb sm~ller size IDCIs must be scaled, coçff1r;ent scaling is p~.. r.. ,~ on ~be block by c~ rf; ~ t scale~ 408 before an lD~X is p~- r.-- .~ ed by ID~ circuit 409. The vidco blocks ou~put by ~CI ci~uit 409 are passed ~hrough sIlmm;r~g element 410, whe~e a p ~;~i~l ~ge may be added to them, prior to output on line 411. As is shown in FIG. 4, the p,~ t~Jd image is ~L~ c~ by motion e~
pr~;c~I 412. Modon CQ-l~ ~r d ~.~ic~. 412 g ~t~5 this y._~l;ct~d image as a funstion of scaled motion vectors received from modon vector scaler 413, and the video blocks output on line 411. Scaling is needed because ~he modon vectors are derived for the highest res ~ tinn sca1e and their values must be adjusted for dle c~rent, lower resoI~ltion scale FIG. S shows a ~;mplifi~d block diagram of a decoder capable of decoflin~ two scalable resoIvtinn layers. The encoded l~ ,~ is input to buffer 501 via line 502. Buffer 501 passes thc bil:~h~ to slice~ 503 which scans ~e ~il;,hea~ or slice start codes and slave slice start codes. Upon d~t~ a slice start code, slice il1entifi~ 503 sends a control sign~ inG 504, which directs the output of c~ i cou~ adder 505 to sli-~ebuffel 506. IJp~n~l~ tb;~ a slave slice s~art code, slice i~l ~ 503 sends a control signal, via line 50~, which direc~s the ou~tput of co~fflri~n~ co~lL~t~ '~ - 505 to line 507. This insures that the low ~ laye~ coe~- will be passu~ to slice bu~fer 506 for stolage, so that thcy may bo ay~ Ldtely eO~ ~1 widl c~,fr~ om dle higher resolution slavcslice which is received a~ter the low reso~ ion layer.
The l~ ~ is passed from slice iden~ifi~ 503 to DMPX/YL decoder 508.
where it is dem~ and docoded to obtain ~ 7PA D~ coeffi~ nt~ These 2~96~7~
cC~r~ cnl~ ~ then y~s¢d by invcrse ~1~AfI~ d inYCrSC scanne~ S10. The conrfifr~ ted, and ~ d J c~n5~l coef~ ntc of Za'o aI~ inscrted to f~m squa~c Wock by co~ffi~nt eoul ~.t~ ~' SOS (square blocks arc required for ~ .f.. ing a~ ~. If this is a low ~esoll~tion laycr, c~effirient count~,.fpr lt' 505 passcs the c~r.'.. ~ to slicc buffer 506; o~ ,.w.se co~fficien~ pz ''- SOS outputs thc c~fr ~ on Line 507.
Slice buffer 506 allows the low resol.u~i~n c~rr~ to be held whilc thc high rssolntion coeffirient~ are decoded. The c~ rr,~ ;~f~l~ are passed from slice buffer 506 and/or line 507 to coeffir;en~ scaler Sl l and IDCT circuit 512 via l~;y~ n select switch 513. The position of resoln~ion selcot switch is controlled by an in(lfc~ p~CCSSol not shown in FIG. S.
Thc video blocks output by IDCI circuit 512 are passcd through s~mming element 514, where a ~ lietcd im~ge may be adfled to them, prio~ to output on line SlS. Motion CC~ J
predictor 516 4. ..~ t~ s ~is predicted image as a function of scaled motion vectors recdved from motion vector scaler 517, and the ~ideo blocks output on linc SlS.
In certain ~rFIir ~inn~, different VLC tables may haw to be employed for each S ~rr~.~n~ video layer. Since the possible mn lengths for the lowe~ res~ on layers a~e s}~r than that of the high re~~ layers, it is possible ~o use shorter VLC tables for ~e lowcr reso'~ltin~ layers. This may reduce the required cc,...~ t ~;ol~ esoul~;cs and memory required to fi~filir~ the video ~li'r,cY~ g Also, 1;1 certain aMl'~ a separate .lu~.ti~. may be utilized for each video scale ~ a11ow i~d~ 'Af ~ quality ~ 1jr- -- cc,~ .>~ g to ~it 20 rates of each layer.
FIG. 6 shows an encoder imrl - - which is similar in ope~ation to the encoder of FIG. 3, but whe~ein each layer in enco~ed via a ~e ~,c a~ ~ ~ 4 ~ i; . .. and vanable-leng~h coder. ~ thi9 partic~ )n~ image bloeks are input on line 601 toJirrw~.lci.lg element 602 and passed to DCI circuit 603 where a DCT is p~. r~ A All of 25 the c~rr;c;~n~ output by DCI circuit ~03 aro ~hen passed to th~ separate 605, and 606), ~ree separa~e scanners (61)7, 608, and 609), ~hree ~eparate run~level Colllput~
(610, 611, and 612), thsee separate layer end 1J1VeeSSI~13 (613, 614, and 615), and three separate va.-'l~ eoders (616, 617, and 618). The layer end p..,cesso.~ are ~ go that ~ layer erld ~ SS~ fr~lit~tine thc enro line of ~e lower laye~s 30 may pass ~( - to higher reso~ ;on layer end pl~cc,ss~l ~ as to dle actual number of co~rfi~ :e~ which need be in~ f d in each layer su~block. CG~ ~r.~ :~,. .t~ that actually belong in a highel~ resolvti-n layer, but are inc1vr1~d in a lower resolvtion layer so as not to break a run of zeros, are coded with the lower resolution laye~. The valiable-length coders for the l;al~i (617) and highest (618) rcsol~tion layers output l,i~ o buf~ers 619 and 9 20968~8 620, ~ .vly. Illis cnables thc output of thc ~ ble l~,ngth coders ~o be p~op~ly se~n~nc~ by MPX 621. MPX 621 ou~puts the s~ ed bibLi~ ~d l~v~head codes to buffer 622 for ~ c ";~ioil sr s~age. Motion c~ on and C~ ;On circuitry 623 c~lcula~P~ a prediction of the imagc bloclc input on line 624 'oased upon various control S pa~ , the signal from local decoder 625, and the image blocl~ sigr.al on line 601. This estim~te is added to the inromillE im~ge signal via ~L;fl~ncing element 602. Motion es~ima~ion and co~ ion circuitry 623 also outputs modon vectors which are i~CO~ t~
in the overhead h~l,.la~ion for the layer being encoded by MPX 621.
Since employing a single 4~ t;Di \~ and variable-length coder gives rise to the 10 F~ -l of b~.s,.alin~ run lengths as large as the en~rc bloc~ length in any re~olntion layer, using s_aller VLC ta'oles requires imrl~ .b .~ g the following n~A;~ coding gui~Plines~
For each re Iayer, in addition to ~he desired num'oer of c~ rl t~, the ~
allowed num'oer of c~ rl ;- o ~ ~, wnich inr~ the rwlllevel ~eeodi7~g e I~ahility, is s~ifi~d at the be~nning of the video s~u .ee If, as a result of a long run of zeros, tnis limi~ ~.
15 going to 'oe e Gc~cA, tne mn is cut and an EOB is sont. The ~ ning ~eros are ser~t with the next resol~ on layer. ~learly9 ;..~ of EOB's in su~block~9 and cutting zero runs will increase tlhe u.. hc 1 The increase '~e~os-n~s si~if ~ when r.~ el ~7OCo~ e c~pa~ility is limited. The actual overhead incroase is image ~ but c~ , ~ly smalle~ t'nan tne o.~ n~d ~l~lu~c~ by the previously hlown method of sending EOBs fo~ each coded 20 Uock of every scale. Ille sig~nal ou~tput by the en~ of FIG. 6 may ~ decoded by a decoder similar to that of FIG. S, but which employs a separate inverse ~ nli~ inverse scanner, and DMPX/VL decode~ f~r each individual res~ n layer.
Thc aboYe~ sc ;hcd ~ ~n provides a highly efficient ~u~,ncy scaling t~ u~ wherein thc numbe.r of cc~ t~ to be inc~ ~ in each su~block is sçle~t~
25 and a code i~ ;. ~;n,@ the number of co~rl';~ S~; wi~in each laye~ is inserted irl the l~it~
at the bG~ of e~ch encoded video ..~llten~-e, It will be 1l~ dersto~l that the par~cular m.~h~ -il,~ are only illustr~ive of the ~ s of thc present i"l,~,n~ , and that various modificad ns co~ be made by those skilled in the ar~ without depardng from ~he scope and SpiIit of the present i~v~.tl~, which is limited only by the claims that follûw.
30 Such ,.~l;r;t~;onC wollld include ent~ schemes involYing morc ~an three res~lnti~n layers, or schemes wherein, prior to cC~rr~i;f ~t ~ Q~ the co.ff~f~ e~ sent in lower re.solllti~ n layers are ~,b~l .J.~d from the co~fl;~ ;e~ being sent in highe~ ~eso~ ion layers.
Claims (13)
1. Method for encoding a digital video signal as at least two sets of coefficients, each of said sets containing a selectable number of coefficients and a code indicative of said selectable number, wherein said coefficients within each of said sets comprise a layer corresponding to a particular image resolution scale, and wherein said encoding is performed so that if the last coefficient within any given set of coefficients is zero, and either initiates or falls within a sequence of zeros, representations of the sequence of zero coefficients, and of the first non-zero coefficient following the sequence of zeros, are included in the set of coefficients.
2. The method of claim 1, wherein if the number coefficients within a given set of coefficients comprising a layer corresponding to a lower image resolution scale, exceeds a predetermined maximum as the result of the length of a sequence of contiguous zeros concluding the set, the concluding sequence of zeros is truncated so the number of contiguous zero coefficients contained within the set is limited to said predetermined maximum, and the set is concluded with an end of block code.
3. The method of claim 2, wherein zeros truncated from a given set of coefficients are encoded at the beginning of a set of coefficients comprising a layer corresponding to a higher image resolution scale.
4. apparatus for encoding a digital video signal as at least two sets of coefficients, wherein said encoding is performed so that if the last coefficient within any given set of coefficients is zero, and initiates or falls within a sequence of zeros, a representation of the sequence of zeros and the first non-zero coefficient following the sequence of zeros, are included in the set of coefficients, comprising:
means for receiving a digital video input signal including a succession of digital representations related to picture elements of a video image;
means for coding said received digital video input signal to a one-dimensional coefficient sequence;
means for compressing said one-dimensional coefficient sequence to a series of run/level number pairs;
means for variable-length coding said series of run/level number pairs;
means for arranging said series of variable-length coded run/level number pairs into at least two sets, each of said sets consisting of a selectable number of coefficients and comprising a layer corresponding to a particular image resolution scale;
means for generating a code corresponding to each of said sets indicative of thenumber of coefficients within each set; and means for inserting each of said generated codes within the set of coefficients to which it corresponds.
means for receiving a digital video input signal including a succession of digital representations related to picture elements of a video image;
means for coding said received digital video input signal to a one-dimensional coefficient sequence;
means for compressing said one-dimensional coefficient sequence to a series of run/level number pairs;
means for variable-length coding said series of run/level number pairs;
means for arranging said series of variable-length coded run/level number pairs into at least two sets, each of said sets consisting of a selectable number of coefficients and comprising a layer corresponding to a particular image resolution scale;
means for generating a code corresponding to each of said sets indicative of thenumber of coefficients within each set; and means for inserting each of said generated codes within the set of coefficients to which it corresponds.
5. The apparatus of claim 4, wherein said means for arranging said series of variable-length coded run/level number pairs into at least two sets of coefficients is adapted to detect if the number of coefficients within any set of coefficients comprising a layer corresponding to a lower image resolution scale exceeds a predetermined maximum as the result of the length of a contiguous sequence of zero coefficients concluding the set, and upon detecting such a condition reduce the number of contiguous zero coefficients contained within the set to said predetermined maximum by truncating the concluding the sequence of zeros and inserting an end of block code at the termination of the truncated set.
6. The apparatus of claim 5, further comprising a means for inserting the zero coefficients truncates from a given set of coefficients corresponding to a lower image resolution scale at the beginning of a set of coefficients comprising a layer corresponding to a higher image resolution scale.
7. Apparatus for encoding a digital video signal as a multi-layer set of coefficients, wherein said encoding is performed so that if the last coefficient within any given set of coefficients is zero, and initiates or falls within a sequence of zeros, a representation of the sequence of zeros and the first non-zero coefficient following the sequence of zeros, are included in the set of coefficients, comprising:
means for receiving a digital video input signal including a succession of digital representations related to picture elements of a video image;
dedicated means for each of said layers adapted to code said received digital video input signal to a one-dimensional coefficient sequence;
dedicated means for each of said layers adapted to compress said one-dimensional coefficient to a series of run/level number pairs;
dedicated means for each of said layers adapted to variable-length code said series of run/level number pairs;
dedicated means for each of said layers adapted to arrange said series of variable-length coded run/level number pairs into a set of coefficients consisting of a selectable number of coefficients and comprising a layer corresponding to a particular image resolution scale;
dedicated means for each of said layers adapted to generate code indicative of the number of coefficients within each set; and dedicated means for each of said layers adapted to insert each of said generatedcodes within a corresponding set of coefficients.
means for receiving a digital video input signal including a succession of digital representations related to picture elements of a video image;
dedicated means for each of said layers adapted to code said received digital video input signal to a one-dimensional coefficient sequence;
dedicated means for each of said layers adapted to compress said one-dimensional coefficient to a series of run/level number pairs;
dedicated means for each of said layers adapted to variable-length code said series of run/level number pairs;
dedicated means for each of said layers adapted to arrange said series of variable-length coded run/level number pairs into a set of coefficients consisting of a selectable number of coefficients and comprising a layer corresponding to a particular image resolution scale;
dedicated means for each of said layers adapted to generate code indicative of the number of coefficients within each set; and dedicated means for each of said layers adapted to insert each of said generatedcodes within a corresponding set of coefficients.
8. The apparatus of claim 7, wherein each of said means adapted for arranging said series of variable-length coded run/level number pairs and dedicated to a layer corresponding to a lower image resolution scale is further adapted to detect if the number of coefficients within a set of coefficients comprising a layer exceeds a predetermined maximum as the result of the length of a contiguous sequence of zero coefficients concluding the set, and upon detecting such a condition reduce the number of coefficients contained within the set to said predetermined maximum by truncating the concluding sequence of zeros and inserting an end of block code at the termination of the truncated set.
9. The apparatus of claim 7, further comprising a means for inserting the coefficients truncated from a given set of coefficients corresponding to a lower image resolution scale at the beginning of a set of coefficients corresponding a layer corresponding to a higher image resolution scale.
10. Method for decoding a digital video signal having at least two sets of coefficients, each of said sets containing a fixed number of coefficients and comprising a layer which corresponds to a particular image resolution scale, said decoding performed so that if the number of coefficients within a given set of coefficients comprising a layer corresponding to a lower image resolution scale is in excess of predetermined maximum number of coefficients which may be decoded for a signal at said lower image resolution scale, the excess coefficients are truncated and only remaining coefficients associated with said lower image resolution scale are utilized in the process of decoding the portion of said digital video signal associated with said lower image resolution scale.
11. The method of claim 10 wherein said excess coefficients included within said given set of coefficients comprising a layer corresponding to a lower image resolution scale are employed in the decoding of the portion of said digital video signal corresponding with a higher image resolution scale.
12. Apparatus for decoding a digital video signal having at least two sets of coefficients, each of said sets containing a fixed number of coefficients comprising a layer corresponding to a particular image resolution scale, wherein said decoding is performed so that if the number of coefficients within a given set of coefficients comprising a layer corresponding to a lower image resolution scale is in excess of a predetermined maximum number of coefficients which may be decoded for a signal at said lower image resolution scale, the excess coefficients are not utilized in the process of decoding the portion of said digital video signal associated with said lower image resolution scale, comprising:
means for receiving said digital video signal;
means for decoding from said received digital video signal said sets of coefficients;
means for reconstituting run/level pairs from said sets of coefficients;
means for reconstituting the particular image resolution scale video layers fromsaid reconstituted run/level pairs.
means for receiving said digital video signal;
means for decoding from said received digital video signal said sets of coefficients;
means for reconstituting run/level pairs from said sets of coefficients;
means for reconstituting the particular image resolution scale video layers fromsaid reconstituted run/level pairs.
13. The apparatus of claim 12, further comprising:
means for storing excess coefficients associated with said layers corresponding to said lower image resolution scale;
means for receiving said stored excess coefficients;
means for reconstituting run/level pairs from said sets of retrieved coefficients;
and means for reconstructing higher image resolution scale video layers utilizing said run/level pairs reconstituted from said sets of coefficients associated with said layers corresponding to said lower image resolution scales.
means for storing excess coefficients associated with said layers corresponding to said lower image resolution scale;
means for receiving said stored excess coefficients;
means for reconstituting run/level pairs from said sets of retrieved coefficients;
and means for reconstructing higher image resolution scale video layers utilizing said run/level pairs reconstituted from said sets of coefficients associated with said layers corresponding to said lower image resolution scales.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US908,134 | 1992-07-02 | ||
| US07/908,134 US5253055A (en) | 1992-07-02 | 1992-07-02 | Efficient frequency scalable video encoding with coefficient selection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2096878A1 CA2096878A1 (en) | 1994-01-03 |
| CA2096878C true CA2096878C (en) | 1999-02-16 |
Family
ID=25425253
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002096878A Expired - Fee Related CA2096878C (en) | 1992-07-02 | 1993-05-25 | Efficient frequency scalable video encoding with coefficient selection |
Country Status (5)
| Country | Link |
|---|---|
| US (2) | US5253055A (en) |
| EP (1) | EP0577327B1 (en) |
| JP (1) | JP3221638B2 (en) |
| CA (1) | CA2096878C (en) |
| DE (1) | DE69320719T2 (en) |
Families Citing this family (126)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB9405914D0 (en) * | 1994-03-24 | 1994-05-11 | Discovision Ass | Video decompression |
| EP0576749B1 (en) | 1992-06-30 | 1999-06-02 | Discovision Associates | Data pipeline system |
| JPS5991480A (en) * | 1982-11-17 | 1984-05-26 | 日本電気株式会社 | Electrooptic display |
| JPH05167863A (en) * | 1991-04-15 | 1993-07-02 | Canon Inc | Method and device for processing image |
| US6680975B1 (en) | 1992-02-29 | 2004-01-20 | Samsung Electronics Co., Ltd. | Signal encoding and decoding system and method |
| KR960006762B1 (en) * | 1992-02-29 | 1996-05-23 | 삼성전자주식회사 | 2-dimensional data scanning selecting circuit for image coding |
| SG45281A1 (en) * | 1992-06-26 | 1998-01-16 | Discovision Ass | Method and arrangement for transformation of signals from a frequency to a time domain |
| US6263422B1 (en) | 1992-06-30 | 2001-07-17 | Discovision Associates | Pipeline processing machine with interactive stages operable in response to tokens and system and methods relating thereto |
| US6112017A (en) | 1992-06-30 | 2000-08-29 | Discovision Associates | Pipeline processing machine having a plurality of reconfigurable processing stages interconnected by a two-wire interface bus |
| US6330665B1 (en) | 1992-06-30 | 2001-12-11 | Discovision Associates | Video parser |
| US5768561A (en) | 1992-06-30 | 1998-06-16 | Discovision Associates | Tokens-based adaptive video processing arrangement |
| US5842033A (en) | 1992-06-30 | 1998-11-24 | Discovision Associates | Padding apparatus for passing an arbitrary number of bits through a buffer in a pipeline system |
| US6079009A (en) | 1992-06-30 | 2000-06-20 | Discovision Associates | Coding standard token in a system compromising a plurality of pipeline stages |
| US6067417A (en) | 1992-06-30 | 2000-05-23 | Discovision Associates | Picture start token |
| US6047112A (en) | 1992-06-30 | 2000-04-04 | Discovision Associates | Technique for initiating processing of a data stream of encoded video information |
| US5809270A (en) | 1992-06-30 | 1998-09-15 | Discovision Associates | Inverse quantizer |
| KR970000761B1 (en) * | 1992-10-07 | 1997-01-18 | 대우전자 주식회사 | Mini high-definition television |
| US5835792A (en) | 1993-06-24 | 1998-11-10 | Discovision Associates | Token-based adaptive video processing arrangement |
| US5861894A (en) | 1993-06-24 | 1999-01-19 | Discovision Associates | Buffer manager |
| US5805914A (en) | 1993-06-24 | 1998-09-08 | Discovision Associates | Data pipeline system and data encoding method |
| US5699544A (en) * | 1993-06-24 | 1997-12-16 | Discovision Associates | Method and apparatus for using a fixed width word for addressing variable width data |
| TW323029U (en) * | 1993-06-30 | 1997-12-11 | Victor Company Of Japan Co Ltd | Processing system for digital video signal |
| BE1007330A3 (en) * | 1993-07-16 | 1995-05-16 | Philips Electronics Nv | Apparatus for transmitting a digital image signal. |
| JP3292221B2 (en) * | 1993-09-14 | 2002-06-17 | ソニー株式会社 | Image compression coding method |
| KR0183688B1 (en) * | 1994-01-12 | 1999-05-01 | 김광호 | Image encoding method and device |
| JP2930089B2 (en) * | 1994-03-17 | 1999-08-03 | 日本ビクター株式会社 | Compressed video information recording method and compressed video information recording device |
| CA2145379C (en) | 1994-03-24 | 1999-06-08 | William P. Robbins | Method and apparatus for addressing memory |
| CA2145365C (en) | 1994-03-24 | 1999-04-27 | Anthony M. Jones | Method for accessing banks of dram |
| CA2145363C (en) | 1994-03-24 | 1999-07-13 | Anthony Mark Jones | Ram interface |
| US5754235A (en) * | 1994-03-25 | 1998-05-19 | Sanyo Electric Co., Ltd. | Bit-rate conversion circuit for a compressed motion video bitstream |
| US5940130A (en) * | 1994-04-21 | 1999-08-17 | British Telecommunications Public Limited Company | Video transcoder with by-pass transfer of extracted motion compensation data |
| US5510842A (en) * | 1994-05-04 | 1996-04-23 | Matsushita Electric Corporation Of America | Parallel architecture for a high definition television video decoder having multiple independent frame memories |
| GB9417138D0 (en) * | 1994-08-23 | 1994-10-12 | Discovision Ass | Data rate conversion |
| US5550847A (en) * | 1994-10-11 | 1996-08-27 | Motorola, Inc. | Device and method of signal loss recovery for realtime and/or interactive communications |
| US5473376A (en) * | 1994-12-01 | 1995-12-05 | Motorola, Inc. | Method and apparatus for adaptive entropy encoding/decoding of quantized transform coefficients in a video compression system |
| US5748244A (en) * | 1995-03-28 | 1998-05-05 | Daewoo Electronics Co., Ltd. | Adaptive runlength coding based on zero and non-zero subblocks |
| US5724453A (en) * | 1995-07-10 | 1998-03-03 | Wisconsin Alumni Research Foundation | Image compression system and method having optimized quantization tables |
| US5831559A (en) * | 1996-01-24 | 1998-11-03 | Intel Corporation | Encoding/decoding video signals using multiple run-val mapping tables |
| US5706002A (en) * | 1996-02-21 | 1998-01-06 | David Sarnoff Research Center, Inc. | Method and apparatus for evaluating the syntax elements for DCT coefficients of a video decoder |
| US5768537A (en) * | 1996-02-22 | 1998-06-16 | International Business Machines Corporation | Scalable MPEG2 compliant video encoder |
| JP3575508B2 (en) * | 1996-03-04 | 2004-10-13 | Ｋｄｄｉ株式会社 | Encoded video playback device |
| US5801778A (en) * | 1996-05-23 | 1998-09-01 | C-Cube Microsystems, Inc. | Video encoding with multi-stage projection motion estimation |
| US6134269A (en) * | 1996-09-25 | 2000-10-17 | At&T Corp | Fixed or adaptive deinterleaved transform coding for image coding and intra coding of video |
| US5953506A (en) * | 1996-12-17 | 1999-09-14 | Adaptive Media Technologies | Method and apparatus that provides a scalable media delivery system |
| US6339614B1 (en) * | 1996-12-20 | 2002-01-15 | Intel Corporation | Method and apparatus for quantizing and run length encoding transform coefficients in a video coder |
| US6128340A (en) * | 1997-03-14 | 2000-10-03 | Sony Corporation | Decoder system with 2.53 frame display buffer |
| US5870144A (en) * | 1997-03-28 | 1999-02-09 | Adaptec, Inc. | Reduced-quality resolution digital video encoder/decoder |
| US20020196853A1 (en) * | 1997-06-04 | 2002-12-26 | Jie Liang | Reduced resolution video decompression |
| JP3469438B2 (en) * | 1997-08-28 | 2003-11-25 | カスタム・テクノロジー株式会社 | Image signal processing method and apparatus, recording medium |
| AU1285099A (en) * | 1997-11-07 | 1999-05-31 | Pipe Dream, Inc | Method for compressing and decompressing motion video |
| KR100249235B1 (en) * | 1997-12-31 | 2000-03-15 | 구자홍 | Hdtv video decoder |
| KR20010071519A (en) * | 1998-06-19 | 2001-07-28 | 벤자민 에프 커틀러 | Decoding an encoded image having a first resolution directly into a decoded image having a second resolution |
| SG77650A1 (en) * | 1998-09-07 | 2001-01-16 | Victor Company Of Japan | A scalable delivery scheme of compressed video |
| US6603883B1 (en) * | 1998-09-08 | 2003-08-05 | Canon Kabushiki Kaisha | Image processing apparatus including an image data encoder having at least two scalability modes and method therefor |
| SE513353C2 (en) * | 1998-10-21 | 2000-08-28 | Ericsson Telefon Ab L M | Partial image retrieval in the compressed domain |
| US6591013B1 (en) | 1999-03-22 | 2003-07-08 | Broadcom Corporation | Switching between decoded image channels |
| US6446037B1 (en) | 1999-08-09 | 2002-09-03 | Dolby Laboratories Licensing Corporation | Scalable coding method for high quality audio |
| KR100827212B1 (en) * | 1999-08-27 | 2008-05-07 | 코닌클리케 필립스 일렉트로닉스 엔.브이. | Scalable coding |
| EP1161840A1 (en) | 1999-08-27 | 2001-12-12 | Koninklijke Philips Electronics N.V. | Scalable coding |
| US6724939B1 (en) * | 1999-08-31 | 2004-04-20 | Telefonaktiebolaget Lm Ericsson | Low complexity JPEG decoder |
| JP2002027469A (en) * | 2000-07-05 | 2002-01-25 | Matsushita Electric Ind Co Ltd | Bit stream converting method and device, and program recording medium |
| US6631163B1 (en) * | 2000-11-14 | 2003-10-07 | Koninklijke Philips Electronics N.V. | Dynamic adaptation of complexity in an MPEG-2 scalable decoder |
| US20020122483A1 (en) * | 2001-03-02 | 2002-09-05 | Matsushita Electric Industrial Co., Ltd. | Encoding circuit and method |
| US20020191698A1 (en) * | 2001-06-15 | 2002-12-19 | Solidstreaming, Inc. | Video data CODEC system with low computational complexity |
| US7197077B2 (en) * | 2001-12-20 | 2007-03-27 | Sbc Technologies Resources, Inc. | Multilevel data compression using a single compression engine |
| US7190840B2 (en) * | 2002-01-07 | 2007-03-13 | Hewlett-Packard Development Company, L.P. | Transform coefficient compression using multiple scans |
| US6993196B2 (en) * | 2002-03-18 | 2006-01-31 | Eastman Kodak Company | Digital image storage method |
| US7129962B1 (en) * | 2002-03-25 | 2006-10-31 | Matrox Graphics Inc. | Efficient video processing method and system |
| PT1467491E (en) | 2002-05-02 | 2007-03-30 | Fraunhofer Ges Forschung | Arithmetical coding of transform coefficients |
| US7006572B2 (en) * | 2002-05-09 | 2006-02-28 | Mitsubishi Electric Research Labs, Inc. | Method and apparatus for decoding video bitstreams to reduced spatial resolutions |
| US7386049B2 (en) * | 2002-05-29 | 2008-06-10 | Innovation Management Sciences, Llc | Predictive interpolation of a video signal |
| US6917713B2 (en) * | 2002-05-29 | 2005-07-12 | Koninklijke Philips Electronics N.V. | System and method for enhancing videos from drift-free scalable bitstream |
| US7212681B1 (en) | 2003-01-15 | 2007-05-01 | Cisco Technology, Inc. | Extension of two-dimensional variable length coding for image compression |
| US7194137B2 (en) * | 2003-05-16 | 2007-03-20 | Cisco Technology, Inc. | Variable length coding method and apparatus for video compression |
| US20130107938A9 (en) * | 2003-05-28 | 2013-05-02 | Chad Fogg | Method And Apparatus For Scalable Video Decoder Using An Enhancement Stream |
| US20050036555A1 (en) * | 2003-08-13 | 2005-02-17 | Lakshmanan Ramakrishnan | Automatic direct memory access engine |
| JP2007520918A (en) * | 2003-11-28 | 2007-07-26 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Method and apparatus for encoding or decoding a bitstream |
| US7889792B2 (en) * | 2003-12-24 | 2011-02-15 | Apple Inc. | Method and system for video encoding using a variable number of B frames |
| EP1709816A2 (en) * | 2004-01-16 | 2006-10-11 | Koninklijke Philips Electronics N.V. | Optimised requantisation for digital data distribution |
| US7869503B2 (en) | 2004-02-06 | 2011-01-11 | Apple Inc. | Rate and quality controller for H.264/AVC video coder and scene analyzer therefor |
| US7453938B2 (en) * | 2004-02-06 | 2008-11-18 | Apple Inc. | Target bitrate estimator, picture activity and buffer management in rate control for video coder |
| US7492820B2 (en) * | 2004-02-06 | 2009-02-17 | Apple Inc. | Rate control for video coder employing adaptive linear regression bits modeling |
| US7986731B2 (en) * | 2004-02-06 | 2011-07-26 | Apple Inc. | H.264/AVC coder incorporating rate and quality controller |
| CN1954614B (en) * | 2004-05-13 | 2011-06-08 | 皇家飞利浦电子股份有限公司 | Method and device for encoding blocks of values |
| US7492956B2 (en) * | 2004-08-18 | 2009-02-17 | Cisco Technology, Inc. | Video coding using multi-dimensional amplitude coding and 2-D non-zero/zero cluster position coding |
| US7499596B2 (en) | 2004-08-18 | 2009-03-03 | Cisco Technology, Inc. | Amplitude coding for clustered transform coefficients |
| US7471841B2 (en) * | 2004-06-15 | 2008-12-30 | Cisco Technology, Inc. | Adaptive breakpoint for hybrid variable length coding |
| US7454076B2 (en) * | 2004-06-15 | 2008-11-18 | Cisco Technology, Inc. | Hybrid variable length coding method for low bit rate video coding |
| US7471840B2 (en) * | 2004-08-18 | 2008-12-30 | Cisco Technology, Inc. | Two-dimensional variable length coding of runs of zero and non-zero transform coefficients for image compression |
| US7454073B2 (en) * | 2004-06-15 | 2008-11-18 | Cisco Technology, Inc. | Video compression using multiple variable length coding processes for multiple classes of transform coefficient blocks |
| US7499595B2 (en) * | 2004-08-18 | 2009-03-03 | Cisco Technology, Inc. | Joint amplitude and position coding for photographic image and video coding |
| US20050286629A1 (en) * | 2004-06-25 | 2005-12-29 | Adriana Dumitras | Coding of scene cuts in video sequences using non-reference frames |
| US7680349B2 (en) | 2004-08-18 | 2010-03-16 | Cisco Technology, Inc. | Variable length coding for clustered transform coefficients in video compression |
| US7620258B2 (en) * | 2004-08-18 | 2009-11-17 | Cisco Technology, Inc. | Extended amplitude coding for clustered transform coefficients |
| US20060193379A1 (en) * | 2005-02-25 | 2006-08-31 | Nokia Corporation | System and method for achieving inter-layer video quality scalability |
| US20060233255A1 (en) * | 2005-04-13 | 2006-10-19 | Nokia Corporation | Fine granularity scalability (FGS) coding efficiency enhancements |
| US20060260752A1 (en) * | 2005-05-23 | 2006-11-23 | Ward/Kraft | System and method for producing small production runs of products having removable coatings |
| US8446960B2 (en) * | 2006-01-13 | 2013-05-21 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Picture coding using adaptive color space transformation |
| US7242328B1 (en) * | 2006-02-03 | 2007-07-10 | Cisco Technology, Inc. | Variable length coding for sparse coefficients |
| JP4876122B2 (en) * | 2006-03-22 | 2012-02-15 | フラウンホッファー−ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ | Coding scheme that enables precision scalability |
| US20080043832A1 (en) * | 2006-08-16 | 2008-02-21 | Microsoft Corporation | Techniques for variable resolution encoding and decoding of digital video |
| US8773494B2 (en) | 2006-08-29 | 2014-07-08 | Microsoft Corporation | Techniques for managing visual compositions for a multimedia conference call |
| US8990305B2 (en) | 2006-10-18 | 2015-03-24 | Microsoft Corporation | Techniques for virtual conferencing servers |
| BRPI0720806B1 (en) * | 2007-01-18 | 2023-03-28 | Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. | SCALABLE QUALITY VIDEO DATA STREAMING |
| US8208745B2 (en) * | 2008-01-31 | 2012-06-26 | Analog Devices, Inc. | Spatial domain video enhancement/scaling system and method |
| KR101426272B1 (en) * | 2008-03-10 | 2014-08-07 | 삼성전자주식회사 | Apparatus of encoding image and apparatus of decoding image |
| ES2934052T3 (en) * | 2008-07-11 | 2023-02-16 | Fraunhofer Ges Forschung | Audio encoder and audio decoder |
| US9110849B2 (en) * | 2009-04-15 | 2015-08-18 | Qualcomm Incorporated | Computing even-sized discrete cosine transforms |
| US9069713B2 (en) * | 2009-06-05 | 2015-06-30 | Qualcomm Incorporated | 4X4 transform for media coding |
| US8762441B2 (en) * | 2009-06-05 | 2014-06-24 | Qualcomm Incorporated | 4X4 transform for media coding |
| US9118898B2 (en) | 2009-06-24 | 2015-08-25 | Qualcomm Incorporated | 8-point transform for media data coding |
| US8451904B2 (en) | 2009-06-24 | 2013-05-28 | Qualcomm Incorporated | 8-point transform for media data coding |
| US9081733B2 (en) * | 2009-06-24 | 2015-07-14 | Qualcomm Incorporated | 16-point transform for media data coding |
| US9075757B2 (en) * | 2009-06-24 | 2015-07-07 | Qualcomm Incorporated | 16-point transform for media data coding |
| US8675730B2 (en) * | 2009-07-13 | 2014-03-18 | Nvidia Corporation | Macroblock grouping in a destination video frame to improve video reconstruction performance |
| US20120236931A1 (en) * | 2010-12-23 | 2012-09-20 | Qualcomm Incorporated | Transform coefficient scan |
| US9490839B2 (en) | 2011-01-03 | 2016-11-08 | Qualcomm Incorporated | Variable length coding of video block coefficients |
| US9824066B2 (en) | 2011-01-10 | 2017-11-21 | Qualcomm Incorporated | 32-point transform for media data coding |
| US9516316B2 (en) | 2011-06-29 | 2016-12-06 | Qualcomm Incorporated | VLC coefficient coding for large chroma block |
| US9338456B2 (en) | 2011-07-11 | 2016-05-10 | Qualcomm Incorporated | Coding syntax elements using VLC codewords |
| JP5653328B2 (en) * | 2011-09-26 | 2015-01-14 | 株式会社東芝 | Image processing apparatus and image processing system |
| JP6141417B2 (en) * | 2012-06-29 | 2017-06-07 | インテル コーポレイション | System, method and computer program product for scalable video coding based on coefficient sampling |
| CN104272735B (en) * | 2013-01-16 | 2018-04-13 | 黑莓有限公司 | For the transform coefficients encoding of the context adaptive binary entropy coding of video |
| US10516898B2 (en) | 2013-10-10 | 2019-12-24 | Intel Corporation | Systems, methods, and computer program products for scalable video coding based on coefficient sampling |
| CN114222140A (en) * | 2016-05-28 | 2022-03-22 | 世宗大学校产学协力团 | Method for decoding video signal |
| JP2021174404A (en) * | 2020-04-28 | 2021-11-01 | 株式会社Preferred Networks | Data generation device, training device, and data generation method |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4179709A (en) * | 1978-01-10 | 1979-12-18 | Bell & Howell Company | Video information bandwidth compression |
| DE3369451D1 (en) * | 1982-10-14 | 1987-02-26 | British Telecomm | Image transmission |
| FR2575351B1 (en) * | 1984-12-21 | 1988-05-13 | Thomson Csf | ADAPTIVE METHOD OF ENCODING AND DECODING A SUITE OF IMAGES BY TRANSFORMATION, AND DEVICES FOR CARRYING OUT SAID METHOD |
| DE3728444A1 (en) * | 1987-08-26 | 1989-03-09 | Thomson Brandt Gmbh | METHOD AND CIRCUIT FOR IMPROVING THE RESOLUTION OF DIGITAL SIGNALS |
| FR2621194B1 (en) * | 1987-09-29 | 1989-12-29 | Labo Electronique Physique | DEVICE FOR CODING DIGITAL VIDEO SIGNALS |
| US4821119A (en) * | 1988-05-04 | 1989-04-11 | Bell Communications Research, Inc. | Method and apparatus for low bit-rate interframe video coding |
| US4984076A (en) * | 1988-07-27 | 1991-01-08 | Kabushiki Kaisha Toshiba | Image compression coding system |
| JPH02226886A (en) * | 1989-02-28 | 1990-09-10 | Sony Corp | Data transmitter |
| JPH0832039B2 (en) * | 1989-08-19 | 1996-03-27 | 日本ビクター株式会社 | Variable length coding method and apparatus thereof |
| US4969040A (en) * | 1989-10-26 | 1990-11-06 | Bell Communications Research, Inc. | Apparatus and method for differential sub-band coding of video signals |
| JP3085465B2 (en) * | 1989-10-31 | 2000-09-11 | オリンパス光学工業株式会社 | Image data encoding apparatus and encoding method |
| US5063608A (en) * | 1989-11-03 | 1991-11-05 | Datacube Inc. | Adaptive zonal coder |
| US5107345A (en) * | 1990-02-27 | 1992-04-21 | Qualcomm Incorporated | Adaptive block size image compression method and system |
| US5001561A (en) * | 1990-05-02 | 1991-03-19 | At&T Bell Laboratories | Embedded coding system for video signals |
| US4999705A (en) * | 1990-05-03 | 1991-03-12 | At&T Bell Laboratories | Three dimensional motion compensated video coding |
-
1992
- 1992-07-02 US US07/908,134 patent/US5253055A/en not_active Expired - Lifetime
-
1993
- 1993-03-17 US US08/032,582 patent/US5278646A/en not_active Expired - Lifetime
- 1993-05-25 CA CA002096878A patent/CA2096878C/en not_active Expired - Fee Related
- 1993-06-24 EP EP93304924A patent/EP0577327B1/en not_active Expired - Lifetime
- 1993-06-24 DE DE69320719T patent/DE69320719T2/en not_active Expired - Lifetime
- 1993-06-30 JP JP16065793A patent/JP3221638B2/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| EP0577327B1 (en) | 1998-09-02 |
| EP0577327A1 (en) | 1994-01-05 |
| US5278646A (en) | 1994-01-11 |
| JPH0698308A (en) | 1994-04-08 |
| DE69320719D1 (en) | 1998-10-08 |
| DE69320719T2 (en) | 1999-02-18 |
| US5253055A (en) | 1993-10-12 |
| JP3221638B2 (en) | 2001-10-22 |
| CA2096878A1 (en) | 1994-01-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2096878C (en) | Efficient frequency scalable video encoding with coefficient selection | |
| Ghanbari | Standard codecs: Image compression to advanced video coding | |
| KR950006772B1 (en) | Compressed video data generating method | |
| EP0562420B1 (en) | Method and apparatus for vector coding of video transform coefficients | |
| US6650784B2 (en) | Lossless intraframe encoding using Golomb-Rice | |
| Richardson | H. 264 and MPEG-4 video compression: video coding for next-generation multimedia | |
| CA2214663C (en) | Fixed or adaptive deinterleaved transform coding for image coding and intra coding of video | |
| US5623308A (en) | Multiple resolution, multi-stream video system using a single standard coder | |
| KR100969645B1 (en) | Adaptive variable length coding | |
| US7486211B2 (en) | Method and system for entropy coding | |
| US4821119A (en) | Method and apparatus for low bit-rate interframe video coding | |
| EP1407603B1 (en) | Dct compression using golomb-rice coding | |
| EP0926897B1 (en) | Video signal processing for electronic watermarking | |
| US20150249836A1 (en) | Method, medium, and apparatus encoding and/or decoding an image using the same coding mode across components | |
| EP1538566A2 (en) | Method and apparatus for scalable video encoding and decoding | |
| US8472734B2 (en) | Method and apparatus for encoding and decoding image | |
| HUE030338T2 (en) | Indication and activation of parameter sets for video coding | |
| EP0847565A1 (en) | Method and apparatus for scanning of transform coefficients | |
| EP0736843B1 (en) | A motion video compression system with adaptive quantisation | |
| Li et al. | Rate-distortion optimized embedding | |
| US20070230811A1 (en) | Method of enhancing entropy-coding efficiency, video encoder and video decoder thereof | |
| Perra et al. | An image browsing application based on JPEG XR | |
| Shahid et al. | Fast protection of H. 264/AVC by selective encryption of CABAC for I & P frames | |
| Van Assche et al. | Evaluation of lossless compression techniques for high-resolution RGB and CMYK color images | |
| Cavigioli | Image Compression: Spelling out the options |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed | ||
| MKLA | Lapsed |
Effective date: 20090525 |