CA2125868A1 - Method and apparatus for compression of images - Google Patents

Abstract

2125868 9312610 PCTABS00023 An image compression method based on symbol matching is disclosed. Precompression of the image is performed prior to symbol matching to improve efficiency. A voting scheme is used in conjuction with a plurality of novel similarity tests to improve symbol matching accuracy. A template composition scheme achieves image enhancement. Other disclosed features provide further advantages.
Apparatus for implementing the image compression method for image transmission, storage, and enhancement are included.

Description

2i2~8~8 WO ~3/12610 ~ - - PCr/US92/10172 t~) Method and Apparatus for Compression of Ima~es Background of the Invention This invention relates to compression o~ images. in particular~ to compression methods -emploving svmbol matching;
, standard 8.5 by 11 inch page, when scanned as a binary imagé at 300 dots per inch (DPI). comprises over 8,000,000 bits of information--about a megabyte. The same page.
when scanned at 200 DPI, the scan rate of fax fine mode, comprises almost hal~ a megab~ te of in~ormation. Because of the large capacity needed to store such images~ some sort of image compression is typically implemented in image storage and transmission de~ices. ~
; One current standard rnethod for binarv image compression, in widespread use as the standard for facsimile transrr~ission, is the CCITT-3 standard. This is a lossless binar~-image compression al~or~thm based on Huffman encoding of a run-len~th encoding of the image. The CCITT-3 standard typlcallv compresses an image by an order of magnit~lde by ~taking into account first-order statistical regularities in the lengths of runs. Thus, a 300 ~DPI imaBe may be compressed from a megabyte to 100 kilobytes.
Symbol matchlng can be used to incrcase compression ratios for binary images which co~sist~primarily o~ machine-pnnted text. Such text is character~ed by many small regions S~ol connected~black~ pi~cels, reprcsenting; characters, surrou~ded by white pixels. A typical document~might ~use only a few hundred different characters representing the letters of the lphabet~in uppcr-~and lowercase in a few different type faces (e.g., Times and Helvetica~
and type styles ~e.g., plain,;bold, italics). In compression techniques employing symbol matching,~similar~characters~ar`e grouped, and their description as an array of black pixels is rëpresented~only once as~a template. The entire binary image is rcpreseIIted as a sequence ol tempiatés~long~.~vith references to their positions ~wit4in the image.~ -In this way, the r~ rcse~ion~coïituns far ~ewer bits than the onginal image. The amount o~ compression that can be~ac~ievcd using ~ymbol matching depends on~how many characters are matched, how tho~ templates~ are represented, ar~d how the character~positions are represented.
Jnfortunatcly, characters that~ appear similar to the human eye are not usually pixel-identical. The printing method, subsequent photocopyingj and, most importantly, the scanning process itsel~ introduce error into ~the original image~ An example is illustrated in Fig. 1,~which depicts two instances o~ the letter "b" in 12 point Courier scanne(l at 200 DPI. The error pixels are also shown~ In order for a symbol matching scheme to significantly contribute to binary image compression, error between two characters must be , ' ? 1~ ~; 4 ( ~
S~8 g~ .Rec'd p~.T/PTD 14 DfC J9g~
allowed, but bounded. Some notion of the matching of two characters based on perceptu;~i sin~ilarity is needed for this purpose. Furthermore, sincc the ma~ching characters are ~
all identical. a way of defilling a template &om the matching character~ is required. Sir.ce such a template is necessarilv not identical to all of the characters uscd to define it. t;r~e representation b~r templates does not encode the image perfectly. But since the characters var~ onl- sli~htlv in appearance from the template. the tifEerence is not perccptible.
The success of symbol matchins in binarv imase compressivn hinges cruciallv on h~
similaritv of ewO chalracters is judged (how characters are matched). .~ matchin~ metr~c allowing large pixel differences between characters is prone to-s-ubstitution errors. F~r example, as illustratet in Fig. 2, an ins~ance of a ;'b" and an instance of an h` in the same font and size catl have a relatively small pi~cel error. lf they are matched. the reconstructed image will be ridtled with error~. On the other hand, if the matchin~ metric is too stringent, then multiple in~tances of thc same character ~Irill not be matched and little compres~ion will be achicved.
'rhe u~e of symbol matching in binary image compressio~ dates back to at lea~t 19 (see "A means for achieving a high de~ree of compaction on ~can~igitizet-printed te~t,' by R.N. Accher and G. I!Jagy, in IEEE ~ranJ~ct~on~ on Computcrl, 23tll):1174-1179.
Nov. 197~). Since then, a ~ariety of compre~ion sy~tem~ emploji~ ~ymbol matching have bcen proposet. ~See Combir~ed symbol matching facsimile dsta compre~ion s~ stem. ' ht-W.K. Pratt ee al., in Procc~dingJ of thc IEEE, 68(7j:786-796, 3111y 1980, and U.S. Patent No~. ~ 4,091,g24, ~,2~8,782, 4,410,9I6, 4,~63,386, ~,499,499, and 4,606,069.) The present invention i~ a compre~sion method employing a substantially impro~ed matching process.
as wdl a~ other impro~red featurc~ whicb increaJe the compre~sion ratio.

Summary of the Imrc~tioa In general, the i~e~gtO~ cludes a ~nety of impro~ed feature~ for methot~ for com-:pressing im~er ~ ich inelud the hs~ic step3 of identifying ~ymbol~ i~ the imagc, asso-ciatin~ ith ~: itentifial ~bol a~ indication of itr loe-tion i~ t~e ima~se, comparins each identiSe~ ol to templ~t from a ~ibr~ry of te~platff to match each identified ymbol to ~ templ-te, u~d tesl~r-tin!s a compre~ed repre~ent~Sio~ of the image includin~
a repre~entation of eu:h te~plate, and a rcferencc to a loc~tioR ~d to a template ~or each symbol in the imagc. Thc im~g~ will typically start a~ biny pi~el-array images. but the methot~ of the i~e~tios~ are also suitable ~or compre~6 ima8 originati~ in other format~
In one a5peet7 the iQ~re~tio~ features, as a fir~t ~tcp, presompre3si~g the ima~e. For ,, SlJBSnTUTE SHEEl-.
,

2 1 .~ 6 ~ PCI/US92/10172 example, for images that start as binary pixel-array images, run-length encoding ma- be used. When run-length encoding is used in the precompression step, the symbols may be identified by associating runs of black pixels in adjacent rows which have at least one black pixel in the same column or in the sarne or adjacent columns.
In another aspect, the invention features generating the library from scratch during image compression bv stori 1g in the librarv each symbol for which no matching template is found as a new template, and defining each template bv a representation determined bv a plurality of the symbols which match it. For images that start as binary pixel-array images~
each template may be defined bv a uni~ue spatial distribution of black pixels determined by the distributions of black pixels of a plurality of the symbols which match the template.
The distribution may be determined, for example, by averaging the distributions of black pixels of the symbols or from starting with one svmbol which matches the template a~d chan~ing a pixel in this svmbol only if a predetermined fraction of the pixels in the same location In the other symbols matching the template disagree with it.
In another aspect the comparing step features identifying error locations, perforrning a plurality of tests on the error locations and on other global features of the symbol and the template, and declaring a match between a symbol and a template only if all of at least one ol a~ set of predetermined~ groups of ~tests are passed. For images that start as binar-pixel-array images, the error locations ~may be error pixel locations which have a black pixel in~only~the~symb:ol~or the template, but not both. The error pixel locations may be run-length encoded. One appropriate test is if the distance between each error pixcl location and~the~nearest black pixel~in~the template or symbol having a white pixel at the error pixel location is less than a predetermined value. Another test is if all contiguous groups of èrror~pixel`~locations are smaller than predetermined sizes and shapes. Yet another test is if the différènce~in~ha6hts bdween~a template and a symbol is less than a predetermined alue,~and~siII~ilarly~for the~ fference in widths. Yet another test is if the ratio of the number~ of ~ bl~c~ pixds in the symbol to thé number of black pixels in the tcmpl~te is within a predetermined ra~l~e. Yet another test is if the ratio of the number of error pi~cel I ocations~to thc minimum of the number of blaclc pixels in the template and the number of black~ pi~cels in the symbol~is~less~than a predeterrnined value. Yet another test is if the ratio o f the number of error pixel locations to the number of black pixels adjacent to a white pixcl in the template is less than a predeterrnined value. Yet another test is if all pieces~of~evidence for a shift~of the symbol relatiYe to the template are consistent, where evi~dence is given by local configurations of error pixels and their adjacent pixels. A novel feature of this invention is the~use of tests (such as the above) based on the sizes, shapes.
and orientations of local configurations of error pixels.
-:

WO 93/12610 2 1 2 S ~ ~ ~ PCI /US92/10172 In another aspect, the invention features representing each template by the boundar-of the spatial distibution of black pixels for the template. In a related aspect, the repre-sentation of each template includes a re~erence to a frame template for defining an outer boundary of the template and a list of hole templates with associated positions within the frame template for defining any inner boundaries of the template. The frame templates and hole templates ma- be represented bv representations of the boundary of the frame templates and hole templates.
In another aspect, the invention features indexing the templates in the library accord-ing to template features. The step of comparing proceeds according to the index. In this way, lhose templates more likely, based on features~ to match a symbol are compared ~o the symbol before templates less likely to match and templates whose features preclude a match are not considered.
In another aspect, the invention ~features reducing the resolution of the template rep-resentations to achieve further compression.
In preferred embodiments, the compressed representation of the ima~e includes a template stream, a template reference stream, and row and column position streams.
Other~ divis;ons of the information into data streams are possible. Preferably at least one of these~streams is~further compressed.
ln ~prekrred embodiments, any template including a region smaller than a predeter-mined~ size~ between two regions làrger than another pretetermuned size is split into more than~one~template. Furthcr,~each template formed by a split is pre~erably compared to other~templates~and mer~ed with any other template it matches. In preferred embodi-ments, symbols~smaller than a predetermined size are eliminated.
In~prefé~rred embodiments, ~the reference to a location for each symbol in the binar~r image is represented by the difference between the actual offset between the given symbol and à~prcceding~'symbol and~ a standard offset for the symbol pair defined by the given symbol an~d'the precetin~ symbol. The stantard oÆset may be the offset last scen ~or the symboi pair or~the~most ~requently seen~o~set for tbe symbol pair. The'difference between the standard~offset and the'actual offset may be determined at a lowcr resolution than the resolution~of the binary imase. ~ ~ ~
In preferred embodiments, an appro~cimation of the reading order of the symbols is dctermined ~and the compressed representation of the binary image is generated according to the reading order. The reading order may be determined by locating symbols on a single line- and~sorting the symbols. The symbols occupying a line may be detetmined by using a beam to search along the line, the size of the beam ~rarying according to the sizes of one or more of the~ adjacent symbois. ~'ertical bounds on the size of the beam may be imposed.
~,, :: -:

WO 93/12610 21 2 5 8 6 8 PCr/US92/10172 In another aspect, the invention is an image transmission svstem (such as a facsimile svstem), an image enhancement svstem, or an image storage svstem operating according to the image compression methods described above.
Compression according to the invention can achieve, for example, compression ratios greater than 25 times that of CCITT-3 for an 8-page 300 DPI document. Each page is thus compressed from a megabvte of raw image to ~ kilobvtes by the method of the invention. It ;s important to note that the compression ratio will increase with the length of the document since a single template library is used throughout. Thus, the invention has particular advantages for image transmission or storage systems dealing with large binary images.

Brief Description of the Drawing Fig. 1 is a diagram illustrating how t~-o similar characters may not be pixel-identical.
Fig. 2 is a diagram illustrating how two distinct characters may have onlv small pixel differences.
Fig. 3 is a block diagram of an image storage system employing image compressionaccording to the method of the inYention.
Fig. 4 is a block diagram of an image transmission system employing image compres-sion according to the method of ~the invention.
Fig. ~ is a~flow chart ~summarizing the steps performed by an image compression system according to the invention.
Fig.~ is a dia~ram demonstrating a method of segmenting a run-length encoded inarv lmage into symbols.
Fig. 7 is a diagram illustrating configurations of error pixels used to infer shifts for the ;shift~ test.
Fig.~8 is a diagram illustratin~ the impro~ement in symbol quality that can be achieve~ by composition of symbols;into templates.
~ .
Fig. ~ is a diagram demonstrating a method of representing a symbol by tracing.the edges of a pi~cd array image.
; - Fig. 10 is a diagram demonstrating a method of representing a symbol as a rame and at least one hole.
Fig. 11 is a diagram illustrating how a skew in the scanning process can make symbols appear on the same line even when thev are not.
Fig. 12 is a diagram demonstrating a method for using a beam to scan for svmbolsin the same line to the left or nght of a current symbol.

, ~

W0 93~12610 21 2 3 ~ ~ 8 PCI /US92/10172 Description of the Preferred Embodiment .~ svstem for image storage using the compression method of the invention is illus~rated in Fig. 3. A processor 12 with storage memory 14 forms the heart of the svstem 10. As showm the processor 12 includes at least one interface lff for accepting representations of irnages. This input interface 1~ may be, for example, a connection to a computer which generates images, or a connection to a scanning device used to capture a representation of an image from a hardcopy of the image. The processor 12 performs the steps of the image compression method of ;the invention, discussed further below. The compressed images are stored in the storage memory 14. The processor 12 also decompresses the stored images~and provides reconstructed representations to at least one interface 18. The output interface 18 may be, for example, a connection to a computer used to process and/or display the image, or ~a connection to a printer for providing a hardcopv of the image. Preferably, the output intèrface 18 can display both images received directly from the input interface 10 and reconstructed lmages called~ up from storage memorv 14. For reasons discusscd below, the storage system illustrated in Fig~ 3 can also be considered an image~enhancement system~when the image compression system of the invention is emb~odied:~in~processor 12. ~
A~ system~for~imap~transmiss~on~using the compression method of the invention isillustrated~in~Fig.~4~ A typical example o{ an image transmission system is a conventional facsiIï~ile-tr~nsm~ssion system. ~At ~the transmitting cnd, the system 20 inclutes a processor ~22~with ~mem~ory~ which ~?ccepts~ represcntations of images via at ~least one interface 24. In a~facsimilé~system,~the~input;~interface 24 is typically a connection to a scanning device used to capture a~representation of ~an image from a hàrdcopy of the image. The processor 22~ compresses thc~input image according to the method o~ the invention, discussed further below. The~processo;r 22 further mcludes an interface 2~ for transmitting the compressed im~6e.~-In~ a;facliimile~ system,~ the output interface 2~ is typic~lly a connection to a tele-phonc~line-Yia~a~motem. At ~he~receiYing end, another processor 28 with memory includes an~interface30~or~accepting~the~transmittedcompressedrepresentationoftheimage. In a~ facsimile~-s=ystem, the iDpUt~interace 30 is again typically a;;connection to a telephone line.~ Thc ~procN60r 28 decompresses the input compressed image accorting to the method of the invention, ant provides~ a reconstructed representation of the image to at least one inte~ce 32. In~ a ~facsimile system, the output interface 32 is typically a connection to a pnnter. ~
An~ image~ compression system according to the invention accepts a representation of an image~and segments the image into symbols. These symbols may represent characters .-, :~ :

: :

WO 93/12610 2 ~ 2 S ~ ~ 8 PCI/US92/10172 (e.g., letters~ numbers, etc.) or graphics (e.g., logos, charts~ etc.) Each identified svmbo! is matched against a library of templa~es. In the preferred embodiment. the template librar-is generated from scratch as new svmbols are identified. E;ach identified svmbol is matched against templates created so far, and it is added to a matching template group if one exists.
If it matches none of them, it forms a new template group and serves as the template ~or that group so that later symbols can be matched to it. Once all of the symbols are grouped into template groups. a representation of each template is composed, ~referablv based on features of many, if not all, of the members of the template group. Also, a representation o f the location (typically by row and column) of each symbol in the image and the svmbol s template group is built. Thus, typically {our streams of information--a template stream, a template code stream, a row position stream, and a column position stream--constitute the compress`ed representation~of the image. Other divisions of the information into streams i s~also possible. The streams of information are each preferablv further compresscd.
Decompression of an image compressed according to the methods of the invention is stràishtforward. First, the streams of data are decompressed if the- were further com-pressed~ as a final step of the compression process. Then the template librarv is read from the~template~stream. Next, as the templat~e code stream is read, the symbol position for each symbol in~the tempiate code~ stream is ~determined from the row and column positions streams,~and a represcntation of the template is overlayed at the deterrnined position.
i~- 5~ A~ fkrwchart ol the steps~for image compression according to the preferred embodiment of the ~in cntion~is~provided~in Fig.~ 5. ~ The ~steps of the method will no~r be takcn up in turn~
PrècomDression of the Imal~e ~ ~
ln~typlcal~ lmplementations o image compression systems, images are provided as bi-nary pixél atrys,~ with ~connected regions o{ black pixels representing characters, graphics~
etc. The pi~cel ~array ~may be~ pr~vided,~for -~ample, by a computer storing the image in such`~a form,~or by~a scanmn~dé*ce~w};lch converts a~hardcopy of a binary image into a pixel~ array. ~
Prior~biDary image compressioD svstems, such as those referenced above, operate di-rectlv OD the~pi~cel array. ;For a 300 DPI, 8.5 by 11 inch binary image, a megabyte of memory is required to store the pixel array, and millions of operations may be required to~scan this memory. To redùce this burden, according to the invention, the pixel array is precompressed before the symbol matching process is com nenced. This saves significant amounts of storage space as well~ as valuable processing time. The step of prccompres-sion makès the entire compression process more efficient in space and time, an important advantage for commercial products.

WO 93/12610 21 2 5 8 ~ ~ PCI /US97/10172 A standard technique for compressing a pixel arra~r representation of a binarv image is to run-length encode it. This turns out to be a very suitable wav to precompress a pi.Yel array for the purposes of the invention. ~ technique for segmenting.a run-length encoded representation of a binarv image into s~ mbols is presented in the next sectlon.~ ariants of run-length encoding ma~ be used to precompress the binar~ image. For example. a pixei arrav representation mav be compressed to a y-axis representation. Ac-cording to this technique each non-empt~ rou in the image ~of constant y, hence the name) is encoded as a list containing~ in order, the row index, and the indices of the first ;~ coordinate at uhich a black pixel region is encountered, the ;~ coordinate at whi~h the region is exited, the second ~ coordinate at which a black pixel region is entered, the coordinaie at uhich the region is e~cited, and so forth. This representation can be efficientl-generated by a raster scanning process. and is quite convenient for further compressionbt svmbol matching. It has the advantage of being a very compact representation for images with a lot of white;space occurring m horizontal bands, e.g., non-skewed images of double-spaced te~ct. ~For further information~on y-a~cis representations see "Representations of contours and regions for efficient~ computer search," by R.D. ~Ierrill, in Communication~
of the ACA~, 16(2):69-82, Februar~ 1973 1 Hereinbelow, the term "run-length encoding-' will refer to anj variant of run-length encoding.
Other~precompression techniques are also appropriate. One such technique invol~es representing a~pixel array as~a~quadtree. A quadtree encodes binary images hierarchicall~-.
A~ square~image~is di~rided into~four subregions corresponding to the northwest, northeast, southwest,~and~southeast quadrants.~If all of the pi~cels in a quadrant are black, the cor~
respon~ding~tree~node is recorded as black; if all pixe}s are white, the node is recorded as ;white. Only if the quadrant- contains~both black and white pi~cels is further subdivision necessar~ ln~this~cas;e,; the~quadrant is recorded as gray, and is further subdivided recur-sivelv.~This representdion is~-space-e~cient for images with large continuous arcas of black or~white,~such~ te~ct images. (For further information on quattrees, see "The quadtree and~ related hiérarchical data s~ructures," bv H. Samet, in Computcr Survey~, 16:187-260.

Yet; another compression ;technique involves the use of chain codes to efficiently rep-resent boundaries. A binary image can be reprcsented as a set of chain codes for each black~ pixel region in the image. ;The~ chain code for a region consists of a starting point and a specification of a series of unit moves along thè 4 or 8 compass directions that trace out the border. First derivative of direction can be used to further compress the representation for sufficiently smooth regions. Since the chain code representation picks out connected regions, it is especiallY appropriate as an image representation on uhich t~

:

::

WO 93/12610 2 ~ ~ 3 ~ 6 ~ PCl/US92/10172 :
perform symbol matching. The chain code representation can be used to calculaSe the area of a region~ a procedure which is important in symbol matching. Furthermore. chain code representations of regions are particularlv efficient for merging. Thus. a mixed grid file ( see below) and chain code initial representation can be efficientlv converted to a pure chain code representation. (For further information on chain codes~ see "Computer processing of line drauing images," b~ H. Freeman, in Computer Surveyl, 6(1)~ 98, ~larch 19/~.) Grid files can be used to exploit the advantages of more than one,compression svstem .~ grid file is made by breahng the binarv image into a regular or non-regular rectangular grid in ~ and y. Each rectangular section is then encoded by some other mechanism such as quadtrees or chain codes.` The gri(l can be chosen so as to maximize the efficienc~ ~Ir the coding for different regions ;of the binary image. ~For further information on grid files~
see "The grid file: An adaptible, symmetric~ multikey file structure," by J. Nievergelt~et al., in .~CAf Tran~action~ on Databa~e Sylteml, 9:38-71, 198~.) Note that the step of precompressing the binary image from a pixel arrav represen-tation to a precompressed;rep~resentation is not required in cases where the provider o~
the binary image has already performed such a compression, or where the provider of the binary image yields a compressed representation automatically ~with no pixel array as an intermediate step). Furthermore, the ma be cases where the binary image is provided in ~a representation compressed ~relative to a pixel array representation, but where conver-sion ~to~anot~her~compressed representation; woult be preferable for the purposes of symbol ~matchlng. All~ such variants are~ wlthin the scope of the invention.
; Se~mentatlon of the Ima~e into SYmbols ft^er the~binary image;is precompressed from~ a pixel array representation to a precom-pressed~répresentatlon,~the Image is segmented into symbols. According to the invention, evèrtr connected~region ol black pi~cels is Identified as a symbol. Therefore, in addition to 1 etters-'^and numbers, grsphics'such' as logos and charts will also be considercd symbols.
This i;s iD contra~t to all prior~approaches. For~example, U.S. Patent No. 4,41Q,916 dis-closes~a'oegmenting method~wherem~ the binary image is segmcnted into symbols ~e.g., characters)~andnon-symbols~(e.g., graphics).
BV way of e~ nple, an efflcient way of ~segmenting a run-length encoded representation of a binary ima~e will be presentcd. The~method works by associating with each black run an identifier for thc character it is a part of.
;In the~preferred embodimeni ~of the se~menting method. each black run in the binary i mage is operated on in top-to-bottom, left-to-right order. For each run. the resion of the pre~rious row directly above the current run is examined, and all of the runs that overlap that region are collected. Each of these runs will have alreadv been assigned to a symbol :

'' :: :
'~' WO ~3/12610 2 ~ ~ 5 ~ ~ ~ PCr/US92/l0172 since they appear before the current run in the top-to-bottorIL order. The current run is added to the symbol associatecl with the leftmost such overlapping run~ and all of the runs associated with the other overlapping runs are reassociated with that svmbol. The reassociation can be done bv a single pointer operation if the associations are appropriatel~
represented.
Two snapshots of the segmenting process are illustrated in Fig. 6. .~fter segmentin~
two rows o~ an image, runs have been assigned to three different svmbols (identified ~ith the labels "a", "b", and "c"). As the runs of the third row are processed. each is given the label of a run above it. In the case of the last run, which has two runs direct1v abo~e i~.
the label of the first such run is used. Later runs (those associated with the label "b in this example) are reassociated with the label of the current run ("c").
Often, small groups of just a few pixels will be introduced in a binary image as a reslllt of scanning or printing error. These characters can disrupt the compression process~ par-ticularly the final compression of the representation of template codes and corresponding svmbol locations, discussed further belou. According to a preferred embodiment of the invention, all symbols that are smaller than a certain threshold (four. sav) are eliminated during the symbol identification process.
Svmbol Matchin,~
In prior methods of symbol matchin& for binary image compression, nu-merical results derived from one or more tests are combined in a weighted manner and the result is thresholded to detern~ine whether or not a symbol matches a template. According to the i nvention, an alternate technique is employed. which has been shown to improve matching accuracy, and therefore, to reduce errors. The method of the invention is a voting scheme.
where a plurality of tests are used for each symbol!temE~late pair. Each test which is passed (indicating similarity between the symbol and template) contributes one vote to a sum of votes. In order for a match to be declared, a predetermined number of votes must be received. More generally, the tests can be groupet into one or more predetermined groups~ and~all tests ~vithin at least-one o~ the Rroups must be passed for a match to be declared. In one embodiment, the vote must be unanimous for a match to be declared.
:
~ or most applications, a good symbol matching process for the purposes of binar~
image compression must take into account the error introduced by scanning. This error can be presumed to be primarily a quantization error due to a shi~t in registration of the characters relative to the scanning grid. Follovr,ing from this, error is expected to occur only at the edges of the symbols (the juncture of black and white pixels). Howe-er, since quantization error is associated with different placements of the same s~mbol relative to t he scanning grid, the overall number of black pixels is expected to be roughlv constant ~or WO 93/12610 2 :12. 5 ~ ~ g PCT/US92/10172 two similar s~rmbols. Furthermore, the number of error pixels is expected to be small for two similar svmbols. Finallv, any evidence for the presumed shift of a character relati~e to the scanning grid should be consistent across the entire s~mbol. These observations suggest the following similaritv tests for detern~ining a match between a s~mbol and a template.
Edge test 1: .~11 error pixels must be within a fixed distance dedg~ of an edge.Edge test 2: No bloclt of error pixels may be larger than dn bV dn square.
Edge test 3: The difference in heights of the symbol and the témplate mnst be les~
than or equal to a fixed bound dJiZc~ and similarly for the widths.
Density test: The ratio of the number of black pixels in the s~mbol to the number of black pixels in the template must be within the range 1 ~t dd~n~ity Error test 1: The ratio of error pixels to the minimum (or equi~alently, maximurh of the number of black pixels in the template and the number of black pixels in the s~mbol must be less than a bound ddiJtortion Error test 2: The ratio of error pixels to black pixels occurrjng on an edge (i.e.
adjacent to a white pixel) in the template or in the svmbol must be less than a bound ded9c di~tortion-- ~ ~ Shift test: All inferred shifts of the symbol relative to the template must be consis-tent. ~ ~ ~
Other ~tests ~are~ also possible.
Edge test~ 1 may be implemented, for example, by checking whether the distance betwen- each ;error pixel location and the nearest black pixel in the template or svmbol having~a~white~pixel at the error pixel location is less than a predetermined value. The inverse~ test~ could also be used, that~ is, the distance between each error pixel location and the nearèst white pixel in the template or symbol having a black pixel at the error pixel location~ could be~checked. ~ ~
T}~e~shift tcst requires a method for inferring the presumcd shih of the symbol relative to~ the te~nplate.~One mcthod for infernng shift is to examine groups of error pixels of the same~source,~that is, whicb are all black pixels in the template or all black pixels in the symbol.~ The overall map of error pixe} locations can be divided into two separate maps of (1) error pi%els that are black in ~he template (the template error map), and (2) error pixels that are black in the symbol (the symbol error map). Contiguous groups of pixels in hese various error maps can be presumed to have arisen from a shift of the symbol relative : ~ :
to the template. For instance, a vertical column of two error pixels in the template error map that are adjacent and to the right of black pixels in the template mav be tlue to the symbol having shifted awa~v from the error pixels toward the adjacent pixels. An example ~;
., , WO 93/12610 2 1 2 ~ 8 ~ 8 PCI/US92/10172 of this configuration is'illustrated in Fig. 7a.
.~nother pattern of error pixels that can be used to infer shifts is a triangle of three contiguous error pixels of the same source. For example~ if the three contiguous error pixels of such a configuration are found in the template error map, and the pixel location that falls within the two-b~,-two square that bounds the three error pixels is black in the template, a shift can be inferred aUaJ from the error pixels toward the adjacent pixel.
If~ on the other hand, the adjacent pixel is white in the template, a shift awav from the adjacent pixel can be inerred. Converse inferred shifts can be inerred from such triangles of error pixels in the symbol error map. E;xamples of these configurations are illustrated in Fig. 7b-e.
Other sizes~ shapes, and orientations of local sets of error pixels from the various error maps can be used to infer shifts~ for the purpose of comparing consistency of shift. Th~
triangle-based test for infernng shifts is used in the preferred embodiment of this method.
The deterinination of whether~ all shifts are deemed consistent can be done by requirin~
that all of the inferred shifts ;are~ldentlcal, or by weaker restrictions, such as requiring that a certain percentage be identical, or ~that no pair be in opposite directions, or that all be ;within a certain angular r'ange of each other. or combinations of these tests. In the preférred~embodiment, Idéntlty of all inferred ~shifts ~s required.
Note~tha t Edge tests 1 and 2~and Shi~t test look specifically at local configurations of ~pi~ s ~in the error maps (t~he~overall, the template, and the symbol maps). The use of catcgoricàl~tests~based on size~, shape, orientation, and relation to adjaccnt pixels of local sets of error~pixels~ls~novel to~this method.~ The particular tests dcscribed here are used in the~p`ref~er~rcd embodiment, but all such~tests~are includcd within thc scopc of the invention.
In order~'to apply~thcse~tests to~a symbd/template pair, the symbol and templatemust~be~regidered to each other, for example, by aligning two corners. Registrations that diffcr~by~one~or~:~more~than onc pixel~are preferably also tested, rcquiring a plurality of potential~regidrations to~ be~:consideret.~
~ e aariolls~;bounts specified in~ each test need not be fi.sed. Preferably they are varied ; as ~a function of ~the~ size of the ~symbols.~ This is because humans pcrcei~re symbol differences as~bei~ng~sa~ient differentially ;with symbol size~ In a small {ont, for instance, the difEerence of only a few perccnt in the number of pixels is significant7 whereas at larger font sizes, that same~ percentage may not bc perceived as significant. The ability to use different bounds ;at diffcrent~sizes can~allow more~aggressive matching at larger sizes without sacrificing distinctions at the smaller sizes.
Bv waY~of example, approprlate values o~ the bounds for the above tests are d~dg~
dn = 2, d~iz~ = 2, dd~nJity = .50~ ddi~tortion = .20 (if fewer than 150 pi~cels) = .40 (if greater , 2~2~i8 ~TIUSg 2/ 1 o 1 7 ~
- -~ than 200 pixels) and interpolated otherwise. and d~a9t dl~tor~on = ~1 1 4 DfC 1993 RePresentation of the Temp!ates In many prior mcthods of binarv imase cornpression using svmbol matchins. :~e first stmbol encountered in each template ~roup ser-es as the exemplar for that ~ro~lp .~ccording to the invention, rather than throw a- av the information about the appearanc.o vf all of the other s~ ols that matched the first one. se~eral or all of the s-mbols are ~ce~i to compose the template rcpresentation. In this uav a template can be compos~d thar appears ~better' than any of the svmbols &om v.hich it was formed. The reconstructe~
binary image can actually be an improved imase because similir~ity among the matchec characters in the binary image is providing furthcr information about the shapes o~ the srmbols. ~rnlike smoothing techniques~ this method reduce~ high frequency ~rror ~itholl, sacrificing high frequency information in general. Because of this property, the method can be used as an image enhancement technique, as well a~ an ima~e compressio~ m~thod.
- An example of the enhanccment that can be expected is illu~trated in Fig. B, where a ma~snified image of the word "here" in its original form in a digitized image, and a magnifiet image of the ~ame word in the reconstructed ima~e, zre presente~ ote the improved quality of the characters~ especially the character rn, which wa achieYed b-composition of a plurality of symbols into template~.
One way this '~avera~ng" proce~ can be carried out is timply to keep a pixel black in s he tcmplate only if th~ majonty of symbols for~Tsng the template ha~rc that pixel black To preYent artif3ct~ of the averaging proces~, which can occur when a small number o~
symbol~ are aYeraged, an ~Iternative compo~ition method i~ to wc the appearance of one of the symbol~, the fir~t ~ymbol encountered, ~y, but to allow the other symbol~ to ~,ote on chan~ins any gi~ren pixel. The vote will succee~ if morc than a 3pecified fraction.
two-thirds, s~y, of the~ other ~ym~ols di~a~ree ~rith the original 5yml)01. For templates composed of large number~ of ~ymbols, the effcct is mucll the sam~ a~ a~eragin~, but ~; for~ t~mplates with small ~ m~eN of ~ymbol~, a greater wdght is Rive~ to onc symbol.
~climinatin~ a~i~ a~ Other pts~eedure~ for tesnplate compodtion sre possible ; whicl~ eS~e ul~4~ di~cussed aboYe.
Tempbte compitio~ my be performed at m~ny ~tage~ turin~ the compres3ion pro-ce~s. ~Accordi~ to one ~o~ment, the "a~reragin~ performed a~ an ongoi~g process a~ symbob are matched to templates in the library of template~ tbi~ mode, the rey-rcsentation~ of the templ~te~ in the library continu~lly chulg~ ymbol~ are matched to the templatc 8roUp3. T~is can ultimately decrease the number of templates ~tored in the library ~i~ce template~ srstu~lly come to re~emble the a~rera~se feD~ture~ of the ~ymbols.
re5ul~in~ in more frequent matches.
, .

: . .
SVBSTITUTE SHEET

.

2~S~
WO93~12610 PCI/US92/10172 .~ccordin~to another embodiment, the "averaging is performed onlv after all s~ mL)ols have been placed in template groups. In this mode the first encountered svmbol (t~ picall~ ) of ant template group is used in the matching process. The averaging that occurs at the end of the process serves to provide an improved representation oi the characteris~ics ,r the s~mbvls in the template group.
Note that in either of the above modes of operation. after the svmbols have beengrouped into templates~ two templates mav be merged if the representation of the t~-.
templates match under the above criterion. Because of the averaging éffect o~ the template composition, t~ho templates ma~ match even though the s~mbols that were matched uith the t~vo templates did not. Template merging can further decrease the number of templates that must be represented. According to one embodiment. templates carl be generated for each page individually by matchin~, and then the templates from the several pages merged.
Di~erent matching criteria mav be used ior the first matching process within the page. and the second merging process across pages.
.~s symbols in a binary image are processed. a library of templates is developed b-augmenting an initial library with new templates as they are found. The initial librarv may include prestored symbols (say those of especially frequent usage in a particular ap-plication) or may be empty. If~ prestored symbols are employed, it may not be necessar-to augment ~the library. Furthermore, it may not be necessary to incorporate the repre-sentations~of thc templates corresponding to the prestored symbols into the compressed representation of the ima~e, since these template representations may also be prestored in the apparatus performing the decompression (which may or may not be the same as the apparatus performin~ the compression~.
The templates~ are preferably stored in the library in such a way that those templates Iikelv to~ match a~ given symbol are accessed first. In prior methods (see, for example, U.S.~ Patènt ~No. 4,606,069) templates are stored accortling to ~requency of occurrence, so as to speed~up~the~time talcen~ to find a matching template. According to the invention, an alternati-ve storiilg scheme is uscd which has been shown to result in increased speed and efflciency. In preferred embodiments, the templates are stored in a hashed structure indexed by features of pertinence to the matching process, such as height, width, number of black pixels, etc.
At least two modes of operation are possible with this indexing scheme. In one mode.
a given symbol is compared to templates until either a match is found or the templates are exhausted. T he templates are compared in an order determined by the inde~cing scheme so tliat those most likely to match are compared first. However, in the preferred mode ol operation, the indexing scheme is used to rule out groups oi templates, and the given WO 93/12~10 2 ~ 2 ~ ~ 6 8 PCI/U~92/10172 s~mbol is not compared to templates ~hhose indexed features would preclude a match The indexing scheme can also be used to adjust the ordering of the tests emplo~ed in the matching process to impro-e the speed with which matches are made.
According to another aspect of the invention, the stored representations of the tem-plates are compressed. Compressed representation of the templates ma- be used through-out the matching and template composition process, or, alternativelv. the templates can be compressed after these processes are complete. This is particularl~ adl,-antageous in image transmission systems where the template representations are to be transmitted. ~ ariotls compression techniques are appropriate. For examyle, the template mav be CCITT-3 en-coded. Or~ a run-length encoded representation mav be further compressed b~ an adapti~e method such as an adaptive Huffman code or a variant o~ Lempel-Ziv compression.
.~lternatively, a change in representation can be used that takes adYantage of t~e connectedness and vertical stability of tvpical characters. Rather than run-length encoding the pixels, the boundarv of the character can be traced. Fig. ~ shows one wa~ this tracing of edges can be performed. In this figure. the leftmost edge of a pixel arrav representation of the letter "b" is traced from the first black pixel in the image. Note the regular structure of the trace. Most moves are "left 0 . This regularity can allow for significant further compression.
Furthermore, the binary image representation can be decomposed into connected re-gions o f the same color pixels. The svmbol can be decomposed into a l'frame" and some "holes" in the frame by eliminating all black runs that occur betwcen the leftmost and rightmost ~black runs in a single row. The removed runs, when parity inverted, constitute the hol~es In~the frame. Fîg. 10 shows a pixel array representation of the letter "b" broken up into a ~frame and an inverted image of the holes (in this example there are two). The latter i~ma~e can itself be compressed recursiYelv by treating it as a separate symbol and placing it~ in;a template group~ The original image of the templates can then be repre-sented- by ~a reference~to `a ~ame template and a list of hole templates wîth their ~ssociated positions in the frame. The ability to use the same hole image in several frame templates allows for further compression. Because of the recursive nature of the compression process.
these holes may themselves have further holes in them, and so on. At each level, the parit~
of the frarne and its holes inverts, white holes on a black frame, black holes on a white frame, etc.
The advantage of the frame and hole representation is that the structure of the char-acters is better represented, especially as concerns the shapes of their boundaries. Conse-quentl.v, representing the templates bv a trace of their boundaries (as described above) is even more effective.

WO 93/12610 ~ 8 ~ 8 PCl /VS92/10172 ~ rror in scanning or printing text can cause neighborïng characters to run together.
By splitting such characters into their t~-o parts, more redundancy can be e.Yploited in compression. because the two parts ma- match with other characters. uhereas the run-together pair is unlikely to. The splitting of run-together characters could be performed during the symbo~ identification step of the process whenever a single vertical run of a fe-(two sayJ pi~cels connects the two halves of a s~mbol. However, according to a preferred embodirnent of the invention, the splitting is performed on the templates after all syn~ ls have been matched into template groups. After a template has beën split into one or more components, each component is preferably compared to the other templates and merged vith a matching template if one is found. Since templates preferabl- represent the "average`' features of the~ symbols in the group, the run-together error ~ill usuall~ ?~e averaged out. Thus, the reconstructed image will actually be improved by the reducti~n of run~together characters.
~ > avoid splitting templates that happen to have such a 'bridge" in them, the t-~,-o halves are preferably required to be of a sufficient size. In particular, each half can be required to be of at least a certain height. Also, one might require that one or both of the halves; match an existing template. Note that the brid~e is preferably not deleted (although it may be averaged~;out in the template representation). Thus, even if symbols which~actu:ally have such a bridge~are split, the bridge will appear when the ima~e is reconstructed.
Represéntation of the Tem~late Codes and Svmbol Locations In order~to allow reconstruction of a binary imaye compressed according to the inven-tion,~ a~reference to the template corresponding to each symbol and the position of each symbol~in two~dimensions must be éncoded. The represcntation of the template references and~the symbol positions is;preferab!y compressed.
The amount of comprcssioD will depend on the initial encoding of the template ref-erences and~ the~symbol podtions. The ide:al; encoding situation for the case where most of~the~symbols~are characts from a tc~ct is one where the symbol information is ordered in readiAg~order, that is, tho order in which a pcrson would rcad the characters. This is because the charactcrs and;~their~ posiiions, when viewed from this perspective, have the least entropy. For instance~ the characters display redundancy due to the rules of spelling o~ the language, due to their positions, due to the standards for aligning characters on a line, due to kerninR, and so ~orth.~
In preferred embodiments, therefore, an approximation of the reading order of the , ~
: characters is reconstructed. Unlike in optical character recoKnition svstems, this task must be handled fully aut~omatically~ and in a way that malces a minimum number of , WO 93/12610 ~ 1 ~ 5 ~ ~ 8 PCI/US92/10172 assumptions about the image. For instance. it should not be assumed that the binar~-image represents te~ct in English~ or even that the Roman alphabet is used. On the other hand. perfect operation is not required. since small errors onlv a~ect the compression ratio.
not the faithfulness of the reconstruction.
Reconstructing reading order is not a simple task for cases in which the original binary image representation was obtained bv scanning an original which ma- be skewecl (the tvpical situation in facsimile svstems!. One solution to reconstructing reading order in~olves tracking the lines in the text. The method described here involves a two-step process. First, all characters in a line are located. Second, the characters are sorted b~
column position to place them in correct reading order.
The first step works as follows. A first character on the page is found by "scanning the stored representation of the binary image from top-to-bottom, left-to-right. This character mav or may not be the first character in a line. The image is then scanned backward character bv character to find characters ostensibly on the same line as the , first character, ultimately ending at the leftmost character of a line. The image is then scanned forward again and all characters on the line are collected. The scanning steps must be carefullv designed, so as not to be fooled by characters on other lines that appear horizontally ali~ned beca~lse of skew in the te~ct lines of the ima~e. Fig. 11 provides an example of how skew in the text lines of an image can make characters appear as if the.~
are on the same line when they are not. In addition, characters composed of two symbols.
such as~ "i", ~"j", ":", an~ "~o" should be grouped together on one line.
; The prefèrred method o scanning starting from a current character works by looking t o the left or right for a character having at least one run within a horizontal "beam". The beam used~ to scan~ for characters is ;illustrated in the e~cample shown in Fig. 12. Note that~ln ;thls ~example, the beam ~otlnd the dot over the "i". The height o~ the beam is preferablv~initial~y the height of the first character located in the scanning process. It is widened~whene~rer a taller character is scanned, so as t~ include the entire height of the taller charactcr. When a shortcr eharacter is scanned, howeYer~ the beam is not shrunk to the n~ew character's height, but preferably to a height computed by taking a weighted running~average of the heights of the last several characters. This may be approximated, ~or example, by computing the new beam height as a sum of (n--l)/n times the old beam heiBht and l/n times the height of the next character, where n is a parameter specifying the aggressiveness of the decay. If several characters ~`all within the beam, all are added to the current line. (They will be sorted into the correct order in a later step in which the characters are sorted by horizontal position.) In order to prevent material from other lines from being inadvertently associated with ,, .

WO 93/12610 2 ~ ~ 5 8 S 8 PCl/US~2/10172 the material from the current line. the beam is preferablv limited in horizontal and ertical extent. lt is never allowed to grou to a height greater than a certain number of pi.Yels.
Furthermore~ characters hori70ntall- farther than a certain number of pixels are preferabl~
not included in the scan.
When all of the characters in a given line are found~ thev are sorted bv horizontal position to approeimate reading order~ and their template code and positions are encoded in that order.
The regularity of positioning of text OIl a page is a potential source of further com-pression. in that the position o{ a character is quite predictable from the position of the previous character. In most cases, it uill be vertically aligned at the bottom and sepa-rated from the previous character by a standard inter-character spacing. In some cases.
because of kerning, for instance~ or because the character is the dot over the "precedin~
"i", or because the character is the first in a line, the inter-character spacing will ~ar~.
ievertheless, the generally regular character spacing can be exploited by encoding not the absolute 2 and y positions o~ the characters but their offsets from the pre-ious charac-ter, for example, the displacement from the lower right corner of the previous character to the lower left corner of the current character. Furthermore, the first two sources of Yariance--kerniIlg and multi-symbol characters--can be handled by using as the default inter-character space the space between the last consecutive occurre.nces of the current characters (i~ they have occurred previously), or by encoding the most frequent value of the Inter-character space for the pair (the mode value) and using the difference from the mode as~ the representation of the inter-character space. ~inallyq the number of times that this default inter-character space can be used can be maxiII~ized by allowin~s near n~isses to be treated as hits. One way to achieve this is to per~orm the check at a reduced resolution.
Of course, the error may propogate. Ii this is unacceptable, the near miss must cause the encoded positlon of the character to differ ~rom the true position bv only a small amount;
a siDgle~ pi: cel is- appropnate.~
The seque~2ce o~ templde codes can be further compressed due to variation in thefrequency of o~currcnce of the various templates by ~Iuffman coding, or, to e~ploit regu-larities in spelling, by Lempel-Ziv compression or similar methods. The symbol position information can also be further compressed by such methods.
In fact, the template code or symbol position information c~n even be indicated in the compressed representation implicitly, rather than -bv explicit inde~cing into the template library and into coordinate positions respectively. For e~ample, the positions of characters matching each template can be given immediately following the template representation itself, thereby obviating the need for an explicit representation of template codes for those .

WO 93/12610 21.~ !5 g ~ ~ P~/US92/10172 svmbols as the associated template is indicated implicitl~.
In summary, through the above methods, typically four streams of information ar~generated: a sequence of templates representations. a sequence of template references cor-responding to the symbols (approximately in reading order i~ the binarv image is primaril-text)~ a sequence of the row positions of the svmbols. and a sequence of the column p.~-sitions of the s~ mbols. These streams in themselves constitute a significantl~ compressed representation of the binarv image. However each of these streams can be further com-pressed by the disclosed techniques.
The method has been shown to achieve, for example, a compression ratio greater than 2~ times that produced by CCITT^3 for an 8-page 300 DPI document. Each page is thus compressed from a megabyte of raw image to ~1 kilobvtes bv the method of the in-ention.
It is ~important to note that since pages of a document are not treated separatelv, t~e compression ratio will increase with the length of the document.
In some embodiments, it mav be desirable to store or transmit the compressed image at a lower resolution than the starting image. For example, one may want to store or transmit a 3~0 DPI document at 100 DPI. This can be accomplished by further compressing the template representations by skipping pixels or in any way reducing the number of pixels bv a~ real or~integer divisor. At decompression, interpolation can be used to recreate the templates.~ Of course, some quality will be lost~ but in many applications, the increase iA ;compression~ achieved is an overriding factor. lt is pre~erable that such reduction in resolution~is not performed at an~earlier stage in the compression process. This is because high ;rcsolutlon~aids the symbol matching process. Therefore, images should be scanned.
and~symbols~should~ be identified and matched, at high resolution, and any reducing of the resolution~:shoul~l be per~ormed only after the template library is established. ComPression of Non-binar~ Ima~es Although the preferred~cmbodiment is described in terms o~ compression of binar-images~having~pixels spatially distributed into rows and coiumns, the in~ention is applicable ;~ to non-binary~ images with different spatial distributions as well. For e~cample, grey-scale or color images might bc compressed b~ classifying the pixels into a first type representing text and~ graphics and a second type representing background, applying the compression meth~d to the patterns of pixels of the first type, and compressin~ the background using a different method. In this way, text documents may be compressed even if printed on a non-white background, while retaining the background information. Furthermore. the decision as to whether a pixel is of the first or second type might not be based on its gre~-~- scale value alone, but also on context or other in~ormation. Finallyj gre~-scale information might be used to distinguish text from graphics and background so that the text can be ., -:, :

WO 93/12610 21 2 ~ ~ ~ $ PCr/US92/10172 compressed by the methods of this invention while the grev-scale graphics are separatel~
compressed ~ith possible half-toning, dithering. or other preliminarv processing. (Certain of these possibilities hold for binary black-and-white images as well.) ~ nother method o~ use with non-binarv images involves classi*ing pixels into svmbols of like coloF matching them based on shape alone, and storing for each symbol an indication of its associated template and its color.
Another method of use with non-binary images involves separating the image into bit planes and compressing some or all of these bit planes using the methods of the invention.
lnsofar as there is a corrolation between certain of the bit planes and dominant symbol-based patterns, this method can compress the images better than non-symbol-matching methods.
The methods of the invention can be used to generate non-binarv images as welh even from binary originals, for example,~by maintaining the templates as grev-scale images obtained bv averaging, rather than ~oting~ on the alue of each pixel. This allovvs an anti-aliased representation of each s~ mbol to be automatically developed, which would be especiallv useful in the invention`s application as an image enhancement method.The image need not be restricted to structuring in terms of evenly-spaced ro~7s and columns. For instance. the scanning process might ha~re differing resolution in the I and y - ~ ~ direciions:(as in standard facsimile images, which are scanned at 200 by 100 pixels, thereby I eading to rectangular, rather than square pixels). Other arrangements of pixels, based on arbitrary fixed or~ random plane tessellations are possible It 1S~ recognized that variations and modifications of the in~ention will occur to those skilled in the art, and it is intended that all such ~rariations and modifications be included within~ the~ scope of the appended claims. For example, applications of the method to compressing~portions of images, either contiguous or non-contiguous. rather than entire i mages, or to subsets of identified symbols~ the subsets distinguished by function (as, ~or example, only textual idcntificd symbols rather than graphical identified symbols), size, shape,~position, or other features, are included within the scope of the invention.

What i9 claimed is:
:;

. .
~ ~0 : .

Claims (84)

Claims

1. A method for compressing images comprising:
precompressing a first representation of an image, said first representation based on pixels of a first type and a second type, to generate a precompressed representation of said image, identifying symbols in said precompressed representation of said image, comparing a plurality of identified symbols to each of at least one template from a library of templates to match identified symbols to templates, and generating a compressed representation of said image including a representation of templates, and an indication of a location and a template for symbols in said image.

2. The method of claim 1 wherein said first representation of an image is precom-pressed by run-length encoding.

3. The method of claim 2 wherein said first representation is based on rows and columns of pixels, and symbols are identified by associating runs of said first type of pixel in adjacent rows of said precompressed representation which have at least one pixel of said first type in the same column or in the same or adjacent columns.

4. A method for compressing images comprising:
identifying symbols in a first representation of an image, comparing a plurality of identified symbols to each of at least one template from a library of templates to match identified symbols to templates, wherein said library is augmented during image compression by storing in said library symbols for which no matching template is found as new templates, and wherein templates are defined by a representation determined by a plurality of the symbols which match, and generating a compressed representation of said image including a representation of templates, and an indication of a location and a template for symbols in said image.

5. The method of claim 4 wherein each template is defined by a unique spatial distri-bution of pixels, said distribution determined by the distributions of pixels of a plurality of the symbols which match said template.

6. The method of claim 5 wherein the distribution of pixels defining a template is determined by averaging the distributions of pixels of a plurality of the symbols which match said template.

7. The method of claim 5 wherein the distribution of pixels defining a template is determined by taking one symbol matching the template and changing a pixel in that symbol only if more than a predetermined fraction of the other symbols matching the template disagree with that pixel.

8. A method for compressing images comprising identifying symbols in a first representation of an image comparing a plurality of identified symbols to each of at least one template in a library of templates to match identified symbols to templates said comparing step including iden-tifying error locations performing at least one test on said error locations and on other global features of the symbol and the template, and declaring a match between a symbol and a template only if all of the tests in at least one of a set of predetermined groups of tests are passed and generating a compressed representation of said image including a representation templates and an indication of a location and a template for symbols in said image

9. The method of claim 8 wherein a match is declared only if every test in said at least one test is passed

10. The method of claim 8 wherein said first representation is based on rows andcolumns of pixels of a first type and a second type said comparing step including identifying error pixel locations which have a pixel of said first type in only said symbol or said template but not both, and performing at least one test on said error pixel locations and other global features of the symbol and the template a match between a symbol and a template being declared only if all of the tests in at least one of a set of predetermined groups of tests are passed.

11. The method of claim 10 wherein said error locations are run-length encoded

12. The method of claim 10 wherein one of said at least one test is passed if the distance between each error pixel location and the nearest pixel of the first type in the template or symbol having a pixel of the second type at the error pixel location is less than a predetermined value.

13. The method of claim 10 wherein one of said at least one test is passed if the distance between each error pixel location and the nearest pixel of the second type in the template or symbol having a pixel of the first type at the error pixel location is less than a predetermined value.

14. The method of claim 10 wherein one of said at least one test depends on the sizes shapes, and orientations of contiguous groups of error pixels in some error map

15. The method of claim 14 wherein one of said at least one test is passed if all continuous groups of error pixel locations are smaller than predetermined patterns of pixels.

16. The method of claim 14 wherein one of said at least one test is passed if inferred shifts of the symbol relative to the template are consistent.

17. The method of claim 16 wherein shifts are inferred from the sizes, shapes, and orientations of contiguous groups of error pixel locations and their relation to adjacent pixels.

18. The method of claim 11 wherein shifts are inferred from triangles of pixels in the template and symbol error maps such that the inferred shift is towards or away from the fourth pixel in the square bounding the triangle depending on whether said fourth pixel has the same or different parity, respectively, as the triangle pixels in the template.

19. The method of claim 10 wherein one of said at least one test is passed if the difference in heights between a template and a symbol is less than a predetermined value.

20. The method of claim 10 wherein one of said at least one test is passed if the difference in widths between a template and a symbol is less than a predetermined value.

21. The method of claim 10 wherein one of said at least one test is passed if the ratio of the number of pixels of the first type in the symbol and the number of pixels of the first type in the template is within a predetermined range.

22. The method of claim 10 wherein one of said at least one test is passed if the ratio of the number of error pixel locations to the number of pixels of the first type in said template or said symbol, or the minimum or maximum of the two, is less than a predetermined value.

23. The method of claim 10 wherein one of said at least one test is passed if the ratio of the number of error pixel locations to the number of pixels of the first type adjacent to a pixel of said second type in said template or said symbol, or the minimum or maximum of the two, is less than a predetermined value.

24. The method of claim 8, 9, or 10 wherein at least one of said at least one test depends on symbol size.

25. A method for compressing images comprising:
identifying symbols in a first representation of an image, comparing a plurality of identified symbols to each of at least one template in a library templates to match identified symbols to templates, wherein each said template is defined by a representation based on a spatial distribution of pixels of a first type, and generating a compressed representation of said image including a representation of templates, and indication of a location and a template for symbols in said image, wherein the representation of a plurality of said templates is a representation of the boundary of the spatial distribution of pixels of the first type for the templates.

26. A method for compressing images comprising:
identifying symbols in a first representation of an image, comparing a plurality of identified symbols to each of at least one template in a library of templates to match identified symbols to templates, and generating a compressed representation of said image including a representation o templates, and an indication of a location and a template for symbols in said image.
wherein the representation of a plurality of said templates comprises a reference to a frame template for defining an outer boundary of said template and a list of hole templates with associated positions within said frame template for defining any inner boundaries of said template.

27. The method of claim 26 wherein said frame templates and hole templates are represented by representations of the boundary of said frame templates and hole templates.

28. A method for compressing images comprising:
identifying symbols in a first representation of an image, comparing a plurality of identified symbols to each of at least one template in a library of templates to match identified symbols to templates, wherein the templates are indexed in said library according to template features. and wherein said step of comparing proceeds according to the index, whereby those templates more likely, based on features, to match a symbol are compared to the symbol before templates less likely to match, and those templates whose indexed features indicate that the likelihood of a match falls below a predetermined threshold are not compared, and generating a compressed representation of said image including a representation of templates, and an indication of a location and a template for symbols in said image.

29. A method for compressing images comprising:
identifying symbols in a first representation of an image, comparing a plurality of identified symbols to each of at least one template in a library of templates to match identified symbols to templates, generating a compressed representation of said image including a representation of templates and an indication of a location and a template for symbols in said image, and reducing the resolution of the representation of a plurality of templates.

30. The method of claim 1, 2, or 3 wherein each template is defined by a unique spatial distribution of pixels, said distribution determined by the distributions of pixels of a plurality of the symbols which match said template.

31. The method of claim 4 and 5 wherein said comparing step includes identifyingerror locations, performing at least one test on said error locations and on other global features of the symbol and the template, and declaring a match between a symbol and a template only if all of the tests in at least one of a set of predetermined groups of tests are passed.

32. The method of claim 1, 2, or 3 wherein said comparing step includes identifying error pixel locations which have a pixel of said first type in only said symbol or said template, but not both, and performing at least one test on said error pixel locations and on other global features of the symbol and the template, a match between a symbol and a template being declared only if all of the tests in at least one of a set of predetermined groups of tests are passed.

33. The method of claim 1, 2, 3, 4, 5, 8, 9, or 10 wherein said first representation of based on pixels of a first type and a second type, and the representation of a plurality of said templates is a representation of the boundary of the spatial distribution of pixels of the first type for the template.

34. The method of claim 1, 2, 3, 4, 5, 8, 9, or 10 wherein the representation of each said template comprises a reference to a frame template for defining an outer boundary of said template and a list of hole templates with associated positions within said frame template for defining any inner boundaries of said template.

35. The method of claim 1, 2, 3, 4, 5, 8, 9, 10, 25, or 26 wherein the templates are indexed in said library according to template features, and wherein said step of comparing proceeds according to the index, whereby those templates more likely, based on features to match a symbol are compared to the symbol before templates less likely to match, and those templates whose indexed features indicate that the likelihood of a match falls below a predetermined threshold are not compared.

36. The method of claim 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, or 28 furthest comprising reducing the resolution of the representation of a plurality of templates.

37. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 wherein said compressed representation of said image includes a plurality of data streams, and where at least one of said streams is further compressed.

38. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 wherein said compressed representation of said image includes a template stream, a template reference stream, and row and column position streams, and wherein at least one of said streams is further compressed.

39. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 further comprising splitting templates comprising a region smaller than a predetermined pattern of pixels into more than one template.

40. The method of claim 39 further comprising comparing templates formed by splits to other templates and merging templates formed by splits which match another template with that template.

41. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 further comprising eliminating symbols smaller than predetermined patterns of pixels.

42. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 wherein the indications of locations for symbols in said image are represented by the difference between the actual offset between a given symbol and a preceding symbol and a standard offset for the symbol pair defined by said given symbol and preceding symbol.

43. The method of claim 42 wherein said standard offset is the offset last seen for the symbol pair.

44. The method of claim 42 wherein said standard offset is the most frequently seen offset for the symbol pair.

45. The method of claim 42 wherein the difference between the standard offset and the actual offset is determined at a lower resolution than the resolution of the first repre-sentation of said image.

46. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 further comprising determining approximate reading order of said symbols and generating said compressed representation of said image according to said reading order.

47. The method of claim 46 wherein said reading order is determined by locating symbols on a single line and sorting the symbols.

48. The method of claim 47 wherein the symbols on a line are determined by usinga beam to search along the line, the size of said beam varying according to the size of one or more adjacent symbols.

49. The method of claim 48 wherein bounds on the size of the beam are imposed.

50. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 applied to an image transmission system.

51. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 applied to a facsimile transmission system.

52. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 applied to an image enhancement system.

53. The method of any of claims 1, 2, 3, 4, 5, 8, 9, 10, 25, 26, 28, or 29 applied to an image storage system.

54. Apparatus for compressing images comprising:
input means for accepting a first representation of an image, and processing means for a) precompressing said first representation of said image to generate a precompressed representation of said image, b) identifying symbols in said precompressed representation of said image, c) comparing a plurality of identified symbols to each of at least one template from library of templates to match identified symbols to templates, and d) generating a compressed representation of said image including a representation of templates, and an indication of a location and a template corresponding to symbols in said image.

55. Apparatus for compressing images comprising:
input means for accepting a first representation of an image, and processing means for a) identifying symbols in said first representation of said image, b) comparing a plurality of identified symbols to each of at least one template from library of templates to match identified symbols to templates, wherein said library is ang-mented during image compression by storing in said library symbols for which no matching template is found as new templates, and wherein templates are defined by representations determined by a plurality of the symbols which match, and c) generating a compressed representation of said image including a representation of templates, and an indication of a location and a template corresponding to symbols in said image.

56. Apparatus for compressing images comprising:
input means for accepting a first representation of an image, and processing means for a) identifying symbols in said first presentation of said image, b) comparing a plurality of identified symbols to each of at least one template from a library of templates to match identified symbols to templates, said comparing step in-cluding identifying error locations, performing at least one test on said error locations and on other global features of the symbol and the template, and declaring a match between a symbol and a template only if all of the tests in at least one of a set of predetermined groups of tests are passed, and c) generating a compressed representation of said image including a representation of templates, and an indication of a location and a template corresponding to symbols in said image.

57. Apparatus for compressing images comprising:
input means for accepting a first representation of an image, and processing means for a) identifying symbols in said first representation of said image, b) comparing a plurality of identified symbols to each of at least one template from a library of templates to match identified symbols to templates, wherein each said template is defined by a representation based on a spatial distribution of pixels of a first type, and c) generating a compressed representation of said image including a representation of templates, and an indication of a location and a template corresponding to symbols in said image, wherein the representation of a plurality of said templates is a representation of the boundary of the spatial distribution of pixels of the first type for the templates.

58. Apparatus for compressing images comprising:
input means for accepting a first representation of an image, and processing means for a) identifying symbols in said first representation of said image, b) comparing a plurality of identified symbols to each of at least one template from a library of templates to match identified symbols to templates, and c) generating a compressed representation of said image including a representation of templates, and an indication of a location and a template corresponding to symbols in said image, wherein the representations of a plurality of said templates comprise a reference to a frame template for defining an outer boundary of the template and a list of hole templates with associated positions within said frame template for defining any inner boundaries of said template.

59. Apparatus for compressing images comprising:
input means for accepting a first representation of an image, and processing means for a) identifying symbols in said first representation of said image, b) comparing a plurality of identified symbols to each of at least one template from a library of templates to match identified symbols to templates, wherein the templates are indexed in said library according to template features, and wherein said step of comparing proceeds according to the index, whereby those templates more likely, based on features.
to match a symbol are compared to the symbol before templates less likely to match, and those templates whose indexed features would preclude a match are not compared, and c) generating a compressed representation of said image including a representation of templates, and an indication of a location and a template corresponding to symbols in said image,

60. Apparatus for compressing images comprising:
input means for accepting a first representation of an image, and processing means for a) identifying symbols in said first representation of said image, b) comparing a plurality of identified symbols to each of at least one template from a library of templates to match identified symbols to a template, c) generating a compressed representation of said image including a representation of templates and an indication of a location and a template corresponding to symbols in said image. and d) reducing the resolution of the representations of a plurality of the templates.

61. The apparatus of any of claims 54-60 further comprising output means for pro-viding said compressed representation of said image.

62. The apparatus of any of claims 54-60 wherein said input means is a scanner.

63. The apparatus of claim 62 wherein said output means is a modem.

64. A method for compressing images comprising:
precompressing a first representation of an image, said first representation based on pixels of a first type and a second type, to generate a precompressed representation of said image, identifying symbols in said precompressed representation of said image, comparing a plurality of identified symbols to each of at least one template from a library of templates to match identified symbols to templates, and generating a compressed representation of said image including an indication of a location and a template for symbols in said image.

65. The method of claim 64 wherein said first representation of an image is precom-pressed by run-length encoding.

66. The method of claim 65 wherein said first representation is based on rows and columns of pixels, and symbols are identified by associating runs of said first type of pixel in adjacent rows of said precompressed representation which have at least one pixel of said first type in the same column or in the same or adjacent columns.

67. A method for compressing images comprising:
identifying symbols in a first representation of an image, comparing a plurality of identified symbols to each of at least one template in a library of templates to match identified symbols to templates, said comparing step including iden-tifying error locations, performing at least one test on said error locations and on other global features of the symbol and the template, and declaring a match between a symbol and a template only if all of the tests in at least one of a set of predetermined groups of tests are passed, and generating a compressed representation of said image including an indication of a location and a template for symbols in said image.

68. The method of claim 67 wherein a match is declared only if every test in said at least one test is passed.

69. The method of claim 67 wherein said first representation is based on rows and columns of pixels of a first type and a second type. said comparing step including identifying error pixel locations which have a pixel of said first type in only said symbol or said template. but not both, and performing at least one test on said error pixel locations and other global features of the symbol and the template. a match between a symbol and template being declared only if all of the tests in at least one of a set of predetermined groups of tests are passed.

70. The method of claim 69 wherein said error locations are run-length encoded.

71. The method of claim 69 wherein one of said at least, one test is passed if the distance between each error pixel location and the nearest pixel of the first type in the template or symbol having a pixel of the second type at the error pixel location is less than a predetermined value.

72. The method of claim 69 wherein one of said at least one test is passed if the distance between each error pixel location ant the nearest pixel of the second type in the template or symbol having a pixel of the first type at the error pixel location is less than a predetermined value.

73. The method of claim 69 wherein one of said at least one test depends on the sizes.

shapes, and orientations of contiguous groups of error pixels in some error map.74. The method of claim 73 wherein one of said at least one test is passed if all contiguous groups of error pixel locations are smaller than predetermined patterns of pixels.

75. The method of claim 73 wherein one of said at least one test is passed if inferred shifts of the symbol relative to the template are consistent.

76. The method of claim 75 wherein shifts are inferred from the sizes, shapes. and orientations of contiguous groups of error pixel locations and their relation to adjacent pixels.

77. The method of claim 70 wherein shifts are inferred from triangles of pixels in the template and symbol error maps such that the inferred shift is towards or away from the fourth pixel in the square bounding the triangle depending on whether said fourth pixel has the same or different parity, respectively, as the triangle pixels in the template.

78. The method of claim 69 wherein one of said at least one test is passed if the difference in heights between a template and a symbol is less than a predetermine value.

79. The method of claim 69 wherein one of said at least on test is passed if thedifference in widths between a template and a symbol is less than a predetermined value.

80. The method of claim 69 wherein one of said at least one test is passed if the ratio of the number of pixels of the first type in the symbol and the number of pixels of the first type in the template is within a predetermined range.

81. The method of claim 69 wherein one of said at least one test is passed if the ratio of the number of error pixel locations to the number of pixels of the first type in said template or said symbol, or the minimum or maximum of the two, is less than a predetermined value.

82. The method of claim 69 wherein one of said at least one test is passed if the ratio of the number of error locations to the number of pixels of the first type adjacent to a pixel of said second type in said template or said symbol, or the minimum or maximum of the two, is less than a predetermined value.

83. The method of claim 67, 68, or 69 wherein at least one of said at least one test depends on symbol size.

84. A method for compressing images comprising:
identifying symbols in a first representation of an image, comparing a plurality of identified symbols to each of at least one template in a library of templates to match identified symbols to templates, wherein the templates are indexed in said library according to template features, and wherein said step of comparing proceeds according to the index, whereby those templates more likely, based on features, to match a symbol are compared to the symbol before templates less likely to match, and those templates whose indexed features indicate that the likelihood of a match falls below a predetermined threshold are not compared, and generating a compressed representation of said image including an indication of a location and a template for symbols in said image.