WO1995030965A1

WO1995030965A1 - Method for recognizing handwritten input

Info

Publication number: WO1995030965A1
Application number: PCT/US1995/005640
Authority: WO
Inventors: John L. C. Seybold
Original assignee: Motorola Inc.
Priority date: 1994-05-10
Filing date: 1995-05-08
Publication date: 1995-11-16
Also published as: US5600735A; CZ285285B6; HU9503881D0; FI955609A0; FI112403B; DE69527487T2; CN1128073A; HUT73908A; PL312986A1; IL113658A0; IL113658A; DE69527487D1; MX9600190A; BR9506217A; JPH09500473A; FI955609A; EP0708944B1; EP0708944A1; EP0708944A4; AU672558B2

Abstract

The present invention determines whether two discrete continuous segments of handwritten input S1 (210) and S2 (220) form part of the same handwritten input or are part of more than one, separate handwritten inputs. The present method calculates one or more substantially parallel distance (260) disposed substantially parallel to the writing axis (230) and compares these distances to one or more predefined thresholds. The predefined thresholds specify minimum distance measures which must be exceeded by the substantially parallel distances (260) for the discrete continuous segments (210, 220) to be judged as belonging to separate segments of handwritten input.

Description

METHOD FOR RECOGNIZING HANDWRITTEN INPUT

Field Of The Invention

This invention relates generally to handwriting recognition, and more particularly to recognition of individual words.

Background of the Invention

Machine recognition of human handwriting is a very difficult problem, and with the recent explosion of pen-based computing devices, has become an important problem to be addressed. Machine recognition of human handwriting has various present applications. One example of the current application for machine recognition of human handwriting is found in personal digital assistants, such as the EO and Newton products. Typically these type of products have a touch sensitive screen upon which a user can impose handwriting. These devices then function to digitize the handwritten input, such as alphanumeric input, and thereafter process the input in an attempt to recognize the information content of the handwriting.

Pursuant to one prior art handwriting recognition technique, one makes a best determination as to the identity of each alphanumeric character in sequence, with the resulting string of characters comprising the result of the recognition activity. There are a variety of drawbacks to this approach. It is hindered by the difficulty of identifying spatial boundaries of the candidate inputs (i: this case alphanumeric characters to be recognized. When these boundaries are not located correctly, it is impossible to recognize the character accurately, since it will either be lacking pieces or will incorporate extraneous material from adjacent characters. One significant problem with machine recognition of human handwriting is the ability to recognize the end of one input and the beginning of the next input. For example, a significant problem exists in locating the end of one handwritten input segment, word, or alphanumeric input, from the beginning of the subsequent handwritten input segment, word, or alphanumeric input. Poor recognition of such breaks in the handwritten input results in poor, inaccurate interpretation of the information content of the handwritten input.

Accordingly, a need exists for a handwriting recognition technique that can detect the end of a first handwritten input segment from the beginning of a second handwritten input segment, in the handwritten input and thereby provide a more accurate interpretation of the information content of the handwritten input.

Brief Description of the Drawings

FIG. 1 Illustrates a flow diagram of operation in accordance with a preferred embodiment of the present invention.

FIG. 2 Illustrates a graphical view of an illustrative display in accordance with a preferred embodiment of the present invention.

FIG. 3 Illustrates a graphical view of an illustrative display in accordance with a preferred embodiment of the present invention.

FIG. 4 Illustrates a graphical view of an illustrative display in accordance with an alternative preferred embodiment of the invention.

FIG. 5 Illustrates a graphical view of an illustrative display in accordance with a preferred embodiment of the present invention. Detailed Description of the Preferred Embodiments

Typically, handwritten character input is collected from the user in the form of discrete continuous segments. A discrete continuous segment consists of one or more pen strokes, where a pen stroke is the mark left by a pen during its period of contact with an input device such as a digitizing tablet or paper. A stroke is represented as a sequence of points sampled at approximately regular intervals by the input device. Each point is described at minimum by an X coordinate and a Y coordinate. Strokes may be captured electronically using a digitizing tablet, or in an alternative embodiment may be derived from a scanned or faxed image through a process of line detection in the image; such methods of capturing input electronically are understood in the art.

Generally, the present invention as disclosed determines whether two discrete continuous segments form part of the same handwritten character input or part of more than one handwritten character input. In the present invention one or more discrete continuous segments are the units of handwritten input being recognized. Handwritten input is input which is captured electronically that includes but is not limited to the following: handwritten input; electronic input; input captured through pressure, such as stamped input; input that is received electronically, such as via facsimile, pager, or other device. For example - the present invention determines whether two discrete continuous segments form part of the same word or whether they form part of separate words. In a preferred method, the present invention calculates one or more substantially parallel distances disposed substantially parallel to the writing axis, and compares these distances to one or more predefined thresholds. The predefined thresholds specify minimum distance measures which must be exceeded by the substantially parallel distances for the discrete continuous segments to be judged as belonging to separate handwritten input, for example separate handwritten words. The writing axis is the line along which the handwritten input is added. The writing direction is the direction in which each subsequent handwritten input is added In English, handwritten input is added typically along a horizontal writing axis with each subsequent alphanumeric input following horizontally after the previous input in a writing direction that is left to right. Various other writing axis and writing directions alternatives are possible with implementation of the teachings of the present invention.

In a preferred embodiment, the handwriting axis is horizontal and the handwritten input forms a series of words. In this preferred embodiment, the substantially parallel distances are calculated horizontally, and the output tells whether discrete continuous segments belong to separate words. In an alternative preferred embodiment, the handwriting axis is horizontal and the handwritten input forms a series of separate characters, which may be alphanumeric characters, ideographic characters as found in languages such as Chinese, or other forms of characters or symbols of written communications. In this alternative embodiment, the output tells whether the discrete continuous segments belong to separate characters. In another preferred embodiment, the handwriting axis is vertical and the handwritten input forms a series of separate characters, which may be alphanumeric characters, ideographic characters, or other handwritten text. In this preferred embodiment, the writing axis is vertical and the substantially parallel distances are aligned vertically; the output tells whether the discrete continuous segments belong to separate characters. In yet another preferred embodiment, the handwriting axis is vertical and the handwritten input forms a series of separate words, alphanumeric input, or other handwritten input, such as a vertical list of words, or numbers. In this preferred embodiment, the writing axis is vertical and the substantially parallel distances are aligned vertically; the output tells whether the discrete continuous segments belong to separate handwritten input, such as separate words. As disclosed above and as will be discussed further, the present invention demonstrates through the disclosure of several of the preferred embodiments that the writing axis may exist at any angle and the handwritten input may be interpreted more generally as corresponding to discrete elements (including but not limited to characters and words) containing one or more discrete continuous segments. The application of the methods described herein to any of various preferred embodiments requires only a change in the coordinate system used and such modifications can be made in accordance with the teachings presented.

Referring now to FIG. 1, a preferred method of the present invention is illustrated. The present invention is applicable to one or more handwritten inputs of discrete continuous segments. The preferred embodiments of the present invention, are applicable to two or more handwritten inputs of discrete continuous segments. The use of only two discrete continuous segments Si and S2 is for illustrative purposes. In the preferred method illustrated in FIG. 1, handwritten input consisting of two discrete continuous segments Si and S₂ (110) is accepted by a device, such as a PDA, or other device. Other devices which function to receive handwritten input include but are not limited to the following: computers, modems, pagers, telephones, digital or interactive or other televisions, devices having a digitizing tablet, facsimile devices, scanning devices, and any device with the ability to capture handwritten input. Preferably, upon acceptance of the handwritten input the substantially perpendicular boundaries bi and b₂ between the discrete continuous segments Si and S₂ are identified (120). The boundaries bi and b2 which are substantially perpendicular to the writing axis are determined by finding the point in the stroke sequence Si that has the largest displacement along the writing direction and the point in the stroke sequence S₂ that has the smallest displacement along the writing direction (120). The substantially perpendicular boundary bi is the displacement values of the point in the stroke sequence Si that has the largest displacement along the writing direction. The substantially perpendicular boundary b₂ is the displacement values of the point in the stroke sequence S₂ that has the smallest displacement along the writing direction. By calculating the displacement in the writing direction of each point in Si in turn and comparing that value to a stored value that is initially a very large negative number, the substantially perpendicular boundary bi that has the largest displacement along the writing direction can be determined. If the calculated displacement value is larger than the stored value, the stored value is replaced with the calculated displacement value. Once all the points have been examined, the stored value will contain the value of the largest displacement value found in the stroke sequence. A similar procedure, starting with a large positive initializing value, can be used to discover the point in S₂ that has the smallest displacement along the writing direction by calculating the displacement in the writing direction of each point in S₂ in turn and comparing that value to a stored value that is initially a very large positive number. If the calculated displacement value is less than the stored value, the stored value is replaced with the calculated displacement value. The preferred method then calculates a first substantially parallel distance d', where d' - b₂ - bi (130).

Referring to FIG. 1, the first substantially parallel distance d' is compared to a first predetermined threshold t'. If d' is greater than or equal to (>■=) t' the preferred method concludes that the first continuous discrete segment Si and the second continuous discrete segment S2 belong to different segments of handwritten input (145). By way of example if d' is greater than (>) t¹ the preferred method concludes Si and S2 are different words, characters, or other elements of handwritten input. The selection of a predetermined threshold t' is a value made in accordance with the specific embodiment. Choice of a threshold t' will be discussed in further detail elsewhere. If the value of d' is smaller, or less than, the threshold value t' further processing occurs. To determine if the discrete continuous segments Si and S2 form separate discrete continuous segments of handwritten input, such as separate words, characters, or other elements, a second substantially parallel distance d" is calculated. The second substantially parallel distance d" is found by calculating several substantially parallel distances and choosing the shortest of those distances. In a preferred embodiment, this is done by first calculating the maximum extent perpendicular to the writing axis subtended by Si and S2 together (150). The extent is then divided into a number of bands of equal height substantially parallel to the writing axis (160). The substantially parallel distance between Si and S2 for each band (170) is then found. The smallest or shortest substantially parallel distance between Si and S2 from among the plurality of bands is selected as d" (180). In the preferred method represented in FIG 1, once the substantially horizontal distance d" is found, the distance d" and the first substantially horizontal distance d' are combined to aid in making a final decision that Si and S2 are part of the same handwritten input or are parts of separate discrete continuous segments of handwritten input. A weighted average of d' and d" is calculated. A preferred weighted average equation is q'd' + q"d"/(q' + q"). It has been found by empirical tests on actual handwriting data that a weighted average of d' and d" is more accurate than either alone. In one preferred embodiment, q' = q", but it may be possible to optimize this further by a judicious choice of q' and q" which are not identical. Additionally, q' and q" can each be equal to zero, but not at the same time. The weighted average is compared against a second threshold t". If the weighted average is less than or smaller than t" the preferred method concludes that Si and S2 are part of the same segment of handwritten input (197). If the weighted average is larger than or equal to t" the preferred method concludes that Si and S2 are different continuous discrete segments of handwritten input and the preferred method concludes with that result (199).

The preferred method of selecting t' and t" is a detail of the specific embodiment. In one embodiment, t' may be set by measuring the value of d¹ across a large set of data and choosing the value that best distinguishes real dividing points from false choices. In another preferred embodiment, t' may be set to a constant value that corresponds to some fraction of the distance between input guides on the input device. In another preferred embodiment, t¹ may vary dynamically as a fraction of the measured extent of Si and S2 perpendicular to the writing axis. In another embodiment, t' may be set explicitly by the user before the method is invoked. In the preferred embodiment illustrated, t^! is set to be 1/3 of the measured extent of Si and S2 perpendicular to the writing axis, which gives a good result for English, alphanumeric text. Many other ways of setting t' may be considered and may be optimal for different solutions. The various ways of determining t" in different embodiments are similar to those described for determining t¹.

In FIG. 2, a graphical representation is given, for a preferred embodiment. In this embodiment, the discrete continuous segments, Si (210) and S₂ (220), correspond to English words, the writing axis (230) is horizontal, and the writing direction (270) is left to right. In this preferred embodiment, the displacement along the writing direction for any point is simply the value of the X coordinate for that point, so the substantially perpendicular boundary bi (240) is the value of the X coordinate of the rightmost point in Si (210) and the substantially perpendicular boundary b₂ (250) is the value of the X coordinate of the leftmost point in S₂ (220). The substantially parallel distance d' (260) is the distance extending between bi and b₂- Alternatively, as illustrated in FIG. 3, the discrete continuous segments Si (310) and S2 (320) correspond to Chinese characters, the writing axis (330) is vertical, and the writing direction (370) is top to bottom. In this embodiment, the displacement along the writing direction (370) for any point is simply the value of the Y coordinate for that point multiplied by -1 (assuming a standard coordinate system in which Y increases as one moves from bottom to top), so the substantially perpendicular boundary bi (340) is the value of the Y coordinate times -1 of the bottommost point in Si (310) and the substantially perpendicular boundary b₂ (350) is the value of the Y coordinate times -1 of the topmost point in S₂

(320).

In other embodiments, the values for bi and b₂ can be calculated by applying a simple geometric rotation of the handwritten input to line up the writing axis with one of the cardinal axes of the coordinate system, and then applying the procedure just described. This is a straightforward operation that will be understood by persons in conjunction with the teachings presented here. Alternatively, other embodiments may calculate the displacements of the points along the writing direction, when the writing direction does not line up with one of the cardinal axes of the coordinate system, by a simple geometric projection of the discrete continuous segments onto the writing axis and then using trigonometry to calculate the displacement of the projected points from the origin of the coordinate system. This is a well-understood mathematical procedure that can be applied to the teachings herein. As stated previously, if d' is smaller than t', the processing continues. In many cases, segments that do in fact belong to separate words or characters are positioned in such a way that d' is small or even negative, but it is still possible to determine that Si and S₂ belong to different words or characters. FIG. 4 shows an example of this situation for an embodiment in which Si (410) and S₂ (420) are English words. In this example, d' (460) is actually negative, and bi (440) is larger than b₂ (450). In this example, the writing axis (430) is horizontal and the writing direction (470) is left to right.

In one embodiment in which the writing axis is horizontal, the maximum extent perpendicular to the writing axis subtended by Si and S₂ is calculated by finding the minimum and maximum Y coordinate values present in Si and S₂ • To find the minimum Y coordinate value, a stored value is initialized with a very large positive value, and then every point in Si and S₂ is examined in turn, and if its Y coordinate value is smaller than the stored value, the stored value is assigned that Y coordinate value. A similar method is used to find the maximum Y coordinate value. By subtracting the maximum and minimum Y coordinate values, the extent perpendicular to the writing axis can be calculated. All points in Si and S₂ lie within this extent.

In a preferred embodiment, this process can be made more efficient by considering only a subset of the points in Si and S₂ such that only a fixed number of points near the boundary in question are considered , since these are the ones most likely to affect the measure being derived.

In an alternative embodiment, in which the writing axis is vertical, the maximum extent perpendicular to the writing axis subtended by Si and S₂ is calculated by finding the minimum and maximum X coordinate values present in Si and S₂ • To find the minimum X coordinate value, a stored value is initialized with a very large positive value, and then every point in Si and S₂ is examined in turn, and if its X coordinate value is smaller than the stored value, the stored value is assigned that X coordinate value. A similar method is used to find the maximum X coordinate value. By subtracting the maximum and minimum X coordinate values, the extent perpendicular to the writing axis can be calculated.

In an alternative embodiment, where the writing axis is neither horizontal nor vertical, the image plane can be rotated to align the writing axis with the X or Y axis of the coordinate system, as described above in the calculation of the substantially parε distance d' when the writing axis was neither horizontal nor vertical.

Once the extent perpendicular to the writing axis has been found, a preferred embodiment divides this extent into bands substantially parallel to the writing axis, such that each band describes a narrow slice through Si and S₂. Within each band, the point in Si with the largest displacement in the writing direction is found, and the point in S₂ with the smallest displacement in the writing direction is found. This can be done efficiently with respect to Si by initializing a stored value for each band to a very large negative number. Each point in S i is checked in turn. First, its displacement perpendicular to the writing axis is checked to see which band it lies in. Since the bands span the entire extent of Si and S₂ perpendicular to the writing axis, it is guaranteed that a band can be found for any point in Si and S₂. If the preferred embodiment described above for calculating the perpendicular extent by considering only a subset of points in Si and S₂, the perpendicular extent may not contain all points in Si and S₂, and so each point must be checked to ensure that it lies within a band. If it does not, it is not considered further. Once the band is identified, the displacement of the point in the writing direction is compared to the stored value for that band, and if the displacement is larger, it is assigned to the stored value. Once all of the points in Si have been checked in this way, the stored value in each band will contain the largest displacement of the point with the largest displacement encountered in that band in Si. If there were no points in Si lying within a given band, the stored value will remain as the very large negative initializing value. A similar procedure, using a second stored value in each band initialized to a large positive number, is used with respect to S₂ to find the minimum displacement of the point with the minimum displacement in the writing direction within each band in S₂. If there are no points for a given band in S₂, the second stored value for that band remains at its large initialized value.

At the end of this operation, for each band there are two stored values corresponding to the largest displacement in the writing direction of Si in the band and the smallest displacement in the writing direction of S₂ in the band. If either stored value has not changed from its very large negative or positive initializing value, then the information for that band is not used in any further calculations. Otherwise, a distance is calculated for each band by subtracting the stored value for the displacement of the point in Si from the stored value for the displacement of the point in S₂. The smallest of these distances is found by examining the distance for each band in turn and storing the smallest one found. This smallest distance is the substantially horizontal distance d". If d" is less than 0, which can occur if Si and S₂ touch or overlap, it is assigned the value 0.

One embodiment in which the writing axis is horizontal and the writing direction is left to right is shown in FIG. 5. In this embodiment, the number of bands (510) is 10. The rightmost point of Si lying within each band is found (520) and the leftmost point of S2 lying within that band is also found (530). This is done with respect to Si by initializing a stored value for each band to a very large negative number. Each point in Si is checked in turn. First, its Y coordinate is checked to see which band it lies in. Since the bands span the entire height, extent (540) of Si and S₂, it is guaranteed that a band can be found for any point in Si and S₂. Once the band is identified, the X coordinate of the point is compared to the stored value for that band, and if the X coordinate is larger, it is assigned to the stored value. Once all of the points in Si have been checked in this way, the stored value in each band will contain the X coordinate of the rightmost point encountered in that band in Si . If there were no points in Si lying within a given band, the stored value will remain as the very large negative initializing value. A similar procedure with respect to S₂, using a second stored value in each band initialized to a large positive number, is used to find the X coordinate of the leftmost point in each band in S₂. As described previously, the value of d" is computed from the stored locations for each band. The smallest, or shortest, substantially parallel distance between Si and S2 from amount the plurality of bands is selected as d". The distance d" is then combined with d' via a weighted average and compared to t". Upon comparison with t" the method of the present invention concludes whether the discrete continuous segments Si and S₂ belong to the same handwritten input or separate handwritten inputs.

It will be apparent to those skilled in the art that the disclosed invention may be modified in numerous ways and may assume many embodiments other than the preferred forms particularly set out and described above. Accordingly, it is intended by the appended claims to cover all modifications of the invention that fall within that fall within the true spirit and scope of the invention and its equivalents.

I Claim:

Claims

1. A method comprising the steps of: receiving handwritten character input, which handwritten character input is comprised of at least a first and a second discrete continuous segment; calculating at least on substantially parallel distance between the first and second discrete continuous segments; using the substantially parallel distance to determine whether the first and second discrete continuous segments belong to separate handwritten character input.

2. The method of claim 1 wherein at least one of the following: a) the step of calculating at least one substantially parallel distance includes the step of calculating a plurality of substantially parallel distances between the first and second discrete continuous segments and, where further selected the step of calculating a plurality of substantially parallel distances further includes the step of identifying the substantially parallel distance having smallest value and, where further selected the step of using the parallel distance to determine whether the first and second discrete continuous segments belong to separate handwritten inputs includes the step of using the substantially parallel distance having the smallest value to determine whether the first and second discrete continuous segments belong to separate handwritten inputs; b) further including the steps of: identifying a substantially perpendicular boundary for each of the first and second discrete continuous segments; calculating a distance between the substantially perpendicular boundary for the first and second discrete continuous segments and, where further selected the step of using the parallel distance to determine whether the first and second discrete continuous segments belong to separate handwritten inputs includes the step of also using the distance between the substantially perpendicular boundary for the first and second discrete continuous segments to determine whether the first and second discrete continuous segments belong to separate handwritten inputs.

3. A method comprising the steps of: receiving handwritten input, which handwritten input is comprised of at least a first and a second discrete continuous segment; identifying a substantially perpendicular boundary for each of the first and second discrete continuous segments; calculating a first distance between the substantially perpendicular boundary for the first and second discrete continuous segments; when the first distance does not at least exceed the first predetermined threshold, calculating at least one substantially parallel distance between the first and second discrete continuous segments; using the first distance and the substantially parallel distance to determine whether the first and second discrete continuous segments belong to separate handwritten inputs.

4. The method of claim 3, wherein the step of using the first distance and the substantially parallel distance to determine whether the first and second discrete continuous segments belong to separate handwritten inputs includes the step of adding a weighted first distance and a weighted substantially parallel distance together to obtain a result and dividing the result by two to obtain a final result, and comparing the final result with a second predetermined threshold.

5. The method of claim 4, wherein at least one of the following: a) the first and second predetermined thresholds are identical to one another; b) the first and second predetermined thresholds are different from one another; c) a weighting factor, as applied to both the first distance and the substantially parallel distance to obtain the weighted first distance and the weighted substantially parallel distance, respectively, is one, such that the first distance is equal to the weighted first distance and the substantially parallel distance is equal to the weighted substantially parallel distance.

6. The method of claim 3, wherein the step of calculating a first distance between the substantially perpendicular boundary for the first and second discrete continuous segments includes the steps of: when the substantially parallel distance is positive in value, using the substantially parallel distance as the substantially parallel distance; when the substantially parallel distance is negative in value, using a predetermined value as the substantially parallel distance and; where further selected the predetermined value is zero.

7. The method of claim 3, wherein the step of calculating at least one substantially parallel distance includes the step of calculating a plurality of substantially parallel distances between the first and second discrete continuous segments.

8. The method of claim 7, wherein the step of calculating a plurality of substantially parallel distances further includes the step of identifying the substantially parallel distance having a smallest value and; where further selected the step of using the first distance and the substantially parallel distance to determine whether the first and second discrete continuous segments belong to separate handwritten inputs includes the step of using the substantially parallel distance having the smallest value and the first distance to determine whether the first and second discrete continuous segments belong to separate handwritten inputs.

9. A method comprising the steps of: receiving handwritten input, which handwritten input is comprised of at least a first and a second discrete continuous segment that are disposed substantially parallel to a writing axis; identifying a boundary that is perpendicular to the writing axis for each of the first and second discrete continuous segments; calculating a first distance between the boundary for the first and second discrete continuous segments; when the first distance at least exceeds a first predetermined threshold, concluding that the first and second discrete continuous segments belong to separate handwritten inputs; when the first distance does not at least exceed the first predetermined threshold, calculating at least one substantially perpendicular distance between the first and second discrete continuous segments, which substantially perpendicular distance is substantially perpendicular to the writing axis; using the first distance and the substantially parallel distance to determine whether the first and second discrete continuous segments belong to separate handwritten inputs.

10. A method of determining that two discrete continuous segments belong to separate handwritten inputs, comprising the steps of: receiving handwritten input, which handwritten input is comprised of at least a first and a second discrete continuous segment; identifying a substantially perpendicular boundary for each of the first and second discrete continuous segments, which substantially perpendicular boundaries are those boundaries that are most adjacent to one another; calculating a separation distance between the substantially perpendicular boundary for the first and second discrete continuous segments; when the separation distance at least exceeds a first predetermined threshold, concluding that the first and second discrete continuous segments belong to separate handwritten inputs; when the separation distance does not at least exceed the first predetermined threshold, calculating a plurality of substantially parallel separation distances between portions of the first and second discrete continuous segments; using the separation distance and a smallest one of the substantially parallel separation distances to determine whether the first and second discrete continuous segments belong to separate handwritten inputs.