WO1991007729A2

WO1991007729A2 - Improvements in methods and apparatus for signature verification

Info

Publication number: WO1991007729A2
Application number: PCT/GB1990/001725
Authority: WO
Inventors: David Nicholas Lloyd Howell; Colin Sefton Hilton; John Scott Bridle; Roger Kenneth Moore; Martin James Russell
Original assignee: The Secretary Of State For Defence In Her Britannic Majesty's Government Of The United Kingdom Of Great Britain And Northern Ireland
Priority date: 1989-11-10
Filing date: 1990-11-09
Publication date: 1991-05-30
Also published as: GB9024411D0; GB8925479D0; GB2237917A; WO1991007729A3

Abstract

A signature to be verified is written on an area (13) which carries horizontal and vertical lines using a pen (10). As the lines are crossed signals representative of the vertical and horizontal components of the pen tip velocity and whether the tip is in contact with the area are passed to a computer (11). A hidden Markov model derived from vertical and horizontal velocities and a ''contact'' signal occurring as a number of authentic signatures are written is stored by the computer. A forward pass of the forward-backward algorithm is used to calculate the probability that the vertical and horizontal components of the pen tip and the contacts between pen tip and the area (13) could have been produced from the hidden Markov model. This probability is used to decide whether the signature is authentic. The hidden Markov model stored by the computer is derived from an initial model based on pen tip velocities and contacts occurring in an authentic signature, and re-estimation carried out using forward and backward passes of the forward-backward algorithm and velocities and contacts from further authentic signatures.

Description

IMPROVEMENTS IN METHODS AND APPARATUS FOR SIGNATURE VERIFICATION The present invention relates to methods and apparatus for signature verification both automatically and as an aid to verification carried out by a person.

Automatic signature verification has uses in financial transactions, for example with cheques and smart cards, and in providing access to facilities, such as computers, or premises.

There are of course many other instances of signature use both past and future which benefit from automatic signature recognition.

In known signature verification apparatus the signatory signs on an electronic input device, numerous types of which are known operating on several different principles. Some operate on the principle described in U.K. Patent No. 1,310,683 and U.S. Patent

No. 3,885,097 to provide x,y coordinates of points in the signature, others interrupt light to a sensor as opaque lines on a writing surface are crossed, as described in U.K. Patent Application

No. 8626411 (Publication No. 2183071) and corresponding United

States Applications Nos. 076209, 137676 and 548192) to provide such features as variations in speed at which different parts of a signature are written, and in other cases a noise generated while the signature is written provides information for signature recognition.

Hidden Markov modelling is a technique that has been used for modelling events that are assumed to have been generated by a statistical process. Hidden Markov Models (HMMs) have been used to model many systems. For example, biological populations, musical styles but in particular HMMs have been used to model speech and natural language. A suggestion that HMMs might be used for signature recognition has been made in the paper "Handwritten Script Recognition Using HMMs" by R. Nag, K.H. Wong and F. Fallside, ICASSP 86, Tokyo, pages 132 to 135. This paper is mainly concerned with recognising handwritten characters by constructing HMMs based on the inclination of script at different points in the characters. HMMs for signature recognition are also briefly mentioned in the above mentioned U.K. Specification No. 2183071.

An HMM is a class of finite state machine in which the output from a particular state is chosen by sampling from possible values with probability governed by probability distribution function(s) (pdf). Such a machine generates a series of outputs, each of which is dependent upon its current internal state. The transition from one state to another is dependent on a set of transition probabilities. According to a first aspect of the present invention there is provided a method of automatic signature verification comprising, for each signatory, the steps of forming a finite state machine particular to the signatures of that signatory based on corresponding measurements of a plurality of signatures from that signatory, at least one of the measurements on each signature being of a feature which is only apparent from values occurring at the time the signature was written, and using the finite state machine to verify signatures alleged to be written by the signatory. According to a second aspect of the present invention there is provided apparatus for signature verification comprising means for deriving and storing a finite state machine for the signature of each person whose signature is to be recognised by the apparatus, each said machine being based on corresponding measurements on a plurality of signatures from that person, and at least one of the measurements of each signature being of a feature which is only apparent from values occurring at the time the signature was written, and means for verifying signatures based on the stored finite. state machines.

Usually the finite state machines are HMMs.

By deriving at least some of the states of the HMMs from dynamic signature features, that is features apparent from values occurring at the time a signature is written, forgery is made much more difficult since in addition to achieving a signature which appears correct, the forger has to mimic dynamic features which are unknown to him.

The HMM states may, for example, be based on Cartesian components of the velocity at which the tip of a pen used in signing travels as different segments of the signature are written, each segment corresponding to a state in the HMM. Thus in each HMM the state pdfs may represent distributions of pen velocity components occurring in writing signatures. Static features may be combined with dynamic features in constructing the HMMs. The invention also relates to methods and apparatus for deriving finite state machines representing signatures wherein at least one of the states of the machine for a signature is based on a dynamic feature of that signature. Methods and apparatus for verifying signatures from stored finite state machines, where each of the said machines is based on a dynamic feature of a signature, form further aspects of the invention.

In an embodiment of the invention, signatures can be written using a pen which interrupts light to a sensor as opaque lines on a writing surface are crossed, as described in the above mentioned U.K. Patent Application No. 8626411.

Each person whose signature is to be recognised writes his signature several times using a pen such as that mentioned in the previous paragraph, and a programmed computer derives an HMM by dividing each signature into segments and using the forward-backward algorithm to re-estimate an initial HMM based on orthogonal components of pen velocity in the segments. Preferably each signature is divided into between thirty and a hundred segments, typically fifty, in deriving the corresponding HMM. The number of states in the HMM usually equals the number of segments used to derive it. An HMM for each person's signature is stored for use when signatures are to be recognised.

To verify a signature, the signature is automatically matched against the stored HMM derived from signatures of the person alleged to be the author of the signature by calculating the logarithm of the likelihood of the model generating the signature to be verified. The likelihood value is calculated by a forward probability or "alpha" pass through the model.

According to a third aspect of the invention there is provided a method of verifying a signature comprising regenerating a signature from a finite state machine model of the signature, and comparing the regenerated signature with a trial signature to be verified in rejecting or accepting the trial signature.

According to a fourth aspect of the invention there is provided apparatus for use in signature verification comprising means for storing data defining the states of a finite state machine model of a signature, and means for generating a display of a representation of the signature from at least some of the said data, whereby a signature to be verified can be compared with the said representation.

The comparison between the displayed representation and the signature to be verified can be carried out either by a person or automatically. Certain embodiments of the invention will now be described by way of example with reference to the accompanying drawings, in which:-

Figure 1 is a schematic diagram of apparatus according to the invention, Figure 2 is a diagram of part of an HMM used to represent a signature,

Figure 3 is a flow diagram of an algorithm for generating HMMs from sample signatures, and

Figure 4 is a flow diagram of an algorithm for regenerating a signature from an HMM.

Apparatus is first described which, in general, can be used for signature recognition or generating HMMs for signature recognition. In the apparatus of Figure 1 a pen 10 is used to write a signature 12 on a region 13 of a document 14 such as, in signature recognition, a cheque. The region 13 has closely ruled horizontal and vertical lines and the signature 12 is written over these lines. The pen may be in the form which provides analogue signals for a computer 11, if the computer is capable of analysing such signals to produce pulses which correspond to line crossings in the signature, or the pen itself may include some form of analogue-to- digital converter which provides pulses which correspond to line crossings. The horizontal lines in the area 13 are a different colour from the vertical lines in that area and the pen 10 generates two analogue or digital signals, each corresponding to line crossings of one respective colour. Thus the computer 11 is able to derive two signals representative of the number of vertical and horizontal lines, respectively, crossed in unit time. The computer therefore has available digital signals representative of vertical and horizontal components of pen velocity as the signature is written. In addition the pen includes a pressure sensitive switch which indicates whether or not the pen tip is in contact with a surface on which the signature is written and, in this way, a "contact" signal is also supplied to the computer 11.

Many other types of graphical input tablet may be used to supply signals which can be analysed to derive vertical and horizontal components of pen-tip velocity (x and y velocities) and an indication of whether the tip is in contact with a writing surface or not. For example an input tablet which provides a stream of digital signals representative of the coordinates of the position of the pen tip when in contact with a writing surface may be used. In order to generate an HMM of a signature the number of sample signatures, typically five, are written on a graphical input tablet, for example in the area 13 of Figure 1, and digital signals representing vertical and horizontal velocity components for the whole signature are stored together with a digital "contact" signal. These signals are used to generate an HMM of the signature by dividing each of the signals into, typically, 50 uniform-length segments and generating a 50 state HMM. The Bakis form of HMM is used in which, in this example, each of the states has three associated Gaussian probability density functions (pdfs), one for each input data dimension (that is x and y velocities and writing surface contact indicator). Each pdf is represented by a mean value and a variance and the Bakis form also requires transition probabilities between states to be zero except for transitions from a state to itself, and transitions from a state to two other specified states. A diagrammatic representation of such an HMM is partially shown in Figure 2 with states represented by circles 20 to 23, 49 and 50. Non-zero transitions are represented by arrows, some of which, such as the arrow 30, point back to the state itself and represent transitions from a state to itself. Other arrows, such as the arrows 31 and 32, which join one state to another, represent transitions between states.

The computer 11 is programmed to hold a number of matrices, one of which is known as matrix A and has dimensions 50 x 3. This matrix contains real valued natural logarithms of probabilities of transitions from state i to state j where i and j index the matrix. The Bakis form requires only 150 non-zero matrix elements, three for each of the 50 states. The matrix B having dimensions of 50 x 3 contains mean and variance values of the state pdfs. A further matrix 0, for input data, has dimensions 3 x a number equal to the number of digital signals expected from a signature of maximum length. The matrix 0 holds and stores the incoming input signals. The input signals representing a signature can be regarded as being in a number of frames, each containing 3 input signal values, x and y velocity and contact value, and one frame occurring each time values of these three signals are provided.

The HMM is generated by re-estimation from an initial model which, as mentioned above, has typically 50 states corresponding to 50 segments. All the transitions of the Bakis model have initial values of unity and the pdfs are represented by the means and variances of each of the x and y velocities and the contact value calculated for each of the initial 50 segments. Although the latter can in practice only take up the values "1" and "0" it is believed that the mean and variance values obtained are useful. At this point the HMM is stored in matrices A and B.

Thus in Figure 3, the flow diagram for HMM generation, the first three operations are to segment the three input signals of one of the sample signatures into 50 segments (operation 40) and calculate the mean and variance for each segment of each input

(operation 42). The "forward" or "alpha" probability values and the "backward" or "beta" probability values are calculated frame by frame, in an operation 43, using the Baum-Welch or forward-backward algorithm as described, for example, by L.A. Liporace in "Maximum likelihood estimation for ultivariate stocastic observations o

Markov sources", IEEE Transactions on Information Theory, 28:

729-734, 1982. Log probabilities are used and it is therefore necessary to add terms in many of the equations. This necessitates the use of an "addlog" function as described by Kingsbury (see appendix B of the paper by S.O. Cox "Hidden Markov Models for

Automatic Speech Recognition: Theory and Application, RSRE

Memorandum No. 4135, Procurement Executive, Ministry of Defence,

RSRE Malvern, Worcestershire, U.K.). The forward-backward algorithm is also described by Cox and by J.N. Holmes in his book "Speech Synthesis and Recognition", Van Nostrand Reinhold (UK) Co.

Ltd. (1988), see Chapter 8. J.N. Holmes also mentions Kingsbury.

The references to Liporace, Cox, Holmes and Kingsbury are hereby incorporated into this specification.

The forward and backward probabilities are stored in two matrices C and D, respectively, and the dimensions of these matrices are the number of states (usually 50) x the number of frames in a signature, that is one matrix element for the forward and backward probability of each state and each frame. Using these matrices an expected value for each of the transition probabilities and each of the means and variances for each state is calculated (operation 44) and stored in two further matrices known as exptrans (dimensions 50 x 3) and expB (dimensions 50 x 6). Storing these values is shown as an accumulation operation 45 since the operation is later repeated for the other signature samples when storing is replaced by accumulation.

The operations 43 to 45 are repeated using the three input signals from another sample signature. The expected values obtained are added in the operation 45 to the corresponding values in matrices exptrans and expB. The process of repeating the operations 43 to 45 is repeated with the three input signals from each remaining sample signature in turn until a test 46 determines that all the signature samples have been used.

The accumulated values are then used to calculate re-estimates of the values of the transitions, means and variances in an operation 47, this being a process akin to normalisation, the results being stored in the A and B matrices. The forward pass of the forward-backward algorithm is carried out in an operation 48 using the transition probabilities, means and variances from the operation 47, and then a test 51, which is explained below, is carried out. At this stage the test 51 causes a jump back to the operation 43. In repeating the operations 43 to 46, each sample signature is used again in turn by reading its three input signals into the matrix 0. The operations 43 to 48 are repeated until, as indicated by the test 51, the increase in the likelihood of the data being produced by the model between successive iterations is below a value such that further iterations do not provide a worthwhile improvement. At this point the parameters stored in the A and B matrices define the final HMM. At each stage the likelihood of the data being produced by the model is given by the value in the last cell of the C matrix (forward probabilities), that is the matrix element corresponding to the last state in the HMM.

Having described the generation of HMMs for signatures, signature verification will now be considered. The signatory signs on using, for example, an apparatus of the type shown in Figure 1 or another apparatus from which x and y velocity signals and contact signals can be generated. Although in signature verification for large-scale applications such as in banking only the pen 10 and the area 13 need to be specially provided since a general purpose computer such as one already installed for such applications can be used to calculate the probability of an unknown signature being genuine.

For verification, the parameters of HMM derived from the sample signatures are stored in equivalents of the A and B matrices but the arrays expB, exptrans and D are not required.

The probability of the new or trial signature data having been generated by the stored HMM is now calculated and can be expressed as P(0|M) which means the probability of the set of observations 0 occurring given values defining a Markov model M.

If 0-| , O2 , 0χ are observations corresponding to frames of the input signals and N is the number of states in the model, then the probability of the joint event of emitting the partial observation sequence 0-| , O2 , 0-t and occupying state Sj at time t is given by αt(J) = ^p(°l- °2 . °t- (state Sj at time t)|M)

Then the completed forward probability pass yields

P(0|M) = ∑ α_T(j) j=l

The x velocity, y velocity and contact signals from the trial signature and the stored HMM parameters are used in a forward pass of the forward-backward algorithm to calculate P(0|M) and the last element in the matrix C provides this value when the forward pass is complete. The probability found is tested against a threshold and the signature is rejected or accepted as a result of this test.

Whether a signature should be accepted at a given level of probability is one which will depend on the application concerned; for example in banking it may be more acceptable to have a threshold probability for acceptance which allows a small percentage of forgeries to be accepted but gives a lower percentage of false rejections. Forgeries are, on the whole, unlikely and in any case most would be discovered by such a threshold level while false rejections create bad feeling against a system if they occur often with genuine users. In another application, such as access to security premises, a lower threshold of pulse acceptances might be better since false rejections can be tolerated by suitably instructed staff. Thus setting the probability threshold for acceptance is a matter of the application concerned but it is also necessary to set the threshold heuristically.

Another aid to recognition is the possibility of regenerating a representation of a genuine signature so that it can be displayed, for example, on a computer screeen. Such a display is useful where acceptance or rejection of a signature is not to be carried out entirely on the basis of the results of calculation but is partly at the discretion of the person present at the time the signature is written or produced. The regenerated signature may also be used as a convenient method of electronic signature storage and in such circumstances authorisation of a signature may be made solely by a person present on the basis of the regenerated signature.

In reproducing the signature it is assumed that the states are arranged in a linear chain and the duration of each state is proportional to the reciprocal of transition back to that state. These durations are calculated and stored before regeneration, together with the total signature time which is the sum of the state durations.

A signature can be regenerated from the mean values of x velocity, y velocity and surface contact held by the HMM for that signature; that is in the B matrix. A test 52 is then carried out to determine whether the mean value of the contact signal of the first state of the HMM is equal to, or greater than one. If so the pen is in contact with the writing surface and a line joining a start point to a next point in the signature can be displayed, when calculated. For this purpose the means of the x velocity, y velocity for that state are retrieved and the position of the next point is calculated (operation 53) and a line joining the start point and the next point is displayed (operation 54). A test 55 now determines whether the next state shall be used to determine the next point for line construction on the basis of the stored duration of the current state. If a change is to be made an operation 56 retrieves the mean values of the next state. A test 57 now follows to decide if the current duration t of the regenerated signature is greater than the calculated total duration T and if not a return to the test 52 occurs. Each time this test indicates that the pen is not in contact with the writing surface, the x coordinate of the start of the next line to be displayed is incremented by a convenient amount (operation 58). If the test 52 indicates contact between pen and writing surface the operations 53 and 54 occur and the next line forming the regenerated signature is displayed. In this way by moving through the HMM states by means of the tests 52, 55 and 57, and the operations 53, 54, 56 and 58, lines forming the whole regenerated signature are eventually displayed.

When a signature is regenerated in this way and a break appears the position chosen to continue the signature is usually incorrect. However, a good idea of the original can be obtained, especially by a person with a little experience. The signature to be authenticated may also be modelled by an HMM and regenerated in the same way so that the two regenerated signatures can be compared by eye or automatically. The problem of breaks in regenerated signatures can be overcome when absolute position of the pen tip is also known, for example when the input tablet is in a form.which provides x and y coordinates since mean values of x and y positions are available to determine the position of the next point after a break.

As an alternative to displaying lines, the regenerated signature may be made up by displaying only the points calculated. Having described several specific embodiments of the invention, it will be clear that the invention can be put into practice in many other ways. For example other dynamic features of a signature may be used in providing variables for the pdfs and non-dynamic features such as pen position may also be included. One pen tip velocity component, only, may be used, preferably in combination with another dynamic or static feature of the signature.

The HMM model may be simplified from the Bakis form by allowing only probabilities from a state to itself or to the next state in the model and different techniques may be used to form the HMM from the sample signatures.

A known alternative to the Baum-Welch algorithm for HMM re-estimation is the Viterbi algorithm and this algorithm may therefore be used both for deriving the HMM model itself and for calculating probability in verification.

Claims

CLAIMS 1. A method of automatic signature verification comprising, for each signatory, the steps of forming a finite state machine particular to the signatures of that signatory based on corresponding measurements of a plurality of signatures from that signatory, at least one of the measurements on each signature being of a feature which is only apparent from values occurring at the time the signature was written, and using the finite state machine to verify signatures alleged to be written by the signatory.

2. A method according to Claim 1 wherein the said measurements include the velocity of the tip of a writing instrument used to write the said plurality of signatures.

3. A method according to Claim 2 wherein the said measurements include two components of the said velocity in directions normal to one another.

4. A method according to Claim 2 or 3 wherein each of the said plurality of signatures is written on a writing surface carrying parallel lines and the said velocity is determined from the rate at which the said lines are crossed by the said tip.

5. A method according to Claim 4 insofar as dependent on Claim 3 wherein the surface has two sets of parallel lines with the lines of one set normal to those of the other, and the sets are such that crossing the lines of one set by the said tip can be distinguished from such crossings of the other set.

6. A method according to any preceding claim wherein the finite state machine has between thirty and one hundred states.

7. A method according to any preceding claim wherein the finite state machine is a hidden Markov model (HMM).

8. A method according to Claim 7 wherein each state of the HMM has not more than two transitions to other states.

9. A method according to any preceding claim wherein forming the finite state machine includes deriving one signal representing each said measurement, dividing each signal into an equal number of segments, and deriving the finite state machine with a number of states dependent on the number of segments.

10. A method according to any preceding claim wherein the states of the finite state machine are defined by at least one probability density function (pdf) represented by the mean and variance of a Gaussian distribution.

11. A method according to Claims 2, 4, 9 and 10 wherein each mean and variance for each state relate to the velocity of the said tip in a said segment which relates to that state.

12. A method according to Claim 2 or any claim dependent thereon wherein one of the said measurements indicates whether the said tip is in contact with a writing surface on which the said plurality of signatures is written.

13. A method according to any preceding claim wherein forming the finite state machine comprises calculating the forward and backward probabilities from signals representing one of the plurality of signatures and an initial form of the finite state machine based on measurements from one of the plurality of signatures, and re-estimating values defining the machine from the forward and backward probabilities, repeating the step of calculating forward and backward probabilities and re-estimating values using the initial form of the finite state machine and input signals from each of a number of the plurality of signatures while accumulating re-estimated values defining the machine, deriving values re-defining the finite state machine from the accumulated values, accumulating a plurality of re-estimated values defining the finite state machine by repeatedly calculating the forward and backward probabilities from the re-defined form of the finite state machine and input signals from each of a number of the plurality of signatures, deriving values re-defining the finite state machine from the accumulated values, and continually repeating the steps of accumulating re-estimated values and calculating the forward probability and deriving values re-defining the finite state machine until the increase in the forward probability calculated for the final state falls below a predetermined threshold, the last said values derived, then defining the final version of the finite state machine.

14. A method according to any preceding claim wherein using the finite state machine to verify signatures comprises calculating the forward probabilities from the finite state machine and measurements from a trial signature to be verified which correspond to the measurements on which the finite state machine is based, and comparing the forward probability calculated for the final state with a predetermined threshold value to determine whether to accept or reject the trial signature.

15. Apparatus for signature verification comprising means for deriving and storing a finite state machine for the signature of each person whose signature is to be recognised by the apparatus, each said machine being based on corresponding measurements on a plurality of signatures from that person, and at least one of the measurements of each signature being of a feature which is only apparent from values occurring at the time the signature was written, and means for verifying signatures based on the stored finite state machines.

16. Apparatus according to Claim 15 wherein the means for deriving and storing is arranged to derive two orthogonal components of the velocity of the tip of a writing instrument used to write the said plurality of signatures and to use the said components as two of the said measurements.

17. Apparatus according to Claim 15 or 16 wherein the means for deriving and storing is arranged to derive a finite state machine for each signature which has between thirty and one hundred states.

18. A method of deriving a finite state machine modelling a signature comprising obtaining corresponding measurements from a plurality of signatures from one signatory, at least one of the measurements being of a feature of the signature which is only apparent from values occurring at the time the signature is written, and forming a finite state machine particular to that signatory based on the measurements obtained.

19. Apparatus for deriving a finite state machine modelling a signature comprising means for obtaining corresponding measurements from a plurality of signatures from one signatory, at least one of the measurements being of a feature of the signature which is only apparent from values occurring at the time the signature is written, and means for forming a finite state machine particular to that signatory based on the measurements obtained.

20. A method of verifying a signature comprising obtaining measurements from a signature to be verified, at least one of the measurements being of a feature of the signature which is only apparent from values occurring at the time the signature is written, determining the probability that the said measurέtoents could have been generated from a stored finite state machine derived from at least one genuine signature, and accepting or rejecting the signature on the basis of the probability determined.

21. Apparatus for verifying a signature comprising means for obtaining measurements from a trial signature to be verified, at least one of the measurements being of a feature of the signature which is only apparent from values occurring at the time the signature is written, means for determining the probability that the said measurements could have been generated from a stored finite state machine derived from at least one genuine signature, and means for accepting or rejecting the signature on the basis of the probability determined.

22. Apparatus according to Claim 21 wherein the means for determining the said probability calculates the forward probability of the final state of the machine from the stored finite state machine and the measurements from the trial signature, and the means for accepting or rejecting the signature uses the forward probability calculated for the final state to determine whether the trial signature is to be accepted or rejected.

23. A method of verifying a signature comprising regenerating a signature from a finite state machine model of the signature, and comparing the regenerated signature with a trial signature to be verified in rejecting or accepting the trial signature.

24. Apparatus for use in signature verification comprising means for storing data defining the states of a finite state machine model of a signature, and means for generating a display of a representation of the signature from at least some of the said data, whereby a signature to be verified can be compared with the said representation.

25. A method of regenerating a signature from a stored finite state machine model thereof, comprising retrieving a series of values from the stored model, each value being at least one of a number of values defining states of the machine, and the values in the series being retrieved from successive states of the machine, and generating a series of visible points and/or lines in a representation of the signature from the series of values.

26. A method according to Claim 24 wherein the values in the series comprise a series of mean velocities of a pen tip writing an authentic signature and generating the visible points and/or lines by calculating point positions from the mean velocities.

27. A method according to Claim 25 wherein the number of visible points and/or lines derived from each machine state depends on the reciprocal of the probability of transition from that state back to the same state.

28. Apparatus for regenerating a signature from a stored finite state machine model thereof, comprising means for retrieving a series of values from the stored model, each value being at least one of a number of values defining states of the machine, and the values in the series being retrieved from successive states of the machine, and means for generating a series of visible points and/or lines in a representation of the signature from the series of values.

29. An apparatus or method of signature recognition substantially as hereinbefore described.