WO2017093294A1

WO2017093294A1 - Pulse oximetry and contactless patient biometric monitoring system

Info

Publication number: WO2017093294A1
Application number: PCT/EP2016/079240
Authority: WO
Inventors: John Cronin; Seth Melvin CRONIN
Original assignee: Koninklijke Philips N.V.
Priority date: 2015-11-30
Filing date: 2016-11-30
Publication date: 2017-06-08

Abstract

A method for monitoring patient biometrics involves acquiring biometric data through the use of a contactless monitoring system and the acquisition of pulse oximetry data by using a pulse oximeter. The acquired pulse oximetry data is associated with the acquired biometric data; both data sets are provided to a patient monitor. The acquired biometric data is compared with biometric data previously stored in the patient monitor. In the event that acquired biometric data does not match the previously stored biometric data, the acquired biometric data is calibrated using a reference data. The calibrated biometric data is then compared with the previously stored biometric data and the previously stored biometric data in the patient monitor is replaced with the acquired biometric data should the quality of the latter be superior to that already maintained in a database.

Description

PULSE OXIMETRY AND CONTACTLESS PATIENT BIOMETRIC MONITORING

SYSTEM

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims the priority benefit of U.S. provisional application number 62/261,299 filed November 30, 2015, the disclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention is generally related to contactless monitoring systems. More specifically, the present invention is related to contactless monitoring and pulse oximetry.

Description of the Related Art

Present day fingerprinting systems are based on wet-ink method of fingerprint acquisition whereby flat images of a three dimensional object— the finger— are acquired. In the process of gathering these flat image, the fingers are pressed against paper or a platen and distortions of the three-dimensional (3D) fingerprints are created. Such distortions may not be uniform throughout a single fingerprint or even between administering operators due to changes in pressure and other factors. Issues from too much or too little ink, smearing, worn fingerprints, and interfering contaminants also make traditional fingerprinting less than ideal.

This is in addition to the requirement of an operator with special training who collects the fingerprints. Collections of the same individual may be inconsistent between operators as a result. Wet-ink collections, too, require materials that must be replaced by operators and involves clean-up for both operators and subjects alike.

Live scan devices address the material and cleanup aspects of wet-ink collections, but can introduce issues of their own. Such devices can suffer from interference from latent fingerprints left on the scanner platen, high failure rates due to variations in skin condition— dry, moist, oily, or worn skin— and variation in prints due to varying amounts of pressure during the scan.

Contactless monitoring systems use optical and electrical elements to obtain biometric data from a patient without needing physical contact with the patient. Contactless fingerprint scanning systems, for example, have a number of potential advantages over conventional fingerprinting systems such as improved image quality, faster image acquisition, nor the need for monitoring of the acquisition process. The spreading of contaminants among subjects is likewise reduced.

U.S. patent number 6,643,531 discloses a system having a finger grip device that has incorporated therein both an oximeter and a fingerprint sensor. U.S. patent publication number discloses a pulse oximeter that includes a fingerprint reader to obtain a fingerprint image of a patient or care-giver or both. And U.S. patent publication number 2006/0074280 discloses a patient identification system with a portable processor, a first sensor, and a second sensor, wherein the first sensor can be a finger clip having any one of a pulse oximeter, fingerprint identifier, or other biometric sensor.

There exists a need in the art for accurate fingerprint identification using contactless fingerprint scanning methods.

SUMMARY OF THE PRESENTLY CLAIMED INVENTION

In a claimed embodiment of the present invention, a method for monitoring patient biometric is disclosed. Through the method, biometric data is acquired using a contactless monitoring system. Pulse oximetry data is acquired using a pulse oximeter. The acquired pulse oximetry data is associated with the acquired biometric data and transmitted to a patient monitor. The acquired biometric data is compared with biometric data previously stored in the patient monitor. The acquired biometric data is then calibrated using reference data if the acquired biometric data does not match the previously stored biometric data. If the calibrated biometric data provides a better match to the reference data than the previously stored biometric data, then the acquired biometric data provides a better match to the reference data than the previously stored biometric data.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a further understanding of the invention, are incorporated herein to illustrate embodiments of the invention. Along with the description, they also serve to explain the principle of the invention. In the drawings:

FIG. 1 illustrates a contactless monitoring system.

FIG. 2A illustrates a diagram of a patient monitor. FIG. 2B illustrates a patient monitor that communicates with the contactless monitoring system.

FIG. 3 is a flowchart illustrating a methodology involving the patient monitor that communicates with the contactless monitoring system.

FIG. 4 is a flowchart illustrating a further methodology of the present invention involving retrieved sensor and patient data.

FIG. 5 is a flowchart that illustrates a further methodology of the present invention and involving the receipt of sensor data and patent ID information from the patient monitor.

FIG. 6 illustrates a flowchart corresponding to a methodology involving the polling of contactless sensors.

FIG. 7 illustrates a table showing various information including fingerprints corresponding to different patients as might be found in a database of the present invention.

DETAILED DESCRIPTION

A contactless monitoring system uses sensors to monitor patient biometrics. A pulse oximeter and fingerprint reading sensors send pulse oximetry and fingerprint data to a patient monitor thus allowing the patient monitor to send a verified plethysmograph and patient identity to the contactless monitoring system. The contactless monitoring system allows for correlation over time with the pulse oximeter such that the contactless system calibrates. With the foregoing in mind, the following are definitions of terms as used in the various embodiments of the present invention.

The term "database" as used herein refers to a collection of data and information organized in such a way as to allow the data and information to be stored, retrieved, updated, and manipulated and to allow them to be presented into one or more formats such as in table form or to be grouped into text, numbers, images, and audio data. The database typically resides in computer memory that includes various types of volatile and non-volatile computer memory. "Database" also refers to conventional databases that may reside locally or that may be accessed from a remote location, e.g., remote network servers.

The term "database" as used herein may also refer to a segment or portion of a larger database, which in this case forms a type of database within a database. Memory wherein the database resides may include high-speed random access memory or non-volatile memory such as magnetic disk storage devices, optical storage devices, and flash memory. Memory where the database resides may also comprise one or more software for processing and organizing data received by and stored into the database.

As used herein, the term "contactless" may refer to varying levels of contact between the fingerprint, for example, and the biometric sensor. For example, the term

"contactless" may refer to a case in which the user places their hands directly on the biometric device, but the fingertips do not make contact with the sensor.

Alternatively, the entire process is completely contactless where the user might stand at a distance from the sensor and puts their hand in the air to have their fingerprint scanned. This type of contactless fingerprint acquisition is typically considered to be true touchless biometrics. Touchless biometrics can provide the benefits of faster subject processing and higher quality data gathering while addressing the known issues that plagued current generation of contactless biometric acquisition techniques.

FIG. 1 illustrates a contactless monitoring system 100. The system uses one or more sensors to monitor patient biometrics. A pulse oximeter 104 and fingerprint reading sensors sends pulse oximetry and fingerprint data to a patient monitor 102, thus allowing the patient monitor 102 to send a verified plethysmograph 106 and patient identity to the contactless monitoring system 100.

The contactless monitoring system 100 of the present invention allows for correlating the identification data with the pulse oximeter data acquired by the pulse oximeter worn by the patient 1 14. Over time, the contactless system 100 calibrates with respect to individual users such that the contactless system 100 may eventually be able to operate alone and be the only necessary component of the overall system. The correlation level may determine if and when a patient can remove the worn pulse oximeter.

FIG. 2A illustrates a diagram of a patient monitor. The patient monitor includes a central bus that connects a memory component 218 with a patient ID database 220, patient monitor database 222, patient ID software 224, and synchronization software 226. Also connected by way of central bus is display 200, power 202, processor 204,

communication unit 206, user interface 208, and signal processor 210. Signal processor 210 receives data and information from a plurality of sensor components including pulse oximeter 212, fingerprint reader 214, and input sensor n 216.

FIG. 2B illustrates a patient monitor that communicates with the contactless monitoring system. Such communication occurs via communication unit 234. The contactless monitoring system includes a central bus connecting display 228, power 230, processor 232, and communication unit 234. Bus further connects user interface 236 and signal processor 238, the latter of which is capable of receiving input from a first input sensor 240, a second input sensor 242, and input sensor n 244. The central bus also connects to a memory component 246 containing contactless monitor database 248, calibration software 250, and instrument handler 252.

FIG. 3 is a flowchart illustrating a methodology involving the patient monitor that communicates with the contactless monitoring system. In the first step of FIG. 3, the system polls the fingerprint reader for fingerprint data (step 300). The polled fingerprint data is then matched with fingerprint data stored in the patient ID database (step 302). The process determines if there is a fingerprint match (step 304). If there is a match, the system then polls the first sensor (pulse oximeter) for sensor data (step 306). Any polled sensor data is then stored in the patient monitor database with the patient ID (step 308).

If there is no fingerprint match in step 304, the system queries as to whether there is any fingerprint having undergone the aforementioned three iterations (step 310). If the number of iterations is less than three with respect to any fingerprint, then the process loops back to step 300. If three iterations have already been performed, the user is then prompted to input patient ID data into the patient monitor (step 312). Patient identification software allows the fingerprint reader to be used as a way of identifying the patient from which the sensor data is connected. This allows the contactless monitoring system to be more accurate by storing patient ID with sensor data.

FIG. 4 is a flowchart illustrating a further methodology of the present invention involving retrieved sensor and patient data. In the first step of FIG. 4, sensor data and the corresponding patient ID are retrieved from the patient monitor database (step 400). The retrieved sensor data and patient ID are then transmitted to the contactless monitoring system (step 402). The received sensor data is calibrated by the contactless monitoring system (step 404) after which the calibrated sensor data is sent to the patient monitor database (step 406). The received calibrated sensor data is then stored in the patient monitor database (step 408). Software that allows the patient monitor to synchronize sensor data with the contactless monitoring system and further allows the patient monitor to receive calibrated data from the contactless monitoring system is stored with the sensor data and patient ID on the patient monitor.

FIG. 5 is a flowchart that illustrates a further methodology of the present invention and involving the receipt of sensor data and patent ID information from the patient monitor. In the first step of FIG. 5, the sensor data and patient ID are received from the patient monitor (step 500). Uncalibrated sensor data is retrieved from the contactless monitor database (step 502). The uncalibrated sensor data is then matched with the sensor data from the patient monitor (step 504).

If there is a match, the contactless sensor is not calibrated and the process ends (step 506). If there is no match found in step 504, at least one calibration coefficient is calculated (step 508). The calculated calibration coefficient is applied to uncalibrated data to generate calibrated data (step 510). The process then determines if the calibrated data is a better match for the sensor data than the uncalibrated data (step 512). If so, the calibrated data is stored in the contactless monitor database (step 514). If the calibrated data is not a better match for the sensor data than the uncalibrated data, the process loops back to step 506.

Software allows the contactless monitor system to calibrate at least one of its sensors by extracting uncalibrated sensor data from said sensor and comparing it to the values of a calibrated sensor such as a pulse oximeter placed on a patient finger. If the data matches, no calibration is needed. If the data does not match, the software calculates a calibration coefficient, calibrates the data, and then tests the calibrated data to determine if it is a better match than the uncalibrated data.

FIG. 6 illustrates a flowchart corresponding to a methodology involving the polling of contactless sensors. In the method of FIG. 6, the contactless sensors are polled in the first step for the purpose of acquiring uncalibrated sensor data (step 600). If newly- acquired uncalibrated sensor data is available, the uncalibrated sensor data is then sent to and stored in the contactless monitor database (step 602). The process then determines whether the patient monitor is connected (step 604). If connected, the calibration software that, in a one possible embodiment, resides in the patient monitor performs a calibration of the newly- acquired sensor data (step 606). If there is no connection, then polling continues in the final step of the method (step 608) and returns to previous step 602.

FIG. 7 illustrates a table showing various information including fingerprints corresponding to different patients as might be found in a database of the present invention. The table as illustrated in FIG. 7 shows, from left to right, the following patient information:

"Patient ID" 700,

"Name" 702,

"Age" 704,

"Height" 706,

"Weight" 708,

"Gender" 710, "Condition" 712, and

"Fingerprint" 714.

The conditions listed such as "Sleep Apnea," "Asthma," and "COPD" are conditions that can be monitored using a pulse oximeter.

In one embodiment involving the acquisition of the patient biometric data using the contactless biometric monitoring system of the present invention, one or more cameras are used to acquire contactless biometric data. The one or more cameras may be also used in conjunction with one or more other contactless biometric identification systems. The one or more cameras may be calibrated before acquiring a three-dimensional fingerprint image.

The camera calibration is preferably performed off-line and before the biometric data acquisition step. Algorithms that compute the intrinsic and extrinsic parameters of a two-camera setup may be utilized. For example, the object used for calibration can be a checkerboard pattern, which is captured in different positions. The obtained images are then processed using an algorithm such as a corner detection algorithm.

During the fingerprint data acquisition step, the finger may be placed on a placeholder with a fixed distance to the cameras in order to allow adjustment of the camera lens focus. The three-dimensional fingerprint data acquisition may be performed using a projector or various types of LEDs, such as white or blue LED to enhance the visibility of fingerprint features such as ridge patterns, along with one or more cameras. In one embodiment, the acquisition set up is such that the finger can be positioned at different orientations.

Following biometric data acquisition, and specifically in the case of fingerprint data, fingerprint reference points are extracted and matched by cross-correlation. After refining the pairs of reference points, the process is followed by three-dimensional surface estimation, image wrapping, and texture enhancement post-processing routines. The methodology preferably allows for the acquisition of pairs of corresponding points on multiple images of the contactless fingerprints. Further, one or more correlation techniques that permit increasing the number of corresponding points estimated in contactless fingerprint images are used.

For example, an individual may stop in front of a scanner and hold up a hand with the palm facing the device. The scanner sensor locates the hand and collects the finger imagery. Alternatively, the process allows the use of a full-hand, multi-finger, and/or rolled equivalent capture. The collected imagery is processed for storage or matching to existing databases. High resolution cameras, too, may be used to provide more detailed information. As a result, high resolution 3D prints might be captured without the need to touch a platen in the device. The high resolution cameras that are used to capture the biometric region of the hand may be configured to produce at least 1000 ppi images although the present invention can operate at a resolution lower than 1000 ppi. Notwithstanding, more minutiae of the friction ridges in the biometric region of the hand can be captured at resolutions around 1000 ppi thereby resulting in a more detailed image.

It should also be noted that the precise number of high resolution cameras used to capture the biometric region of the hand is not fixed. The greater the pixel resolution of a camera, the larger the area that can be captured in focus at 1000 ppi. Thus, fewer cameras may be used if they are of higher resolution as long as they can still collectively capture all surfaces of the biometric region. The capabilities and physical characteristics of the cameras and lenses used will also help determine how far the cameras should be from each other, as well as how far they should be from the respective sections of the biometric region. The types of cameras and lenses will then in turn dictate the necessary dimensions of the enclosure to accommodate the cameras.

In a preferred embodiment of the present invention, a freely posed hand is captured including all five fingerprints, finger flats, palm prints, and hand geometry at high resolution (i.e., 1000 ppi) in a few seconds. The system would then produce ink-like prints of those biometrics that can be matched by a legacy AFIS database. Necessary precautions are taken to ensure movement of the subject fingers does not impact scan quality.

In a still further embodiment of the present invention, 2D fingerprint images may be acquired to reconstruct 3D fingerprints. In such an embodiment, the invention would utilize a shape from shading approach whereby the 2D and 3D images can also be

simultaneously matched and used for the performance improvement. Since these 2D images are acquired at-a-distance, the contrast of the intensity between valley and ridge is low.

Preferably, image contrast enhancement is performed before subjecting the fingerprint images to a filter-based fingerprint enhancement algorithm such as Homomorphic filtering for contrast improvement.

Various 2D minutiae extraction algorithms may be utilized including but not limited to the NBIS's MINDTCT function. The fingerprint template generated from the MINDTCT identifies the minutiae as [x, y, _, q], where q is the quality of the minutia. A maximum score of all reference minutia pair is preferably chosen from the final score of two templates (similar to as also used for generating scores using 3D minutiae). Preferably, multiple images of the same finger, under different illuminations, are simultaneously acquired and used for generating 2D fingerprint matching score for every user or subject.

The acquired finger images are acquired using contactless imaging setup and the average/expected distance between the camera and the finger is approximately 10 cm. In one embodiment, multiple symmetrically distributed LED illuminators may be used to acquire fingerprint data. An illumination sequence and image acquisition timing process may be synchronized and controlled by a computer. The position of LEDs on the acquired images is preferably calibrated as a part of said sequence. Acquired images may then be down sampled after edge detection, boundary scanning, and similar post processing routines to extract a given region of interest (ROI) measured in pixels.

The present invention is not intended to be restricted to the several exemplary embodiments of the invention described above. Other variations that may be envisioned by those skilled in the art are intended to fall within the disclosure.

Claims

CLAIMS:

1. A method for acquiring patient biometric data, the method comprising:

acquiring biometric data using a contactless monitoring system; acquiring pulse oximetry data using a pulse oximeter;

associating the acquired pulse oximetry data with the acquired biometric data; transmitting the associated acquired pulse oximetry and acquired biometric data to a patient monitor;

comparing the acquired biometric data with a biometric data previously stored in the patient monitor;

calibrating the acquired biometric data using a reference data if the acquired biometric data does not match the previously stored biometric data;

replacing the previously stored biometric data in the patient monitor with the acquired biometric data when the acquired biometric data provides a better match to the reference data than the previously stored biometric data.

2. The method of claim I, wherein the contactless monitoring system collects fingerprint data.

3. The method of claim 2, wherein the contactless monitoring system is touchless.

4. The method of claim 2, wherein the contactless monitoring system involves one or more cameras.

5. The method of claim I, further comprising calibrating the contactless monitoring system prior to acquiring biometric data.

6. The method of claim 5, wherein the contactless monitoring system is a two- camera system and calibrating the contactless monitoring system involves presenting a checkerboard pattern, captured images of which are processed using a corner detection algorithm.

7. The method of claim 2, wherein the contactless monitoring system operates in conjunction with a series of LED projectors that project white or blue light that enhances the visibility of fingerprint features captured as biometric data.

8. The method of claim 7, wherein the enhanced fingerprint features are ridge data.

9. The method of claim 6, wherein the contactless monitoring system extracts and cross-correlates fingerprint data previously acquired as the biometric data.

10. The method of claim 9, further comprising performing three-dimensional surface estimation of the fingerprint data.

1 1. The method of claim 9, further comprising performing image wrapping on the fingerprint data.

12. The method of claim 9, further comprising performing texture enhancement on the fingerprint data.

13. The method of claim 9, wherein the cross-correlation increases the number of corresponding points analyzed as a part of the fingerprint data.

14. The method of claim 1, wherein the biometric data is fingerprint data acquired in conjunction with data about a user hand.

15. The method of claim 14, wherein the data acquired from the user hand is rolled hand data.

16. The method of claim 14, wherein the data acquired from the user hand is multi-finger data.

17. The method of claim 14, wherein the data acquired from the user hand is full hand data.