US8320998B2 - Vein imaging apparatus, positional displacement interpolation method, and program - Google Patents
Vein imaging apparatus, positional displacement interpolation method, and program Download PDFInfo
- Publication number
- US8320998B2 US8320998B2 US12/775,211 US77521110A US8320998B2 US 8320998 B2 US8320998 B2 US 8320998B2 US 77521110 A US77521110 A US 77521110A US 8320998 B2 US8320998 B2 US 8320998B2
- Authority
- US
- United States
- Prior art keywords
- vein
- positional displacement
- light
- image
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/48—Other medical applications
- A61B5/4887—Locating particular structures in or on the body
- A61B5/489—Blood vessels
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/14—Vascular patterns
- G06V40/145—Sensors therefor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
- A61B2562/02—Details of sensors specially adapted for in-vivo measurements
- A61B2562/0233—Special features of optical sensors or probes classified in A61B5/00
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
- A61B2562/04—Arrangements of multiple sensors of the same type
- A61B2562/046—Arrangements of multiple sensors of the same type in a matrix array
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/14—Vascular patterns
Definitions
- the present invention relates to a vein imaging apparatus, a positional displacement interpolation method, and a program.
- Biometric individual authentication is highly important technology for protecting the rights in the future network society.
- biometric individual authentications particularly attract attention as a technique for protecting a field that may not be protected by encryption alone.
- a biometric individual authentication using fingerprint and iris may not solve the issue of counterfeiting.
- an individual authentication technique using a part of the vein pattern that may not be easily imaged from the outside is expected to be a biometric individual authentication for the next generation because of accuracy of determination and difficulty for counterfeiting and spoofing.
- Japanese Patent Application Laid-Open No. 2004-296531 describes a technique for detecting the amount of positional displacement caused by thermal expansion and directly controlling the position of an optical component based on the amount of positional displacement.
- a vein imaging apparatus including a lens array including a plurality of light-receiving lenses disposed in an array, a near-infrared light emission source which is arranged at an end of the lens array and emits a near-infrared light to a part of a living body, an imaging element including a vein image data generation region for generating image data of a vein based on the near-infrared light that was condensed by the lens array and that was scattered in the living body and transmitted through the vein, and a positional displacement detection data generation region that includes a shielded section in which pixels are shielded from the light and an opening section in which pixels are not shielded from the light, and generates data for detecting a positional displacement that is used to detect, based on the light received via the opening section, variation in an image focus position according to an imaging temperature, where a plurality of pixels are assigned to one of the light-receiving lenses, an image focus position detection unit for detecting
- the image focus position detection unit detects the image focus position at the imaging temperature based on the data for detecting positional displacement obtained from the positional displacement detection amount data generation region of the imaging element. Further, the positional displacement amount estimation unit estimates the amount of displacement of the image focus position at the imaging temperature based on the image focus position detected by the image focus position detection unit. Further, the pixel selection unit selects based on the amount of displacement, a pixel for generating the vein image data used to generate the vein image from among the plurality of pixels corresponding to one of the light-receiving lenses.
- the vein imaging apparatus preferably further includes a drive control unit for performing drive control of at least one of the near-infrared light emission source and the imaging element.
- the drive control unit preferably performs drive control of the near-infrared light emission source and/or the imaging element based on the amount of displacement transmitted from the positional displacement amount estimation unit.
- the vein imaging apparatus may further include a vein pattern extraction unit that extracts a vein pattern from the vein image generated using the vein image data, and a vein image interpolation unit that performs interpolation processing on the vein image, from which the vein pattern is extracted, based on the amount of displacement estimated by the positional displacement amount estimation unit.
- the vein image interpolation unit may perform interpolation processing on the vein image by using the vein image data obtained from the pixels, located around a reference unit region serving as a reference unit for pixel selection by the pixel selection unit, from among the plurality of pixels corresponding to one of the light-receiving lenses.
- the pixel selection unit may select a pixel included in a region in which the sum of the light quantity detected by the pixel is the largest from among the plurality of pixels corresponding to one of the light-receiving lenses.
- the vein imaging apparatus may further include a temperature estimation unit for estimating the imaging temperature based on the amount of displacement obtained from the positional displacement amount estimation unit.
- the vein imaging apparatus may further include a warning unit that gives a warning when the amount of displacement output from the positional displacement amount estimation unit or the imaging temperature output from the temperature estimation unit is equal to or more than a predetermined threshold value.
- a positional displacement interpolation method including the steps of detecting a image focus position at imaging temperature based on data for detecting positional displacement obtained from a positional displacement detection data generation region of a vein imaging apparatus including a lens array including a plurality of light-receiving lenses disposed in an array, a near-infrared light emission source which is arranged at an end of the lens array and emits a near-infrared light to a part of a living body, and an imaging element including a vein image data generation region for generating image data of a vein based on the near-infrared light that was condensed by the lens array and that was scattered in the living body and transmitted through the vein, and the positional displacement detection data generation region that includes a shielded section in which pixels are shielded from the light and an opening section in which pixels are not shielded from the light, and generates data for detecting a positional displacement that is used to detect, based on the light received via the opening
- a positional displacement can be corrected between an imaging element and an optical component caused by thermal expansion, and the apparatus can be produced smaller.
- FIG. 1 is a block diagram for illustrating a configuration of a vein imaging apparatus according to a first embodiment of the present invention
- FIG. 2 is an explanatory diagram for illustrating a vein imaging apparatus according to the embodiment
- FIG. 3 is an explanatory diagram for illustrating a vein imaging apparatus according to the embodiment
- FIG. 4A is an explanatory diagram for illustrating an image taken by a microlens array
- FIG. 4B is an explanatory diagram for illustrating an image taken by a microlens array
- FIG. 5A is an explanatory diagram for illustrating an imaging element according to the embodiment.
- FIG. 5B is an explanatory diagram for illustrating an imaging element according to the embodiment.
- FIG. 6A is an explanatory diagram for illustrating an imaging element according to the embodiment.
- FIG. 6B is an explanatory diagram for illustrating an imaging element according to the embodiment.
- FIG. 6C is an explanatory diagram for illustrating an imaging element according to the embodiment.
- FIG. 7 is an explanatory diagram for illustrating an imaging element according to the embodiment.
- FIG. 8A is an explanatory diagram for illustrating a pixel selection unit according to the embodiment.
- FIG. 8B is an explanatory diagram for illustrating a pixel selection unit according to the embodiment.
- FIG. 8C is an explanatory diagram for illustrating a pixel selection unit according to the embodiment.
- FIG. 9 is an explanatory diagram for illustrating a method for obtaining data from a particular pixel
- FIG. 10 is an explanatory diagram for illustrating a method for obtaining data from a particular pixel
- FIG. 11 is a flow diagram for illustrating a positional displacement interpolation method according to the embodiment.
- FIG. 12 is a block diagram for illustrating a configuration of a vein imaging apparatus according to a second embodiment of the present invention.
- FIG. 13 is a block diagram for illustrating a hardware configuration of a vein imaging apparatus according to each embodiment of the present invention.
- CMOS Complementary Metal Oxide Semiconductor
- a microlens array i.e., a type of lens array and large sensor are used, which allows the vein imaging apparatus to be produced thinner.
- An object of each embodiment of the present invention is to realize an automatic correction of deterioration caused by an environmental temperature and decreased imaging performance that may be caused by the environmental temperature, mechanical precision, and displacement, which are the issues of the vein imaging apparatus using a flat, large sensor device such as MLA and TFT.
- a microlens array is a device suitable for reading a vein pattern located several millimeters below the skin.
- a microlens arranged on the microlens array may be a lens for reducing the position of an object, a vein, by one half.
- the size of an image focused by the microlens is one-half the actual size. Therefore, when, for example, one microlens corresponds to eight-by-eight pixels, only with four-by-four pixels in the center of the eight-by-eight pixels, the combined image of the MLA is obtained.
- a fixed pixel selection method used in related art has issues of deterioration in the image quality and the like because there occur discontinuity and distortion at a boundary of a microlens.
- a microlens array formed with a plastic resin is known to have a displacement of one pixel or more, which may cause an issue that taken images may be the greatly deteriorated and an authentication may be performed erroneously unless any measure is taken.
- the below-described vein imaging apparatus aims to automate a correction by means of image processing and optimum sampling of each member at each temperature, by using pixels surrounding an imaging region of the imaging element for detection of displacement between members.
- FIG. 1 is a block diagram for illustrating the configuration of the vein imaging apparatus according to the present embodiment.
- FIG. 2 is a plan view of the vein imaging apparatus according to the present embodiment.
- FIG. 3 is a cross sectional diagram taken along line A-A of FIG. 2 .
- the vein imaging apparatus 10 includes, for example, three units, i.e., an imaging unit, an image processing unit, and an authentication processing unit, and further includes a storage unit 139 .
- the imaging unit performs processing of imaging a part of a living body (for example, a finger). As shown in FIG. 1 , this imaging unit mainly includes, for example, a microlens array 101 , a near-infrared light emission source 105 , an imaging element 109 , and a drive control unit 121 .
- the image processing unit performs processing when obtaining picture data (image data) related to a vein that is generated by the imaging unit, and performs various image processings on the obtained image data, and thus generates image (vein image) of veins which are in the inside of the living body.
- this image processing unit mainly includes, for example, a pixel data dividing unit 123 , a image focus position detection unit 125 , a positional displacement estimation unit 127 , a warning unit 129 , a pixel selection unit 131 , and a vein image interpolation unit 133 .
- the authentication processing unit performs authentication processing of the vein image generated by the image processing unit. As shown in FIG. 1 , this authentication processing unit mainly includes, for example, a vein pattern extraction unit 135 and an authentication unit 137 .
- the microlens array (MLA) 101 condenses near-infrared light, which was emitted from the later-described near-infrared light emission source 105 to a part of a living body and transmitted through a vein inside the living body (which is also referred to hereinafter as vein transmitted light), onto the later-described imaging element 109 .
- This microlens array 101 includes a plurality of light-receiving lens as described later.
- the microlens array 101 is made of, for example, a material that is more likely to be affected by heat than glass material. By using such material, it becomes possible to inexpensively bulk-produce microlens array of any size by means of, for example, molding.
- An example of such material that is more likely to be affected by heat than glass material includes a plastic resin.
- the near-infrared light emission source 105 emits near-infrared light having a predetermined wavelength band onto a part of a living body placed on the vein imaging apparatus 10 . Because the near-infrared light has characteristics that it is well transmitted through body tissues and absorbed by hemoglobin (reduced hemoglobin) in blood, if the near-infrared light is emitted on the finger, palm or back of a hand, veins distributed inside the finger, palm or back of the hand appear as a shadow in an image. The shadow of veins that appears in an image is called a vein pattern.
- the near-infrared light emission source 105 emits near-infrared light having a wavelength of about 600 nm to 1300 nm or, preferably, about 700 nm to 900 nm.
- the wavelength of the near-infrared light emitted by the near-infrared light emission source 105 is less than 600 nm or more than 1300 nm, the percentage of light that is absorbed by hemoglobin in blood decreases, and it becomes difficult to obtain a suitable vein pattern. Also, if the wavelength of the near-infrared light emitted by the near-infrared light emission source 105 is about 700 nm to 900 nm, the near-infrared light is specifically absorbed by both deoxygenated hemoglobin and oxygenated hemoglobin, and it is therefore possible to obtain a suitable vein pattern.
- a near-infrared light emission source 105 a light emitting diode (LED) may be used, for example. Further, in stead of using a light emitting diode having the above wavelength band, a combination of a light emitting diode capable of emitting light containing the above wavelength band and a filter for optically limiting the band of emitted light may be used. Further, the near-infrared light emission source 105 may be combined with an optical quantity adjustment filter that adjusts the distribution of light emitted by the light source.
- LED light emitting diode
- emission timing of the near-infrared light and the intensity of the emitted near-infrared light and the like are controlled by the later-described drive control unit 121 .
- the imaging element 109 has an imaging surface with a plurality of pixels 111 arranged in a lattice structure, and generates vein image data with near-infrared light based on vein transmitted light focused by the microlens array 101 .
- the imaging element 109 can be used, for example, a CCD-image sensor, a CMOS-image sensor, a Thin Film Transistor (TFT)-image sensor, and the like.
- the imaging element 109 outputs the generated vein image data. Further, the imaging element 109 may record the generated vein image data in the later-described storage unit 139 .
- the plurality of pixels 111 are assigned to one light-receiving lens of the microlens array 101 as described later. Therefore, in the vein imaging apparatus 10 according to the present embodiment, the near-infrared light (vein transmitted light) condensed by the one light-receiving lens is imaged with the plurality of pixels 111 .
- the Pixel scanning timing and the like of this imaging element 109 are controlled by the later-described drive control unit 121 .
- the drive control unit 121 can be realized by, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like.
- the drive control unit 121 performs drive control of the near-infrared light emission source 105 and the imaging element 109 .
- the drive control unit 121 adjusts the driving control of the near-infrared light emission source 105 and the imaging element 109 based on information related to the amount of positional displacement transmitted from the later-described positional displacement amount estimation unit 129 . More specifically, the drive control unit 121 performs driving control based on a predetermined synchronization signal and the like.
- the drive control unit 121 controls from which pixels 111 the information is obtained, and controls the scanning timing of the pixels 111 constituting the imaging element 109 .
- the drive control unit 121 performs drive control related to the emission timing and the emission intensity of the near-infrared light also to the near-infrared light emission source 105 .
- the drive control unit 121 performs drive control along a certain direction of the imaging element 109 as such drive control of the imaging element 109 in which pixels along the certain direction are controlled in units of the number of pixels.
- the imaging element 109 is considered to include, for example, seven pixels.
- the drive control unit 121 performs the driving control by dividing the pixels into seven groups in a direction along this cutting-plane line.
- the drive control unit 121 can reference various parameters and databases recorded in the later-described storage unit 139 .
- the microlens array 101 of the vein imaging apparatus 10 includes, for example, the plurality of microlenses 103 , i.e., light-receiving lens as shown in FIG. 2 , and the microlenses 103 are arranged in a lattice-pattern on a predetermined board.
- Each microlens 103 guides vein transmitted light that entered the microlens 103 through a incidence plane to the imaging element 109 (specifically, the pixels 111 of the imaging element 109 ), which is described later, as shown in FIG. 3 , for example.
- the microlens array 101 is a lens array with a small curvature of field and with no distortion in the depth direction, and therefore suitable image data can be obtained by using such a microlens array 101 .
- the focal position of each microlens 103 constituting the microlens array 101 is set at the position of a vein layer where a vein V exists, which is an imaging target of the vein imaging apparatus 10 .
- Human skin is known to have a three-layer structure including an epidermis layer, a dermis layer and a subcutaneous tissue layer, and the above-described vein layer exists in the dermis layer.
- the dermis layer is located at about 0.1 mm to 0.3 mm below the finger surface and has a thickness of about 2 mm to 3 mm.
- the focal position of the microlens 103 at the position where the dermis layer exists (e.g. at the position that is about 1.5 mm to 2.0 mm below the finger surface), it is becomes possible to efficiently condense the light transmitted through the vein layer.
- the number of the microlenses 103 disposed in the microlens array 101 according to the embodiment is not limited to the example shown in FIG. 2 .
- the number of the microlenses 103 disposed in the microlens array 101 according to the embodiment may be set freely according to the size of a living body to be imaged, the size of the imaging element 109 or the like.
- a plurality of light emitting diodes which are an example of the near-infrared light emission source 105 , are arranged at the opposed ends of the microlens array 101 as shown in FIG. 2 , for example.
- the ends at which the light emitting diodes are arranged preferably correspond to the upper end and the lower end of a part of a living body (which is a finger FG in the example shown in FIGS. 2 and 3 ).
- the number of the near-infrared light emission sources 105 according to the embodiment is not limited to the example shown in FIG. 2 , and it may be set freely according to the size of the microlens array 101 , an emission area of the near-infrared light emission sources 105 or the like.
- a directivity control plate 107 is placed between the microlens array 101 and the near-infrared light emission source 105 as shown in FIGS. 2 and 3 , for example.
- This directivity control plate 107 controls the directivity of direct light 12 that is emitted from the near-infrared light emission sources 105 in such a way that the direct light 12 does not directly enter the microlenses 103 of the microlens array 101 .
- the near-infrared light that is emitted from the near-infrared light emission sources 105 propagates upward to the surface of the finger FG and enters the finger FG as the direct light 12 as shown in FIG. 3 , for example.
- the direct light 12 that entered the finger FG propagates while scattering in all directions.
- a part of such scattered light travels as rear scattering light 13 through the above-described vein layer from the backside to the finger surface, and passes through the vein V on its way.
- the vein transmitted light that passed through the vein enters the respective microlenses 103 constituting the microlens array 101 .
- the directivity control plate 107 is placed at the boundary between the adjacent microlenses 103 .
- This directivity control plate 107 makes it possible to control of the directivity of the vein transmitted light, and the light that entered each microlens 103 can be separated from the adjacent microlenses 103 . Accordingly, in the vein imaging apparatus 10 according to the embodiment, it becomes possible to select the vein transmitted light to be condensed on the imaging element 109 (specifically, the pixel 111 ).
- FIG. 4A and FIG. 4B are explanatory diagram for illustrating images taken by the microlens array.
- a taken image is such a image whose up and down side and left and right side are respectively reversed from an original image as shown in FIG. 4A , for example.
- a plurality of pixels 111 are assigned to one light-receiving lens (microlens 103 )
- an image whose up and down side and left and right side are reversed is created for all the pixels 111 that are assigned to one microlens 103 .
- nine (3 ⁇ 3) pixels 111 are assigned to one microlens 103 as shown in FIG. 4B , an image whose up and down side and left and right side are reversed is created for each of the nine pixels 111 .
- the vein imaging apparatus 10 performs interpolation processing of images using image data generated by each of the plurality of pixels 111 corresponding to one of the microlenses 103 .
- FIG. 5A , FIG. 5B , FIG. 6A to FIG. 6C are explanatory diagrams for illustrating the imaging element according to the present embodiment.
- a region formed with the pixels 111 in the imaging element 109 is divided into, for example, two regions 151 and 153 as shown in FIG. 5A .
- the one region 151 is a vein image data generation region used to generate vein image data.
- the other region 153 is a positional displacement detection data generation region for generating positional displacement detection data used to detect positional displacement occurred due to thermal expansion caused by an environmental temperature and the like.
- vein image data generation region 151 a plurality of pixels (not shown) is arranged in an array, and the vein transmitted light condensed by the plurality of microlenses 103 of the microlens array 101 reaches the pixels 111 .
- the vein image data output by the vein image data generation region 151 are data related to the intensity of the light detected by the pixels that generated the image data.
- the positional displacement detection data generation region 153 is a plurality of pixels (not shown) arranged in an array. Further, the positional displacement detection data generation region 153 is shielded by a shielding film from the light, for example, as shown in FIG. 5A . Opening sections 155 without the shielding film are arranged in some areas of the positional displacement detection data generation region 153 .
- the number of the opening sections 155 arranged on the positional displacement detection data generation region 153 is not limited to the number of the opening sections 155 shown in the figure, and can be set to any number according to, e.g., a desired degree of accuracy of positional displacement detection result or the size of the imaging element 109 .
- the size of the positional displacement detection data generation region 153 arranged on the imaging element 109 can be determined according to, e.g., the size of the microlens array 101 used together with the imaging element 109 .
- the positional displacement detection data generation region 153 itself is shielded from external light by means of any method, and light emitted by an illumination light source enters into the opening sections 155 . There occurs a displacement of the position of the pixel at which the light emitted from the illumination light source, after passing through the opening section 155 , is focused because of, e.g., thermal expansion caused by variation in the environmental temperature and individual differences generated when the apparatus itself is manufactured. Then, the vein imaging apparatus 10 can know the size and the direction of the positional displacement by determining which pixel in the region 153 has detected a pixel signal representing the light entered from the opening section 155 .
- FIG. 5B shows an exemplary side view showing the positional displacement detection data generation region 153 .
- the positional displacement detection data generation region 153 is shielded from the external light by a shielding member 157 , so that the external light does not enter the positional displacement detection data generation region 153 .
- An illumination light source LS for detecting positional displacement is arranged inside of the shielding member 157 .
- the illumination light source LS for detecting positional displacement may be a light source arranged solely for detecting positional displacement, or may be a common light source that serves not only for detecting positional displacement but also for taking the vein image.
- the light emitted from the illumination light source LS is detected by the pixels via the opening section 155 .
- the positional displacement detection data generation region 153 is shielded from the external light by the shielding member 157 , and therefore, the light detected by the positional displacement detection data generation region 153 is emitted from the illumination light source LS.
- FIG. 5B is merely an example of the positional displacement detection data generation region 153 . It should be noted that the configuration of the positional displacement detection data generation region 153 according to the present embodiment is not limited to what is shown in FIG. 5B .
- such positional displacement detection data generation region 153 may be arranged at one end of the imaging element 109 along one side of the imaging element 109 as shown in FIG. 5A .
- the positional displacement detection data generation region 153 may be arranged along opposing sides of the imaging element 109 as shown in FIG. 6A and FIG. 6B , or may be arranged along four sides of the imaging element 109 as shown in FIG. 6C .
- the opening section 155 are omitted in FIG. 6A to FIG. 6C .
- FIG. 1 is referenced again.
- the configuration of the image processing unit of the vein imaging apparatus 10 according to the present embodiment will be described in detail.
- the pixel data dividing unit 123 is realized by, for example, CPU, ROM, and RAM. As shown in FIG. 7 , for example, the pixel data dividing unit 123 determines, based on pulses for scanning the imaging element 109 input from the drive control unit 121 , which of the two regions of the imaging element 109 has output the pixel data transmitted from the imaging element 109 . As the example in FIG. 7 shows, the pixel data dividing unit 123 uses three kinds of pulses, namely, a pulse for synchronization in a vertical (or horizontal) direction of the imaging element, a pulse for obtaining data for detecting positional displacement, and a pulse for obtaining vein image data, in order to obtain outputs from the two regions of the imaging element 109 . Therefore, the pixel data dividing unit 123 can determine, based on these pulses, whether the data for detecting positional displacement are transmitted or the vein image data are transmitted.
- the pixel data dividing unit 123 transmits to the later-described image focus position detection unit 125 the data obtained during a period in which the pulse for obtaining data for detecting positional displacement (namely, data for detecting positional displacement) is in the Hi state.
- the pixel data dividing unit 123 transmits to the later-described pixel selection unit 131 the data obtained during a period in which the pulse for obtaining vein image data (namely, vein image data) is in the Hi state.
- the image focus position detection unit 125 is realized by, for example, CPU, ROM, RAM and the like.
- the image focus position detection unit 125 references the data for detecting positional displacement transmitted from the pixel data dividing unit 123 , and detects which pixel of the positional displacement detection data generation region 153 has detected the light entered from the opening sections 155 arranged in the positional displacement detection data generation region 153 .
- the image focus position detection unit 125 can determine the image focus position of the light entered from the opening section 155 by determining in which part of the positional displacement detection data generation region 153 the pixel having detected the light entered from the opening section 155 is located.
- the image focus position detection unit 125 may also determine the image focus position of the light entered from the opening section 155 by detecting a region in which the total light quantity of the light detected by the positional displacement detection data generation region 153 is the largest, instead of identifying the pixel that has detected the light. For example, the image focus position detection unit 125 sets the size of the region of the imaging element corresponding to the opening section 155 as a base unit in considering the total light quantity, and searches per this base unit the data for detecting positional displacement transmitted from the positional displacement detection data generation region 153 . The image focus position detection unit 125 can determine, as the image focus position of the light, a part of the region 153 at which the total light quantity is the largest.
- the image focus position detection unit 125 transmits information about the image focus position related to the determined light (hereinafter referred to as “image focus position information”) to the later-described positional displacement amount estimation unit 127 .
- the positional displacement amount estimation unit 127 is realized by, for example, CPU, ROM, RAM and the like.
- the positional displacement amount estimation unit 127 estimates based on the image focus position information transmitted from the image focus position detection unit 125 , the size and the direction of positional displacement occurring in the vein imaging apparatus 10 .
- the vein imaging apparatus 10 may have positional displacement between the microlens array and the imaging element (more specifically, displacement in the positional relationship between the MLA, the directivity control plate, and the imaging element) due to, e.g., thermal expansion caused by an environmental temperature when the vein is imaged and assembly error arising when the apparatus is assembled.
- the positional displacement amount estimation unit 127 can estimate the size and the direction of the positional displacement by identifying how much and in what direction the image focus position represented by the transferred image focus position information is displaced with respect to an original image focus position without the above-described positional displacement.
- the positional displacement amount estimation unit 127 transmits the information representing the size and the direction of positional displacement (hereinafter, referred to as positional displacement information) to the warning unit 129 , the pixel selection unit 131 , and the vein image interpolation unit 133 , which will be described later.
- the warning unit 129 is realized by, for example, CPU, ROM, RAM and the like.
- the warning unit 129 references the positional displacement information transmitted from the positional displacement amount estimation unit 127 .
- the warning unit 129 determines that it is difficult to perform normal vein imaging processing (furthermore, the vein authentication processing) and outputs a warning.
- the warning unit 129 may determine that the apparatus itself is under an environment that does not allow the apparatus to perform normal operation, and may output a warning accordingly.
- the warning unit 129 may stop the vein imaging processing and the vein authentication processing being carried out by the apparatus itself.
- the pixel selection unit 131 is realized by, for example, CPU, ROM, RAM and the like.
- the pixel selection unit 131 selects based on the positional displacement information transmitted from the positional displacement amount estimation unit 127 , a pixel for generating vein image data used to generate vein image from among the plurality of pixels 111 corresponding to one of the microlenses 103 . Further, the pixel selection unit 131 may select a pixel included in a region in which the total light quantity of the light is the largest from among the plurality of pixels 111 corresponding to one of the microlenses 103 based on the positional displacement information transmitted from the positional displacement amount estimation unit 127 .
- FIG. 8A to FIG. 8C are explanatory diagrams for illustrating the pixel selection processing performed by the pixel selection unit 131 .
- FIG. 8A shows a case where there is no positional displacement occurring in the vein imaging apparatus 10 .
- the pixel selection unit 131 selects 4 ⁇ 4 pixels located in the central portion from among 8 ⁇ 8 pixels corresponding to one of the microlenses 103 .
- FIG. 8B shows a case where the positional displacement is integral multiples of the pixel (namely, there occurs a displacement equivalent to integral multiples of the pixel, such as 1 pixel, 2 pixels, 3 pixels, . . . ).
- the amount of positional displacement includes one pixel in the left direction and one pixel in the upward direction compared with FIG. 8A , and accordingly the pixel selection unit 131 selects 16 pixels included in the reference unit region displaced to left by one pixel and upward by one pixel.
- FIG. 8C shows a case where the positional displacement is not integral multiples of the pixel (namely, there occurs a displacement equivalent to real number multiples, such as 0.5 pixel, 1.3 pixels, . . . ).
- a displacement equivalent to real number multiples such as 0.5 pixel, 1.3 pixels, . . .
- information that should be originally taken by only one pixel is extending over two or more pixels, and a so-called crosstalk occurs.
- the image quality deteriorates, and the spatial frequency of the image decreases, so that it is difficult for the apparatus to recognize a thin vein as a vein.
- the pixel selection unit 131 selects not only the pixels included in the reference unit region but also pixels, detecting light, located around the pixels included in the reference unit region.
- the pixel selection unit 131 transmits, to the later-described vein image interpolation unit 133 , the information about the thus selected pixels (for example, information for identifying the selected pixels) and the vein image data obtained from the selected pixels.
- the vein image interpolation unit 133 is realized by, for example, CPU, ROM, RAM and the like.
- the vein image interpolation unit 133 generates the vein image based on the vein image data transmitted from the pixel selection unit 131 . Further, the vein image interpolation unit 133 performs interpolation processing on the generated vein image based on the positional displacement information transmitted from the positional displacement amount estimation unit 127 .
- Examples of the interpolation processing performed by the vein image interpolation unit 133 include denoising processing of the generated vein image.
- the vein image interpolation unit 133 may also perform processing for improving the image quality of the vein image by integrating a plurality of frame images according to the amount of positional displacement transmitted from the positional displacement amount estimation unit 127 .
- the plurality of frame images are integrated, it takes more time to perform the processing, and the user of the vein imaging apparatus 10 has to wait for a longer time. However, it is possible to curb the occurrence of the situation where a vein may not be imaged due to an environmental temperature and the like (furthermore, the vein authentication processing may not be carried out).
- the amount of positional displacement transmitted from the positional displacement amount estimation unit 127 is not integral multiples of the pixel (real number multiples of the pixel), the vein image interpolation unit 133 performs the following interpolation processing using, for example, multi-tap interpolation filter.
- the vein image interpolation unit 133 performs interpolation processing (composition processing) of the vein image by using not only the image data obtained from the pixels included in the reference unit region but also the image data, obtained from the pixels that are located around the pixels included in the reference unit region and are detecting light.
- the interpolation processing of the image using the neighboring pixels may be performed not only when the amount of positional displacement is not integral multiples of the pixel but also when the amount of positional displacement is integral multiples of the pixel.
- the vein image interpolation unit 133 transmits, to the later-described vein pattern extraction unit 135 , the vein image on which the interpolation processing was performed.
- the vein pattern extraction unit 135 is realized by, for example, CPU, ROM, RAM and the like.
- the vein pattern extraction unit 135 has, for example, a function of performing preprocessing of the vein pattern extraction on the vein image transmitted from the vein image interpolation unit 133 , a function of extracting the vein pattern, and a function of performing a postprocessing of the vein pattern extraction.
- Examples of the above preprocessing of the vein pattern extraction include a processing of detecting an outline of a finger from the vein image and recognizing at which position of the vein image the finger is located, and a processing of rotating the taken image using the detected outline of the finger and correcting the angle of the taken image.
- the above extraction of the vein pattern is performed by applying a differential filter to the taken image on which the outline detection processing and the angle correction processing are completed.
- the differential filter is a filter that outputs a large value as an output value at a part where a difference between a pixel of interest and a neighboring pixel is large.
- the differential filter is a filter that enhances a line or an edge in an image by the operation using a difference in gradation value between a pixel of interest and pixels in its neighborhood.
- image data v(x, y) is generated as represented by the following Expression 1.
- expression 1 “*” indicates convolution integral.
- a differentiation filter such as a primary space differentiation filter or a secondary space differentiation filter may be used as the above-described differential filter.
- the primary space differentiation filter is a filter that calculates a difference in gradation value between a pixel of interest and an adjacent pixel in the horizontal direction and the vertical direction
- a secondary space differentiation filter is a filter that extracts, for a pixel of interest, a part where the amount of change in difference in gradation value is large.
- the following Laplacian of Gaussian (LOG) filter may be used.
- the LOG filter (Expression 3) is represented by a second order derivative of a Gaussian filter (Expression 2), which is a smoothing filter using the Gaussian function.
- ⁇ indicates a standard deviation of the Gaussian function, which is a variable indicating the degree of smoothing of the Gaussian filter.
- ⁇ in the following Expression 3 is a parameter indicating a standard deviation of the Gaussian function as in Expression 2, and an output value when performing LOG filtering can be changed by changing a value of ⁇ .
- Examples of the above postprocessing of the vein pattern extraction include threshold processing performed on the taken image to which the differential filter was applied, binarization processing, and thinning processing. After the above postprocessing, a skeleton of the vein pattern can be extracted.
- the vein pattern extraction unit 135 transmits the vein pattern and the skeleton thus extracted to the later-described authentication unit 137 .
- the vein pattern extraction unit 135 may store the extracted vein pattern and the skeleton in the later-described storage unit 139 .
- the vein pattern extraction unit 135 may further store a parameter generated when performing each processing, the progress of processing or the like in the storage unit 139 .
- the authentication unit 137 is realized by, for example, CPU, ROM, RAM and the like.
- the authentication unit 137 authenticates the vein pattern by collating the vein pattern generated by the vein pattern extraction unit 135 with an already-registered template.
- the vein pattern authentication unit 137 authenticates the generated vein pattern based on the vein pattern that is generated by the vein pattern extraction unit 135 and the template of the vein pattern that has been registered.
- the vein pattern authentication unit 137 requests the storage unit 139 , which is described later, to disclose the registered vein pattern and compares the acquired registered vein pattern with the vein pattern transferred from the vein pattern extraction unit 135 .
- the comparison between the registered vein pattern and the transferred vein pattern can be carried out based on a correlation coefficient, which is calculated as follows, for example. In a case where, as a result of the comparison, the registered vein pattern is determined to be similar to the transmitted vein pattern, the authentication unit 137 determines that the authentication of the transmitted vein pattern is successful. When the registered vein pattern is determined not to be similar to the transmitted vein pattern, the authentication unit 137 determines that the authentication has failed.
- the authentication unit 137 may associate the authentication result with an authentication time and the like, and may record the authentication result as an authentication history in the storage unit 139 .
- the authentication unit 137 may associate the authentication result with an authentication time and the like, and may record the authentication result as an authentication history in the storage unit 139 .
- the authentication unit 137 transmits, to the warning unit 129 , a message indicating that the authentication processing has failed for the predetermined number of times or more.
- the warning unit 129 By transmitting such information to the warning unit 129 , when it is determined that a normal vein authentication processing may not be performed due to thermal expansion caused by an environmental temperature and the like, it is possible to warn the user of the vein imaging apparatus 10 that the authentication might not be normally performed.
- the storage unit 139 stores registered vein patterns of the users of the vein imaging apparatus 10 and other data associated with the registered vein patterns.
- the storage unit 139 may store the vein image data generated by the imaging unit, the vein image generated by the vein image interpolation unit 133 , and vein pattern and the like extracted by the vein pattern extraction unit 135 .
- the storage unit 139 may be stored various programs, data, and the like that are needed in the interpolation processing performed by the vein image interpolation unit 133 .
- the storage unit 139 may store various parameters or progress of processing that are necessary to be stored while the vein imaging apparatus 10 performs certain processing, various kinds of databases or the like. This storage unit 139 can be freely read and written by each processing unit included in the imaging unit, the image processing unit, and the authentication processing unit.
- FIG. 9 and FIG. 10 are explanatory diagrams for illustrating the method for obtaining data from the particular pixel.
- the imaging element 109 of the vein imaging apparatus 10 is a multi-layer element.
- FIG. 9 shows an example of a case where the imaging element 109 is a multi-layer element made of three layers.
- the imaging element 10 performs line-scanning in a longitudinal direction of a finger, namely, in a direction along y-axis in the figure.
- the direction along the y-axis in the figure will be referred to as vertical direction.
- a direction perpendicular to the vertical direction, namely, a direction along x-axis in the figure will be referred to as horizontal direction.
- image data is output by the drive control unit 121 in units of horizontal lines along a temporal axis of vertical synchronization.
- data for some pixels disposed along the horizontal direction is output to the first layer shown in FIG. 9 in synchronization
- data for some pixels disposed along the horizontal direction output to the second layer
- data for some pixels disposed along the horizontal direction is output to the third layer.
- the imaging element 109 can output with the multiple layers.
- the pixel selection unit 131 transmits information about the pixels to be selected to the drive control unit 121 , and for the drive control unit 121 to select an output with a certain layer of the multi-layer element and to select a particular pixel on the horizontal line by the timing control.
- the drive control unit 121 can select data provided by any pixel.
- the divisional driving within the vertical line and the divisional driving within the horizontal line may be used in combination.
- each of the above-described elements may be constituted using a general-purpose member or circuit, or it may be constituted by the hardware specialized to the function of each element. Further, the function of each element may be entirely realized by a CPU or the like. It is thereby possible to change the configuration to be used as appropriate according to the technique level when implementing the embodiment.
- a computer-readable recording medium in which the above computer program is stored may also be provided.
- the recording medium may be, for example, a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory.
- the above computer program may be distributed via, e.g., a network instead of using the recording medium.
- the vein imaging apparatus 10 may be implemented in an information processing apparatus such as a computer or a server, a mobile terminal such as a cellular phone or a PHS or a portable information terminal (PDA), an automated-teller machine (ATM), an access management apparatus. Further, the vein imaging apparatus 10 according to the present embodiment may be implemented in various kinds of apparatuses such as a game machine, a controller of a game machine or the like.
- the registered vein patterns previously registered as templates are assumed to be recorded in the vein imaging apparatus 10 .
- the registered vein patterns may be stored in a recording medium such as a DVD medium, a Blu-ray medium, a compact flash (registered trademark), a memory stick, or an SD memory card, an IC card or an electronic device equipped with a contactless IC chip or the like, or may be stored in a server that is connected to the vein imaging apparatus 10 via a communication network such as the Internet.
- FIG. 11 is a flow diagram for illustrating the positional displacement interpolation method according to the present embodiment.
- the user of the vein imaging apparatus 10 places a part of the living body such as a finger on the microlens array 101 of the vein imaging apparatus 10 .
- the imaging unit of the vein imaging apparatus 10 performs an imaging processing of the part of the living body placed thereon (step S 101 ).
- the image focus position detection unit 125 of the vein imaging apparatus 10 detects the image focus position of the light emitted from the light source used for detecting the positional displacement by using data output from the positional displacement detection data generation region 153 of the imaging element 109 . Subsequently, the positional displacement amount estimation unit 127 estimates the size and the direction of the positional displacement occurring in the vein imaging apparatus 10 based on the image focus position information about the image focus position detected by the image focus position detection unit 125 (step S 103 ). The positional displacement amount estimation unit 127 transmits the positional displacement information including the size and the direction of the positional displacement to the warning unit 129 , the pixel selection unit 131 , and the vein image interpolation unit 133 .
- the warning unit 129 to which the positional displacement information was transmitted, makes a determination on the amount of positional displacement included in the positional displacement information (step S 105 ), and determines whether the amount of positional displacement exceeds a threshold value at which a warning is required.
- the vein imaging apparatus 10 outputs a warning on a display screen (step S 107 ).
- the pixel selection unit 131 When there does not occur the positional displacement according to which a warning is required, the pixel selection unit 131 performs a selection processing of a pixel in view of the positional displacement based on the transmitted positional displacement information (step S 109 ). More specifically, the pixel selection unit 131 selects a pixel outputting image data used for generating the vein image from among the plurality of pixels corresponding to one of the microlenses 103 , for each of the microlenses 103 constituting the microlens array 101 .
- the vein image interpolation unit 133 generates the vein image by using the image data obtained from the pixel selected by the pixel selection unit 131 .
- the vein image interpolation unit 133 performs interpolation processing on the generated vein image according to the amount of positional displacement (step S 111 ). More specifically, the vein image interpolation unit 133 performs integration processing of the plurality of frame images, denoising processing, and interpolation processing of the image using the neighboring pixels.
- the vein image interpolation unit 133 transmits the vein image on which the interpolation processing was performed to the vein pattern extraction unit 135 .
- the vein pattern extraction unit 135 extracts the vein pattern from the transmitted vein image (step S 113 ), and transmits the extracted vein pattern to the authentication unit 137 .
- the authentication unit 137 performs authentication processing of the transmitted vein pattern by using the vein pattern transmitted from the vein pattern extraction unit 135 and the registered vein patterns (templates) stored in the storage unit 139 and the like (step S 115 ).
- the amount of positional displacement is estimated after the living body is imaged.
- the vein imaging apparatus 10 may estimate the positional displacement occurring in the apparatus in advance before imaging the living body.
- FIG. 12 is a block diagram for illustrating the configuration of the vein imaging apparatus according to the present embodiment.
- the vein imaging apparatus 10 includes, for example, three units, i.e., the imaging unit, the image processing unit, and the authentication processing unit.
- the imaging unit and the authentication processing unit according to the present embodiment have the same configuration and achieve almost the same effects as the imaging unit and the authentication processing unit of the vein imaging apparatus 10 according to the first embodiment of the present invention, and accordingly, the detailed description thereof is omitted.
- the image processing unit mainly includes, for example, the pixel data dividing unit 123 , the image focus position detection unit 125 , the positional displacement amount estimation unit 127 , the warning unit 129 , the pixel selection unit 131 , the vein image interpolation unit 133 , and a temperature estimation unit 141 .
- the pixel data dividing unit 123 , the image focus position detection unit 125 , the pixel selection unit 131 , the vein image interpolation unit 133 according to the present embodiment have the same configuration and achieve almost the same effects as each processing unit according to the first embodiment of the present invention, and accordingly, the detailed description thereof is omitted.
- the positional displacement amount estimation unit 127 has the same configuration and achieve almost the same effects as the positional displacement amount estimation unit 127 according to the first embodiment of the present invention, except that the positional displacement information obtained from the estimation result is not transmitted to the warning unit 129 but is transmitted to the temperature estimation unit 141 . Accordingly, the detailed description thereof is omitted.
- the temperature estimation unit 141 is realized by, for example, CPU, ROM, RAM and the like.
- the temperature estimation unit 141 estimates a temperature at which the vein imaging apparatus 10 performed the imaging processing, based on the positional displacement information transmitted from the positional displacement amount estimation unit 127 .
- the temperature at which the vein imaging apparatus 10 performed the imaging processing may be an external temperature at the location in which the vein imaging apparatus 10 is installed, or may be a temperature that the vein imaging apparatus 10 attains.
- the temperature estimation unit 141 has a database storing correspondence relationship between the amount of positional displacement occurring in the vein imaging apparatus 10 and the temperature at which the imaging processing is performed, and estimates the temperature from the amount of positional displacement based on this database.
- This database may relates to, instead of the correspondence relationship between the amount of positional displacement and the temperature, an expansion coefficient calculated from the amount of positional displacement and the temperature at which the imaging processing is performed.
- the database may express this relationship by an expression representing the correspondence relationship between these two parameters.
- the above-described data base may be generated by measuring the amount of positional displacement or the expansion coefficient and the like while varying the temperature during the production of the vein imaging apparatus 10 .
- the temperature estimation unit 141 transmits the temperature obtained from the estimation result to the warning unit 129 . Further, the temperature estimation unit 141 may transmit the temperature obtained from the estimation result to the drive control unit 121 . Based on information about the temperature transmitted from the temperature estimation unit 141 , the drive control unit 121 can control the intensity of the near-infrared light emitted from the near-infrared light emission source 105 and can control the light-receiving time, the frame rate, and the like of the imaging element 109 .
- the warning unit 129 according to the present embodiment has the same configuration and achieve almost the same effects as the warning unit 129 according to the first embodiment, except that the warning unit 129 gives a warning based on not the positional displacement information transmitted from the positional displacement amount estimation unit 127 but based on the information about the temperature transmitted from the temperature estimation unit 141 . Accordingly, the detailed description thereof is omitted.
- each of the above-described elements may be constituted using a general-purpose member or circuit, or it may be constituted by hardware specialized to the function of each element. It is thereby possible to change the hardware configuration to be used as appropriate according to the technique level when implementing the embodiment.
- a computer-readable recording medium in which the above computer program is stored may also be provided.
- the recording medium may be, for example, a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory.
- the above computer program may be distributed via, e.g., a network instead of using the recording medium.
- the positional displacement interpolation method performed by the vein imaging apparatus according to the present embodiment has substantially the same flow and achieves almost the same effects as the positional displacement interpolation method performed by the vein imaging apparatus according to the first embodiment of the present invention, and accordingly, the detailed description thereof is omitted.
- FIG. 13 is a block diagram for illustrating a hardware configuration of the vein imaging apparatus 10 according to an embodiment of the present invention.
- the vein imaging apparatus 10 includes the microlens array 101 , the near-infrared light emission source 105 , and the imaging element 109 .
- the vein imaging apparatus 10 includes a CPU 901 , a ROM 903 , and a RAM 905 .
- the vein imaging apparatus 10 includes a host bus 907 , a bridge 909 , an external bus 911 , an interface 913 , an input device 915 , an output device 917 , a storage device 919 , a drive 921 , a connection port 923 , and a communication device 925 .
- the CPU 901 functions as a processing unit and a control unit, and it controls the whole or a part of operation in the vein imaging apparatus 10 according to various kinds of programs stored in the ROM 903 , the RAM 905 , the storage device 919 or a removable recording medium 927 .
- the ROM 903 stores a program to be used by the CPU 901 , a processing parameter and the like.
- the ROM 903 stores a program to be used by the CPU 901 , a processing parameter and so on.
- the RAM 905 primarily stores programs used by the CPU 901 in the execution, parameters and the like that are changed during the execution.
- the CPU 901 , the ROM 903 and the RAM 905 are connected with one another through the host bus 907 , which is an internal bus such as a CPU bus.
- the host bus 907 is connected to the external bus 911 such as a Peripheral Component Interconnect/Interface (PCI) bus via the bridge 909 .
- PCI Peripheral Component Interconnect/Interface
- the input device 915 is an operating means to be operated by a user, such as a mouse, a keyboard, a touch panel, buttons, a switch or a lever, for example.
- the input device 915 may be a remote controlling means (or a remote control) with an infrared ray or another radio wave, or an externally connected device 929 compatible with the operation of the vein imaging apparatus 10 , such as a cellular phone or a PDA.
- the input device 915 includes an input control circuit that generates an input signal based on information input by a user using the above operating means and outputs it to the CPU 901 , for example. By operating this input device 915 , a user of the vein imaging apparatus 10 can input various kinds of data or give an instruction of a processing operation to the vein imaging apparatus 10 .
- the output device 917 includes an apparatus capable of visually or audibly notifying obtained information to the user.
- Examples of such apparatus include a display device such as a CRT display device, a liquid crystal display device, a plasma display device, an EL display device or a lamp, an audio output device such as a speaker or a headphone, or a printer, a cellular phone or a facsimile.
- the output device 917 outputs, for example, results obtained by various processing by the vein imaging apparatus 10 .
- the display device displays, as a text or an image, a result obtained by various processing of the vein imaging apparatus 10 .
- the audio output device converts an audio signal containing reproduced audio data, acoustic data or the like into an analog signal and outputs it.
- the storage device 919 is a device for data storage that is configured as an example of a storage unit of the vein imaging apparatus 10 .
- the storage device 919 may include a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device or the like.
- This storage device 919 stores a program to be executed by the CPU 901 , various data, or various data acquired from the outside, for example.
- the drive 921 is a reader/writer for a recording medium, which is built in the vein imaging apparatus 10 or attached thereto.
- the drive 921 reads information that is recorded in the removable recording medium 927 such as a magnetic disk, an optical disk, a magneto-optical disk or semiconductor memory which is attached thereto and outputs the information to the RAM 905 . Further, the drive 921 can write information into the removable recording medium 927 such as a magnetic disk, an optical disk, a magneto-optical disk or semiconductor memory which is attached thereto.
- Examples of the removable recording medium 927 include a DVD medium, an HD-DVD medium, and a Blu-ray medium.
- the removable recording medium 927 examples include a compact flash (registered trademark) (CF), a memory stick, and a secure digital (SD) memory card. Further, the removable recording medium 927 may be an integrated circuit (IC) card equipped with a contactless IC chip or an electronic appliance.
- CF compact flash
- SD secure digital
- the removable recording medium 927 may be an integrated circuit (IC) card equipped with a contactless IC chip or an electronic appliance.
- the connection port 923 is a port for directly connecting devices to the vein imaging apparatus 10 .
- Examples of the connection port 923 include a universal serial bus (USB) port, an IEEE 1394 port such as i.Link, and a small computer system interface (SCSI) port.
- examples of the connection port 923 include an RS-232C port, an optical audio terminal, and a high-definition multimedia interface (HDMI) port.
- USB universal serial bus
- HDMI high-definition multimedia interface
- the communication device 925 is a communication interface that is constituted by a communication device or the like for connecting to a communication network 931 , for example.
- the communication device 925 may be a communication card for wired or wireless local area network (LAN), Bluetooth, or wireless USB (WUSB).
- the communication device 925 may be a router for optical communication, a router for asymmetric digital subscriber line (ADSL), or a modem for each kind of communication.
- This communication device 925 can transmit and receive a signal or the like in conformity to a prescribed protocol such as TCP/IP on the Internet or with other communication devices, for example.
- the communication network 931 that is connected to the communication device 925 includes a wired or wireless network or the like, and it may be the Internet, home LAN, infrared data communication, radio wave communication, satellite communication or the like.
- the neighboring pixels around the imaging region are used to detect the positional displacement between the microlens array and the imaging element, so that the vein imaging apparatus can automate interpolation processing by means of image processing and optimum sampling of the vain imaging apparatus at each temperature.
- the vein imaging apparatus estimates the amount of positional displacement based on data obtained from a region arranged on a portion of the imaging element, and performs the interpolation processing of the image based on the positional displacement. As a result, the above-described issues can be solved. Further, according to each embodiment of the present invention, it is also possible to handle not only the positional displacement caused by thermal expansion but also the positional displacement caused by assembly error during production.
Abstract
Description
-
- Regarding configuration of imaging unit
- Regarding an example of structure of imaging unit
- Regarding image obtained by microlens array
- Regarding imaging element
- Regarding configuration of image processing unit
- Regarding configuration of authentication processing unit
- Regarding Obtaining data from particular pixel
Claims (19)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2009117985A JP5326792B2 (en) | 2009-05-14 | 2009-05-14 | Vein imaging device, positional deviation interpolation method and program |
JPP2009-117985 | 2009-05-14 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20100292578A1 US20100292578A1 (en) | 2010-11-18 |
US8320998B2 true US8320998B2 (en) | 2012-11-27 |
Family
ID=43069084
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/775,211 Active 2030-10-21 US8320998B2 (en) | 2009-05-14 | 2010-05-06 | Vein imaging apparatus, positional displacement interpolation method, and program |
Country Status (3)
Country | Link |
---|---|
US (1) | US8320998B2 (en) |
JP (1) | JP5326792B2 (en) |
CN (1) | CN101884537B (en) |
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090002488A1 (en) * | 2007-06-28 | 2009-01-01 | Vincent Luciano | Automatic alignment of a contrast enhancement system |
US20100208949A1 (en) * | 2007-07-20 | 2010-08-19 | Hideo Sato | Vein Authentication Apparatus, Imaging Apparatus for Vein Authentication, and Vein Illuminating Method |
US8712498B2 (en) | 2006-01-10 | 2014-04-29 | Accuvein Inc. | Micro vein enhancer |
US8750970B2 (en) | 2006-01-10 | 2014-06-10 | Accu Vein, Inc. | Micro vein enhancer |
US20140221813A1 (en) * | 2012-03-19 | 2014-08-07 | Genetic Innovations, Inc. | Devices, systems, and methods for virtual staining |
US8838210B2 (en) | 2006-06-29 | 2014-09-16 | AccuView, Inc. | Scanned laser vein contrast enhancer using a single laser |
US9061109B2 (en) | 2009-07-22 | 2015-06-23 | Accuvein, Inc. | Vein scanner with user interface |
US9072426B2 (en) | 2012-08-02 | 2015-07-07 | AccuVein, Inc | Device for detecting and illuminating vasculature using an FPGA |
US9345427B2 (en) | 2006-06-29 | 2016-05-24 | Accuvein, Inc. | Method of using a combination vein contrast enhancer and bar code scanning device |
US9492117B2 (en) | 2006-01-10 | 2016-11-15 | Accuvein, Inc. | Practitioner-mounted micro vein enhancer |
US9854977B2 (en) | 2006-01-10 | 2018-01-02 | Accuvein, Inc. | Scanned laser vein contrast enhancer using a single laser, and modulation circuitry |
US10238294B2 (en) | 2006-06-29 | 2019-03-26 | Accuvein, Inc. | Scanned laser vein contrast enhancer using one laser |
US20190095675A1 (en) * | 2017-09-22 | 2019-03-28 | Fujitsu Limited | Image processing apparatus and image processing method |
US10376147B2 (en) | 2012-12-05 | 2019-08-13 | AccuVeiw, Inc. | System and method for multi-color laser imaging and ablation of cancer cells using fluorescence |
US10813588B2 (en) | 2006-01-10 | 2020-10-27 | Accuvein, Inc. | Micro vein enhancer |
US11051697B2 (en) | 2006-06-29 | 2021-07-06 | Accuvein, Inc. | Multispectral detection and presentation of an object's characteristics |
US11253198B2 (en) | 2006-01-10 | 2022-02-22 | Accuvein, Inc. | Stand-mounted scanned laser vein contrast enhancer |
US11278240B2 (en) | 2006-01-10 | 2022-03-22 | Accuvein, Inc. | Trigger-actuated laser vein contrast enhancer |
USD999379S1 (en) | 2010-07-22 | 2023-09-19 | Accuvein, Inc. | Vein imager and cradle in combination |
Families Citing this family (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6010898B2 (en) * | 2011-11-16 | 2016-10-19 | ソニー株式会社 | Biological measuring device, biological measuring method, program, and recording medium |
JP5990906B2 (en) * | 2011-12-19 | 2016-09-14 | ソニー株式会社 | Measuring device, measuring method, program, and recording medium |
EP2793685A1 (en) * | 2011-12-22 | 2014-10-29 | Infrared Imaging Systems, Inc. | Apparatus and method for imaging vasculature and sub-dermal structures by trans-illuminating nir light |
CN106132299B (en) | 2014-03-31 | 2020-04-14 | 索尼公司 | Measuring device, measuring method, program, and recording medium |
JP6539877B2 (en) * | 2014-03-31 | 2019-07-10 | ソニー株式会社 | Measuring device, measuring method, program and recording medium |
US9558392B2 (en) * | 2015-02-12 | 2017-01-31 | Korecen Co., Ltd. | Finger vein authentication system |
JPWO2017018150A1 (en) * | 2015-07-29 | 2018-02-22 | 富士フイルム株式会社 | Optical sensor device, optical sensor unit and optical sensor system |
CN107977935A (en) * | 2017-11-15 | 2018-05-01 | 合肥中科奔巴科技有限公司 | A kind of vena metacarpea image texture filtering reinforcement method |
JP2019140230A (en) * | 2018-02-09 | 2019-08-22 | ソニーセミコンダクタソリューションズ株式会社 | Solid-state image sensor, electronic device and manufacturing method for electronic device |
CN109684983B (en) * | 2018-12-19 | 2021-02-26 | 武汉华星光电半导体显示技术有限公司 | Fingerprint monitoring device and electronic equipment |
US11068701B2 (en) * | 2019-06-13 | 2021-07-20 | XMotors.ai Inc. | Apparatus and method for vehicle driver recognition and applications of same |
TW202128079A (en) * | 2019-10-17 | 2021-08-01 | 德商百靈佳殷格翰維美迪加股份有限公司 | Examination apparatus for medical examination of an animal |
TWI729813B (en) * | 2020-05-18 | 2021-06-01 | 友達光電股份有限公司 | Blood vessel image forming system |
CN113132647B (en) * | 2021-04-25 | 2023-01-20 | 京东方科技集团股份有限公司 | Data processing method, data processing device, storage medium and electronic equipment |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6485413B1 (en) * | 1991-04-29 | 2002-11-26 | The General Hospital Corporation | Methods and apparatus for forward-directed optical scanning instruments |
JP2004296531A (en) | 2003-03-25 | 2004-10-21 | Fuji Photo Film Co Ltd | Exposure system |
US20050180620A1 (en) * | 2002-05-09 | 2005-08-18 | Kiyoaki Takiguchi | Method of detecting biological pattern, biological pattern detector, method of biological certificate and biological certificate apparatus |
US20070003112A1 (en) * | 2005-06-30 | 2007-01-04 | Fujitsu Limited | Biometrics authentication method biometrics authentication device and blood vessel image reading device |
US20080205711A1 (en) * | 2007-02-26 | 2008-08-28 | Hitachi Maxell, Ltd. | Biometric information acquisition device |
US20110001814A1 (en) * | 2008-03-04 | 2011-01-06 | Ricoh Company, Ltd. | Personal authentication device and electronic device |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3703424B2 (en) * | 2000-10-25 | 2005-10-05 | キヤノン株式会社 | IMAGING DEVICE, ITS CONTROL METHOD, CONTROL PROGRAM, AND STORAGE MEDIUM |
JP4293013B2 (en) * | 2003-02-28 | 2009-07-08 | 日本電気株式会社 | Image display device and manufacturing method thereof |
JP2004287183A (en) * | 2003-03-24 | 2004-10-14 | Fuji Photo Film Co Ltd | Digital camera |
JP4095659B2 (en) * | 2005-06-13 | 2008-06-04 | 株式会社日立製作所 | Vein authentication device |
JP2008036226A (en) * | 2006-08-08 | 2008-02-21 | Sony Corp | Vein authentication imaging unit and electronic equipment |
JP4788526B2 (en) * | 2006-08-30 | 2011-10-05 | 株式会社日立製作所 | Finger vein authentication input device |
JP4621214B2 (en) * | 2007-01-17 | 2011-01-26 | 日本放送協会 | Stereo image capturing position adjusting device, stereo image capturing position adjusting method and program thereof, and stereo image capturing system |
JP2009017156A (en) * | 2007-07-04 | 2009-01-22 | Omron Corp | Imaging apparatus and monitoring system and method |
JP4954819B2 (en) * | 2007-07-23 | 2012-06-20 | リコー光学株式会社 | Blood vessel image input device and blood vessel image reading device |
KR20100038362A (en) * | 2007-07-30 | 2010-04-14 | 소니 주식회사 | Imaging device |
-
2009
- 2009-05-14 JP JP2009117985A patent/JP5326792B2/en not_active Expired - Fee Related
-
2010
- 2010-05-06 US US12/775,211 patent/US8320998B2/en active Active
- 2010-05-07 CN CN2010101771997A patent/CN101884537B/en not_active Expired - Fee Related
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6485413B1 (en) * | 1991-04-29 | 2002-11-26 | The General Hospital Corporation | Methods and apparatus for forward-directed optical scanning instruments |
US20050180620A1 (en) * | 2002-05-09 | 2005-08-18 | Kiyoaki Takiguchi | Method of detecting biological pattern, biological pattern detector, method of biological certificate and biological certificate apparatus |
US20070116330A1 (en) * | 2002-05-09 | 2007-05-24 | Sony Corporation | Living-tissue pattern detecting method, living-tissue pattern detecting device, biometric authentication method, and biometric authentication device |
US20080118114A1 (en) * | 2002-05-09 | 2008-05-22 | Sony Corporation | Living-tissue pattern detecting method, living-tissue pattern detecting device, biometric authentication method, and biometric authentication device |
US7415139B2 (en) * | 2002-05-09 | 2008-08-19 | Sony Corporation | Living-tissue pattern detecting method, living-tissue pattern detecting device, biometric authentication method, and biometric authentication device |
US7869624B2 (en) * | 2002-05-09 | 2011-01-11 | Sony Corporation | Compact biometric authentication device and associated methodology of imaging and detecting living-tissue patterns |
JP2004296531A (en) | 2003-03-25 | 2004-10-21 | Fuji Photo Film Co Ltd | Exposure system |
US20070003112A1 (en) * | 2005-06-30 | 2007-01-04 | Fujitsu Limited | Biometrics authentication method biometrics authentication device and blood vessel image reading device |
US20080205711A1 (en) * | 2007-02-26 | 2008-08-28 | Hitachi Maxell, Ltd. | Biometric information acquisition device |
US20110001814A1 (en) * | 2008-03-04 | 2011-01-06 | Ricoh Company, Ltd. | Personal authentication device and electronic device |
Cited By (64)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10258748B2 (en) | 2006-01-10 | 2019-04-16 | Accuvein, Inc. | Vein scanner with user interface for controlling imaging parameters |
US11642080B2 (en) | 2006-01-10 | 2023-05-09 | Accuvein, Inc. | Portable hand-held vein-image-enhancing device |
US11638558B2 (en) | 2006-01-10 | 2023-05-02 | Accuvein, Inc. | Micro vein enhancer |
US8712498B2 (en) | 2006-01-10 | 2014-04-29 | Accuvein Inc. | Micro vein enhancer |
US11484260B2 (en) | 2006-01-10 | 2022-11-01 | Accuvein, Inc. | Patient-mounted micro vein enhancer |
US8750970B2 (en) | 2006-01-10 | 2014-06-10 | Accu Vein, Inc. | Micro vein enhancer |
US11399768B2 (en) | 2006-01-10 | 2022-08-02 | Accuvein, Inc. | Scanned laser vein contrast enhancer utilizing surface topology |
US8818493B2 (en) | 2006-01-10 | 2014-08-26 | Accuvein, Inc. | Three-dimensional imaging of veins |
US11357449B2 (en) | 2006-01-10 | 2022-06-14 | Accuvein, Inc. | Micro vein enhancer for hands-free imaging for a venipuncture procedure |
US9042966B2 (en) | 2006-01-10 | 2015-05-26 | Accuvein, Inc. | Three dimensional imaging of veins |
US9044207B2 (en) | 2006-01-10 | 2015-06-02 | Accuvein, Inc. | Micro vein enhancer for use with a vial holder |
US11278240B2 (en) | 2006-01-10 | 2022-03-22 | Accuvein, Inc. | Trigger-actuated laser vein contrast enhancer |
US11253198B2 (en) | 2006-01-10 | 2022-02-22 | Accuvein, Inc. | Stand-mounted scanned laser vein contrast enhancer |
US9125629B2 (en) | 2006-01-10 | 2015-09-08 | Accuvein, Inc. | Vial-mounted micro vein enhancer |
US11191482B2 (en) | 2006-01-10 | 2021-12-07 | Accuvein, Inc. | Scanned laser vein contrast enhancer imaging in an alternating frame mode |
US11172880B2 (en) | 2006-01-10 | 2021-11-16 | Accuvein, Inc. | Vein imager with a dual buffer mode of operation |
US11109806B2 (en) | 2006-01-10 | 2021-09-07 | Accuvein, Inc. | Three dimensional imaging of veins |
US10813588B2 (en) | 2006-01-10 | 2020-10-27 | Accuvein, Inc. | Micro vein enhancer |
US9492117B2 (en) | 2006-01-10 | 2016-11-15 | Accuvein, Inc. | Practitioner-mounted micro vein enhancer |
US10617352B2 (en) | 2006-01-10 | 2020-04-14 | Accuvein, Inc. | Patient-mounted micro vein enhancer |
US10500350B2 (en) | 2006-01-10 | 2019-12-10 | Accuvein, Inc. | Combination vein contrast enhancer and bar code scanning device |
US10470706B2 (en) | 2006-01-10 | 2019-11-12 | Accuvein, Inc. | Micro vein enhancer for hands-free imaging for a venipuncture procedure |
US9788788B2 (en) | 2006-01-10 | 2017-10-17 | AccuVein, Inc | Three dimensional imaging of veins |
US9788787B2 (en) | 2006-01-10 | 2017-10-17 | Accuvein, Inc. | Patient-mounted micro vein enhancer |
US9854977B2 (en) | 2006-01-10 | 2018-01-02 | Accuvein, Inc. | Scanned laser vein contrast enhancer using a single laser, and modulation circuitry |
US9949688B2 (en) | 2006-01-10 | 2018-04-24 | Accuvein, Inc. | Micro vein enhancer with a dual buffer mode of operation |
US9345427B2 (en) | 2006-06-29 | 2016-05-24 | Accuvein, Inc. | Method of using a combination vein contrast enhancer and bar code scanning device |
US11051755B2 (en) | 2006-06-29 | 2021-07-06 | Accuvein, Inc. | Scanned laser vein contrast enhancer using a retro collective mirror |
US11523739B2 (en) | 2006-06-29 | 2022-12-13 | Accuvein, Inc. | Multispectral detection and presentation of an object's characteristics |
US10238294B2 (en) | 2006-06-29 | 2019-03-26 | Accuvein, Inc. | Scanned laser vein contrast enhancer using one laser |
US10357200B2 (en) | 2006-06-29 | 2019-07-23 | Accuvein, Inc. | Scanning laser vein contrast enhancer having releasable handle and scan head |
US8838210B2 (en) | 2006-06-29 | 2014-09-16 | AccuView, Inc. | Scanned laser vein contrast enhancer using a single laser |
US9186063B2 (en) | 2006-06-29 | 2015-11-17 | Accu Vein, Inc. | Scanned laser vein contrast enhancer using one laser for a detection mode and a display mode |
US9226664B2 (en) | 2006-06-29 | 2016-01-05 | Accuvein, Inc. | Scanned laser vein contrast enhancer using a single laser |
US11051697B2 (en) | 2006-06-29 | 2021-07-06 | Accuvein, Inc. | Multispectral detection and presentation of an object's characteristics |
US8730321B2 (en) | 2007-06-28 | 2014-05-20 | Accuvein, Inc. | Automatic alignment of a contrast enhancement system |
US11132774B2 (en) | 2007-06-28 | 2021-09-28 | Accuvein, Inc. | Automatic alignment of a contrast enhancement system |
US11847768B2 (en) | 2007-06-28 | 2023-12-19 | Accuvein Inc. | Automatic alignment of a contrast enhancement system |
US10580119B2 (en) | 2007-06-28 | 2020-03-03 | Accuvein, Inc. | Automatic alignment of a contrast enhancement system |
US9760982B2 (en) | 2007-06-28 | 2017-09-12 | Accuvein, Inc. | Automatic alignment of a contrast enhancement system |
US10713766B2 (en) | 2007-06-28 | 2020-07-14 | Accuvein, Inc. | Automatic alignment of a contrast enhancement system |
US9430819B2 (en) | 2007-06-28 | 2016-08-30 | Accuvein, Inc. | Automatic alignment of a contrast enhancement system |
US10096096B2 (en) | 2007-06-28 | 2018-10-09 | Accuvein, Inc. | Automatic alignment of a contrast enhancement system |
US20090002488A1 (en) * | 2007-06-28 | 2009-01-01 | Vincent Luciano | Automatic alignment of a contrast enhancement system |
US20100208949A1 (en) * | 2007-07-20 | 2010-08-19 | Hideo Sato | Vein Authentication Apparatus, Imaging Apparatus for Vein Authentication, and Vein Illuminating Method |
US8649568B2 (en) * | 2007-07-20 | 2014-02-11 | Sony Corporation | Vein authentication apparatus, imaging apparatus for vein authentication, and vein illuminating method |
US10518046B2 (en) | 2009-07-22 | 2019-12-31 | Accuvein, Inc. | Vein scanner with user interface |
US9789267B2 (en) | 2009-07-22 | 2017-10-17 | Accuvein, Inc. | Vein scanner with user interface |
US11826166B2 (en) | 2009-07-22 | 2023-11-28 | Accuvein, Inc. | Vein scanner with housing configured for single-handed lifting and use |
USD999380S1 (en) | 2009-07-22 | 2023-09-19 | Accuvein, Inc. | Vein imager and cradle in combination |
US9061109B2 (en) | 2009-07-22 | 2015-06-23 | Accuvein, Inc. | Vein scanner with user interface |
USD999379S1 (en) | 2010-07-22 | 2023-09-19 | Accuvein, Inc. | Vein imager and cradle in combination |
USD998152S1 (en) | 2010-07-22 | 2023-09-05 | Accuvein, Inc. | Vein imager cradle |
US20140221813A1 (en) * | 2012-03-19 | 2014-08-07 | Genetic Innovations, Inc. | Devices, systems, and methods for virtual staining |
US11510617B2 (en) | 2012-08-02 | 2022-11-29 | Accuvein, Inc. | Device for detecting and illuminating the vasculature using an FPGA |
US9782079B2 (en) | 2012-08-02 | 2017-10-10 | Accuvein, Inc. | Device for detecting and illuminating the vasculature using an FPGA |
US9072426B2 (en) | 2012-08-02 | 2015-07-07 | AccuVein, Inc | Device for detecting and illuminating vasculature using an FPGA |
US10568518B2 (en) | 2012-08-02 | 2020-02-25 | Accuvein, Inc. | Device for detecting and illuminating the vasculature using an FPGA |
US10517483B2 (en) | 2012-12-05 | 2019-12-31 | Accuvein, Inc. | System for detecting fluorescence and projecting a representative image |
US11439307B2 (en) | 2012-12-05 | 2022-09-13 | Accuvein, Inc. | Method for detecting fluorescence and ablating cancer cells of a target surgical area |
US10376147B2 (en) | 2012-12-05 | 2019-08-13 | AccuVeiw, Inc. | System and method for multi-color laser imaging and ablation of cancer cells using fluorescence |
US10376148B2 (en) | 2012-12-05 | 2019-08-13 | Accuvein, Inc. | System and method for laser imaging and ablation of cancer cells using fluorescence |
US20190095675A1 (en) * | 2017-09-22 | 2019-03-28 | Fujitsu Limited | Image processing apparatus and image processing method |
US10867153B2 (en) * | 2017-09-22 | 2020-12-15 | Fujitsu Limited | Image processing apparatus and image processing method |
Also Published As
Publication number | Publication date |
---|---|
JP5326792B2 (en) | 2013-10-30 |
US20100292578A1 (en) | 2010-11-18 |
CN101884537B (en) | 2012-06-20 |
JP2010267086A (en) | 2010-11-25 |
CN101884537A (en) | 2010-11-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8320998B2 (en) | Vein imaging apparatus, positional displacement interpolation method, and program | |
US8111878B2 (en) | Vein authentication device and vein authentication method | |
US20100292579A1 (en) | Vein imaging apparatus, vein image interpolation method, and program | |
US8229179B2 (en) | Finger vein authentication apparatus and finger vein authentication method | |
US7884327B2 (en) | Vein imaging apparatus, vein imaging method and vein authentication apparatus | |
US9122900B2 (en) | Biometric authentication device and computer readable, non-transitory medium | |
US8275174B2 (en) | Vein pattern management system, vein pattern registration apparatus, vein pattern authentication apparatus, vein pattern registration method, vein pattern authentication method, program, and vein data configuration | |
US8320997B2 (en) | Vein authentication apparatus and template registration method | |
US20100226545A1 (en) | Vein pattern management system, vein pattern registration apparatus, vein pattern authentication apparatus, vein pattern registration method, vein pattern authentication method, program, and vein data configuration | |
US20090290763A1 (en) | Vein authentication apparatus and vein authentication method | |
US8270681B2 (en) | Vein pattern management system, vein pattern registration apparatus, vein pattern authentication apparatus, vein pattern registration method, vein pattern authentication method, program, and vein data configuration | |
JP2010240215A (en) | Vein depth determination apparatus, vein depth determination method and program | |
EP2530620A1 (en) | Biometric information process device, biometric information process method, and computer readable medium | |
US8320639B2 (en) | Vein pattern management system, vein pattern registration apparatus, vein pattern authentication apparatus, vein pattern registration method, vein pattern authentication method, program, and vein data configuration | |
JP5056662B2 (en) | Subcutaneous pattern acquisition device, subcutaneous pattern acquisition method, and structure template | |
JP5176556B2 (en) | Vein authentication device and vein authentication method | |
JP5299632B2 (en) | Personal authentication device, program and recording medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SONY CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SATO, HIDEO;REEL/FRAME:024353/0861 Effective date: 20100317 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |
|
AS | Assignment |
Owner name: SONY GROUP CORPORATION, JAPAN Free format text: CHANGE OF NAME;ASSIGNOR:SONY CORPORATION;REEL/FRAME:059499/0066 Effective date: 20210401 |
|
AS | Assignment |
Owner name: MOFIRIA CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SONY GROUP CORPORATION;REEL/FRAME:059618/0159 Effective date: 20210712 |