US20090096783A1 - Three-dimensional sensing using speckle patterns - Google Patents
Three-dimensional sensing using speckle patterns Download PDFInfo
- Publication number
- US20090096783A1 US20090096783A1 US12/282,517 US28251707A US2009096783A1 US 20090096783 A1 US20090096783 A1 US 20090096783A1 US 28251707 A US28251707 A US 28251707A US 2009096783 A1 US2009096783 A1 US 2009096783A1
- Authority
- US
- United States
- Prior art keywords
- images
- speckle pattern
- light source
- image
- diffuser
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B5/00—Optical elements other than lenses
- G02B5/18—Diffraction gratings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/557—Depth or shape recovery from multiple images from light fields, e.g. from plenoptic cameras
Definitions
- the present invention relates generally to methods and systems for mapping of three-dimensional (3D) objects, and specifically to 3D optical imaging using speckle patterns.
- primary speckle When a coherent beam of light passes through a diffuser and is projected onto a surface, a primary speckle pattern can be observed on the surface.
- the primary speckle is caused by interference among different components of the diffused beam.
- the term “primary speckle” is used in this sense in the present patent application and in the claims, in distinction to secondary speckle, which is caused by diffuse reflection of coherent light from the rough surface of an object
- Hart describes the use of a speckle pattern in a high-speed 3D imaging system, in Taiwanese Patent TW 527528 B and in U.S. patent application Ser. No. 09/616,606, whose disclosures are incorporated herein by reference.
- the system includes a single-lens camera subsystem with an active imaging element and CCD element, and a correlation processing subsystem.
- the active imaging element can be a rotating aperture which allows adjustable non-equilateral spacing between defocused images to achieve greater depth of field and higher sub-pixel displacement accuracy.
- a speckle pattern is projected onto an object, and images of the resulting pattern are acquired from multiple angles. The images are locally cross-correlated using an image correlation technique, and the surface is resolved by using relative camera position information to calculate the three-dimensional coordinates of each locally-correlated region.
- a random speckle pattern is projected upon a 3D surface and is imaged by a plurality of cameras to obtain a plurality of two-dimensional digital images.
- the two-dimensional images are processed to obtain a three-dimensional characterization of the surface.
- Embodiments of the present invention perform accurate, real-time mapping of 3D objects using primary speckle patterns.
- the methods and systems that are described in the above-mentioned PCT patent application, as well as further embodiments described hereinbelow, are capable of performing such 3D mapping using a single coherent light source and a single image sensor, which is held stationary at a fixed angle relative to the light source.
- a reference image of the speckle pattern is captured initially on a reference surface of known profile.
- the 3D profile of an object is then determined by capturing an image of the speckle pattern projected on the object, and comparing the image to the reference image.
- successive images of the speckle pattern on the object are captured as the object moves.
- Each image is compared with one or more of its predecessors in order to track the motion of the object in three dimensions.
- the light source and image sensor are held in a linear alignment that permits rapid, accurate motion tracking by computing one-dimensional correlation coefficients between successive images.
- novel illumination and image processing schemes are used to enhance the accuracy, depth of field, and computational speed of the 3D mapping system.
- apparatus for 3D mapping of an object including:
- an illumination assembly including a coherent light source and a diffuser, which are arranged to project a primary speckle pattern on the object;
- a single image capture assembly which is arranged to capture images of the primary speckle pattern on the object from a single, fixed location and angle relative to the illumination assembly;
- a processor which is coupled to process the images of the primary speckle pattern captured at the single, fixed angle so as to derive a 3D map of the object.
- the apparatus includes a mount, which is attached to the illumination assembly and the image capture assembly so as to hold the image capture assembly in a fixed spatial relation to the illumination assembly.
- the image capture assembly includes an array of detector elements arranged in a rectilinear pattern defining first and second, mutually-perpendicular axes, and objective optics, which have an entrance pupil and are arranged to focus the image onto the array, wherein the illumination assembly and image capture assembly are aligned by the mount so as to define a device axis that is parallel to the first axis and passes through the entrance pupil and through a spot at which a beam emitted by the coherent light source passes through the diffuser.
- the processor is arranged to derive the 3D map by finding an offset along only the first axis between the primary speckle pattern captured in one or more of the images and a reference image of the primary speckle pattern.
- the processor is arranged to derive the 3D map by finding respective offsets between the primary speckle pattern on multiple areas of the object captured in one or more of the images and a reference image of the primary speckle pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly.
- the image capture assembly is located at a predetermined spacing from the illumination assembly, and the respective offsets are proportional to the respective distances in a ratio that is determined by the spacing.
- the primary speckle pattern projected by the illumination assembly includes speckles having a characteristic size, and the size of the speckles in the images varies across the image by a tolerance that depends on the spacing, wherein the spacing is selected so as to maintain the tolerance within a predefined bound.
- the processor is arranged to relate the respective offsets to respective coordinates in the 3D map using a parametric model of distortion in the image capture assembly. Further additionally or alternatively, the processor is arranged to find the respective offsets by finding an initial match between the primary speckle pattern in a first area of the object and a corresponding area of the reference image at a first offset relative to the first area, and to apply a region growing procedure, based on the first offset, to find the respective offsets of pixels adjacent to the first area.
- the processor is arranged to process a succession of the images captured while the object is moving so as to map a 3D movement of the object, wherein the object is a part of a human body, and wherein the 3D movement includes a gesture made by the part of the human body, and wherein the processor is coupled to provide an input to a computer application responsively to the gesture.
- the illumination assembly includes a beam former, which is arranged to reduce a variation of a contrast of the speckle pattern created by the diffuser over a sensing volume of the apparatus.
- the beam former includes a diffractive optical element (DOE) and a lens arranged to define a Fourier plane of the diffuser, wherein the DOE is located in the Fourier plane.
- DOE diffractive optical element
- the beam former may be arranged to reduce a divergence of light emitted from the diffuser or to equalize an intensity of light emitted from the diffuser across a plane transverse to an optical axis of the illumination assembly.
- the processor includes an optical correlator
- the optical correlator includes a diffractive optical element (DOE) containing a reference speckle pattern
- the image capture assembly includes a lenslet array, which is arranged to project multiple sub-images of the object onto the DOE so as to generate respective correlation peaks that are indicative of 3D coordinates of the object.
- DOE diffractive optical element
- the coherent light source has a coherence length that is less than 1 cm.
- the primary speckle pattern includes speckles having a characteristic size, and the illumination assembly is configured so as to permit the characteristic size of the speckles to be adjusted by varying a distance between the coherent light source and the diffuser.
- a method for 3D mapping of an object including:
- apparatus for 3D mapping of an object including:
- an illumination assembly including a coherent light source, having a coherence length less than 1 cm, and a diffuser, which are arranged to project a primary speckle pattern on the object;
- an image capture assembly which is arranged to capture images of the primary speckle pattern on the object
- a processor which is coupled to process the images of the primary speckle pattern so as to derive a 3D map of the object.
- the coherence length of the coherent light source is less than 0.5 mm. Additionally or alternatively, the coherent light source has a divergence greater than 5°.
- FIG. 1 is a schematic, pictorial illustration of a system for 3D mapping, in accordance with an embodiment of the present invention
- FIG. 2 is a schematic top view of a speckle imaging device, in accordance with an embodiment of the present invention.
- FIG. 3 is a flow chart that schematically illustrates a method for 3D mapping, in accordance with an embodiment of the present invention
- FIG. 4 is a schematic side view of an illumination assembly used in a system for 3D mapping, in accordance with another embodiment of the present invention.
- FIG. 5 is a schematic side view of a beam former, in accordance with an embodiment of the present invention.
- FIG. 6 is a schematic side view of a beam former, in accordance with yet another embodiment of the present invention.
- FIG. 7 is a schematic side view of an optical correlator used in a system for 3D mapping, in accordance with a further embodiment of the present invention.
- FIG. 1 is a schematic, pictorial illustration of a system 20 for 3D mapping, in accordance with an embodiment of the present invention.
- System 20 comprises a speckle imaging device 22 , which generates and projects a primary speckle pattern onto an object 28 and captures an image of the primary speckle pattern appearing on the object. Details of the design and operation of device 22 are shown in the figures that follow and are described hereinbelow with reference thereto.
- An image processor 24 processes image data generated by device 22 in order to derive a 3D map of object 28 .
- Image processor 24 which performs such reconstruction, may comprise a general-purpose computer processor, which is programmed in software to carry out the functions described hereinbelow.
- the software may be downloaded to processor 24 in electronic form, over a network, for example, or it may alternatively be provided on tangible media, such as optical, magnetic, or electronic memory media.
- processor 24 may be implemented in dedicated hardware, such as a custom or semi-custom integrated circuit or a programmable digital signal processor (DSP).
- DSP programmable digital signal processor
- processor 24 is shown in FIG. 1 , by way of example, as a separate unit from imaging device 22 , some or all of the processing functions of processor 24 may be performed by suitable dedicated circuitry within the housing of the imaging device or otherwise associated with the imaging device.
- the 3D map that is generated by processor 24 may be used for a wide range of different purposes.
- the map may be sent to an output device, such as a display 26 , which shows a pseudo-3D image of the object.
- object 28 comprises all or a part (such as a hand) of the body of a subject.
- system 20 may be used to provide a gesture-based user interface, in which user movements detected by means of device 22 control an interactive computer application, such as a game, in place of tactile interface elements such as a mouse, joystick or other accessory.
- system 20 may be used to create 3D maps of objects of other types, for substantially any application in which 3D coordinate profiles are needed.
- FIG. 2 is a schematic top view of device 22 , in accordance with an embodiment of the present invention.
- An illumination assembly 30 comprises a coherent light source 32 , typically a laser, and a diffuser 33 .
- the term “light” in the context of the present patent application refers to any sort of optical radiation, including infrared and ultraviolet, as well as visible light.
- the beam of light emitted by source 32 passes through diffuser 33 at a spot 34 of radius w 0 , and thus generates a diverging beam 36 .
- the primary speckle patterns created by diffuser 34 at distances Z obj1 and Z obj2 are to a good approximation linearly-scaled versions of one another, as long as Z obj1 and Z obj2 are within a range of distances given by the axial size of the speckle pattern ⁇ Z at the object distance Z obj ,
- An image capture assembly 38 captures an image of the speckle pattern that is projected onto object 28 .
- Assembly 38 comprises objective optics 39 , which focus the image onto an image sensor 40 .
- sensor 40 comprises a rectilinear array of detector elements 41 , such as a CCD or CMOS-based image sensor array.
- Optics 39 have an entrance pupil 42 , which together with the dimensions of the image sensor defines a field of view 44 of the image capture assembly.
- the sensing volume of device 22 comprises an overlap area 46 between beam 36 and field of view 44 .
- the characteristic transverse speckle size projected by illumination assembly 30 (as defined by the second-order statistics of the speckle pattern) at a distance Z obj is
- the size of the speckles imaged onto sensor 40 should be between one and ten pixels, depending on range and resolution requirements, i.e., each speckle imaged onto sensor 40 by optics 39 should span between one and ten detector elements 41 in the horizontal direction. In typical applications, a speckle size between two and three pixels gives good results.
- the speckle size may be adjusted by varying the distance between light source 32 and diffuser 33 , since the radius w 0 of spot 34 increases with distance from the light source.
- the speckle parameters of illumination assembly 30 can be controlled simply by laterally shifting the light source, without the use of lenses or other optics.
- Illumination assembly 30 can be adjusted in this manner to work with image sensors of different size and resolution and with objective optics of varying magnification.
- an inexpensive light source such as a laser diode, with high divergence (5° or greater) and short coherence length (less than 1 cm, and in some cases even shorter than 0.5 mm) may be used in system 20 with good effect.
- Illumination assembly 30 and image capture assembly 38 are held in a fixed spatial relation by a mount 43 .
- the mount comprises a housing that holds the assemblies.
- any other suitable sort of mechanical mount may be used to maintain the desired spatial relation between the illumination and image capture assemblies.
- the configuration of device 22 and the processing techniques described hereinbelow make it possible to perform 3D mapping using the single image capture assembly, without relative movement between the illumination and image capture assemblies and without moving parts.
- Image capture assembly 38 thus captures images at a single, fixed angle relative to illumination assembly 30 .
- mount 43 hold assemblies 30 and 38 so that the axis passing through the centers of entrance pupil 42 and spot 34 is parallel to one of the axes of sensor 40 .
- the axis passing through pupil 42 and spot 34 should be parallel to one of the array axes, which is taken for convenience to be the X-axis.
- Z-coordinates of points on the object may thus be determined by measuring shifts in the X-coordinates of the speckles in the image captured by assembly 38 relative to a reference image taken at a known distance Z.
- the group of speckles in each area of the captured image are compared to the reference image to find the most closely-matching group of speckles in the reference image.
- the relative shift between the matching groups of speckles in the image gives the Z-direction shift of the area of the captured image relative to the reference image.
- the shift in the speckle pattern may be measured using image correlation or other image matching computation methods that are known in the art.
- the shift of the speckle pattern with ⁇ Z will be strictly in the X-direction, with no Y-component of the shift (as long as distortion due to optics 39 is negligible). Therefore, the image matching computation is simplified and need only seek the closest matching group of speckles subject to X-shift.
- processor may use a parametric model to compensate for the deviation.
- the known deviation may be measured or otherwise modeled, and the processor may then check copies of areas of the current image that are shifted by appropriate (X,Y) shifts relative to the reference image according to the parametric model of the deviation in order to find the actual 3D coordinates of the object surface.
- the operating parameters of system 20 are chosen so that S ⁇ Z obj .
- S since the Z-direction resolution of system 20 depends on the ratio S/Z obj , S must be large enough, relative to the intended working distance of the system, so that the desired resolution can be achieved.
- S ⁇ Z obj the respective distances from the illumination and image capture assemblies to each object point are close, but generally not exactly equal. Therefore, the scale of the speckles in the images of the speckle pattern that are captured by assembly 38 will vary across area 46 by some tolerance ⁇ .
- Computational methods known in the art some of which are described in the above-mentioned PCT patent application, may be used to compensate for these scale variations in matching areas of the current image to corresponding areas of the reference image.
- ⁇ be maintained within some predetermined bound, depending on the matching window size, as well as on the characteristic speckle size.
- ⁇ should be limited so that scaling of a characteristic window varies by no more than about 30% of a single speckle size. Given a diagonal angle ⁇ of the field of view of the image capture assembly 38 ,
- psize(Z obj ) is the size of the pixel at Z obj .
- FIG. 3 is a flow chart that schematically illustrates a method for 3D mapping using system 20 , in accordance with an embodiment of the present invention. This method is based, inter alia, on the realization that the speckle pattern that is projected by illumination assembly 30 does not change substantially over time. Therefore, an individual image of the speckle pattern that is projected onto an object, captured by image capture assembly 38 at a fixed location and angle relative to assembly, may be used to accurately compute a 3D map of the object.
- device 22 is calibrated by projecting the speckle pattern from assembly 30 onto an object of known spatial profile at a known distance from the device, at a calibration step 50 .
- a planar object extending across area 46 at a known distance Z obj is used as a calibration target for this purpose.
- Image capture assembly 38 captures a reference image of the object, which is stored in a memory of processor 24 .
- This calibration step may be carried out at the time of manufacture, and the reference image stored in the memory will then be usable in the field as long as there is no uncontrolled relative motion among the different components of device 22 .
- the reference image may be saved in a data-reduced form, such as a threshold-based binary image, that is appropriate for the matching algorithm that is to be used.
- system 20 When system 20 is ready for use, it is actuated to capture an image of the object of interest (object 28 in this example) using device 22 , at an initial image capture step 52 .
- Processor 24 compares this image to the speckle pattern in the stored calibration image, at a map computation step 54 . Dark areas of the image, in which the pixel values are below some threshold value (or otherwise not containing relevant speckle information), are typically classified as shadow areas, from which depth (Z) information cannot be derived.
- the remainder of the image may be binarized, possibly using an adaptive threshold, as is known in the art, or otherwise data-reduced for efficient matching to the reference image.
- Processor 24 selects a certain window within the non-shadow part of the image, and compares the sub-image within the window to parts of the reference image until the part of the reference image that best matches the sub-image is found.
- assemblies 30 and 38 are aligned along the X-axis, as described above and shown in FIG. 2 , it is sufficient for processor to compare the sub-image to parts of the reference image that are displaced in the X-direction relative to the sub-image (subject to scaling of the speckle pattern by up to the scaling factor ⁇ , as noted above).
- the processor uses the transverse offset of the sub-image relative to the matching part of the reference image to determine the Z-coordinate of the area of the surface of object 28 within the sub-image, based on the principle of triangulation explained above.
- Processor 24 may optionally analyze the speckle distortion in order to estimate the slant angle, and thus improve the accuracy of 3D mapping.
- Processor 24 may use the map coordinates of this first window as a start point for determining the coordinates of neighboring areas of the image. Specifically, once the processor has found a high correlation between a certain area in the image and a corresponding area in the reference image, the offset of this area relative to the reference image can serve as a good predictor of the offsets of neighboring pixels in the image. The processor attempts to match these neighboring pixels to the reference image with an offset equal to or within a small range of the initially-matched area. In this manner, the processor grows the region of the matched area until it reaches the edges of the region. The processor thus proceeds to determine the Z-coordinates of all non-shadow areas of the image, until it has completed the 3D profile of object 28 . This approach has the advantage of providing fast, robust matching even using small windows and images with poor signal/noise ratio. Details of computational methods that may be used for this purpose are described in the above-mentioned PCT patent application.
- processor 24 will have computed a complete 3D map of the part of the object surface that is visible in the initial image.
- the method may readily be extended, however, to capture and analyze successive images in order to track 3D motion of the object, at a next image step 56 .
- Device 22 captures the successive images at some predetermined frame rate, and processor 24 updates the 3D map based on each successive image.
- the 3D maps may be computed with respect to the stored, calibrated reference image if desired. Alternatively, since the object will generally not move too much from one image frame to the next, it is frequently more efficient to use each successive image as a reference image for the next frame.
- processor 24 may compare each successive image to the preceding image in order to compute the X-direction shift of the speckles in each sub-image relative to the same speckles in the preceding image, at a shift computation step 58 .
- the shift is no more than a few pixels, so that the computation can be performed rapidly and efficiently.
- processor 24 outputs the updated 3D map, at a new map output step 60 .
- This process of image capture and update may thus proceed indefinitely.
- system 20 is capable of operating and outputting map coordinates at real-time video rates, on the order of 30 frames/sec or even faster, while using simple, low-cost imaging and processing hardware.
- efficient image matching computation and region growing, as described above may enable system 20 to operate at video speed even when local shifts cannot be computed from preceding images.
- a computer (which may comprise processor 24 or may receive the 3D maps output by the processor) identifies a certain volume or volumes in the 3D maps that correspond to parts of the user's body, such as the arm, hand, and/or fingers, and possibly the head, torso, and other extremities, as well.
- the computer is programmed to identify gestures corresponding to certain movements of these body parts and to control computer applications in response to these gestures. Examples of such gestures and applications include:
- FIG. 4 is a schematic side view of an illumination assembly 70 that may be used in system 20 in order to enhance the useful depth range of the system, in accordance with an embodiment of the present invention.
- Assembly 70 comprises source 32 and diffuser 33 , together with a beam former 72 .
- the beam former is designed to create beam 74 having reduced divergence over an intermediate region 76 , while still preserving the linear scaling of the speckle pattern with axial distance Z within this region.
- high speckle contrast is maintained in images of object 28 throughout region 76 , so that the range of depths covered by the 3D mapping system is increased.
- a number of optical designs that may be used to achieve this enhanced performed in region 76 are described below.
- FIG. 5 is a schematic side view of beam former 72 , in accordance with an embodiment of the present invention.
- the beam former comprises a diffractive optical element (DOE) 80 and an axicon 82 .
- DOE 80 may be butted against diffuser 33 , or even incorporated as an etched or deposited layer on the surface of the diffuser itself.
- Various diffractive designs may be used to reduce the beam divergence in region 76 .
- DOE 80 may comprise a pattern of concentric rings, centered on the optical axis of source 32 , with a random distribution of ring radii.
- Axicon 82 has a conical profile centered on the optical axis, i.e., it is a sort of rotationally-symmetrical prism.
- Both DOE 80 and axicon 82 have the effect of creating long regions of focus along the optical axis, so that either of these elements could be used alone to create a region of reduced beam divergence.
- the reduction of divergence can be further enhanced by using the two elements together.
- FIG. 6 is a schematic side view of a beam former 90 , in accordance with another embodiment of the present invention.
- Beam former 90 comprises a DOE 92 and lenses 94 and 96 , having focal length F.
- the lenses are separated from diffuser 33 and from DOE 92 by distances equal to their focal lengths, so that the DOE is located in the Fourier plane of the diffuser.
- the Fourier transform of the diffuser is multiplied by the transmission function of the DOE.
- the speckle pattern is multiplied by the Fourier transform of the pattern on the DOE.
- the DOE pattern may be chosen so that its Fourier transform provides reduced divergence, as shown above in FIG. 4 , and/or more uniform illumination across the illumination beam.
- the latter object can be achieved by designing element 92 with lower transmittance in its central region than in the periphery (opposite to the angular intensity distribution of the beam from diffuser 33 , which tends to be brighter in the center and fall off with increasing angle from the optical axis).
- Other designs of DOE 92 or DOE 80 ( FIG. 5 ) for the purposes of achieving more uniform speckle contrast over the volume of interest will be apparent to those skilled in the art and are considered to be within the scope of the present invention.
- FIG. 7 is a schematic side view of an optical correlator 110 that may be used in system 20 to determine Z-coordinates of areas of object 28 , in accordance with an embodiment of the present invention.
- Correlator 110 uses optical techniques to carry out some of the functions of processor 24 that were described above.
- the correlator is capable of determining coordinates of multiple areas of the object in parallel at very high speed, almost instantaneously. It is therefore useful particularly in applications that are characterized by rapid object motion.
- a lenslet array 116 forms multiple sub-images of object 28 under speckle illumination by assembly 30 .
- An array 118 of apertures limits the fields of view of the lenslets in array 116 , so that each sub-image contains light from only a narrow angular range.
- a second lenslet array 120 projects the sub-images onto a DOE 122 .
- Array 120 is separated from the plane of the sub-images by a distance equal to the focal length of the lenslets in the array, and is separated from the plane of DOE 122 by an equal distance.
- a rear lenslet array 124 is located between DOE 122 and sensor 40 , separated from each by a distance equal to the focal length of the lenslets.
- DOE 122 contains a reference diffraction pattern that is the spatial Fourier transform of the reference speckle pattern to which the speckle image of object 28 is to be compared.
- the reference diffraction pattern may be the Fourier transform of the calibration speckle image formed at step 50 ( FIG. 3 ), using a flat surface at a known distance from the illumination source.
- the reference diffraction pattern may be deposited or etched on the surface of the DOE.
- DOE 122 may comprise a spatial light modulator (SLM), which is driven to project the reference diffraction pattern dynamically.
- SLM spatial light modulator
- correlator 110 multiplies the sub-images of the object (formed by the lenslets in array 116 ) by the reference speckle pattern in Fourier space. Therefore, the intensity distribution projected onto sensor 40 by lenslet array 124 corresponds to the cross-correlation of each sub-image with the reference speckle pattern.
- the intensity distribution on the sensor will comprise multiple correlation peaks, each peak corresponding to one of the sub-images. The transverse offset of each peak relative to the axis of the corresponding sub-image (as defined by the corresponding aperture in array 118 ) is proportional to the transverse displacement of the speckle pattern on the corresponding area of object 28 .
- This displacement is proportional to the Z-direction displacement of the area relative to the plane of the reference speckle pattern, as explained above.
- the output of sensor 40 may be processed to determine the Z-coordinate of the area of each sub-image, and thus to compute a 3D map of the object.
Abstract
Description
- This application claims the benefit of U.S.
Provisional Patent Application 60/785,187, filed Mar. 24, 2006. This application is a continuation-in-part of PCT Patent Application PCT/IL2006/000335, filed Mar. 14, 2006, which claims the benefit of U.S.Provisional Patent Application 60/724,903, filed Oct. 11, 2005. All of these related applications are assigned to the assignee of the present patent application, and their disclosures are incorporated herein by reference. - The present invention relates generally to methods and systems for mapping of three-dimensional (3D) objects, and specifically to 3D optical imaging using speckle patterns.
- When a coherent beam of light passes through a diffuser and is projected onto a surface, a primary speckle pattern can be observed on the surface. The primary speckle is caused by interference among different components of the diffused beam. The term “primary speckle” is used in this sense in the present patent application and in the claims, in distinction to secondary speckle, which is caused by diffuse reflection of coherent light from the rough surface of an object
- Hart describes the use of a speckle pattern in a high-
speed 3D imaging system, in Taiwanese Patent TW 527528 B and in U.S. patent application Ser. No. 09/616,606, whose disclosures are incorporated herein by reference. The system includes a single-lens camera subsystem with an active imaging element and CCD element, and a correlation processing subsystem. The active imaging element can be a rotating aperture which allows adjustable non-equilateral spacing between defocused images to achieve greater depth of field and higher sub-pixel displacement accuracy. A speckle pattern is projected onto an object, and images of the resulting pattern are acquired from multiple angles. The images are locally cross-correlated using an image correlation technique, and the surface is resolved by using relative camera position information to calculate the three-dimensional coordinates of each locally-correlated region. - Another speckle-based 3D imaging technique is described by Hunter et al., in U.S. Pat. No. 6,101,269, whose disclosure is incorporated herein by reference. A random speckle pattern is projected upon a 3D surface and is imaged by a plurality of cameras to obtain a plurality of two-dimensional digital images. The two-dimensional images are processed to obtain a three-dimensional characterization of the surface.
- Embodiments of the present invention perform accurate, real-time mapping of 3D objects using primary speckle patterns. The methods and systems that are described in the above-mentioned PCT patent application, as well as further embodiments described hereinbelow, are capable of performing such 3D mapping using a single coherent light source and a single image sensor, which is held stationary at a fixed angle relative to the light source.
- In one aspect of the invention, a reference image of the speckle pattern is captured initially on a reference surface of known profile. The 3D profile of an object is then determined by capturing an image of the speckle pattern projected on the object, and comparing the image to the reference image.
- In another aspect of the invention, successive images of the speckle pattern on the object are captured as the object moves. Each image is compared with one or more of its predecessors in order to track the motion of the object in three dimensions. In one embodiment, which is described hereinbelow, the light source and image sensor are held in a linear alignment that permits rapid, accurate motion tracking by computing one-dimensional correlation coefficients between successive images.
- In some embodiments, novel illumination and image processing schemes are used to enhance the accuracy, depth of field, and computational speed of the 3D mapping system.
- There is therefore provided, in accordance with an embodiment of the present invention, apparatus for 3D mapping of an object, including:
- an illumination assembly, including a coherent light source and a diffuser, which are arranged to project a primary speckle pattern on the object;
- a single image capture assembly, which is arranged to capture images of the primary speckle pattern on the object from a single, fixed location and angle relative to the illumination assembly; and
- a processor, which is coupled to process the images of the primary speckle pattern captured at the single, fixed angle so as to derive a 3D map of the object.
- In some embodiments, the apparatus includes a mount, which is attached to the illumination assembly and the image capture assembly so as to hold the image capture assembly in a fixed spatial relation to the illumination assembly. In one embodiment, the image capture assembly includes an array of detector elements arranged in a rectilinear pattern defining first and second, mutually-perpendicular axes, and objective optics, which have an entrance pupil and are arranged to focus the image onto the array, wherein the illumination assembly and image capture assembly are aligned by the mount so as to define a device axis that is parallel to the first axis and passes through the entrance pupil and through a spot at which a beam emitted by the coherent light source passes through the diffuser. Thus, the processor is arranged to derive the 3D map by finding an offset along only the first axis between the primary speckle pattern captured in one or more of the images and a reference image of the primary speckle pattern.
- In some embodiments, the processor is arranged to derive the 3D map by finding respective offsets between the primary speckle pattern on multiple areas of the object captured in one or more of the images and a reference image of the primary speckle pattern, wherein the respective offsets are indicative of respective distances between the areas and the image capture assembly. Typically, the image capture assembly is located at a predetermined spacing from the illumination assembly, and the respective offsets are proportional to the respective distances in a ratio that is determined by the spacing. In a disclosed embodiment, the primary speckle pattern projected by the illumination assembly includes speckles having a characteristic size, and the size of the speckles in the images varies across the image by a tolerance that depends on the spacing, wherein the spacing is selected so as to maintain the tolerance within a predefined bound.
- Additionally or alternatively, the processor is arranged to relate the respective offsets to respective coordinates in the 3D map using a parametric model of distortion in the image capture assembly. Further additionally or alternatively, the processor is arranged to find the respective offsets by finding an initial match between the primary speckle pattern in a first area of the object and a corresponding area of the reference image at a first offset relative to the first area, and to apply a region growing procedure, based on the first offset, to find the respective offsets of pixels adjacent to the first area.
- In a disclosed embodiment, the processor is arranged to process a succession of the images captured while the object is moving so as to map a 3D movement of the object, wherein the object is a part of a human body, and wherein the 3D movement includes a gesture made by the part of the human body, and wherein the processor is coupled to provide an input to a computer application responsively to the gesture.
- In some embodiments, the illumination assembly includes a beam former, which is arranged to reduce a variation of a contrast of the speckle pattern created by the diffuser over a sensing volume of the apparatus. In one embodiment, the beam former includes a diffractive optical element (DOE) and a lens arranged to define a Fourier plane of the diffuser, wherein the DOE is located in the Fourier plane. The beam former may be arranged to reduce a divergence of light emitted from the diffuser or to equalize an intensity of light emitted from the diffuser across a plane transverse to an optical axis of the illumination assembly.
- In one embodiment, the processor includes an optical correlator, the optical correlator includes a diffractive optical element (DOE) containing a reference speckle pattern, and the image capture assembly includes a lenslet array, which is arranged to project multiple sub-images of the object onto the DOE so as to generate respective correlation peaks that are indicative of 3D coordinates of the object.
- In some embodiments, the coherent light source has a coherence length that is less than 1 cm. Additionally or alternatively, the primary speckle pattern includes speckles having a characteristic size, and the illumination assembly is configured so as to permit the characteristic size of the speckles to be adjusted by varying a distance between the coherent light source and the diffuser.
- There is also provided, in accordance with an embodiment of the present invention, a method for 3D mapping of an object, including:
- illuminating an object with a beam of diffused coherent light from a light source so as to project a primary speckle pattern on the object;
- capturing images of the primary speckle pattern on the object from a single, fixed location and angle relative to the light source; and
- processing the images of the primary speckle pattern captured at the single, fixed angle so as to derive a 3D map of the object.
- There is additionally provided, in accordance with an embodiment of the present invention, apparatus for 3D mapping of an object, including:
- an illumination assembly, including a coherent light source, having a coherence length less than 1 cm, and a diffuser, which are arranged to project a primary speckle pattern on the object;
- an image capture assembly, which is arranged to capture images of the primary speckle pattern on the object; and
- a processor, which is coupled to process the images of the primary speckle pattern so as to derive a 3D map of the object.
- In one embodiment, the coherence length of the coherent light source is less than 0.5 mm. Additionally or alternatively, the coherent light source has a divergence greater than 5°.
- The present invention will be more fully understood from the following detailed description of the embodiments thereof, taken together with the drawings in which:
-
FIG. 1 is a schematic, pictorial illustration of a system for 3D mapping, in accordance with an embodiment of the present invention; -
FIG. 2 is a schematic top view of a speckle imaging device, in accordance with an embodiment of the present invention; -
FIG. 3 is a flow chart that schematically illustrates a method for 3D mapping, in accordance with an embodiment of the present invention; -
FIG. 4 is a schematic side view of an illumination assembly used in a system for 3D mapping, in accordance with another embodiment of the present invention; -
FIG. 5 is a schematic side view of a beam former, in accordance with an embodiment of the present invention; -
FIG. 6 is a schematic side view of a beam former, in accordance with yet another embodiment of the present invention; and -
FIG. 7 is a schematic side view of an optical correlator used in a system for 3D mapping, in accordance with a further embodiment of the present invention. -
FIG. 1 is a schematic, pictorial illustration of asystem 20 for 3D mapping, in accordance with an embodiment of the present invention.System 20 comprises aspeckle imaging device 22, which generates and projects a primary speckle pattern onto anobject 28 and captures an image of the primary speckle pattern appearing on the object. Details of the design and operation ofdevice 22 are shown in the figures that follow and are described hereinbelow with reference thereto. - An
image processor 24 processes image data generated bydevice 22 in order to derive a 3D map ofobject 28. The term “3D map,” as used in the present patent application and in the claims, refers to a set of 3D coordinates representing the surface of an object. The derivation of such a map based on image data may also be referred to as “3D reconstruction.”Image processor 24, which performs such reconstruction, may comprise a general-purpose computer processor, which is programmed in software to carry out the functions described hereinbelow. The software may be downloaded toprocessor 24 in electronic form, over a network, for example, or it may alternatively be provided on tangible media, such as optical, magnetic, or electronic memory media. Alternatively or additionally, some or all of the functions of the image processor may be implemented in dedicated hardware, such as a custom or semi-custom integrated circuit or a programmable digital signal processor (DSP). Althoughprocessor 24 is shown inFIG. 1 , by way of example, as a separate unit fromimaging device 22, some or all of the processing functions ofprocessor 24 may be performed by suitable dedicated circuitry within the housing of the imaging device or otherwise associated with the imaging device. - The 3D map that is generated by
processor 24 may be used for a wide range of different purposes. For example, the map may be sent to an output device, such as adisplay 26, which shows a pseudo-3D image of the object. In the example shown inFIG. 1 , object 28 comprises all or a part (such as a hand) of the body of a subject. In this case,system 20 may be used to provide a gesture-based user interface, in which user movements detected by means ofdevice 22 control an interactive computer application, such as a game, in place of tactile interface elements such as a mouse, joystick or other accessory. Alternatively,system 20 may be used to create 3D maps of objects of other types, for substantially any application in which 3D coordinate profiles are needed. -
FIG. 2 is a schematic top view ofdevice 22, in accordance with an embodiment of the present invention. Anillumination assembly 30 comprises a coherentlight source 32, typically a laser, and adiffuser 33. (The term “light” in the context of the present patent application refers to any sort of optical radiation, including infrared and ultraviolet, as well as visible light.) The beam of light emitted bysource 32 passes throughdiffuser 33 at aspot 34 of radius w0, and thus generates a divergingbeam 36. As explained in the above-mentioned PCT Patent Application PCT/IL2006/000335, the primary speckle patterns created bydiffuser 34 at distances Zobj1 and Zobj2 are to a good approximation linearly-scaled versions of one another, as long as Zobj1 and Zobj2 are within a range of distances given by the axial size of the speckle pattern ΔZ at the object distance Zobj, -
- An
image capture assembly 38 captures an image of the speckle pattern that is projected ontoobject 28.Assembly 38 comprisesobjective optics 39, which focus the image onto animage sensor 40. Typically,sensor 40 comprises a rectilinear array ofdetector elements 41, such as a CCD or CMOS-based image sensor array.Optics 39 have anentrance pupil 42, which together with the dimensions of the image sensor defines a field ofview 44 of the image capture assembly. The sensing volume ofdevice 22 comprises anoverlap area 46 betweenbeam 36 and field ofview 44. - The characteristic transverse speckle size projected by illumination assembly 30 (as defined by the second-order statistics of the speckle pattern) at a distance Zobj is
-
- The inventors have found that for optimal image processing performance, the size of the speckles imaged onto
sensor 40 should be between one and ten pixels, depending on range and resolution requirements, i.e., each speckle imaged ontosensor 40 byoptics 39 should span between one and tendetector elements 41 in the horizontal direction. In typical applications, a speckle size between two and three pixels gives good results. - It can be seen from the formula above for ΔX that the speckle size may be adjusted by varying the distance between
light source 32 anddiffuser 33, since the radius w0 ofspot 34 increases with distance from the light source. Thus, the speckle parameters ofillumination assembly 30 can be controlled simply by laterally shifting the light source, without the use of lenses or other optics.Illumination assembly 30 can be adjusted in this manner to work with image sensors of different size and resolution and with objective optics of varying magnification. Given the small speckle size that is mandated by the above parameters, an inexpensive light source, such as a laser diode, with high divergence (5° or greater) and short coherence length (less than 1 cm, and in some cases even shorter than 0.5 mm) may be used insystem 20 with good effect. -
Illumination assembly 30 andimage capture assembly 38 are held in a fixed spatial relation by amount 43. In the embodiment shown inFIG. 2 , the mount comprises a housing that holds the assemblies. Alternatively, any other suitable sort of mechanical mount may be used to maintain the desired spatial relation between the illumination and image capture assemblies. The configuration ofdevice 22 and the processing techniques described hereinbelow make it possible to perform 3D mapping using the single image capture assembly, without relative movement between the illumination and image capture assemblies and without moving parts.Image capture assembly 38 thus captures images at a single, fixed angle relative toillumination assembly 30. - To simplify the computation of the 3D map and of changes in the map due to motion of
object 28, as described hereinbelow, it is desirable that mount 43hold assemblies entrance pupil 42 andspot 34 is parallel to one of the axes ofsensor 40. In other words, taking the rows and columns of the array ofdetector elements 41 to define mutually-perpendicular X- and Y-axes (with the origin on the optical axis of objective optics 39), the axis passing throughpupil 42 andspot 34 should be parallel to one of the array axes, which is taken for convenience to be the X-axis. The advantages of this arrangement are explained further hereinbelow. - The respective optical axes of
assemblies 30 and 38 (which pass through the centers ofspot 34 andpupil 42, respectively) are separated by a distance S. Therefore, variations in Zobj will cause distortions of the speckle pattern in images of the object captured byimage capture assembly 38. Specifically, by triangulation, it can be seen inFIG. 2 that a Z-direction shift of a point on the object, δZ, will engender a concomitant transverse shift δX in the speckle pattern observed in the image, such that -
- Z-coordinates of points on the object, as well as shifts in the Z-coordinates over time, may thus be determined by measuring shifts in the X-coordinates of the speckles in the image captured by
assembly 38 relative to a reference image taken at a known distance Z. In other words, the group of speckles in each area of the captured image are compared to the reference image to find the most closely-matching group of speckles in the reference image. The relative shift between the matching groups of speckles in the image gives the Z-direction shift of the area of the captured image relative to the reference image. The shift in the speckle pattern may be measured using image correlation or other image matching computation methods that are known in the art. Some exemplary methods are described in the above-mentioned PCT patent application. Another method that is particularly useful in conjunction withdevice 22 is described in U.S.Provisional Patent Application 60/785,202, filed Mar. 24, 2006, which is assigned to the assignee of the present patent application and whose disclosure is incorporated herein by reference. - Furthermore, in the arrangement shown in
FIG. 2 , in which the X-axis passing throughpupil 42 andspot 34 is parallel to the X-axis ofsensor 40, the shift of the speckle pattern with δZ will be strictly in the X-direction, with no Y-component of the shift (as long as distortion due tooptics 39 is negligible). Therefore, the image matching computation is simplified and need only seek the closest matching group of speckles subject to X-shift. In other words, to determine δZ of an area in the current image relative to a reference image (which may be any previous image of the speckle pattern), it is necessary only to check X-shifted copies of areas of the current image against the reference image in order to find the value of the shift δX that gives the best match to the reference image. - Alternatively, if the geometrical alignment of the elements of
device 22 deviates from the above criteria, or if lens distortions are significant, processor may use a parametric model to compensate for the deviation. In other words, the known deviation may be measured or otherwise modeled, and the processor may then check copies of areas of the current image that are shifted by appropriate (X,Y) shifts relative to the reference image according to the parametric model of the deviation in order to find the actual 3D coordinates of the object surface. - Typically, for convenience of construction and computation, the operating parameters of
system 20 are chosen so that S<<Zobj. (On the other hand, since the Z-direction resolution ofsystem 20 depends on the ratio S/Zobj, S must be large enough, relative to the intended working distance of the system, so that the desired resolution can be achieved.) As long as S<<Zobj, the respective distances from the illumination and image capture assemblies to each object point are close, but generally not exactly equal. Therefore, the scale of the speckles in the images of the speckle pattern that are captured byassembly 38 will vary acrossarea 46 by some tolerance γ. Computational methods known in the art, some of which are described in the above-mentioned PCT patent application, may be used to compensate for these scale variations in matching areas of the current image to corresponding areas of the reference image. - In general, however, to avoid placing too great a computational load on
processor 24, it is desirable that γ be maintained within some predetermined bound, depending on the matching window size, as well as on the characteristic speckle size. Generally, the inventors have found that γ should be limited so that scaling of a characteristic window varies by no more than about 30% of a single speckle size. Given a diagonal angle θ of the field of view of theimage capture assembly 38, -
- Therefore, substantial scale invariance of the local speckle pattern for window of size N is obtained when
-
- wherein psize(Zobj) is the size of the pixel at Zobj. Under these conditions, the Z-direction shifts of the object in successive image frames captured by
assembly 38 can generally be computed without explicitly taking speckle scaling variations into account. -
FIG. 3 is a flow chart that schematically illustrates a method for 3Dmapping using system 20, in accordance with an embodiment of the present invention. This method is based, inter alia, on the realization that the speckle pattern that is projected byillumination assembly 30 does not change substantially over time. Therefore, an individual image of the speckle pattern that is projected onto an object, captured byimage capture assembly 38 at a fixed location and angle relative to assembly, may be used to accurately compute a 3D map of the object. - Before mapping an object,
device 22 is calibrated by projecting the speckle pattern fromassembly 30 onto an object of known spatial profile at a known distance from the device, at acalibration step 50. Typically, a planar object extending acrossarea 46 at a known distance Zobj is used as a calibration target for this purpose.Image capture assembly 38 captures a reference image of the object, which is stored in a memory ofprocessor 24. This calibration step may be carried out at the time of manufacture, and the reference image stored in the memory will then be usable in the field as long as there is no uncontrolled relative motion among the different components ofdevice 22. To save memory and simplify subsequent computation, the reference image may be saved in a data-reduced form, such as a threshold-based binary image, that is appropriate for the matching algorithm that is to be used. - When
system 20 is ready for use, it is actuated to capture an image of the object of interest (object 28 in this example) usingdevice 22, at an initialimage capture step 52.Processor 24 compares this image to the speckle pattern in the stored calibration image, at amap computation step 54. Dark areas of the image, in which the pixel values are below some threshold value (or otherwise not containing relevant speckle information), are typically classified as shadow areas, from which depth (Z) information cannot be derived. The remainder of the image may be binarized, possibly using an adaptive threshold, as is known in the art, or otherwise data-reduced for efficient matching to the reference image. -
Processor 24 selects a certain window within the non-shadow part of the image, and compares the sub-image within the window to parts of the reference image until the part of the reference image that best matches the sub-image is found. Whenassemblies FIG. 2 , it is sufficient for processor to compare the sub-image to parts of the reference image that are displaced in the X-direction relative to the sub-image (subject to scaling of the speckle pattern by up to the scaling factor γ, as noted above). The processor uses the transverse offset of the sub-image relative to the matching part of the reference image to determine the Z-coordinate of the area of the surface ofobject 28 within the sub-image, based on the principle of triangulation explained above. If this area of the object surface is slanted, rather than being oriented in an X-Y plane, then the speckle pattern in the sub-image will exhibit distortion.Processor 24 may optionally analyze the speckle distortion in order to estimate the slant angle, and thus improve the accuracy of 3D mapping. -
Processor 24 may use the map coordinates of this first window as a start point for determining the coordinates of neighboring areas of the image. Specifically, once the processor has found a high correlation between a certain area in the image and a corresponding area in the reference image, the offset of this area relative to the reference image can serve as a good predictor of the offsets of neighboring pixels in the image. The processor attempts to match these neighboring pixels to the reference image with an offset equal to or within a small range of the initially-matched area. In this manner, the processor grows the region of the matched area until it reaches the edges of the region. The processor thus proceeds to determine the Z-coordinates of all non-shadow areas of the image, until it has completed the 3D profile ofobject 28. This approach has the advantage of providing fast, robust matching even using small windows and images with poor signal/noise ratio. Details of computational methods that may be used for this purpose are described in the above-mentioned PCT patent application. - At the conclusion of the above steps,
processor 24 will have computed a complete 3D map of the part of the object surface that is visible in the initial image. The method may readily be extended, however, to capture and analyze successive images in order to track 3D motion of the object, at anext image step 56.Device 22 captures the successive images at some predetermined frame rate, andprocessor 24 updates the 3D map based on each successive image. The 3D maps may be computed with respect to the stored, calibrated reference image if desired. Alternatively, since the object will generally not move too much from one image frame to the next, it is frequently more efficient to use each successive image as a reference image for the next frame. - Thus,
processor 24 may compare each successive image to the preceding image in order to compute the X-direction shift of the speckles in each sub-image relative to the same speckles in the preceding image, at ashift computation step 58. Typically, the shift is no more than a few pixels, so that the computation can be performed rapidly and efficiently. After each new image is processed in this manner,processor 24 outputs the updated 3D map, at a newmap output step 60. This process of image capture and update may thus proceed indefinitely. Because of the ease of computation of successive 3D maps,system 20 is capable of operating and outputting map coordinates at real-time video rates, on the order of 30 frames/sec or even faster, while using simple, low-cost imaging and processing hardware. Furthermore, efficient image matching computation and region growing, as described above, may enablesystem 20 to operate at video speed even when local shifts cannot be computed from preceding images. - These capabilities of
system 20 make it suitable for use in a wide range of applications, and particularly in implementing machine interfaces based on human gestures. In such an interface, a computer (which may compriseprocessor 24 or may receive the 3D maps output by the processor) identifies a certain volume or volumes in the 3D maps that correspond to parts of the user's body, such as the arm, hand, and/or fingers, and possibly the head, torso, and other extremities, as well. The computer is programmed to identify gestures corresponding to certain movements of these body parts and to control computer applications in response to these gestures. Examples of such gestures and applications include: -
- Mouse translation and clicking—The computer interprets motion of the user's hand and fingers as though the user were moving a mouse on a table and clicking the mouse buttons.
- Freehand pointing to, selection and translation of objects on the computer screen.
- Computer games, in which user gestures may hit, grasp, move and release real or virtual objects used in the game.
- Computer interface for handicapped users, based on sensing a limited range of motions that the user is capable of making.
- Typing on a virtual keyboard.
Other applications will be apparent to those skilled in the art.
- Returning now to
FIG. 2 , asbeam 36 spreads beyond the Rayleigh distance, the intensity of illumination falling onobject 28 drops in approximate proportion to Z2. The contrast of the speckle pattern projected on the object will drop accordingly, particularly when there is strong ambient light at the wavelength ofsource 32. The range of depth (Z-coordinates) over whichsystem 20 provides useful results may thus be limited on account of weak illumination at large Z. This limitation may be mitigated by methods of adaptive control and image processing, as are known in the art. Some suitable methods of this sort are described in the above mentioned PCT Patent Application PCT/IL2006/000335. Alternatively or additionally, optical beam forming may be applied to improve the illumination profile, as described hereinbelow. -
FIG. 4 is a schematic side view of anillumination assembly 70 that may be used insystem 20 in order to enhance the useful depth range of the system, in accordance with an embodiment of the present invention.Assembly 70 comprisessource 32 anddiffuser 33, together with a beam former 72. The beam former is designed to createbeam 74 having reduced divergence over anintermediate region 76, while still preserving the linear scaling of the speckle pattern with axial distance Z within this region. As a result, high speckle contrast is maintained in images ofobject 28 throughoutregion 76, so that the range of depths covered by the 3D mapping system is increased. A number of optical designs that may be used to achieve this enhanced performed inregion 76 are described below. -
FIG. 5 is a schematic side view of beam former 72, in accordance with an embodiment of the present invention. The beam former comprises a diffractive optical element (DOE) 80 and anaxicon 82.DOE 80 may be butted againstdiffuser 33, or even incorporated as an etched or deposited layer on the surface of the diffuser itself. Various diffractive designs may be used to reduce the beam divergence inregion 76. For example,DOE 80 may comprise a pattern of concentric rings, centered on the optical axis ofsource 32, with a random distribution of ring radii.Axicon 82 has a conical profile centered on the optical axis, i.e., it is a sort of rotationally-symmetrical prism. BothDOE 80 andaxicon 82 have the effect of creating long regions of focus along the optical axis, so that either of these elements could be used alone to create a region of reduced beam divergence. The reduction of divergence can be further enhanced by using the two elements together. -
FIG. 6 is a schematic side view of a beam former 90, in accordance with another embodiment of the present invention. Beam former 90 comprises aDOE 92 andlenses diffuser 33 and fromDOE 92 by distances equal to their focal lengths, so that the DOE is located in the Fourier plane of the diffuser. Thus, the Fourier transform of the diffuser is multiplied by the transmission function of the DOE. In the far field, the speckle pattern is multiplied by the Fourier transform of the pattern on the DOE. - The DOE pattern may be chosen so that its Fourier transform provides reduced divergence, as shown above in
FIG. 4 , and/or more uniform illumination across the illumination beam. The latter object can be achieved by designingelement 92 with lower transmittance in its central region than in the periphery (opposite to the angular intensity distribution of the beam fromdiffuser 33, which tends to be brighter in the center and fall off with increasing angle from the optical axis). Other designs ofDOE 92 or DOE 80 (FIG. 5 ) for the purposes of achieving more uniform speckle contrast over the volume of interest will be apparent to those skilled in the art and are considered to be within the scope of the present invention. -
FIG. 7 is a schematic side view of an optical correlator 110 that may be used insystem 20 to determine Z-coordinates of areas ofobject 28, in accordance with an embodiment of the present invention. Correlator 110, in other words, uses optical techniques to carry out some of the functions ofprocessor 24 that were described above. The correlator is capable of determining coordinates of multiple areas of the object in parallel at very high speed, almost instantaneously. It is therefore useful particularly in applications that are characterized by rapid object motion. - A
lenslet array 116 forms multiple sub-images ofobject 28 under speckle illumination byassembly 30. Anarray 118 of apertures limits the fields of view of the lenslets inarray 116, so that each sub-image contains light from only a narrow angular range. Asecond lenslet array 120 projects the sub-images onto aDOE 122.Array 120 is separated from the plane of the sub-images by a distance equal to the focal length of the lenslets in the array, and is separated from the plane ofDOE 122 by an equal distance. Arear lenslet array 124 is located betweenDOE 122 andsensor 40, separated from each by a distance equal to the focal length of the lenslets. -
DOE 122 contains a reference diffraction pattern that is the spatial Fourier transform of the reference speckle pattern to which the speckle image ofobject 28 is to be compared. For example, the reference diffraction pattern may be the Fourier transform of the calibration speckle image formed at step 50 (FIG. 3 ), using a flat surface at a known distance from the illumination source. In this case, the reference diffraction pattern may be deposited or etched on the surface of the DOE. Alternatively,DOE 122 may comprise a spatial light modulator (SLM), which is driven to project the reference diffraction pattern dynamically. - In either case, correlator 110 multiplies the sub-images of the object (formed by the lenslets in array 116) by the reference speckle pattern in Fourier space. Therefore, the intensity distribution projected onto
sensor 40 bylenslet array 124 corresponds to the cross-correlation of each sub-image with the reference speckle pattern. In general, the intensity distribution on the sensor will comprise multiple correlation peaks, each peak corresponding to one of the sub-images. The transverse offset of each peak relative to the axis of the corresponding sub-image (as defined by the corresponding aperture in array 118) is proportional to the transverse displacement of the speckle pattern on the corresponding area ofobject 28. This displacement, in turn, is proportional to the Z-direction displacement of the area relative to the plane of the reference speckle pattern, as explained above. Thus, the output ofsensor 40 may be processed to determine the Z-coordinate of the area of each sub-image, and thus to compute a 3D map of the object. - Although the embodiments described above relate to the specific configuration of
system 20 and design ofdevice 22 that are described above, certain principles of the present invention may similarly be applied in systems and devices of other types for speckle-based 3D mapping. For example, aspects of the embodiments described above may be applied in systems in that use multiple image capture assemblies, or in which the image capture assembly and the illumination assembly are movable relative to one another. - It will thus be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.
Claims (44)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/282,517 US8390821B2 (en) | 2005-10-11 | 2007-03-08 | Three-dimensional sensing using speckle patterns |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US72490305P | 2005-10-11 | 2005-10-11 | |
PCT/IL2006/000335 WO2007043036A1 (en) | 2005-10-11 | 2006-03-14 | Method and system for object reconstruction |
US78518706P | 2006-03-24 | 2006-03-24 | |
PCT/IL2007/000306 WO2007105205A2 (en) | 2006-03-14 | 2007-03-08 | Three-dimensional sensing using speckle patterns |
US12/282,517 US8390821B2 (en) | 2005-10-11 | 2007-03-08 | Three-dimensional sensing using speckle patterns |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IL2006/000335 Continuation-In-Part WO2007043036A1 (en) | 2005-03-30 | 2006-03-14 | Method and system for object reconstruction |
PCT/IL2007/000306 A-371-Of-International WO2007105205A2 (en) | 2005-10-11 | 2007-03-08 | Three-dimensional sensing using speckle patterns |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/605,340 Continuation-In-Part US20110096182A1 (en) | 2005-10-11 | 2009-10-25 | Error Compensation in Three-Dimensional Mapping |
US13/748,617 Continuation US9063283B2 (en) | 2005-10-11 | 2013-01-24 | Pattern generation using a diffraction pattern that is a spatial fourier transform of a random pattern |
Publications (2)
Publication Number | Publication Date |
---|---|
US20090096783A1 true US20090096783A1 (en) | 2009-04-16 |
US8390821B2 US8390821B2 (en) | 2013-03-05 |
Family
ID=38509871
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/282,517 Active 2029-03-29 US8390821B2 (en) | 2005-10-11 | 2007-03-08 | Three-dimensional sensing using speckle patterns |
US13/748,617 Active 2026-10-07 US9063283B2 (en) | 2005-10-11 | 2013-01-24 | Pattern generation using a diffraction pattern that is a spatial fourier transform of a random pattern |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/748,617 Active 2026-10-07 US9063283B2 (en) | 2005-10-11 | 2013-01-24 | Pattern generation using a diffraction pattern that is a spatial fourier transform of a random pattern |
Country Status (5)
Country | Link |
---|---|
US (2) | US8390821B2 (en) |
JP (1) | JP5174684B2 (en) |
KR (1) | KR101331543B1 (en) |
CN (1) | CN101496033B (en) |
WO (1) | WO2007105205A2 (en) |
Cited By (127)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080106746A1 (en) * | 2005-10-11 | 2008-05-08 | Alexander Shpunt | Depth-varying light fields for three dimensional sensing |
US20080240502A1 (en) * | 2007-04-02 | 2008-10-02 | Barak Freedman | Depth mapping using projected patterns |
US20090183125A1 (en) * | 2008-01-14 | 2009-07-16 | Prime Sense Ltd. | Three-dimensional user interface |
US20090185274A1 (en) * | 2008-01-21 | 2009-07-23 | Prime Sense Ltd. | Optical designs for zero order reduction |
US20100020078A1 (en) * | 2007-01-21 | 2010-01-28 | Prime Sense Ltd | Depth mapping using multi-beam illumination |
US20100034457A1 (en) * | 2006-05-11 | 2010-02-11 | Tamir Berliner | Modeling of humanoid forms from depth maps |
US20100177164A1 (en) * | 2005-10-11 | 2010-07-15 | Zeev Zalevsky | Method and System for Object Reconstruction |
US20100284082A1 (en) * | 2008-01-21 | 2010-11-11 | Primesense Ltd. | Optical pattern projection |
US20100290698A1 (en) * | 2007-06-19 | 2010-11-18 | Prime Sense Ltd | Distance-Varying Illumination and Imaging Techniques for Depth Mapping |
US20100309301A1 (en) * | 2007-12-04 | 2010-12-09 | Sirona Dental Systems Gmbh | Recording method for obtaining an image of an object and recording device |
US20110052006A1 (en) * | 2009-08-13 | 2011-03-03 | Primesense Ltd. | Extraction of skeletons from 3d maps |
US20110080475A1 (en) * | 2009-10-07 | 2011-04-07 | Microsoft Corporation | Methods And Systems For Determining And Tracking Extremities Of A Target |
US20110080336A1 (en) * | 2009-10-07 | 2011-04-07 | Microsoft Corporation | Human Tracking System |
US20110081044A1 (en) * | 2009-10-07 | 2011-04-07 | Microsoft Corporation | Systems And Methods For Removing A Background Of An Image |
US20110096182A1 (en) * | 2009-10-25 | 2011-04-28 | Prime Sense Ltd | Error Compensation in Three-Dimensional Mapping |
US20110114857A1 (en) * | 2009-11-15 | 2011-05-19 | Primesense Ltd. | Optical projector with beam monitor |
US20110134114A1 (en) * | 2009-12-06 | 2011-06-09 | Primesense Ltd. | Depth-based gain control |
US20110154201A1 (en) * | 2009-12-22 | 2011-06-23 | Akira Nakanishi | Video Reproducing Apparatus and Video Reproducing Method |
US20110158508A1 (en) * | 2005-10-11 | 2011-06-30 | Primesense Ltd. | Depth-varying light fields for three dimensional sensing |
US20110188054A1 (en) * | 2010-02-02 | 2011-08-04 | Primesense Ltd | Integrated photonics module for optical projection |
US20110187878A1 (en) * | 2010-02-02 | 2011-08-04 | Primesense Ltd. | Synchronization of projected illumination with rolling shutter of image sensor |
US20110211754A1 (en) * | 2010-03-01 | 2011-09-01 | Primesense Ltd. | Tracking body parts by combined color image and depth processing |
US20110211044A1 (en) * | 2010-03-01 | 2011-09-01 | Primesense Ltd. | Non-Uniform Spatial Resource Allocation for Depth Mapping |
WO2011146259A2 (en) | 2010-05-20 | 2011-11-24 | Irobot Corporation | Mobile human interface robot |
US20120105326A1 (en) * | 2010-11-03 | 2012-05-03 | Samsung Electronics Co., Ltd. | Method and apparatus for generating motion information |
WO2012091807A2 (en) | 2010-12-30 | 2012-07-05 | Irobot Corporation | Mobile human interface robot |
WO2012091814A2 (en) | 2010-12-30 | 2012-07-05 | Irobot Corporation | Mobile robot system |
US20120313875A1 (en) * | 2011-06-13 | 2012-12-13 | Sharp Kabushiki Kaisha | Manual operating device |
US20120313896A1 (en) * | 2011-06-07 | 2012-12-13 | Sony Corporation | Information processing apparatus, information processing method, and program |
US20130031517A1 (en) * | 2011-07-28 | 2013-01-31 | Dustin Freeman | Hand pose interaction |
US20130033459A1 (en) * | 2010-04-13 | 2013-02-07 | Nokia Corporation | Apparatus, method, computer program and user interface |
US20130055120A1 (en) * | 2011-08-24 | 2013-02-28 | Primesense Ltd. | Sessionless pointing user interface |
US8456517B2 (en) | 2008-07-09 | 2013-06-04 | Primesense Ltd. | Integrated processor for 3D mapping |
US8462207B2 (en) | 2009-02-12 | 2013-06-11 | Primesense Ltd. | Depth ranging with Moiré patterns |
US8493496B2 (en) | 2007-04-02 | 2013-07-23 | Primesense Ltd. | Depth mapping using projected patterns |
US20130222874A1 (en) * | 2012-02-29 | 2013-08-29 | Lg Electronics Inc. | Holographic display device and method for generating hologram using redundancy of 3d video |
WO2013130734A1 (en) | 2012-02-29 | 2013-09-06 | Irobot Corporation | Mobile robot |
US8582867B2 (en) | 2010-09-16 | 2013-11-12 | Primesense Ltd | Learning-based pose estimation from depth maps |
US8594425B2 (en) | 2010-05-31 | 2013-11-26 | Primesense Ltd. | Analysis of three-dimensional scenes |
US8599484B2 (en) | 2010-08-10 | 2013-12-03 | Asahi Glass Company, Limited | Diffractive optical element and measuring device |
WO2014033614A1 (en) | 2012-08-27 | 2014-03-06 | Koninklijke Philips N.V. | Patient-specific and automatic x-ray system adjustment based on optical 3d scene detection and interpretation |
US20140118496A1 (en) * | 2012-10-31 | 2014-05-01 | Ricoh Company, Ltd. | Pre-Calculation of Sine Waves for Pixel Values |
US8717417B2 (en) | 2009-04-16 | 2014-05-06 | Primesense Ltd. | Three-dimensional mapping and imaging |
WO2014071254A1 (en) | 2012-11-01 | 2014-05-08 | Eyecam, LLC | Wireless wrist computing and control device and method for 3d imaging, mapping, networking and interfacing |
US8749796B2 (en) | 2011-08-09 | 2014-06-10 | Primesense Ltd. | Projectors of structured light |
US8786682B2 (en) | 2009-03-05 | 2014-07-22 | Primesense Ltd. | Reference image techniques for three-dimensional sensing |
US20140307307A1 (en) * | 2013-04-15 | 2014-10-16 | Microsoft Corporation | Diffractive optical element with undiffracted light expansion for eye safe operation |
US8872762B2 (en) | 2010-12-08 | 2014-10-28 | Primesense Ltd. | Three dimensional user interface cursor control |
US8881051B2 (en) | 2011-07-05 | 2014-11-04 | Primesense Ltd | Zoom-based gesture user interface |
US8891827B2 (en) | 2009-10-07 | 2014-11-18 | Microsoft Corporation | Systems and methods for tracking a model |
US8908277B2 (en) | 2011-08-09 | 2014-12-09 | Apple Inc | Lens array projector |
US8918213B2 (en) | 2010-05-20 | 2014-12-23 | Irobot Corporation | Mobile human interface robot |
US8918209B2 (en) | 2010-05-20 | 2014-12-23 | Irobot Corporation | Mobile human interface robot |
WO2014202720A1 (en) | 2013-06-19 | 2014-12-24 | Koninklijke Philips N.V. | Calibration of imagers with dynamic beam shapers |
US8930019B2 (en) | 2010-12-30 | 2015-01-06 | Irobot Corporation | Mobile human interface robot |
US8935005B2 (en) | 2010-05-20 | 2015-01-13 | Irobot Corporation | Operating a mobile robot |
US8933876B2 (en) | 2010-12-13 | 2015-01-13 | Apple Inc. | Three dimensional user interface session control |
US8959013B2 (en) | 2010-09-27 | 2015-02-17 | Apple Inc. | Virtual keyboard for a non-tactile three dimensional user interface |
US20150054923A1 (en) * | 2012-06-30 | 2015-02-26 | Microsoft Corporation | Depth sensing with depth-adaptive illumination |
US8995057B2 (en) | 2010-11-02 | 2015-03-31 | Asahi Glass Company, Limited | Diffractive optical element and measurement instrument |
US9002099B2 (en) | 2011-09-11 | 2015-04-07 | Apple Inc. | Learning-based estimation of hand and finger pose |
US9019267B2 (en) | 2012-10-30 | 2015-04-28 | Apple Inc. | Depth mapping with enhanced resolution |
US9024872B2 (en) | 2011-04-28 | 2015-05-05 | Sharp Kabushiki Kaisha | Head-mounted display |
US9030498B2 (en) | 2011-08-15 | 2015-05-12 | Apple Inc. | Combining explicit select gestures and timeclick in a non-tactile three dimensional user interface |
US9030528B2 (en) | 2011-04-04 | 2015-05-12 | Apple Inc. | Multi-zone imaging sensor and lens array |
US9030529B2 (en) | 2011-04-14 | 2015-05-12 | Industrial Technology Research Institute | Depth image acquiring device, system and method |
US9035876B2 (en) | 2008-01-14 | 2015-05-19 | Apple Inc. | Three-dimensional user interface session control |
US9036158B2 (en) | 2010-08-11 | 2015-05-19 | Apple Inc. | Pattern projector |
TWI485361B (en) * | 2013-09-11 | 2015-05-21 | Univ Nat Taiwan | Measuring apparatus for three-dimensional profilometry and method thereof |
US9047507B2 (en) | 2012-05-02 | 2015-06-02 | Apple Inc. | Upper-body skeleton extraction from depth maps |
US20150168553A1 (en) * | 2013-12-16 | 2015-06-18 | Samsung Electronics Co., Ltd. | Event filtering device and motion recognition device thereof |
US9066087B2 (en) | 2010-11-19 | 2015-06-23 | Apple Inc. | Depth mapping using time-coded illumination |
US9098931B2 (en) | 2010-08-11 | 2015-08-04 | Apple Inc. | Scanning projectors and image capture modules for 3D mapping |
US9122311B2 (en) | 2011-08-24 | 2015-09-01 | Apple Inc. | Visual feedback for tactile and non-tactile user interfaces |
US9131136B2 (en) | 2010-12-06 | 2015-09-08 | Apple Inc. | Lens arrays for pattern projection and imaging |
US9158375B2 (en) | 2010-07-20 | 2015-10-13 | Apple Inc. | Interactive reality augmentation for natural interaction |
US9157790B2 (en) | 2012-02-15 | 2015-10-13 | Apple Inc. | Integrated optoelectronic modules with transmitter, receiver and beam-combining optics for aligning a beam axis with a collection axis |
US20150301591A1 (en) * | 2012-10-31 | 2015-10-22 | Audi Ag | Method for inputting a control command for a component of a motor vehicle |
US9201237B2 (en) | 2012-03-22 | 2015-12-01 | Apple Inc. | Diffraction-based sensing of mirror position |
US9201501B2 (en) | 2010-07-20 | 2015-12-01 | Apple Inc. | Adaptive projector |
US9217665B2 (en) | 2013-01-31 | 2015-12-22 | Hewlett Packard Enterprise Development Lp | Viewing-angle imaging using lenslet array |
US9229534B2 (en) | 2012-02-28 | 2016-01-05 | Apple Inc. | Asymmetric mapping for tactile and non-tactile user interfaces |
US9285874B2 (en) | 2011-02-09 | 2016-03-15 | Apple Inc. | Gaze detection in a 3D mapping environment |
US9296109B2 (en) | 2007-03-20 | 2016-03-29 | Irobot Corporation | Mobile robot for telecommunication |
US9330324B2 (en) | 2005-10-11 | 2016-05-03 | Apple Inc. | Error compensation in three-dimensional mapping |
US9377865B2 (en) | 2011-07-05 | 2016-06-28 | Apple Inc. | Zoom-based gesture user interface |
US9377863B2 (en) | 2012-03-26 | 2016-06-28 | Apple Inc. | Gaze-enhanced virtual touchscreen |
US9459758B2 (en) | 2011-07-05 | 2016-10-04 | Apple Inc. | Gesture-based interface with enhanced features |
US9477018B2 (en) | 2010-08-06 | 2016-10-25 | Asahi Glass Company, Limited | Diffractive optical element and measurement device |
US20160335773A1 (en) * | 2015-05-13 | 2016-11-17 | Oculus Vr, Llc | Augmenting a depth map representation with a reflectivity map representation |
US9528906B1 (en) | 2013-12-19 | 2016-12-27 | Apple Inc. | Monitoring DOE performance using total internal reflection |
US20170032530A1 (en) * | 2015-07-30 | 2017-02-02 | Canon Kabushiki Kaisha | Information processing apparatus, information processing method, and storage medium |
US9582889B2 (en) | 2009-07-30 | 2017-02-28 | Apple Inc. | Depth mapping based on pattern matching and stereoscopic information |
CN107209960A (en) * | 2014-12-18 | 2017-09-26 | 脸谱公司 | For system, the device and method of the user interface for providing reality environment |
US9965471B2 (en) | 2012-02-23 | 2018-05-08 | Charles D. Huston | System and method for capturing and sharing a location based experience |
EP3327546A1 (en) * | 2016-11-24 | 2018-05-30 | Industrial Technology Research Institute | Interactive display device and interactive display system |
US10013070B2 (en) * | 2016-03-29 | 2018-07-03 | Korea Electronics Technology Institute | System and method for recognizing hand gesture |
US10012831B2 (en) | 2015-08-03 | 2018-07-03 | Apple Inc. | Optical monitoring of scan parameters |
US10043279B1 (en) | 2015-12-07 | 2018-08-07 | Apple Inc. | Robust detection and classification of body parts in a depth map |
US10073004B2 (en) | 2016-09-19 | 2018-09-11 | Apple Inc. | DOE defect monitoring utilizing total internal reflection |
US10114452B2 (en) | 2015-03-11 | 2018-10-30 | Panasonic Intellectual Property Management Co., Ltd. | Motion detection system |
US10154234B2 (en) * | 2016-03-16 | 2018-12-11 | Omnivision Technologies, Inc. | Image sensor with peripheral 3A-control sensors and associated imaging system |
US10186034B2 (en) | 2015-01-20 | 2019-01-22 | Ricoh Company, Ltd. | Image processing apparatus, system, image processing method, calibration method, and computer-readable recording medium |
US10366278B2 (en) | 2016-09-20 | 2019-07-30 | Apple Inc. | Curvature-based face detector |
US10402993B2 (en) | 2016-03-30 | 2019-09-03 | Samsung Electronics Co., Ltd. | Structured light generator and object recognition apparatus including the same |
US10499039B2 (en) | 2016-12-15 | 2019-12-03 | Egismos Technology Corporation | Path detection system and path detection method generating laser pattern by diffractive optical element |
US20190392987A1 (en) * | 2018-06-20 | 2019-12-26 | Murata Manufacturing Co., Ltd. | Inductor and method for producing the same |
US10527711B2 (en) * | 2017-07-10 | 2020-01-07 | Aurora Flight Sciences Corporation | Laser speckle system and method for an aircraft |
US10600235B2 (en) | 2012-02-23 | 2020-03-24 | Charles D. Huston | System and method for capturing and sharing a location based experience |
US10649523B2 (en) * | 2017-04-24 | 2020-05-12 | Magic Leap, Inc. | System for detecting six degrees of freedom of movement by tracking optical flow of backscattered laser speckle patterns |
US10664104B2 (en) | 2016-03-30 | 2020-05-26 | Seiko Epson Corporation | Image recognition device, image recognition method, and image recognition unit |
WO2020136658A1 (en) * | 2018-12-28 | 2020-07-02 | Guardian Optical Technologies Ltd | Systems, devices and methods for vehicle post-crash support |
US10769401B2 (en) | 2016-01-14 | 2020-09-08 | Seiko Epson Corporation | Image recognition device, image recognition method and image recognition unit |
US10775508B1 (en) * | 2016-08-19 | 2020-09-15 | Apple Inc. | Remote sensing device |
US10775936B2 (en) | 2016-01-13 | 2020-09-15 | Seiko Epson Corporation | Image recognition device, image recognition method and image recognition unit |
US10937239B2 (en) | 2012-02-23 | 2021-03-02 | Charles D. Huston | System and method for creating an environment and for sharing an event |
US11016613B2 (en) | 2016-01-13 | 2021-05-25 | Seiko Epson Corporation | Image recognition device, image recognition method and image recognition unit |
US11057608B2 (en) | 2016-01-04 | 2021-07-06 | Qualcomm Incorporated | Depth map generation in structured light system |
US11118901B2 (en) | 2017-12-01 | 2021-09-14 | Omron Corporation | Image processing system and image processing method |
US11314399B2 (en) | 2017-10-21 | 2022-04-26 | Eyecam, Inc. | Adaptive graphic user interfacing system |
US11422292B1 (en) | 2018-06-10 | 2022-08-23 | Apple Inc. | Super-blazed diffractive optical elements with sub-wavelength structures |
US11423566B2 (en) * | 2018-11-20 | 2022-08-23 | Carl Zeiss Industrielle Messtechnik Gmbh | Variable measuring object dependent camera setup and calibration thereof |
US11506762B1 (en) | 2019-09-24 | 2022-11-22 | Apple Inc. | Optical module comprising an optical waveguide with reference light path |
US11536981B2 (en) | 2018-06-11 | 2022-12-27 | AGC Inc. | Diffractive optical element, projection device, and measurement device |
US11632535B2 (en) * | 2019-12-31 | 2023-04-18 | Peking University | Light field imaging system by projecting near-infrared spot in remote sensing based on multifocal microlens array |
US11681019B2 (en) | 2019-09-18 | 2023-06-20 | Apple Inc. | Optical module with stray light baffle |
US11754767B1 (en) | 2020-03-05 | 2023-09-12 | Apple Inc. | Display with overlaid waveguide |
Families Citing this family (77)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2007105205A2 (en) | 2006-03-14 | 2007-09-20 | Prime Sense Ltd. | Three-dimensional sensing using speckle patterns |
FR2921719B1 (en) | 2007-09-28 | 2010-03-12 | Noomeo | METHOD FOR CONSTRUCTING A SYNTHESIS IMAGE OF A THREE-DIMENSIONAL SURFACE OF A PHYSICAL OBJECT |
GB2463724B (en) * | 2008-09-26 | 2011-05-04 | Cybula Ltd | Forming 3D images |
FR2940423B1 (en) * | 2008-12-22 | 2011-05-27 | Noomeo | DENSE RECONSTRUCTION THREE-DIMENSIONAL SCANNING DEVICE |
JP5654583B2 (en) | 2009-06-17 | 2015-01-14 | 3シェイプ アー/エス | Focus control device |
CN102022979A (en) | 2009-09-21 | 2011-04-20 | 鸿富锦精密工业(深圳)有限公司 | Three-dimensional optical sensing system |
US8786757B2 (en) | 2010-02-23 | 2014-07-22 | Primesense Ltd. | Wideband ambient light rejection |
US8670029B2 (en) * | 2010-06-16 | 2014-03-11 | Microsoft Corporation | Depth camera illuminator with superluminescent light-emitting diode |
US9348111B2 (en) | 2010-08-24 | 2016-05-24 | Apple Inc. | Automatic detection of lens deviations |
IL208568B (en) * | 2010-10-07 | 2018-06-28 | Elbit Systems Ltd | Mapping, detecting and tracking objects in an arbitrary outdoor scene using active vision |
EP2466560A1 (en) | 2010-12-20 | 2012-06-20 | Axis AB | Method and system for monitoring the accessibility of an emergency exit |
US8717488B2 (en) | 2011-01-18 | 2014-05-06 | Primesense Ltd. | Objective optics with interference filter |
US9052512B2 (en) | 2011-03-03 | 2015-06-09 | Asahi Glass Company, Limited | Diffractive optical element and measuring apparatus |
JP5948949B2 (en) * | 2011-06-28 | 2016-07-06 | 旭硝子株式会社 | Diffractive optical element and measuring device |
JP5948948B2 (en) * | 2011-03-03 | 2016-07-06 | 旭硝子株式会社 | Diffractive optical element and measuring device |
CN102859320A (en) * | 2011-04-28 | 2013-01-02 | 三洋电机株式会社 | Information acquisition device and object detection device |
EP2530442A1 (en) | 2011-05-30 | 2012-12-05 | Axis AB | Methods and apparatus for thermographic measurements. |
US8971572B1 (en) | 2011-08-12 | 2015-03-03 | The Research Foundation For The State University Of New York | Hand pointing estimation for human computer interaction |
FR2980292B1 (en) | 2011-09-16 | 2013-10-11 | Prynel | METHOD AND SYSTEM FOR ACQUIRING AND PROCESSING IMAGES FOR MOTION DETECTION |
WO2013067526A1 (en) | 2011-11-04 | 2013-05-10 | Remote TelePointer, LLC | Method and system for user interface for interactive devices using a mobile device |
DE102011121696A1 (en) * | 2011-12-16 | 2013-06-20 | Friedrich-Schiller-Universität Jena | Method for 3D measurement of depth-limited objects |
EP2611169A1 (en) | 2011-12-27 | 2013-07-03 | Thomson Licensing | Device for the acquisition of stereoscopic images |
SI2618316T1 (en) | 2012-01-23 | 2018-12-31 | Novomatic Ag | Wheel of fortune with gesture control |
CN103424077A (en) * | 2012-05-23 | 2013-12-04 | 联想(北京)有限公司 | Motion detection device, detection method and electronic equipment |
CN102681183B (en) * | 2012-05-25 | 2015-01-07 | 合肥鼎臣光电科技有限责任公司 | Two-way three-dimensional imaging and naked-eye three-dimensional display system based on lens array |
WO2014003796A1 (en) * | 2012-06-30 | 2014-01-03 | Hewlett-Packard Development Company, L.P. | Virtual hand based on combined data |
US9152234B2 (en) | 2012-12-02 | 2015-10-06 | Apple Inc. | Detecting user intent to remove a pluggable peripheral device |
NL2010213C2 (en) | 2013-01-31 | 2014-08-04 | Lely Patent Nv | Camera system, animal related system therewith, and method to create 3d camera images. |
JP6044403B2 (en) * | 2013-03-18 | 2016-12-14 | 富士通株式会社 | Imaging apparatus, imaging method, and imaging program |
CN103268608B (en) * | 2013-05-17 | 2015-12-02 | 清华大学 | Based on depth estimation method and the device of near-infrared laser speckle |
JP2016524709A (en) * | 2013-06-06 | 2016-08-18 | ヘプタゴン・マイクロ・オプティクス・プライベート・リミテッドHeptagon Micro Optics Pte. Ltd. | Sensor system with active illumination |
CN105324631B (en) | 2013-06-19 | 2018-11-16 | 苹果公司 | integrated structured light projector |
US9208566B2 (en) | 2013-08-09 | 2015-12-08 | Microsoft Technology Licensing, Llc | Speckle sensing for motion tracking |
WO2015030127A1 (en) | 2013-09-02 | 2015-03-05 | 旭硝子株式会社 | Diffraction optical element, projection device, and measurement device |
EP2894546B1 (en) * | 2014-01-13 | 2018-07-18 | Facebook Inc. | Sub-resolution optical detection |
WO2015118120A1 (en) | 2014-02-07 | 2015-08-13 | 3Shape A/S | Detecting tooth shade |
WO2015148604A1 (en) | 2014-03-25 | 2015-10-01 | Massachusetts Institute Of Technology | Space-time modulated active 3d imager |
WO2015152829A1 (en) | 2014-04-03 | 2015-10-08 | Heptagon Micro Optics Pte. Ltd. | Structured-stereo imaging assembly including separate imagers for different wavelengths |
US10455212B1 (en) * | 2014-08-25 | 2019-10-22 | X Development Llc | Projected pattern motion/vibration for depth sensing |
USD733141S1 (en) | 2014-09-10 | 2015-06-30 | Faro Technologies, Inc. | Laser scanner |
US9841496B2 (en) | 2014-11-21 | 2017-12-12 | Microsoft Technology Licensing, Llc | Multiple pattern illumination optics for time of flight system |
US9881235B1 (en) | 2014-11-21 | 2018-01-30 | Mahmoud Narimanzadeh | System, apparatus, and method for determining physical dimensions in digital images |
TWI564754B (en) * | 2014-11-24 | 2017-01-01 | 圓剛科技股份有限公司 | Spatial motion sensing device and spatial motion sensing method |
WO2016103271A2 (en) * | 2014-12-27 | 2016-06-30 | Guardian Optical Technologies Ltd. | System and method for detecting surface vibrations |
FI126498B (en) * | 2014-12-29 | 2017-01-13 | Helmee Imaging Oy | Optical measuring system |
US9958758B2 (en) * | 2015-01-21 | 2018-05-01 | Microsoft Technology Licensing, Llc | Multiple exposure structured light pattern |
US10509147B2 (en) | 2015-01-29 | 2019-12-17 | ams Sensors Singapore Pte. Ltd | Apparatus for producing patterned illumination using arrays of light sources and lenses |
US9817159B2 (en) | 2015-01-31 | 2017-11-14 | Microsoft Technology Licensing, Llc | Structured light pattern generation |
US9530215B2 (en) * | 2015-03-20 | 2016-12-27 | Qualcomm Incorporated | Systems and methods for enhanced depth map retrieval for moving objects using active sensing technology |
US10001583B2 (en) | 2015-04-06 | 2018-06-19 | Heptagon Micro Optics Pte. Ltd. | Structured light projection using a compound patterned mask |
US9525863B2 (en) | 2015-04-29 | 2016-12-20 | Apple Inc. | Time-of-flight depth mapping with flexible scan pattern |
WO2016195684A1 (en) * | 2015-06-04 | 2016-12-08 | Siemens Healthcare Gmbh | Apparatus and methods for a projection display device on x-ray imaging devices |
US10474297B2 (en) | 2016-07-20 | 2019-11-12 | Ams Sensors Singapore Pte. Ltd. | Projecting a structured light pattern onto a surface and detecting and responding to interactions with the same |
US10241244B2 (en) | 2016-07-29 | 2019-03-26 | Lumentum Operations Llc | Thin film total internal reflection diffraction grating for single polarization or dual polarization |
US10481740B2 (en) | 2016-08-01 | 2019-11-19 | Ams Sensors Singapore Pte. Ltd. | Projecting a structured light pattern onto a surface and detecting and responding to interactions with the same |
US10158845B2 (en) | 2017-01-18 | 2018-12-18 | Facebook Technologies, Llc | Tileable structured light projection for wide field-of-view depth sensing |
US10620447B2 (en) | 2017-01-19 | 2020-04-14 | Cognex Corporation | System and method for reduced-speckle laser line generation |
JP7136094B2 (en) | 2017-05-26 | 2022-09-13 | Agc株式会社 | Diffractive optical element, projection device and measurement device |
US11494897B2 (en) | 2017-07-07 | 2022-11-08 | William F. WILEY | Application to determine reading/working distance |
KR20200028446A (en) | 2017-08-31 | 2020-03-16 | 에스지 디제이아이 테크놀러지 코., 엘티디 | Solid state light detection and ranging (LIDAR) systems, and systems and methods to improve solid state light detection and ranging (LIDAR) resolution |
JP6856784B2 (en) * | 2017-08-31 | 2021-04-14 | エスゼット ディージェイアイ テクノロジー カンパニー リミテッドSz Dji Technology Co.,Ltd | Solid-state photodetection and range-finding (LIDAR) systems, systems and methods for improving solid-state light detection and range-finding (LIDAR) resolution. |
US10310281B1 (en) | 2017-12-05 | 2019-06-04 | K Laser Technology, Inc. | Optical projector with off-axis diffractive element |
US10545457B2 (en) | 2017-12-05 | 2020-01-28 | K Laser Technology, Inc. | Optical projector with off-axis diffractive element and conjugate images |
US10317684B1 (en) | 2018-01-24 | 2019-06-11 | K Laser Technology, Inc. | Optical projector with on axis hologram and multiple beam splitter |
CN110161786B (en) | 2018-02-12 | 2021-08-31 | 深圳富泰宏精密工业有限公司 | Light projection module, three-dimensional image sensing device and sensing method thereof |
CN108663800B (en) * | 2018-04-16 | 2021-03-19 | 华东交通大学 | Optical encryption and decryption method, device and system |
US11675114B2 (en) | 2018-07-23 | 2023-06-13 | Ii-Vi Delaware, Inc. | Monolithic structured light projector |
CN109541875B (en) * | 2018-11-24 | 2024-02-13 | 深圳阜时科技有限公司 | Light source structure, optical projection module, sensing device and equipment |
WO2020171749A1 (en) * | 2019-02-18 | 2020-08-27 | Fingerprint Cards Ab | Optical biometric imaging device and method of operating an optical biometric imaging device |
US11029408B2 (en) * | 2019-04-03 | 2021-06-08 | Varjo Technologies Oy | Distance-imaging system and method of distance imaging |
US10509128B1 (en) | 2019-04-12 | 2019-12-17 | K Laser Technology, Inc. | Programmable pattern optical projector for depth detection |
GB2589121A (en) * | 2019-11-21 | 2021-05-26 | Bae Systems Plc | Imaging apparatus |
EP4090244A4 (en) * | 2020-01-17 | 2024-01-17 | Antishock Tech Ltd | System and method for monitoring fluid management to a patient |
US11888289B2 (en) * | 2020-03-30 | 2024-01-30 | Namuga, Co., Ltd. | Light source module allowing differential control according to distance to subject and method for controlling the same |
WO2022005362A1 (en) * | 2020-06-30 | 2022-01-06 | Kneedly Ab | Solution for determination of supraphysiological body joint movements |
EP3993385A1 (en) | 2020-10-29 | 2022-05-04 | Universitat de València | A multiperspective photography camera device |
CN114255233B (en) * | 2022-03-01 | 2022-05-31 | 合肥的卢深视科技有限公司 | Speckle pattern quality evaluation method and device, electronic device and storage medium |
Citations (89)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4336978A (en) * | 1978-12-26 | 1982-06-29 | Canon Kabushiki Kaisha | Method for optically making a diffusion plate |
US4542376A (en) * | 1983-11-03 | 1985-09-17 | Burroughs Corporation | System for electronically displaying portions of several different images on a CRT screen through respective prioritized viewports |
US4802759A (en) * | 1986-08-11 | 1989-02-07 | Goro Matsumoto | Three-dimensional shape measuring apparatus |
US4843568A (en) * | 1986-04-11 | 1989-06-27 | Krueger Myron W | Real time perception of and response to the actions of an unencumbered participant/user |
US5075562A (en) * | 1990-09-20 | 1991-12-24 | Eastman Kodak Company | Method and apparatus for absolute Moire distance measurements using a grating printed on or attached to a surface |
US5483261A (en) * | 1992-02-14 | 1996-01-09 | Itu Research, Inc. | Graphical input controller and method with rear screen image detection |
US5630043A (en) * | 1995-05-11 | 1997-05-13 | Cirrus Logic, Inc. | Animated texture map apparatus and method for 3-D image displays |
US5636025A (en) * | 1992-04-23 | 1997-06-03 | Medar, Inc. | System for optically measuring the surface contour of a part using more fringe techniques |
US5835218A (en) * | 1995-07-18 | 1998-11-10 | Insutrial Technology Institute | Moire interferometry system and method with extended imaging depth |
US5838428A (en) * | 1997-02-28 | 1998-11-17 | United States Of America As Represented By The Secretary Of The Navy | System and method for high resolution range imaging with split light source and pattern mask |
US5856871A (en) * | 1993-08-18 | 1999-01-05 | Applied Spectral Imaging Ltd. | Film thickness mapping using interferometric spectral imaging |
US5909312A (en) * | 1996-10-02 | 1999-06-01 | Ramot University Authority For Applied Research & Industrial Development Ltd. | Phase-only filter for generating an arbitrary illumination pattern |
US6041140A (en) * | 1994-10-04 | 2000-03-21 | Synthonics, Incorporated | Apparatus for interactive image correlation for three dimensional image production |
US6081269A (en) * | 1992-03-12 | 2000-06-27 | International Business Machines Corporation | Image processing system and method for generating data representing a number of points in a three-dimensional space from a plurality of two-dimensional images of the space |
US6084712A (en) * | 1998-11-03 | 2000-07-04 | Dynamic Measurement And Inspection,Llc | Three dimensional imaging using a refractive optic design |
US6088105A (en) * | 1998-04-04 | 2000-07-11 | Joh. & Ernst Link Gmbh & Co. Kg | Measuring unit for determining dimensions of test pieces, preferably of hollow bodies, in particular, of bores of workpieces, and method for measuring such dimensions |
US6101269A (en) * | 1997-12-19 | 2000-08-08 | Lifef/X Networks, Inc. | Apparatus and method for rapid 3D image parametrization |
US6099134A (en) * | 1996-09-27 | 2000-08-08 | Hitachi, Ltd. | Liquid crystal display device |
US6100517A (en) * | 1995-06-22 | 2000-08-08 | 3Dv Systems Ltd. | Three dimensional camera |
US6108036A (en) * | 1996-03-25 | 2000-08-22 | Sharp Kabushiki Kaisha | Imaging apparatus having a spatial filter and image shifting mechanism controller based on an image mode |
US6167151A (en) * | 1996-12-15 | 2000-12-26 | Cognitens, Ltd. | Apparatus and method for 3-dimensional surface geometry reconstruction |
US6259561B1 (en) * | 1999-03-26 | 2001-07-10 | The University Of Rochester | Optical system for diffusing light |
US6262740B1 (en) * | 1997-08-01 | 2001-07-17 | Terarecon, Inc. | Method for rendering sections of a volume data set |
US6268923B1 (en) * | 1999-10-07 | 2001-07-31 | Integral Vision, Inc. | Optical method and system for measuring three-dimensional surface topography of an object having a surface contour |
US6301059B1 (en) * | 2000-01-07 | 2001-10-09 | Lucent Technologies Inc. | Astigmatic compensation for an anamorphic optical system |
US20020041327A1 (en) * | 2000-07-24 | 2002-04-11 | Evan Hildreth | Video-based image control system |
US20020075456A1 (en) * | 2000-12-20 | 2002-06-20 | Olympus Optical Co., Ltd. | 3D image acquisition apparatus and 3D image acquisition method |
US6494837B2 (en) * | 2000-06-10 | 2002-12-17 | Medison Co., Ltd. | System and method for three-dimensional ultrasound imaging using a steerable probe |
US6495848B1 (en) * | 1998-05-14 | 2002-12-17 | Orametrix, Inc. | Evaluation of projection pattern for transitions in pattern to determine spatial structure of 3D surfaces |
US20030048237A1 (en) * | 2000-02-07 | 2003-03-13 | Seiji Sato | Display system with no eyeglasses |
US20030057972A1 (en) * | 1999-07-26 | 2003-03-27 | Paul Pfaff | Voltage testing and measurement |
US20030156756A1 (en) * | 2002-02-15 | 2003-08-21 | Gokturk Salih Burak | Gesture recognition system using depth perceptive sensors |
US20040001145A1 (en) * | 2002-06-27 | 2004-01-01 | Abbate Jeffrey A. | Method and apparatus for multifield image generation and processing |
US6686921B1 (en) * | 2000-08-01 | 2004-02-03 | International Business Machines Corporation | Method and apparatus for acquiring a set of consistent image maps to represent the color of the surface of an object |
US6731391B1 (en) * | 1998-05-13 | 2004-05-04 | The Research Foundation Of State University Of New York | Shadow moire surface measurement using Talbot effect |
US6741251B2 (en) * | 2001-08-16 | 2004-05-25 | Hewlett-Packard Development Company, L.P. | Method and apparatus for varying focus in a scene |
US20040105580A1 (en) * | 2002-11-22 | 2004-06-03 | Hager Gregory D. | Acquisition of three-dimensional images by an active stereo technique using locally unique patterns |
US6751344B1 (en) * | 1999-05-28 | 2004-06-15 | Champion Orthotic Investments, Inc. | Enhanced projector system for machine vision |
US6754370B1 (en) * | 2000-08-14 | 2004-06-22 | The Board Of Trustees Of The Leland Stanford Junior University | Real-time structured light range scanning of moving scenes |
US6759646B1 (en) * | 1998-11-24 | 2004-07-06 | Intel Corporation | Color interpolation for a four color mosaic pattern |
US20040130730A1 (en) * | 2002-11-21 | 2004-07-08 | Michel Cantin | Fast 3D height measurement method and system |
US20040130790A1 (en) * | 2002-09-20 | 2004-07-08 | Sales Tasso R. M. | Random microlens array for optical beam shaping and homogenization |
US20040174770A1 (en) * | 2002-11-27 | 2004-09-09 | Rees Frank L. | Gauss-Rees parametric ultrawideband system |
US6810135B1 (en) * | 2000-06-29 | 2004-10-26 | Trw Inc. | Optimized human presence detection through elimination of background interference |
US20040213463A1 (en) * | 2003-04-22 | 2004-10-28 | Morrison Rick Lee | Multiplexed, spatially encoded illumination system for determining imaging and range estimation |
US6813440B1 (en) * | 2000-10-10 | 2004-11-02 | The Hong Kong Polytechnic University | Body scanner |
US20040218262A1 (en) * | 2003-02-21 | 2004-11-04 | Chuang Yung-Ho | Inspection system using small catadioptric objective |
US20040228519A1 (en) * | 2003-03-10 | 2004-11-18 | Cranial Technologies, Inc. | Automatic selection of cranial remodeling device trim lines |
US6825985B2 (en) * | 2001-07-13 | 2004-11-30 | Mems Optical, Inc. | Autostereoscopic display with rotated microlens and method of displaying multidimensional images, especially color images |
US6841780B2 (en) * | 2001-01-19 | 2005-01-11 | Honeywell International Inc. | Method and apparatus for detecting objects |
US20050052637A1 (en) * | 2003-09-10 | 2005-03-10 | Shaw Eugene L. | Tire inspection apparatus and method |
US20050111705A1 (en) * | 2003-08-26 | 2005-05-26 | Roman Waupotitsch | Passive stereo sensing for 3D facial shape biometrics |
US6937348B2 (en) * | 2000-01-28 | 2005-08-30 | Genex Technologies, Inc. | Method and apparatus for generating structural pattern illumination |
US20050200925A1 (en) * | 1999-12-10 | 2005-09-15 | Xyz Imaging, Inc. | Holographic printer |
US20050200838A1 (en) * | 2003-09-10 | 2005-09-15 | Shaw Eugene L. | Plurality of light sources for inspection apparatus and method |
US20050231465A1 (en) * | 2004-04-15 | 2005-10-20 | Depue Marshall T | Optical device that measures distance between the device and a surface |
US20050271279A1 (en) * | 2004-05-14 | 2005-12-08 | Honda Motor Co., Ltd. | Sign based human-machine interaction |
US20060017656A1 (en) * | 2004-07-26 | 2006-01-26 | Visteon Global Technologies, Inc. | Image intensity control in overland night vision systems |
US7006952B1 (en) * | 1999-02-19 | 2006-02-28 | Sanyo Electric Co., Ltd. | 3-D model providing device |
US20060072851A1 (en) * | 2002-06-15 | 2006-04-06 | Microsoft Corporation | Deghosting mosaics using multiperspective plane sweep |
US7076024B2 (en) * | 2004-12-01 | 2006-07-11 | Jordan Valley Applied Radiation, Ltd. | X-ray apparatus with dual monochromators |
US20060156756A1 (en) * | 2005-01-20 | 2006-07-20 | Becke Paul E | Phase change and insulating properties container and method of use |
US7112774B2 (en) * | 2003-10-09 | 2006-09-26 | Avago Technologies Sensor Ip (Singapore) Pte. Ltd | CMOS stereo imaging system and method |
US20060221218A1 (en) * | 2005-04-05 | 2006-10-05 | Doron Adler | Image sensor with improved color filter |
US7120228B2 (en) * | 2004-09-21 | 2006-10-10 | Jordan Valley Applied Radiation Ltd. | Combined X-ray reflectometer and diffractometer |
US20060269896A1 (en) * | 2005-05-27 | 2006-11-30 | Yongqian Liu | High speed 3D scanner and uses thereof |
US20070060336A1 (en) * | 2003-09-15 | 2007-03-15 | Sony Computer Entertainment Inc. | Methods and systems for enabling depth and direction detection when interfacing with a computer program |
US20070057946A1 (en) * | 2003-07-24 | 2007-03-15 | Dan Albeck | Method and system for the three-dimensional surface reconstruction of an object |
US7194105B2 (en) * | 2002-10-16 | 2007-03-20 | Hersch Roger D | Authentication of documents and articles by moiré patterns |
US7231069B2 (en) * | 2000-03-31 | 2007-06-12 | Oki Electric Industry Co., Ltd. | Multiple view angles camera, automatic photographing apparatus, and iris recognition method |
US20070165243A1 (en) * | 2004-02-09 | 2007-07-19 | Cheol-Gwon Kang | Device for measuring 3d shape using irregular pattern and method for the same |
US7256899B1 (en) * | 2006-10-04 | 2007-08-14 | Ivan Faul | Wireless methods and systems for three-dimensional non-contact shape sensing |
US20080031513A1 (en) * | 2000-07-14 | 2008-02-07 | Massachusetts Institute Of Technology | Method and system for high resolution, ultra fast 3-D imaging |
US7335898B2 (en) * | 2004-07-23 | 2008-02-26 | Ge Healthcare Niagara Inc. | Method and apparatus for fluorescent confocal microscopy |
US7369685B2 (en) * | 2002-04-05 | 2008-05-06 | Identix Corporation | Vision-based operating method and system |
US20080106746A1 (en) * | 2005-10-11 | 2008-05-08 | Alexander Shpunt | Depth-varying light fields for three dimensional sensing |
US7385708B2 (en) * | 2002-06-07 | 2008-06-10 | The University Of North Carolina At Chapel Hill | Methods and systems for laser based real-time structured light depth extraction |
US20080198355A1 (en) * | 2006-12-27 | 2008-08-21 | Cambridge Research & Instrumentation, Inc | Surface measurement of in-vivo subjects using spot projector |
US20080212835A1 (en) * | 2007-03-01 | 2008-09-04 | Amon Tavor | Object Tracking by 3-Dimensional Modeling |
US20080240502A1 (en) * | 2007-04-02 | 2008-10-02 | Barak Freedman | Depth mapping using projected patterns |
US7433024B2 (en) * | 2006-02-27 | 2008-10-07 | Prime Sense Ltd. | Range mapping using speckle decorrelation |
US20080247670A1 (en) * | 2007-04-03 | 2008-10-09 | Wa James Tam | Generation of a depth map from a monoscopic color image for rendering stereoscopic still and video images |
US20080278572A1 (en) * | 2007-04-23 | 2008-11-13 | Morteza Gharib | Aperture system with spatially-biased aperture shapes and positions (SBPSP) for static and dynamic 3-D defocusing-based imaging |
US20090183152A1 (en) * | 2008-01-16 | 2009-07-16 | Dell Products, Lp | Method to Dynamically Provision Additional Computer Resources to Handle Peak Database Workloads |
US20090183125A1 (en) * | 2008-01-14 | 2009-07-16 | Prime Sense Ltd. | Three-dimensional user interface |
US20090185274A1 (en) * | 2008-01-21 | 2009-07-23 | Prime Sense Ltd. | Optical designs for zero order reduction |
US20100007717A1 (en) * | 2008-07-09 | 2010-01-14 | Prime Sense Ltd | Integrated processor for 3d mapping |
US20100013860A1 (en) * | 2006-03-08 | 2010-01-21 | Electronic Scripting Products, Inc. | Computer interface employing a manipulated object with absolute pose detection component and a display |
US20100303289A1 (en) * | 2009-05-29 | 2010-12-02 | Microsoft Corporation | Device for identifying and tracking multiple humans over time |
Family Cites Families (78)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS6079108U (en) * | 1983-11-08 | 1985-06-01 | オムロン株式会社 | speckle rangefinder |
JPH0762869B2 (en) | 1986-03-07 | 1995-07-05 | 日本電信電話株式会社 | Position and shape measurement method by pattern projection |
JP2714152B2 (en) * | 1989-06-28 | 1998-02-16 | 古野電気株式会社 | Object shape measurement method |
GB9116151D0 (en) | 1991-07-26 | 1991-09-11 | Isis Innovation | Three-dimensional vision system |
JP3353365B2 (en) * | 1993-03-18 | 2002-12-03 | 静岡大学長 | Displacement and displacement velocity measuring device |
CN1174611A (en) * | 1994-09-05 | 1998-02-25 | 米高技术有限公司 | Diffraction surfaces and method for manufacture thereof |
JPH08186845A (en) | 1994-12-27 | 1996-07-16 | Nobuaki Yanagisawa | Focal distance controlling stereoscopic-vision television receiver |
DE19638727A1 (en) | 1996-09-12 | 1998-03-19 | Ruedger Dipl Ing Rubbert | Method for increasing the significance of the three-dimensional measurement of objects |
WO1998028593A1 (en) | 1996-12-20 | 1998-07-02 | Pacific Title And Mirage, Inc. | Apparatus and method for rapid 3d image parametrization |
JPH10327433A (en) | 1997-05-23 | 1998-12-08 | Minolta Co Ltd | Display device for composted image |
DE19736169A1 (en) | 1997-08-20 | 1999-04-15 | Fhu Hochschule Fuer Technik | Method to measure deformation or vibration using electronic speckle pattern interferometry |
US6438272B1 (en) | 1997-12-31 | 2002-08-20 | The Research Foundation Of State University Of Ny | Method and apparatus for three dimensional surface contouring using a digital video projection system |
GB2352901A (en) | 1999-05-12 | 2001-02-07 | Tricorder Technology Plc | Rendering three dimensional representations utilising projected light patterns |
US6377700B1 (en) | 1998-06-30 | 2002-04-23 | Intel Corporation | Method and apparatus for capturing stereoscopic images using image sensors |
JP3678022B2 (en) | 1998-10-23 | 2005-08-03 | コニカミノルタセンシング株式会社 | 3D input device |
US8965898B2 (en) | 1998-11-20 | 2015-02-24 | Intheplay, Inc. | Optimizations for live event, real-time, 3D object tracking |
CN2364507Y (en) * | 1999-03-18 | 2000-02-16 | 香港生产力促进局 | Small non-contact symmetric imput type three-D profile scanning head |
AU3994799A (en) * | 1999-05-14 | 2000-12-05 | 3Dmetrics, Incorporated | Color structured light 3d-imaging system |
JP2001141430A (en) | 1999-11-16 | 2001-05-25 | Fuji Photo Film Co Ltd | Image pickup device and image processing device |
US6700669B1 (en) | 2000-01-28 | 2004-03-02 | Zheng J. Geng | Method and system for three-dimensional imaging using light pattern having multiple sub-patterns |
US6639684B1 (en) | 2000-09-13 | 2003-10-28 | Nextengine, Inc. | Digitizer using intensity gradient to image features of three-dimensional objects |
JP3689720B2 (en) | 2000-10-16 | 2005-08-31 | 住友大阪セメント株式会社 | 3D shape measuring device |
JP2002152776A (en) | 2000-11-09 | 2002-05-24 | Nippon Telegr & Teleph Corp <Ntt> | Method and device for encoding and decoding distance image |
JP2002213931A (en) | 2001-01-17 | 2002-07-31 | Fuji Xerox Co Ltd | Instrument and method for measuring three-dimensional shape |
JP2002365023A (en) | 2001-06-08 | 2002-12-18 | Koji Okamoto | Apparatus and method for measurement of liquid level |
US7811825B2 (en) | 2002-04-19 | 2010-10-12 | University Of Washington | System and method for processing specimens and images for optical tomography |
KR100624405B1 (en) | 2002-10-01 | 2006-09-18 | 삼성전자주식회사 | Substrate for mounting optical component and method for producing the same |
US7539340B2 (en) | 2003-04-25 | 2009-05-26 | Topcon Corporation | Apparatus and method for three-dimensional coordinate measurement |
CA2435935A1 (en) | 2003-07-24 | 2005-01-24 | Guylain Lemelin | Optical 3d digitizer with enlarged non-ambiguity zone |
US7250949B2 (en) | 2003-12-23 | 2007-07-31 | General Electric Company | Method and system for visualizing three-dimensional data |
US20050135555A1 (en) | 2003-12-23 | 2005-06-23 | Claus Bernhard Erich H. | Method and system for simultaneously viewing rendered volumes |
US8134637B2 (en) | 2004-01-28 | 2012-03-13 | Microsoft Corporation | Method and system to increase X-Y resolution in a depth (Z) camera using red, blue, green (RGB) sensing |
CA2575704C (en) | 2004-07-30 | 2014-03-04 | Extreme Reality Ltd. | A system and method for 3d space-dimension based image processing |
JP2006128818A (en) | 2004-10-26 | 2006-05-18 | Victor Co Of Japan Ltd | Recording program and reproducing program corresponding to stereoscopic video and 3d audio, recording apparatus, reproducing apparatus and recording medium |
IL165212A (en) | 2004-11-15 | 2012-05-31 | Elbit Systems Electro Optics Elop Ltd | Device for scanning light |
EP1875162B1 (en) | 2005-04-06 | 2014-06-11 | Dimensional Photonics International, Inc. | Determining positional error of an optical component using structured light patterns |
US7560679B1 (en) | 2005-05-10 | 2009-07-14 | Siimpel, Inc. | 3D camera |
US20110096182A1 (en) | 2009-10-25 | 2011-04-28 | Prime Sense Ltd | Error Compensation in Three-Dimensional Mapping |
WO2007105205A2 (en) | 2006-03-14 | 2007-09-20 | Prime Sense Ltd. | Three-dimensional sensing using speckle patterns |
JP5001286B2 (en) | 2005-10-11 | 2012-08-15 | プライム センス リミティド | Object reconstruction method and system |
US8018579B1 (en) | 2005-10-21 | 2011-09-13 | Apple Inc. | Three-dimensional imaging and display system |
US20070133840A1 (en) | 2005-11-04 | 2007-06-14 | Clean Earth Technologies, Llc | Tracking Using An Elastic Cluster of Trackers |
US7856125B2 (en) | 2006-01-31 | 2010-12-21 | University Of Southern California | 3D face reconstruction from 2D images |
CN101501442B (en) | 2006-03-14 | 2014-03-19 | 普莱姆传感有限公司 | Depth-varying light fields for three dimensional sensing |
US7869649B2 (en) | 2006-05-08 | 2011-01-11 | Panasonic Corporation | Image processing device, image processing method, program, storage medium and integrated circuit |
US8488895B2 (en) | 2006-05-31 | 2013-07-16 | Indiana University Research And Technology Corp. | Laser scanning digital camera with pupil periphery illumination and potential for multiply scattered light imaging |
US8139142B2 (en) | 2006-06-01 | 2012-03-20 | Microsoft Corporation | Video manipulation of red, green, blue, distance (RGB-Z) data including segmentation, up-sampling, and background substitution techniques |
EP2050067A1 (en) | 2006-08-03 | 2009-04-22 | Alterface S.A. | Method and device for identifying and extracting images of multiple users, and for recognizing user gestures |
US7737394B2 (en) | 2006-08-31 | 2010-06-15 | Micron Technology, Inc. | Ambient infrared detection in solid state sensors |
EP2064675B1 (en) | 2006-09-04 | 2012-11-21 | Koninklijke Philips Electronics N.V. | Method for determining a depth map from images, device for determining a depth map |
WO2008061259A2 (en) | 2006-11-17 | 2008-05-22 | Celloptic, Inc. | System, apparatus and method for extracting three-dimensional information of an object from received electromagnetic radiation |
US8090194B2 (en) | 2006-11-21 | 2012-01-03 | Mantis Vision Ltd. | 3D geometric modeling and motion capture using both single and dual imaging |
US7840031B2 (en) | 2007-01-12 | 2010-11-23 | International Business Machines Corporation | Tracking a range of body movement based on 3D captured image streams of a user |
WO2008087652A2 (en) | 2007-01-21 | 2008-07-24 | Prime Sense Ltd. | Depth mapping using multi-beam illumination |
WO2008120217A2 (en) | 2007-04-02 | 2008-10-09 | Prime Sense Ltd. | Depth mapping using projected patterns |
US7835561B2 (en) | 2007-05-18 | 2010-11-16 | Visiongate, Inc. | Method for image processing and reconstruction of images for optical tomography |
US8494252B2 (en) | 2007-06-19 | 2013-07-23 | Primesense Ltd. | Depth mapping using optical elements having non-uniform focal characteristics |
US20100182406A1 (en) | 2007-07-12 | 2010-07-22 | Benitez Ana B | System and method for three-dimensional object reconstruction from two-dimensional images |
JP4412362B2 (en) | 2007-07-18 | 2010-02-10 | 船井電機株式会社 | Compound eye imaging device |
US20090060307A1 (en) | 2007-08-27 | 2009-03-05 | Siemens Medical Solutions Usa, Inc. | Tensor Voting System and Method |
DE102007045332B4 (en) | 2007-09-17 | 2019-01-17 | Seereal Technologies S.A. | Holographic display for reconstructing a scene |
KR100858034B1 (en) | 2007-10-18 | 2008-09-10 | (주)실리콘화일 | One chip image sensor for measuring vitality of subject |
US8384997B2 (en) | 2008-01-21 | 2013-02-26 | Primesense Ltd | Optical pattern projection |
DE102008011350A1 (en) | 2008-02-27 | 2009-09-03 | Loeffler Technology Gmbh | Apparatus and method for real-time detection of electromagnetic THz radiation |
US8121351B2 (en) | 2008-03-09 | 2012-02-21 | Microsoft International Holdings B.V. | Identification of objects in a 3D video using non/over reflective clothing |
US8035806B2 (en) | 2008-05-13 | 2011-10-11 | Samsung Electronics Co., Ltd. | Distance measuring sensor including double transfer gate and three dimensional color image sensor including the distance measuring sensor |
US8462207B2 (en) | 2009-02-12 | 2013-06-11 | Primesense Ltd. | Depth ranging with Moiré patterns |
US8786682B2 (en) | 2009-03-05 | 2014-07-22 | Primesense Ltd. | Reference image techniques for three-dimensional sensing |
US8717417B2 (en) | 2009-04-16 | 2014-05-06 | Primesense Ltd. | Three-dimensional mapping and imaging |
US8503720B2 (en) | 2009-05-01 | 2013-08-06 | Microsoft Corporation | Human body pose estimation |
EP2275990B1 (en) | 2009-07-06 | 2012-09-26 | Sick Ag | 3D sensor |
WO2011013079A1 (en) | 2009-07-30 | 2011-02-03 | Primesense Ltd. | Depth mapping based on pattern matching and stereoscopic information |
US8773514B2 (en) | 2009-08-27 | 2014-07-08 | California Institute Of Technology | Accurate 3D object reconstruction using a handheld device with a projected light pattern |
US8830227B2 (en) | 2009-12-06 | 2014-09-09 | Primesense Ltd. | Depth-based gain control |
US8320621B2 (en) | 2009-12-21 | 2012-11-27 | Microsoft Corporation | Depth projector system with integrated VCSEL array |
US8982182B2 (en) | 2010-03-01 | 2015-03-17 | Apple Inc. | Non-uniform spatial resource allocation for depth mapping |
US8330804B2 (en) | 2010-05-12 | 2012-12-11 | Microsoft Corporation | Scanned-beam depth mapping to 2D image |
US8654152B2 (en) | 2010-06-21 | 2014-02-18 | Microsoft Corporation | Compartmentalizing focus area within field of view |
-
2007
- 2007-03-08 WO PCT/IL2007/000306 patent/WO2007105205A2/en active Application Filing
- 2007-03-08 CN CN2007800166255A patent/CN101496033B/en active Active
- 2007-03-08 JP JP2008558981A patent/JP5174684B2/en active Active
- 2007-03-08 US US12/282,517 patent/US8390821B2/en active Active
- 2007-03-08 KR KR1020087025030A patent/KR101331543B1/en active IP Right Grant
-
2013
- 2013-01-24 US US13/748,617 patent/US9063283B2/en active Active
Patent Citations (99)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4336978A (en) * | 1978-12-26 | 1982-06-29 | Canon Kabushiki Kaisha | Method for optically making a diffusion plate |
US4542376A (en) * | 1983-11-03 | 1985-09-17 | Burroughs Corporation | System for electronically displaying portions of several different images on a CRT screen through respective prioritized viewports |
US4843568A (en) * | 1986-04-11 | 1989-06-27 | Krueger Myron W | Real time perception of and response to the actions of an unencumbered participant/user |
US4802759A (en) * | 1986-08-11 | 1989-02-07 | Goro Matsumoto | Three-dimensional shape measuring apparatus |
US5075562A (en) * | 1990-09-20 | 1991-12-24 | Eastman Kodak Company | Method and apparatus for absolute Moire distance measurements using a grating printed on or attached to a surface |
US5483261A (en) * | 1992-02-14 | 1996-01-09 | Itu Research, Inc. | Graphical input controller and method with rear screen image detection |
US6081269A (en) * | 1992-03-12 | 2000-06-27 | International Business Machines Corporation | Image processing system and method for generating data representing a number of points in a three-dimensional space from a plurality of two-dimensional images of the space |
US5636025A (en) * | 1992-04-23 | 1997-06-03 | Medar, Inc. | System for optically measuring the surface contour of a part using more fringe techniques |
US5856871A (en) * | 1993-08-18 | 1999-01-05 | Applied Spectral Imaging Ltd. | Film thickness mapping using interferometric spectral imaging |
US6041140A (en) * | 1994-10-04 | 2000-03-21 | Synthonics, Incorporated | Apparatus for interactive image correlation for three dimensional image production |
US5630043A (en) * | 1995-05-11 | 1997-05-13 | Cirrus Logic, Inc. | Animated texture map apparatus and method for 3-D image displays |
US6100517A (en) * | 1995-06-22 | 2000-08-08 | 3Dv Systems Ltd. | Three dimensional camera |
US5835218A (en) * | 1995-07-18 | 1998-11-10 | Insutrial Technology Institute | Moire interferometry system and method with extended imaging depth |
US6108036A (en) * | 1996-03-25 | 2000-08-22 | Sharp Kabushiki Kaisha | Imaging apparatus having a spatial filter and image shifting mechanism controller based on an image mode |
US6099134A (en) * | 1996-09-27 | 2000-08-08 | Hitachi, Ltd. | Liquid crystal display device |
US5909312A (en) * | 1996-10-02 | 1999-06-01 | Ramot University Authority For Applied Research & Industrial Development Ltd. | Phase-only filter for generating an arbitrary illumination pattern |
US6438263B2 (en) * | 1996-12-15 | 2002-08-20 | Dan Albeck | Apparatus and method for 3-dimensional surface geometry reconstruction |
US20010016063A1 (en) * | 1996-12-15 | 2001-08-23 | Cognitens, Ltd. | Apparatus and method for 3-dimensional surface geometry reconstruction |
US6167151A (en) * | 1996-12-15 | 2000-12-26 | Cognitens, Ltd. | Apparatus and method for 3-dimensional surface geometry reconstruction |
US5838428A (en) * | 1997-02-28 | 1998-11-17 | United States Of America As Represented By The Secretary Of The Navy | System and method for high resolution range imaging with split light source and pattern mask |
US6262740B1 (en) * | 1997-08-01 | 2001-07-17 | Terarecon, Inc. | Method for rendering sections of a volume data set |
US6101269A (en) * | 1997-12-19 | 2000-08-08 | Lifef/X Networks, Inc. | Apparatus and method for rapid 3D image parametrization |
US6088105A (en) * | 1998-04-04 | 2000-07-11 | Joh. & Ernst Link Gmbh & Co. Kg | Measuring unit for determining dimensions of test pieces, preferably of hollow bodies, in particular, of bores of workpieces, and method for measuring such dimensions |
US6731391B1 (en) * | 1998-05-13 | 2004-05-04 | The Research Foundation Of State University Of New York | Shadow moire surface measurement using Talbot effect |
US6495848B1 (en) * | 1998-05-14 | 2002-12-17 | Orametrix, Inc. | Evaluation of projection pattern for transitions in pattern to determine spatial structure of 3D surfaces |
US6084712A (en) * | 1998-11-03 | 2000-07-04 | Dynamic Measurement And Inspection,Llc | Three dimensional imaging using a refractive optic design |
US6759646B1 (en) * | 1998-11-24 | 2004-07-06 | Intel Corporation | Color interpolation for a four color mosaic pattern |
US7006952B1 (en) * | 1999-02-19 | 2006-02-28 | Sanyo Electric Co., Ltd. | 3-D model providing device |
US6259561B1 (en) * | 1999-03-26 | 2001-07-10 | The University Of Rochester | Optical system for diffusing light |
US6751344B1 (en) * | 1999-05-28 | 2004-06-15 | Champion Orthotic Investments, Inc. | Enhanced projector system for machine vision |
US20030057972A1 (en) * | 1999-07-26 | 2003-03-27 | Paul Pfaff | Voltage testing and measurement |
US6803777B2 (en) * | 1999-07-26 | 2004-10-12 | Paul Pfaff | Voltage testing and measurement |
US6268923B1 (en) * | 1999-10-07 | 2001-07-31 | Integral Vision, Inc. | Optical method and system for measuring three-dimensional surface topography of an object having a surface contour |
US7009742B2 (en) * | 1999-12-10 | 2006-03-07 | Xyz Imaging, Inc. | Holographic printer |
US20050200925A1 (en) * | 1999-12-10 | 2005-09-15 | Xyz Imaging, Inc. | Holographic printer |
US6301059B1 (en) * | 2000-01-07 | 2001-10-09 | Lucent Technologies Inc. | Astigmatic compensation for an anamorphic optical system |
US6937348B2 (en) * | 2000-01-28 | 2005-08-30 | Genex Technologies, Inc. | Method and apparatus for generating structural pattern illumination |
US20030048237A1 (en) * | 2000-02-07 | 2003-03-13 | Seiji Sato | Display system with no eyeglasses |
US7231069B2 (en) * | 2000-03-31 | 2007-06-12 | Oki Electric Industry Co., Ltd. | Multiple view angles camera, automatic photographing apparatus, and iris recognition method |
US6494837B2 (en) * | 2000-06-10 | 2002-12-17 | Medison Co., Ltd. | System and method for three-dimensional ultrasound imaging using a steerable probe |
US6810135B1 (en) * | 2000-06-29 | 2004-10-26 | Trw Inc. | Optimized human presence detection through elimination of background interference |
US20080031513A1 (en) * | 2000-07-14 | 2008-02-07 | Massachusetts Institute Of Technology | Method and system for high resolution, ultra fast 3-D imaging |
US20090016642A1 (en) * | 2000-07-14 | 2009-01-15 | Massachusetts Institute Of Technology | Method and system for high resolution, ultra fast 3-d imaging |
US20020041327A1 (en) * | 2000-07-24 | 2002-04-11 | Evan Hildreth | Video-based image control system |
US20080018595A1 (en) * | 2000-07-24 | 2008-01-24 | Gesturetek, Inc. | Video-based image control system |
US6686921B1 (en) * | 2000-08-01 | 2004-02-03 | International Business Machines Corporation | Method and apparatus for acquiring a set of consistent image maps to represent the color of the surface of an object |
US6754370B1 (en) * | 2000-08-14 | 2004-06-22 | The Board Of Trustees Of The Leland Stanford Junior University | Real-time structured light range scanning of moving scenes |
US6813440B1 (en) * | 2000-10-10 | 2004-11-02 | The Hong Kong Polytechnic University | Body scanner |
US20020075456A1 (en) * | 2000-12-20 | 2002-06-20 | Olympus Optical Co., Ltd. | 3D image acquisition apparatus and 3D image acquisition method |
US7013040B2 (en) * | 2000-12-20 | 2006-03-14 | Olympus Optical Co., Ltd. | 3D image acquisition apparatus and 3D image acquisition method |
US6841780B2 (en) * | 2001-01-19 | 2005-01-11 | Honeywell International Inc. | Method and apparatus for detecting objects |
US6825985B2 (en) * | 2001-07-13 | 2004-11-30 | Mems Optical, Inc. | Autostereoscopic display with rotated microlens and method of displaying multidimensional images, especially color images |
US6741251B2 (en) * | 2001-08-16 | 2004-05-25 | Hewlett-Packard Development Company, L.P. | Method and apparatus for varying focus in a scene |
US20030156756A1 (en) * | 2002-02-15 | 2003-08-21 | Gokturk Salih Burak | Gesture recognition system using depth perceptive sensors |
US7369685B2 (en) * | 2002-04-05 | 2008-05-06 | Identix Corporation | Vision-based operating method and system |
US7385708B2 (en) * | 2002-06-07 | 2008-06-10 | The University Of North Carolina At Chapel Hill | Methods and systems for laser based real-time structured light depth extraction |
US20060072851A1 (en) * | 2002-06-15 | 2006-04-06 | Microsoft Corporation | Deghosting mosaics using multiperspective plane sweep |
US20040001145A1 (en) * | 2002-06-27 | 2004-01-01 | Abbate Jeffrey A. | Method and apparatus for multifield image generation and processing |
US6859326B2 (en) * | 2002-09-20 | 2005-02-22 | Corning Incorporated | Random microlens array for optical beam shaping and homogenization |
US20040130790A1 (en) * | 2002-09-20 | 2004-07-08 | Sales Tasso R. M. | Random microlens array for optical beam shaping and homogenization |
US7194105B2 (en) * | 2002-10-16 | 2007-03-20 | Hersch Roger D | Authentication of documents and articles by moiré patterns |
US20040130730A1 (en) * | 2002-11-21 | 2004-07-08 | Michel Cantin | Fast 3D height measurement method and system |
US20040105580A1 (en) * | 2002-11-22 | 2004-06-03 | Hager Gregory D. | Acquisition of three-dimensional images by an active stereo technique using locally unique patterns |
US20040174770A1 (en) * | 2002-11-27 | 2004-09-09 | Rees Frank L. | Gauss-Rees parametric ultrawideband system |
US20040218262A1 (en) * | 2003-02-21 | 2004-11-04 | Chuang Yung-Ho | Inspection system using small catadioptric objective |
US20040228519A1 (en) * | 2003-03-10 | 2004-11-18 | Cranial Technologies, Inc. | Automatic selection of cranial remodeling device trim lines |
US7127101B2 (en) * | 2003-03-10 | 2006-10-24 | Cranul Technologies, Inc. | Automatic selection of cranial remodeling device trim lines |
US20040213463A1 (en) * | 2003-04-22 | 2004-10-28 | Morrison Rick Lee | Multiplexed, spatially encoded illumination system for determining imaging and range estimation |
US20070057946A1 (en) * | 2003-07-24 | 2007-03-15 | Dan Albeck | Method and system for the three-dimensional surface reconstruction of an object |
US20050111705A1 (en) * | 2003-08-26 | 2005-05-26 | Roman Waupotitsch | Passive stereo sensing for 3D facial shape biometrics |
US20050052637A1 (en) * | 2003-09-10 | 2005-03-10 | Shaw Eugene L. | Tire inspection apparatus and method |
US20050200838A1 (en) * | 2003-09-10 | 2005-09-15 | Shaw Eugene L. | Plurality of light sources for inspection apparatus and method |
US20070060336A1 (en) * | 2003-09-15 | 2007-03-15 | Sony Computer Entertainment Inc. | Methods and systems for enabling depth and direction detection when interfacing with a computer program |
US7112774B2 (en) * | 2003-10-09 | 2006-09-26 | Avago Technologies Sensor Ip (Singapore) Pte. Ltd | CMOS stereo imaging system and method |
US20070165243A1 (en) * | 2004-02-09 | 2007-07-19 | Cheol-Gwon Kang | Device for measuring 3d shape using irregular pattern and method for the same |
US20050231465A1 (en) * | 2004-04-15 | 2005-10-20 | Depue Marshall T | Optical device that measures distance between the device and a surface |
US20050271279A1 (en) * | 2004-05-14 | 2005-12-08 | Honda Motor Co., Ltd. | Sign based human-machine interaction |
US7335898B2 (en) * | 2004-07-23 | 2008-02-26 | Ge Healthcare Niagara Inc. | Method and apparatus for fluorescent confocal microscopy |
US20060017656A1 (en) * | 2004-07-26 | 2006-01-26 | Visteon Global Technologies, Inc. | Image intensity control in overland night vision systems |
US7120228B2 (en) * | 2004-09-21 | 2006-10-10 | Jordan Valley Applied Radiation Ltd. | Combined X-ray reflectometer and diffractometer |
US7551719B2 (en) * | 2004-09-21 | 2009-06-23 | Jordan Valley Semiconductord Ltd | Multifunction X-ray analysis system |
US7076024B2 (en) * | 2004-12-01 | 2006-07-11 | Jordan Valley Applied Radiation, Ltd. | X-ray apparatus with dual monochromators |
US20060156756A1 (en) * | 2005-01-20 | 2006-07-20 | Becke Paul E | Phase change and insulating properties container and method of use |
US20060221218A1 (en) * | 2005-04-05 | 2006-10-05 | Doron Adler | Image sensor with improved color filter |
US20060269896A1 (en) * | 2005-05-27 | 2006-11-30 | Yongqian Liu | High speed 3D scanner and uses thereof |
US20080106746A1 (en) * | 2005-10-11 | 2008-05-08 | Alexander Shpunt | Depth-varying light fields for three dimensional sensing |
US7433024B2 (en) * | 2006-02-27 | 2008-10-07 | Prime Sense Ltd. | Range mapping using speckle decorrelation |
US20100013860A1 (en) * | 2006-03-08 | 2010-01-21 | Electronic Scripting Products, Inc. | Computer interface employing a manipulated object with absolute pose detection component and a display |
US7256899B1 (en) * | 2006-10-04 | 2007-08-14 | Ivan Faul | Wireless methods and systems for three-dimensional non-contact shape sensing |
US20080198355A1 (en) * | 2006-12-27 | 2008-08-21 | Cambridge Research & Instrumentation, Inc | Surface measurement of in-vivo subjects using spot projector |
US20080212835A1 (en) * | 2007-03-01 | 2008-09-04 | Amon Tavor | Object Tracking by 3-Dimensional Modeling |
US20080240502A1 (en) * | 2007-04-02 | 2008-10-02 | Barak Freedman | Depth mapping using projected patterns |
US20080247670A1 (en) * | 2007-04-03 | 2008-10-09 | Wa James Tam | Generation of a depth map from a monoscopic color image for rendering stereoscopic still and video images |
US20080278572A1 (en) * | 2007-04-23 | 2008-11-13 | Morteza Gharib | Aperture system with spatially-biased aperture shapes and positions (SBPSP) for static and dynamic 3-D defocusing-based imaging |
US20090183125A1 (en) * | 2008-01-14 | 2009-07-16 | Prime Sense Ltd. | Three-dimensional user interface |
US20090183152A1 (en) * | 2008-01-16 | 2009-07-16 | Dell Products, Lp | Method to Dynamically Provision Additional Computer Resources to Handle Peak Database Workloads |
US20090185274A1 (en) * | 2008-01-21 | 2009-07-23 | Prime Sense Ltd. | Optical designs for zero order reduction |
US20100007717A1 (en) * | 2008-07-09 | 2010-01-14 | Prime Sense Ltd | Integrated processor for 3d mapping |
US20100303289A1 (en) * | 2009-05-29 | 2010-12-02 | Microsoft Corporation | Device for identifying and tracking multiple humans over time |
Cited By (214)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080106746A1 (en) * | 2005-10-11 | 2008-05-08 | Alexander Shpunt | Depth-varying light fields for three dimensional sensing |
US9330324B2 (en) | 2005-10-11 | 2016-05-03 | Apple Inc. | Error compensation in three-dimensional mapping |
US8050461B2 (en) | 2005-10-11 | 2011-11-01 | Primesense Ltd. | Depth-varying light fields for three dimensional sensing |
US8400494B2 (en) | 2005-10-11 | 2013-03-19 | Primesense Ltd. | Method and system for object reconstruction |
US9066084B2 (en) | 2005-10-11 | 2015-06-23 | Apple Inc. | Method and system for object reconstruction |
US20100177164A1 (en) * | 2005-10-11 | 2010-07-15 | Zeev Zalevsky | Method and System for Object Reconstruction |
US20110158508A1 (en) * | 2005-10-11 | 2011-06-30 | Primesense Ltd. | Depth-varying light fields for three dimensional sensing |
US8374397B2 (en) | 2005-10-11 | 2013-02-12 | Primesense Ltd | Depth-varying light fields for three dimensional sensing |
US20100034457A1 (en) * | 2006-05-11 | 2010-02-11 | Tamir Berliner | Modeling of humanoid forms from depth maps |
US8249334B2 (en) | 2006-05-11 | 2012-08-21 | Primesense Ltd. | Modeling of humanoid forms from depth maps |
US20100020078A1 (en) * | 2007-01-21 | 2010-01-28 | Prime Sense Ltd | Depth mapping using multi-beam illumination |
US8350847B2 (en) | 2007-01-21 | 2013-01-08 | Primesense Ltd | Depth mapping using multi-beam illumination |
US9296109B2 (en) | 2007-03-20 | 2016-03-29 | Irobot Corporation | Mobile robot for telecommunication |
US8150142B2 (en) | 2007-04-02 | 2012-04-03 | Prime Sense Ltd. | Depth mapping using projected patterns |
US8493496B2 (en) | 2007-04-02 | 2013-07-23 | Primesense Ltd. | Depth mapping using projected patterns |
US20080240502A1 (en) * | 2007-04-02 | 2008-10-02 | Barak Freedman | Depth mapping using projected patterns |
US20100290698A1 (en) * | 2007-06-19 | 2010-11-18 | Prime Sense Ltd | Distance-Varying Illumination and Imaging Techniques for Depth Mapping |
US8494252B2 (en) | 2007-06-19 | 2013-07-23 | Primesense Ltd. | Depth mapping using optical elements having non-uniform focal characteristics |
US20100309301A1 (en) * | 2007-12-04 | 2010-12-09 | Sirona Dental Systems Gmbh | Recording method for obtaining an image of an object and recording device |
US8957954B2 (en) | 2007-12-04 | 2015-02-17 | Sirona Dental Systems Gmbh | Recording method for obtaining an image of an object and recording device |
US9035876B2 (en) | 2008-01-14 | 2015-05-19 | Apple Inc. | Three-dimensional user interface session control |
US20090183125A1 (en) * | 2008-01-14 | 2009-07-16 | Prime Sense Ltd. | Three-dimensional user interface |
US8166421B2 (en) | 2008-01-14 | 2012-04-24 | Primesense Ltd. | Three-dimensional user interface |
US20110069389A1 (en) * | 2008-01-21 | 2011-03-24 | Primesense Ltd. | Optical designs for zero order reduction |
US8384997B2 (en) | 2008-01-21 | 2013-02-26 | Primesense Ltd | Optical pattern projection |
US20110075259A1 (en) * | 2008-01-21 | 2011-03-31 | Primesense Ltd. | Optical designs for zero order reduction |
US9239467B2 (en) | 2008-01-21 | 2016-01-19 | Apple Inc. | Optical pattern projection |
US8630039B2 (en) | 2008-01-21 | 2014-01-14 | Primesense Ltd. | Optical designs for zero order reduction |
US20090185274A1 (en) * | 2008-01-21 | 2009-07-23 | Prime Sense Ltd. | Optical designs for zero order reduction |
US20100284082A1 (en) * | 2008-01-21 | 2010-11-11 | Primesense Ltd. | Optical pattern projection |
US8456517B2 (en) | 2008-07-09 | 2013-06-04 | Primesense Ltd. | Integrated processor for 3D mapping |
US8462207B2 (en) | 2009-02-12 | 2013-06-11 | Primesense Ltd. | Depth ranging with Moiré patterns |
US8786682B2 (en) | 2009-03-05 | 2014-07-22 | Primesense Ltd. | Reference image techniques for three-dimensional sensing |
US8717417B2 (en) | 2009-04-16 | 2014-05-06 | Primesense Ltd. | Three-dimensional mapping and imaging |
US9582889B2 (en) | 2009-07-30 | 2017-02-28 | Apple Inc. | Depth mapping based on pattern matching and stereoscopic information |
US8565479B2 (en) | 2009-08-13 | 2013-10-22 | Primesense Ltd. | Extraction of skeletons from 3D maps |
US20110052006A1 (en) * | 2009-08-13 | 2011-03-03 | Primesense Ltd. | Extraction of skeletons from 3d maps |
US9659377B2 (en) | 2009-10-07 | 2017-05-23 | Microsoft Technology Licensing, Llc | Methods and systems for determining and tracking extremities of a target |
US8867820B2 (en) * | 2009-10-07 | 2014-10-21 | Microsoft Corporation | Systems and methods for removing a background of an image |
US20170278251A1 (en) * | 2009-10-07 | 2017-09-28 | Microsoft Technology Licensing, Llc | Systems and methods for removing a background of an image |
US8861839B2 (en) | 2009-10-07 | 2014-10-14 | Microsoft Corporation | Human tracking system |
US9821226B2 (en) | 2009-10-07 | 2017-11-21 | Microsoft Technology Licensing, Llc | Human tracking system |
US20110080475A1 (en) * | 2009-10-07 | 2011-04-07 | Microsoft Corporation | Methods And Systems For Determining And Tracking Extremities Of A Target |
US10147194B2 (en) * | 2009-10-07 | 2018-12-04 | Microsoft Technology Licensing, Llc | Systems and methods for removing a background of an image |
US8963829B2 (en) | 2009-10-07 | 2015-02-24 | Microsoft Corporation | Methods and systems for determining and tracking extremities of a target |
US8970487B2 (en) | 2009-10-07 | 2015-03-03 | Microsoft Technology Licensing, Llc | Human tracking system |
US8891827B2 (en) | 2009-10-07 | 2014-11-18 | Microsoft Corporation | Systems and methods for tracking a model |
US20110080336A1 (en) * | 2009-10-07 | 2011-04-07 | Microsoft Corporation | Human Tracking System |
US9679390B2 (en) | 2009-10-07 | 2017-06-13 | Microsoft Technology Licensing, Llc | Systems and methods for removing a background of an image |
US9522328B2 (en) | 2009-10-07 | 2016-12-20 | Microsoft Technology Licensing, Llc | Human tracking system |
US8542910B2 (en) | 2009-10-07 | 2013-09-24 | Microsoft Corporation | Human tracking system |
US8564534B2 (en) | 2009-10-07 | 2013-10-22 | Microsoft Corporation | Human tracking system |
US20110081044A1 (en) * | 2009-10-07 | 2011-04-07 | Microsoft Corporation | Systems And Methods For Removing A Background Of An Image |
US9582717B2 (en) | 2009-10-07 | 2017-02-28 | Microsoft Technology Licensing, Llc | Systems and methods for tracking a model |
US8897495B2 (en) | 2009-10-07 | 2014-11-25 | Microsoft Corporation | Systems and methods for tracking a model |
US20110096182A1 (en) * | 2009-10-25 | 2011-04-28 | Prime Sense Ltd | Error Compensation in Three-Dimensional Mapping |
US20110114857A1 (en) * | 2009-11-15 | 2011-05-19 | Primesense Ltd. | Optical projector with beam monitor |
US8492696B2 (en) | 2009-11-15 | 2013-07-23 | Primesense Ltd. | Optical projector with beam monitor including mapping apparatus capturing image of pattern projected onto an object |
US20110134114A1 (en) * | 2009-12-06 | 2011-06-09 | Primesense Ltd. | Depth-based gain control |
US8830227B2 (en) | 2009-12-06 | 2014-09-09 | Primesense Ltd. | Depth-based gain control |
US20110154201A1 (en) * | 2009-12-22 | 2011-06-23 | Akira Nakanishi | Video Reproducing Apparatus and Video Reproducing Method |
US8413053B2 (en) * | 2009-12-22 | 2013-04-02 | Kabushiki Kaisha Toshiba | Video reproducing apparatus and video reproducing method |
US20150292709A1 (en) * | 2010-02-02 | 2015-10-15 | Apple Inc. | Integrated photonics module for optical projection |
US20110188054A1 (en) * | 2010-02-02 | 2011-08-04 | Primesense Ltd | Integrated photonics module for optical projection |
US20110187878A1 (en) * | 2010-02-02 | 2011-08-04 | Primesense Ltd. | Synchronization of projected illumination with rolling shutter of image sensor |
US9736459B2 (en) | 2010-02-02 | 2017-08-15 | Apple Inc. | Generation of patterned radiation |
US20110211754A1 (en) * | 2010-03-01 | 2011-09-01 | Primesense Ltd. | Tracking body parts by combined color image and depth processing |
US20150145961A1 (en) * | 2010-03-01 | 2015-05-28 | Apple Inc. | Non-uniform spatial resource allocation for depth mapping |
US20110211044A1 (en) * | 2010-03-01 | 2011-09-01 | Primesense Ltd. | Non-Uniform Spatial Resource Allocation for Depth Mapping |
US8982182B2 (en) * | 2010-03-01 | 2015-03-17 | Apple Inc. | Non-uniform spatial resource allocation for depth mapping |
US8787663B2 (en) | 2010-03-01 | 2014-07-22 | Primesense Ltd. | Tracking body parts by combined color image and depth processing |
US10291905B2 (en) * | 2010-03-01 | 2019-05-14 | Apple Inc. | Non-uniform spatial resource allocation for depth mapping |
US20130033459A1 (en) * | 2010-04-13 | 2013-02-07 | Nokia Corporation | Apparatus, method, computer program and user interface |
US9535493B2 (en) * | 2010-04-13 | 2017-01-03 | Nokia Technologies Oy | Apparatus, method, computer program and user interface |
WO2011146259A2 (en) | 2010-05-20 | 2011-11-24 | Irobot Corporation | Mobile human interface robot |
US8935005B2 (en) | 2010-05-20 | 2015-01-13 | Irobot Corporation | Operating a mobile robot |
US9498886B2 (en) | 2010-05-20 | 2016-11-22 | Irobot Corporation | Mobile human interface robot |
US8918213B2 (en) | 2010-05-20 | 2014-12-23 | Irobot Corporation | Mobile human interface robot |
US8918209B2 (en) | 2010-05-20 | 2014-12-23 | Irobot Corporation | Mobile human interface robot |
US9014848B2 (en) | 2010-05-20 | 2015-04-21 | Irobot Corporation | Mobile robot system |
US9902069B2 (en) | 2010-05-20 | 2018-02-27 | Irobot Corporation | Mobile robot system |
US9400503B2 (en) | 2010-05-20 | 2016-07-26 | Irobot Corporation | Mobile human interface robot |
US8594425B2 (en) | 2010-05-31 | 2013-11-26 | Primesense Ltd. | Analysis of three-dimensional scenes |
US8781217B2 (en) | 2010-05-31 | 2014-07-15 | Primesense Ltd. | Analysis of three-dimensional scenes with a surface model |
US8824737B2 (en) | 2010-05-31 | 2014-09-02 | Primesense Ltd. | Identifying components of a humanoid form in three-dimensional scenes |
US9158375B2 (en) | 2010-07-20 | 2015-10-13 | Apple Inc. | Interactive reality augmentation for natural interaction |
US9201501B2 (en) | 2010-07-20 | 2015-12-01 | Apple Inc. | Adaptive projector |
US9477018B2 (en) | 2010-08-06 | 2016-10-25 | Asahi Glass Company, Limited | Diffractive optical element and measurement device |
US8599484B2 (en) | 2010-08-10 | 2013-12-03 | Asahi Glass Company, Limited | Diffractive optical element and measuring device |
US9098931B2 (en) | 2010-08-11 | 2015-08-04 | Apple Inc. | Scanning projectors and image capture modules for 3D mapping |
US9036158B2 (en) | 2010-08-11 | 2015-05-19 | Apple Inc. | Pattern projector |
US8582867B2 (en) | 2010-09-16 | 2013-11-12 | Primesense Ltd | Learning-based pose estimation from depth maps |
US8959013B2 (en) | 2010-09-27 | 2015-02-17 | Apple Inc. | Virtual keyboard for a non-tactile three dimensional user interface |
US8995057B2 (en) | 2010-11-02 | 2015-03-31 | Asahi Glass Company, Limited | Diffractive optical element and measurement instrument |
US20120105326A1 (en) * | 2010-11-03 | 2012-05-03 | Samsung Electronics Co., Ltd. | Method and apparatus for generating motion information |
US9066087B2 (en) | 2010-11-19 | 2015-06-23 | Apple Inc. | Depth mapping using time-coded illumination |
US9167138B2 (en) | 2010-12-06 | 2015-10-20 | Apple Inc. | Pattern projection and imaging using lens arrays |
US20160004145A1 (en) * | 2010-12-06 | 2016-01-07 | Apple Inc. | Pattern projection and imaging using lens arrays |
US10345684B2 (en) * | 2010-12-06 | 2019-07-09 | Apple Inc. | Pattern projection and imaging using lens arrays |
US9131136B2 (en) | 2010-12-06 | 2015-09-08 | Apple Inc. | Lens arrays for pattern projection and imaging |
US8872762B2 (en) | 2010-12-08 | 2014-10-28 | Primesense Ltd. | Three dimensional user interface cursor control |
US8933876B2 (en) | 2010-12-13 | 2015-01-13 | Apple Inc. | Three dimensional user interface session control |
WO2012091807A2 (en) | 2010-12-30 | 2012-07-05 | Irobot Corporation | Mobile human interface robot |
EP2769809A1 (en) | 2010-12-30 | 2014-08-27 | iRobot Corporation | Mobile robot system |
US8930019B2 (en) | 2010-12-30 | 2015-01-06 | Irobot Corporation | Mobile human interface robot |
WO2012091814A2 (en) | 2010-12-30 | 2012-07-05 | Irobot Corporation | Mobile robot system |
US9285874B2 (en) | 2011-02-09 | 2016-03-15 | Apple Inc. | Gaze detection in a 3D mapping environment |
US9342146B2 (en) | 2011-02-09 | 2016-05-17 | Apple Inc. | Pointing-based display interaction |
US9454225B2 (en) | 2011-02-09 | 2016-09-27 | Apple Inc. | Gaze-based display control |
US9030528B2 (en) | 2011-04-04 | 2015-05-12 | Apple Inc. | Multi-zone imaging sensor and lens array |
US9030529B2 (en) | 2011-04-14 | 2015-05-12 | Industrial Technology Research Institute | Depth image acquiring device, system and method |
US9024872B2 (en) | 2011-04-28 | 2015-05-05 | Sharp Kabushiki Kaisha | Head-mounted display |
US20120313896A1 (en) * | 2011-06-07 | 2012-12-13 | Sony Corporation | Information processing apparatus, information processing method, and program |
US9766796B2 (en) * | 2011-06-07 | 2017-09-19 | Sony Corporation | Information processing apparatus, information processing method, and program |
US20120313875A1 (en) * | 2011-06-13 | 2012-12-13 | Sharp Kabushiki Kaisha | Manual operating device |
US9459758B2 (en) | 2011-07-05 | 2016-10-04 | Apple Inc. | Gesture-based interface with enhanced features |
US9377865B2 (en) | 2011-07-05 | 2016-06-28 | Apple Inc. | Zoom-based gesture user interface |
US8881051B2 (en) | 2011-07-05 | 2014-11-04 | Primesense Ltd | Zoom-based gesture user interface |
US8869073B2 (en) * | 2011-07-28 | 2014-10-21 | Hewlett-Packard Development Company, L.P. | Hand pose interaction |
US20130031517A1 (en) * | 2011-07-28 | 2013-01-31 | Dustin Freeman | Hand pose interaction |
US8749796B2 (en) | 2011-08-09 | 2014-06-10 | Primesense Ltd. | Projectors of structured light |
US8908277B2 (en) | 2011-08-09 | 2014-12-09 | Apple Inc | Lens array projector |
US9030498B2 (en) | 2011-08-15 | 2015-05-12 | Apple Inc. | Combining explicit select gestures and timeclick in a non-tactile three dimensional user interface |
US20130055120A1 (en) * | 2011-08-24 | 2013-02-28 | Primesense Ltd. | Sessionless pointing user interface |
US9218063B2 (en) * | 2011-08-24 | 2015-12-22 | Apple Inc. | Sessionless pointing user interface |
US9122311B2 (en) | 2011-08-24 | 2015-09-01 | Apple Inc. | Visual feedback for tactile and non-tactile user interfaces |
US9002099B2 (en) | 2011-09-11 | 2015-04-07 | Apple Inc. | Learning-based estimation of hand and finger pose |
US9157790B2 (en) | 2012-02-15 | 2015-10-13 | Apple Inc. | Integrated optoelectronic modules with transmitter, receiver and beam-combining optics for aligning a beam axis with a collection axis |
US9651417B2 (en) | 2012-02-15 | 2017-05-16 | Apple Inc. | Scanning depth engine |
US10936537B2 (en) | 2012-02-23 | 2021-03-02 | Charles D. Huston | Depth sensing camera glasses with gesture interface |
US10600235B2 (en) | 2012-02-23 | 2020-03-24 | Charles D. Huston | System and method for capturing and sharing a location based experience |
US11449460B2 (en) | 2012-02-23 | 2022-09-20 | Charles D. Huston | System and method for capturing and sharing a location based experience |
US9977782B2 (en) | 2012-02-23 | 2018-05-22 | Charles D. Huston | System, method, and device including a depth camera for creating a location based experience |
US9965471B2 (en) | 2012-02-23 | 2018-05-08 | Charles D. Huston | System and method for capturing and sharing a location based experience |
US11783535B2 (en) | 2012-02-23 | 2023-10-10 | Charles D. Huston | System and method for capturing and sharing a location based experience |
US10937239B2 (en) | 2012-02-23 | 2021-03-02 | Charles D. Huston | System and method for creating an environment and for sharing an event |
US9229534B2 (en) | 2012-02-28 | 2016-01-05 | Apple Inc. | Asymmetric mapping for tactile and non-tactile user interfaces |
WO2013130734A1 (en) | 2012-02-29 | 2013-09-06 | Irobot Corporation | Mobile robot |
US9285770B2 (en) * | 2012-02-29 | 2016-03-15 | Lg Electronics Inc. | Holographic display device and method for generating hologram using redundancy of 3D video |
US8958911B2 (en) | 2012-02-29 | 2015-02-17 | Irobot Corporation | Mobile robot |
US20130222874A1 (en) * | 2012-02-29 | 2013-08-29 | Lg Electronics Inc. | Holographic display device and method for generating hologram using redundancy of 3d video |
US9201237B2 (en) | 2012-03-22 | 2015-12-01 | Apple Inc. | Diffraction-based sensing of mirror position |
US11169611B2 (en) | 2012-03-26 | 2021-11-09 | Apple Inc. | Enhanced virtual touchpad |
US9377863B2 (en) | 2012-03-26 | 2016-06-28 | Apple Inc. | Gaze-enhanced virtual touchscreen |
US9047507B2 (en) | 2012-05-02 | 2015-06-02 | Apple Inc. | Upper-body skeleton extraction from depth maps |
US9523571B2 (en) * | 2012-06-30 | 2016-12-20 | Microsoft Technology Licensing, Llc | Depth sensing with depth-adaptive illumination |
US20150054923A1 (en) * | 2012-06-30 | 2015-02-26 | Microsoft Corporation | Depth sensing with depth-adaptive illumination |
US9904998B2 (en) | 2012-08-27 | 2018-02-27 | Koninklijke Philips N.V. | Patient-specific and automatic x-ray system adjustment based on optical 3D scene detection and interpretation |
WO2014033614A1 (en) | 2012-08-27 | 2014-03-06 | Koninklijke Philips N.V. | Patient-specific and automatic x-ray system adjustment based on optical 3d scene detection and interpretation |
US9019267B2 (en) | 2012-10-30 | 2015-04-28 | Apple Inc. | Depth mapping with enhanced resolution |
US20140118496A1 (en) * | 2012-10-31 | 2014-05-01 | Ricoh Company, Ltd. | Pre-Calculation of Sine Waves for Pixel Values |
US20150301591A1 (en) * | 2012-10-31 | 2015-10-22 | Audi Ag | Method for inputting a control command for a component of a motor vehicle |
US9661304B2 (en) * | 2012-10-31 | 2017-05-23 | Ricoh Company, Ltd. | Pre-calculation of sine waves for pixel values |
US9612655B2 (en) * | 2012-10-31 | 2017-04-04 | Audi Ag | Method for inputting a control command for a component of a motor vehicle |
WO2014071254A1 (en) | 2012-11-01 | 2014-05-08 | Eyecam, LLC | Wireless wrist computing and control device and method for 3d imaging, mapping, networking and interfacing |
US11262841B2 (en) | 2012-11-01 | 2022-03-01 | Eyecam Llc | Wireless wrist computing and control device and method for 3D imaging, mapping, networking and interfacing |
US9217665B2 (en) | 2013-01-31 | 2015-12-22 | Hewlett Packard Enterprise Development Lp | Viewing-angle imaging using lenslet array |
US9959465B2 (en) * | 2013-04-15 | 2018-05-01 | Microsoft Technology Licensing, Llc | Diffractive optical element with undiffracted light expansion for eye safe operation |
US10929658B2 (en) | 2013-04-15 | 2021-02-23 | Microsoft Technology Licensing, Llc | Active stereo with adaptive support weights from a separate image |
US10928189B2 (en) | 2013-04-15 | 2021-02-23 | Microsoft Technology Licensing, Llc | Intensity-modulated light pattern for active stereo |
US10816331B2 (en) | 2013-04-15 | 2020-10-27 | Microsoft Technology Licensing, Llc | Super-resolving depth map by moving pattern projector |
US20140307307A1 (en) * | 2013-04-15 | 2014-10-16 | Microsoft Corporation | Diffractive optical element with undiffracted light expansion for eye safe operation |
US10268885B2 (en) | 2013-04-15 | 2019-04-23 | Microsoft Technology Licensing, Llc | Extracting true color from a color and infrared sensor |
WO2014202720A1 (en) | 2013-06-19 | 2014-12-24 | Koninklijke Philips N.V. | Calibration of imagers with dynamic beam shapers |
TWI485361B (en) * | 2013-09-11 | 2015-05-21 | Univ Nat Taiwan | Measuring apparatus for three-dimensional profilometry and method thereof |
US9927523B2 (en) * | 2013-12-16 | 2018-03-27 | Samsung Electronics Co., Ltd. | Event filtering device and motion recognition device thereof |
US20150168553A1 (en) * | 2013-12-16 | 2015-06-18 | Samsung Electronics Co., Ltd. | Event filtering device and motion recognition device thereof |
US9528906B1 (en) | 2013-12-19 | 2016-12-27 | Apple Inc. | Monitoring DOE performance using total internal reflection |
CN107209960A (en) * | 2014-12-18 | 2017-09-26 | 脸谱公司 | For system, the device and method of the user interface for providing reality environment |
EP3234685A4 (en) * | 2014-12-18 | 2018-06-13 | Facebook, Inc. | System, device and method for providing user interface for a virtual reality environment |
CN107209960B (en) * | 2014-12-18 | 2021-01-01 | 脸谱科技有限责任公司 | System, apparatus and method for providing a user interface for a virtual reality environment |
US10559113B2 (en) | 2014-12-18 | 2020-02-11 | Facebook Technologies, Llc | System, device and method for providing user interface for a virtual reality environment |
US10186034B2 (en) | 2015-01-20 | 2019-01-22 | Ricoh Company, Ltd. | Image processing apparatus, system, image processing method, calibration method, and computer-readable recording medium |
US10621694B2 (en) | 2015-01-20 | 2020-04-14 | Ricoh Company, Ltd. | Image processing apparatus, system, image processing method, calibration method, and computer-readable recording medium |
US10114452B2 (en) | 2015-03-11 | 2018-10-30 | Panasonic Intellectual Property Management Co., Ltd. | Motion detection system |
US9947098B2 (en) * | 2015-05-13 | 2018-04-17 | Facebook, Inc. | Augmenting a depth map representation with a reflectivity map representation |
US20160335773A1 (en) * | 2015-05-13 | 2016-11-17 | Oculus Vr, Llc | Augmenting a depth map representation with a reflectivity map representation |
US10006762B2 (en) * | 2015-07-30 | 2018-06-26 | Canon Kabushiki Kaisha | Information processing apparatus, information processing method, and storage medium |
US20170032530A1 (en) * | 2015-07-30 | 2017-02-02 | Canon Kabushiki Kaisha | Information processing apparatus, information processing method, and storage medium |
US10012831B2 (en) | 2015-08-03 | 2018-07-03 | Apple Inc. | Optical monitoring of scan parameters |
US10043279B1 (en) | 2015-12-07 | 2018-08-07 | Apple Inc. | Robust detection and classification of body parts in a depth map |
US11057608B2 (en) | 2016-01-04 | 2021-07-06 | Qualcomm Incorporated | Depth map generation in structured light system |
US10775936B2 (en) | 2016-01-13 | 2020-09-15 | Seiko Epson Corporation | Image recognition device, image recognition method and image recognition unit |
US11016613B2 (en) | 2016-01-13 | 2021-05-25 | Seiko Epson Corporation | Image recognition device, image recognition method and image recognition unit |
US10769401B2 (en) | 2016-01-14 | 2020-09-08 | Seiko Epson Corporation | Image recognition device, image recognition method and image recognition unit |
US10154234B2 (en) * | 2016-03-16 | 2018-12-11 | Omnivision Technologies, Inc. | Image sensor with peripheral 3A-control sensors and associated imaging system |
US10013070B2 (en) * | 2016-03-29 | 2018-07-03 | Korea Electronics Technology Institute | System and method for recognizing hand gesture |
US10664104B2 (en) | 2016-03-30 | 2020-05-26 | Seiko Epson Corporation | Image recognition device, image recognition method, and image recognition unit |
US10489924B2 (en) | 2016-03-30 | 2019-11-26 | Samsung Electronics Co., Ltd. | Structured light generator and object recognition apparatus including the same |
US11035548B2 (en) | 2016-03-30 | 2021-06-15 | Samsung Electronics Co., Ltd. | Structured light generator and object recognition apparatus including the same |
US10402993B2 (en) | 2016-03-30 | 2019-09-03 | Samsung Electronics Co., Ltd. | Structured light generator and object recognition apparatus including the same |
US10775508B1 (en) * | 2016-08-19 | 2020-09-15 | Apple Inc. | Remote sensing device |
US10073004B2 (en) | 2016-09-19 | 2018-09-11 | Apple Inc. | DOE defect monitoring utilizing total internal reflection |
US10366278B2 (en) | 2016-09-20 | 2019-07-30 | Apple Inc. | Curvature-based face detector |
EP3327546A1 (en) * | 2016-11-24 | 2018-05-30 | Industrial Technology Research Institute | Interactive display device and interactive display system |
US10499039B2 (en) | 2016-12-15 | 2019-12-03 | Egismos Technology Corporation | Path detection system and path detection method generating laser pattern by diffractive optical element |
US11762455B2 (en) * | 2017-04-24 | 2023-09-19 | Magic Leap, Inc. | System for detecting six degrees of freedom of movement by tracking optical flow of backscattered laser speckle patterns |
US11150725B2 (en) * | 2017-04-24 | 2021-10-19 | Magic Leap, Inc. | System for detecting six degrees of freedom of movement by tracking optical flow of backscattered laser speckle patterns |
US10649523B2 (en) * | 2017-04-24 | 2020-05-12 | Magic Leap, Inc. | System for detecting six degrees of freedom of movement by tracking optical flow of backscattered laser speckle patterns |
US20220057858A1 (en) * | 2017-04-24 | 2022-02-24 | Magic Leap, Inc. | System for detecting six degrees of freedom of movement by tracking optical flow of backscattered laser speckle patterns |
US10527711B2 (en) * | 2017-07-10 | 2020-01-07 | Aurora Flight Sciences Corporation | Laser speckle system and method for an aircraft |
US11327149B2 (en) | 2017-07-10 | 2022-05-10 | Aurora Flight Sciences Corporation | Laser speckle system and method for an aircraft |
US11314399B2 (en) | 2017-10-21 | 2022-04-26 | Eyecam, Inc. | Adaptive graphic user interfacing system |
US11118901B2 (en) | 2017-12-01 | 2021-09-14 | Omron Corporation | Image processing system and image processing method |
US11422292B1 (en) | 2018-06-10 | 2022-08-23 | Apple Inc. | Super-blazed diffractive optical elements with sub-wavelength structures |
US11536981B2 (en) | 2018-06-11 | 2022-12-27 | AGC Inc. | Diffractive optical element, projection device, and measurement device |
US20190392987A1 (en) * | 2018-06-20 | 2019-12-26 | Murata Manufacturing Co., Ltd. | Inductor and method for producing the same |
US11600438B2 (en) * | 2018-06-20 | 2023-03-07 | Murata Manufacturing Co., Ltd. | Inductor and method for producing the same |
US11423566B2 (en) * | 2018-11-20 | 2022-08-23 | Carl Zeiss Industrielle Messtechnik Gmbh | Variable measuring object dependent camera setup and calibration thereof |
WO2020136658A1 (en) * | 2018-12-28 | 2020-07-02 | Guardian Optical Technologies Ltd | Systems, devices and methods for vehicle post-crash support |
US11681019B2 (en) | 2019-09-18 | 2023-06-20 | Apple Inc. | Optical module with stray light baffle |
US11506762B1 (en) | 2019-09-24 | 2022-11-22 | Apple Inc. | Optical module comprising an optical waveguide with reference light path |
US11632535B2 (en) * | 2019-12-31 | 2023-04-18 | Peking University | Light field imaging system by projecting near-infrared spot in remote sensing based on multifocal microlens array |
US11754767B1 (en) | 2020-03-05 | 2023-09-12 | Apple Inc. | Display with overlaid waveguide |
Also Published As
Publication number | Publication date |
---|---|
US9063283B2 (en) | 2015-06-23 |
JP5174684B2 (en) | 2013-04-03 |
US8390821B2 (en) | 2013-03-05 |
WO2007105205A3 (en) | 2009-04-23 |
US20130136305A1 (en) | 2013-05-30 |
KR101331543B1 (en) | 2013-11-20 |
KR20080111474A (en) | 2008-12-23 |
CN101496033A (en) | 2009-07-29 |
JP2009531655A (en) | 2009-09-03 |
CN101496033B (en) | 2012-03-21 |
WO2007105205A2 (en) | 2007-09-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8390821B2 (en) | Three-dimensional sensing using speckle patterns | |
US7433024B2 (en) | Range mapping using speckle decorrelation | |
JP2009531655A5 (en) | ||
US8050461B2 (en) | Depth-varying light fields for three dimensional sensing | |
US8374397B2 (en) | Depth-varying light fields for three dimensional sensing | |
US7675020B2 (en) | Input apparatus and methods having diffuse and specular tracking modes | |
TWI585436B (en) | Method and apparatus for measuring depth information | |
CN103649674B (en) | Measuring equipment and messaging device | |
US8761495B2 (en) | Distance-varying illumination and imaging techniques for depth mapping | |
RU2502136C2 (en) | Combined object capturing system and display device and associated method | |
JP4392377B2 (en) | Optical device that measures the distance between the device and the surface | |
US20160286202A1 (en) | Three Dimensional Depth Mapping Using Dynamic Structured Light | |
EP3403131A1 (en) | Depth mapping using structured light and time of flight | |
KR102102291B1 (en) | Optical tracking system and optical tracking method | |
CN110462688B (en) | Three-dimensional contour determination system and method using model-based peak selection | |
US9354719B2 (en) | Optical navigation devices | |
CN117528209A (en) | Image pickup module, electronic device, focusing method, focusing device and readable storage medium | |
Hashimoto et al. | Mirror Based Framework for Human Body Measurement | |
Um et al. | Short range 3D depth sensing via multiple intensity differentiation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: PRIME SENSE LTD, ISRAEL Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHPUNT, ALEXANDER;ZALEVSKY, ZEEV;REEL/FRAME:021762/0354 Effective date: 20081026 |
|
AS | Assignment |
Owner name: PRIMESENSE LTD., ISRAEL Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:KREOS CAPITAL III LIMITED;REEL/FRAME:025646/0666 Effective date: 20100620 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: PAT HOLDER NO LONGER CLAIMS SMALL ENTITY STATUS, ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: STOL); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: APPLE INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PRIMESENSE LTD.;REEL/FRAME:034293/0092 Effective date: 20140828 |
|
AS | Assignment |
Owner name: APPLE INC., CALIFORNIA Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICATION # 13840451 AND REPLACE IT WITH CORRECT APPLICATION # 13810451 PREVIOUSLY RECORDED ON REEL 034293 FRAME 0092. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNOR:PRIMESENSE LTD.;REEL/FRAME:035624/0091 Effective date: 20140828 |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |