Methods and systems for laser based real-time structured light depth extraction

IIIIIIIH

US007385708B2

(12) United States Patent

Ackerman et al.

(io) Patent No.: (45) Date of Patent:

US 7,385,708 B2 Jun. 10, 2008

(54) METHODS AND SYSTEMS FOR LASER

BASED REAL-TIME STRUCTURED LIGHT
DEPTH EXTRACTION

(75) Inventors: Jeremy D. Ackerman, Chapel Hill, NC (US); Kurtis P. Keller, Hillsborough, NC (US)

(73) Assignee: The University of North Carolina at Chapel Hill, Chapel Hill, NC (US)

( * ) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 114 days.

(21) Appl. No.: 10/515,305

(22) PCT Filed: Jun. 9, 2003

(86) PCT No.: PCT/US03/17987

§ 371 (c)(1),

(2), (4) Date: Apr. 27, 2005

(87) PCT Pub. No.: WO03/105289 PCT Pub. Date: Dec. 18, 2003

(65) Prior Publication Data

US 2005/0219552 Al Oct. 6, 2005

Related U.S. Application Data

(60) Provisional application No. 60/386,871, filed on Jun. 7, 2002.

(51) Int. CI.

G01B 11/24 (2006.01)

(52) U.S. CI 356/603

(58) Field of Classification Search 356/603,

356/604, 610 See application file for complete search history.

Laser-based methods and systems for real-time structured light depth extraction are disclosed. A laser light source (100) produces a collimated beam of laser light. A pattern generator (102) generates structured light patterns including a plurality of pixels. The beam of laser light emanating from the laser light source (100) interacts with the patterns to project the patterns onto the object of interest (118). The patterns are reflected from the object of interest (118) and detected using a high-speed, low-resolution detector (106). A broadband light source (111) illuminates the object with broadband/light, and a separate high-resolution, low-speed detector (108) detects broadband light reflected from the object (118). A real-time structured light depth extraction engine/controller (110) based on the transmitted and reflected patterns and the reflected broadband light.

64 Claims, 7 Drawing Sheets

OTHER PUBLICATIONS

Zitnick et al., "Multi-Base Stereo Using Surface Extraction," Visual Interface Inc., (Nov. 24, 1996).

Bajura et al., "Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient," Proceedings of SIGGRAPH 92, vol. 2 ( No. 26), p. 203-210, ( Jul. 20, 1992). Ohbuchi et al., "Incremental Volume Reconstruction and Rendering for 3D Ultrasound Imaging," Visualization in Biomedical Computing, SPIE Proceedings, p. 312-323, ( Oct. 13, 1992). State et al., "Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Lracking," Computer Graphics Proceedings, Proceedings of SIGGRAPH 96—Annual Conference Series (New Orleans, Louisiana), p. 429-438, (Aug. 4, 1996). State et al., "Technologies for Augmented Reality Systems," Computer Graphics Proceedings, Proceedings of SIGGRAPH 96, Annual Conference Series (New Orleans, Louisiana), p. 439-446, ( Aug. 4, 1996).

Garrett et al., "Real-Time Incremental Visualization of Dynamic Ultrasound Volumes Using Parallel BSP Trees," Proceedings of IEEE Visualization of Dynamic Ultrasound Volumes Using Parallel BSP Trees, p. 235-240, ( Oct. 27, 1996).

Jacobs et al., "Managing Latency in Complex Augmented Reality Systems," Proceedings of 1997 Symposium on Interactive 3D Graphics, Annual Conference Series, ACM SIGGRAPH (Providence, Rhode Island), p. 49-54, ( Apr. 27, 1997). Livingston et al., "Magnetic Tracker Calibration for Improved Augmented Reality Registration," Presence: Teleoperators and Virtual Environments, MIT Press, vol. 6 (No. 5), p. 532-546, ( Oct. 21, 1997).

Advertisement, "Inspeck 3DC 3D Capturor," Inspeck 3DC 3D Capturor (www.inspeck.com) (1998).

Advertisement, "Virtual 3D High Speed Non-Contact Surface Perception," Virtual 3-D Technologies Corporation (www. virtual3dtech.com), ( Dec. 21, 1998).

Advertisement, "Virtuoso," Visual Interface, Inc., Visual Interface,
Inc. (www.visint.com), ( Dec. 21, 1998).
Advertisement, "Virtuoso," Visual Interface, Inc., (1998).
Ohbuchi "Incremental Acquisition and Visualization of 3D
Ultrasound Images", Ph.D. Dissertation, UNC-CH Computer Sci-
ence Technical Report TR95-023, (1993).

Ohbuchi et al. "An Incremental Volume Rendering Algorithm for
Interactive 3D Ultrasound Imaging", UNC-CH Computer Science
Technical Report TR91-023, (1991).

Fuchs et al. "Towards Performing Ultrasound-Guided Needle Biopsies from Within a Head-Mounted Display", 4th International Conference, VBC '96, Hamburg, Germany, (Sep. 22-25, 1996). Depiero et al., "3-D Computer Vision Using Structured Light: Design, Calibration and Implementation Issues," The University of Tennessee, p. 1-46, (1996).

Fuchs et al., "Augmented Reality Visualization for Laparoscopic Surgery," MICCAI, vol. 11 ( No. 13), p. 934-943, (Oct. 1998). State et al., "Case Study: Observing a Volume Rendered Fetus within a Pregnant Patient," IEEE Visualization Conference, 5fh ed., p. 364-368, (1994).

* cited by examiner