WO2011011193A1

WO2011011193A1 - Integrated display in a hand-held three-dimensional metrology system

Info

Publication number: WO2011011193A1
Application number: PCT/US2010/041045
Authority: WO
Inventors: Nathan E. Wallace; Timothy I. Fillion
Original assignee: Dimensional Photonics International, Inc.
Priority date: 2009-07-21
Filing date: 2010-07-06
Publication date: 2011-01-27

Abstract

Described is a user-manipulated imaging device for measuring a three-dimensional surface of an object. The device includes an imager configured for acquiring two-dimensional images of the surface and a device housing coupled to the imager and configured for manual positioning of the imager. The device also includes a processor in communication with the imager and configured to generate three-dimensional surface data based on the two-dimensional images. The device further includes a display coupled to the device housing and in communication with at least one of the imager and the processor. The display shows images of the surface and is observable within a field of view of the user while the device housing is manually positioned within the field of view and relative to the surface. In various embodiments, the display shows the two-dimensional images and representations of the three-dimensional surface data.

Description

DPI-019PC

INTEGRATED DISPLAY IN A HAND-HELD THREE- DIMENSIONAL METROLOGY SYSTEM

RELATED APPLICATION

This application claims the benefit of the earlier filing date of U.S. Provisional Patent Application Serial No. 61/227,255, filed July 21 , 2009, titled "Integrated Display in a Hand-Held Three-Dimensional Metrology System," the entirety of which is incorporated herein by reference.

FIELD OF THE INVENTION

The invention relates to the field of three-dimensional imaging and more specifically to the field of displaying non-contact surface measurement data for dental and medical applications.

BACKGROUND OF THE INVENTION

A variety of precision non-contact three-dimensional (3D) metrology systems have been developed for dental and medical applications.

Conventional systems typically include a handheld camera or scanner connected to a processing unit that communicates with a display monitor. The display monitor presents a variety of information to the user. The information can include control options, acquired images, and operator assistance information such as an indication of an optimal focus condition. This configuration requires the user to look in two directions, that is, to look at the position of the handheld device with respect to the patient and to look at the display monitor to determine that proper images are being acquired. Thus the time and effort to obtain the desired measurement data is adversely affected by the requirement for the user to alternately view the position of the device and view the acquired images. DPI-019PC

SUMMARY

In one aspect, the invention features a method of displaying

information for a user-manipulated 3D imaging device. The method includes acquiring a plurality of two-dimensional (2D) images of a surface of an object with an imaging device manipulated by a user in position relative to the surface of the object and within a field of view of the user. The 2D images are processed to generate three-dimensional surface data for the surface of the object. Measurement data are displayed to the user within the field of view of the user during continued manipulation of the imaging device. In one embodiment, the displayed measurement information includes the two-dimensional images acquired by the imaging device and, in another embodiment, the displayed information includes a representation of the 3D surface data.

In another aspect, the invention features a user-manipulated imaging device for measuring a 3D surface of an object. The imaging device includes an imager, a device housing, a processor and a display. The imager is configured for acquiring 2D images of a surface of the object. The device housing is coupled to the imager and configured for manipulation by a user to position the imager relative to the surface of the object. The processor communicates with the imager and is configured to generate 3D surface data for the surface based on the 2D images. The display is coupled to the device housing and communicates with at least one of the imager and the processor. The display shows images of the surface observable within a field of view of the user while the device housing is manually positioned within the field of view of the user relative to the surface. In one embodiment, the display shows the 2D images of the surface acquired by the imager and, in another embodiment, the display shows a representation of the 3D surface data generated by the processor. DPI-019PC

BRIEF DESCRIPTION OF THE DRAWINGS

The above and further advantages of this invention may be better understood by referring to the following description in conjunction with the accompanying drawings, in which like numerals indicate like structural elements and features in the various figures. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.

FIG. 1 illustrates a 3D imaging device that projects a structured light pattern onto an object. FIG. 2 is a flowchart representation of an embodiment of a

measurement procedure using a hand-held 3D imaging device according to the invention.

FIG. 3 illustrates an embodiment of a user-manipulated imaging device according to the invention. FIG. 4A illustrates an embodiment of a user-manipulated imaging device according to the invention and showing a display panel in an open position.

FIG. 4B illustrates the user-manipulated imaging device of FIG. 4A showing the display panel in a closed position. DETAILED DESCRIPTION

In brief overview, the invention relates to a user-manipulated 3D metrology device such as a hand-held camera or scanning device. The device includes an integrated display monitor that provides the user with convenient access to control options, acquired images, and operator assistance indications within a field of view of the user. Advantageously, the location of the operating tip of the device relative to the object being measured can be viewed without the need to redirect the view of the user to DPI-019PC a display monitor. For medical and dental 3D metrology devices, the user positions and aligns the device to a patient while simultaneously viewing a display of the acquired images or data. As a result, measurement data are obtained with less time and operator effort than is required for conventional user-manipulated 3D metrology devices.

The present teaching will now be described in more detail with reference to exemplary embodiments thereof as shown in the accompanying drawings. While the present teaching is described in conjunction with various embodiments and examples, it is not intended that the present teaching be limited to such embodiments. On the contrary, the present teaching encompasses various alternatives, modifications and equivalents, as will be appreciated by those of skill in the art. Those of ordinary skill in the art having access to the teaching herein will recognize additional implementations, modifications and embodiments, as well as other fields of use, which are within the scope of the present disclosure as described herein.

In a typical dental or medical 3D camera or scanner imaging system, a series of 2D intensity images of an object surface is acquired where the illumination for each image can vary. In some systems, structured light patterns are projected onto the surface and detected in each 2D intensity image. FIG. 1 shows an example of a 3D imaging system 10 in which the structured light pattern is generated by a projector 14 as a pair of

overlapping coherent optical beams 16A and 16B that illuminate the object 18. The 3D imaging system 10 may be constructed to operate in accordance with the principles described in U.S. Patent No. 5,870, 191 , titled "Apparatus and Methods for Surface Contour Measurement," incorporated herein by reference in its entirety. A CCD camera 22 is used to acquire images of the illuminated object 18. The fringe pattern 26 resulting from the interference of the two beams 16 is varied between successive 2D images acquired by the camera 22. For example, the fringes in the fringe pattern 26 can be shifted by changing the phase difference between the two beams 16. DPI-019PC

A processor 30 calculates the distance from the camera 22 to the object surface for each image pixel based on the intensity values for the pixel in the 2D images. Thus the process creates a set of 3D coordinates, that is, a "point cloud," for the object surface. In a dynamic 3D imaging system, a series of point clouds is acquired while the camera or scanner is in motion relative to the object surface. For example, the imaging system can be a handheld device that a user manually positions relative to the object surface. In some applications, multiple objects surfaces are measured by moving the device relative to the objects so that surfaces obscured from view of the device in one position are observable by the device in another position. A processor registers the overlapped region of adjacent point clouds, using a 3D correlation technique or other registration technique, to transform each successive point cloud into an initial coordinate space. The successive point clouds are thus "stitched" into a common reference space.

Referring to FIG. 2, at the start of an embodiment of a measurement procedure 100 according to the invention, the user aligns and positions (step 1 10) the hand-held imaging device relative to the patient while acquiring 2D images of a patient area of interest. The 2D images are processed (step 120) to generate 3D surface data of the area of interest. The user simultaneously observes measurement images in a display while controlling (step 130) the positioning and motion of the handheld imaging device with respect to the patient. The images in the display can be the acquired 2D images. Alternatively, the displayed images can be 3D surface representations generated by processing the acquired 2D images. By way of examples, the 3D surface representations can be 3D wire-mesh

representations of point clouds or artificial surface displays that comprise simple geometrical shapes (e.g., triangles) between neighboring points in point clouds. DPI-019PC

Providing a display that is in communication with the processor and mounted to or otherwise integrated with the 3D imaging device according to the principles of the invention permits the user to see the acquired 2D images, 3D surface representation, other display information or

combinations of such images and information simultaneous with the observation and continued manipulation of the 3D imaging device relative to the patient. Thus the user can more easily and rapidly complete the measurement procedure than would be possible using a conventional handheld dental or medical imaging device. Other displayed information can include operator assistance information such as a slide bar shown along the edge of the display to indicate measured position within a usable imaging range, the distance to a surface of the object being measured, and a color box to indicate the current mode of the device, such as idle, preview and scan modes. In one embodiment, the display includes a touchscreen that permits the user to input selection data while maintaining the handheld device in proper position relative to the patient. Control options shown on the touchscreen display can include, by way of example, preview, scan and stop function activation "buttons;" save and redo buttons presented at the completion of a scan, and input data buttons. For example, in dental applications, the input data buttons can be used to indicate the jaw to be imaged (upper or lower) or particular teeth to be imaged for a partial jaw scan.

In another embodiment illustrated in FIG. 3, the imaging device 34 includes a miniature display 38 similar to the displays typically used in mass-produced cell phones for consumers. The miniature display 38 can be embedded in a side of the device housing 42 and optionally has a viewing surface that is flush with the housing 42. By way of a specific example, the miniature display 38 may have a 1.8 inch diagonal viewing area. In one embodiment, the display is a compact liquid crystal display (LCD). DPI-019PC

In an alternative embodiment, a display 46 is integral to a panel 50 that is pivotally attached to a side of a device housing 54 for the 3D imaging device 58 shown in FIG. 4A and FIG. 4B. The panel 50 is small enough to be compatible with the overall dimensions of the device 58 and yet include a display 46 that is large enough to present detailed images to the user. By way of a specific example, the display 46 can have a four inch diagonal viewing area. FIG. 4A shows the panel 50 in an open position in which the user views the displayed images, 3D representations and information.

FIG. 4B shows the panel 50 in a closed position such that the panel 50 is substantially parallel and adjacent to the side of the device housing 54. The closed position is intended for when the device 58 is stored or otherwise not in use for extended periods of time.

In the embodiments described above, the device according to the invention is generally described as a handheld device; however, the invention also contemplates that the device can be manually adjusted or manipulated by a user without being directly held by hand.

While the invention has been shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention.

What is claimed is:

Claims

DPI-019PC CLAIMS

1. A method of displaying information for a user-manipulated three- dimensional imaging device, the method comprising:

acquiring a plurality of two-dimensional images of a surface of an object with an imaging device manipulated by a user in position relative to the surface of the object and within a field of view of the user;

processing the two-dimensional images to generate three-dimensional surface data for the surface of the object; and

displaying measurement information to the user within the field of view of the user during continued manual manipulation of the imaging device.

2. The method of claim 1 wherein the displayed measurement

information comprises the two-dimensional images acquired by the imaging device.

3. The method of claim 1 wherein the displayed measurement

information comprises a representation of the three-dimensional surface data.

4. The method of claim 1 wherein the displayed measurement

information comprises operator assistance information. DPI-019PC

5. The method of claim 4 wherein the operator assistance information comprises a distance to the surface of the object.

6. A user-manipulated imaging device for measuring a three-dimensional surface of an object, comprising:

an imager configured for acquiring two-dimensional images of a surface of an object;

a device housing coupled to the imager and configured for

manipulation by a user to position the imager relative to the surface of the object;

a processor in communication with the imager and configured to generate three-dimensional surface data for the surface based on the two- dimensional images; and

a display coupled to the device housing and in communication with at least one of the imager and the processor, the display showing images of the surface observable within a field of view of the user while the device housing is manually positioned within the field of view of the user relative to the surface.

7. The user-manipulated device of claim 6 wherein the images shown in the display are the two-dimensional images of the surface acquired by the imager.

8. The user-manipulated device of claim 6 wherein the images shown in the display are representations of the three-dimensional surface data generated by the processor. DPI-019PC

9. The user-manipulated device of claim 6 wherein the display shows operator assistance information.

10. The user-manipulated device of claim 9 wherein the operator assistance information includes a distance to the surface of the object.

11. The user-manipulated device of claim 6 wherein the display is a touchscreen display configured to receive data input from the user.

12. The user-manipulated device of claim 6 wherein the display comprises a liquid crystal display (LCD).

13. The user-manipulated device of claim 6 wherein the display comprises a display panel pivotably secured to a side of the device housing, the display panel extending away from a surface of the device housing while in an open position and extending substantially parallel to the surface of the device housing while in a closed position, and wherein images of the surface are observable to the user while the display panel is in the open position.

14. The user-manipulated device of claim 6 wherein the display comprises a viewing surface that is substantially flush with a side of the device housing. DPI-019PC

15. The user-manipulated device of claim 6 further comprising a projector in communication with the processor and configured for projecting a structured light pattern onto the surface of the object.

16. The user-manipulated device of claim 15 wherein the projector comprises a source of coherent optical beams for illuminating the surface of the object with a fringe pattern.