WO2011033186A1

WO2011033186A1 - Method for the three-dimensional digitisation of a surface, comprising the projection of a combined pattern

Info

Publication number: WO2011033186A1
Application number: PCT/FR2010/000610
Authority: WO
Inventors: Benoît BOCQUILLON; Ludovic Brethes
Original assignee: Noomeo
Priority date: 2009-09-15
Filing date: 2010-09-08
Publication date: 2011-03-24
Also published as: FR2950157A1

Abstract

The invention relates to the use of a structured pattern in the construction of a three-dimensional digital model of a physical surface (2), in which the pattern is projected onto the surface (2), said pattern being formed by combining a primary pattern (28) comprising an array of spaced-apart continuous straight fringes (29) and a secondary anisotropic pattern (30) extending through the spaces between the fringes (29).

Description

Method for three-dimensional digitization of a surface comprising the projection of a combined pattern

The invention relates to the construction of three-dimensional numerical models of physical surfaces by optical measurement, without contact. This technique is commonly called three-dimensional scanning, 3D scanning, or, in English terminology, 3D scanning.

The architecture of vision systems for 3D digitizing a surface conventionally comprises two components:

a physical component: the scanner (preferably portable), which acquires images of the surface, generally by a succession of shots (point, continuous or continuous) thereof;

a software component for processing the images produced by the scanner, this processing making it possible to carry out a three-dimensional reconstruction of surface synthesis, for example in a computer-assisted design (CAD) or computer-aided design (CAD / CAM) environment.

The acquisition of the images is generally carried out by projecting on the surface to be digitized a structured luminous pattern having a predetermined shape (lines, squares, speckles, etc.), then by optical capture of the projected pattern. Treatments are applied to the captured images, making it possible to calculate the spatial coordinates of a selection of points to reconstruct the three-dimensional model of the surface.

On the basis of this same fundamental principle, many solutions have been proposed, which vary according to three main axes: scanner architecture; the nature and form of the projected motif; the reconstruction methodology.

In the first place, scanners are generally divided into two types: single-camera scanners (monocular vision), and multi-camera scanners (binocular vision or stereovision, multi-ocular vision).

In scanners operating in monocular vision, only one image of the deformed pattern is produced. This image is then compared to a reference image of the undistorted pattern to perform the reconstruction. Patent documents US 5003 166 (MIT), FR 2842591 (ENSMA / CNRS), WO2007 / 043036 (PRIME SENSE), US 2009/0059241 (ARTEC) and FR 2921 719 (NOOMEO) illustrate this architecture.

In scanners operating in binocular or multiocular vision, which are those which interest us here, at least two images of the deformed pattern are produced simultaneously, and a comparison of these images is carried out to carry out the reconstruction. International application WO 2006/094409 (CREAFORM) illustrates this architecture.

Secondly, in the case of the projected pattern, it may be repetitive, for example in the form of lines or segments (see US 5003 166 above), or non-repetitive, for example in the form of a speckle (cf. 2842 591 and FR 2921 719 cited above).

Lastly, with regard to the reconstruction methodology, it can vary from one solution to another in the choice of algorithms, but generally comprises three steps:

rectification of images;

- matching;

the calculation of the spatial coordinates of points of the digitized surface.

In stereovision, the rectification of images is preferable to facilitate their subsequent exploitation. In addition to correcting the distortions induced by the optical systems, images are generally applied to an epipolar rectification treatment to make the epipolar lines of the two images parallel and aligned.

The matching or correlation then consists in matching in the stereoscopic images the homologues or stereo- corresponding, that is to say the projections in the image planes of the same point of the physical surface. The difference between the coordinates of the stereo-correspondents is called disparity.

Once the disparities are calculated, the spatial coordinates of the corresponding points of the surface are then calculated by triangulation, the result of the calculations providing a cloud of points located precisely in space. From a scatter plot, mapping techniques provide a continuous surface. The rectification of images is a well-known problem, for which powerful and robust algorithms have existed for some time now, cf. for example P. Lasserre and P. Grandjean, "Stereo improvements", in IEEE International Conference on Advanced Robotics, Barcelona, 1995.

Similarly, once the mapping is done, the triangulation calculations do not pose any particular problems.

On the other hand, the problem of matching is much more complex, and is currently the subject of much research to achieve efficient, fast and robust solutions.

For an overview of some mapping techniques (and, incidentally, a better understanding of epipolar geometry in stereovision), cf. for example V. Lemonde, "Stereovision on vehicle: from Self-Calibration to Obstacle Detection", CNRS / I NSA Thesis, November 2005, S. Chambon, "Stereoscopic mapping of color images in the presence occultations ", Thesis Paul Sabatier University, December 2005, and W. Souid-Miled," Stereoscopic mapping by convex variational approaches ", Thesis University Paris-Est / I NRIA, December 2007.

The complexity of matching increases with the number of points you want to match. Indeed, while it is relatively simple to select a small number of points of interest (for example, located in a highly contrasting environment making it possible to easily detect the corresponding stereo-speakers), a denser pairing quickly encounters difficulties of analysis because of the presence of ambiguities in certain areas of the images which sin for example by a too homogeneous aspect, or even by the total absence of content (for example due to occultation phenomena which do not fail to appear, especially for surfaces with pronounced reliefs).

It appears that a poor design of the pattern to be projected on the surface to be digitized is the source of numerous misregistration errors and / or insufficient reconstruction density.

Mismatch errors lead to the achievement of clearly aberrant points that must be eliminated by means of complex filtering techniques. This elimination requires power and calculation time, it decreases the total number of useful points - and therefore the density of the reconstruction.

We can try to remedy an insufficiency of density by reiterating the operations of acquisition and treatment, so as to generate several clouds of points which it is then necessary to recalibrate in the space because of the offsets between two taken of successive views. But the techniques of registration are complex, and require a power and / or a computation time proportional to the number of clouds to be recalibrated.

Repetitive patterns, such as alternating black and white lines or grids (see for example Figure 6 of US 5,003,166 cited above), which have the advantage of offering good contrast, are difficult to exploit in the stereovision systems because they generate matching errors due to the isotropic nature of the pattern, so offsets may occur when looking for correlation windows.

The pseudo-random motifs, of the mouchetis type (see for example FIG. 7B of the document US Pat. No. 5,003,166 or the document FR 2,921,719 cited above), have the natural advantage of not being repetitive, so that the risks offset when looking for correlation windows are weak. On the other hand, their contrast is mediocre, which complicates the search for points of interest.

The aim of the invention is to propose a solution that overcomes the aforementioned drawbacks, and facilitates a reliable and dense reconstruction of a three-dimensional numerical model of a surface.

For this purpose, the invention proposes a use of a structured pattern in the construction of a three-dimensional numerical model of a physical surface, in which the pattern is projected onto the surface, this pattern being formed by the combination of a primary pattern comprising an array of spaced continuous rectilinear fringes, and an anisotropic secondary pattern extending into the interframe spaces.

Such a combination makes it possible to combine the advantages of the isotropic patterns, by offering a good contrast, and anisotropic patterns, while minimizing the risk of confusion when identifying correlation windows. The invention also proposes a three-dimensional scanning system comprising a device for projecting a luminous pattern, and a stereovision system, in which the projection device is arranged to project a luminous pattern as described above.

The invention also proposes a method for constructing a three-dimensional digital model of a physical surface using a pattern as described above, this method comprising the following operations:

Projection of the structured pattern on the surface,

Acquisition and storage of at least one two-dimensional image of the illuminated surface,

Construction of the digital model from the image thus acquired.

According to a first embodiment, the primary pattern is monooriented, the fringes extending in a single direction.

According to a second embodiment, the primary pattern is bi-oriented, the fringes extending in two directions, which may be perpendicular so that the fringes define a grid.

The fringes can be clear, and be bordered by dark lines between which the secondary pattern extends.

Alternatively, the fringes are dark; they can be lined with light lines lined with dark lines.

As for the secondary reason, it includes, for example, scabs or pictograms.

Other objects and advantages of the invention will become apparent in the light of the description given hereinafter with reference to the appended drawings in which:

Figure 1 is a perspective view illustrating a three-dimensional scanning device comprising a capture apparatus connected to a computer, applied to the scanning of the surface of an object, in this case the face of a statue;

Figure 2 is a schematic side view illustrating the main optical components of the apparatus of Figure 1, placed in a shooting situation; Figure 3 is a plan view of a pattern for projection on a surface to be digitized, according to a first embodiment;

Figure 4 is a detail view of the pattern of Figure 3;

FIG. 5 is a plan view of a pattern for projecting onto a surface to be digitized, according to a second variant embodiment;

Figure 6 is a detail view of the pattern of Figure 5;

Figure 7 is a plan view of a pattern for the projection on a surface to be digitized, according to a third embodiment;

Figure 8 is a detail view of the pattern of Figure 7;

Figure 9 is a plan of a pattern for projection on a surface to be digitized, according to a fourth embodiment; FIG. 10 is a view of a detail of the pattern of FIG. 9;

Figure 11 shows a pair of structured stereoscopic images of the pattern of Figures 3 and 4 projected on the face of Figure 1;

Fig. 12 shows a pair of textured stereoscopic images of the face of Fig. 1;

FIG. 13 is a perspective view of a reconstructed point cloud from the pair of structured stereoscopic images of FIG.

FIG. 14 is a perspective view showing two non-recalibrated point clouds, reconstructed from two pairs of structured stereoscopic images captured during two successive shots of the face of the statue of FIG. 1;

FIG. 15 is a view similar to FIG. 14, showing the two scattered point clouds;

FIG. 16 is a perspective view showing a plurality of recalibrated point clouds, reconstructed from a series of pairs of stereoscopic images captured during a sequence of successive shots of the face of the statue of the figure 1 .

FIG. 1 shows a non-contact optical scanning system 1 for constructing a digital model three-dimensional, in the form of a synthetic image in CAD (computer-assisted drafting) or CAD (computer-aided design) environment, of a physical surface 2, for example the envelope of a real physical object 3. In the occurrence, this object 3 is a statue but it could be any other object having one or more surfaces to be digitized. Generally, the surface 2 to be digitized is embossed, that is to say non-planar, but the system 1, as well as the method implemented described hereinafter, allow a fortiori the digitization of flat surfaces. (Note that to meet the formal requirements for patent drawings, the photograph of the statue in Figure 1 has been filtered and converted to two-color display, hence its granular appearance).

The scanning system 1 comprises an optical acquisition device 4 (called capture), called a scanner, equipped with a portable box 5 provided with a handle 6 for its capture and manipulation. The handle 6 carries, on a front face, a manual trigger 7 whose operation commands the shooting.

The system 1 also comprises a central processing unit (CPU or CPU) of the images captured by the scanner 4, in the form of a processor on which is implemented a software application for building the digital models from the captured images. More specifically, this software application includes instructions for carrying out the calculation operations described below.

The processor can be embedded in the scanner 4 or, as illustrated in FIG. 1, relocated by being integrated in a computer 8 equipped with a graphical interface and connected to the scanner 4 via a communication interface 9 wired (or wireless), for example in the form of a USB-type computer bus.

The scanner 4 is designed to work in stereovision. As can be seen in FIGS. 1 and 2, it comprises, mounted in the housing 5:

a bright projector;

a system 1 1 of stereovision comprising a pair of optical acquisition devices 12, 13, of the camera type

(ie designed for continuous shooting, for example at the normal frame rate of 24 frames per second), or cameras (that is to say shooting occasionally, possibly burst);

an optical sighting system 14;

an inertial central unit, configured to provide at any time an invariant absolute reference, constructed from two orthogonal directions identifiable by the central unit 15, namely the magnetic north and the vertical.

It is assumed in the following that the optical acquisition devices 12, 13 are cameras, which, if necessary, can be used as cameras.

The projector 10 comprises a light source (not visible), a focusing optics 16 and a slide (not visible) interposed between the light source and the focusing optics 16.

The light source is preferably a non-coherent source of white light, in particular of the filament or halogen type, or also of the diode (LED) type.

The focusing optics 16 of the projector 10 define a main optical axis 17 passing through the light source. This optical axis 17 defines an optical axis of projection of the scanner 4.

The cameras 12, 13 each comprise:

a focusing optics 18 defining an optical axis 19 (respectively 20) of sight,

a photosensitive sensor 21, for example of the CCD type, which is in the form of a square or rectangular plate placed, facing the focusing optics 18, on the optical axis 19, 20, that is to say perpendicular to it and so that the axis 19, 20 passes through the center of the sensor 21.

The optical centers of the cameras 12, 13 are spaced from one another by a distance called base, their optical axes 19, 20 converging towards a point 22 situated on the optical axis 17 of the projector 10, in the field of this one. In practice, the cameras 12, 13 will be oriented such that the point of convergence of their optical axes 19, 20 lies in a median plane situated in the projection field and in the sharpness zone (also called depth of field) of the projector 10. The sensor 21 of each camera 12, 13 is placed at a predetermined distance from the focusing optics 18, in an image plane depending on its focal length and such that the object plane corresponding to this image plane is in the sharpness zone of the camera. projector 10.

As shown in FIG. 2, the projector 10 and the cameras 12, 13 are fixed on a common one-piece plate 23, made of a sufficiently rigid material to minimize the risks of misalignment of the cameras 1 2, 13 (such a misalignment is called scaling). In practice, the plate 23 is made of an aluminum alloy.

As can be seen in FIG. 1, the handle 6 of the scanner 4 extends parallel to the base, so that the natural hold of the scanner 4 is vertical, the cameras 12, 13 extending one above the other. the other. By convention, however, the left camera is referred to as the upper camera 12 (located on the thumb side) and the right camera as the lower camera 13.

The aiming system 14 is arranged to allow, prior to the shooting, a positioning of the scanner 4 at a distance from the surface 2 to be digitized such that the projected pattern is included in the field of the projector 10 - and therefore net .

As illustrated in FIG. 2, the aiming system 14 comprises two laser pointers 24 rigidly mounted on the plate 23, designed to emit each, when the trigger 7 is depressed, a linear light beam 25, 26 producing on any surface illuminated, a light spot substantially punctual.

The pointers 24 are positioned angularly on the plate 23 so that the beams 25, 26 emitted cross at a point of intersection 27 located on the axis 17 of the projector 10, in the sharpening zone thereof and upstream of the point 22 of convergence of the optical axes 19, 20 of the cameras 12, 13.

The slide is disposed between the light source and the optic 16 of the projector 10, on the axis 17, that is to say perpendicularly to the axis 1 7 and so that it passes through the center of the slide. The slide is placed at a predetermined distance from the optics 16 of which it constitutes an object plane, so that the image of the pattern is sharp in an image plane perpendicular to the axis 17. In practice, the image plane n ' is not unique, the focusing optics of the projector 10 being designed (or adjusted) to have a certain depth of field, such that the image of the projected pattern is sharp in a certain area of sharpness that extends between two spaced planes perpendicular to the axis 17.

The slide is in the form of a translucent or transparent plate (glass or plastic), square or rectangular, on which the pattern is printed by a conventional process (transfer, offset, screen printing, flexography, laser, spray jet). ink, microlithography, etc.).

The pattern is structured, that is to say it has alternations of light areas and dark areas of a predetermined shape. The constitution of the pattern (color, contrast) is not random: it is known in every respect. The pattern can be done in grayscale or color, but is preferably made in black and white, maximizing the contrast between light and dark areas.

More specifically, the pattern comprises the combination of two sub-patterns, namely:

A regular primary pattern 28 comprising an array of spaced, continuous rectilinear fringes 29, and

a secondary, generally anisotropic, i.e., irregular and non-repetitive secondary pattern in any direction of space.

FIGS. 3 to 10 show four variants of patterns that can be used in the digitization system.

According to an embodiment illustrated in FIGS. 3 to 8 which illustrate three variants, the secondary pattern 30 comprises scabs 31 extending in the spaces between the fringes 29. According to another embodiment illustrated in FIGS. 9 and 10, the secondary pattern 30 includes pictograms arranged pseudo-randomly.

A scabby secondary pattern as shown in FIGS. 3 to 8 can be generated automatically by means of a computer program for performing pseudorandom patterns. One example is Ken Perlin's algorithm generating a well-known type of mouchetis called Perlin noise, which has the advantage of being anisotropic. In detail, this type of pattern includes an interweaving of black and white spots that give a grainy appearance to the pattern. The term "scab" refers to this interlacing. In English, this type of pattern is also called "Speckle". As will be seen in more detail below, the primary pattern 28 has the function of locally generating a strong, easily detectable contrast between white light areas and dark areas, while the secondary pattern 30 functions by its anisotropic nature. , to locate in the image with certainty any viewing window.

In the first variant of the first embodiment, shown in its entirety in FIG. 3 and in detail on an enlarged scale in FIG. 4, the primary pattern 28 is single-oriented and comprises a series of clear fringes 29 (white in color). occurrence) which extend vertically in the orientation illustrated in Figures 3 and 4, each bordered on either side by two parallel strips 32 (black in this case) parallel, similar.

The width of the fringes 29 is constant, as is the spacing between fringes 29.

As can be seen in FIG. 3, and in more detail in FIG. 4, the scabs 31 of the secondary pattern 30 extend in strips in the spaces between the fringes 29, and more precisely between the black strips 32 bordering the white fringes 29. .

Given the mono-directional nature of the primary pattern 28, there is a preferred orientation of the general pattern, relative to the positioning of the system 1 1 stereovision. Indeed, the image is printed on the slide - or it is oriented - so that the fringes 29 extend perpendicularly to the base, that is to say to the axis connecting the optical centers of the cameras 12, 13. In other words, the fringes 29 extend perpendicular to the plane containing the optical axes 19, 20 of the cameras 12, 13, so that in stereoscopic vision the fringes 29 extend vertically on the images captured by the cameras 12, 13.

Be that as it may, the presence of clear fringes 29, bordered with black bands 32, makes it possible to provide the pattern with contrasting zones that can be identified during image processing, as will be explained in more detail below.

This type of pattern is advantageous in comparison with a simple speckle which appears to be of low contrast, since the black and white spots of the slide would, by projection, form grayscale spots. As for an isotropic pattern made for example of single fringes, it is by nature not adapted to stereovision, because it is not locatable and therefore unsuitable for matching.

In the second variant, shown as a whole in FIG. 5 and in enlarged detail in FIG. 6, the primary pattern 28 is bi-oriented and comprises a series of light fringes 29 (white in this case) which extend both horizontally and vertically to form a grid. Each horizontal and vertical fringe 29 is bordered, on both sides, by discontinuous black (black) lines, which together form a network of regularly spaced isolated squares with black edges.

As can be seen in FIG. 5, and in more detail in FIG. 6, the scabs 31 of the secondary pattern 30 extend in blocks in the spaces between the fringes 29, that is to say in the squares delimited by the black bands 32.

This second variant of the pattern has, compared to the first variant, the advantage of being bi-oriented, so that there is no preferred direction of projection of the pattern on the surface 2, with respect to the orientation of the scanner 4. In addition, the multiplication of the fringes 29 makes it possible to increase the density of the contrasting zones of the pattern, in this case at the level of the fringes 29.

In the third variant, shown as a whole in FIG. 7 and in enlarged detail in FIG. 8, the primary pattern 28 is bi-oriented and comprises a series of dark fringes 29 (black in this case) which extend both horizontally and vertically to form a grid. Each horizontal and vertical fringe 29 is bordered, on both sides, by discontinuous light strips 32 (white in this case) which together form a network of regularly spaced isolated squares with white edges. In addition, the edges 32 of each square are lined, inward, with black bands 33, so that each square has a double perimeter, consisting of a white outer perimeter and a black inner perimeter. In addition to having the same benefit as the second variant of being bi-oriented, this third variant also has the advantage of further increasing the density of contrasting areas of the pattern.

In the second embodiment, illustrated as a whole in FIG. 9 and in detail on an enlarged scale in FIG. primary 28 is bi-oriented, and comprises a series of clear fringes 29 (white in this case) that extend both horizontally and vertically to form a grid. The squares delimited by the fringes 29 have a dark background (black in this case) on which stand out pictograms 34 of light color (white in this case).

As can be seen in FIG. 9, only a limited number of different pictograms 34 can be used. However, in order to make the secondary pattern 30 generally anisotropic, the pictograms 34 are distributed pseudo-randomly and so that the vicinity of each pictogram 34 (for example the adjacent pictograms 34 located on the same line, or on the same column) is different from the neighborhoods of all the other pictograms 34. This pattern has the advantage of being more contrasted than the scab patterns, which facilitates matching operations (see below).

The operation of the three-dimensional scanning system 1 is now described. This operation is based on the succession of the following steps, performed after a prior calibration step of the stereovision system 11:

- Digitization of surface 2;

Rectification of images;

Reconstruction of point clouds;

Recalibration of point clouds;

Construction of the three-dimensional model.

Calibration

The calibration of the stereovision system 1 1 is a necessary preliminary step, without which it is not possible to properly process the information captured by the cameras 12, 13.

The calibration of a single camera (in monocular vision) is limited to estimate the intrinsic parameters of the camera as well as its position relative to an absolute reference frame. Calibration of the system 1 1 stereovision is more complex because it involves performing a double calibration, which includes not only the estimation of the intrinsic parameters of each camera 12, 13, but also the relative position and orientation of the cameras 12, 13 relative to each other. In practice, the calibration consists of matching the two cameras 12, 13 by implementing in the digitization system CPU 1 a given number of structural and optical parameters of the stereovision system 1 1, in particular the position of the optical centers and the angle between the optical axes 19, 20.

This calibration is not necessarily performed before each use of the system 1. It is preferably carried out at the factory, using a test pattern (for example a checkerboard) whose geometry is known exactly.

It is possible to calibrate the stereovision system 1 1 according to a known procedure and algorithms. For this purpose, the skilled person may in particular refer to V. Lemonde, "Stereovision embedded on vehicle: self-calibration to the detection of obstacles", Thesis, CNRS / I NSA, November 2005, cited above, or B. Bocquillon, "Contributions to the autocalibration of cameras: models and solutions guaranteed by interval analysis", Thesis, University of Toulouse, October 2008.

scanning

Scanning is the step of acquiring one or more pairs of stereoscopic images of the surface 2. More specifically, the scanning procedure comprises the succession of the following operations.

In the first place, the scanner 4 is positioned facing the surface 2 to be digitized - in this case facing the face of the statue.

A pulse on the trigger 7 causes the lighting of the pointers 24. The scanner 4 can then be positioned correctly, the concordance of the light spots produced by the lasers on the surface 2 signifying that it is located in the field of the projector 10 and cameras 12, 13.

Once positioned the scanner 4, the acquisition procedure is then initiated by a long press on the trigger 7. The procedure continues as long as the trigger 7 is held down. This procedure consists of the following steps:

a.1) Lighting the projector 10. This lighting causes the projection of the pattern on the surface 2.

a.2) Extinction of pointers (to avoid camera blindness

12, 13) and acquisition by the cameras 12, 13 of a first couple two-dimensional stereoscopic images of the surface 2 thus illuminated. These images are said to be structured because they contain the structured pattern projected on the surface 2, according to the two different points of view of the cameras 12, 13. Although the system 1 1 of stereovision is not necessarily oriented horizontally, conventionally we call image left and G is the structured image produced by the left camera 12, and right image - denoted D - the structured image produced by the right camera 13. The acquisition of an image comprises in fact two steps. First the shooting, during which the image is printed on the photosensitive cells of the sensor 21, then the reading of the image by the on-board electronics of the camera 12, 13, with progressive setting of the image, in digital form, in a buffer embedded in the camera. The sensors 21 being blocked during their reading, the cameras are momentarily unusable and consequently put on hold,

a.3) Memorization of the G-D structured stereoscopic image pair: once the G and D images have been buffered, they can be transmitted to the system CPU 1 1 where they are stored for processing (see the rectification and reconstruction phases described below). A pair of G-D structured stereoscopic images of the face 2 of the statue 1 is illustrated in FIG. 13. This figure is provided for illustrative purposes, the G-D images not necessarily being displayed on the graphical interface of the computer 8.

a.4) Shutdown of the projector 10. This shutdown can be controlled immediately after the acquisition of the structured images G and D, for example during the reading of the images, or during their storage by the CPU of the system 1 1.

a.5) Acquisition by the cameras 12, 1 3 of a second pair of two-dimensional stereoscopic images of the unlit (and not pointed) surface 2. This operation is similar to the operation a.2), except that the images are not structured, the pattern has not been projected on the surface 2.

The images of the unlit surface 2 are made in natural light (ambient), possibly embellished with a white light produced for example by light-emitting diodes. (LED) equipping the scanner 4. Thus, according to a particular embodiment, the scanner 4 is equipped in front of two circular series of LEDs surrounding the optics 18 of the cameras 12, 13 and oriented along the axes 19, 20 of these this.

These images are said to be textured, because in the absence of the projected motif they contain the natural texture of the surface 2. The left image is denoted G '; the right image D '. The images G 'and D' are read by the internal electronics of the cameras 12, 13 and stored in the buffer memory. Meanwhile, the cameras 12, 13 are put on hold.

a.6) Memorization of textured images G'-D '. Once the images G 'and D' have been buffered, they are transmitted to the CPU of the system 1 1 where they are stored for processing with the pair of structured stereoscopic images G-D. textured stereoscopic image G'-D 'of the face 2 of the statue 1 is illustrated in Fig. 14. (Note that to meet the formal requirements for patent drawings, the images shown in Fig. 14 have been filtered and converted for a display in two colors, hence their granular appearance not to be confused with scab).

This acquisition procedure therefore provides a quadruplet of images, hereinafter referred to as "quad", composed of the two pairs of stereoscopic images: a pair of structured images GD and a pair of textured images G'-D ' .

The procedure, ie the sequence of operations a.1) to a.6), can be repeated as much as necessary, in order to obtain a series of quads. During this rehearsal, it is possible to scan the surface 2 to cover it completely and shoot it continuously. In order to increase the density of the reconstruction (see below), it is also preferable to make several acquisitions from the same point of view, or from similar points of view in order to obtain an overlap of the zones of the surface 2 covered by the cameras 12, 13.

In order to avoid the drift of the scanner 4 out of the field of the projector 10 during the scanning of the surface 2, the pointers 24 can be activated permanently - with the exception of the sequences of shots, failing which the images would be unusable - so to allow the user to keep the distance between the scanner 4 and the surface 2 relatively constant. Given the shutter speeds of the cameras 12, 13 (of the order of a few milliseconds), the eye does not perceive the extinction of the lasers 24 during shooting, the user having the illusion of a continuous pointing.

Given the precision of the CCDs and the memory capacity of the system 1, conclusive tests could be conducted with the following parameters:

Frequency of the acquisition (1 quad): 4 Hz (4 quads / s)

- Shutter speed (shooting): about 5 ms

Reading time for each image: 28 to 50 ms

CCD sensor size: 1024x768 pixels

8-bit gray scale (0-255)

Memory space required for each image: 0.7 MB

Rectification

The raw images of each quad, obtained by the acquisition procedure described above, are affected by defects, notably a distortion and an absence of parallelism of the epipolar lines between images of the same stereoscopic pair.

Also, before submitting the images to the processing intended to allow the reconstruction of the three-dimensional model of the surface 2, is it preferable to apply to them a pretreatment including the following operations, programmed in the form of instructions in the CPU of the computer 8 for their implementation:

Rectification of the distortion (for example by means of known algorithms), aimed at straightening the straight lines that appear curved on the images;

Epipolar rectification, which allows, for a pair of stereoscopic images, to correspond at any point of the left image a horizontal line of the right image, and vice versa.

Like the distortion rectification, epipolar rectification can be conducted using conventional algorithms. The skilled person can refer to the documents mentioned in the introduction.

Reconstruction The reconstruction step aims at calculating, from the pair of structured stereoscopic images GD, a cloud of three-dimensional points of the surface to be digitized.

The reconstruction comprises two phases, programmed in the form of instructions in the CPU of the computer 8 for the implementation of:

Mapping (M EC)

Triangulation Matching

The MEC consists of matching points of the pair of G-D structured images.

The CEM is performed in two time periods: an initial, relatively coarse MEC; then a final, finer MEC based on the results of the initial MEC.

The initial M EC consists in carrying out for each pixel P _G of coordinates of the image G a local correlation consisting of:

Select in the image G a neighborhood, called correlation window, centered on the pixel P _G ;

- Compare the correlation window of the pixel P _G with a series of windows of correlation of the same size (in the methods of block matching) or of size d ifferente centered on a pixel P _D of coordinates (k, l) of the image D, located on the line of the image D containing the epipolar line corresponding to the pixel P _G ; each correlation window of the image D corresponds to a correlation measure (or score);

In this series, select the correlation window that maximizes the correlation score - or mimicry, depending on whether sim ilarity or dissimilarity criteria are used. The pixel P _D located in the center of this window is then matched to the pixel P _G , the pixels P _G and P _D being denominated as homologous or stereocomponent.

We can then calculate a first, so-called initial estimate of the disparity D (x) between the views 1 and 2 at the pixel P _G :

D (P _G ) = x (P _D ) - x (P _G ) Where x is a measure parallel to the horizontal lines of the G and D (abscissa) images.

The initial MEC is relatively robust. It is all the more so because the secondary pattern 30, pseudo-random, makes it possible to minimize mismatches.

It would be possible, from the coordinates x (P _G ) and x (P _D ) corresponding stereo pixels, and through a calculation of triangulation, calculate the position of a point of the surface 2 whose projections on the images G and D would be located respectively in the pixels P _G and P _D. However, the accuracy of the positioning of this point, particularly in depth, would be insufficient to meet the objectives set by the applicant, namely a depth tolerance of 0.1 mm.

Indeed, disparity and depth accuracy can be related by the following equation:

_{Az =} work ² . _{Ap (1)} focal-base

Or :

Δζ (expressed in mm) is the depth tolerance, measured along the optical axis 17 of the projector 10;

ztravaii (expressed in mm) is approximately the normal working distance of the scanner 4, i.e. the average distance between the scanner 4 and the surface 2 to be digitized, measured along the optical axis 17 of the projector 10 ;

focal length (expressed in pixel equivalents) is the focal length of the cameras 12, 13;

base (expressed in mm) is the distance between the optical centers of the cameras 12, 13;

AD (expressed in pixels) the tolerance on disparity.

It is assumed that the system 11 has the following configuration: ^z work between 350 and 500 mm (ie 425 mm on average) focal length 1700 pixels

base 140 mm.

Given this configuration (which can of course vary), and assuming that it is desired to achieve an accuracy of 0.1 mm, the accuracy of the disparity must be 0.1 pixel. In in other words, the disparity must be precise to the sub-pixel, and more precisely to the tenth of a pixel. Such precision is not attainable by means of an initial MEK as described above.

However, as we will now see, the results of the initial MEC can be used as a basis for the fine MEC, which is more accurate but based primarily on the primary motive points.

The final MEC includes the following operations, programmed as instructions in the CPU of the computer 8 for their implementation:

b.1) Detection in image G of local peaks of light intensity.

These can be vertices or troughs, which are formally equivalent. In practice, the vertices are detected. This detection can be carried out using known algorithms, such as the algorithm of Biais and Rioux (see F. Biais and M. Rioux, "Real-time numerical peak detector", in Signal processing, 1986, vol. 1, No. 2, pp.145-155). Concretely, from the image G, we establish a map of a derivative in each pixel P _G (i, j) of the light intensity corresponding to this pixel.

Insofar as the luminous intensity is not a continuous but discrete function, the derivative is computed in a discrete manner, for example equal to the slope of the segment joining two successive points whose abscissa values are those of the corresponding pixels, and the ordinates the values of the corresponding light intensities (on a scale from 0 to 255 for eight-bit gray levels).

It is then decreed that a pixel corresponds to a peak when a function derived from the luminous intensity changes sign between this pixel and the neighboring pixel situated on the same line P _G (i + 1, j). More precisely, and according to the algorithmic chosen, we can decree that the pixel corresponds to a peak when the function derived from the luminous intensity changes from sign to rising, that is to say that it is negative to the pixel P _G (i, j) ^and positive to the pixel P _G (i + 1, j). b.2) Calculation of the coordinates of the points of the image G, called extremums (in practice they are maximums) and noted below

E _G , corresponding to the peaks detected. These points are not pixels. These are points whose coordinates in the image G are real numbers.

For this purpose, for a peak detected in the pixel P _G (i, j) during the operation b.1), a window is chosen containing the pixel P _G (i, j), for example the window comprising P _G (i, j) ^and ^'are the two pixels enclosing on the same line, or P _G (i - l, j) and P _G (i + 1, j).

A continuous interpolation function of the luminous intensity of the pixels of this window is then calculated, and then the abscissa x _G of the extremum E _G , that is to say of the point which maximizes this function, is calculated. interpolation. The interpolation function being continuous, the abscissa x _G is not complete: it is a real number.

b.3) For each extremum E _G thus calculated, determination, in the second D image of the torque GD, of the corresponding stereo of the pixel P _G (i, j), that is to say of the pixel P _D () paired, at the initial MEC, with the pixel P _G (i, j) of the image G containing the extremum E _G.

b.4) Selection, in the image D, of a window containing the pixel P _D (fe, Z).

This window comprises for example P _D (/ c, /) and the pixels flanking it on the same line, ie P _D (c - 1.0 and P _D (/ c + 1, /).

b.5) Detection in this window of a peak of light intensity.

b.6) If such a peak exists, calculation of a continuous interpolation function of the luminous intensity of the pixels of this window, and calculation of the abscissa x _D of the extremum E _D , that is to say say the point that maximizes this interpolation function. As for the extremum E _G , the abscissa of the extremum E _D is not complete since it is a real number.

b.7) Apartment of the extremums E _c and E _D. One can possibly calculate the disparity:

D = x _D - x _G

The abscissae x _D and x _G being, as we have seen, real numbers, the precision obtained on the disparity is much less than 0.1 pixel, in accordance with the objectives indicated above.

Steps b.1) to b.7) are repeated for all light intensity peaks, the mapping produces a pair of pairs of points E _c and E _D paired for the GD image pair. As we have seen, the contrast is lower in the secondary pattern 30 than in the primary pattern 28. Also the detection of peaks is less reliable in the areas of the images G and D corresponding to the secondary pattern 30, than in the zones corresponding to the primary pattern 28, in which the alternation of the light and dark areas is clearer.

Also, for precision purposes - but at the expense of the reconstruction density - is it preferable to limit the fine ECM to a selection of points of interest located in the fringes 29 of the primary pattern 28, that it is by therefore necessary to locate. This location can be performed by vertical scanning of the image (in the case of the first pattern version) or by combining a vertical scan and a horizontal scan (in the case of the second version and the third version of the pattern), and by estimating the similarity of the neighborhoods. The fringes 29 are characterized in fact by a directional isotropy which results in a homogeneity of the neighborhoods of the points of the fringes 29 in the direction in which it extends.

triangulation

Since the extremums E _G and E _D are homologous (or stereo- corresponding), they are the projections, respectively in the left image G and in the right image D, of the same point of the surface 2 to be digitized.

Therefore, the coordinates of this point can be easily calculated by triangulation from the positions of E _G and E _D.

Thus, from the set of doublets of extremums E _G and E _D , the triangulation makes it possible to obtain a cloud of points which forms a first blank, scattered, of the three-dimensional model of the surface to be digitized.

Such a cloud of points is represented in FIG. 15. This cloud results from the succession of the MEC and triangulation phases applied to the image pair GD of FIG. 11. It can be seen from FIG. 13 that the cloud is oriented, and includes several groups of points all located in vertical parallel planes. This illustrates the MEC technique described above, in which the fine MEC was limited to a selection of points of interest located in the fringes 29 of the primary reason 28, because of the precision of their location in the G and D images.

retiming

The reconstruction being repeated for all the pairs of GD structured images captured during the multiple shots of the surface 2, we obtain a series of scattered scattered point clouds scattered in space, which it It is therefore necessary to recalibrate to merge them before starting the meshing procedure for constructing the three-dimensional model of the surface 2. There is shown in Figure 14 two reconstructed point clouds from two successive quads, before resetting. The registration operations are programmed as instructions in the CPU of the computer 8 for their implementation.

From the point of view of the cameras 12, 1 3, the scanned surface 2 has evolved in space, between two successive shots, in a reference linked to the scanner 4, in practice a Cartesian landmark whose center coincides with the image focus of the left camera 12, and whose axes are respectively:

x: the line parallel to the horizontal of the sensor 21 of the left camera 12 passing through its image focus, located in the center of the sensor 21;

y: the line parallel to the verticals of the sensor 21 of the left camera 12 and passing through its image focus;

z: the optical axis 19 of the left camera 12.

The registration aims to evaluate the movements of the scanner 12 between two successive shots, then to apply to the point clouds resulting from these shots transformations (called "poses" and each consisting of the combination of a rotation and a translation) corresponding to these displacements, in order to align the marks of each cloud of points.

For this purpose, data from:

of the inertial unit 15 of the scanner 4, which, as we have seen, provides at all times the magnetic north and the vertical and therefore behaves like a three-dimensional compass; the pair of textured stereoscopic images G'-D '. Calculation of rotation

The calculation of the rotation of the data from the inertial unit 15 is used.

Let there be a pair of successive point clouds (N _{i (} N _{i + 1} ), which made it possible to reconstruct two respective point clouds, and it is desired to calculate the relative rotation of the scanner 4 between N, and N _{i + 1} , in order to apply the opposite of this rotation to the second cloud to set it in rotation on the first.

The data of the inertial unit 1 5 make it possible to calculate: the absolute rotation matrix, denoted by R, the transformation matrix of the reference associated with the scanner 4 towards the terrestrial reference at the iteration i,

the absolute rotation matrix, denoted R _{i + 1} , the transformation matrix of the reference associated with the scanner 4 towards the terrestrial reference at the iteration i + 1.

The relative rotation matrix between the cloud Nj and the cloud N _{i + 1} , denoted R _j ⁺¹ , is equal to the product of the matrix R; by the inverse matrix of the matrix R _{i + 1} :

DE + 1 - D-1. D

The inverse of this rotation is applied to the points of the cloud N in order to set it in rotation on the cloud Nj.

Calculation of translation

The scanner 4 having been moved between the two successive quads corresponding to the clouds Nj and N _{i + 1} (for an acquisition frequency of 4 Hz, there is in practice a time shift of approximately 250 ms between the quads), the successive pairs of Structured stereoscopic images G _r Dj and G _{i + 1} -D _{i + 1} , on the basis of which the clouds have been reconstructed, are unusable only for the calculation of the translation between clouds.

Indeed, the projected pattern has moved on the surface 2 to be digitized, so that no reliable correlation can be made directly between the left images Gj and G _{i + 1} (respectively between the straight images Dj and D _{i + 1} ) pairs of images G _r Dj and G _{i + 1} -D _{i + 1} Also, for the computation of the translation, one exploits the pairs of stereoscopic images textured G ' _r D'i and G' _{i + 1} -D ' _{i + 1} of the quads.

We begin by selecting in G'j a set of points of interest (in practice between 200 and 300), by means of a filtering algorithm for detecting singularities.

Then we search in G ' _{i + 1 for} the corresponding points of G' ,, by means of a point tracking algorithm (for example the Lucas-Kanade point-tracking algorithm, see B. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," in Proceedings of Imaging Understanding Workshop, pp. 121-130, 1981) based on surface texture similarities of 2, acquired during the acquisition of textured images G'-D '(hence the usefulness of these images as part of the registration).

We thus have two sets of points:

Pi ^G 'points of interest of G'j matched in G' _{i + 1}

^Ρ ί + Ί points G ' _{i +} i matched at the points Pj ^G '

The points P _j ^G 'are then reinjected in the structured G-image, and for each of these points, an initial MEC is performed (or the results of the initial MEC already performed for these points are reused, see above). image Dj of the same stereoscopic pair G _r D ;, then one carries out a reconstruction to obtain a first cloud of points of reduced density, expressed in the reference associated with the cloud Nj and comprising a number of points lower than this one.

The same operation is carried out for the points P _j + Ί in the stereoscopic pair G _{i + 1} -D _{i + 1} so as to obtain a second cloud of points of reduced density, expressed in the reference associated with the cloud N _{i + 1} and deemed to be the evolution in the space of the first cloud of points of reduced density.

This reinjection supposes either to have first corrected the textured images G ' _j and G' _{i + 1} , or to rectify the points Pj ^G 'and Pj + Ί, so as to correspond at least locally the landmarks of the textured images with those structured images in which the points are reinjected.

We then apply the inverse of the rotation R | ⁺¹ to the second cloud of reduced density to rotate it in rotation on the first cloud of reduced density. Then, for each point Pj ^G 'of the first cloud of reduced density, the distance separating it (in space) from its corresponding Pg in the second cloud of reduced density is calculated. The average of these distances is extracted, which one decrees equal to the translation undergone by the cloud of points N _{i + 1} with respect to the first Nj.

Each pose, which includes a rotation and a translation, is thus complete. We then apply the inverse of this pose to all the points of the cloud N _{i + 1} to translate it in translation on the cloud Nj. FIG. 15 shows the clouds of FIG. 14, recalibrated.

The operations just described are repeated with the succession of scatter plots, which are thus recalibrated with respect to each other. Model construction

The construction of the numerical model of the surface 2 is carried out from the clouds of points resulting from the resetting.

Since many areas of the object have been captured several times (there are overlapping areas between the successive views), the final point cloud, obtained by aggregation of the points clouds resulting from the resetting, has a high density (see figure 16, which shows ten clouds of successive points recalibrated).

We can then perform from the points of the cloud a mesh, to obtain a second model draft consisting of contiguous triangles, then an interpolation calculation allowing from this second draft to reconstruct a true surface model based on functions continuous mathematics, compatible with CAD systems.

Claims

1. Use of a structured pattern in the construction of a three-dimensional digital model of a physical surface (2), in which the pattern is projected onto the surface (2), characterized in that the pattern is formed by the combination of a primary pattern (28) comprising an array of spaced rectilinear fringes (29) and an anisotropic secondary pattern (30) extending into the fringe spaces (29).

Use according to claim 1, wherein the primary pattern (28) is mono-oriented, the fringes (29) extending in a single direction.

3. Use according to claim 1, wherein the primary pattern (28) is bi-oriented, the fringes (29) extending in two directions.

4. Use according to claim 3, wherein the two directions are perpendicular, the fringes (29) defining a grid.

5. Use according to one of claims 1 to 4, wherein the fringes (29) are clear.

The use according to the claim, wherein the light fringes (29) are bordered by dark lines (32) between which the secondary pattern (30) extends.

7. Use according to one of claims 1 to 4, wherein the fringes (29) are dark.

8. Use according to claim 7, wherein the dark fringes (29) are lined with light lines (32) lined with dark lines (33).

9. Use according to one of claims 1 to 8, wherein the secondary pattern (30) comprises scabs (31) or pictograms (34).

10. A method of constructing a three-dimensional digital model of a physical surface (2) implementing a use according to one of claims 1 to 9, said method comprising the following operations:

Projecting the structured pattern on the surface (2), Acquisition and storage of at least one two-dimensional image of the illuminated surface,

Construction of the digital model from the image thus acquired.

1 1. System (1) for three-dimensional scanning comprising a device (10) for projecting a light pattern and a system (1 1) for stereovision, characterized in that the projection device (10) is arranged to project a light pattern formed by the combination of a primary pattern (28) comprising an array of spaced continuous rectilinear fringes (29), and an anisotropic secondary pattern (30) extending into the fringe spaces (28).