WO2001098762A1

WO2001098762A1 - Automatic inspection system with x-ray imaging

Info

Publication number: WO2001098762A1
Application number: PCT/US2000/040237
Authority: WO
Inventors: Christopher B. Jackson; David Fishbaine; Steven K. Case
Original assignee: Cyberoptics Corporation
Priority date: 2000-06-19
Filing date: 2000-06-19
Publication date: 2001-12-27
Also published as: AU2000264047A1

Abstract

A method and apparatus for using x-rays (622-624) and statistical appearance modeling (500) to assess article quality is provided. In one embodiment, a plurality of x-ray sources (608, 610) sequentially provide x-rays (622, 624) at differing incident angles such that components (618, 620) lying on the article (602) on different planes can be distinguished from one another. The statistical appearance modeling (SAM) (500) can then be applied to selected distinguished features (618, 620) on the article (602) for assessment. In another embodiment, a single x-ray source (700) is used to create a single x-ray image. Then, a priori knowledge of the article is applied to the single x-ray image to distinguish features (618, 620) on one plane of the article (602) from features on another plane. Statistical appearance modeling (500) is then applied to the so distinguished features (618, 620) in order to assess article quality.

Description

AUTOMATIC INSPECTION SYSTEM WITH X-RAY IMAGING

FIELD OF INVENTION

This invention relates to inspection equipment and to methods of inspection for articles of manufacture, with particular emphasis on the area of assembling components onto a printed circuit board, as in electronic assembly.

BACKGROUND OF THE INVENTION

The invention relates to inspection of articles of manufacture such as printed circuit boards. This is a main and extremely important application of the invention and will be mainly used for explaining the invention, but as will be indicated hereinafter the invention has wider application than the inspection of printed circuit boards. In the manufacture of printed circuit boards a rigid sheet of a synthetic composition is used as a substrate, and onto this substrate are provided copper conductors in a pattern established by a conventional photoresist and etching process.

By silk screening, a solder paste is applied to the copper conductors at locations where electrical components such as capacitors and resistors are to be applied, and where processor chips and the like, which usually have multiple terminals to be connected to the conductors, are to be applied. The components and terminals stick to the paste, and then the board and the applied components and chips are passed into an oven where the solder paste forms a secure solder connection with the component and chip terminals. These boards have two sides and the process is repeated in respect of the second side.

These boards can be extremely complicated and small, and any one board may have a vast number of components and consequently a vast number of electrical connections. Printed circuit boards are now produced in large quantities, and as they are expensive and are used in expensive equipment, it is important that they be produced accurately, with minimum wastage. Unfortunately, because of the manufacturing methods available, wastage (because of rejects), is still higher than in other industries.

Typical faults on printed circuit boards comprise inaccuracy of placement of components on the board, which might mean that the components are not correctly electrically connected in the board, and or electrical connections are not made, or that there is insufficient paste deposits leading to poor connections or too much paste leading to short circuits, and so on.

The industry experiences so much difficulty in providing reliable production that extensive inspection is necessary to check the accuracy of manufacture of these boards. The current inspection equipment is either too expensive, or too slow, or too inaccurate, or suffers from combinations of these disadvantages. There is therefore a huge demand for accurate inspection equipment, and if such equipment is faster, and/or less expensive than existing equipment, so much the better.

As to the equipment that is available and the methods used for inspection of printed circuit boards, the most basic method is a manual one. Inspecting operators simply used their vision to detect the faults indicated above. To assist, the operator has a mask which he or she puts over the board and examines the board, section by section, to check that the components are correctly positioned and are electrically connected, and there are no electrical shorts. Manual inspection, in fact, can be extremely accurate, but the trouble with it is that operators tend to be accurate at the start of a shift, but as time goes by, their accuracy drops. Also, it is a very slow method, and is labor intensive, which is expensive.

Another method is to scan the board with a telecentric camera, that is to say a camera with telecentric lenses, which views in parallel beam imaging, to. photograph a section of the board, and use software to analyze the image. The camera and board are relatively movable so that any required section of the board can be viewed. In one case, the camera is mounted on a stationary gantry, and the board can be moved past the camera. In another case, the board is stationary and the camera is movable.

In using a_. telecentric camera, problems due to parallax can be overcome, but there is a disadvantage with this arrangement in that the camera can only view the image in two directions, which is not a problem as long as the board is exactly flat, but although normally these boards are nominally fiat, exact flatness is difficult, if not impossible to achieve for various reasons. Steps are taken to keep the board flat to the degree necessary to make the method accurate, but this is not easy, and in fact the board is sometimes clamped in an effort to keep it sufficiently flat to make the results acceptable. It should be borne in mind at this time that the dimensions of the degree of out of flatness (i.e. warpage) of the board which are involved are measured in microns, but even warpage of the board to this minute degree can have a serious effect on the accuracy of the inspection.

Previous automated methods have also suffered from the inability of gray scale models to accurately match a component under test. Such models were unable to compensate for variations in shape, colour, lighting and the like, which resulted in numerous identifications of properly placed components as having been improperly placed (i.e. "false calls"). Another disadvantage of the above method is that the equipment is expensive and involves an expensive, precision X-Y gantry. This method also tends to be slow, and requires precision mechanical registration of each section of the board, which is viewed by the camera. In the present invention we have provided inspection equipment which gives greater accuracy than the known methods, provides a low "false call" rate and does not require the use of expensive equipment.

SUMMARY OF THE INVENTION

In one embodiment of the present invention there is provided inspection equipment comprising means creating at least two x-ray beams, which pass through the surface or object to be inspected. The beams are arranged to pass through the surface or object at different angles, such that components on different planes can be distinguished from one another in the x-ray images.

In another embodiment of the present invention, there is provided inspection equipment comprising means creating a single x-ray viewing beam, and means for receiving some form of a priori knowledge of object position which knowledge is correlated with the x-ray image to distinguish components on different planes from one another.

The present invention preferably employs a statistical model of a component or of a reference point, called a SAM model. The SAM model incorporates variability in shape, lighting and the like of the component and its immediate vicinity. The SAM models are preferably used during inspection of a printed circuit board, where a search area for a specific component is defined and a SAM model corresponding to the component which is expected to be found within the search area is applied to points within the search area. At each point within the search area, the SAM model is reconstructed to take into account the specific variations of that portion of the search area, and the reconstructed SAM model applied to each of the points within the search area. At each point within the search area, a measure of fit is computed, and the point at which the measure of fit is optimized is used as the best-fit point representative of the actual location of the component on the board.

It can be seen therefore that the invention has potential in the particular field of printed circuit board inspection. It is to be mentioned however, that the invention has other uses in and outside of the printed circuit board industry.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will now be described by way of example, with reference to the accompanying drawings, wherein:

Fig. 1A shows a section of printed circuit board, which is to be inspected by the equipment according to the invention;

Fig. IB is an enlarged view showing how vertical distortion of the board also leads to lateral displacement of component;

Fig. 2 is a diagrammatic side elevation of equipment according to a first embodiment of the invention;

Fig. 3 is an enlarged side view showing the optical system of the embodiment of the invention shown in Fig. 2;

Fig. 4 is an enlarged perspective view showing the optical effect which applies when the printed circuit board is distorted;

Fig. 5 shows the spacing of the images of a reference point as seen by the two _* cameras in Fig 4;

. . Fig..6 is a view similar to Fig. 2 showing an alternative embodiment of the present invention;

Fig. 7 is a flow chart of the method of the invention;

Fig. 8 is a schematic representation of a SAM model; and

Fig. 9 is an overall block diagram of the system of the present invention.

Fig. 10 is an overall block diagram of a system in accordance with an embodiment of the present invention.

Figs. 11 A through 1 IE illustrate x-ray imaging in a manner that distinguishes components on one plane from that of another.

Figs. 12A through 12C illustrate a method of analyzing x-ray images using a priori knowledge to distinguish components on one plane from that of another.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Although much of the description of embodiments of the invention is related to inspection systems using cameras, and the visible spectrum, additional embodiments are described with respect to Figs. 10-12.

Referring to Fig.1, a printed circuit board to be inspected is indicated by reference numeral 10, and it is shown in this example as having thereon a processing chip 12, components 14 and printed circuit conductor wires 16. It is to be pointed out that base components may be extremely small, and very tightly packed on the board. It is usual to have such items attached to both sides of the board. The objective of the present invention is to provide an inspection means for the board whereby the correct positioning of the various items on the board can be checked. This is done by scanning by means of closed circuit television cameras as will be explained. Referring momentarily to Figs. 2 and 3, the equipment for performing the scanning is shown diagramatically in these figures, and comprises a pair of conveyor belts 18 and 20 which are spaced by a distance to enable the board 10 to be supported therebetween. The spacing between the conveyors 18 and 20 can be adjusted to accommodate boards of different sizes.

The inspection cameras are located vertically above the board 10, and they are arranged in pairs such as are indicated by reference numerals 22 and 24 in Fig. 2. There is a bank of camera pairs A, B, C and D and so on arranged in a direction transverse to the direction, indicated by arrow 26 in which the board is transported by the conveyors 18 and 20. The conveyors are arranged to operate in a stepping fashion so that the board 10 steps past the fields of view of the cameras so as to be photographed progressively in strips which lie in the direction of arrow 26, and are arranged in parallel and side by side in a direction at right angles to direction 26 and indicated by arrow 28 in Fig. 3. Thereby, the cameras are arranged to photograph all of the board, and the photograph of the board can be reconstructed on a display screen 30 of electronic computing equipment 32 to which the outputs of the cameras are directed.

Pre-loaded into the computing equipment 32, is a model of the printed circuit board so that the computing equipment can compare what is viewed by the cameras, and the model details, to indicate whether or not the board is of satisfactory manufactured quality or has to be rejected. A comparison will be mainly to ensure that the items on the board are correctly and exactly positioned, but the comparison can also check items such excess solder or shortage of solder, which faults respectively could mean short-circuiting or imperfect electrical connection. The process is detailed below.

Reverting to Fig. 1 A, the rectangular and overlapping areas II and 12 respectively represent the images as seen by the cameras 22 and 24 at the first step in the inspection process. Fig. 2 illustrates that these images are generated by divergent beams 36 and 38 of which the beams axes lie at an angle X to one another. Such angle may be in the order of 3 degrees, but the net effect is that the cameras 22 and 24 look at the board in a stereovision manner and by arranging degrees that the images II and 12 overlap, accurate re-creation of a mosaic image of the board on the screen 30 can be achieved by "stitching" the images II and 12 when they are processed electronically, but retaining the stereo nature of the information in the mosaic image. This is done by pre-programming the computing equipment 32 with information concerning reference points such as the vias 40 and 42 which exist on the board 10 and relating their position to the fiducials, such as 44, which are also on the board 10.

If it is considered that the images II and 12 overlap lengthwise of the board, the images from the side-by-side cameras also overlap sidewise as shown by image 13, and can be stitched in this direction also, so that a complete mosaic picture of the board 10 can be built up by the electronics 32 and a comparison with the model input 34 can be made accurately. Any arrangement of cameras in lengthwise and sidewise directions may be adopted to provide the stitching of image facility.

. . By stereoscopically viewing the board using two cameras 22 and 24 in each pair, compensation can be made for any out of flatness (i.e. warpage) of board 10. Such out of flatness or distortion arises for the reasons given herein, and by way of explanation, Fig. 4 shows the board 10 in its actual distorted shape, whereas reference 10A indicates the optimum flat configuration of the board 10 (which rarely exists in practice).

If one now examines the beam paths 36 and 38, and considers, for example, the via reference point 40 on the board 10, it would be seen that in actual fact the point 40 is displaced downwards from where the computing system would expect such point to be located (in the plane 10A).

In Fig. 5, the outputs from the cameras 22 and 24 therefore show two images 40A and 40B as being displaced one relative to the other in that the cameras 22 and 24 would be looking for the images 40 A and 40B to be in the plane 10 if the board 10 were at the correct distance from the cameras.

This spacing, S, between the images 40A and 40B is calculated which in turn gives an indication of the extent of displacement of the actual via 40, and so a height compensation factor can be provided. If this height compensation factor is not included in the comparison, then spurious results can be provided and this is demonstrated by reference to Fig. IB. Fig. IB shows an enlarged elevation at position 50 where the board ideally would be expected to be, and a component is shown at 52. The component has a width 54 and the inspection electronics would be looking for component 52 to be in the position shown and to exhibit the width 54. However, if the board is distorted as shown at 56, the component 52 will in fact not only be deflected downwards, but will also be displaced laterally by distance D, and if the electronics does not compensate the board profile as a result of the distortion, what the electronics will see in looking at position 4 will only be part of the component 52 and it may conclude that component 4 is therefore "out of position". When the height or warpage compensation and profile shape, however, is taken into account, the electronics will calculate that there has been distortion of the board downwards and lateral movement of the component 52 and therefore will not reject component 52 but rather will accept it in positioa 58. A tile-by-tile, piecewise linear fit is preferred, but other methods are acceptable for use with the present invention.

. . The stereoscopic inspection of the board; therefore, provides improved performance of the equipment, without requiring expensive devices as are employed in the known methods.

In the embodiment described, pairs of relatively inexpensive and relatively poor resolution CCTN cameras are used, and there is no need to make any attempt to mechanically flatten the board during inspection. A typical camera resolution is 760 x 575 pixels. In another embodiment again there is no need to mechanically flatten the board, but a single camera can be used in place of each pair, the single camera being of a higher resolution quality but arranging to have its beam split to provide the two stereo images at each step. Such an arrangement is illustrated in Fig. 6. The method of operation is otherwise similar to what has already been described.

In Fig. 6, the high resolution camera 60 has a viewing beam 62 which impinges upon a beam splitter prism 64 which splits the beam into two identical but oppositely directed beams 66 and 68. These stereo beams 66 and 68, respectively, impinge upon mirrors 70 and 72, resulting in the provision of incident stereo beams 74 and 76 which view the board 10 optically in the identical manner as do the beams 36 and 38. The advantage of this arrangement is that both beams 74 and 76 are generated by the same camera, and the registration of the stereo image tiles and the processing of the information are slightly similar.

It has been mentioned that the images are stitched together by viewing reference points on the board. These reference points are real, but the system can also be made to work with virtual reference points provided, for example, by spots of light, and Fig. 6 shows one possible arrangement wherein pencil reference beams 78 and 80 travel through the same optical system as the camera beam but are set to impinge upon a common spot 82 to form a reference point. If the board 10 is distorted or warped as described in relation to Fig. 4, the viewing of that reference spot will produce two images in a manner similar to that shown in Fig. 5. The present invention provides equipment and method enabling the accurate high speed inspection of surfaces and objects, such as printed circuit boards, without the need for adopting expensive gantry XY devices, or telecentric cameras or expensive mounting device for clamping the board flat.

The invention of course has wider application as indicated herein, and in one example stereo viewing can be used for viewing other spots to provide an indication of volume of the solder in that spot. Indeed, the concept of viewing image regions II and 12 and relating these to reference points such as 40 and 42 followed by the stitching of the images to provide an accurate representation constitutes a novel aspect, even if the viewing beams are arranged in parallel as long as they diverge and overlap.

The present invention is also able to be practiced with electromagnetic radiation of varying wavelengths. A x-ray source would replace the cameras and appropriate x-ray receivers would be employed to record an image of the article which is being viewed. Additional hardware in the x-ray embodiment would perform the same functions as disclosed herein. In fact, it is apparent that even a line scan image of the article, where an image of the article is built up line by line while the article and a linear detector moves relative to each other, is contemplated under the method and apparatus of the present invention. In a line scan embodiment of the present invention, a series of collected outputs from a linear detector would be necessary to provide a single image of the article, and another series of collected outputs from the linear detector would be necessary to construct the mosaic image.

Fig. 10 is an exemplary system block diagram of an x-ray embodiment of the present invention. The block diagram of Fig. 10 is similar to that of Fig. 8 (which will be described later in the specification) and similar components are numbered similarly. The primary distinction between the system shown in Fig. 10 and that shown in Fig. 8 is that system 616 provides x-ray sources 608 and 610 where the system shown in Fig. 8 provides cameras. Additionally, system 616 includes an x-ray detector 614 that is adapted to detect x-rays emanating from either of sources 608 and 610, passing through board 602 and falling upon detector 614. One example of an x-ray detector is comprised of a means for converting x-rays to visible light, such as a phosphor; a means for intensifying the resulting visible 2D image to levels detectable by a standard camera such as an image intensifier; and a camera such as a lens coupled to a charge coupled device (CCD) camera. Those skilled in the art will recognize that a moving linescan sensor can also be used in an x-ray detector. Unlike other embodiments of the present invention, x-ray embodiments are generally unconcerned with feature height calculation^' relative to other portions of the board. Instead, such embodiments focus upon applying statistical modeling (which will be described later in the specification) to x-ray images. System 616 is similar to a system shown in Fig. 8 in_. that multiple x-ray sources are provided while multiple cameras are provided in the system shown in Fig. 8. The provision of multiple x-ray sources and the sequential operation of x-ray sources 608 and 610 allows system 616 to distinguish between components which lie on one plane from other components which lie-on another plane.

Fig. 11A illustrates board 602 having a pair of features, or targets, 618 and 620 disposed on opposite sides from one another. X-rays emanating from source 608 are illustrated diagrammatically at arrow 622 as passing through board 602 substantially perpendicularly to board 602. X-rays 622 pass through targets 618, 620 and board 602, and fall upon detector 614 which provides an image related to the integrated density of x-rays at a given point. Fig. 1 IB is an exemplary drawing showing targets 618 and 620 as forming a cross pattern. Viewing solely the cross pattern of Fig. 1 IB, it is impossible to distinguish which component lies on top of which, or even if the components themselves lie on different planes. Referring to Fig. 11C, x- rays from source 610 are illustrated at arrow 624 as passing through targets 618, 620 and board 622 at an angle relative to x-rays 622. Thus, the image formed when using x-ray source 610 (Fig. 1 ID) illustrates that components lying in different planes will shift to varying degrees depending on the vertical distance to which they are separated and the variation in the angle of incidence. Thus, the image of component 620 lying below board 602 appears in Fig. 1 ID to be shifted to the right when compared to Fig. 1 IB. Such shifting indicates that target 620 lies on a different plane than that of target 618. Fig. 1 IE illustrates the distinction between Figs. 1 IB and 1 ID by showing images 618-1 and 618-2, which correspond to x-ray images acquired from sources 608 and 610, respectively. Those skilled in the art will notice that target 620 does not appear to have moved, or has moved to a lesser degree, than target 618. Preferably, x-ray sources 608 and 610 are energized in sequence such that the acquisition of images 1 IB and 1 ID are temporally spaced. In this manner, the multiple x-ray source embodiment of the present invention is able to distinguish between components lying on different planes using a single detector. Once such distinction is made, images corresponding to components on a given plane can be provided to the SAM module, as will be described in further detail later in the Specification. ^~

Figs. 12A through 12C illustrate a method of distinguishing components on one plane from that of another and applying the SAM model. Specifically, Fig. 12A illustrates x-ray source 700, which like x-ray sources 608 and 610, can be any suitable x-ray source. X-rays emanating from source 700 pass through component 618, board 602, and component 620, and fall upon detector 614 to form the image shown in Fig. 12B. Since detector 614 detects essentially the integrated density of x-rays impacting a given area, the. cross hatched area representing the intersection of components 618 and 620 will be darker than other areas. Since a singular x-ray source, 700, is used to provide the image illustrated in Fig. 12B, it is necessary to use external information, a priori knowledge, to distinguish between components 618 and 620. Examples of such a priori knowledge include information transmitted from a computer-aided- design (CAD) system, user-provided information, or any other suitable information. Using such a priori information, the system is able to distinguish components 618 and 620 from one another -l ias illustrated in Fig. 12C. In one embodiment, the overlapping, or occluded region, is essentially subtracted from the model of each component to provide new component models 618-new and 620-new. Additionally, the occluded area 630 is also separated. Thus, three models 618-new, 620-new and 630 are provided to the SAM module for testing. Further, when using this embodiment, the SAM software is generally trained using examples of 618-new, 620-new and 630.

The remainder of the description of the present invention will now be completed with respect to the multiple camera embodiments of the present invention. Further, description of the statistical appearance modeling (SAM) will also be provided with respect to the multiple camera embodiment. However, as described above, the use of such modeling with transmissive imaging, such as x-ray imaging, is expressly contemplated.

The following discussion refers to Fig. 7 and provides a more detailed understanding of how the statistical appearance modeling process is used with the mosaic image in the preferred embodiment of the present invention. Precise and accurate measurement of component position relies on establishing the shape of the surface (substrate) upon which the component is mounted which is required to accurately take account of the path length distance between points on a curved surface and to overcome the errors arising from the use on non- telecentric optics in the imaging device in the preferred embodiment-"

At box 302, the tiled images are acquired by the camera pairs. At box 304, the positions of reference points visible in both images of the stereo pairs, are established. The discrepancy in location between the measured positions of reference points in each of the stereo images, and measurements describing the positions of all the cameras in the system obtained during a calibration process, allows the system to establish the distance between the reference points and the cameras, which imaged the reference points. This distance is used to establish the height of the reference point above a reference plane, which is established during system calibration. Due to limitations in processing power and the need for the system to work within the cycle time of the manufacturing process producing the articles, it is necessary to restrict the number of points at which this height measurement is made, and so a sparsely populated height map of the entire surface of the substrate of the article is created.

In order to produce a complete description of the three-dimensional shape of the substrate it is necessary to interpolate between the points in the height map. Different mathematical models identified below can be imposed upon the height map data.

1. "Sag" where the substrate is assumed to have drooped between the rails of the conveyor system. The height of the surface of the substrate is defined by: ^z(^χ, y) ^{~ aχl'} +bx + c so that the surface of the substrate is exclusively quadratic with respect to x. The parameters {a, b, c} are determined from n measured reference points (xi, yi, z;) (1 < i < n) in a least squares fashion, which fits this model to the height map.

2. "Thickness/tilt" where the substrate is thick enough to be stiff and therefore relatively flat and the effect to compensate for is due only to the thickness and tilt of a plane, given by: ax + by + cz + d = 0

The parameters (a, b, c, d} are, again, determined from the measured reference points (x;, y;, z;) (1 < i < n) in a least squares fashion, which fits this model to the height map.

3. "Warp" where the substrate is considered to be warped in many directions simultaneously. Here, the (x;, y;, z;) (1 < i < n) measurements of the reference points are used to construct an interpolating surface known as a "thin plate spline". The thin plate spline uses a model of pliable material which is "bent" to match exactly the heights z; at the point (xi, yi) and does so such that the amount of conceptual energy required to bend the plate is minimized. The result is a continuous surface, z=f(x,y), which interpolates the measured reference points, (i.e. z;=f(xi, yi)) is fitted to the height map.

At box 306, the positions of reference points are located and a compensated coordinate system for the article is computed. The reference points must appear in the row overlaps defined by the image acquisition process, or the row overlaps must be set from the positions of available reference points. The location of the reference points can be determined in a number of ways but employing a SAM model of the reference points is preferred. One way is to use an example image, where a user defines the coordinates of a suitable candidate reference points. A second way is to use design information for the article (e.g. CAD, Gerber) which defines the position of suitable reference points. A third way is to analyze the image of the article using alternative image processing and analysis algorithms (e.g. Hough Transform) to determine the positions of objects of particular characteristic shapes.

It is advantageous that a 'golden' article is not required to define the position of the reference points because it is sometimes not possible to obtain a perfect article. However, should design information be available for the article, greater precision and accuracy can gained by comparing the actual positions of the reference points with their expected positions.

The stereo overlapping tiles are stitched together to form a mosaic image where it is necessary to move the article or the imaging device array, in order to capture image data for larger articles. An image registration process takes place simultaneously to the height mapping process described above. In this way, a mosaic image of many rows of stereo pairs of image tiles is built up to form a height and movement corrected mosaic image. The entire process by which the mosaic image is produced is called "stitching".

Having established a rigorous mathematical image of the topology of the substrate, and having accounted for the errors introduced into the image acquisition process by virtue of imprecise mechanics, accurate measurements of location of certain features (e.g., components) can now be made.

Components mounted on the substrate of the article are located using statistical appearance models, as shown in box 308 The use of statistical appearance modeling (SAM) results in a very reliable assessment of component presence and a very accurate .and repeatable assessment of component position. The objective is to measure component position with respect to the coordinate system of the article, typically based on the commonly found fiducials marks on the printed circuit board. The inspection procedure involves first detecting and measuring the position of these fiducials and then producing a coordinate system from their measured positions (box 306). The detection and position measurement of the fiducials is achieved using SAM, although other image analysis methods are within the scope of the present invention. The substrate suffers from errors resulting from the positioning of the article under the imaging device array and the normal tolerances associated with the manufacturing process (these errors result in distortions in the printed/etched pattern on the substrate).

Having now established a compensated coordinate system of the article in the mosaic image at box 306, all components which are to be inspected are detected and their positions measured at boxes 308-312. For each kind of component there exists a corresponding SAM model and a list of acceptance criteria that must be satisfied in order for an inspection for a component to be deemed to be passed. The criteria include, but are not limited to, tolerance in x, y, angle of skew and a measure of how likely the component in question is described by its associated SAM model, expressed as a probability of fit.

The measurement of the position of the fiducials and the components must take account of their height in relation to the reference plane, so that their position on the substrate is accurately established. In order for a SAM model to be used to detect and measure the position of a component, the model corresponding to the component or feature is applied in the vicinity of where the component or feature is to be expected at a range of angles in the mosaic image, as indicated in box 308. The location process, at box 310, evaluates the correspondence between the SAM model and the feature at all points within a defined search area and establishes the best-fit points (one from each stereovision set of information in the mosaic image) at which the SAM best describes the data in the search area. This process returns a best-fit x,y coordinate, an angle of skew and the probability that the SAM model has properly described the component. The discrepancy between the x, y coordinates of the best-fit points allows the distance of the surface of the component from the imaging devices to be computed, which as in the stitching process allows a height measurement to be computed with respect to the reference plane. This height measurement is then used to compute a height compensated coordinate of the projection of the position of the surface of the component onto the substrate at box 312. In box 314, the corrected x, y, skew angle and probability measures are then tested against the acceptance criteria for this style of component, and the inspection for this component passes if these measures are within the acceptance criteria specified. However, if the inspection indicates that the component is outside of the acceptance criteria, then the board can be either discarded, scheduled for re- work or appropriate warning given to an operator (box 316).

In another embodiment of the invention, the compensated location for a component can be computed from the best-fit location obtained from one of the stereo images in which the component appears, in combination with a height estimate of the component obtained from CAD and other component design information.

An overall block diagram of the system is shown in Fig. 8. A printed circuit board 402 rests on a conveyor belt 400. Conveyor belt 400 is actuated by motor and motor drive 404, which operates under instructions from a computer 406. Computer 406 is multi-processor computer of standard design and includes circuits for acquiring and digitizing images from two banks of video cameras 408, 410 and a man machine interface 412 consisting of keyboard, mouse and screen. Computer 406 controls the movement of the board 402 with a precision of +/- 0.5 mm, in such a way to position the board for acquisition of the images. Computer 406 also directs acquisition of partial images of the board 402 from banks 408,410. An illuminator 414 provides lighting for image acquisition.

The SAM model 500 shown in Fig. 9 allows proper account to be taken of legitimate variability in shape, color, lighting, surface patterns and the like. This results in a reliable determination as to whether the feature is present, absent, of the correct type or an accurate assessment as to where it is placed on the article. SAM model 500 describes the value of every point of intensity (pixel) in an image of an object, it also describes how each point of intensity can vary in value with respect to all other points of intensity. There are two distinct stages in the use of SAM. The first involves the construction of a SAM model by analysis of images of a variety of features of the same type, as shown at 502 - 508, where it is clear that any number of examples may be used in creating a SAM model. The second involves using the SAM model to detect and locate the feature it describes, in an image where such a feature is detected, but which may or may not be present, as detailed above with respect to fiducial marks and to components.

A SAM model is constructed by collecting together example images of the feature of interest, pre-processing the image data from each example and turning each into a one- dimensional vector of pixel values. Given all the vectors, one for each example, a mean vector X_mea_n is produced representing the average appearance of the feature, -as given in Equation 1:

1 " N & ^J

where X_j is an nxl vector of pixel values, n is the number of pixels in each vector and N is the number of example images.

Each example can now be represented as a zero centered vector expressing its variance from the mean, called a mean adjusted vector, given by Equation 2:

X ⁼ X, — Xmean

A compact description of how each example varies from the mean can now be generated by constructing a co-variance matrix, S, using all the mean adjusted vectors from the previous stage, as stated in Equation 3:

where x' is an nxl matrix and x'i^T is an lxn matrix, so that the product of the two matrices is an nxn symmetric matrix.

The eigensystem of the co-variance matrix, S, is given by Equation 4;

where λ_k is the k^Λ eigen value of the co-variance matrix, pk are the orthogonal eigenvectors and

{1 7 = k 0 j ≠ k

The eigensystem yields an orthogonal system of eigenvectors which represent the particular ways, called modes of variation, in which the pixels of images of the feature vary in shape, colour, lighting, surface patterns and the like, the eigenvalues representing the magnitude of each of the modes of variation. Only the most significant modes of variation are retained, so as to reduce the affects of random noise, which exists in the image data of each example. Only the more significant modes of variation, which explain typically 95% of the variability in the examples, are used, as shown in Equation 5 :

where P is an nxm matrix, P^τ is an mxn matrix, and PP^T=1. A SAM model 500 has now been constructed which allows the reconstruction of any example, seen or unseen, whose appearance (pixel intensity or gray values) lies within the bounds dictated by the magnitudes of the modes of variation.

Next, the SAM model is used to reconstruct itself to more closely conform to the component under test as detailed at box 308 in Fig. 7. A newly reconstructed model is computed at a range of angles for each point in the vicinity of the component under test (i.e. the search area). The best-fit reconstructed model occurs at the best-fit location. Equation 6 shows the general form of reconstruction: x = x mean +Pb

Solving Equation 6 for b, the parameter vector, yields Equation 7: b = P^Υ(x-x_mean)

Once the b vector values are computed, an overall quality of fit measure of how well the reconstructed SAM model fits the component under test is computed, as a function of the Manhanobolis distance and residual error. In particular, the Mahanobilis distance provides a normalized measure of how well the SAM model describes the candidate and is given by Equation 8:

J f αnhαl

where m is the number of example images used to construct the SAM model.

The other measure of quality of fit evaluates the residual error between the component under test and the reconstructed SAM model, as given by Equation 9:

where n is the number of pixels in the model, r- is the error of the j pixel between the relevant search area pixels and the reconstructed approximation of the SAM model, and λ_r^ is the variance of η over the examples stored previously.

The overall "quality of fit" is derived in Equation 10: .

J fit J mαnhob ' J resid

The value f _& is transformed into a "probability of fit" representative of the probability that the model describes the component under test by assuming that the f a values from a population of examples follow a chi-squared distribution. The probability of fit value, Pf, is preferably computed using the incomplete gamma function, given by:

The use of SAM models allows a very reliable decision to be made as to whether or not a feature is present in an image, as the reconstruction can smoothly interpolate between all the examples used in constructing the model to produce an appearance of the feature which, although not seen before as an example, has been varied from the mean. The newly generated appearance of the feature, however, is consistent with the variability captured during the model construction phase. Also, the best-fit location returned from the location process is very accurate and repeatable because the variability in the feature is properly described by the model.

When making a comparison between a feature and a corresponding SAM model there will be two sources of error; one will be caused by the difference between the feature and the model, the other will be caused by random noise resulting from the image acquisition process and other factors. The location process and the SAM model which the system uses remove the first source of error leaving only the second source of error, which being at a much lower level, results in improvements in measurement performance and detection reliability over systems which do not employ this method.

Although the present invention has been described with reference to preferred embodiments, workers skilled in the art will recognize that changes can be made in form and detail without departing from the spirit and scope of the invention. For instance, insubstantial changes in the logical flow of the steps of the present invention which realize the substantially the same function will be recognized as within the scope of the present invention. Additionally, the optics discussed for use in the present invention should not be used to limit the invention, and it will be recognized that other schemes from which a height map of a surface of interest may be computed, and the location of features of interest on the surface compensated by the height are within the scope of the present invention. Image analysis methods other than the ones presented in this disclosure are also within the scope of the present invention, as long as they are able to be reconstructed to more accurately comport with an image under test. Finally, the present invention is not limited to use in the area of electronics assembly inspection machines, but may be used in other inspection and manufacturing systems, which must accurately identify the presence or absence of a certain item on a surface with variations in its planarity.

Claims

WHAT IS CLAIMED IS:

1. A method of assessing a quality of an article, the method comprising: collecting a plurality of x-ray images of the article; analyzing the images to distinguish between features on different planes; and applying a statistical model corresponding to a selected feature on the article to at least one of the images to provide the assessment of article quality.

2. The method of claim 1, wherein the images are x-ray images that at least partially overlap.

3. The method of claim 2, and further comprising: providing a plurality of x-ray sources adapted to provide x-ray beams at different angles of incidence from one another; and providing a detector adapted to provide the x-ray images based upon x-ray intensity emanating through the article.

4. The method of claims 3 wherein the x-ray sources are energized sequentially to provide the images.

5. The method of claim 4 wherein the detector remains unmoved during x-ray source sequencing.

6. The method of claim 1, and further comprising: providing a single x-ray source adapted to generate a plurality of temporally-spaced x-ray beams having different angles of incidence relative to the article; and providing a detector adapted to provide the x-ray images based upon x-ray intensity emanating through the article.

7. A method of assessing a quality of an article, the method comprising: collecting a single x-ray image of the article; collecting a priori knowledge of the article; applying the a priori knowledge to the single x-ray image to distinguish at least one feature on the article; and computing a characteristic of the feature using statistical appearance modeling; and determining the quality of the article based upon the statistical appearance modeling of the feature.

8. The method of claim 7 wherein the article includes multiple features disposed on different planes with respect to one another, and overlapping one another with respect to x-rays passing therethrough.

9. The method of claim 7 wherein the a priori knowledge is provided from a computer-aided-design (CAD) package.

10. The method of claim 7 wherein applying the a priori knowledge to the single x- ray image provides a plurality of image examples to which the statistical appearance modeling is applied.

11. The method of claim 10 wherein at least one of the examples represents an area of intersection between a plurality of features on the article.

12. A system for assessing a quality of an article, the system comprising: a first x-ray source disposed to provide x-rays at a first angle relative to the article; a second x-ray source adapted to provide x-rays at a second angle relative to the article, wherein the second angle is different than the first angle; a detector adapted to provide x-ray images based upon x-rays falling thereon; and a processor adapted to apply statistical appearance modeling to an x-ray image of at least one feature on the article.

13. The system of claim 12, wherein one of the x-ray sources is adapted to provide x- rays substantially perpendicular to a plane of the article.