CA2061865C - Methods and apparatus for optically determining the acceptability of products - Google Patents
Methods and apparatus for optically determining the acceptability of products Download PDFInfo
- Publication number
- CA2061865C CA2061865C CA002061865A CA2061865A CA2061865C CA 2061865 C CA2061865 C CA 2061865C CA 002061865 A CA002061865 A CA 002061865A CA 2061865 A CA2061865 A CA 2061865A CA 2061865 C CA2061865 C CA 2061865C
- Authority
- CA
- Canada
- Prior art keywords
- image
- product
- images
- value
- filter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
- B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
- B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
- B07C5/04—Sorting according to size
- B07C5/10—Sorting according to size measured by light-responsive means
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C3/00—Registering or indicating the condition or the working of machines or other apparatus, other than vehicles
- G07C3/14—Quality control systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Abstract
The acceptability of the appearance of objects such as consumer products is determined by forming an initial discriminant function as a composite of a relatively small number of images which are known to be acceptable. This initial discriminant function is then used to gather statistical information about how a first relatively large number of images compares to the initial discriminant function. Thereafter, this statistical information is.used to select or aid in the selection of acceptable images from a second relatively large number of images, and the images selected as acceptable are used to refine the discriminant function. The refined discriminant function is then used (e.g., during actual product inspection) to determine which objects have an acceptable appearance and which do not.
Description
2fl~~.~~
METHODS AND APPARATUS FOR
OPTICALLY DETE'RMINTNG THE A~CEP'~ABILITY OF PRODUCTS
Background of the Invention This invention relates to product inspection methods and apparatus, and more particularly to methods and apparatus for optically determining whether or not a product has an acceptable appearance.
For many products such as consumer goods like 1o packaged foods, beverages,~cleaning products, health and beauty aids, cigarettes, cigars, etc., it is very important that the external appearance of the product or its packaging be uniform and defect-free. Yet these products are typically produced in such large.
quantities and at such high speeds that some form of automated optical inspection is practically essential.
It is highly desirable for an optical inspection system to be able to test all or substantially all parts of ~,:..
the product image so that defects of any kind occurring anywhere in the image can be detected. At the same time the inspection system should not reject products having minor but acceptable deviations from the ideal product.
For even a relatively simple product image such as a cigarette pack, an inspection system must be initially supplied with a tremendous amount of information in order to enable the system to inspect all or substantially all portions of tY.se image with the 2~~~.~~5 sophistication required to discriminate between acceptable products (i.e., products having the ideal appearance or an appearance acceptably close to the ideal) and unacceptable products which should be rejected because of defects in appearance or appearance which is not sufficiently.close to the ideal.
Identifying and entering this information into the inspection system apparatus typically requires a very high level of skill and/or large amounts of operator 1o time. 'Moreover, this data identification and entry task must be repeated each time a new or even slightly different product is to be inspected.
It is therefore an object of this invention to improve and simplify optical inspection systems.
It is a more particular object of this invention to provide optical inspection systems which greatly reduce the level of operator skill and amount of operator time required to set up the.system to inspect a new or different product.
Summary of the Invention These and other objects of the invention are accomplished in accordance with the principles of the invention by providing optical inspection systems which form an initial discriminant function or "filter" from a composite of a relatively small number of '°first phase'° images which the operator of the system determines. to be acceptable images. A relatively simple technique (e.g., a logical OR function) is preferably used to form this composite. The ability to produce the initial discriminant function quickly using such a simple combination of a small number of images facilitates rapid "start-up" of the systems Thereafter the system~uses the initial discriminant function to process a relatively large number of representative '°second phase" images in order to compute statistical information about the images in relation to the initial discriminant function. In particular, the system uses the initial discriminant function to compute a processed value for each second phase image. These processed values will typically have an approximately normal or Gaussian distribution. The upper and lower limits of a central portion of this distribution containing a first statistically large number of the processed values are identified as first threshold values'. The upper and. lower limits of a central portion of this distribution containing a second statistically even larger number"of the processed values are identified as second threshold values.
In a subsequent third phase of the operation of the system, the first and second threshold values are used in the processing of a relatively large number of "third phase" images. In particular, the system uses a discriminant function (initially the.above-mentioned initial discriminant function) to compute a processed value for each successive third phase image.
If this processed value for a given third phase image is between the first threshold values, that third phase image is automatically used to refine (e.g., using a Widrow-Hoff-type adaptive training process) the discriminant function for subsequent use., If the processed value for a third phase image is~not between the second threshold values, that third phase image is automatically discarded. As a third possibility, if the processed value for a third phase image is not between the first threshold values but fs between the second threshold values, the operator of the~system is given the choice as to whether or not that image should be discarded (i.e., because the image looks unacceptable) or used to refine the discriminant YW
~ $ ~.
function for subsequent use (i.e., because. the image looks acceptable). w When the third phase is completed, the system is ready for actual product inspection using the refined discriminant function and the first threshold values. In actual product inspection, the system uses the refined discriminant function to compute a processed value for each product image. I:~ the processed value for a product image is between the first threshold values, the product is aoc~spted as having an acceptable appearance. If the processed value for a product image is not._betwsen the first threshold values, the product is rejected as having an unacceptable appearance.
The system greatly reduces the level of operator skill and amount of operator time required to set the system up for a product inspection task. The operator is only required to identify ~ relatively small number of acceptable images (e.g., 25) during the first phase. The initial discriminant function is then computed~automatically using a simple and rapid technique such as a logical t7R of the small number of first phase images. The entire second phase may also be automatic. And during the third phase, the operator is only required to decide on the acceptability of the relatively small number of images whose processed values fall outside the first threshold values but between the second threshold values.
Further features of the invention, its nature and various advantages will be more apparent from the accompanying drawings and the following detailed description of the preferred embodiments.
_5_ $r~ Description of the 3~rawi.nas FIG. 1 is a simplified schematic'block diagram of an illustrative embadiment of an optical product inspection apparatus constructed in accordance with the principles of this invention.
FIGS. 2a and 2b (referred to collectively as FIG. 2) are a flow chart of an illustrative optical product inspection method in accordance with this invention.
1 FIGS. 3a-3c (referred to collectively as FIG. 3) are a flow chart of an illustrative, more detailed embodiment of one of the steps shown in FIG. 2.
FIGS. 4a and 4b (referred to collectively as FIG. 4) are a flow chart of an illustrative, more detailed embodiment of another one of the steps shown in FIG. 2.
FIGS. 5a-5c (referred to collectively as FIG. 5) are a flow chart of an illustrative, more detailed embodiment of still another one of the steps shown in FIG. 2.
FIG. 6 is a flow chart of an illustrative, more detailed embodiment of yet another one of the steps shown in FIG. 2.
FIG. 7 is a diagram of a dot product spectrum which is useful in explaining certain features of the invention.
FIG. 8 is a histogram diagram useful In eacplaining certain features of the invention.
FIG. 9 shows several equations which may be employed in accordance with the invention:
Detailed Desari~tion of the Preferred Fanbcadiments As shown in FIG. Z, a typical product inspection system 10 constructed in accordance with _ g _ this invention includes conveyor apparatus 20 for conveying the objects or products 12 to be inspected, one after another, from left to right as viewed in the FIG. Each time conventional product sensor 22 detects a product 12 at a predetermined location A opposite conventional camera 24, conventional processor 26 (which includes conventional imaging hardware) causes conventional light sources 30 to briefly illuminate the product, thereby allowing camera 24 to capture what is to effectively a still image of the product. This still image is fed to processor 26 which digitizes and further processes the image. Processor 26 is augmented by conventional video display 32~~and conventional data entry device 34 (e. g., a keyboard, mouse, and/or touch screen elements associated with display 32). Processor 26 can cause display 32 to display a product image captured by camera 24, and can augment that display with other information such as the outline of an acceptable product image and/or outlines of certain features of an acceptable product image. The operator may use this augmenting information to help determine whether the product image being displayed is acceptable. The operator may use data entry device 34 to control the overall operation of the system, as well as to respond to inquiries from the system (e.g., as to whether or not the operator fudges the product image currently shown on display 32 to be acceptable).
The system may be set up to perform a product inspection by operating it substantially as though it were inspecting products, i.e., by using conveyor 20 to convey representative products one after another past camera 24 and by using the other elements of the system to process the images of those praducts as described in detail below. . During actual product inspection, processor .26 determines whether the image of each successive product 12 is acceptable, and when that product reaches a controllable branch 20B in conveyor 20, processor 26 controls that branch so that acceptable products 12A are directed to accepted product conveyor 20A, while unacceptable products 12R
are directed to rejected product conveyor 20R.
While FIG. 1 suggests that system 10 operates on a single elevational image of products 12, it will be apparent to those skilled in the art that the system could be set up to test multiple images of the products taken from different angles and including perspective views so that as many surfaces of the objects are inspected as are desired. Similarly, although the system will be explained in terms of monochrome (e. g., Z5 black and white) images, it will be apparent to those skilled in the art how the system can be modified to inspect in full color. Thus camera 24 may be a conventional NTSC or RGB compatible camera. Processor 26 may be a suitably programmed conventional 386 personal computer workstation such as a CAT386 workstation available from Comark Corp. of Medfield, Massachusetts with a conventional IM~-1280 imaging hardware system available from MATROX Electronic Systems Limited of Dorval, Quebec, Canada.
An overview of a preferred embodiment of the method of this invention is shown in FIG. 2. Basically the depicted embodiment comprises a training portion, including three successive phases 1, 2, and 3 (shown in boxes 100, 200, and 300, respectively), and actual product inspection (shown in box 400j. During the three training phases, the system "learns", by appropriately processing product images with appropriate but relatively limited input from the human operator of the system, how to discriminate between good and bad images. Thereafter, during actual product g inspection, the system uses this °'knowledge°' to accept or reject products.
In training phase 1 (step 100 in FIG. 2) an, initial discriminant function F (which may be thought of as a two-dimensional matrix commensurate with the two-dimensional data for the product images or product image portions to be inspected) is formed from the data I for a relatively small number of °'first phase"
images. Although this initial discriminant function l0 could be computed in many other ways in accordance with this invention, in the preferred embodiment (shown .in detail in FIG. 3) a relatively simple technique (i.e., a logical OR of the phase 1 images) is used in order to allow a relatively small and inexpensive processor 26 to perform the necessary calculations without requiring more time than the operator of the system needs to provide the necessary inputs regarding each successive first phase image. Accordingly, as shown in FIG. 3 training phase 1 starts with step 102, and.in step 104 various program constants are initialized and inputs are read (e.g., from the memory which is~part of processor 26 and/or from data entry device 34). For example, step 104 may include selection of an image outline overlay to be displayed with product images on display 32 to help the operator judge the acceptability of images. Step 104 may also include selection of the boundaries of the portion or portions of the image to be processed. As another example, step 104 may include selection of a threshold to be used in binarizing the image data as discussed below. Any other necessary system initialization tasks may be performed as part of step 104.
In step 106 the system acquires the data for the first .of the first phase images. This is done by having camera 24 capture a product image as described above. Processor 26 then digitizes this image in full gray scale and causes display 32 to display this gray scale image with any augmenting information (such as an outline overlay) selected in step 104. The operator then indicates (via data entry device 34) whether or not the displayed image is acceptable.
If so, control passes to step 108. If not, step 106 is repeated with new product images until an image acceptable to the operator is found.
In step 108 the first acceptable image is preprocessed. This preferably includes edge detecting the gray scale image so that pixels at or near significant changes in image brightness are emphasized (e.g., increased in value) relative to other pixels which are de-emphasized (e.g., decreased in value). Edge detection is a well-known technique. After edge detection, the edge detected image is preferably binarized so that all pixels having values on one side of a predetermined binarization threshold level (which may have been selected in step 104) are assigned one binary value (e.g., 1 ), while all pixels having values on the other side of the binarization threshold level are assigned the other binary value (e.g., 0).
In step 110 the initial discriminant function F is set equal to the first acceptable image data from step 108.
In step 112 the sum of the dot products of the discriminant function and the phase 1 image data is initialized. Because at this point F
and I
are the same, the initial dot product of F and I is just the sum of the pixel values of I.
-In step 114 an index value i is set equal to 1.
In step 116 the system acquires the next acceptable image. Step 116 is therefore an exact 5 repetition of above--described step 106.
In step 118 the data for the next acceptable image tacquired in step 116) is preprocessed exactly as described above in connection with step 108.
In step 120 the initial discriminant function 10 is updated with the new image data by computing the logical OR of the new image data and the old initial discriminant function data to produce a new initial discriminant function. In other... words, for each pixel location in which either or both of the image data and the old initial discriminant function data are 1, the new initial discriminant data value is l, while for each pixel location in which both the image data and the old initial discriminant data are o, the new initial discriminant function data valu~ i's 0.
In step 122 the sum of the dot products of the discriminant function and the phase 1 image data is updated for the current image. Because the 1-valued pixel locations in each image are always a subset of the 1--valued pixel locations in F, each new dot product is just the sum of the pixel values in the current image I.
In step 124 the index i is incremented by 1, and in step 126 the new value of i. is~tested to determine whether it is greater than or equal to 25.
This is an arbitrary number which determines how many first phase images will be used to compute the initial discriminant function. Although any other number could be used, 25 has been found to give good results. If i has not yet reached 25, control returns to step 116 arid steps 116-.126 are repeated until the test in step 126 is satisfied and control consequently passes to step 128.
In step 128 the average of the dot products of F and each of the first phase images is'computed by dividing PSUM by 25 (the number of first phase images).
In step 130 training phase Z (step 200 in FIG. 2) begins. The initial discriminant function F
from the last performance of step 120 and the average dot product are saved.
l0 In step 200 the initial discriminant function F is used to compute statistical information about the images being processed. Again, althaugh this can be done in other ways in accordance-.with this invention, a preferred embodiment of step 200 is shown in FIG. 4 and will now be described by way of illustration.
Training phase 2 starts in step 202. In step 204 index value i is set equal to 1, variable S'UM is set equal to 0, and variable SUMS~RS (for sum of squares) is also set equal to 0.
In step 205 the initial hinary discriminant function F is converted to bipolar form using initial positive and negative values such that the'final discriminant function values take advantage of the full arithmetic range of processor 26. To reflect this in the average dot product, the average dot product is also scaled by the same scale factor in step 205. For example, if processor 26 performs 8-bit arithmetic with values between -128 and +127, the initial values now used in function F may be -50 (for pixel locations where the F value was formerly 0) and +50 (for pixel values where the F value was formerly 1), and the average dot product from step 128 may be multiplied by 50.
In step 206 a product image is acquired in a manner similar to above-described step 106, except that la during training phase 2 the operator of the system is not required to determine whether the image is acceptable. Accordingly, all the images received during phase 2 are used. These images can therefore be expected to exhibit the normal range of variation for the product images that the system will subsequently encounter during actual product inspection. In addition, because no interaction with the~operator of the system is required during this phase, the phase 2 l0 images can be processed much fastex (e. g., at actual product inspection rates) than the phase 1 images.
In step 208 the image data acquired in step 206 is preprocessed exactly as described above in connection with step 108.
In step 210 the dot product P of the resealed initial discriminant function F from step 205 and the image data I from step 208 is calculated. This calculation involves multiplying the value of F at each pixel location by the value of I at that pixel location, and then summing all of the resulting products to produce the dot product P. Rlsewhere in this specification the more generic term "processed value" is sometimes used for the dot product P. It will be noted that if I is identical to F, P will be a certain number, but if.I differs from F at certain pixel locations, P will be greater or less than that number. The amount by which P differs from that number is a measure of how much I differs from F. In practice, the values of P.will typically exhibit an approximately normal (i.e.,~approximately Gaussian) distribution about some mean or average value.
In step 212 the variable SUMSQRS is incremented by the square of the value of P from step 210, and the index value i is incremented by 1.
In step 214 the index value i is compared to an arbitrary number (e.g., 1000) which is the predetermined number of images to be processed in phase 2. Although any sufficiently large number of images can be processed in phase 2, 1000 images have been found to give good results. If the test in step 214 is not satisfied, control returns to step 206 where processing of the next phase 2 image begins. When 1000 phase 2 images have been processed as described above, l0 the test in step 214 is satisfied and control passes from step 214 to step 216.
In step 216 the index value i is reduced by 1 to reverse the last incrementing",of that value.
In step 218 the resealed average dot product from step 205 and the value of the SeTMSQRS variable are used to compute the standard deviation of the previously computed dot products.
In step 220 two first threshold~values and two second threshold values are selected so that the distribution of phase 2 dot products is subdivided by these threshold values as shown in FIG. ?. For example, the first threshold values may be chosen so that a fraction fi of the dot products are greater than the upper one of these threshold values and the same fraction of dot products are less than the lower one of these threshold values. The fraction f.i is preferably .
significantly greater than one-half the fraction of images which are expected to be defective in order to minimize the possibility that any unacceptable images have dot products that are greater than the upper or less than the lower of these first threshold values.
Images with dot products between the first threshold values are.therefore automatically acceptable as indicated in FIG. ?.
20~1~6~
The second threshold values are chosen so that a smaller fraction f2 of the dot products are greater than the upper one of these threshold values and the same smaller fraction of dot products are less than the lower one of these threshold values. The fraction f2 is preferably significantly smaller than one-half the fraction of images which are expected to be defective in order to minimize the possibility that any acceptable images have dot products that are less 1o than the lower or greater than the upper one of these second'threshold values. Images with dot products outside the region bounded by the second threshold values are therefore automatically rejectable as "gross rejects" as indicated in FIG. 7. Images with dot products outside the region bounded by thQ first threshold values but inside the region bounded by the second threshold values are "marginal rejects°' as indicated in FIG: 7. operator intervention is required to determine whether such an image should be accepted or rejected.
It may be convenient and appropriate to choose the threshold values described above assuming the distribution of dot products to be Gaussian as shown, for example, in FIG. 8, and therefore characterized by a standard deviation (given by the equation in step 128). In that case, the thresholds can be defined by the equations shown in FIG. 9. The average dot product used in these equations is the resealed average dot product from step 205. The alpha coefficients used in these equations with the standard deviation are selected so as to achieve the target fractions fl and f2 for.a Gaussian distribution. These values can be readily selected with the aid of available tables of the properties of the Gaussian distribution. The most preferred approach is to select _ 15 _ the first threshold values without assuming a Gaussian distribution (i.e., as described prior to the above discussion of the Gaussian distribution), and to use the second method (i.e., the Gaussian distribution assumption) to select the second threshold values.
Note that in FIG. 8 the region A corresponds to the "acceptable" region of FIG. 7, the regions B correspond to the ''gross reject°' regions of FIG. Z, and the regions C correspond to the "marginal reject"regions L0 of FIGS 7. Thus region A includes dot products known to be associated with clearly acceptable images, whereas regions B include dot products known to be associated with clearly unacceptable images. Regions C
are those along the distribution of dot products P
which may be marginally acceptable. Adaptive training is performed in phase 3 as discussed below with respect to dat products lying in region A, and also with respect to dot products ly3.ng in regions C which the operator of the system determines to be acceptable.
After the second threshold values are calculated in step 220, control passes to step 222 to begin training phase 3 (step 300 in FIG. 2). The first and second threshold values from step 220 are saved, as are F and the rescaled average dot product from step 205.
In training phase 3 (step 300 in FIG. 2) the statistical information from phase 2 is used with the image data from another statistically significant number of images to refine the initial discriminant function F. Again, although this can be done in other ways in accordance with this invention, a preferred embodiment of training phase 3 is shown in FIG. 5 which will now be described by way of illustration.
.As shown in FIG. 5, training phase 3 begins in step 302, and in step 310 an index value i is 2~~~.~~
initialized to 0, and a counter -- used during phase 3 to count the number of marginally acceptable images which the operator of the system decides to accept --is also initialized to 0.
In step 312 a phase 3 image is acquired exactly as described above in connection with step 206, and in step 314 the data for this image is preprocessed as described above in connection with step 208.
In step 316 the dot product P of the discrilninant function and the image data from step 314 is calculated.
In step 318 the dot product value P from step 316 is compared to the second threshold values from step 220. If P is outside the range bounded by these second threshold values, control passes to step 320 where the image is rejected and control is.then returned to step 312 to begin the acquisition and processing of the next phase 3 image. On the other hand, if P is in the range bounded by the second, threshold values, control passes from step 318 to step 322.
In step 322 the value of P from step 316 is compared to the first threshold values from step 220.
If P is outside the range bounded by these first threshold values, control passes to step 324. Step 324 is reached whenever the image is neither.automatically rejectable as unacceptable (because the associated dot product is outside the limits defined by the second threshold values), nor automatically acceptable (because the associated dot product is inside the limits defined by the first threshold values).
Accordingly, in step 324 the operator ofthe system is asked to intervene and decide whether oranot the image is acceptable. The image is displayed on;display 32 (as in connection with step 106 above. y"If the 2fl~~.8~~
_ i?
operator responds (again as in connection with step 106) that the image is unacceptable, control passes to step 320 where the image is rejected, and thereafter processing of the next phase 3 image begins as described above. On the other hand, if the operator responds that the image is acceptable, control passes from step 324 to step 326 where the counter nacc is incremented. Thereafter, control passes to step 328.
Returning to the other branch from step 322, if P is ZO not outside the limits defined by the first threshold values, the image is automatically acceptable and control passes directly from step 322 to step 328.
Step 328 is performed only whsn~tha current third phase image has been determined to be an acceptable image. In most cases the system will have made this determination automatically because the dot .
product P for the image is between the first threshold values and the image is therefore obviously acceptable.
In a few cases, however, the operator will have been required to assist with this determination as described above in connection with step 324. Accordingly, for the most part the processing of images can proceed as rapidly during phase 3 as during phase 2.' Only rarely will the operator be required to intervene as a result of the performance of step 324. Moreover,. operator intervention should.be required even less frequently as phase 3 proceeds and the discriminant function is progressively refined as will now be described.
Step 328 begins the process of refining the resealed discriminant function using the data from the image which has just been determined to be acceptable.
This discriminant function refining process is repeated for each acceptable phase 3 image. In step 328 an "error" value equal to the difference between the average P value from step 205 and P from step 316 is 29~~.865 calculated. In step 329 a value N equal co the number of pixels which are "on" in the image datasis calculated. In step 330 a correction value equal to the error value from step 328 divided by the value of N
from step 329 is calculated. In step 332 the binary image data for the current phase 3 imae~e is resealed using the correction value from step 330.._.Ln particular, each pixel value of 3. is changed to the correction value, while each pixel value of 0 is 1o unaltered.
In step 334 the resealed discrianinant function is refined by incrementing each pixel value by the value associated with that pixel in the resealed image data from step 332. Step 334 is an "adaptive training" step analogous to the Widrow-Hoff training algorithm sometimes used in signal processing (see, for example, B. Widrow and S.D. Stearns, Adat~tive Signal Processing, Prentice-Hall,~Englewood Clifs, 1985).
Accordingly, as step 334 is performed for,successive acceptable third phase images, the resealed discriminant function becomes better and better at producing dot products (as in step 316) which are clearly differentiated between those associated with acceptable images (P within the range bounded by the first threshold values) and those associated with unacceptable images (P outside,the range bounded by the second threshold values). Accordingly, as phase 3 progresses, there should be less and less..,need to' perform step 324, and the amount of input~required from the operator of the system should decrease.
In step 336 the index value i is incremented.
In step 338 this index value is compared to a phase 3 cut-off value (e. g., 2000 acceptable phase.3 images).
If the index value is less than the cut-off value, control passes from step 338 to step 312 where _ ,,~
processing of the next phase 3 image begins. As soon as step 338 detects that the index value has xeached the cut-off value, control passes from step 338 to step 340.
In step 340 the rati~ of the counter value nacc to the index value i is compared to aw' predetermined threshold value. If this radio exceeds the threshold value, the system is still tentatively rejecting too many images which the operator of the l0 system~has found acceptable in step 324. Whis indicates that the discriminant functi~n F' is still in need of further refinement. accordingly,"~~ontrol passes from step 340 to step 310~~rhers therprocessing of another series of phase 3 images begins'~again. 4n the other hand, if the ratio in step 340 is less than the threshold, the refining of discriminant function F' is judged complete, and training phase 3 is concluded by passing control to step~342 where actual product inspection begins (step 400 in FTG. 2).
An illustrative embodiment of actual product inspection (step 400 in FIG. 2) is shown in FIG. 6.
This process begins with step 402, and instep 404 an image is acquired as in step 206. In step 406 the data for this image is preprocessed as in step 208. In step 408 the dot product P of the refined discriminant function from training phase 3 and the image data from step 406 is calculated. In step 410 P is tested to determine whether or not it is in the range between the first threshold values from step 220. If so, the system deems the image acceptable and control passes to step 412 in order to accept the product (i.e., direct it to accepted product conveyor 201 in FIG. 1). If the step 410 test is not satisfied, the system deems the image unacceptable and control passes to step 4.14 in order to reject the product (i.e., direct it to rejected product conveyor 20R in FIG. 1).. After either step 412 or 414, control returns to step 404 to begin processing of the next image.
Tt will be understood that the foregoing is merely illustrative of the principles of this invention and that carious modifications can be made by those skilled in the art without departing from the scope and spirit of the invention. For example, each image formed by elements 24 and 26 can be bro.ken~down into a to plural~.ty of predetermined segments, and the data for each of the segments can be separately processed just as the data for the whole image ~.s processed in the foregoing discussion. During actual. product inspection all segments must satisfy the test of step'410 in order for the product to be accepted. As another example of a modification within the scope of this invention, .".,.
discriminant function F° can continue to b_e refined during actual product inspection by updatihg it in accordance with any acceptable product. image or images exactly as described abode .in connection with trainincr phase 3. In another modification withjn -the scope of the in~zention bipolar values -1 and +1 are used instead pf binary values throughout and the discussion of, for e~le, steps 108, 118, 208, 314 and 406 should be constn~ed accordingly.
It should be understood that both binary and bipolar values are merely examples and any other two values can be used instead and are within -the scope of the invention.
METHODS AND APPARATUS FOR
OPTICALLY DETE'RMINTNG THE A~CEP'~ABILITY OF PRODUCTS
Background of the Invention This invention relates to product inspection methods and apparatus, and more particularly to methods and apparatus for optically determining whether or not a product has an acceptable appearance.
For many products such as consumer goods like 1o packaged foods, beverages,~cleaning products, health and beauty aids, cigarettes, cigars, etc., it is very important that the external appearance of the product or its packaging be uniform and defect-free. Yet these products are typically produced in such large.
quantities and at such high speeds that some form of automated optical inspection is practically essential.
It is highly desirable for an optical inspection system to be able to test all or substantially all parts of ~,:..
the product image so that defects of any kind occurring anywhere in the image can be detected. At the same time the inspection system should not reject products having minor but acceptable deviations from the ideal product.
For even a relatively simple product image such as a cigarette pack, an inspection system must be initially supplied with a tremendous amount of information in order to enable the system to inspect all or substantially all portions of tY.se image with the 2~~~.~~5 sophistication required to discriminate between acceptable products (i.e., products having the ideal appearance or an appearance acceptably close to the ideal) and unacceptable products which should be rejected because of defects in appearance or appearance which is not sufficiently.close to the ideal.
Identifying and entering this information into the inspection system apparatus typically requires a very high level of skill and/or large amounts of operator 1o time. 'Moreover, this data identification and entry task must be repeated each time a new or even slightly different product is to be inspected.
It is therefore an object of this invention to improve and simplify optical inspection systems.
It is a more particular object of this invention to provide optical inspection systems which greatly reduce the level of operator skill and amount of operator time required to set up the.system to inspect a new or different product.
Summary of the Invention These and other objects of the invention are accomplished in accordance with the principles of the invention by providing optical inspection systems which form an initial discriminant function or "filter" from a composite of a relatively small number of '°first phase'° images which the operator of the system determines. to be acceptable images. A relatively simple technique (e.g., a logical OR function) is preferably used to form this composite. The ability to produce the initial discriminant function quickly using such a simple combination of a small number of images facilitates rapid "start-up" of the systems Thereafter the system~uses the initial discriminant function to process a relatively large number of representative '°second phase" images in order to compute statistical information about the images in relation to the initial discriminant function. In particular, the system uses the initial discriminant function to compute a processed value for each second phase image. These processed values will typically have an approximately normal or Gaussian distribution. The upper and lower limits of a central portion of this distribution containing a first statistically large number of the processed values are identified as first threshold values'. The upper and. lower limits of a central portion of this distribution containing a second statistically even larger number"of the processed values are identified as second threshold values.
In a subsequent third phase of the operation of the system, the first and second threshold values are used in the processing of a relatively large number of "third phase" images. In particular, the system uses a discriminant function (initially the.above-mentioned initial discriminant function) to compute a processed value for each successive third phase image.
If this processed value for a given third phase image is between the first threshold values, that third phase image is automatically used to refine (e.g., using a Widrow-Hoff-type adaptive training process) the discriminant function for subsequent use., If the processed value for a third phase image is~not between the second threshold values, that third phase image is automatically discarded. As a third possibility, if the processed value for a third phase image is not between the first threshold values but fs between the second threshold values, the operator of the~system is given the choice as to whether or not that image should be discarded (i.e., because the image looks unacceptable) or used to refine the discriminant YW
~ $ ~.
function for subsequent use (i.e., because. the image looks acceptable). w When the third phase is completed, the system is ready for actual product inspection using the refined discriminant function and the first threshold values. In actual product inspection, the system uses the refined discriminant function to compute a processed value for each product image. I:~ the processed value for a product image is between the first threshold values, the product is aoc~spted as having an acceptable appearance. If the processed value for a product image is not._betwsen the first threshold values, the product is rejected as having an unacceptable appearance.
The system greatly reduces the level of operator skill and amount of operator time required to set the system up for a product inspection task. The operator is only required to identify ~ relatively small number of acceptable images (e.g., 25) during the first phase. The initial discriminant function is then computed~automatically using a simple and rapid technique such as a logical t7R of the small number of first phase images. The entire second phase may also be automatic. And during the third phase, the operator is only required to decide on the acceptability of the relatively small number of images whose processed values fall outside the first threshold values but between the second threshold values.
Further features of the invention, its nature and various advantages will be more apparent from the accompanying drawings and the following detailed description of the preferred embodiments.
_5_ $r~ Description of the 3~rawi.nas FIG. 1 is a simplified schematic'block diagram of an illustrative embadiment of an optical product inspection apparatus constructed in accordance with the principles of this invention.
FIGS. 2a and 2b (referred to collectively as FIG. 2) are a flow chart of an illustrative optical product inspection method in accordance with this invention.
1 FIGS. 3a-3c (referred to collectively as FIG. 3) are a flow chart of an illustrative, more detailed embodiment of one of the steps shown in FIG. 2.
FIGS. 4a and 4b (referred to collectively as FIG. 4) are a flow chart of an illustrative, more detailed embodiment of another one of the steps shown in FIG. 2.
FIGS. 5a-5c (referred to collectively as FIG. 5) are a flow chart of an illustrative, more detailed embodiment of still another one of the steps shown in FIG. 2.
FIG. 6 is a flow chart of an illustrative, more detailed embodiment of yet another one of the steps shown in FIG. 2.
FIG. 7 is a diagram of a dot product spectrum which is useful in explaining certain features of the invention.
FIG. 8 is a histogram diagram useful In eacplaining certain features of the invention.
FIG. 9 shows several equations which may be employed in accordance with the invention:
Detailed Desari~tion of the Preferred Fanbcadiments As shown in FIG. Z, a typical product inspection system 10 constructed in accordance with _ g _ this invention includes conveyor apparatus 20 for conveying the objects or products 12 to be inspected, one after another, from left to right as viewed in the FIG. Each time conventional product sensor 22 detects a product 12 at a predetermined location A opposite conventional camera 24, conventional processor 26 (which includes conventional imaging hardware) causes conventional light sources 30 to briefly illuminate the product, thereby allowing camera 24 to capture what is to effectively a still image of the product. This still image is fed to processor 26 which digitizes and further processes the image. Processor 26 is augmented by conventional video display 32~~and conventional data entry device 34 (e. g., a keyboard, mouse, and/or touch screen elements associated with display 32). Processor 26 can cause display 32 to display a product image captured by camera 24, and can augment that display with other information such as the outline of an acceptable product image and/or outlines of certain features of an acceptable product image. The operator may use this augmenting information to help determine whether the product image being displayed is acceptable. The operator may use data entry device 34 to control the overall operation of the system, as well as to respond to inquiries from the system (e.g., as to whether or not the operator fudges the product image currently shown on display 32 to be acceptable).
The system may be set up to perform a product inspection by operating it substantially as though it were inspecting products, i.e., by using conveyor 20 to convey representative products one after another past camera 24 and by using the other elements of the system to process the images of those praducts as described in detail below. . During actual product inspection, processor .26 determines whether the image of each successive product 12 is acceptable, and when that product reaches a controllable branch 20B in conveyor 20, processor 26 controls that branch so that acceptable products 12A are directed to accepted product conveyor 20A, while unacceptable products 12R
are directed to rejected product conveyor 20R.
While FIG. 1 suggests that system 10 operates on a single elevational image of products 12, it will be apparent to those skilled in the art that the system could be set up to test multiple images of the products taken from different angles and including perspective views so that as many surfaces of the objects are inspected as are desired. Similarly, although the system will be explained in terms of monochrome (e. g., Z5 black and white) images, it will be apparent to those skilled in the art how the system can be modified to inspect in full color. Thus camera 24 may be a conventional NTSC or RGB compatible camera. Processor 26 may be a suitably programmed conventional 386 personal computer workstation such as a CAT386 workstation available from Comark Corp. of Medfield, Massachusetts with a conventional IM~-1280 imaging hardware system available from MATROX Electronic Systems Limited of Dorval, Quebec, Canada.
An overview of a preferred embodiment of the method of this invention is shown in FIG. 2. Basically the depicted embodiment comprises a training portion, including three successive phases 1, 2, and 3 (shown in boxes 100, 200, and 300, respectively), and actual product inspection (shown in box 400j. During the three training phases, the system "learns", by appropriately processing product images with appropriate but relatively limited input from the human operator of the system, how to discriminate between good and bad images. Thereafter, during actual product g inspection, the system uses this °'knowledge°' to accept or reject products.
In training phase 1 (step 100 in FIG. 2) an, initial discriminant function F (which may be thought of as a two-dimensional matrix commensurate with the two-dimensional data for the product images or product image portions to be inspected) is formed from the data I for a relatively small number of °'first phase"
images. Although this initial discriminant function l0 could be computed in many other ways in accordance with this invention, in the preferred embodiment (shown .in detail in FIG. 3) a relatively simple technique (i.e., a logical OR of the phase 1 images) is used in order to allow a relatively small and inexpensive processor 26 to perform the necessary calculations without requiring more time than the operator of the system needs to provide the necessary inputs regarding each successive first phase image. Accordingly, as shown in FIG. 3 training phase 1 starts with step 102, and.in step 104 various program constants are initialized and inputs are read (e.g., from the memory which is~part of processor 26 and/or from data entry device 34). For example, step 104 may include selection of an image outline overlay to be displayed with product images on display 32 to help the operator judge the acceptability of images. Step 104 may also include selection of the boundaries of the portion or portions of the image to be processed. As another example, step 104 may include selection of a threshold to be used in binarizing the image data as discussed below. Any other necessary system initialization tasks may be performed as part of step 104.
In step 106 the system acquires the data for the first .of the first phase images. This is done by having camera 24 capture a product image as described above. Processor 26 then digitizes this image in full gray scale and causes display 32 to display this gray scale image with any augmenting information (such as an outline overlay) selected in step 104. The operator then indicates (via data entry device 34) whether or not the displayed image is acceptable.
If so, control passes to step 108. If not, step 106 is repeated with new product images until an image acceptable to the operator is found.
In step 108 the first acceptable image is preprocessed. This preferably includes edge detecting the gray scale image so that pixels at or near significant changes in image brightness are emphasized (e.g., increased in value) relative to other pixels which are de-emphasized (e.g., decreased in value). Edge detection is a well-known technique. After edge detection, the edge detected image is preferably binarized so that all pixels having values on one side of a predetermined binarization threshold level (which may have been selected in step 104) are assigned one binary value (e.g., 1 ), while all pixels having values on the other side of the binarization threshold level are assigned the other binary value (e.g., 0).
In step 110 the initial discriminant function F is set equal to the first acceptable image data from step 108.
In step 112 the sum of the dot products of the discriminant function and the phase 1 image data is initialized. Because at this point F
and I
are the same, the initial dot product of F and I is just the sum of the pixel values of I.
-In step 114 an index value i is set equal to 1.
In step 116 the system acquires the next acceptable image. Step 116 is therefore an exact 5 repetition of above--described step 106.
In step 118 the data for the next acceptable image tacquired in step 116) is preprocessed exactly as described above in connection with step 108.
In step 120 the initial discriminant function 10 is updated with the new image data by computing the logical OR of the new image data and the old initial discriminant function data to produce a new initial discriminant function. In other... words, for each pixel location in which either or both of the image data and the old initial discriminant function data are 1, the new initial discriminant data value is l, while for each pixel location in which both the image data and the old initial discriminant data are o, the new initial discriminant function data valu~ i's 0.
In step 122 the sum of the dot products of the discriminant function and the phase 1 image data is updated for the current image. Because the 1-valued pixel locations in each image are always a subset of the 1--valued pixel locations in F, each new dot product is just the sum of the pixel values in the current image I.
In step 124 the index i is incremented by 1, and in step 126 the new value of i. is~tested to determine whether it is greater than or equal to 25.
This is an arbitrary number which determines how many first phase images will be used to compute the initial discriminant function. Although any other number could be used, 25 has been found to give good results. If i has not yet reached 25, control returns to step 116 arid steps 116-.126 are repeated until the test in step 126 is satisfied and control consequently passes to step 128.
In step 128 the average of the dot products of F and each of the first phase images is'computed by dividing PSUM by 25 (the number of first phase images).
In step 130 training phase Z (step 200 in FIG. 2) begins. The initial discriminant function F
from the last performance of step 120 and the average dot product are saved.
l0 In step 200 the initial discriminant function F is used to compute statistical information about the images being processed. Again, althaugh this can be done in other ways in accordance-.with this invention, a preferred embodiment of step 200 is shown in FIG. 4 and will now be described by way of illustration.
Training phase 2 starts in step 202. In step 204 index value i is set equal to 1, variable S'UM is set equal to 0, and variable SUMS~RS (for sum of squares) is also set equal to 0.
In step 205 the initial hinary discriminant function F is converted to bipolar form using initial positive and negative values such that the'final discriminant function values take advantage of the full arithmetic range of processor 26. To reflect this in the average dot product, the average dot product is also scaled by the same scale factor in step 205. For example, if processor 26 performs 8-bit arithmetic with values between -128 and +127, the initial values now used in function F may be -50 (for pixel locations where the F value was formerly 0) and +50 (for pixel values where the F value was formerly 1), and the average dot product from step 128 may be multiplied by 50.
In step 206 a product image is acquired in a manner similar to above-described step 106, except that la during training phase 2 the operator of the system is not required to determine whether the image is acceptable. Accordingly, all the images received during phase 2 are used. These images can therefore be expected to exhibit the normal range of variation for the product images that the system will subsequently encounter during actual product inspection. In addition, because no interaction with the~operator of the system is required during this phase, the phase 2 l0 images can be processed much fastex (e. g., at actual product inspection rates) than the phase 1 images.
In step 208 the image data acquired in step 206 is preprocessed exactly as described above in connection with step 108.
In step 210 the dot product P of the resealed initial discriminant function F from step 205 and the image data I from step 208 is calculated. This calculation involves multiplying the value of F at each pixel location by the value of I at that pixel location, and then summing all of the resulting products to produce the dot product P. Rlsewhere in this specification the more generic term "processed value" is sometimes used for the dot product P. It will be noted that if I is identical to F, P will be a certain number, but if.I differs from F at certain pixel locations, P will be greater or less than that number. The amount by which P differs from that number is a measure of how much I differs from F. In practice, the values of P.will typically exhibit an approximately normal (i.e.,~approximately Gaussian) distribution about some mean or average value.
In step 212 the variable SUMSQRS is incremented by the square of the value of P from step 210, and the index value i is incremented by 1.
In step 214 the index value i is compared to an arbitrary number (e.g., 1000) which is the predetermined number of images to be processed in phase 2. Although any sufficiently large number of images can be processed in phase 2, 1000 images have been found to give good results. If the test in step 214 is not satisfied, control returns to step 206 where processing of the next phase 2 image begins. When 1000 phase 2 images have been processed as described above, l0 the test in step 214 is satisfied and control passes from step 214 to step 216.
In step 216 the index value i is reduced by 1 to reverse the last incrementing",of that value.
In step 218 the resealed average dot product from step 205 and the value of the SeTMSQRS variable are used to compute the standard deviation of the previously computed dot products.
In step 220 two first threshold~values and two second threshold values are selected so that the distribution of phase 2 dot products is subdivided by these threshold values as shown in FIG. ?. For example, the first threshold values may be chosen so that a fraction fi of the dot products are greater than the upper one of these threshold values and the same fraction of dot products are less than the lower one of these threshold values. The fraction f.i is preferably .
significantly greater than one-half the fraction of images which are expected to be defective in order to minimize the possibility that any unacceptable images have dot products that are greater than the upper or less than the lower of these first threshold values.
Images with dot products between the first threshold values are.therefore automatically acceptable as indicated in FIG. ?.
20~1~6~
The second threshold values are chosen so that a smaller fraction f2 of the dot products are greater than the upper one of these threshold values and the same smaller fraction of dot products are less than the lower one of these threshold values. The fraction f2 is preferably significantly smaller than one-half the fraction of images which are expected to be defective in order to minimize the possibility that any acceptable images have dot products that are less 1o than the lower or greater than the upper one of these second'threshold values. Images with dot products outside the region bounded by the second threshold values are therefore automatically rejectable as "gross rejects" as indicated in FIG. 7. Images with dot products outside the region bounded by thQ first threshold values but inside the region bounded by the second threshold values are "marginal rejects°' as indicated in FIG: 7. operator intervention is required to determine whether such an image should be accepted or rejected.
It may be convenient and appropriate to choose the threshold values described above assuming the distribution of dot products to be Gaussian as shown, for example, in FIG. 8, and therefore characterized by a standard deviation (given by the equation in step 128). In that case, the thresholds can be defined by the equations shown in FIG. 9. The average dot product used in these equations is the resealed average dot product from step 205. The alpha coefficients used in these equations with the standard deviation are selected so as to achieve the target fractions fl and f2 for.a Gaussian distribution. These values can be readily selected with the aid of available tables of the properties of the Gaussian distribution. The most preferred approach is to select _ 15 _ the first threshold values without assuming a Gaussian distribution (i.e., as described prior to the above discussion of the Gaussian distribution), and to use the second method (i.e., the Gaussian distribution assumption) to select the second threshold values.
Note that in FIG. 8 the region A corresponds to the "acceptable" region of FIG. 7, the regions B correspond to the ''gross reject°' regions of FIG. Z, and the regions C correspond to the "marginal reject"regions L0 of FIGS 7. Thus region A includes dot products known to be associated with clearly acceptable images, whereas regions B include dot products known to be associated with clearly unacceptable images. Regions C
are those along the distribution of dot products P
which may be marginally acceptable. Adaptive training is performed in phase 3 as discussed below with respect to dat products lying in region A, and also with respect to dot products ly3.ng in regions C which the operator of the system determines to be acceptable.
After the second threshold values are calculated in step 220, control passes to step 222 to begin training phase 3 (step 300 in FIG. 2). The first and second threshold values from step 220 are saved, as are F and the rescaled average dot product from step 205.
In training phase 3 (step 300 in FIG. 2) the statistical information from phase 2 is used with the image data from another statistically significant number of images to refine the initial discriminant function F. Again, although this can be done in other ways in accordance with this invention, a preferred embodiment of training phase 3 is shown in FIG. 5 which will now be described by way of illustration.
.As shown in FIG. 5, training phase 3 begins in step 302, and in step 310 an index value i is 2~~~.~~
initialized to 0, and a counter -- used during phase 3 to count the number of marginally acceptable images which the operator of the system decides to accept --is also initialized to 0.
In step 312 a phase 3 image is acquired exactly as described above in connection with step 206, and in step 314 the data for this image is preprocessed as described above in connection with step 208.
In step 316 the dot product P of the discrilninant function and the image data from step 314 is calculated.
In step 318 the dot product value P from step 316 is compared to the second threshold values from step 220. If P is outside the range bounded by these second threshold values, control passes to step 320 where the image is rejected and control is.then returned to step 312 to begin the acquisition and processing of the next phase 3 image. On the other hand, if P is in the range bounded by the second, threshold values, control passes from step 318 to step 322.
In step 322 the value of P from step 316 is compared to the first threshold values from step 220.
If P is outside the range bounded by these first threshold values, control passes to step 324. Step 324 is reached whenever the image is neither.automatically rejectable as unacceptable (because the associated dot product is outside the limits defined by the second threshold values), nor automatically acceptable (because the associated dot product is inside the limits defined by the first threshold values).
Accordingly, in step 324 the operator ofthe system is asked to intervene and decide whether oranot the image is acceptable. The image is displayed on;display 32 (as in connection with step 106 above. y"If the 2fl~~.8~~
_ i?
operator responds (again as in connection with step 106) that the image is unacceptable, control passes to step 320 where the image is rejected, and thereafter processing of the next phase 3 image begins as described above. On the other hand, if the operator responds that the image is acceptable, control passes from step 324 to step 326 where the counter nacc is incremented. Thereafter, control passes to step 328.
Returning to the other branch from step 322, if P is ZO not outside the limits defined by the first threshold values, the image is automatically acceptable and control passes directly from step 322 to step 328.
Step 328 is performed only whsn~tha current third phase image has been determined to be an acceptable image. In most cases the system will have made this determination automatically because the dot .
product P for the image is between the first threshold values and the image is therefore obviously acceptable.
In a few cases, however, the operator will have been required to assist with this determination as described above in connection with step 324. Accordingly, for the most part the processing of images can proceed as rapidly during phase 3 as during phase 2.' Only rarely will the operator be required to intervene as a result of the performance of step 324. Moreover,. operator intervention should.be required even less frequently as phase 3 proceeds and the discriminant function is progressively refined as will now be described.
Step 328 begins the process of refining the resealed discriminant function using the data from the image which has just been determined to be acceptable.
This discriminant function refining process is repeated for each acceptable phase 3 image. In step 328 an "error" value equal to the difference between the average P value from step 205 and P from step 316 is 29~~.865 calculated. In step 329 a value N equal co the number of pixels which are "on" in the image datasis calculated. In step 330 a correction value equal to the error value from step 328 divided by the value of N
from step 329 is calculated. In step 332 the binary image data for the current phase 3 imae~e is resealed using the correction value from step 330.._.Ln particular, each pixel value of 3. is changed to the correction value, while each pixel value of 0 is 1o unaltered.
In step 334 the resealed discrianinant function is refined by incrementing each pixel value by the value associated with that pixel in the resealed image data from step 332. Step 334 is an "adaptive training" step analogous to the Widrow-Hoff training algorithm sometimes used in signal processing (see, for example, B. Widrow and S.D. Stearns, Adat~tive Signal Processing, Prentice-Hall,~Englewood Clifs, 1985).
Accordingly, as step 334 is performed for,successive acceptable third phase images, the resealed discriminant function becomes better and better at producing dot products (as in step 316) which are clearly differentiated between those associated with acceptable images (P within the range bounded by the first threshold values) and those associated with unacceptable images (P outside,the range bounded by the second threshold values). Accordingly, as phase 3 progresses, there should be less and less..,need to' perform step 324, and the amount of input~required from the operator of the system should decrease.
In step 336 the index value i is incremented.
In step 338 this index value is compared to a phase 3 cut-off value (e. g., 2000 acceptable phase.3 images).
If the index value is less than the cut-off value, control passes from step 338 to step 312 where _ ,,~
processing of the next phase 3 image begins. As soon as step 338 detects that the index value has xeached the cut-off value, control passes from step 338 to step 340.
In step 340 the rati~ of the counter value nacc to the index value i is compared to aw' predetermined threshold value. If this radio exceeds the threshold value, the system is still tentatively rejecting too many images which the operator of the l0 system~has found acceptable in step 324. Whis indicates that the discriminant functi~n F' is still in need of further refinement. accordingly,"~~ontrol passes from step 340 to step 310~~rhers therprocessing of another series of phase 3 images begins'~again. 4n the other hand, if the ratio in step 340 is less than the threshold, the refining of discriminant function F' is judged complete, and training phase 3 is concluded by passing control to step~342 where actual product inspection begins (step 400 in FTG. 2).
An illustrative embodiment of actual product inspection (step 400 in FIG. 2) is shown in FIG. 6.
This process begins with step 402, and instep 404 an image is acquired as in step 206. In step 406 the data for this image is preprocessed as in step 208. In step 408 the dot product P of the refined discriminant function from training phase 3 and the image data from step 406 is calculated. In step 410 P is tested to determine whether or not it is in the range between the first threshold values from step 220. If so, the system deems the image acceptable and control passes to step 412 in order to accept the product (i.e., direct it to accepted product conveyor 201 in FIG. 1). If the step 410 test is not satisfied, the system deems the image unacceptable and control passes to step 4.14 in order to reject the product (i.e., direct it to rejected product conveyor 20R in FIG. 1).. After either step 412 or 414, control returns to step 404 to begin processing of the next image.
Tt will be understood that the foregoing is merely illustrative of the principles of this invention and that carious modifications can be made by those skilled in the art without departing from the scope and spirit of the invention. For example, each image formed by elements 24 and 26 can be bro.ken~down into a to plural~.ty of predetermined segments, and the data for each of the segments can be separately processed just as the data for the whole image ~.s processed in the foregoing discussion. During actual. product inspection all segments must satisfy the test of step'410 in order for the product to be accepted. As another example of a modification within the scope of this invention, .".,.
discriminant function F° can continue to b_e refined during actual product inspection by updatihg it in accordance with any acceptable product. image or images exactly as described abode .in connection with trainincr phase 3. In another modification withjn -the scope of the in~zention bipolar values -1 and +1 are used instead pf binary values throughout and the discussion of, for e~le, steps 108, 118, 208, 314 and 406 should be constn~ed accordingly.
It should be understood that both binary and bipolar values are merely examples and any other two values can be used instead and are within -the scope of the invention.
Claims (20)
1. A method of determining the acceptability of a product by generating a filter from a first set of acceptable images of the product, comparing the filter with each of a second set of images of the product to produce a processed value for each image in the second set, the processed values having a distribution of values, comparing the filter with each of a third set of images of the product to produce a further processed value for each image in the third set, and comparing each further processed value to the distribution to determine the acceptability of the product associated with the image having the further processed value, characterized by:
generating from the distribution a first range and a second range of processed values, the first range comprising processed values associated with acceptable images of the product, and the second range being spaced from said first range and comprising processed values associated with unacceptable images of the product;
determining whether each further processed value is outside both of the first and second ranges and, if so, selecting the associated image only if said image is acceptable; and adaptively training the filter with the selected image to produce a modified filter whereby comparison of the modified filter with the selected image produces a modified processed value closer to the first range.
generating from the distribution a first range and a second range of processed values, the first range comprising processed values associated with acceptable images of the product, and the second range being spaced from said first range and comprising processed values associated with unacceptable images of the product;
determining whether each further processed value is outside both of the first and second ranges and, if so, selecting the associated image only if said image is acceptable; and adaptively training the filter with the selected image to produce a modified filter whereby comparison of the modified filter with the selected image produces a modified processed value closer to the first range.
2. The method according to Claim 1, wherein generating the filter from the first set of acceptable images of the product comprises combining the first set of acceptable images using a logical OR operation.
3. The method as claimed in Claim 2, wherein the step of combining the first set of acceptable images includes the steps of:
edge detecting each image; and associating a first value with each portion of each edge detected image which has a value on one side of a predetermined threshold value, and associating a second value with all other portions of each edge detected image prior to combining the images using the logical OR operation.
edge detecting each image; and associating a first value with each portion of each edge detected image which has a value on one side of a predetermined threshold value, and associating a second value with all other portions of each edge detected image prior to combining the images using the logical OR operation.
4. The method according to Claim 3, wherein the first and second values are binary 0 and 1.
5. The method according to any one of Claims 1 to 4, wherein the adaptive training of the filter is analogous to a Widrow-Hoff training algorithm.
6. The method according to any one of Claims 1 to 5, wherein the selected image is selected manually based on the appearance of the product associated with the image.
7. The method according to any one of Claims 1 to 6, characterised by adaptively training the filter with the associated image if the further processed value is in the first range.
8. The method according to any one of Claims 1 to 7, wherein the processed values of each image in the second set are the dot product of the filter and each of the second set of images.
9. The method according to any one of Claims 1 to 8, characterised by subdividing each of the acceptable images into a plurality of image portions and performing all of the foregoing steps upon at least some of the image portions individually.
10. Apparatus for determining the acceptability of a product by generating a filter from a first set of acceptable images of the product, comparing the filter with each of a second set of images of the product to produce a processed value for each image in the second set, the processed values having a distribution of values, comparing the filter with each of a third set of images of the product to produce a further processed value for each image in the third set, and comparing each further processed value to the distribution to determine the acceptability of the product associated with the image having the further processed value, characterized by:
means for generating from the distribution a first range and a second range of processed values, the first range comprising processed values associated with acceptable images of the product, and the second range being spaced from the first range and comprising processed values associated with unacceptable images of the product;
means for determining whether each further processed value is outside both of the first and second ranges and, if so, selecting the associated image only if said image is acceptable; and means for adaptively training the filter with the selected image to produce a modified filter whereby the comparison of the modified filter with the selected image produces a modified processed value closer to the first range.
means for generating from the distribution a first range and a second range of processed values, the first range comprising processed values associated with acceptable images of the product, and the second range being spaced from the first range and comprising processed values associated with unacceptable images of the product;
means for determining whether each further processed value is outside both of the first and second ranges and, if so, selecting the associated image only if said image is acceptable; and means for adaptively training the filter with the selected image to produce a modified filter whereby the comparison of the modified filter with the selected image produces a modified processed value closer to the first range.
11. The apparatus according to Claim 10, wherein the filter is generated from the first set of acceptable images of the product by means for combining the first set of acceptable images using a logical OR operation.
12. The apparatus according to Claim 11, wherein the means for combining the first set of acceptable images comprises:
means for edge detecting each image; and means for associating a first value with etch portion of each edge detected image which has a value on one side of a predetermined threshold value, and associating a second value with all other portions of each edge detected image prior to combining the images using the logical OR operation.
means for edge detecting each image; and means for associating a first value with etch portion of each edge detected image which has a value on one side of a predetermined threshold value, and associating a second value with all other portions of each edge detected image prior to combining the images using the logical OR operation.
13. The apparatus according to Claim 12, wherein the first and second values are binary 1 and 0.
14. The apparatus as claimed in any one of Claims 10 to 13, wherein the means for adaptively training performs a function analogous to a Widrow-Hoff training algorithm.
15. The apparatus according to any one of Claims 10 to 14, wherein the means for determining includes means for allowing manual selection based on the appearance of the product associated with the image.
16. The apparatus according to any one of Claims 10 to 15, characterized by means for adaptively training the filter with an image associated with a further processed value which is in the first range.
17. The apparatus according to any one of Claims 10 to 16, characterized by means for forming the dot product of the filter and each of the second set of images to form the processed values of each image in the said second set.
18. The apparatus according to any one of Claims 10 to 17, characterized by a video camera for forming at least one of the first, second and third sets of images, and by means for positioning a product in the field of view of the video camera.
19. The apparatus according to Claim 18, characterized by means for illuminating the product in the field of view of the video camera.
20. The apparatus according to Claim 18 or 19, wherein the positioning means comprises a conveyor for conveying products one after another through the field of view of the video camera.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US661,809 | 1991-02-27 | ||
| US07/661,809 US5146510A (en) | 1989-02-09 | 1991-02-27 | Methods and apparatus for optically determining the acceptability of products |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2061865A1 CA2061865A1 (en) | 1992-08-28 |
| CA2061865C true CA2061865C (en) | 2004-10-26 |
Family
ID=24655202
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002061865A Expired - Fee Related CA2061865C (en) | 1991-02-27 | 1992-02-26 | Methods and apparatus for optically determining the acceptability of products |
Country Status (10)
| Country | Link |
|---|---|
| US (1) | US5146510A (en) |
| EP (1) | EP0501784B1 (en) |
| JP (1) | JP3126474B2 (en) |
| KR (1) | KR100225836B1 (en) |
| AT (1) | ATE173554T1 (en) |
| CA (1) | CA2061865C (en) |
| DE (1) | DE69227603T2 (en) |
| DK (1) | DK0501784T3 (en) |
| ES (1) | ES2124241T3 (en) |
| HK (1) | HK1013872A1 (en) |
Families Citing this family (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA1311063C (en) * | 1988-12-16 | 1992-12-01 | Tokumichi Murakami | Digital signal processor |
| US5353356A (en) * | 1989-02-09 | 1994-10-04 | Waugh Richard M | Product gauge methods and apparatus for use in the optical determination of the acceptability of products |
| US5237621A (en) * | 1991-08-08 | 1993-08-17 | Philip Morris Incorporated | Product appearance inspection methods and apparatus employing low variance filter |
| CA2132515C (en) * | 1992-03-20 | 2006-01-31 | Glen William Auty | An object monitoring system |
| US5822208A (en) * | 1994-04-12 | 1998-10-13 | Bay Instrumentation & Technology Co. | Method and apparatus for predicting errors in a manufacturing process |
| US6201576B1 (en) * | 1998-06-26 | 2001-03-13 | Lucent Technologies Inc. | Apparatus and method for detecting an NTSC signal in an HDTV transmission signal |
| US6169600B1 (en) | 1998-11-20 | 2001-01-02 | Acuity Imaging, Llc | Cylindrical object surface inspection system |
| DE19938964A1 (en) * | 1999-08-17 | 2001-03-08 | Siemens Ag | Method and device for video monitoring of process plants |
| JP4743805B2 (en) * | 2000-04-06 | 2011-08-10 | ルネサスエレクトロニクス株式会社 | Appearance inspection method and apparatus |
| US7668362B2 (en) * | 2000-05-03 | 2010-02-23 | Aperio Technologies, Inc. | System and method for assessing virtual slide image quality |
| US9092841B2 (en) | 2004-06-09 | 2015-07-28 | Cognex Technology And Investment Llc | Method and apparatus for visual detection and inspection of objects |
| US7545949B2 (en) * | 2004-06-09 | 2009-06-09 | Cognex Technology And Investment Corporation | Method for setting parameters of a vision detector using production line information |
| WO2005008517A1 (en) * | 2003-07-18 | 2005-01-27 | Commonwealth Scientific And Industrial Research Organisation | A method and system for selecting one or more variables for use with a statistical model |
| JP5134365B2 (en) | 2004-05-27 | 2013-01-30 | アペリオ・テクノロジーズ・インコーポレイテッド | System and method for generating and visualizing a three-dimensional virtual slide |
| US8127247B2 (en) * | 2004-06-09 | 2012-02-28 | Cognex Corporation | Human-machine-interface and method for manipulating data in a machine vision system |
| US20050276445A1 (en) * | 2004-06-09 | 2005-12-15 | Silver William M | Method and apparatus for automatic visual detection, recording, and retrieval of events |
| US8891852B2 (en) | 2004-06-09 | 2014-11-18 | Cognex Technology And Investment Corporation | Method and apparatus for configuring and testing a machine vision detector |
| US7636449B2 (en) | 2004-11-12 | 2009-12-22 | Cognex Technology And Investment Corporation | System and method for assigning analysis parameters to vision detector using a graphical interface |
| US9292187B2 (en) | 2004-11-12 | 2016-03-22 | Cognex Corporation | System, method and graphical user interface for displaying and controlling vision system operating parameters |
| US7720315B2 (en) | 2004-11-12 | 2010-05-18 | Cognex Technology And Investment Corporation | System and method for displaying and using non-numeric graphic elements to control and monitor a vision system |
| US20060116902A1 (en) * | 2004-11-29 | 2006-06-01 | Canon U.S.A., Inc. | Method and apparatus for workflow |
| US8164622B2 (en) | 2005-07-01 | 2012-04-24 | Aperio Technologies, Inc. | System and method for single optical axis multi-detector microscope slide scanner |
| DE102005053224A1 (en) | 2005-11-06 | 2007-05-24 | Prt Patent Registration Service & Trade Ltd. | Device for non-invasive removal of strontium from human body, comprises two electrodes, switch control, source of DC-voltage and transistor |
| DE202006020357U1 (en) | 2006-09-21 | 2008-10-30 | Safetec Gmbh | Electrode and device for electro-foot bath |
| US8103085B1 (en) | 2007-09-25 | 2012-01-24 | Cognex Corporation | System and method for detecting flaws in objects using machine vision |
| US8873862B2 (en) * | 2011-06-28 | 2014-10-28 | Rakuten, Inc. | Product image processor, product image processing method, information recording medium, and program |
| US9651499B2 (en) | 2011-12-20 | 2017-05-16 | Cognex Corporation | Configurable image trigger for a vision system and method for using the same |
| US10297020B2 (en) | 2015-09-18 | 2019-05-21 | Datalogic Ip Tech S.R.L. | Stereoscopic system and method for quality inspection of cigarettes in cigarette packer machines |
| DE102019104822A1 (en) * | 2019-02-26 | 2020-08-27 | Wago Verwaltungsgesellschaft Mbh | Method and device for monitoring an industrial process step |
Family Cites Families (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3835332A (en) * | 1973-06-04 | 1974-09-10 | Eastman Kodak Co | Inspection apparatus for detecting defects in a web |
| JPS5218136A (en) * | 1975-08-01 | 1977-02-10 | Hitachi Ltd | Signal processing unit |
| US4030068A (en) * | 1976-01-12 | 1977-06-14 | Decision Data Computer Corporation | Optical character recognition system |
| GB1545117A (en) * | 1976-05-25 | 1979-05-02 | Nat Res Dev | Comparison apparatus eg for use in character recognition |
| US4053056A (en) * | 1976-07-19 | 1977-10-11 | Amf Incorporated | Cigarette package inspection apparatus |
| US4097845A (en) * | 1976-11-01 | 1978-06-27 | Rush-Presbyterian-St. Luke's Medical Center | Method of and an apparatus for automatic classification of red blood cells |
| US4110737A (en) * | 1977-08-22 | 1978-08-29 | The Singer Company | Character recognition apparatus for serially comparing an unknown character with a plurality of reference characters |
| DE3070721D1 (en) * | 1980-12-18 | 1985-07-04 | Ibm | Process for inspecting and automatically classifying objects presenting configurations with dimensional tolerances and variable rejecting criteria depending on placement, apparatus and circuits therefor |
| US4482971A (en) * | 1982-01-18 | 1984-11-13 | The Perkin-Elmer Corporation | World wide currency inspection |
| GB8331248D0 (en) * | 1983-11-23 | 1983-12-29 | Kearney & Trecker Marwin Ltd | Inspecting articles |
| US4589141A (en) * | 1984-03-12 | 1986-05-13 | Texas Instruments Incorporated | Apparatus for automatically inspecting printed labels |
| JP2607457B2 (en) * | 1984-09-17 | 1997-05-07 | 株式会社東芝 | Pattern recognition device |
| US4759074A (en) * | 1986-10-28 | 1988-07-19 | General Motors Corporation | Method for automatically inspecting parts utilizing machine vision and system utilizing same |
| IT1207600B (en) * | 1987-01-20 | 1989-05-25 | Gd Spa | METHOD FOR ELECTRIC OPTICAL CONTROL OF PACKAGES |
| JPS63257083A (en) * | 1987-04-14 | 1988-10-24 | Nec Corp | Pattern recognition system |
| US5014327A (en) * | 1987-06-15 | 1991-05-07 | Digital Equipment Corporation | Parallel associative memory having improved selection and decision mechanisms for recognizing and sorting relevant patterns |
| US4952805A (en) * | 1987-08-20 | 1990-08-28 | Fuji Photo Film Co., Ltd. | Method of judging the presence or absence of a limited irradiation field, method of selecting a correct irradiation field, and method of judging correctness or incorrectness of an irradiation field |
| JP2597370B2 (en) * | 1987-10-14 | 1997-04-02 | 株式会社ヒューテック | Method for detecting significant differences in sheet-like test materials |
| JPH0782542B2 (en) * | 1988-01-29 | 1995-09-06 | 株式会社スキャンテクノロジー | Print inspection method, print inspection device, and automatic printed matter distribution system |
| JPH01269034A (en) * | 1988-02-26 | 1989-10-26 | R J Reynolds Tobacco Co | Package inspection system |
| US4972494A (en) * | 1988-02-26 | 1990-11-20 | R. J. Reynolds Tobacco Company | Package inspection system |
| GB8810357D0 (en) * | 1988-04-30 | 1988-06-08 | Univ Manchester | Data analysis method & apparatus |
| US4952062A (en) * | 1988-08-05 | 1990-08-28 | Bean Iii Vern W | Method and apparatus for detecting flaws in fabric |
| US4975972A (en) * | 1988-10-18 | 1990-12-04 | At&T Bell Laboratories | Method and apparatus for surface inspection |
| US4972262A (en) * | 1988-10-27 | 1990-11-20 | Honeywell Inc. | Real time edge detection |
| US4974261A (en) * | 1988-11-15 | 1990-11-27 | Matsushita Electric Works, Ltd. | Optical surface inspection method |
| US5046111A (en) * | 1989-02-09 | 1991-09-03 | Philip Morris Incorporated | Methods and apparatus for optically determining the acceptability of products |
-
1991
- 1991-02-27 US US07/661,809 patent/US5146510A/en not_active Expired - Lifetime
-
1992
- 1992-02-26 DE DE69227603T patent/DE69227603T2/en not_active Expired - Fee Related
- 1992-02-26 ES ES92301631T patent/ES2124241T3/en not_active Expired - Lifetime
- 1992-02-26 DK DK92301631T patent/DK0501784T3/en active
- 1992-02-26 AT AT92301631T patent/ATE173554T1/en not_active IP Right Cessation
- 1992-02-26 EP EP92301631A patent/EP0501784B1/en not_active Expired - Lifetime
- 1992-02-26 CA CA002061865A patent/CA2061865C/en not_active Expired - Fee Related
- 1992-02-27 JP JP04078540A patent/JP3126474B2/en not_active Expired - Fee Related
- 1992-02-27 KR KR1019920003044A patent/KR100225836B1/en not_active IP Right Cessation
-
1998
- 1998-12-24 HK HK98115330A patent/HK1013872A1/en not_active IP Right Cessation
Also Published As
| Publication number | Publication date |
|---|---|
| US5146510A (en) | 1992-09-08 |
| EP0501784A3 (en) | 1994-02-16 |
| HK1013872A1 (en) | 1999-09-10 |
| JPH05126757A (en) | 1993-05-21 |
| DK0501784T3 (en) | 1999-06-23 |
| DE69227603D1 (en) | 1998-12-24 |
| ATE173554T1 (en) | 1998-12-15 |
| JP3126474B2 (en) | 2001-01-22 |
| ES2124241T3 (en) | 1999-02-01 |
| KR920016977A (en) | 1992-09-25 |
| CA2061865A1 (en) | 1992-08-28 |
| DE69227603T2 (en) | 1999-06-24 |
| EP0501784A2 (en) | 1992-09-02 |
| EP0501784B1 (en) | 1998-11-18 |
| KR100225836B1 (en) | 1999-10-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2061865C (en) | Methods and apparatus for optically determining the acceptability of products | |
| US5353356A (en) | Product gauge methods and apparatus for use in the optical determination of the acceptability of products | |
| US5189708A (en) | Methods and apparatus for optically determining the acceptability of products | |
| Elster | Detection of cracks in eggs using machine vision | |
| US5818953A (en) | Optical characterization method | |
| Edreschi et al. | Classification of potato chips using pattern recognition | |
| US5732147A (en) | Defective object inspection and separation system using image analysis and curvature transformation | |
| Miller et al. | Peach defect detection with machine vision | |
| US5537670A (en) | Product appearance inspection methods and apparatus employing low variance filter | |
| CA2093310A1 (en) | Product inspection method and apparatus | |
| JPH0411073B2 (en) | ||
| US5924575A (en) | Method and apparatus for color-based sorting of titanium fragments | |
| US5208870A (en) | Image inspection methods and apparatus | |
| Tao | Spherical transform of fruit images for on-line defect extraction of mass objects | |
| Uluişik et al. | Image processing based machine vision system for tomato volume estimation | |
| Mohamed et al. | Development of a real-time machine vision prototype to detect external defects in some agricultural products | |
| Omid et al. | Implementation of an efficient image processing algorithm for grading raisins | |
| JPH05332754A (en) | Inspecting method for inspection device | |
| Wen et al. | Brightness-invariant image segmentation for on-line fruit defect detection | |
| JP3962525B2 (en) | Image generation device and sorting classification determination device used for sorting agricultural products | |
| JP2003216930A (en) | Method and apparatus for inspecting discoloration | |
| Eaton et al. | Colour analysis of strawberries on a real time production line | |
| JP4002387B2 (en) | Agricultural product grade judging method and grade judging device | |
| Nie et al. | Segmentation method of citrus surface defect based on brightness correction | |
| Budhi et al. | The Identification of the Apples (Malus Sylvestris) Skin Wax Coating Using the Edge Detection Method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |