EP1026516B1

EP1026516B1 - Method and apparatus for analyzing an ultrasonic image of a carcass

Info

Publication number: EP1026516B1
Application number: EP00300805A
Authority: EP
Inventors: Yujun Liu; James R. Stouffer; Greg Snider
Original assignee: Micro Beef Technologies Ltd
Current assignee: Micro Beef Technologies Ltd
Priority date: 1999-02-05
Filing date: 2000-02-02
Publication date: 2006-10-04
Anticipated expiration: 2020-02-02
Also published as: CA2298074C; DE60031021D1; US6099473A; EP1026516A3; CA2298074A1; ATE341767T1; EP1026516A2

Abstract

The present disclosure teaches a system for analyzing ultrasonic image that provides an output of a measurement of muscle width from an ultrasonic image input of an outline of a muscle from an animal or carcass. The muscle that is used in the preferred embodiment is the longissimus dorsi muscle when an ultrasonic scan is taken in a transverse direction relative to the backbone. The analysis is done with a computer that receives the electronic input of rows and columns of gray level pixel data from an ultrasonic scan image of the outline of the muscle of the animal or carcass. The software is set to select a region of the ultrasonic image input to analyze to determine a first edge of the muscle. The selected region is divided into subregions Sj,k. J designates a row and ranges between 1 and n. K designates a column and ranges between 1 and o such that o is greater than 1. The subregions are aligned in rows and columns throughout the ultrasonic image input. The software calculates a sum of the gray level pixel data for each of the subregions Sj,k then compares the sums to determine which of the subregions Sj,k has the highest sum within each row j. The software defines a position of the first edge of the muscle by comparing the highest sum within each row j. This position is then used to calculate a relative muscle width when compared to a defined second edge of the muscle. The second edge can be defined as one edge of the selected region of the ultrasonic image input or it can be defined using the same steps used to define the first edge. <IMAGE>

Description

FIELD OF THE INVENTION

The invention pertains to the field of ultrasonic animal and carcass evaluation, and more particularly relates to analyzing an ultrasonic image of an animal or carcass.

BACKGROUND OF THE INVENTION

Evaluating and grading meat animals, both live and slaughtered, has historically been performed by humans. Because of this it is very difficult to achieve accuracy, efficiency and consistency. Both producers and packers demand an objective means of classifying their animals accurately according to their carcass real values. However, since an accurate, quick, and consistent grading system has not been put into place, producers are not being paid for the true value of their animals. Currently, producers are paid on an average basis. The price differential between a high-yield and a low-yield grade is less than it should be. Therefore, it is important to the hog and beef industries that improved or new technologies must be developed in their evaluation systems in order to be able to accurately measure the hog and beef carcass characteristics that are of significant value.
Labor costs and inconsistent grading are significant problems in the meat processing industry. Attempts have been made to automate the grading and inspection systems involved in meat processing. For example see Patent no. 4,931,933, entitled, "Application of Knowledge-Based System for Grading Meat" granted to Chen et al, and Patent Number 5,079,951, entitled "Ultrasonic Carcass Inspection" granted to Raymond et al. However, these systems are overly complicated and do not provide an efficient method of accurately measuring the Longissimus dorsi muscle depth and fat composition.
European patent application number 0337661, published Octobre 18, 1989, entitled "Method and apparatus for grading of live animals and live carcases", discloses ultrasonic scanning of live animals and carcasses, particularly for grading purposes.
The Longissimus dorsi muscle is one of the most valuable portions of beef or pork and is also an excellent indication of the value of the rest of the animal or carcass. Therefore, most analysis of animals or carcasses with ultrasound concentrates on this muscle.
Ultrasonic images of the Longissimus dorsi (rib eye muscle in beef and loin eye muscle in hogs) have been used to evaluate livestock. U.S. Patent No. 5,339,815 (Liu et al.) discloses a method and apparatus wherein an ultrasonic transducer is centered in a longitudinal direction over the last few ribs of the animal or carcass and the ultrasonic image is of a ribline, a Longissimus dorsi muscle and fat layers above the muscle such that the specified window starts below the ribline of the animal or carcass. A fat depth is determined from a distance between the second interface line and a specified plane of contact between the animal or carcass and the ultrasonic transducer adjusted for any positioning equipment or stand-off gel. A muscle depth is determined from a distance between the first and second interfaces line. The output of the system includes the fat depth and the muscle depth for the animal or carcass from which the image was taken.
Longissimus dorsi or ribeye muscle cross-sectional area is currently obtained by manually tracing around the perceived outline of the muscle from an ultrasonic image. Some ultrasonic scanners, like the latest model we have been using [Aloka, 1990a], provide the capability of approximating the area with an ellipse. Due to its low degree of accuracy and the relatively large time requirement, this feature is seldom used. It is, however, more common for the images to be recorded on a video tape and the area analysis done at a later time. This analysis is still a very time consuming process. Because of the quality of the image, accurately tracing the 1.d. muscle area can be done only by trained technicians. It is, therefore, very difficult to achieve efficiency, consistency and accuracy. The teachings of Patent No. 5, 339,815 provided a method to automatically determine the area of the muscle when the ultrasonic scan image input is transverse with respect to a backbone of the animal or carcass. However, there are faster and easier methods for measuring the width of the muscle.

SUMMARY OF THE INVENTION

The present invention teaches a system for analyzing ultrasonic image that provides an output of a measurement of muscle width from an ultrasonic image input of an outline of a muscle from an animal or carcass. The muscle that is used in the preferred embodiment is the longissimus dorsi muscle when an ultrasonic scan is taken in a transverse direction relative to the backbone. The analysis is done with a computer that receives the electronic input of rows and columns of gray level pixel data from an ultrasonic scan image of the outline of the muscle of the animal or carcass.
The software is set to select a region of the ultrasonic image input to analyze to determine a first edge of the muscle. The selected region is divided into subregions Sj_,k. J designates a row and ranges between 1 and n. K designates a column and ranges between 1 and o such that o is greater than 1. The subregions are aligned in rows and columns throughout the ultrasonic image input. The software calculates a sum of the gray level pixel data for each of the subregions Sj,k then compares the sums to determine which of the subregions Sj,k has the highest sum within each row j. The software defines a position of the first edge of the muscle by comparing the highest sum within each row j. This position is then used to calculate a relative muscle width when compared to a defined second edge of the muscle. The second edge can be defined as one edge of the selected region of the ultrasonic image input or it can be defined using the same steps used to define the first edge.
The system is intended to be used in combination with a system for determining relative muscle depth. When used in combination the system can calculate a relative muscle area from the relative muscle width and relative muscle depth. Furthermore, the system can compare the relative muscle area to a measured weight of the animal or carcass and assign a relative value for use in further production of the animal or processing of the carcass.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1: shows a representative selected region of an ultrasonic image for analysis.
Fig. 2: shows a representation of an ultrasonic transducer positioned to scan a longissimus dorsi muscle, with a transducer positioned in a transverse direction with respect to an animal's backbone.
Fig. 3: is a flow chart of the basic steps for determining an interface within the image.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Several computer algorithms have been developed for subcutaneous fat thickness, longissimus dorsi muscle area, and intramuscular fat or marbling measurements from ultrasonic images of live animal and carcasses. The applicant's have previously developed a method of automatically determining the depth of the longissmus dorsi muscle in an ultrasonic image and were granted U.S. Patent No. 5,339,815 for this system. The present invention expands upon the teachings of this patent by teaching a method of determining a relative width of the longissimus dorsi muscle area in an ultrasonic image.
The invention disclosed in U.S. Patent No. 5,339,815 (Liu et al.), introduces an automated system for ultrasonic carcass evaluation that uses the disclosed sliding z test for depth measurement and then measures the width of the muscle as set forth below. In essence, there is a difference in the ultrasound images of the muscle and the tissue adjacent the muscle. For determining the depth of the muscle, the edge detection technique involves finding the region where the difference between neighboring pixels reaches the computed threshold. The threshold is computed based on mean difference and standard deviation. This technique for determining the muscle depth always finds the region with a large gray value change that may not be the brightest spot.
For determining the width, the edge detection technique is to search the region with the highest average of gray values. The brightest area in average is found, which represents the interface between the muscle and surrounding tissue.
For this application, the term "AutoD" refers to the implementation of the technique disclosed in the '815 patent. The term "AutoW" refers to the implementation of the technique which is the subject of the present disclosure.
The present invention can be used in a variety of embodiments. In the way of example, the study of the longissimus dorsi muscle of swine is discussed herein. The characteristics of this muscle are currently the most important in determining the value of an animal or carcass. Specifically, the longitudinal images with respect to backbone of the animal or carcass are used in evaluation. Various positioning devices can assist in proper and consistent positioning of the ultrasonic transducer.

The Depth Measurement

Various edge detection algorithms for processing images have been developed, and most of them are typically modeled to detect a step edge in some optimal way. Detecting ideal step edges is a simple process. In real images, however, noise corrupts the edges and thus makes the edge detection a rather complicated process.
The AutoD system uses a process of measuring the distance between two detected boundaries and applies this technique as a method for on-line high speed evaluation of fat thickness in animal carcasses. Computational efficiency, therefore, is the major concern in developing the technique. The one-dimensional sliding Z-test algorithm is used for the AutoD portion of the present invention for both edge and peak detection for the depth component. This algorithm requires the standard deviation of the whole area of interest in an image to be calculated only once. It not only simplifies the problem of edge detection but it is also very computationally efficient. The AutoD system and the Z-test are described extensively in U.S. Patent No. 5,339,815.

The Width Measurement

AutoD has proven to be an extremely useful tool in evaluating carcasses and livestock. In an expansion of this utility a method of automatically determining the width of the muscle was developed. The automatic width method (AutoW) searches for the left or right end of the muscle from an ultrasonic image. It assumes that there is an interface at the end of the muscle which produces the brightest image.
The ultrasonic image is composed of rows and columns of pixels that have a particular grey value. The higher the grey value the "brighter" the image for a particular pixel. Generally speaking, evaluation of animals and carcasses is performed with transducers and ultrasound equipment that generate a rectangular image. The rectangular image includes the region of the muscle to be examined. Depending upon the application, a smaller portion of the total image may be selected for analysis.
In any event, the image, or a portion thereof, is selected for analysis is subdivided into smaller regions. Typically the selected region is divided into 5 equally spaced rows and a large number of columns about 3 pixels wide, as shown in Fig, 1, although each of these parameters can be changed by the user.
The following variables are used in the calculation:

a) Y = total height in pixels of the selected image region
b) X = total width in pixels of the selected image region
c) n = the number of rows (subregions) in the selected image region
d) m = the width in pixels of the subdivided region

The image shown in Fig. 1 is Y pixels high and X pixels wide, such that the height of each subregion is Y/n. The value n is run-time defined with a default of 5. Changing n also affects the AutoD component of the invention.
For each subregion, the gray values of image pixels in m vertical lines (the default value of m is 3, the same default value used by AutoD) are summed up in a sliding fashion pixel by pixel from left to right. Each sum is from (m*Y/n) pixels. Therefore, there are (X / m) sums. The method is looking for the brightest spot as it "slides" from left to right along each of the rows. The position with the maximum of the sums (Xmax) is saved.
There is an Xmax for each of the n rows making up the sub regions. In total, there are n Xmax values, each from a subregion. A smoothed average of the n Xmax is assumed to be the end of the muscle. The smoothed average is runtime defined that is preferably one of the following:

1) Arithmetic average: (Xmax1 + Xmax2 + ... + Xmaxn) /n
2) Median: Sort Xmax in order. The median is the middle value if n is odd and the average of middle two values if n is even.
3) Trimmed average: Sort Xmax in order and trim off the minimum and maximum.

The Trimmed average is the average of the remaining n - 2 values.
The default method of calculating the smoothed average is Median. Changing the smoothed average can also affect AutoD. AutoW can be implemented with AutoD on or off, thereby affecting the size of the rectangular region of an image to search. All values are in pixels When AutoD is on, the automatic region bounded for searching the left end of the muscle is defined by

Top: (The y location of the fat and muscle interface line)+ 10
Left: (The x location of the left end of the fat and muscle interface line)
Bottom: Top + (Muscle depth in pixels) /3
Right: Left + (The length of the fat and muscle interface line) / 3

The automatic region bounded for searching the right end of the muscle is defined by

Top: (The y location of the fat and muscle interface line) + 10 + (Muscle depth in pixels) / 3
Left: (The x location of the left end of the fat and muscle interface line) + (The length of the fat and muscle interface line) / 3 * 2
Bottom: Top +Muscle depth in pixels) / 3 s
Right: Left + (The length of the fat and muscle Interface line) / 3

When AutoD is off, the automatic region bounded for searching the left end of the muscle is defined by

Top: 90
Left: 80
Bottom: Top + 240 / 3 = 170
Right: Left + 280/3 = 173

Top: 90 + 240 3 = 170
Left: 80 + 280 / 3 * 2 = 266
Bottom: Top + 240 / 3 = 250
Right: Left + 280 / 3 = 359

The pixel width measured by AutoW is the horizontal distance between the left and right ends of the muscle. This value is converted to the selected unit of measurement based on the probe calibration.
The analysis is done with a computer that receives the electronic input of rows and columns of gray level pixel data from an ultrasonic scan image of the outline of the muscle of the animal or carcass. The software is set to select a region of the ultrasonic image input to analyze to determine a first edge of the muscle. The selected region is divided into subregions S_j,k. J designates a row and ranges between 1 and n. K designates a column and ranges between 1 and o such that o is greater than 1. The subregions are aligned in rows and columns throughout the ultrasonic image input. The software calculates a sum of the gray level pixel data for each of the subregions Sj,k then compares the sums to determine which of the subregions Sj,k has the highest sum within each row j. The software defines a position of the first edge of the muscle by comparing the highest sum within each row j. This position is then used to calculate a relative muscle width when compared to a defined second edge of the muscle. The second edge can be defined as one edge of the selected region of the ultrasonic image input or it can be defined using the same steps used to define the first edge.
AutoW can be used either with AutoD or independently. When Auto W is activated in either case, the left or right side of an image is set to a fixed location if the parameter associated with that side is set to a non-negative value. There are six input parameters associated with AutoW:

1) Activate Check: When selected, AutoW is activated. When activated, AutoW will make a width measurement on an image after the image is capture or opened from a file. If AutoD is activated, AutoW will be made after AutoD. AutoW can be activated only if it is available and enabled.
2) Output Col: Enter a column code from A to IV (Default code is G). The AutoW measurement is in this column with 'aMW' as its default column heading. The user preferably can change any column heading to a more meaningful term.
3) Left: Enter any value from -511 to 511 in column pixels. When negative, AutoW searches for the left end of the image; otherwise, AutoW uses this value and sets it as the left end of the image. The default value is 76.
4) Right: Enter any value from -511 to 511 in column pixels, When negative, AutoW searches for the right end of the image; otherwise, AutoW uses this value and sets it as the right end of the image, The default value is -1.
5) Top: Enter any value from -479 to 479 in row pixels. When negative, AutoW searches within the AutoD muscle depth region if AutoD is activated and within rows from 80 to 240 if AutoD is not activated; otherwise, AutoW uses this value as its top and searches below it, The default value is -1.
6) Bottom: Enter any value from -479 to 479 in row pixels- When negative, AutoW searches within the AutoD muscle depth region if AutoD is activated and within rows from 80 to 240 if AutoD is not activated; otherwise, AutoW uses this value as its bottom and searches above it. The default value is-1.

Fig. 3 is a flow chart of the basic steps to determining an interface within the image. An input is provided of an ultrasonic scan image of the muscle and fat area of the animal or carcass comprising rows and columns of pixel data. (Box 20) A window of rows and columns of pixels within the image input is selected. (Box 21) The window is divided into subregions (Box 22) both horizontally and vertically and the sums of each subregion is determined (Box 23). The max Sum for each subregion within a horizontal region is determined (Box 24). Then the position of the max Sum for each horizontal region is compared to the other horizontal regions (Box 25). Finally and an average of the max Sums is used to determine the position of the right side of the muscle (Box 26).

Application

The present invention teaches a method of automatically recognizing fat and muscle interfaces of an animal or carcass from an ultrasonic image of a muscle and fat area. Fig. 2 shows a representation of the positioning of a transducer 5 in a transverse direction with respect to the animal's backbone, specifically a beef carcass. The transducer 5 is positioned such that the left side of the image runs through an indent 7 in the top of the 1.d. muscle and continues through the bottom left corner of the muscle 8. The line between these two points are marked a cross hatched line 10.
From empirical study it has been determined that the proportion of the muscle to the left of the line is the same relative to the total muscle. Therefore, for speed in analysis and for consistent operation, the preferred embodiment is to have the user position the transducer such that the left side of the ultrasonic image starts along this line 10. Therefore, the width of the muscle measured by assuming that the left side of the ultrasonic image is the left side of the muscle and then determining the position right side of the muscle 12. This way the computer does not have to search for both sides of the muscle. This is true for both live animals and carcasses.
In addition, the area of the muscle is calculated by determining the area between the left side of the image, the right side of the muscle and the top and bottom of the muscle. This area is roughly a rectangle, but the muscle is slightly more elliptical in reality. This is also fairly consistent between animals and a standard proportion of the measured area is actually muscle.
The analysis can correct for this and the portion of the muscle to the left of the line 10, however, the importance of the invention is to provide an objective measurement that can be compared to the same measurement made in other carcasses or animals for determining a relative value. In other words, if the measurement is off by a certain percentage it does not matter so long as the measurement is off by that percentage for all measurements. The producers and processors are concerned about percent lean and relative size of the 1.d. muscle when compared to the overall weight of the animal or carcass.
This invention may be used alone, but the preferred implementation is to use the AutoW in a combined system with AutoD and other ultrasound analysis tools (e.g. marbling or intra muscular fat analysis) for both live animal and carcass evaluation. Some processors are already using % lean as measured by AutoD to determine how much to pay producers.
The teachings of the present invention are efficient enough to be implemented in a real time system. The transducer can be positioned manually or automatically on carcasses or animals and then the images can be processed fast enough to allow real time evaluation and sorting. This is extremely important in a practical application of the present invention. Meat processors or breeders will be able to use the present system to sort animals or carcasses based upon the information provided. Such efficient sorting can result in a more profitable processing of carcasses in that only the more valuable carcasses will be selected to go through the more expensive processing steps. Breeders can efficiently select stock for breeding or slaughter based upon the information provided by the present system.
The system can be built in such a way that it can automatically make the decision as to whether or not there is a valid image, regardless of the existence of an animal or carcass identification on the image. Freezing and releasing an image does not alter the previous inputs to the surrounding area including the ID field. This decision must also be made fast enough for near real-time operation since all the input information will be lost during the decision making period. Hence, the algorithm used for this purpose must be simple but efficient.
If the interval image between two animals or carcasses is black or very low in image intensity, compared with a normal ultrasonic image from a animal or carcass, then the image intensity can be used to verify whether or not there is a desired image. By analyzing the images, it was found that normal ultrasonic images had average gray values greater than 30, about 12% of the maximum intensity. Although the image intensity can be controlled by the machine operator, an image with intensity lower than 12% of the maximum is hardly visible. This is a very simple mechanism for image verification but either too low or too high a threshold selected may result in a loss of useful image.
The timing for triggering a measurement depends on both the software execution speed and the on site speed of a particular application. For instance, the chain speed of a large modem commercial abattoir can be as high as 1200 hogs or 400 beef per hour. This speed must be matched for practical application of an automated system in commercial slaughter houses. Suppose that one set of the automated system is used for hogs in a packing plant which operates at the chain speed of 1200 carcasses per hour, and that an operator or robot is able to correctly locate the ultrasonic transducer and to obtain a quality image from each hog carcass passed by. This means that, with the image source passed through the image grabber board, the system must be capable of digitizing an image and making all pre-specified measurements within 3 seconds for each hog (3600 seconds /1200 hogs).
Image Capture Hardware. The image capture hardware used for the verification of the teachings of the present invention included the Cortex-I and CX100 from ImageNation and a video digitizer PCMIA card from MRT Micro, Inc. of Del Ray Beach, Florida. Once the ultrasonic image is digitized using these image digitizing devices, the AutoD and AutoW analyses no longer depend on the image capture hardware.
Computer Hardware. The system used a portable 486 PC and a Pentium PC.
Software requirement. Microsoft Visual C++ ver. 1.5 was used to develop the AUSKey software. The final product is a stand-alone executable software package whose Windows version runs on Windows 3.1 or higher and whose DOS version runs on DOS 3.0 or higher.
Ultrasonic Equipment. The equipment used to acquire ultrasonic images from beef and swine was a real time ultrasonic scanner Aloka SSD-500V with 3.5 Mhz linear array transducers [Aloka, 1990a, 1990b and 1990c]. The images can be recorded in VHS video tapes with a regular video cassette recorder and then played back for processing. The video output on the ultrasonic unit will normally connect to the image grabber board for immediate processing in an permanent on-line operation.

Claims

A method of muscle width measurement from an ultrasonic image of the outline of a muscle of an animal or carcass, comprising the following steps:
a) receiving an input of rows and columns of gray level pixel data from the ultrasonic scan image including the outline of the muscle of the animal or carcass into an electronic input device operatively connected to a computer processor,

b) selecting a region of the ultrasonic image input to analyze in order to determine a first edge of the muscle,

c) dividing the region selected in step b) into subregions S_j,k, wherein j designates a row and ranges between 1 and n and wherein k designates a column and ranges between 1 and o such that o is greater than 1, such that the subregions are aligned in rows and columns throughout the ultrasonic image input,

d) calculating the sum of the gray level pixel data for each of the subregions S_j,k,

e) comparing the sums of the gray level pixel data for each of the subregions S_j,k to determine which of the subregions S_j,k has the highest sum of the gray level pixel data within each row j,

f) comparing the subregions S_j,k with the highest sum of the gray level pixel data for each row j to define the position of the first edge of the muscle, and

g) providing a measurement of the relative width of the muscle, by using the defined position of the first edge of the muscle and comparing the position to a second edge of the muscle as output.
The method of claim 1, wherein the second edge is defined as one edge of the selected region of the ultrasonic image input.
The method of claim 1 wherein the second edge is defined using the same steps used to define the first edge.
The method of claim 1, wherein the muscle is a longissimus dorsi muscle and the ultrasonic image input is an ultrasonic image input of the longissimus dorsi muscle in a transverse direction with respect to the backbone of the animal or carcass.
The method of claim 1, wherein the method is used in combination with a method for determining relative muscle depth.
The method of claim 5, wherein a relative muscle area is calculated from the relative muscle width and relative muscle depth.
The method of claim 6, wherein the relative muscle area is compared to a measured weight of the animal or carcass and assigned a relative value for use in further production of the animal or processing of the carcass.
A system of muscle width measurement from an ultrasonic image of the outline of a muscle of an animal or carcass comprising:
a computer having a computer processor operatively connected to a storage device; and

an electronic input device operatively connected to the computer processor for receiving an input of rows and columns of gray level pixel data from an ultrasonic scan image including the outline of the muscle of the animal or carcass;
wherein the computer carries out a method comprising the steps of:
a) selecting a region of the ultrasonic image input to analyze in order to determine a first edge of the muscle;

b) dividing the selected region of the ultrasonic image input into subregions S_j,k, wherein j designates a row and ranges between 1 and n and wherein k designates a column and ranges between 1 and o such that o is greater than 1, such that the subregions are aligned in rows and columns throughout the ultrasonic image input;

c) calculating a sum of the gray level pixel data for each of the subregions S_j,k;

d) comparing the sums of the gray level pixel data for each of the subregions S_j,k to determine which of the subregions S_j,k has the highest sum of the gray level pixel data within each row j;

e) comparing the subregions S_j,k with the highest sum of the gray level pixel data for each row j to define the position of the first edge of the muscle; and

f) measuring the relative width of the muscle, by using the defined position of the first edge of the muscle and comparing the position to a second edge of the muscle as output.
The system of claim 8 wherein the second edge is defined as one edge of the selected region of the ultrasonic image input.
The system of claim 8 wherein the second edge is defined using the same steps used to define the first edge.
The system of claim 8, wherein the muscle is a longissimus dorsi muscle and the ultrasonic image input is an ultrasonic image input of the longissimus dorsi muscle in a transverse direction with respect to the backbone of the animal or carcass.
The system of claim 8, wherein the system is used in combination with a system for determining relative muscle depth.
The system of claim 12, wherein the method further comprises the step of calculating a relative muscle area from the relative muscle width and relative muscle depth.
The system of claim 13, wherein the method further comprises the steps of comparing the relative muscle area to a measured weight of the animal or carcass and assigning a relative value for use in further production of the animal or processing of the carcass.