US7065235B2 - Radiographic scoring method - Google Patents

Radiographic scoring method Download PDF

Info

Publication number
US7065235B2
US7065235B2 US10/237,311 US23731102A US7065235B2 US 7065235 B2 US7065235 B2 US 7065235B2 US 23731102 A US23731102 A US 23731102A US 7065235 B2 US7065235 B2 US 7065235B2
Authority
US
United States
Prior art keywords
interest
region
image
displayed
reference stage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US10/237,311
Other versions
US20030053673A1 (en
Inventor
Piet Dewaele
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Agfa HealthCare NV
Original Assignee
Agfa Gevaert NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Agfa Gevaert NV filed Critical Agfa Gevaert NV
Assigned to AGFA-GEVAERT reassignment AGFA-GEVAERT ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DEWAELE, PIET
Publication of US20030053673A1 publication Critical patent/US20030053673A1/en
Application granted granted Critical
Publication of US7065235B2 publication Critical patent/US7065235B2/en
Assigned to AGFA HEALTHCARE N.V. reassignment AGFA HEALTHCARE N.V. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AGFA-GEVAERT N.V.
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10116—X-ray image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H70/00—ICT specially adapted for the handling or processing of medical references
    • G16H70/60—ICT specially adapted for the handling or processing of medical references relating to pathologies

Definitions

  • the present invention relates to computer-assisted radiological scoring of radiographic images.
  • Radiographs can be captured to detect and identify lesions, to diagnose an underlying pathology from radiological manifestations, to assess the existence and extent of a fracture etc.
  • the radiographer or clinician already knows the pathology but wishes to assess the degree of severity of the disease or the developmental stage of the patient.
  • rating refers to the process of assigning a stage to an anatomical site and scoring refers to the attribution of a value to the stage, we will call both processes ‘scoring’ in the sequel.
  • Radiological scoring methods which adhere to the aforementioned principles, are e.g. bone age determination (also known as skeletal maturity assessment), rheumatoid arthritis scoring, osteo-arthritis scoring, ankylosing spondylitis scoring, and osteoporosis assessment.
  • a first example of an application wherein radiographic scoring is used, is skeletal maturity assessment, a procedure frequently performed in paediatric radiology.
  • a common method for assessing the skeletal maturity is the atlas method, and the hand-wrist was the first area of the body for which atlases became available.
  • the hand-wrist was the first area of the body for which atlases became available.
  • the most frequently used method is the Greulich and Pyle (GP) method.
  • the radiograph of non-dominant hand wrist is compared to a reference series of hand-wrist radiographs displayed in this atlas.
  • Each hand-wrist radiograph corresponds to a certain year of bone age.
  • the age interval between successive hand-wrist plates varies from 3 to 6 months.
  • the reference radiograph which globally compares best with the clinical image, is selected as the best match and its associated age is called the bone age.
  • This atlas-derived bone age is then compared with the chronological age of the patient, and the age difference is used for diagnostic purposes.
  • the comparison is a global one. Depending on the weight the radiologist attaches to the similarity or dissimilarity of specific skeletal sites on the hand-wrist radiograph, ambiguous results may be obtained.
  • TW 2 method Tanner and White house method
  • Scores are assigned to grades of skeletal maturity indicators, and the sum of scores is later transformed to a skeletal age (also called bone age). Slight modifications to the reference values and charts have recently been made to reflect changes in the population, resulting in the TW 3 method. Descriptions and manual rating of the stages of the bones, however, have not been altered.
  • the TW 2 method is currently effectuated in paediatric radiology as follows.
  • a radiograph from the non-dominant hand and wrist is made either by (a) conventional screen/film recording system, or (b) by digital means such as film digitisation, a computed radiography system or a direct radiography system (based on direct or indirect flat panel, or CCD).
  • a digital image on the other hand may be either printed on film and viewed conventionally on a light box, or it may be displayed on a viewing station.
  • the corresponding skeletal site in the actual hand-wrist radiograph is searched for visually. Specific salient anatomic details of the skeletal site are retained mentally.
  • the score, corresponding to the matching stage, is written down on paper or typed in a computer spreadsheet or database.
  • This iterative process of hypothesis generation and hypothesis verification as to the actual stage of a skeletal site is repeated s for every skeletal site.
  • the total score (on a scale between 0 . . . 1000) is translated into a bone age by a table look up operation. A different table is applied depending on the gender of the patient.
  • the bone age derived is compared with the chronological age of the patient and the difference between them is used as an element in the clinical diagnosis process.
  • Another example of an application wherein radiographic scoring is used, is rheumatoid arthritis.
  • RA scoring Conventional rheumatoid arthritis scoring (RA scoring) is based on comparison of the actual film radiography of hands and feet with reference pictures printed on film.
  • Scoring proceeds by displaying standard reference films and the actual radiography on a light box, and writing down scores associated with the matching stage of each joint.
  • Radiological assessment serves as a recognized standard for the evaluation of rheumatoid arthritis.
  • epidemiological and therapeutic evaluation scoring or grading systems have been proposed on X-ray images of specific skeletal sites.
  • X-ray images have the advantages to be able to record the history of the damage of a joint, because the damage is mostly irreversible. Previous images can be recalled in a later stage, to assess the evolution and to subject them to newly developed scoring systems. The effectivity of drugs can be traced.
  • a displayed reference stage image which most optimally matches with a displayed region of interest can be selected by
  • a reference stage image is an image which is representative for a developmental stage of a specific skeletal site. These reference stage images are commonly selected by an expert. In most cases these reference images are accepted as world wide standards.
  • the step of determining a region of interest is guided by means of a stencil consisting of a sketch of a body part of the patient that is to be evaluated.
  • the sketch depicts at least one of the outline of the following three items: the body part, bone contours of bones within the body part, anatomical contours, as well as region of interest delineations superimposed on said sketch.
  • a region of interest is determined and extracted on a display device by manual operation.
  • a digital image representation of a region of interest is extracted from a digital image representation of said image by re-sampling and interpolation.
  • a stencil is selected according to identification data of said radiographic image such as gender, age and examination type (hand, foot etc.).
  • the image data in the regions of interest in the displayed image are extracted.
  • the extracted regions of interest are subjected to rotation or to an affine transformation so as to have them in upright position.
  • Regions of interest may also be subjected to image enhancing processing (see below).
  • the ensemble of ROI's is subjected to the remainder of the scoring steps: display in proximity to the reference stage images, selection of the best matching reference stage image, associating of a score number and combining of score numbers of all regions of interest into an overall score number of the entire image.
  • This procedure is referred to as manual scoring procedure.
  • a semi-automated method can be applied wherein all steps of the method described higher are also applied but wherein additional decision support is generated to assist the selection of the optimally matching reference stage image.
  • additional support is generated to help to assess whether a specific feature is radiologically manifested in the region of interest. More specifically for each of the regions of interest (ROI) a sequence of specific image processing operators is applied to compute ROI specific feature values and to subject the computed feature values to a judgment device in order to determine presence or absence of a feature. Presence or absence of a feature will be used for assessing which reference stage image is most optimally matching.
  • ROI regions of interest
  • This system is called semi-automated scoring since the ultimate decision whether or not a computer-suggested decision is accepted, is still made by a radiologist. It may be possible in the current method that not for all regions of interest automatic feature based support is available. Furthermore it may also be possible that the features are not computed exhaustively for all stages of a specific region of interest.
  • Image processing is e.g. spatial reformatting and/or processing for the purpose of enhancing the intrinsic image quality.
  • multi scale contrast enhancement can be applied, as will be explained further on.
  • Another aspect of this invention relates to a computer program product as set out in the claims.
  • a computer program product is meant to encompass a software product in the form of an electric signal as well as a computer program on a computer readable medium such as a CD-ROM.
  • Still another aspect relates to a computer readable medium such as a CD-ROM comprising program code adapted to carry out the above method.
  • the ROI extraction and display in the neighborhood of the reference stage images alleviates the problem of repeated focus of attention towards the correct skeletal site.
  • the available dynamic range of the output medium can be used optimally to render all relevant image details.
  • the current invention is particularly suited to implement different variations of the Sharp score in rheumatoid arthritis, since the ROI stencil can be customized so as to include only an abbreviated set of ROI's.
  • a score based on a reduced number of ROI's, can also be derived from a superset including more ROI's having been evaluated previously.
  • the risk to evaluate a wrong ROI is substantially reduced, because the ROI stencil depicting fewer skeletal sites serves as a reliable hinting mechanism.
  • the task of focusing on a ROI and scoring is alternated for all skeletal sites.
  • the order has been changed, first all ROI's are determined and next the scoring is performed.
  • the invention is particularly useful since by first extracting regions of interest around each vertebra, the visual search process is greatly simplified. No confusion with other vertebrae is possible when presenting each ROI in juxtaposition with the reference radiographs and graphic pictures.
  • FIG. 1 illustrates the different steps of a scoring method according to the present invention
  • FIG. 2 shows a stencil used for bone age determination
  • FIG. 3 is an example of automatically segmented RUS bones, wherein FIG. 3A shows the initial bounding box placement for automatic segmentation of the associated bones and FIG. 3B shows the segmentation of the phalanxes and carpal bones.
  • the interface regions at the extremal ends of these bones determine the ROI's for bone age scoring,
  • FIG. 4 is a more detailed illustration of a specific embodiment of the method of the present invention.
  • FIG. 5 is a general overview of a scoring system
  • FIG. 6 is an example of a user interface with static ROI display
  • FIG. 7 is an example of a user interface with dynamic ROI display
  • FIG. 8 is an example of a longitudinal maturity study.
  • FIG. 1 A digital image representation of an image to be subjected to radiographic scoring is acquired ( FIG. 1 , numeral 1 ).
  • the radiation image can for example be recorded on a photostimulable phosphor screen.
  • the screen carrying the radiation image is then read out by scanning it with stimulating radiation and detecting the image-modulated light which is emitted upon stimulation and by converting the image-wise modulated light into a signal representation of the radiation image.
  • the radiation image can be stored in a direct radiation sensor rendering a signal representation of a radiation image.
  • Still further alternatives for acquiring a digital signal representation of a radiation image such as scanning of radiographic film carrying a radiation image, are possible.
  • the acquired image representation is next applied to a display device and the image is displayed ( FIG. 1 , numeral 1 ).
  • the ROI (region of interest) positioning stage at least one ROI is determined on the displayed image ( FIG. 1 , step 2 ).
  • the hand In the following explanation will be referred to the hand as a body part that is subjected to evaluation. However, the present invention is not limited to scoring methods using the hand as a body part to be evaluated.
  • the region of interest is the image region covering the joint or bone relevant for the examination.
  • the ROI positioning stage aims at positioning all regions of interest (ROI's) on the body part that is examined. Although each of the ROI's may be extracted separately and analyzed immediately, a more convenient work flow is obtained by first extracting all ROI's simultaneously before processing them.
  • ROI positioning can be performed in an operator-guided way or alternatively in an automatic way.
  • a method using a digital stencil can be used to facilitate the positioning.
  • a stencil is a digital sketch of a hand wrist image depicting the outline of the hand and all hand bone contours. The delineations of all possible ROI's are superimposed on these outlines.
  • a hand-wrist image is taken as an example.
  • Other body parts may be examined depending on the type of application and the radiographic scoring method.
  • these ROI outlines will typically be rectangular regions around joints of the tubular (radius, ulna, metacarpal and phalange) bones and around the carpal bones.
  • FIG. 2 shows a stencil used for bone age determination.
  • the stencil shows all rectangles of regions of interest (dashed lines) superimposed on an outline of the diagnostic entities.
  • Each of the ROI outlines is mapped or dragged by the operator (translated e.g. by mouse and arrow keys) to its actual location in the displayed image, and the stencil serves as a guideline to indicate the intended image region the ROI should encompass, with sufficient precision for further analysis.
  • Basic operations such as translation, rotation and scaling may be applied to the stencil ROI so that the resulting ROI fully comprises its associated hand bone without irrelevant bones being present which would make the identification of the target bone ambiguous.
  • the manual placement may be speeded up by first translating, rotating and scaling the stencil in its entirety so that it roughly coincides with the actual hand and bony outlines, and by further adjusting the position of the individual ROI's so that each ROI accurately covers its intended bone/joint location.
  • the ROI's may be sequentially displayed in the stencil and positioned in the actual region. Size and orientation of the rectangles may be modified e.g. by corner point dragging to match the actual size and the orientation of the displayed image.
  • the stencil must preferably reflect this in order to more faithfully represent the actual skeletal sites. Therefore, preferably a number of hand-wrist stencils are stored as a function of chronological age, gender, race etc. In accordance with the patient's identification data such as age, race, gender, etc. which may be entered into the computer, the corresponding sketch can be selected.
  • the chronological age may not be representative for the skeletal maturity status of the patient (the bone age scoring is precisely meant to detect significant differences from it), it serves as a rough estimate for the initial selection of the stencil.
  • the ROI positioning can be automated by image processing algorithms, so that no user interaction is required in the placement of the ROI regions.
  • Automatic placement can be based on first automatically segmenting the bones based on Point Distribution Models (PDM) and Active Shape Models (ASM) such as disclosed in the prior art (Cootes TF and Taylor CJ, Active Shape Models—“smart snakes”, Proc. British Machine Vision Conference, Springer Verlag 1992, p. 256–275).
  • PDM Point Distribution Models
  • ASM Active Shape Models
  • An example for segmenting the individual vertebrae in the spine is given in (Smyth et al, Automatic measurement of vertebral shape using active shape models, Image and Vision Computing, Vol. 15, p.
  • the ROI's searched for may be located in the following ways with respect to the segmentation of the bones:
  • FIG. 3 is an example of automatically segmented RUS bones.
  • FIG. 3A shows the initial bounding box placement for automatic segmentation of the associated bones and
  • FIG. 3B shows the segmentation of the phalanxes and carpal bones.
  • the interface regions at the extremal ends of these bones determine the ROI's for bone age scoring.
  • FIG. 5 A general overview of a scoring system is shown in FIG. 5 .
  • the system comprises an engine and a user interface.
  • the engine controls the navigation through different screens (e.g. one screen per region of interest), retrieves and supplies the previous scores and regions of interest from the database, controls juxtaposing of the regions of interest and the reference stage images on the display device, initiates appropriate image processing of the region of interest (e.g. contrast enhancement) and geometric manipulation (e.g. zooming) to optimize visual matching, initiates computation of ROI features on the radiologist's request for decision support, updates spreadsheet and database for newly retained region of interest scores.
  • screens e.g. one screen per region of interest
  • controls juxtaposing of the regions of interest and the reference stage images on the display device controls juxtaposing of the regions of interest and the reference stage images on the display device
  • initiates appropriate image processing of the region of interest e.g. contrast enhancement
  • geometric manipulation e.g. zooming
  • the summation device requires the individual score values of the regions of interest (and feature values) as input values and computes and outputs a total overall score of the radiation image.
  • the total score output by the summation device may be subjected to LUT (look up table) based transforms (the exact transform being dependent on the patient identification data) to convert the overall score to a clinical score value.
  • LUT look up table
  • a report may finally be generated.
  • a radiologist or radiology operator visually compares each ROI with reference stages so as to find the most optimal matching stage. This procedure is performed successively for every ROI.
  • a data base ( FIG. 1 , numeral 5 ) is composed in advance.
  • the database comprises for each individual ROI a number of reference stage images corresponding with different developmental stages, each of these reference images having an associated score.
  • FIGS. 6 and 7 show different embodiments of the organization of a graphical user interface on a computer screen used for performing the scoring procedure on each ROI.
  • a number of reference images having an associated score number are displayed for each of a number of stages.
  • all reference images according to a certain stage are displayed on the same row (referred to as the row-wise arrangement). Reference images corresponding with different stages are displayed in different rows. If the number of stages (hence the number of rows) exceeds the display capacity of the screen, a scroll able window can be used. In the illustrated embodiment the leftmost two columns depict a set of reference images, two images per stage. Up to 8 stages (stage B to I) are used in the bone age assessment.
  • This embodiment of a user interface according to the present invention further comprises means for displaying one or more regions of interest ROI of an image to be scored.
  • the display of the region of interest is situated in close proximity to the display of the reference image(s).
  • the third column is reserved for the display of the actual ROI to be classified.
  • a fourth column is provided which depicts a sketch of the bone with emphasis of particular anatomic features.
  • a fifth column may also be provided which displays a list of salient anatomical feature(s) in textual format, one or more for each of the different stages.
  • Means may be provided for displaying a tick close to the textual information for ticking off the textual information so as to indicate the fact that the radiologist perceives the feature in the actual ROI of the skeletal region.
  • the columns may be interchanged without substantially changing the classification process.
  • the actual ROI image may be placed in between the two reference images, instead of being placed between the reference image and the sketch image.
  • more than two reference images per stage may be used to achieve broader coverage of the grey value appearance of a certain stage.
  • a horizontal arrangement may be adopted, and the actual ROI may be slided horizontally through the scoring screen. Because all reference elements of a certain stage are positioned in the same column, this disposition is called the column-wise arrangement.
  • Two modes are available for displaying the ROI's in the scoring screen.
  • a first mode is the static mode in which each ROI is displayed in juxtaposition with all reference stage images.
  • a second mode is the dynamic mode in which a single ROI is displayed juxtaposed to reference stage images. This ROI is shifted up and downwards by user interaction e.g. by mouse movement, and is juxtaposed to the next or previous reference stage images.
  • the reference stage which most optimally matches with the displayed ROI is selected ( FIG. 1 , numeral 7 ) and the score number associated with the selected reference stage is retained (numeral 8 ) for generating the total score number of the total image (numeral 9 ).
  • the total score can be generated as the sum of the individual scores of the ROI's. Additionally an index can be calculated, being a more complex accumulation of feature values pertaining to individual ROI's (e.g. SDI—spinal deformity index).
  • the interactive comparison of only (a) region(s) of interest of the hand-wrist X-ray image has major advantages over the comparison of the entire hand-wrist X-ray image with a reference image.
  • the actual region of interest only comprises the skeletal site to be classified without any extraneous anatomical detail visual comparison and stage assignment is made easier than in prior art methods. Also because in the present user interface a region of interest can be juxtaposed to the reference images and the sketch, the visual comparison and stage assignment is greatly facilitated over the prior art method.
  • a radiographic feature of a skeletal site that assists in the assessment of the skeletal maturity is called a maturity indicator.
  • a maturity indicator must be determined during the maturation of every child to be useful in the assessment of maturity.
  • the grades of a maturity indicator appear in a fixed sequence for each bone, irrespective of stature, weight, health or other socio-economic factors.
  • Stages and corresponding scores can be recalled from the archived data and displayed during assessment of the maturity of the current skeletal site.
  • the score to be assigned to a ROI of this site under current evaluation cannot be less than the previously assigned score.
  • This property can be implemented in a specific embodiment of the user interface according to the present invention by disabling all reference stages that are less mature or that correspond to a less deteriorated stage, thus narrowing the search in a plausible manner. Such stages can be disabled e.g. by graying them out.
  • Still further narrowing the set of assignable stages can be achieved by marking, e.g. ticking off a characteristic feature in the textual information section on the scoring screen of the user interface.
  • the user interface is then organized such that whenever such a feature of a certain stage is deemed to be present, no less mature stages can be selected. Reversely, when a feature of a certain stage is believed not to be present, said stage and all more mature stages are put to a non-selectable state.
  • the time sequence of the archived regions of interests for a particular skeletal site can be displayed along with the chronological time at which each radiograph has been taken, so that the evolution over time can be assessed.
  • each maturity indicator can be determined and displayed graphically, thus showing how long the bone remained at the same level of maturity.
  • Time t 1 is called baseline and is the point in time at which the first scoring was made. Distances along the time axis are proportional to the time of actual exposure of the radiographs. Time t n+1 is a time point in future at which a new scoring is planned, depending on indications derived from the visual analysis of the ROI sequences.
  • the screen is made scroll able to allow for a large number m of ROI's or a large number of scoring studies over time.
  • the availability of the reference data in digital format enables more easily customisation towards the ethnic type.
  • the TW 2 atlas has been established for the Caucasian type. Differentiation towards ethnic type is achieved either by supplying reference pictures and associated scores according to the patient's ethnic type, or by substituting new value of the TW 2 scores only and keeping the classical reference pictures. This type of customisation obviates the need for establishing different atlases in the current operation of the TW 2 method.
  • a TW 3 atlas has recently been introduced.
  • the operational principle is identical.
  • the calculation of the total score number has been changed relative to the TW 2 atlas in order to match better with the characteristics of the present population.
  • the scores needed be written down in the patient dossier or added manually into the appropriate database record.
  • no recording of the score is needed because the score can be automatically entered according to the row position of the ROI on the screen.
  • the score can be entered in the spreadsheet and database, running in a background process.
  • the assessor may navigate through the screens to review previously accorded stages, and on leave of the program, an overview of all assigned scores, total score and corresponding skeletal age can be displayed.
  • the embodiments of the scoring methods according to the present invention may have other features distinguishing them from the conventional film-based operation.
  • Each of the stages of the scoring methods is examined as to their prominent image features.
  • the appearance of these features may be enhanced in a feature-specific way.
  • FIG. 4 The application of ROI specific image processing is shown in FIG. 4 which figure gives a more detailed overview of a specific embodiment of the present invention.
  • An image is decomposed into detail images at different resolution levels and a residual image.
  • the detail images are then modified by applying at least one non linear conversion according to a conversion function which is non-linear, monotonically increasing, odd, and has a slope that gradually decreases with increasing argument values.
  • Next modified detail images and the residual images are recombined to yield a reconstructed, processed image.
  • Latitude reduction can e.g. be applied for enhanced soft-tissue swelling visualization (RA Larsen scoring) or for enhanced visualization of the onset of ossification (bone age scoring).
  • RA Larsen scoring enhanced soft-tissue swelling visualization
  • bone age scoring enhanced visualization of the onset of ossification
  • Edge enhancement can e.g. be applied prior to width measurement between edges (RA scoring and osteoartritis scoring) or to make assessment more reliable as to whether a border is convex or concave (bone age scoring).
  • Multi scale contrast enhancement can e.g. be applied to assess the presence of erosions (RA scoring) or to make assessment more reliable as to whether an epiphyses caps the metaphysics (bone age scoring).
  • n(i) of multi scale processing can be performed as depicted in FIG. 4 .
  • Static display mode ( FIG. 6 ): all different processing of a ROI are displayed simultaneously in juxtaposition with the reference pictures. User interaction is minimal in this mode (no cursor movements). Selection of the matching stage is achieved by clicking on a row entity (either the most similar reference picture or the particular processed ROI).
  • ROI a1 refers to a region of interest processed to enhance feature 1 of stage a
  • ROI a2 refers to a region of interest processed to enhance feature 2 of stage a etc.
  • Dynamic display mode ( FIG. 7 ): the displayed processing of a ROI is changed in accordance to the cursor up and down movements.
  • the attributed stage corresponds to the row position of the ROI on leave of the screen.
  • ROI c1 refers to a region of interest processed to enhance feature 1 of stage c
  • ROI c2 refers to a region of interest processed to enhance feature 2 of stage c.
  • the user may navigate back and forth through the screens corresponding to the different skeletal sites to be scored, e.g. to revise the score attributed to a previously examined skeletal site.
  • the reference images are preferably also subjected to multi scale processing (identical processing conditions as those applied to the ROI) so as to enable comparison under identical image quality conditions between reference pictures and the processed ROI.
  • a zooming functionality to magnify the ROI (and corresponding reference images) may be implemented to achieve better visibility of small details, such as e.g. the corner of the epiphyses when it is believed to cap the metaphysics (bone age scoring) or to assess the presence or absence of an erosion (RA).
  • small details such as e.g. the corner of the epiphyses when it is believed to cap the metaphysics (bone age scoring) or to assess the presence or absence of an erosion (RA).
  • ROI-specific processing is generally better than first globally optimizing the image quality of the entire image and next cutting out the ROI's.
  • the ROI image quality is higher because the image statistics used in the enhancement procedure are based on pixels of the ROI only.

Abstract

In a method for radiographic scoring of a radiographic image at least one region of interest is determined. Each of the determined regions of interest is displayed in close proximity to a number of reference stages each having an associated score. For each region of interest a reference stage image best matching the displayed region of interest is selected and the score number associated with the selected reference stage image is associated with the region of interest. Score numbers retained for each of the regions of interest are combined to form the score number of the radiographic image.

Description

FIELD OF THE INVENTION
The present invention relates to computer-assisted radiological scoring of radiographic images.
BACKGROUND OF THE INVENTION
In radiological practice, radiographic images are used for a variety of diagnostic purposes. Radiographs can be captured to detect and identify lesions, to diagnose an underlying pathology from radiological manifestations, to assess the existence and extent of a fracture etc.
There exist also a domain of specialized radiological examination procedures, which cannot be classified into any of the enumerated tasks, which is the field of radiological scoring.
In radiographic scoring methods,the radiographer or clinician already knows the pathology but wishes to assess the degree of severity of the disease or the developmental stage of the patient.
Several scoring methods exist in the state of the art. All scoring methods have the following specific characteristics in common:
    • (1) a number of pre-determined anatomical sites on a radiograph is examined; the number of anatomical sites being constant for a given method;
    • (2) the specific appearance of the anatomical site is rated against a number of reference pictures, each reference picture corresponding to a developmental or pathological stage, and the rating results in a stage being assigned; the number of reference stages being constant for the given anatomical site;
    • (3) each stage has an associated numerical score, the value of which in general differs according to the stage and the skeletal site;
    • (4) all scores are totaled and translated into a clinically useful index.
Although rating refers to the process of assigning a stage to an anatomical site and scoring refers to the attribution of a value to the stage, we will call both processes ‘scoring’ in the sequel.
Radiological scoring methods, which adhere to the aforementioned principles, are e.g. bone age determination (also known as skeletal maturity assessment), rheumatoid arthritis scoring, osteo-arthritis scoring, ankylosing spondylitis scoring, and osteoporosis assessment.
In the following scoring methods applied to different applications will be explained.
A first example of an application wherein radiographic scoring is used, is skeletal maturity assessment, a procedure frequently performed in paediatric radiology.
Biologically, it is desirable to assess the maturity of the whole skeleton, but because of several practical difficulties such as the time needed for the assessment of so many bones, the expense of radiographic film, the risk of excessive radiation exposure etc. a specific area is chosen for the assessment of skeletal maturity.
An area of the human skeleton, which has received considerable attention as a source of clinically relevant maturity indicator, is the hand-wrist. Based on a radiological examination of skeletal development of areas of the non-dominant hand and wrist, the bone age is assessed and compared to the chronological age. A discrepancy between these two values indicates abnormalities in skeletal development. This examination is performed on children with growth abnormalities to affirm clinical suspect, to predict height at adult age, or to monitor the effect of treatment of metabolic diseases.
A common method for assessing the skeletal maturity is the atlas method, and the hand-wrist was the first area of the body for which atlases became available. Currently, there are two frequently used atlas methods operating on a hand-wrist radiograph.
The most frequently used method (78%) is the Greulich and Pyle (GP) method. In this method, the radiograph of non-dominant hand wrist (mostly the left hand) is compared to a reference series of hand-wrist radiographs displayed in this atlas. Each hand-wrist radiograph corresponds to a certain year of bone age. The age interval between successive hand-wrist plates varies from 3 to 6 months. The reference radiograph, which globally compares best with the clinical image, is selected as the best match and its associated age is called the bone age. This atlas-derived bone age is then compared with the chronological age of the patient, and the age difference is used for diagnostic purposes.
The manual effectuation of this method is prone to error or susceptible to ambiguity.
Firstly, different radiologist performing this procedure may have different training experience; hence substantial inter-observer and intra-observer variance may result from the subjective nature of this comparison.
Second, the comparison is a global one. Depending on the weight the radiologist attaches to the similarity or dissimilarity of specific skeletal sites on the hand-wrist radiograph, ambiguous results may be obtained.
A method, which aims at minimizing the subjective observer errors, is the Tanner and White house method (TW2 method). Scores are assigned to grades of skeletal maturity indicators, and the sum of scores is later transformed to a skeletal age (also called bone age). Slight modifications to the reference values and charts have recently been made to reflect changes in the population, resulting in the TW3 method. Descriptions and manual rating of the stages of the bones, however, have not been altered.
The TW2 method is currently effectuated in paediatric radiology as follows.
A radiograph from the non-dominant hand and wrist is made either by (a) conventional screen/film recording system, or (b) by digital means such as film digitisation, a computed radiography system or a direct radiography system (based on direct or indirect flat panel, or CCD).
In case a film is used, it is viewed on a light box. A digital image on the other hand may be either printed on film and viewed conventionally on a light box, or it may be displayed on a viewing station.
Knowledge of gender and chronological age of the patient are the only two clinical data needed to start the analysis.
For each skeletal site indicated in the atlas for which a score needs to be established, the corresponding skeletal site in the actual hand-wrist radiograph is searched for visually. Specific salient anatomic details of the skeletal site are retained mentally.
Then the attention is directed to the different stages depicted in the atlas to identify the most similar one.
Use is made of (1) the reference pictures (two radiographic reproductions per stage in the TW2 atlas, to reflect the range of variation within the stage), (2) a sketch or drawing of the skeletal site depicting the outline of the bones, the radio-opaque lines on or within the margin of the epiphyses and bones and pointers to salient features of the particular stage, and (3) one or more criterions per stage, which are textual explanations hinting at the most prominent image features of the stage.
In the event of doubt, whether a particular feature of a certain stage is present or not in the actual image, visual attention is redirected back to the hand-wrist image on the light box or computer display.
As a matter of course, the correct skeletal site to be assessed needs again be relocated in the hand-wrist radiograph before such confirming visual analysis can be effectuated.
In many cases, if not all, several switches back and forth between hand-wrist radiograph and the reference stages in the atlas are needed before a conclusive assignment of a reference stage can be made.
The score, corresponding to the matching stage, is written down on paper or typed in a computer spreadsheet or database.
This iterative process of hypothesis generation and hypothesis verification as to the actual stage of a skeletal site is repeated s for every skeletal site.
In the full TW2 method, 20 skeletal sites in the hand-wrist need to be assessed this way; therefore this scoring method can be time-consuming, demanding and error-prone.
The total score (on a scale between 0 . . . 1000) is translated into a bone age by a table look up operation. A different table is applied depending on the gender of the patient.
The bone age derived is compared with the chronological age of the patient and the difference between them is used as an element in the clinical diagnosis process.
Analogous to the Greulich and Pyle method, care must be exercised when interpreting the bone age to chronological age difference for patients belonging to a race other than the Caucasian type, because the atlas shows reference pictures and associated scores for this ethnic type only.
Unfortunately, although reliable, the above method is complex to operate for several reasons:
    • It requires a well-trained radiologist or radiology operator.
    • Because it requires the assessment of a great number of different skeletal sites, a manual method is time consuming.
    • The application of the method by means of analogue screen-film radiography is particularly cumbersome. It requires 3 distinct media to effectuate this procedure:
      • An X-ray film of the hand or other skeletal sites, displayed on a light box. With the emergence of digital radiography modalities (film digitisation, computed radiography, digital radiography sensors), the digital image may be displayed on a computer display instead. However, such electronic medium still is physically distinct from the remaining components.
      • The TW2 atlas illustrating the different reference skeletal stages to compare with, and depicting the conversion table to manually convert the total score into a bone age.
      • Pencil/Paper to note and add the scores of each selected stage in the patient dossier. Alternatively, electronic spreadsheets may be used in conjunction with a database to store and compute the total score and the conversion to bone age. This way of operation is particularly cumbersome when scores determined at regular time intervals need to be retrieved and compared for clinical evaluation over time.
    • Both TW2 and GP atlases have been established for the Caucasian racial type. Therefore, the bone age derived from these atlases is in principle valid for this type only.
Another example of an application wherein radiographic scoring is used, is rheumatoid arthritis.
Conventional rheumatoid arthritis scoring (RA scoring) is based on comparison of the actual film radiography of hands and feet with reference pictures printed on film.
To establish the database of reference pictures, all stages of a joint or a group of joints are printed on one sheet of film in life size.
Scoring proceeds by displaying standard reference films and the actual radiography on a light box, and writing down scores associated with the matching stage of each joint.
Among other indicators, the most specific deformations in RA are erosions and joint space narrowing. The radiological study of these deformations is complementary; therefore, both are being assessed in a scoring method.
Radiological assessment serves as a recognized standard for the evaluation of rheumatoid arthritis. For standardized, epidemiological and therapeutic evaluation, scoring or grading systems have been proposed on X-ray images of specific skeletal sites. X-ray images have the advantages to be able to record the history of the damage of a joint, because the damage is mostly irreversible. Previous images can be recalled in a later stage, to assess the evolution and to subject them to newly developed scoring systems. The effectivity of drugs can be traced.
These advantages however, can only be obtained when the images are of sufficient diagnostic quality: (a) for serial assessment, the positioning of the joints during radiation exposure are of substantial importance, (b) correct exposure is important because overexposure or underexposure affects e.g. the correct assessment of erosions, and (c) the images recorded with sufficient resolution are essential for a reliable evaluation of previous erosions.
Still another example of an application wherein radiographic scoring is used, is osteoporosis scoring.
Spine or hip fracture due to minor or no trauma is an essential feature of symptomatic osteoporosis. Conventional radiographs are widely used to confirm or disprove suspected osteoporotic fractures, and may also demonstrate progress of existing fractures or development of new ones.
The scoring systems using conventional radiographs suffer the same drawback of other film-based diagnostic scoring systems. As indicated, the problem of repeated re-location of the anatomical site is even more compound because the spine is a highly repetitive structure of vertebrae having almost identical shape.
OBJECTS OF THE INVENTION
It is an object of the present invention to provide a user-friendly radiographic scoring method that overcomes the drawbacks of the prior art scoring methods, such as the need for different media and the need for repeated focusing of the attention between images on different media.
Further objects will become apparent from the detailed description given hereinafter.
SUMMARY OF THE INVENTION
The above-mentioned objects are realised by a method of radiographic scoring of a radiographic image comprising the steps of
    • acquiring (1) a digital image representation of an image to be subjected to radiographic scoring,
    • applying (2) the acquired image representation to a display device and generating a displayed image,
    • determining (3) at least one region of interest on the displayed image,
    • retrieving (4) for each of the determined regions of interest a number of reference stage images from a database (5), each reference stage image having an associated score number and
    • displaying (6) the reference stage images on said display device,
    • selecting (7) a displayed reference stage image which most optimally matches with a displayed region of interest,
    • associating (8) a score number corresponding with the selected reference stage image with the displayed region of interest,
    • combining (9) score numbers associated with said regions of interest into an overall score number pertaining to said radiographic image.
A displayed reference stage image which most optimally matches with a displayed region of interest can be selected by
    • displaying each of said region(s) of interest and the reference stage images pertaining to the displayed region of interest in close proximity to each other, and
    • visually comparing a displayed region of interest and displayed reference stage images pertaining to the displayed region of interest,
    • selecting a displayed reference image which visually most optimally matches with a displayed region of interest.
A reference stage image is an image which is representative for a developmental stage of a specific skeletal site. These reference stage images are commonly selected by an expert. In most cases these reference images are accepted as world wide standards.
In one embodiment the step of determining a region of interest is guided by means of a stencil consisting of a sketch of a body part of the patient that is to be evaluated. The sketch depicts at least one of the outline of the following three items: the body part, bone contours of bones within the body part, anatomical contours, as well as region of interest delineations superimposed on said sketch.
In one embodiment a region of interest is determined and extracted on a display device by manual operation.
The following steps are performed:
    • displaying a stencil comprising a sketch of a body part to be evaluated and region of interest delineations superimposed on said sketch,
    • mapping or dragging the region of interest delineations onto the actual position of the body part to be scored in the displayed image,
    • adjusting the shape of the region of interest by rotation and scaling so that it substantially matches the actual geometry of the local anatomy in the displayed image,
    • selecting image data within the dragged delineations as region of interest.
In one embodiment a digital image representation of a region of interest is extracted from a digital image representation of said image by re-sampling and interpolation.
Preferably a stencil is selected according to identification data of said radiographic image such as gender, age and examination type (hand, foot etc.).
After the regions of interest superimposed on the skeletal outlines in the stencil, are mapped to the correct position in the image, the image data in the regions of interest in the displayed image are extracted. Preferably the extracted regions of interest are subjected to rotation or to an affine transformation so as to have them in upright position.
Regions of interest may also be subjected to image enhancing processing (see below).
Next the ensemble of ROI's is subjected to the remainder of the scoring steps: display in proximity to the reference stage images, selection of the best matching reference stage image, associating of a score number and combining of score numbers of all regions of interest into an overall score number of the entire image.
This procedure is referred to as manual scoring procedure.
Alternatively a semi-automated method can be applied wherein all steps of the method described higher are also applied but wherein additional decision support is generated to assist the selection of the optimally matching reference stage image. In this semi-automated method additional support is generated to help to assess whether a specific feature is radiologically manifested in the region of interest. More specifically for each of the regions of interest (ROI) a sequence of specific image processing operators is applied to compute ROI specific feature values and to subject the computed feature values to a judgment device in order to determine presence or absence of a feature. Presence or absence of a feature will be used for assessing which reference stage image is most optimally matching. This system is called semi-automated scoring since the ultimate decision whether or not a computer-suggested decision is accepted, is still made by a radiologist. It may be possible in the current method that not for all regions of interest automatic feature based support is available. Furthermore it may also be possible that the features are not computed exhaustively for all stages of a specific region of interest.
In an enhanced semi-automatic operation mode all features applicable for a specific region of interest are computed and input to a judgment device to derive the stage of the region of interest. The difference with the above semi-automated method is that, in order to derive a suggestion as to the stage of the region of interest, all relevant features covering the whole developmental stage of the spectrum of the region of interest must be computed and used in the classification process. The only task left for the radiologist is then to accept or reject the computer suggestions for individual features. This form of semi-automated scoring will therefore be much faster, but because of the plurality of input features, the judgment device is faced with a more complex task. Bayesian classifiers and artificial neural networks are used for this purpose.
Data within a region of interest and/or data of a reference stage image may be subjected to image processing. Image processing is e.g. spatial reformatting and/or processing for the purpose of enhancing the intrinsic image quality. For example multi scale contrast enhancement can be applied, as will be explained further on.
Another aspect of this invention relates to a computer program product as set out in the claims.
Further details of the different steps performed by the computer program product are set out in the dependent claims.
A computer program product is meant to encompass a software product in the form of an electric signal as well as a computer program on a computer readable medium such as a CD-ROM.
Still another aspect relates to a computer readable medium such as a CD-ROM comprising program code adapted to carry out the above method.
Specific features for preferred embodiments of the invention are set out in the dependent claims.
It will be clear that the scoring method according to the present invention copes with the problems of the prior art.
The ROI extraction and display in the neighborhood of the reference stage images alleviates the problem of repeated focus of attention towards the correct skeletal site.
Furthermore, by applying image processing such as multi-scale enhancement, the available dynamic range of the output medium can be used optimally to render all relevant image details.
The current invention is particularly suited to implement different variations of the Sharp score in rheumatoid arthritis, since the ROI stencil can be customized so as to include only an abbreviated set of ROI's.
Alternatively, a score, based on a reduced number of ROI's, can also be derived from a superset including more ROI's having been evaluated previously.
The risk to evaluate a wrong ROI is substantially reduced, because the ROI stencil depicting fewer skeletal sites serves as a reliable hinting mechanism.
In the prior art the task of focusing on a ROI and scoring is alternated for all skeletal sites. In one embodiment of the present invention the order has been changed, first all ROI's are determined and next the scoring is performed.
In the field of osteoporosis the invention is particularly useful since by first extracting regions of interest around each vertebra, the visual search process is greatly simplified. No confusion with other vertebrae is possible when presenting each ROI in juxtaposition with the reference radiographs and graphic pictures.
For bone age determination, errors that might occur in the prior art due to inadvertently exchanging score numbers for boys and girls, are avoided.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates the different steps of a scoring method according to the present invention,
FIG. 2 shows a stencil used for bone age determination,
FIG. 3 is an example of automatically segmented RUS bones, wherein FIG. 3A shows the initial bounding box placement for automatic segmentation of the associated bones and FIG. 3B shows the segmentation of the phalanxes and carpal bones. The interface regions at the extremal ends of these bones determine the ROI's for bone age scoring,
FIG. 4 is a more detailed illustration of a specific embodiment of the method of the present invention,
FIG. 5 is a general overview of a scoring system,
FIG. 6 is an example of a user interface with static ROI display,
FIG. 7 is an example of a user interface with dynamic ROI display,
FIG. 8 is an example of a longitudinal maturity study.
 DETAILED DESCRIPTION OF THE INVENTION
Image Acquisition and Display
A digital image representation of an image to be subjected to radiographic scoring is acquired (FIG. 1, numeral 1).
A large variety of image acquisition systems are applicable. The radiation image can for example be recorded on a photostimulable phosphor screen. The screen carrying the radiation image is then read out by scanning it with stimulating radiation and detecting the image-modulated light which is emitted upon stimulation and by converting the image-wise modulated light into a signal representation of the radiation image.
In an alternative embodiment the radiation image can be stored in a direct radiation sensor rendering a signal representation of a radiation image.
Still further alternatives for acquiring a digital signal representation of a radiation image, such as scanning of radiographic film carrying a radiation image, are possible.
The acquired image representation is next applied to a display device and the image is displayed (FIG. 1, numeral 1).
ROI Positioning and Selection
Next, in the ROI (region of interest) positioning stage at least one ROI is determined on the displayed image (FIG. 1, step 2).
In the following explanation will be referred to the hand as a body part that is subjected to evaluation. However, the present invention is not limited to scoring methods using the hand as a body part to be evaluated.
In this context, the region of interest is the image region covering the joint or bone relevant for the examination. The ROI positioning stage aims at positioning all regions of interest (ROI's) on the body part that is examined. Although each of the ROI's may be extracted separately and analyzed immediately, a more convenient work flow is obtained by first extracting all ROI's simultaneously before processing them.
ROI positioning can be performed in an operator-guided way or alternatively in an automatic way.
In the case of operator-guided ROI positioning a method using a digital stencil can be used to facilitate the positioning.
In this context, a stencil is a digital sketch of a hand wrist image depicting the outline of the hand and all hand bone contours. The delineations of all possible ROI's are superimposed on these outlines.
As has already been explained higher, a hand-wrist image is taken as an example. Other body parts may be examined depending on the type of application and the radiographic scoring method.
For example for the TW2 analysis, these ROI outlines will typically be rectangular regions around joints of the tubular (radius, ulna, metacarpal and phalange) bones and around the carpal bones.
FIG. 2 shows a stencil used for bone age determination. The stencil shows all rectangles of regions of interest (dashed lines) superimposed on an outline of the diagnostic entities.
Each of the ROI outlines is mapped or dragged by the operator (translated e.g. by mouse and arrow keys) to its actual location in the displayed image, and the stencil serves as a guideline to indicate the intended image region the ROI should encompass, with sufficient precision for further analysis.
Basic operations such as translation, rotation and scaling may be applied to the stencil ROI so that the resulting ROI fully comprises its associated hand bone without irrelevant bones being present which would make the identification of the target bone ambiguous.
The manual placement may be speeded up by first translating, rotating and scaling the stencil in its entirety so that it roughly coincides with the actual hand and bony outlines, and by further adjusting the position of the individual ROI's so that each ROI accurately covers its intended bone/joint location.
An alternative for speeding up is the semi-automated positioning relative to a few user-defined points.
Because the superposition of all ROI's in the stencil may be visually confusing when they overlap substantially (this is especially the case for the carpal region of interest), the ROI's may be sequentially displayed in the stencil and positioned in the actual region. Size and orientation of the rectangles may be modified e.g. by corner point dragging to match the actual size and the orientation of the displayed image.
Because the hand-wrist anatomy changes its aspect in accordance with age, the stencil must preferably reflect this in order to more faithfully represent the actual skeletal sites. Therefore, preferably a number of hand-wrist stencils are stored as a function of chronological age, gender, race etc. In accordance with the patient's identification data such as age, race, gender, etc. which may be entered into the computer, the corresponding sketch can be selected.
Although the chronological age may not be representative for the skeletal maturity status of the patient (the bone age scoring is precisely meant to detect significant differences from it), it serves as a rough estimate for the initial selection of the stencil.
In an alternative embodiment, the ROI positioning can be automated by image processing algorithms, so that no user interaction is required in the placement of the ROI regions.
Automatic placement can be based on first automatically segmenting the bones based on Point Distribution Models (PDM) and Active Shape Models (ASM) such as disclosed in the prior art (Cootes TF and Taylor CJ, Active Shape Models—“smart snakes”, Proc. British Machine Vision Conference, Springer Verlag 1992, p. 256–275). An example for segmenting the individual vertebrae in the spine is given in (Smyth et al, Automatic measurement of vertebral shape using active shape models, Image and Vision Computing, Vol. 15, p. 575–581, 1997), for the hand and hip in (Behiels et al, Active shape model based segmentation of digital X-ray images, proceedings 2and international conference on medical image computing and computer-assisted intervention—MICCAI′99, lecture notes in computer science, Vol. 1979, p. 128–137, 1999).
Secondly, depending on the scoring method, the ROI's searched for may be located in the following ways with respect to the segmentation of the bones:
  • at outer borders of the bony segmented regions (which is the case for RUS bone age scoring because the proximal and distal interface regions of neighboring bones contain the epiphyses and corresponding metaphyses; and for rheumatoid arthritis scoring, because these regions are affected by joint space narrowing and is erosion defects), around the bony region (which is the case for individual vertebra of a spine, such that each ROI contains the segmented vertebra and part of both neighboring vertebrae), inside the bony region at pre-determined locations given by human anatomy (which is the case for carpal bone age scoring when the region of carpal bones is segmented as a whole, because the each carpal bone has a fixed position with respect to all other carpal bones contained in the ROI).
FIG. 3 is an example of automatically segmented RUS bones. FIG. 3A shows the initial bounding box placement for automatic segmentation of the associated bones and FIG. 3B shows the segmentation of the phalanxes and carpal bones. The interface regions at the extremal ends of these bones determine the ROI's for bone age scoring.
Scoring Procedure—General Overview
The methodology used to achieve radiographic scoring according to the present invention will be described using bone age assessment as an example.
Whenever necessary, details specific for other clinical scoring methods( rheumatoid arthritis, osteo-arthritis, osteoporosis and ankylosing spondylitis) will be indicated.
A general overview of a scoring system is shown in FIG. 5.
The system comprises an engine and a user interface. The engine controls the navigation through different screens (e.g. one screen per region of interest), retrieves and supplies the previous scores and regions of interest from the database, controls juxtaposing of the regions of interest and the reference stage images on the display device, initiates appropriate image processing of the region of interest (e.g. contrast enhancement) and geometric manipulation (e.g. zooming) to optimize visual matching, initiates computation of ROI features on the radiologist's request for decision support, updates spreadsheet and database for newly retained region of interest scores.
The summation device requires the individual score values of the regions of interest (and feature values) as input values and computes and outputs a total overall score of the radiation image.
It may further compute sums of features which are output as an index (as will be explained further on).
The total score output by the summation device may be subjected to LUT (look up table) based transforms (the exact transform being dependent on the patient identification data) to convert the overall score to a clinical score value.
A report may finally be generated.
Comparison of ROI's with Reference Stages
Referring again to FIG. 1 (numeral 4), in the following step a radiologist or radiology operator visually compares each ROI with reference stages so as to find the most optimal matching stage. This procedure is performed successively for every ROI.
To this end a data base (FIG. 1, numeral 5) is composed in advance. The database comprises for each individual ROI a number of reference stage images corresponding with different developmental stages, each of these reference images having an associated score.
To facilitate this comparison, a graphical user interface has been developed. FIGS. 6 and 7 show different embodiments of the organization of a graphical user interface on a computer screen used for performing the scoring procedure on each ROI.
In the user interface a number of reference images having an associated score number are displayed for each of a number of stages.
In one embodiment all reference images according to a certain stage are displayed on the same row (referred to as the row-wise arrangement). Reference images corresponding with different stages are displayed in different rows. If the number of stages (hence the number of rows) exceeds the display capacity of the screen, a scroll able window can be used. In the illustrated embodiment the leftmost two columns depict a set of reference images, two images per stage. Up to 8 stages (stage B to I) are used in the bone age assessment.
This embodiment of a user interface according to the present invention further comprises means for displaying one or more regions of interest ROI of an image to be scored. The display of the region of interest is situated in close proximity to the display of the reference image(s). In the displayed embodiment the third column is reserved for the display of the actual ROI to be classified.
In the illustrated embodiment a fourth column is provided which depicts a sketch of the bone with emphasis of particular anatomic features.
A fifth column may also be provided which displays a list of salient anatomical feature(s) in textual format, one or more for each of the different stages.
Means may be provided for displaying a tick close to the textual information for ticking off the textual information so as to indicate the fact that the radiologist perceives the feature in the actual ROI of the skeletal region.
It will be clear that the columns may be interchanged without substantially changing the classification process. For example, the actual ROI image may be placed in between the two reference images, instead of being placed between the reference image and the sketch image. Also, more than two reference images per stage may be used to achieve broader coverage of the grey value appearance of a certain stage.
Instead of a vertical arrangement of the reference data, a horizontal arrangement may be adopted, and the actual ROI may be slided horizontally through the scoring screen. Because all reference elements of a certain stage are positioned in the same column, this disposition is called the column-wise arrangement.
Two modes are available for displaying the ROI's in the scoring screen.
A first mode is the static mode in which each ROI is displayed in juxtaposition with all reference stage images.
A second mode is the dynamic mode in which a single ROI is displayed juxtaposed to reference stage images. This ROI is shifted up and downwards by user interaction e.g. by mouse movement, and is juxtaposed to the next or previous reference stage images.
Selection of Matching Reference Stage
The reference stage which most optimally matches with the displayed ROI is selected (FIG. 1, numeral 7) and the score number associated with the selected reference stage is retained (numeral 8) for generating the total score number of the total image (numeral 9). The total score can be generated as the sum of the individual scores of the ROI's. Additionally an index can be calculated, being a more complex accumulation of feature values pertaining to individual ROI's (e.g. SDI—spinal deformity index).
The interactive comparison of only (a) region(s) of interest of the hand-wrist X-ray image has major advantages over the comparison of the entire hand-wrist X-ray image with a reference image.
Because the actual region of interest only comprises the skeletal site to be classified without any extraneous anatomical detail visual comparison and stage assignment is made easier than in prior art methods. Also because in the present user interface a region of interest can be juxtaposed to the reference images and the sketch, the visual comparison and stage assignment is greatly facilitated over the prior art method.
Maturity Indicator
A radiographic feature of a skeletal site that assists in the assessment of the skeletal maturity is called a maturity indicator. A maturity indicator must be determined during the maturation of every child to be useful in the assessment of maturity. By definition, the grades of a maturity indicator appear in a fixed sequence for each bone, irrespective of stature, weight, health or other socio-economic factors.
This characteristic of human growth can be exploited in the user interface according to the present invention in the following way.
Stages and corresponding scores can be recalled from the archived data and displayed during assessment of the maturity of the current skeletal site. In the assumption that the scores previously assigned to a specific skeletal site are correct, the score to be assigned to a ROI of this site under current evaluation cannot be less than the previously assigned score.
Likewise in rheumatoid arthritis previous damage to the skeletal site cannot be undone. The score number of the last evaluation is carried forward to the present evaluation.
This property can be implemented in a specific embodiment of the user interface according to the present invention by disabling all reference stages that are less mature or that correspond to a less deteriorated stage, thus narrowing the search in a plausible manner. Such stages can be disabled e.g. by graying them out.
Still further narrowing the set of assignable stages can be achieved by marking, e.g. ticking off a characteristic feature in the textual information section on the scoring screen of the user interface. The user interface is then organized such that whenever such a feature of a certain stage is deemed to be present, no less mature stages can be selected. Reversely, when a feature of a certain stage is believed not to be present, said stage and all more mature stages are put to a non-selectable state.
Furthermore, to clarify the decision support, the time sequence of the archived regions of interests for a particular skeletal site can be displayed along with the chronological time at which each radiograph has been taken, so that the evolution over time can be assessed.
The duration of each maturity indicator can be determined and displayed graphically, thus showing how long the bone remained at the same level of maturity.
In order not to miss stages, schedules of serial (follow-up) radiographs can be planned relative to the inception of maturity indicators as opposed to classical chronological age planning. This feature is termed the longitudinal maturity study and the screen layout is depicted in FIG. 8. Time t1 is called baseline and is the point in time at which the first scoring was made. Distances along the time axis are proportional to the time of actual exposure of the radiographs. Time tn+1 is a time point in future at which a new scoring is planned, depending on indications derived from the visual analysis of the ROI sequences. The screen is made scroll able to allow for a large number m of ROI's or a large number of scoring studies over time.
While the statement, that the sequence of maturity indicators is always the same, is true for the grades of a single indicator for a particular bone, it is not accurate if-applied to several indicators of various bones. Precisely because each skeletal site is assessed separately, this characteristic of local differences in human bone growth does not influence the precision of the TW2 method, in contrast to the global method of Greulich and Pyle where all skeletal sites must be assessed simultaneously.
The availability of the reference data in digital format enables more easily customisation towards the ethnic type. Currently, the TW2 atlas has been established for the Caucasian type. Differentiation towards ethnic type is achieved either by supplying reference pictures and associated scores according to the patient's ethnic type, or by substituting new value of the TW2 scores only and keeping the classical reference pictures. This type of customisation obviates the need for establishing different atlases in the current operation of the TW2 method.
A TW3 atlas has recently been introduced. The operational principle is identical. The calculation of the total score number has been changed relative to the TW2 atlas in order to match better with the characteristics of the present population.
In the prior art, the scores needed be written down in the patient dossier or added manually into the appropriate database record. In one embodiment of the present invention no recording of the score is needed because the score can be automatically entered according to the row position of the ROI on the screen. On exit of the screen and continuation to the next screen of the next skeletal site, the score can be entered in the spreadsheet and database, running in a background process. The assessor may navigate through the screens to review previously accorded stages, and on leave of the program, an overview of all assigned scores, total score and corresponding skeletal age can be displayed. Graphs, depicting the progression of skeletal maturity of each skeletal site, and overall bone age progression with respect to chronological age, complement the skeletal maturity report.
The embodiments of the scoring methods according to the present invention may have other features distinguishing them from the conventional film-based operation.
Each of the stages of the scoring methods is examined as to their prominent image features. The appearance of these features may be enhanced in a feature-specific way.
The application of ROI specific image processing is shown in FIG. 4 which figure gives a more detailed overview of a specific embodiment of the present invention.
In U.S. Pat. No. 5,805,721 image enhancement based on multi-scale image decomposition is disclosed.
An image is decomposed into detail images at different resolution levels and a residual image. The detail images are then modified by applying at least one non linear conversion according to a conversion function which is non-linear, monotonically increasing, odd, and has a slope that gradually decreases with increasing argument values. Next modified detail images and the residual images are recombined to yield a reconstructed, processed image.
Using a suitable transformation of Laplacian pyramid coefficients, different visual aspects of the regions of interest can be enhanced.
Latitude reduction can e.g. be applied for enhanced soft-tissue swelling visualization (RA Larsen scoring) or for enhanced visualization of the onset of ossification (bone age scoring).
Edge enhancement can e.g. be applied prior to width measurement between edges (RA scoring and osteoartritis scoring) or to make assessment more reliable as to whether a border is convex or concave (bone age scoring).
Multi scale contrast enhancement can e.g. be applied to assess the presence of erosions (RA scoring) or to make assessment more reliable as to whether an epiphyses caps the metaphysics (bone age scoring).
If there are more criteria per stage, all applicable processing of the ROI can be performed and displayed simultaneously. Let m be the total number of ROI's, and i denoting the sequence number of a ROI, then a ROI-specific number n(i) of multi scale processing can be performed as depicted in FIG. 4.
These optimally processing of the ROI's are input to the scoring system.
Two modes of displaying the processed ROI's in the scoring screen can be made available:
Static display mode (FIG. 6): all different processing of a ROI are displayed simultaneously in juxtaposition with the reference pictures. User interaction is minimal in this mode (no cursor movements). Selection of the matching stage is achieved by clicking on a row entity (either the most similar reference picture or the particular processed ROI). In this figure ROIa1 refers to a region of interest processed to enhance feature 1 of stage a, ROIa2 refers to a region of interest processed to enhance feature 2 of stage a etc.
Dynamic display mode (FIG. 7): the displayed processing of a ROI is changed in accordance to the cursor up and down movements. The attributed stage corresponds to the row position of the ROI on leave of the screen. In this figure ROIc1 refers to a region of interest processed to enhance feature 1 of stage c, ROIc2 refers to a region of interest processed to enhance feature 2 of stage c.
The user may navigate back and forth through the screens corresponding to the different skeletal sites to be scored, e.g. to revise the score attributed to a previously examined skeletal site.
To achieve optimal matching performance, the reference images are preferably also subjected to multi scale processing (identical processing conditions as those applied to the ROI) so as to enable comparison under identical image quality conditions between reference pictures and the processed ROI.
A zooming functionality, to magnify the ROI (and corresponding reference images) may be implemented to achieve better visibility of small details, such as e.g. the corner of the epiphyses when it is believed to cap the metaphysics (bone age scoring) or to assess the presence or absence of an erosion (RA).
In this way a drawback of the conventional atlas-based method is overcome, namely that in the conventional atlas-based method the illustrations are small, implying that it difficult to observe e.g. the erosive and joint space abnormalities characterizing the different stages of rheumatoid arthritis.
ROI-specific processing is generally better than first globally optimizing the image quality of the entire image and next cutting out the ROI's. By first cutting out the ROI's from the unprocessed images and next subjecting them to image enhancement, the ROI image quality is higher because the image statistics used in the enhancement procedure are based on pixels of the ROI only.
Evidently, such stage-specific processing is impossible with the prior art using film images. Furthermore, it will be clear that the conventional atlas-based comparison method as used in the prior art is unable to provide the aforementioned improved functionalities.

Claims (22)

1. A method of radiographic scoring of a radiographic image comprising the steps of:
acquiring a digital image representation of an image to be subjected to radiographic scoring,
applying the acquired image representation to a display device and generating a displayed image,
determining and extracting at least one region of interest on the displayed image,
retrieving for each said at least one extracted region of interest a corresponding region of interest in reference stage images from a database, each said reference stage image having an associated score number,
displaying the extracted regions of interest from the displayed image and the corresponding reference regions of interest from said reference stage images in juxtaposition on the same display screen,
selecting a displayed optimal region of interest within said reference regions of interest which most optimally matches with the extracted region of interest from said displayed image,
associating with the region of interest from the displayed image a score number corresponding with the selected region of interest from the reference stage images, and
combining score numbers associated with said regions of interest into an overall score number pertaining to said radiographic image.
2. A method according to claim 1 wherein the displayed reference stage image which most optimally matches with the displayed region of interest is selected by:
displaying each said region of interest and the reference stage images pertaining to the displayed region of interest in close proximity to each other, and
visually comparing the displayed region of interest and the displayed reference stage images pertaining to the displayed region of interest, and
selecting the displayed reference image which visually most optimally matches with the displayed region of interest.
3. A method according to claim 1 wherein the step of determining the region of interest is guided by means of a stencil consisting of a sketch of a body part to be evaluated depicting at least one of the outline of the body part, bone contours and anatomical contours within said body part, in addition to region of interest delineations superimposed on said sketch.
4. A method according to claim 1 wherein a digital image representation of the region of interest is extracted from a digital image representation of said image by resampling and interpolation.
5. A method according to claim 3 wherein said stencil is selected on the basis of identification data such as gender, age, examination type.
6. A method according to claim 3 wherein the region of interest comprising a body part in said stencil, said region of interest being defined by said region of interest delineations, is mapped onto an actual position of the body part within an image to be scored.
7. A method according to claim 1 wherein data within the region of interest is subjected to image processing.
8. A method according to claim 1 wherein the reference stage image is subjected to image processing.
9. A method according to claim 7 wherein said image processing comprises a spatial reformatting process.
10. A method according to claim 8 wherein said image processing comprises a spatial reformatting process.
11. A method according to claim 7 wherein said image processing is an image enhancement process.
12. A method according to claim 8 wherein said image processing is an image enhancement process.
13. A computer readable medium encoded with a computer program adapted to carry out the following steps when run on a computer:
displaying a radiographic image on a display device connected to said computer so as to generate a displayed image,
upon indication by an operator of a region of interest in said displayed image, selecting and extracting said region of interest, and displaying said extracted region of interest on a display device,
retrieving for a selected region of interest reference stage images from a database accessible via said computer, each of said reference stage images having an associated score number, displaying the retrieved reference stage images in juxtaposition with the said extracted region of interest on the same display device,
selecting a displayed reference stage image which
optimally matches with a displayed region of interest, associating with the displayed region of interest a score number which corresponds with the selected reference stage image,
combining score numbers associated with said regions of interest into an overall score number pertaining to said radiographic image.
14. A computer readable medium encoded with a computer program according to claim 13 wherein a digital image representation of a region of interest is extracted from a digital image representation of said image by re-sampling and interpolation.
15. A computer readable medium encoded with a computer program according to claim 13 wherein a displayed reference stage image which most optimally matches with a displayed region of interest is selected by:
displaying each of said regions of interest and the reference stage images pertaining to the displayed region of interest in close proximity to each other,
visually comparing a displayed region of interest and displayed reference stage images pertaining to the displayed region of interest, and
selecting a displayed reference image which visually most optimally matches with a displayed region of interest.
16. A computer readable medium encoded with a computer program according to claim 13 wherein the determining of a region of interest is guided by means of a stencil consisting of a sketch of a body part to be evaluated depicting at least one of the outline of the body part, bone contours and anatomical contours within said body part and region of interest delineations superimposed on said sketch.
17. A computer readable medium encoded with a computer program according to claim 13 wherein a region of interest comprising a body part in said stencil, said region of interest being defined by said region of interest delineations, is mapped onto the actual position of that body part within an image to be scored.
18. A computer readable medium encoded with a computer program according to claim 13 wherein data within a region of interest are subjected to image processing.
19. A computer readable medium encoded with a computer program according to claim 13 wherein a reference stage image is subjected to image processing.
20. A computer readable medium encoded with a computer program according to claim 13 wherein said image processing comprises a spatial reformatting process.
21. A computer readable medium encoded with a computer program according to claim 17 wherein said image processing is an image enhancement process.
22. A computer readable medium encoded with a computer program according to claim 18 wherein said image processing is an image enhancement process.
US10/237,311 2001-09-18 2002-09-09 Radiographic scoring method Expired - Fee Related US7065235B2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP01000470A EP1293925A1 (en) 2001-09-18 2001-09-18 Radiographic scoring method
EP01000470.3 2001-09-18

Publications (2)

Publication Number Publication Date
US20030053673A1 US20030053673A1 (en) 2003-03-20
US7065235B2 true US7065235B2 (en) 2006-06-20

Family

ID=8176064

Family Applications (1)

Application Number Title Priority Date Filing Date
US10/237,311 Expired - Fee Related US7065235B2 (en) 2001-09-18 2002-09-09 Radiographic scoring method

Country Status (3)

Country Link
US (1) US7065235B2 (en)
EP (1) EP1293925A1 (en)
JP (1) JP2003175023A (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030229299A1 (en) * 2002-06-05 2003-12-11 Fuji Photo Film Co., Ltd. Method of and system for evaluating joint and computer program for carrying out the method
US20050244036A1 (en) * 2004-04-19 2005-11-03 Henry Rusinek Method and apparatus for evaluating regional changes in three-dimensional tomographic images
US20070064981A1 (en) * 2003-09-17 2007-03-22 Meijer Eric L Repeated examination reporting
US20070269086A1 (en) * 2006-03-22 2007-11-22 Kerwin William S Conditional shape model for image processing
US20080103834A1 (en) * 2006-10-25 2008-05-01 Bruce Reiner Method and apparatus of providing a radiation scorecard
US20090046953A1 (en) * 2004-04-29 2009-02-19 Koninklijke Philips Electronics N V Image Processing Apparatus And Method
US20110060247A1 (en) * 2009-09-10 2011-03-10 General Electric Company Methods and apparatus for measuring bone lengths
US20110058725A1 (en) * 2009-09-10 2011-03-10 General Electric Company Methods and apparatus for measuring body circumference
US20110058726A1 (en) * 2009-09-10 2011-03-10 General Electric Company System and method for performing bone densitometer measurements
US20110270623A1 (en) * 2007-10-25 2011-11-03 Bruce Reiner Method and apparatus of determining a radiation dose quality index in medical imaging
US8300911B1 (en) 2009-09-10 2012-10-30 General Electric Company Methods and apparatus for measuring visceral fat mass
US20120290957A1 (en) * 2011-05-12 2012-11-15 Jonathan Chernilo User interface for medical diagnosis
US20130064467A1 (en) * 2009-12-17 2013-03-14 Elta Systems Ltd. Method and system for enhancing an image
US11308662B2 (en) * 2016-08-02 2022-04-19 Shanghai United Imaging Healthcare Co., Ltd. System and method for image reconstruction

Families Citing this family (126)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ATE454845T1 (en) 2000-10-30 2010-01-15 Gen Hospital Corp OPTICAL SYSTEMS FOR TISSUE ANALYSIS
US9295391B1 (en) 2000-11-10 2016-03-29 The General Hospital Corporation Spectrally encoded miniature endoscopic imaging probe
DE10297689B4 (en) 2001-05-01 2007-10-18 The General Hospital Corp., Boston Method and device for the determination of atherosclerotic coating by measurement of optical tissue properties
GB2380888A (en) * 2001-10-13 2003-04-16 Hewlett Packard Co Automatic determination of regions of interest in an image
US7355716B2 (en) 2002-01-24 2008-04-08 The General Hospital Corporation Apparatus and method for ranging and noise reduction of low coherence interferometry LCI and optical coherence tomography OCT signals by parallel detection of spectral bands
US8054468B2 (en) * 2003-01-24 2011-11-08 The General Hospital Corporation Apparatus and method for ranging and noise reduction of low coherence interferometry LCI and optical coherence tomography OCT signals by parallel detection of spectral bands
CN1741768A (en) * 2003-01-24 2006-03-01 通用医疗有限公司 System and method for identifying tissue using low-coherence interferometry
EP1611470B1 (en) 2003-03-31 2015-10-14 The General Hospital Corporation Speckle reduction in optical coherence tomography by path length encoded angular compounding
EP2011434A3 (en) 2003-06-06 2009-03-25 The General Hospital Corporation Process and apparatus for a wavelength tuned light source
EP2278287B1 (en) 2003-10-27 2016-09-07 The General Hospital Corporation Method and apparatus for performing optical imaging using frequency-domain interferometry
GB2414616A (en) * 2004-05-28 2005-11-30 Sony Uk Ltd Comparing test image with a set of reference images
TW200529794A (en) * 2004-03-03 2005-09-16 Chi-Wen Hsieh Method of automatically evaluating age of bone and skeleton via hand X-ray photograph
JP2005276002A (en) 2004-03-26 2005-10-06 Fuji Photo Film Co Ltd System for supporting diagnosis and method to be used for the same system
US7394946B2 (en) 2004-05-18 2008-07-01 Agfa Healthcare Method for automatically mapping of geometric objects in digital medical images
WO2005119574A2 (en) * 2004-05-26 2005-12-15 Guardian Technologies International, Inc. System and method for identifying objects of interest in image data
US7840048B2 (en) * 2004-05-26 2010-11-23 Guardian Technologies International, Inc. System and method for determining whether there is an anomaly in data
JP4750786B2 (en) * 2004-05-29 2011-08-17 ザ ジェネラル ホスピタル コーポレイション Chromatic dispersion compensation process, system and software configuration using refractive layer in optical coherence tomography (OCT) imaging
US7447408B2 (en) * 2004-07-02 2008-11-04 The General Hospital Corproation Imaging system and related techniques
EP1782020B1 (en) 2004-08-06 2012-10-03 The General Hospital Corporation Process, system and software arrangement for determining at least one location in a sample using an optical coherence tomography
JP5324095B2 (en) 2004-08-24 2013-10-23 ザ ジェネラル ホスピタル コーポレイション Method and apparatus for imaging blood vessel segments
WO2006024014A2 (en) 2004-08-24 2006-03-02 The General Hospital Corporation Process, system and software arrangement for measuring a mechanical strain and elastic properties of a sample
KR101269455B1 (en) 2004-09-10 2013-05-30 더 제너럴 하스피탈 코포레이션 System and method for optical coherence imaging
US7366376B2 (en) 2004-09-29 2008-04-29 The General Hospital Corporation System and method for optical coherence imaging
EP1815377A2 (en) 2004-11-18 2007-08-08 Dental Medicine America, Llc Describing a periodontal disease state
US7995210B2 (en) * 2004-11-24 2011-08-09 The General Hospital Corporation Devices and arrangements for performing coherence range imaging using a common path interferometer
US8922781B2 (en) 2004-11-29 2014-12-30 The General Hospital Corporation Arrangements, devices, endoscopes, catheters and methods for performing optical imaging by simultaneously illuminating and detecting multiple points on a sample
US20080159605A1 (en) 2005-03-15 2008-07-03 Ramsay Thomas E Method for characterizing an image source utilizing predetermined color spaces
US20060269135A1 (en) * 2005-03-15 2006-11-30 Ramsay Thomas E System and method for identifying objects of interest in image data
AU2006229785A1 (en) * 2005-03-24 2006-10-05 Optasia Medical Limited Method and system for characterization of knee joint morphology
JP2008538612A (en) * 2005-04-22 2008-10-30 ザ ジェネラル ホスピタル コーポレイション Configuration, system, and method capable of providing spectral domain polarization sensitive optical coherence tomography
JP5684452B2 (en) * 2005-04-28 2015-03-11 ザ ジェネラル ホスピタル コーポレイション System, method and software apparatus for evaluating information related to anatomical structures by optical interferometry
US9020955B2 (en) * 2005-05-12 2015-04-28 Hewlett-Packard Development Company, L.P. Compositional balance driven content retrieval
EP1889037A2 (en) * 2005-06-01 2008-02-20 The General Hospital Corporation Apparatus, method and system for performing phase-resolved optical frequency domain imaging
JP5144506B2 (en) * 2005-06-02 2013-02-13 トードベルグ、ハンス・ヘンリク Determination method of skeletal maturity
KR101387454B1 (en) 2005-08-09 2014-04-22 더 제너럴 하스피탈 코포레이션 Apparatus, methods and storage medium for performing polarization-based quadrature demodulation in optical coherence tomography
US20070081699A1 (en) * 2005-09-29 2007-04-12 General Electric Company Systems, methods and apparatus for diagnosis of disease from categorical indices
US20070081701A1 (en) * 2005-09-29 2007-04-12 General Electric Company Systems, methods and apparatus for tracking progression and tracking treatment of disease from categorical indices
US20070081700A1 (en) * 2005-09-29 2007-04-12 General Electric Company Systems, methods and apparatus for creation of a database of images from categorical indices
US7872759B2 (en) 2005-09-29 2011-01-18 The General Hospital Corporation Arrangements and methods for providing multimodality microscopic imaging of one or more biological structures
US7929737B2 (en) * 2005-09-29 2011-04-19 General Electric Company Method and system for automatically generating a disease severity index
US7889348B2 (en) * 2005-10-14 2011-02-15 The General Hospital Corporation Arrangements and methods for facilitating photoluminescence imaging
GB2431793B (en) 2005-10-31 2011-04-27 Sony Uk Ltd Image processing
EP1971848B1 (en) * 2006-01-10 2019-12-04 The General Hospital Corporation Systems and methods for generating data based on one or more spectrally-encoded endoscopy techniques
JP4690204B2 (en) * 2006-01-16 2011-06-01 富士フイルム株式会社 Image reproduction apparatus and program thereof
JP2009523574A (en) * 2006-01-18 2009-06-25 ザ ジェネラル ホスピタル コーポレイション System and method for generating data using one or more endoscopic microscopy methods
WO2007084995A2 (en) 2006-01-19 2007-07-26 The General Hospital Corporation Methods and systems for optical imaging of epithelial luminal organs by beam scanning thereof
WO2007084903A2 (en) 2006-01-19 2007-07-26 The General Hospital Corporation Apparatus for obtaining information for a structure using spectrally-encoded endoscopy techniques and method for producing one or more optical arrangements
US20070171433A1 (en) * 2006-01-20 2007-07-26 The General Hospital Corporation Systems and processes for providing endogenous molecular imaging with mid-infrared light
WO2007149603A2 (en) 2006-02-01 2007-12-27 The General Hospital Corporation Apparatus for applying a plurality of electro-magnetic radiations to a sample
US10426548B2 (en) 2006-02-01 2019-10-01 The General Hosppital Corporation Methods and systems for providing electromagnetic radiation to at least one portion of a sample using conformal laser therapy procedures
WO2007092911A2 (en) * 2006-02-08 2007-08-16 The General Hospital Corporation Methods, arrangements and systems for obtaining information associated with an anatomical sample using optical microscopy
WO2007101026A2 (en) 2006-02-24 2007-09-07 The General Hospital Corporation Methods and systems for performing angle-resolved fourier-domain optical coherence tomography
EP3150110B1 (en) 2006-05-10 2020-09-02 The General Hospital Corporation Processes, arrangements and systems for providing frequency domain imaging of a sample
US7720577B2 (en) 2006-05-17 2010-05-18 The Boeing Company Methods and systems for data link front end filters for sporadic updates
US7676064B2 (en) 2006-05-17 2010-03-09 The Boeing Company Sensor scan planner
US7999849B2 (en) 2006-05-17 2011-08-16 The Boeing Company Moving object detection
US7702183B1 (en) 2006-05-17 2010-04-20 The Boeing Company Methods and systems for the detection of the insertion, removal, and change of objects within a scene through the use of imagery
US20100165335A1 (en) * 2006-08-01 2010-07-01 The General Hospital Corporation Systems and methods for receiving and/or analyzing information associated with electro-magnetic radiation
EP3006920A3 (en) * 2006-08-25 2016-08-03 The General Hospital Corporation Apparatus and methods for enhancing optical coherence tomography imaging using volumetric filtering techniques
US8838213B2 (en) 2006-10-19 2014-09-16 The General Hospital Corporation Apparatus and method for obtaining and providing imaging information associated with at least one portion of a sample, and effecting such portion(s)
WO2008087556A2 (en) * 2007-01-16 2008-07-24 Optasia Medical, Limited Image processing systems and methods
US7949019B2 (en) * 2007-01-19 2011-05-24 The General Hospital Wavelength tuning source based on a rotatable reflector
JP5507258B2 (en) * 2007-01-19 2014-05-28 ザ ジェネラル ホスピタル コーポレイション Apparatus and method for controlling measurement depth in optical frequency domain imaging
WO2008118781A2 (en) 2007-03-23 2008-10-02 The General Hospital Corporation Methods, arrangements and apparatus for utilizing a wavelength-swept laser using angular scanning and dispersion procedures
US10534129B2 (en) 2007-03-30 2020-01-14 The General Hospital Corporation System and method providing intracoronary laser speckle imaging for the detection of vulnerable plaque
US20100128957A1 (en) * 2007-04-12 2010-05-27 Konica Minolta Medical & Graphic, Inc. Bone disease evaluating system
WO2008131082A1 (en) * 2007-04-17 2008-10-30 The General Hospital Corporation Apparatus and methods for measuring vibrations using spectrally-encoded endoscopy techniques
WO2009018456A2 (en) * 2007-07-31 2009-02-05 The General Hospital Corporation Systems and methods for providing beam scan patterns for high speed doppler optical frequency domain imaging
US7933021B2 (en) * 2007-10-30 2011-04-26 The General Hospital Corporation System and method for cladding mode detection
WO2009060355A1 (en) * 2007-11-08 2009-05-14 Philips Intellectual Property & Standards Gmbh Computer-aided diagnosis with queries based on regions of interest
KR100942699B1 (en) * 2007-12-10 2010-02-17 한국전자통신연구원 Method and system for extracting distal radius metaphysis
US20090225324A1 (en) * 2008-01-17 2009-09-10 The General Hospital Corporation Apparatus for providing endoscopic high-speed optical coherence tomography
US7898656B2 (en) 2008-04-30 2011-03-01 The General Hospital Corporation Apparatus and method for cross axis parallel spectroscopy
JP5607610B2 (en) 2008-05-07 2014-10-15 ザ ジェネラル ホスピタル コーポレイション Apparatus for determining structural features, method of operating apparatus and computer-accessible medium
US8430816B2 (en) * 2008-05-20 2013-04-30 General Electric Company System and method for analysis of multiple diseases and severities
US8126249B2 (en) * 2008-05-30 2012-02-28 Optasia Medical Limited Methods of and system for detection and tracking of osteoporosis
EP2288948A4 (en) 2008-06-20 2011-12-28 Gen Hospital Corp Fused fiber optic coupler arrangement and method for use thereof
US9254089B2 (en) 2008-07-14 2016-02-09 The General Hospital Corporation Apparatus and methods for facilitating at least partial overlap of dispersed ration on at least one sample
US8588485B2 (en) * 2008-11-25 2013-11-19 Carestream Health, Inc. Rendering for improved diagnostic image consistency
US8937724B2 (en) * 2008-12-10 2015-01-20 The General Hospital Corporation Systems and methods for extending imaging depth range of optical coherence tomography through optical sub-sampling
WO2010090837A2 (en) * 2009-01-20 2010-08-12 The General Hospital Corporation Endoscopic biopsy apparatus, system and method
EP2382456A4 (en) * 2009-01-26 2012-07-25 Gen Hospital Corp System, method and computer-accessible medium for providing wide-field superresolution microscopy
CA2749670A1 (en) * 2009-02-04 2010-08-12 The General Hospital Corporation Apparatus and method for utilization of a high-speed optical wavelength tuning source
BR112012001042A2 (en) * 2009-07-14 2016-11-22 Gen Hospital Corp fluid flow measurement equipment and method within anatomical structure.
HUE051135T2 (en) 2010-03-05 2021-03-01 Massachusetts Gen Hospital Systems which provide microscopic images of at least one anatomical structure at a particular resolution
JP5438647B2 (en) * 2010-03-31 2014-03-12 富士フイルム株式会社 Radiation imaging system
US9401047B2 (en) * 2010-04-15 2016-07-26 Siemens Medical Solutions, Usa, Inc. Enhanced visualization of medical image data
US9069130B2 (en) 2010-05-03 2015-06-30 The General Hospital Corporation Apparatus, method and system for generating optical radiation from biological gain media
WO2011149972A2 (en) 2010-05-25 2011-12-01 The General Hospital Corporation Systems, devices, methods, apparatus and computer-accessible media for providing optical imaging of structures and compositions
JP5778762B2 (en) 2010-05-25 2015-09-16 ザ ジェネラル ホスピタル コーポレイション Apparatus and method for spectral analysis of optical coherence tomography images
JP6066901B2 (en) 2010-06-03 2017-01-25 ザ ジェネラル ホスピタル コーポレイション Method for apparatus and device for imaging structures in or in one or more luminal organs
US8934685B2 (en) * 2010-09-21 2015-01-13 General Electric Company System and method for analyzing and visualizing local clinical features
EP2632324A4 (en) 2010-10-27 2015-04-22 Gen Hospital Corp Apparatus, systems and methods for measuring blood pressure within at least one vessel
US20140142861A1 (en) * 2010-11-06 2014-05-22 Oklahoma Medical Research Foundation Biomarkers For Predicting Progressive Joint Damage
WO2013013049A1 (en) 2011-07-19 2013-01-24 The General Hospital Corporation Systems, methods, apparatus and computer-accessible-medium for providing polarization-mode dispersion compensation in optical coherence tomography
WO2013029047A1 (en) 2011-08-25 2013-02-28 The General Hospital Corporation Methods, systems, arrangements and computer-accessible medium for providing micro-optical coherence tomography procedures
US9341783B2 (en) 2011-10-18 2016-05-17 The General Hospital Corporation Apparatus and methods for producing and/or providing recirculating optical delay(s)
EP2833776A4 (en) 2012-03-30 2015-12-09 Gen Hospital Corp Imaging system, method and distal attachment for multidirectional field of view endoscopy
WO2013177154A1 (en) 2012-05-21 2013-11-28 The General Hospital Corporation Apparatus, device and method for capsule microscopy
JP6227652B2 (en) 2012-08-22 2017-11-08 ザ ジェネラル ホスピタル コーポレイション System, method, and computer-accessible medium for fabricating a miniature endoscope using soft lithography
WO2014117130A1 (en) 2013-01-28 2014-07-31 The General Hospital Corporation Apparatus and method for providing diffuse spectroscopy co-registered with optical frequency domain imaging
WO2014120791A1 (en) 2013-01-29 2014-08-07 The General Hospital Corporation Apparatus, systems and methods for providing information regarding the aortic valve
US11179028B2 (en) 2013-02-01 2021-11-23 The General Hospital Corporation Objective lens arrangement for confocal endomicroscopy
EP2967491B1 (en) 2013-03-15 2022-05-11 The General Hospital Corporation A transesophageal endoscopic system for determining a mixed venous oxygen saturation of a pulmonary artery
EP2997354A4 (en) 2013-05-13 2017-01-18 The General Hospital Corporation Detecting self-interefering fluorescence phase and amplitude
WO2015010133A1 (en) 2013-07-19 2015-01-22 The General Hospital Corporation Determining eye motion by imaging retina. with feedback
EP4349242A2 (en) 2013-07-19 2024-04-10 The General Hospital Corporation Imaging apparatus and method which utilizes multidirectional field of view endoscopy
US9668652B2 (en) 2013-07-26 2017-06-06 The General Hospital Corporation System, apparatus and method for utilizing optical dispersion for fourier-domain optical coherence tomography
WO2015105870A1 (en) 2014-01-08 2015-07-16 The General Hospital Corporation Method and apparatus for microscopic imaging
WO2015116986A2 (en) 2014-01-31 2015-08-06 The General Hospital Corporation System and method for facilitating manual and/or automatic volumetric imaging with real-time tension or force feedback using a tethered imaging device
US10228556B2 (en) 2014-04-04 2019-03-12 The General Hospital Corporation Apparatus and method for controlling propagation and/or transmission of electromagnetic radiation in flexible waveguide(s)
US10912462B2 (en) 2014-07-25 2021-02-09 The General Hospital Corporation Apparatus, devices and methods for in vivo imaging and diagnosis
WO2016022072A1 (en) * 2014-08-06 2016-02-11 Singapore Health Services Pte Ltd Method and system for automatically identifying joints within a subject's body
US9833198B2 (en) * 2015-02-04 2017-12-05 Francis J. Stapleton System and method for obtaining an objective dental health analysis
CN105989174B (en) * 2015-03-05 2019-11-01 欧姆龙株式会社 Region-of-interest extraction element and region-of-interest extracting method
US10595941B2 (en) * 2015-10-30 2020-03-24 Orthosensor Inc. Spine measurement system and method therefor
KR101977174B1 (en) * 2017-09-13 2019-05-10 이재준 Apparatus, method and computer program for analyzing image
CN108056786B (en) * 2017-12-08 2020-06-30 浙江大学 Bone age detection method and device based on deep learning
CN108491770B (en) * 2018-03-08 2023-05-30 李书纲 Data processing method based on fracture image
EP3847951A4 (en) * 2018-09-04 2022-04-20 Bonewise Inc. Bone age assessment device, method and recording medium for recording program
CN110349653A (en) * 2019-07-15 2019-10-18 杭州依图医疗技术有限公司 Display methods, equipment and the storage medium of image analysing computer data
US10939044B1 (en) * 2019-08-27 2021-03-02 Adobe Inc. Automatically setting zoom level for image capture
CN110782450B (en) * 2019-10-31 2020-09-29 北京推想科技有限公司 Hand carpal development grade determining method and related equipment
WO2021118068A1 (en) * 2019-12-09 2021-06-17 주식회사 뷰노 Medical image generation method and device using same
US20210398280A1 (en) * 2020-06-23 2021-12-23 Vuno Inc. Bone age assessment method for bone image
CN113221881B (en) * 2021-04-30 2022-03-22 浙大城市学院 Multi-level smart phone screen defect detection method

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4437161A (en) * 1981-06-29 1984-03-13 Siemens Gammasonics Inc. Medical imaging apparatus
US4791934A (en) * 1986-08-07 1988-12-20 Picker International, Inc. Computer tomography assisted stereotactic surgery system and method
US5740266A (en) * 1994-04-15 1998-04-14 Base Ten Systems, Inc. Image processing system and method
US5839438A (en) 1996-09-10 1998-11-24 Neuralmed, Inc. Computer-based neural network system and method for medical diagnosis and interpretation
EP1000470A2 (en) 1997-07-01 2000-05-17 QUALCOMM Incorporated Method and apparatus for controlling signal power in a communication system
WO2000033252A1 (en) 1998-11-25 2000-06-08 General Electric Company Methods and apparatus for calcification scoring

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4437161A (en) * 1981-06-29 1984-03-13 Siemens Gammasonics Inc. Medical imaging apparatus
US4791934A (en) * 1986-08-07 1988-12-20 Picker International, Inc. Computer tomography assisted stereotactic surgery system and method
US5740266A (en) * 1994-04-15 1998-04-14 Base Ten Systems, Inc. Image processing system and method
US5839438A (en) 1996-09-10 1998-11-24 Neuralmed, Inc. Computer-based neural network system and method for medical diagnosis and interpretation
EP1000470A2 (en) 1997-07-01 2000-05-17 QUALCOMM Incorporated Method and apparatus for controlling signal power in a communication system
WO2000033252A1 (en) 1998-11-25 2000-06-08 General Electric Company Methods and apparatus for calcification scoring

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"A Digital Hand Atlas for Bone Age Assessment of Children-An Application of PACS Database" , H.K. Huang et al., SPIE vol. 3662, Feb. 1999, pp. 178-184.
"Finding-Specific Display Presets for Computed Radiography Soft- Copy Reading", Katherine P. Andriole et al., Journal of Digital Imaging, vol. 12, No. 2, May 1999, pp. 3-5.

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030229299A1 (en) * 2002-06-05 2003-12-11 Fuji Photo Film Co., Ltd. Method of and system for evaluating joint and computer program for carrying out the method
US7634301B2 (en) * 2003-09-17 2009-12-15 Koninklijke Philips Electronics N.V. Repeated examination reporting
US20070064981A1 (en) * 2003-09-17 2007-03-22 Meijer Eric L Repeated examination reporting
US8280482B2 (en) * 2004-04-19 2012-10-02 New York University Method and apparatus for evaluating regional changes in three-dimensional tomographic images
US20050244036A1 (en) * 2004-04-19 2005-11-03 Henry Rusinek Method and apparatus for evaluating regional changes in three-dimensional tomographic images
US20090046953A1 (en) * 2004-04-29 2009-02-19 Koninklijke Philips Electronics N V Image Processing Apparatus And Method
US20070269086A1 (en) * 2006-03-22 2007-11-22 Kerwin William S Conditional shape model for image processing
US7894664B2 (en) 2006-03-22 2011-02-22 University Of Washington Conditional shape model for image processing
US8538776B2 (en) * 2006-10-25 2013-09-17 Bruce Reiner Method and apparatus of providing a radiation scorecard
US20080103834A1 (en) * 2006-10-25 2008-05-01 Bruce Reiner Method and apparatus of providing a radiation scorecard
US8412544B2 (en) * 2007-10-25 2013-04-02 Bruce Reiner Method and apparatus of determining a radiation dose quality index in medical imaging
US20110270623A1 (en) * 2007-10-25 2011-11-03 Bruce Reiner Method and apparatus of determining a radiation dose quality index in medical imaging
US20110058725A1 (en) * 2009-09-10 2011-03-10 General Electric Company Methods and apparatus for measuring body circumference
US8280138B2 (en) 2009-09-10 2012-10-02 General Electric Company System and method for performing bone densitometer measurements
US8295570B2 (en) 2009-09-10 2012-10-23 General Electric Company Methods and apparatus for measuring body circumference
US8300911B1 (en) 2009-09-10 2012-10-30 General Electric Company Methods and apparatus for measuring visceral fat mass
US20110058726A1 (en) * 2009-09-10 2011-03-10 General Electric Company System and method for performing bone densitometer measurements
US20110060247A1 (en) * 2009-09-10 2011-03-10 General Electric Company Methods and apparatus for measuring bone lengths
US20130064467A1 (en) * 2009-12-17 2013-03-14 Elta Systems Ltd. Method and system for enhancing an image
US8724918B2 (en) * 2009-12-17 2014-05-13 Elta Systems Ltd. Method and system for enhancing an image
US20120290957A1 (en) * 2011-05-12 2012-11-15 Jonathan Chernilo User interface for medical diagnosis
US8935628B2 (en) * 2011-05-12 2015-01-13 Jonathan Chernilo User interface for medical diagnosis
US11308662B2 (en) * 2016-08-02 2022-04-19 Shanghai United Imaging Healthcare Co., Ltd. System and method for image reconstruction
US11869120B2 (en) 2016-08-02 2024-01-09 Shanghai United Imaging Healthcare Co., Ltd. System and method for image reconstruction

Also Published As

Publication number Publication date
JP2003175023A (en) 2003-06-24
US20030053673A1 (en) 2003-03-20
EP1293925A1 (en) 2003-03-19

Similar Documents

Publication Publication Date Title
US7065235B2 (en) Radiographic scoring method
US5740267A (en) Radiographic image enhancement comparison and storage requirement reduction system
US6901277B2 (en) Methods for generating a lung report
US7130457B2 (en) Systems and graphical user interface for analyzing body images
Ruikar et al. Automated fractured bone segmentation and labeling from CT images
US9020235B2 (en) Systems and methods for viewing and analyzing anatomical structures
Pietka et al. Integration of computer assisted bone age assessment with clinical PACS
US8355553B2 (en) Systems, apparatus and processes for automated medical image segmentation using a statistical model
US6697506B1 (en) Mark-free computer-assisted diagnosis method and system for assisting diagnosis of abnormalities in digital medical images using diagnosis based image enhancement
EP2100275B1 (en) Comparison workflow automation by registration
US20030028401A1 (en) Customizable lung report generator
Forsyth et al. Assessment of an automated cephalometric analysis system
EP3100209B1 (en) Extraction of information from an image and inclusion thereof in a clinical report
US20040151358A1 (en) Medical image processing system and method for processing medical image
CN111210897A (en) Processing medical images
JP2004195213A (en) Initialization method of model-based interpretation of radiograph
US10950019B2 (en) Automatic layout apparatus, automatic layout method, and automatic layout program
CN111986182A (en) Auxiliary diagnosis method, system, electronic device and storage medium
CN113744847A (en) Method and system for medical image annotation tool
Reeves et al. Large-scale image region documentation for fully automated image biomarker algorithm development and evaluation
Piętka et al. Role of radiologists in CAD life-cycle
Pietka et al. Computer-assisted bone age assessment: graphical user interface for image processing and comparison
Parascandolo et al. Computer aided diagnosis: state-of-the-art and application to musculoskeletal diseases
Meesters et al. Multi atlas-based muscle segmentation in abdominal CT images with varying field of view
CN113674261B (en) Bone detection method, system, electronic device and storage medium

Legal Events

Date Code Title Description
AS Assignment

Owner name: AGFA-GEVAERT, BELGIUM

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DEWAELE, PIET;REEL/FRAME:013285/0717

Effective date: 20020703

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: AGFA HEALTHCARE N.V., BELGIUM

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AGFA-GEVAERT N.V.;REEL/FRAME:020254/0713

Effective date: 20071108

FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.)

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.)

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20180620

FP Lapsed due to failure to pay maintenance fee

Effective date: 20180620