WO2009095775A1

WO2009095775A1 - Digital scanner

Info

Publication number: WO2009095775A1
Application number: PCT/IB2009/000160
Authority: WO
Inventors: Shamus Fairhall
Original assignee: Shamus Fairhall
Priority date: 2008-01-28
Filing date: 2009-01-28
Publication date: 2009-08-06

Abstract

A digital scanner (1) allows three-dimensional imaging of an object such as a human foot. The scanner (1) includes one or more cameras (6) and mirrors (8) arranged around an object enabling the cameras (6) to receive multiple images of the object. The apparatus may also provide an indication of pressure exerted on a body part by using the colour variation in the skin of the body part to indicate the pressure over one or more areas of the body part.

Description

DIGITAL SCANNER

Field of Invention

The invention relates generally to scanning apparatus and methods for scanning objects such as body parts. More particularly, embodiments of the invention provide for three- dimensional scanning of feet to produce a three-dimensional model which can be used to create foot orthotics and/or medical grade footwear.

Background to the Invention

The traditional way of manufacturing orthotics has been to cast the patients in a non weight bearing position using Plaster of Paris raps and/or slabs. These casts are then sent off for use in the manufacturing of orthotics. This system has proved to work in that it is easy to do is portable and the set up time and costs are minimal. However it is also very messy, the results are variable and non repeatable. It requires a high degree of skill to get it right and the casts that the orthotics are manufactured of are often damaged in transit to the manufacturer. Shipping of the casts also accounts for a fair amount of the time frame the person waits to receive the finished product.

Clinical practices have began to change as well with weight bearing casting starting to be used in both corrected and non-corrected positions. In the last decade several companies have developed "electronic casting" devices to address the mess and accuracy issues. The best of these devices have also addressed the repeatability issues and some devices have reduced the skill level required to get an appropriate clinical outcome. Most address the shipping issue by manufacturing on the spot or sending the data by the internet to a central fabrication centre for CAM (Computer Aided Manufacturing).

A method exists for forming three-dimensional images using lasers and cameras wherein a line is projected by a laser emitter onto the object to be imaged and a camera is offset from the emitter by a set distance. The camera takes an image of the laser line which is distorted by being projected onto the surface. The laser line is moved across the object until multiple images are taken which represents the entire structure. Computer software calculates the distance between the calibrated undistorted line and the distorted line of each image and builds a three-dimensional representation of the object. This method is particularly time consuming as the line has to be scanned across the entire surface, this produces particular problems because the object that is scanned have to remain completely stationery for an extended period of time.

United States patent 5,689,446 describes a device for creating a digital representation of a foot. The underside is measured by placing the foot on a platform with gauge pins. The gauge pins are elevated using a diaphragm, locked in place and the foot is removed. An image of the conforming gauge pin array is acquired and the image is compared to a reference image to determine the height of the gauge pins. The top of the foot is scanned by projecting a light bar onto the object, taking an image of the distorted bar, and then sweeping the light bar by a known amount. This process is repeated until the entire foot has been scanned. The heel of the foot is scanned similarly with the additional use of a mirror. The images of the distorted light bar is compared with reference images and the profile is determined.

A problem with this system is that the accuracy on the underside of the foot depends on the resolution of the array of gauge pins. Another problem is that the act of scanning the rest of the foot is time consuming, as the user has to remain stationery while the light beam is scanned across the foot.

United States patent 5,911 ,126 describes a method for digitising three-dimensional sensing of the shape of bodies or body parts. The body or body part is covered with a flexible tight fitting envelope with a high contrast pattern on the surface thereof. The body or body part is inserted into a device and is imaged by several cameras. The cameras cooperate in the manner of stereo cameras having overlapping image areas and in which the digitised data of the three-dimensional shape are obtained by automatic coordination of corresponding patterns recorded in the images.

A problem with this system is that multiple cameras have to be used which is costly and it increases the complexity of the device.

Other systems exists that use stereoscopic cameras or flatbed scanners. Some systems use projectors to project lines and patterns onto objects which are recorded with a camera, the lines and patterns are then compared to reference scans and a three- dimensional profile is built. A problem with these systems are they are capable only of producing a half image, or hollow shell digital model of the object being scanned.

Most of these devices are very large and heavy, the cheapest is USD30.000 up to USD250,000 for some of the high end systems. None have been widely used the podiatrists due to mainly the cost and size. This has been especially true in many countries where most practices are small independent practices. Plaster of Paris casting is still the most commonly used practice by far.

Summary of the Invention

It is an object of the invention to provide an apparatus and/or method for producing a three-dimensional digital image of an object, or which provides an indication of pressure or blood flow in a body part, that mitigates at least some of the aforementioned problems.

Alternatively, it is an object of the invention to provide at least a useful choice to the public.

According to a first aspect of the invention, there is provided digital scanner apparatus for the three-dimensional imaging of an object, the digital scanner comprising:

one or more cameras; and,

a plurality of reflective means arranged around an object enabling the cameras to receive multiple images of the object.

Preferable, the object is a foot.

Preferably, the reflective means are mirrors.

Preferably, the cameras are digital cameras.

Preferably, the object is covered with a tight fitting envelope which has a pattern on it. Preferably, the tight fitting envelope is a sock.

Preferably, the pattern is random.

Preferably, the image taken by the cameras overlap enabling a three-dimensional image to be formed.

Preferably, the apparatus includes a light source.

Preferably, the light source is capable of overpowering ambient light.

Preferably, the apparatus includes a clear surface on which the object can rest while the image is taken.

According to a second aspect of the invention, there is provided a method of producing a three-dimensional digital image of an object using the apparatus according to the first embodiment, the method comprising:

placing an object in the apparatus according to the first embodiment; and,

taking multiple images of the object at substantially the same time and using the images to form a three-dimensional digital representation.

Preferably, the image from the camera is processed by a computer or such device to produce a digital three-dimensional representation of the object.

According to a third aspect of the invention, there is provided a method of determining an indication of pressure exerted on a body part, the method comprising:

subjecting the body part to pressure; taking an image of the body part under pressure; and

using the colour variation in the skin of the body part to indicate the pressure over one or more areas of the body part.

Preferably, the calculation is performed by a computer.

The invention may also broadly be said to consist in any novel feature or combination of features disclosed herein.

Brief Description of the Drawings

Embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings in which:

Figure 1 shows an isometric sketch of imaging or scanning apparatus.

Figure 2 shows a front elevation in cross section of the apparatus of Figure 1.

Figure 2a shows a diagrammatic spatial arrangement of mirrors that may be used in apparatus such as that of Figure 1 to image one side of an object.

Figure 3 shows images (of each side) of a foot in a sock obtained using the apparatus of the preceding figures.

Figure 4 shows images (of each side) of a calibration block obtained using the apparatus of the preceding figures.

Figure 5 shows a screen shot of an example of a 3D image provided by the invention.

Figures 6 to 8 are flow charts showing use or operation of the invention. Figures 9 to 13 show images produced by the invention representative of a pressure scan of a foot.

Detailed Description of Preferred Embodiments

Referring to figure 1 , scanning apparatus is shown, generally referenced 1. The apparatus is but one embodiment of the present invention, and allows a human foot to be scanned or imaged to allow a three dimensional (3D) representation of the foot to be generated from the images obtained. In one embodiment the apparatus 1 produces images that are suitable for 3D model generation software to convert to 3 dimensional anatomically accurate vim I (Virtual Reality Modeling Language) models.

The apparatus 1 includes a base unit 2 on which a foot plate 3 is mounted. In the embodiment described the foot plate 3 is constructed from a transparent material and is able to carry the weight of a typical human being, so that a user may stand on the foot plate, as will be described further below. A calibration block (not shown) may be provided as will be described further below.

A lid 4 is hingedly or pivotally connected to the base 2, allowing the base 2 to be uncovered so that a user may step onto the foot plate 3, or alternatively allowing the foot plate 3 to be covered when not in use so that the unit may be stored or transported. The orientation and alignment of the lid 4 with respect to the base is achieved using a hinge or linkage mechanism (not shown). In the position shown in Figure 1 , the lid 4 is in an open position, ready for an object, such as a human foot, to be placed on foot plate 3. Although the term foot plate is used in this document, the apparatus is not limited to use for scanning feet.

Arms 5 project from lid 4 and may be provided to support flash units 7. In a preferred embodiment the arms 5 are connected to the lid 4 and/or base unit 2 by a linkage arrangement such that they allow flash units 7 to be disposed in an appropriate position in use, as will be described further below. Arms 5 may also carry mirrors if desired. The lid 4 also includes one or more cameras 6. In the embodiment shown, two cameras 6 are provided, one camera 6 being arranged at each front corner of the lid 4. Flash units 7 and optical instruments such as mirrors 8 allow the cameras 6 to capture images of an object placed on the footrest, as will be described further below.

The apparatus 1 may also include devices such as an electronic circuit (generally referenced 9) for operating the flash units and appropriate communications apparatus such as a USB hub 10, along with a power supply and/or a battery and charger 11.

Figure 2 shows a diagrammatic front elevation of the apparatus 1 , showing the relationship between the cameras and the mirrors when the lid 4 is in a closed position. In use the object to be scanned is placed centrally on the footrest 3. An example of a possible spatial arrangement of mirrors 8 for one side of that apparatus, to provide images from one side of the object to be imaged, is also shown in Figure 2A.

The mirrors are used to facilitate a complete three-dimensional scan of an object such as a foot. The apparatus uses stereo lithographic technology in a way that is unique. As can be seen from Figures 2 and 2A, there are 5 mirrors on each side, making a total of ten mirrors within the embodiment shown to allow the capture of a complete three dimensional, digital image of a foot or other object placed upon the clear glass footrest 3 within it with just one digital scan. An example of the images produced can be seen in Figure 3.

The mirrors are placed within the device in such a way that each scan produces ten (or more, or fewer depending on the number of mirrors used) overlapping views of the foot. These combined views cover the entire surface of the foot. While the foot is being scanned it rests upon the transparent footplate 3 which is capable of handling the weight of the person whose foot is being scanned, so that the image captured may represent the foot under load if required.

If desirable the load may be varied by simply instructing the person whose foot or feet are being scanned to shift their weight as required. In a preferred embodiment six calibrated photographs are sent to the server when the machine goes on line. Once received by a server, digital software specifically developed for the application extracts a three dimensional digital image of one or both feet by combining all images and correcting distortions caused by perspective. The result is shown in Figure 5.

The cameras 6 provide bifocal vision of the object to be imaged or scanned. In the embodiment shown the cameras are 3.5 Megapixel or greater and are spaced at least 150mm apart. In this embodiment the cameras have a Software Development Kit to allow control from a computer or software and have the ability to operate at a high shutter speed to allow for the very bright light from the flash units 7. Those skilled in the art will appreciate that the invention may be implemented using other reflective devices such as prisms, and that a single camera and flash may be used if appropriate mirrors, prisms or other optical instruments are provided.

The mirrors 8 allow images to be obtained at substantially the same instant and also allow overlapping images to be obtained for a full 360 degrees around the object to be imaged. The fewer cameras greatly reduce cost, and the ability to take multiple images at the same time avoids problems with movement of the object (e.g. a user moving his or her foot during the imaging process). Mirrors also reduce the size of the apparatus and those skilled in the art will appreciate that other forms of mirror (such as convex or oval mirrors) or similar devices may provide further improvements.

Mirrors 8 also have the advantage that they decrease the real space required to provide the required focal length of the cameras ( in this embodiment about 700mm minimum), resulting in greater compactness than an alternative such as a booth design without mirrors. Some of the mirrors 8 may be arranged in the lid 4 or supported by the arms 5.

In the embodiment shown, four remote flashes are used. Two of these are provided on arms as shown in Figure 2. The other two can be provided beneath the footplate 3. The flashes allow ambient light to be drowned out, and so eliminate problems which otherwise occur (for example having to image in darkness). To further minimise ambient light entry into the cameras, the shutter speed is very short, being in the order of 1 millisecond. The flashes 7 are positioned so as not to interfere with the images captured, and have special filters. Three filters are provided for each flash unit in the embodiment described. There are two light dispersion filters to spread light evenly across the object to be imaged, and a polarization filter to condition the light, reducing glare. The filters are discussed further below. The conditioned flash units, in conjunction with the high shutter speed of the cameras, allow the apparatus to be used across a wide variety of ambient light conditions. This provides a significant advantage as the apparatus can be used in a standard environment, such as a retail environment for example, without the requirement for a dark room, or a light impervious seal around the object being imaged.

The flashes 7 are timed to light as soon as a camera is activated, and quickly enough to fit within the 1 millisecond shutter window. One side fires at a time, in sync with each camera 6. The cameras 6 do not fire simultaneously, to prevent the light from one side's flash over-exposing the sensor on the complimentary camera. The layout enables uniform bright lighting of the foot due to flashes only, whilst remaining out of view of the complementary camera. A flash activation circuit enables the rapid ( < 1 millisecond) firing of a single flashbulb from multiple flash capacitors.

In one embodiment, light is dispersed from the bulbs of the flashes 7 using fresnel light diffusers to give even illumination of the foot. Horizontal polarizing filters are applied on the flash bulbs, to complement vertical polarizing filters on the camera lenses, such that no light from the flashes may enter the camera directly, or via reflection in the mirrors. This diminishes light entering the camera from specular reflections, and favours light that has been scattered from the foot alone.

The cameras 6 may be operated a very short space of time apart, so a flash circuit may be provided to operate the flash units 7 associated with each camera accordingly.

The base unit 2 in the embodiment described is attached to the lid 4 in such a way that the upper mirrors 8 fold into the unit when not in use. However, other arrangements may be used.

In the embodiment shown a calibration block (not shown) is used as a reference (known pattern, shade, shape and colour) to calculate the parameters of the imaged object and to determine lens distortion. Images of the calibration block as captured by the cameras of the embodiment described are shown in Figure 4. A sock or similar tight-fitting garment having high contrast markings is placed over the foot or object to be imaged. The markings on the sock provide reference points for determining the 3D representation, and the sock has the added benefit of preventing cross-contamination between users. The colour of the sock allows a determination to be made of what is to be imaged and what is simply background matter. A four way stretch lycra may be used as the sock fabric. The size and shape of the dots remains consistent regardless of the scanning angle. The sock also has the advantage of being a consumable item. Images obtained using the sock are seen in Figure 3.

Referring now to Figures 6 to 8, the invention allows a stand-alone unit to be provided which interfaces with software (which may be provided at a server for example) immediately it goes on line, automatically undertaking all tasks requested (for example requested by an orthotics client or practitioner). Many of the advantages to the client or practitioner offered by the invention arise because of the nature of the comprehensive digital scan it generates of the foot or feet. Other machines that seek to fulfil the same task with laser scanning rely on large electronic files to relay the information needed. These take time to download and further time and operator input to process. However digital information from the apparatus 1 is automatically formatted by computer into a VRML that is then on forwarded automatically as per the instructions of the customer or practitioner expressed through their MSQL database.

These instructions generally include that the information be formatted as G Code for a CNC mill. The system sends this information without the need of further interference to the appropriate CNC mill thereby facilitating a solid representation of the device to be milled from a variety of materials. All that is required of the mill operator is that they load the desired blanks into the machine for milling.

The invention allows anyone properly authorised to access all information needed to track the process from start to finish. Pop up screens are automatically generated to capture additional information. Such data includes client data, manufacturing data, shipping data and billing data. The invention can also capture and generate catalogue images and quotes for different products. No other device that attempts to perform the same task in terms of image capture possesses this capacity for additional data capture and generation. Referring now to Figures 9 to 13, the invention incorporates software technology to make a three dimensional digital image of the foot showing relative pressure at any point. It does this by reading the different colour of skin in the foot resulting from pressure. Skin colour is basically made up of differing ratios. The skin changes colour when pressed under load as blood is forced away from the surface and software is capable of reading these colour variations and rendering them into an image displaying them by showing a three dimensional digital image of the foot in graduated tones.

This is generated in the embodiment described as a hot/cold image made up of blue and white tones where white represents areas of highest pressure and blue the lowest pressure areas. The ratio of blue and white clearly shows the variations of pressure over the entire three dimensional, rotatable image.

The apparatus allows a pressure scan to be produced by working on a set of observations and assumptions, these being that there are two consistent colors in skin. They are yellow and green. The ratio of these colours in the skin depends on the amount of blood in or near the skin. Pressure forces blood from the skin in proportion to the amount of pressure applied. Therefore, the ratio of yellow to green is in proportion to the amount of pressure.

Software takes an image of the plantar surface of the bare foot in the same way as the plantar scan described above. This image is heavily illuminated with a quartz halogen very white light to illuminate the colours. The software then reforms the image using the reference points to give a flat image.

The software then extracts just the yellow and green tones from this image and reforms it as a hot to cold view based on the ratio, white being hot and blue being cold. This image is then translated to a movable rotating 3D model. Thus:

1. The client stands with a bare foot on the glass between the reference points.

2. With the client in an even weight bearing position the cam photo-scan is taken.

3. The process is now repeated for the opposite foot. 4. From here the color pressure chart and 3D model are automatically shown on screen

These scans are used for diagnosis of pressure issues such as metatarsalga as well as to graphically show the client the issues orthotics are addressing. They can also be printed out and sent to other parties such as insurance companies as documented evidence. Furthermore, they can be used to track changes in the foot over time.

In one embodiment the image captured by the camera or scanner is processed as described below.

The image as captured is shown in Figure 9. The next step is to delete the background and determine the orientation of the foot. The orientation is determined by fitting an ellipse to the foot, the angle of ellipse can be calculated by using eigenvector. As the front of foot is wider than the heel, the centre of gravity should be nearer to the front of the foot. Therefore, if the centre is above the middle point, the foot can be assumed to be in an upward orientation. If not, then the image is rotated to provide the desired orientation.

A blue/green filter is then applied to the image. Each pixel of the image the image is processed and is assigned a value according to the formula:

Value = 1 - blue / green

This formula is the simplest example and in practice is changed according to differing situations and environments.

All values are normalized to the number between 0 and 1. The pressure map is based on these values. For example if the value is bigger than 5/6, the result can be: red is 255, green is 255 and blue is (value ^* 6 - 5) * 255. If the value is bigger than 0.5 and smaller than 5/6, the result can be: red is 255, green is (value - 0.5) * 3 * 255 and blue is 0. The final pressure map can be displayed by different colours similar to a heat map (as shown in Figure 11).

The pressure from high to low can be displayed by white, yellow, red, pink, dark pink etcetera. The user can set the number of colour layers required. The finial pressure map can also be displayed in 3D model with different pressure values in the form of a contour or topographical map as shown in Figures 11 and 12.

Diagnosis software may also be used to determine one or more of the following; Hind-foot pronation neutral supination Mid-foot pes cavus neutral pes planus

Forefoot Metatarsalgia. Foot length Foot width

The diagnosis process is as follows: The foot is separated into four parts: Toes (0 - 1/7 length),

Fore foot (1/7 - 3/7),

Mid foot or arch (3/7 - 5/7) Hind foot (5/7 - 1).

From the pressure map referred to above, a determination can be made as to whether the foot is a neutral hind-foot as shown in Fig 13, reference A. The white colour indicates highest pressure. If the centre of gravity of the white area is close to the middle of heel, the hind-foot is neutral.

If the centre of gravity is towards the body mid-line position, the foot is pronated. If the centre of gravity is towards the body's lateral position, the foot is supinated. Using the same rationale if certain colours or shadings are deleted (for example deleting the pink colour), the type of foot arch can be distinguished as shown in Figure 13 reference B. If there is the colour white, as shown in Figure 13 reference C, then there may be metatarsalgia or related issues as this shows a high pressure loading over the width of that area.

It will be seen that these tools can also be used for body parts other than feet.

It should be further noted that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications may be made without departing from the spirit and scope of the invention and without diminishing its attendant advantages. It is therefore intended that such changes and modifications be included within the scope of the invention.

Claims

1. Digital scanner apparatus for the three-dimensional imaging of an object, the digital scanner comprising:

one or more cameras; and,

2. Apparatus as claimed in claim 1 wherein the reflective means are mirrors.

3. Apparatus as claimed in claim 1 or claim 2 wherein the images received by the cameras overlap.

4. Apparatus as claimed in any one of the preceding claims wherein the apparatus includes a light source sufficient to drown out ambient light.

5. Apparatus as claimed in claim 4 wherein the light source includes a filter.

6. A method of producing a three-dimensional digital image of an object using the apparatus according to claim 1 , the method comprising:

placing an object in the apparatus; and,

7. A method of determining an indication of pressure exerted on a body part, the method comprising:

Subjecting the body part to pressure;

taking an image of the body part under pressure; and using the colour variation in the skin of the body part to indicate the pressure over one or more areas of the body part.

8. A method as claimed in claim 7 including the step of determining an indication of the relative pressure difference over one or more areas of the body part.

9. A method as claimed in claim 7 or claim 8 wherein the body part comprises the plantar surface of a human foot.

10. Digital scanner apparatus substantially as herein described.

11. A method of producing a three-dimensional digital image of an object substantially as herein described.

12. A method of determining the relative pressure exerted on a body part substantially as herein described.