WO2003049657A1

WO2003049657A1 - A method and an aid, in particular for the visually impaired

Info

Publication number: WO2003049657A1
Application number: PCT/DK2002/000842
Authority: WO
Inventors: Anders TØNNESEN
Original assignee: Blindsight Aps
Priority date: 2001-12-12
Filing date: 2002-12-11
Publication date: 2003-06-19
Also published as: AU2002361943A1

Abstract

A method for localising, by means of wave signals, moving objects, in particular for the visually impaired comprising transmitting of wave signals, receiving echoes from objects and producing a different signal in response to said transmitted and received signals. The different signal is adapted to correspond to the frequency and dynamic area of the range of the human hearing and is presented to the human ear comprising information about the movements of said object.

Description

A method and an aid, in particular for the visually impaired

The invention relates to a method for localising, by means of wave signals, moving objects, in particular for the visually impaired. Thus, the invention is not limited to the visually impaired; but in the following explanation of the invention, it will be exemplified by the visually impaired.

An example of a patent based on the above-mentioned technique is disclosed in DE 31 33 645 A1 , where the distance to a single object is presented to the user by frequency at a fixed sound level. The bandwidth to the user is very limited and there is only one dimension - the frequency coupled to the distance measured.

Another example is taught in DE 44 02 764 A1 , where the intensity of the reflected signal is transformed to a vibration unit with a bandwidth of 10 Hz.

One patent, US 5 268 692, relates to a safe stopping distance detector which is a much more complex system for concurrent distance measurement of several objects. The transmitted signal is modulated by a microprocessor and the reflected signal is signal-conditioned and is finally reduced to a binary alarm, after evaluation by the microprocessor. The information achievable by this system is also very limited: alarm yes/on.

Yet another patent, US5 818 381 , relates to a highly integrated system, where the radar unit is provided in the form of a pair of spectacles, and part of the radar is integrated in the spectacle frame. The transmitted radio wave is modulated to enable distance measurement, and the transmitted radio wave is extremely focused. This patent does not, however, disclose much about the amount of information communicated to the user.

It is a common feature of the prior art that the known aids are either simple or relatively inexpensive or superior and much more complex. It is the object of the invention to provide a method and an aid, whereby a considerably increased amount of information is communicated than was the case with the prior art, while simultaneously the complexity of the signal processing circuits is reduced considerably.

This object is obtained in that, within a relatively narrow horizontal angle area, wave signals are emitted at a transmitter frequency; that reflected signals are received, the frequency of which deviates from the transmitter frequency in response to the speed of a reflecting object; that a difference signal is produced in response to said transmitted and received signals; that the difference signal is adapted so as to correspond to the frequency and dynamics area of the range of the human hearing; and that the adapted difference signal is introduced to a user as a sound signal, where the information relating to the movement of objects is represented by frequency variations of the sound signal.

The invention is based on the circumstance that the human brain is very flexible and can be trained to form a "visual impression" of the surroundings by receiving a relatively complex "sound picture" provided the brain is trained.

The more the user trains, the more complex sound pictures he/she will be able to interpret, and in accordance with one embodiment the information is represented by frequency variations within a considerable part of the audible frequency range.

Apart from the user being able to tell the various frequencies apart, he/she is of course also able to distinguish between the volumes of the individual signal frequencies. This can be used to obtain an impression of how far away an object is situated, which, however, presupposes some degree of training of the user in case an object moves in a direction that forms an angle to the emitted and transmitted wave signals. However, in practice the user is capable of learning to form a very adequate impression of the distance anyway, but the invention can also be expanded to include an actual measurement of distance by emission of impulse signals, such that the time delay until the reflected signal is received is an expression of the distance. The distance measurement could be presented to the user by a further sound signal which is characteristic of the distance.

Until the user is fully trained it may be convenient to attenuate certain parts of the difference signal, eg by exclusion of some frequencies or by exclusion of signals having an amplitude below a given threshold value.

The signals can be emitted as ultrasound, but in accordance with a preferred embodiment the radar principle is applied in the form of a Doppler radar that produces the above-mentioned difference signal. By using two or more Doppler radars, whose main directions are dissimilar, it is possible to achieve "stereo sound", meaning that the directivity of the information communicated to the user becomes considerably more sensitive.

The visually impaired constitute a very exposed group of people in the traffic, and in accordance with a preferred embodiment the radar frequencies are selected to be such that the difference frequencies that are produced by objects in the traffic give rise to sound signals with frequencies that are comprised within a convenient area of the audible frequency range. Alternatively the difference frequency signal can be converted by frequency shifting or other signal processing, whereby an optimal sound picture is accomplished that the brain can be trained to comprehend.

The invention also relates to a portable aid for the visually impaired and comprises a Doppler radar for producing a difference signal on the basis of an emitted and a received, reflected signal; a circuit configured for adapting the frequency of said signals, in such a manner that the difference signal is to be comprised within the audible human frequency range; a sound emitter for converting the difference signal into a sound signal. Thereby a personal aid is accomplished, wherein the difference signal from the radars are presented acoustically to the user, whereby the user is aided when he/she has to orient in unknown surroundings. Prior to the acoustic presentation, the difference signal or signals are to be subjected to signal processing in order for the signal to be adapted to the human hearing. This signal processing could be, but is not limited to, a frequency shift and a compression of the signal, thereby causing - from a frequency point of view - the difference signal to be located widely within the range of human hearing; the dynamics area also being adapted. The use of two Doppler radars that are, in the in-use position, arranged for emitting signals in unlike horizontal directions, enables the user to acquire improved directional sensitivity, and the signals from each of said two radars being transmitted essentially to each their earphone, it is possible to accomplish a "stereo image" if the correlation is very like the perception of the human hearing of sound emitters in the surroundings.

Now the invention will be explained in further detail by the following embodiment shown in the drawing, wherein:

FIGURE 1 is a schematic view of a Doppler radar;

FIGURE 2 is a schematic view of a preferred embodiment of a Doppler radar applied for the visually impaired;

FIGURE 3 is a visualization of the area covered by the antenna, when the user is holding the apparatus in accordance with the preferred embodiment of the invention;

FIGURE 4 is an explanation of the preferred embodiment of the invention with stereo Doppler radar applied, when an object moves from one side to another; FIGURE 5 is a visualization of the technique used by the user with the preferred embodiment of the invention to follow a hedge.

FIGURE 6 illustrates an example of how to avoid a bicycle.

FIGURE 7 are examples of possible embodiments of the invention with relation to handling the invention during use.

FIGURE 8 shows a possible way of achieving a broader covering area of the radar units.

Best mode for carrying out the invention:

A preferred embodiment of the present invention for the visually impaired is described in FIGURES 1 and 2.

Reference is now made to FIG 1 , where the oscillator 2 is supplied with energy from the power supply 1 , 3. The transmitter/receiver antenna 4 emits the radio waves emitted by the oscillator 2 and receives the reflected radio waves from the surroundings. A portion of the emitted radio waves is mixed with the received radio waves in the mixer 5, and hence the difference signal between the transmitted and received radio waves is found on the output 6 of the mixer 5 in the form of the IF output 6. According to the invention, the oscillator 2 is generating a sufficient high-frequency radio wave, meaning that the output of the mixer 6 is mainly within the range of the human hearing 20- 20KHz for the most common speeds in the surroundings of the visually impaired, 1-200 kmph. In the preferred embodiment of the invention, the frequency of the oscillator is 24.125 Ghz, meaning that a movement of 200 kmph gives an audio frequency of 4.5 KHz.

Turning now to FIG 2, the complete system of the preferred embodiment of the invention can be seen. As the equipment is portable and lightweight, the energy source for the system must be carried along with the system. In the preferred embodiment the battery 10 is integrated with the total system into one unit and a set of earphones 16. The power supply unit 11 transforms the energy supplied by the battery 10 to a stable voltage supply to feed the radar unit 12, the preamplifier 13, the analogue/digital converter (ADC) 14, the processor 17, the digital/analogue converter (DAC) 18 and the amplifier 15. The radar unit 12 generates the IF-output as seen in FIG 1 - object 6. This output from the radar unit 12 is fed into a preamplifier 13, where the signal is amplified to correspond to the input level of the analogue/digital converter 14. The analogue/digital converter 14 is controlled by the processor 17, and hence the output of the preamplifier 13 is converted into digital values at a rate controlled by the processor 17 by the analogue/digital converter 14. The processor 17 is a general-purpose processor that manipulates the digital data stream for even better compliance with the human hearing.

In the preferred embodiment of the invention, this manipulation can be, but is not limited to, an alteration of the dynamic range of the audio signal, a noise- reduction algorithm or a frequency shift of the audio signal. After the processor 17 has manipulated the data-stream from the analogue/digital converter 14, the stream is fed to the DAC 18 and the signal is now an audio signal ready to the presented to the user. First, however, the signal must be amplified by an amplifier 15 and afterwards the signal is fed to the earphones 16. The audio signal can also be fed to the user by a loudspeaker integrated in the unit, thereby obsoleting the cord from the unit to the earphones 16. In either embodiment, it is very important that the earphones 16 or loudspeaker is capable of reproducing the audio signal at a sufficiently high sound level for the visually impaired to hear the sounds.

FIG 3 shows the unit in the preferred embodiment, held by the user 30 and illustrates the area in which the user is able to hear and locate movement 32. When the unit 33, held by the user 30, is viewed from the side 37 it can be seen that the area covered 32 by the radar unit 33 is large compared to the body of the user 30. When viewed from the top 38, the unit 36 is held by the user 31 , it can be seen that the area covered left to right 34, 35 is divided into two areas - one to the left of the user 35 and one to the right 34. By letting each of the areas 34, 35 be covered by each one radar unit integrated into the preferred embodiment of the invention 36, the effect of the stereo perspective occurs. When the radar unit covering the left area 35 is connected to a earphone in the left ear of the user, and the radar unit covering the right area 34 is connected to the earphone of the right ear, the user is able to locate the lateral movement by using the stereo perspective.

FIG 4 demonstrates the stereo perspective in the preferred embodiment of the invention. The stereo perspective is implemented by using two Doppler radar units 20, 21 , where each unit is placed level next to each other, but the two Doppler radar units 20, 21 are angled by a certain degree 25; in the preferred embodiment this angle is 30 degrees. By presenting the output of radar unit 20 to the user's left ear, and the output of the other radar unit 21 to the user's right ear, the user is able to locate the object 22 in front of him. Objects 23, 24 are visualizations of the voltage output of the Doppler radar units 20, 21 during the movement of the object 22 from the left to the right. In FIG 4 the movement from left to right has been divided into three distinct time-periods A, B and C, separated by the covering areas of the two antennas. If the movement in front of the user 22 is moving from the left to the right, the user will experience that the sound generated by the movement is starting in the left ear 23 during time period A, where the sound will be heard only in the left ear. Then the sound will be in both ears 23, 24 during time-period B, and while finishing the movement from the left to the right, the sound will be in the right ear 24 only during time-period C. This experience of the movement is especially important when the object that moves is not generating sound like for instance a bicycle. Silent vehicles that were previously undetectable by the visually impaired can now be detected and located.

FIG 5 shows one of the applications of the invention, namely following a hedge by using the stereoperspective. To the visually impaired 44, walking straight down a pavement 41 is very difficult because of the missing landmarks you would otherwise see. In FIG 5 a visually impaired person 44 is walking forwards on a pavement 41. A hedge 40 follows the left side of the pavement 41. The user 44 holds the invention 47 in the direction of movement, meaning that the user 44 can hear the reflection of the pavement 41 in his right ear. This is due to the fact that the pavement 41 is covered by the radar unit 42, which corresponds to the audio signal given in the user's 44 right ear. In his left ear, the user 44 can hear the reflection of the hedge 40, because the radar unit 43 covering the hedge 40 generates this audio signal. Because the user 44 is moving, the audio signal reflects the speed of the user's 44 movement. As the two radar units are covering very different textures - namely the pavement 41 and the hedge 40 the audio signals generated from the movement are very different. This is because the pavement 41 is a very planar and even surface, where the amount of reflection is very alike. On the other hand, the surface of the hedge 40 is inherently very varying and hence the amount of reflection differs all the time. The reflection from the right radar unit 42 generates a very simple signal 45, which will be very easy for the user 44 to differ from the audio signal generated by the left radar unit 46, as this signal will contain many different frequencies depending of the nature of the hedge 43. What the user has to do is to keep the sound in the right ear very simple and smooth 45, while maintaining the signal with many frequencies in the left ear 46. The user 44 does that by changing the direction of walking, while still moving forwards.

Referring to FIG 6, a visually impaired person is trying to cross a street. The visually impaired person 49 is facing the traffic, standing on the pavement 52. Checking the street for moving objects, he/she swings the apparatus 48 according to the invention from side to side to verify the presence of moving objects in both directions of traffic. In the example, a bicycle 53 moves from left to the right in the illustration; the bicycle moving at a speed designated v meters per second. What the visually impaired person will hear in the headset is a tone, where the frequency of the tone reflects the speed of the bicycle. The frequency of the radio wave reflected by the moving object will be shifted in frequency based on the following formula:

f r = ft / (1 - v / c)

where, fr = frequency received ft = frequency transmitted v = velocity of object c = speed of light

The Doppler radar subtracts the signal transmitted from the signal received, meaning that the difference signal equals:

fd = ft - fr

Using the above formula , rearranging the terms and assuming v « c:

fd = v ^* ft/c

With numbers where ft=24, 125 GHz and c=3 ^* 10⁸ m/sec

fd = v (in kmph) * 22

Meaning: if the bicycle moves at 10 kmph, the frequency of the tone is 220 Hz, if the bicycle moves at 20 kmph, the frequency of the tone is 440 Hz. Theoretically, the visually impaired person 49 is able to calculate the speed of the bicycle 53, but in practice it is often enough to evaluate the speed based on training of the brain of the visually impaired person 49. So the visually impaired person 49 is able to detect the speed of the bicycle 53.

The amplitude of the tone will reflect the size of the moving object. As the bicycle 53 moves into the area covered by the left radar unit 50, the amount of radio waves reflected by the bicycle will increase and hence the amplitude of the tone increases. Now the visually impaired person 49 is able to detect and evaluate the distance to the bicycle 53.

Based on the fact that the preferred embodiment of the invention is equipped with 2 radar units with cover area 50,51 and the fact that the bicycle 53 moves from left to right, the visually impaired person 49 will experience a tone starting with a weak amplitude and a frequency reflecting the speed of the bicycle 53 in his left ear. The tone will increase in amplitude as the bicycle 53 to an increasing degree enters the cover area of the left radar unit 50. When the bicycle 53 enters the common area 50,51 , the amplitude will be stable, but the tone will move from the left ear to both ears, indicating that the moving object 53 is in front of you. Eventually the tone will be in the right ear only to finally fade away. Then the bicycle 53 has passed and the visually impaired person 49 can safely cross the street. So the visually impaired person 49 is able to detect the direction of movement of the bicycle 53.

Turning now to Figure 7, where four possible embodiments of the invention are shown. The preferred embodiment 60 consists of a small, portable unit. This embodiment is associated with the advantage of extreme flexibility and handiness. The drawback is that one hand is occupied. An alternative embodiment 61 integrates the radar units in a cap's brim, presenting the advantage that both hands are free during use of the unit. The unit can be integrated with the white cane 62 giving the advantage that the signal value of the well-known white can is preserved. The last embodiment shown here 63 integrates the Doppler radar unit in a belt.

Figure 8 shows a way of widening the area covered by the Doppler radar units. If the radar units 70, 71 , 72 are placed angled relative to each other with equal angles 73, it is obvious that the radio wave fields will overlap each other 75, 77 and the total area covered by the radar units 74, 75, 76, 77, 78 will increase. If eg three radar units 70, 71 , 72 are used, the middle radar unit 71 could point forwards, while one is angled to the left 70 and another angled to the right 72 relative to the middle unit 71. If an object 79 is moving from the left towards the right, the object 79 would generate an audio signal from the left radar unit 70, followed by a signal from the left 70 and the middle 71 radar unit, while the object 79 is in the area 75 covered by both units. Later the audio signal will be present only from the middle radar unit 71 while in the area 76, followed by two units 71 , 72 while in their common area 77, ending by giving contributions to the audio signal in one radar unit 72 only, while the object is within the respective covering area 78 of the unit 72. At the end of the movement, the object 79 leaves any of the radar units covering areas and hence makes no contributions to the audio signals. If the radio signals from the radar unit 70, 71 , 72 are mixed and presented to the user in the following way:

Left ear: Full audio signal from left radar unit 70 and damped audio signal from the middle unit 71 ;

Right ear: Damped audio signal from the middle unit 71 and full audio signal from the right unit (72)

then the user will still experience the same directivity as a stereo unit, while increasing the possible area where movement is detectable.

In the above-described scenario where an object is moving from the left to the right, the sound will be heard first in the left ear, followed by a transition to both ears, and at the end of the movement, the sound will disappear in the right ear. This gives the user the same experience as if the object 79 were a noisy object that the human hearing could easily place based on the sound level, phase differences, etc. Especially in case the object 79 were a silent bicycle, this technique enables the visually impaired to detect a bicycle, meaning that the user (the visually impaired person) could avoid an accident he may otherwise have caused. By using a Doppler radar any movement in the area covered by the radar unit, will give a difference signal between the transmitted and reflected radio wave. By basing the Doppler radar on a very high radio-wave frequency, any movement will contribute with a audio signal which is within the range of the human hearing. By integrating the radio wave oscillator and radio wave mixer into the same unit, the entire system is so small, that it can be carried by the user. The difference signal from the mixer can vary on two parameters - the frequency and the sound level. The frequency will relate to the speed of the object relative to the speed of the radar unit; it applies in particular that more objects at distinct speeds within the area covered by the radar will each contribute to the mixer output with each their distinct frequency. The sound level will be dependent on the reflectivity of each of the objects, especially the size and the reflectivity against the used radio wave frequency. In contrast to existing products, the invention conveys a very complex "sound- picture" to the user, where each object within the area covered by the radar contributes to the total "sound-picture". The advantages of the "sound- picture", apart from it representing a complete picture of the surroundings, is that it is very similar to the way in which humans use their normal hearing to locate movement.

In the preferred embodiment of the invention, the apparatus is used as a supplement to the ordinary hearing, not a substitute. This eases the user's adaptation to the invention, as the ordinary hearing is not cut off.

The preferred embodiment of the invention can be used by means of two methods:

A. The user does not move the unit; here moving objects within the area covered by the radar unit can be heard. By using the unit in this way, silent and moving objects that cannot be located using normal hearing, can be located within the area covered by the radar unit. It is a particular fact that the area covered by the radar unit can have a range of many meters. B. The user moves the unit, and hereby generates an audio signal that varies in frequency and sound level. The frequency will reflect the speed of the radar relative to the surroundings. The sound level will relate to the total amount of objects within the range of the radar unit, reflecting the radio waves. In particular objects that move relative to the surroundings and hence the radar unit, will each contribute with a distinct frequency and sound level (depending on the moving object reflectivity and speed relative to the radar unit). Hence the functionality achieved by using the invention by method A is not lost by using the unit by method B. This is valid regardless of the number of moving objects.

Hence the user can obtain a precise and comprehensive "sound-picture" of the surroundings, and thereby improving orientation in known as well as unknown surroundings. Likewise the unit can give an impression of the number, speed, size and distance of moving objects in the area covered by the radar unit. In particular, the user can move the unit and obtain a signature of the surroundings.

It is particularly interesting to use two or more radar units, thereby enabling the directional detection of moving objects. This way of detecting objects is much alike the way the human hearing in general detects directional information, as the signals from the units are correlated.

According to the invention the output from the mixer of the radar unit is used as an audio generator, where it is possible to deduct information about the surroundings from the audio signal. By using the unit by method A, B the visually impaired can obtain an overview of unknown surroundings; in particular, it is possible to locate both stationary and moving objects. By using method B it is possible to detect the surroundings in the direction of interest to the visually impaired.

Compared to the competitors on the market, the invention differs in two aspects : A. The output from the radar unit's mixer is only manipulated slightly, and the full information from the radar unit is kept as intact as possible, as this signal is meant to be understood by the user and be comprehended as a "sound- picture" that reflects the surroundings. The way of localising the objects is very similar to that of the ordinary human hearing, where a source of sound is located based on sound level, echo, phase difference etc.

B. To enhance the directivity two or more radar units can be applied. By using two or more audio generators (eg. a set of earphones) it is possible to achieve that the output from the radar units is comprehended as an ordinary source of sound, and hence the user will place the objects relative to the user.

Claims

C l a i m s

1. A method for localising, by means of wave signals, moving objects, characterised in that owithin a relatively narrow horizontal angle area, wave signals are emitted at a transmitter frequency; othat reflected signals are received, the frequency of which deviates from the transmitter frequency in response to the speed of a reflecting object; othat a difference signal is produced in response to said transmitted and received signals; othat the difference signal is adapted so as to correspond to the frequency and dynamics area of the range of the human hearing; and othat the adapted difference signal is introduced to a user as a sound signal, where the information relating to the movement of objects is represented by frequency variations of the sound signal

2. A method according to claim 1 , characterised in that the information is represented by frequency variations within a considerable portion of the audible frequency range.

3. A method according to claims 1-2, characterised in that the amplitude of said sound signal is controlled dependent on the strength of said difference signal.

4. A method according to claims 1-3, characterised in that additional pulses are transmitted and echoes are received to calculate the distance to an object.

5. A method according to claim 4, characterised in that the sound signal is combined with a further sound representing the calculated distance.

6. A method according to claims 1-5, characterised in that parts of said difference signals are suppressed.

7. A method according to claims 1-6, characterised in that ultrasound signals are used.

8. A method according to claims 1-6, characterised in that a Doppler radar is used.

9. A method according to claim 8, characterised in that two or more Doppler radars are used, whose main transmission directions in the horizontal plane are dissimilar.

10. A method according to claim 8 or 9, characterised in that the method is executed as an aid for the visually impaired.

11. A method according to claim 10, characterised in that the radar frequency is selected such that the difference frequency is located within the audible range when the speed of recorded objects correspond to usual traffic speeds.

12. A method according to any one of claims 1-11 , characterised in that the difference frequency is frequency-shifted, whereby the resulting difference frequency is comprised within the audible range when the speeds of the recorded objects correspond to usual traffic speeds.

13. A portable aid for the visually impaired, characterised in that it comprises oa Doppler radar for producing a difference signal on the basis of an emitted and a received, reflected signal; oa circuit configured for adapting the frequency of said signals, in such a manner that the difference signal is to be comprised within the audible human frequency range; oa sound emitter for converting the difference signal into a sound signal.

14. A portable aid according to claim 13, characterised in that it comprises two Doppler radars that are, in the in-use position, arranged for emitting signals in unlike horizontal directions.

15. A portable aid according to claim 14, characterised in that it comprises a pair of sound emitters for being arranged in connection with the user's ears; and that the difference signal for the left Doppler radar is preferably transmitted to the left sound emitter, while the difference signal from the right Doppler radar is preferably transmitted to the right sound emitter.