WO2002039429A1 - Method of processing auditory data - Google Patents
Method of processing auditory data Download PDFInfo
- Publication number
- WO2002039429A1 WO2002039429A1 PCT/US2000/031228 US0031228W WO0239429A1 WO 2002039429 A1 WO2002039429 A1 WO 2002039429A1 US 0031228 W US0031228 W US 0031228W WO 0239429 A1 WO0239429 A1 WO 0239429A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- stream
- signal
- frequency
- cochlea
- electrical signal
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/06—Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids
- G10L2021/065—Aids for the handicapped in understanding
Definitions
- the present invention is related to a method of transforming an electrical signal representative of a sound wave as a step in the electrical stimulation of a mammalian cochlea or for the purpose of effecting data compression of the electrical signal .
- the human cochlea is a complex biochemical- electrical organ of the inner ear that translates sound waves into electrochemical impulses in the auditory nerve.
- the human cochlea is a coil having a wound, sound receiving surface, known as the basilar membrane, of approximately 32 mm in length.
- Nerve fibers emanating from the various regions of the cochlea are associated with the frequencies that most efficiently stimulate those regions, and the brain, which receives neural impulses from the distributed fibers, maps those frequencies in accord with this association.
- the nerve stimulated by this traveling wave is associated, in the brain, with the frequency of the sound both due to this mapping of the locus associated with frequency and due to the timing of nerve impulses which tend to reflect the periodicities of lower frequencies.
- These time patterns of impulses carry information about single frequencies and about the relative magnitudes and phases of multiple frequency components in sounds. For this reason both the spatial mapping of frequencies and the complex timing relationships of the nerve impulses they evoke contribute to the full perception of sounds including speech.
- the relative timing of auditory events at the two ears provides crucial information to a listener. For example, the difference in the times of arrival for sound vibrations at the two ears provides the listener with information about the direction in which the sound has traveled.
- the signal processing mechanisms of cochlear implants did not stimulate the " cochlea in conformity with the timing of the arriving sound to the point where, even for those patients who " -were equipped with binaural implants, patients could determine the direction from which sound was arriving.
- this patent application addresses some problems encountered in the field of data compression of electrical signals representative of sound waves for the purposes of efficient storage, transmission, and reproduction.
- One currently popular form of data compression of sound wave signals is included in the "Motion Picture Experts Group Layer 3 Audio Coding" or more simply “MPEG Layer 3.
- MPEG Layer 3 Audio Coding
- the creation of an MPEG Layer 3 signal is not a real time process. Because of this, it is not suitable for use in telephony or other real time processes .
- the first aspect of the present invention is a method for the real-time transformation of an electrical • signal representative of a sound wave that includes the steps of providing an electrical signal representative of a sound wave passing said electrical signal, in parallel, through a number of bandpass filters to create a set of time domain real and imaginary band limited signals. Next, a stream of instantaneous phase angle and magnitude values for each of said set of time domain real and imaginary band limited signals is computed. Thirdly, a stream of electrical pulses or other digital representation of the phase and magnitude information is " computed for delivery to a cochlear implant or transmission for decoding and synthesis of the original sound.
- the present invention is a method for effecting hearing restoration by the electrical stimulation of a human cochlea, comprising providing a cochlear implant assembly, including a microphone, a signal processing assembly connected to the microphone and a set of electrodes contacting the cochlea and being operatively connected to the signal processing assembly. Also, the microphone receives sound waves and translates them into an electrical signal and the signal processing assembly detects predefined events in the electrical signal in each frequency band out of a set of frequency bands and emits a set of signals in response to each detection of a predefined event. Additionally, at least one of the set of electrodes electrically stimulates the cochlea in response to each set of signals.
- the present invention is a method for effecting hearing restoration by the electrical stimulation of the cochlea of a human, comprising providing a cochlear implant assembly, including a microphone, a signal processing assembly connected to the microphone, a set of electrodes contacting the cochlea and being operatively connected to the signal processing assembly. Also, the microphone receives sound waves and translates them into an electrical signal and the signal processing assembly iterativ ' ely chooses a frequency-magnitude pair in each frequency band out of a predefined set of frequency bands, each frequency-magnitude pair being representative of the sound in the frequency band. Additionally, the electrodes are stimulated in response to the frequency magnitude pairs .
- the present -.invention is a method for effecting hearing restoration by the electrical stimulation of a human cochlea, '.comprising providing a cochlear implant assembly, including a microphone a signal processing assembly connected to the microphone and a set of electrodes contacting the cochlea and being operatively connected to the signal processing assembly.
- the microphone and the signal processing assembly form a set of abstracted frequency-magnitude pairs based on a sound signal received by the microphone
- a plurality of the electrodes cooperatively simulate the sound of all magnitude-frequency pairs.
- the present invention is a method for the real time data compression of an auditory signal, comprising the steps of converting the auditory signal into a digital electronic signal having an initial sampling rate, in real time and forming a time sequence of abstracted parameter values, representative of the auditory signal, in real time. Additionally, the time sequence is encoded to form an encoded time sequence that includes a full representation of the abstracted parameter values less often than the initial sampling rate of the auditory signal.
- FIG. 1 is a block diagram of an assemblage of hardware that may serve as a host for the present invention.
- FIG. 2 is a block diagram of a flow of signal processing according to preferred embodiment of the present invention.
- FIG. 3 is a table showing the division of a ' portion of the auditory spectrum into frequency bands according to a preferred embodiment of the present invention.
- FIG. 4 is a pair of graphs in the signal amplitude versus frequency domain showing the spectrum of an auditory signal (top) and a set of events (bottom) consisting of an instantaneous frequency and amplitude pair that may be derived from the auditory signal.
- FIG. 5 is a graph in the real and imaginary coordinate system showing a vector comprised of the instantaneous phase and magnitude of a signal time sample from one bandpass filter.
- FIG. 6 is a graph in the real and imaginary coordinate system showing a time sequence of vectors, each comprised of the instantaneous phase and magnitude of a signal time sample.
- FIG. 7 is a pair of graphs, the top graph shows the instantaneous frequency and magnitude of a signal time sample and the bottom graph the electrical current applied to a set of cochlear implant electrodes to represent the instantaneous frequency and magnitude on the basilar membrane of the cochlea.
- FIG. 1 represents the set of physical elements that perform the signal processing that is the subject matter of the present invention, and is presented to help the reader understand the context of the present invention.
- a microphone 10 creates an analogue signal that the A/D convertor 12 changes to a digital stream.
- This digital stream is sent to a digital signal processing (DSP) chip 14 and a microprocessor 16, which together determine the amplitude and timing for each electrode to stimulate the cochlea and formats and outputs this information in a predetermined serial format.
- DSP digital signal processing
- the DSP chip may incorporate the functions of the microprocessor in its computational load.
- a modulator 18 places the information into a transcutaneous medium, such as radio frequency (RF) , and transmits it to transcutaneous demodulator 20.
- RF radio frequency
- a demultiplexer 22 divides the serial stream of information yielded by demodulator 20 into a set of pulse magnitude and timing commands for the drivers 24, each of which produces the electrical pulses to drive a particular electrode 26.
- the digital signal processing in a preferred embodiment begins with the Hubert transforming to produce an analytic signal and bandpass filtering (block 50) of this digitized analytic signal, into a set of frequency bands. This step is performed in the DSP chip 14.
- the dimensions of the resultant bands are indicated by the table of FIG.3. Other sets of bands may also be desirable and would fall within the scope of the present invention. It may be desirable to tailor the bandwidth set to the individual patient.
- FIG. 4 (bottom) provides an indication of this division, with one vertical line lying within the frequency scope of each bandpass filter.
- the product of each bandpass filter is a time domain set of complex samples comprising an analytic signal, each having a real and imaginary component.
- each real and imaginary filter time sample is mapped to a single complex vector whose successive time samples have magnitude and frequency properties.
- the vector magnitude (henceforth referred to as "instantaneous magnitude") and real/ imaginary phase (whose rate of change is “instantaneous frequency”) are computed in the microprocessor 16.
- the phase difference between neighboring time samples is then computed to determine the instantaneous frequency of each band pass filtered signal (block 52 in FIG. 2) .
- FIG. 4 shows a set of lines 66 having a height equal to the instantaneous magnitude and positioned at the instantaneous frequency points.
- a code could be constructed for including the instantaneous frequency/magnitude information for each frequency band to meet criteria specific to the end use.
- Perceptual criteria including masking and other known factors eliminate the need to transmit many samples.
- lower frequencies are sampled more than needed to accurately represent them under the Nyquist criteria, updates at significantly longer intervals relative to the original digitized signal are possible.
- the elimination of signal events degrades the available information in a continuous fashion in contrast to the generation of large amounts of distortion as samples of the original signal are eliminated.
- Various compression schemes known to those familiar with signal processing would provide a means for optimally representing the information in a serial or parallel bit stream.
- FIG. 6 shows a sequence of real/imaginary vectors numbered according to each sample ' s relative time position in a sequence of samples. A similar sequence is constructed for each of the frequency bands shown in FIG. 4.
- the cochlea is electrically stimulated at a time that is a uniform time delay from each such real axis crossing. The exact timing of the real axis crossing is determined through standard interpolation and is well within the limits of human discrimination for timing sound events in one ear or between the ears for low frequencies.
- the instantaneous frequency and magnitude at the moment of the event must, for each frequency band, be translated into a set of electrode stimulating pulses (block 54) .
- the basic goal is to create a flow of electricity through the basilar membrane that will electrically stimulate the auditory nerve endings in a close proximity to the way they would be stimulated by a sound wave having the computed instantaneous magnitude and instantaneous frequency.
- the response of the cochlea to a sound wave at a single frequency is not limited to a single point on the cochlea. Rather, the traveling wave that is created has a significant effect over about 1 mm of cochlear length.
- a feature of the present invention is the continuous mapping to cochlear loci of stimulus frequency. This enables the support of large numbers of • electrode contacts located in high density along the basilar membrane.
- a and b are the relative currents at -electrodes 1 and 2 relative to a distant reference.
- FIG. 7 shows a single frequency and its magnitude. The frequency would be located at a position near to one of the electrodes drawn below it . The nearest electrode is to the left of the frequency line. That electrode is driven along with its two neighboring electrodes. In case a, the shaded electrodes are driven with the currents described by the equation on page 7.
- the central electrode carries a+b and the flanking electrodes carry -a and -b.
- b ⁇ a To steer the current toward the right hand flanking electrode, b ⁇ a.
- the purpose is to attempt to steer electric current nearer to the nerves that encode that frequency. The technique works in some cases, but not in all.
- the shaded electrodes are driven in a quadruple configuration.
- the central electrode carries 2a and the flanking electrodes each carry -a.
- the primary stimulus may be achieved by passing current between an electrode in row 112 and its corresponding electrode in row 114. By restricting the spread of the current field this may achieve a stronger stimulation of the auditory nerves of interest .
- the signals representing the pulse magnitude to be delivered to each electrode are multiplexed into a serial signal having a predetermined format (block 56) by the microprocessor 16.
- the signal is then modulated onto a medium, such as RF at encoder 18 (block 58) and transmitted into the body.
- the signal is sent by way of a percutaneous connector, eliminating the need for elements 18 and 20.
- a subcutaneous receiver and demodulator 60 receives and demodulates the signal from transmitter 18.
- the signal is then demultiplexed by a demultiplexer 24, into a set of channels equal to the number of electrodes 26 (block 62) and used to stimulate a set of electrode drivers 24 (block 54) , which in turn stimulate the electrodes 26 that contact the cochlea.
- a demultiplexer 24 As used in this patent application the term "set" may refer to a set containing a single element only.
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2000/031228 WO2002039429A1 (en) | 2000-11-09 | 2000-11-09 | Method of processing auditory data |
AU2001214884A AU2001214884A1 (en) | 2000-11-09 | 2000-11-09 | Method of processing auditory data |
EP00977215A EP1342232A1 (en) | 2000-11-09 | 2000-11-09 | Method of processing auditory data |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2000/031228 WO2002039429A1 (en) | 2000-11-09 | 2000-11-09 | Method of processing auditory data |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2002039429A1 true WO2002039429A1 (en) | 2002-05-16 |
Family
ID=21741995
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2000/031228 WO2002039429A1 (en) | 2000-11-09 | 2000-11-09 | Method of processing auditory data |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP1342232A1 (en) |
AU (1) | AU2001214884A1 (en) |
WO (1) | WO2002039429A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104783926B (en) * | 2014-01-20 | 2016-09-28 | 中国科学院声学研究所 | A kind of quickly multiple tracks synchronizes cochlear implant equipment performance assessment system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5651071A (en) * | 1993-09-17 | 1997-07-22 | Audiologic, Inc. | Noise reduction system for binaural hearing aid |
US5737719A (en) * | 1995-12-19 | 1998-04-07 | U S West, Inc. | Method and apparatus for enhancement of telephonic speech signals |
US5792072A (en) * | 1994-06-06 | 1998-08-11 | University Of Washington | System and method for measuring acoustic reflectance |
-
2000
- 2000-11-09 WO PCT/US2000/031228 patent/WO2002039429A1/en not_active Application Discontinuation
- 2000-11-09 EP EP00977215A patent/EP1342232A1/en not_active Withdrawn
- 2000-11-09 AU AU2001214884A patent/AU2001214884A1/en not_active Abandoned
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5651071A (en) * | 1993-09-17 | 1997-07-22 | Audiologic, Inc. | Noise reduction system for binaural hearing aid |
US5792072A (en) * | 1994-06-06 | 1998-08-11 | University Of Washington | System and method for measuring acoustic reflectance |
US5737719A (en) * | 1995-12-19 | 1998-04-07 | U S West, Inc. | Method and apparatus for enhancement of telephonic speech signals |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104783926B (en) * | 2014-01-20 | 2016-09-28 | 中国科学院声学研究所 | A kind of quickly multiple tracks synchronizes cochlear implant equipment performance assessment system |
Also Published As
Publication number | Publication date |
---|---|
EP1342232A1 (en) | 2003-09-10 |
AU2001214884A1 (en) | 2002-05-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6480820B1 (en) | Method of processing auditory data | |
US8135152B2 (en) | Method and apparatus for envelope detection and enhancement of pitch cue of audio signals | |
EP0190836B1 (en) | Auditory stimulation using cw and pulsed signals | |
AU2009233699B2 (en) | Electrical stimulation of the acoustic nerve with coherent fine structure | |
US20030044034A1 (en) | Cochlear implants and apparatus/methods for improving audio signals by use of frequency-amplitude-modulation-encoding (FAME) strategies | |
US9674621B2 (en) | Auditory prosthesis using stimulation rate as a multiple of periodicity of sensed sound | |
US20170080229A1 (en) | Interaural Coherence Based Cochlear Stimulation Using Adapted Envelope Processing | |
US9808624B2 (en) | Interaural coherence based cochlear stimulation using adapted fine structure processing | |
EP3056022B1 (en) | Arrangement and method for extracting temporal features from spike-like signals | |
AU2014321433B2 (en) | Dynamic stimulation channel selection | |
CN109982744B (en) | Inter-binaural coherence based cochlear stimulation using an adapted fine structure process | |
EP3302696B1 (en) | Patient specific frequency modulation adaption | |
US20070203535A1 (en) | Cochlear implants and apparatus/methods for improving audio signals by use of frequency-amplitude-modulation-encoding (FAME) strategies | |
EP1342232A1 (en) | Method of processing auditory data | |
CN110772713B (en) | Cochlear stimulation system with improved method for determining temporal fine structure parameters | |
CN109963615B (en) | Inter-binaural coherence based cochlear stimulation using an adapted envelope process | |
EP4221817B1 (en) | Patient specific frequency mapping procedure for hearing implant electrode arrays | |
AU2002323421A1 (en) | Cochlear implants and apparatus/methods for improving audio signals by use of frequency-amplitude-modulation-encoding (FAME) strategies |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2000977215 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 2000977215 Country of ref document: EP |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
WWW | Wipo information: withdrawn in national office |
Ref document number: 2000977215 Country of ref document: EP |