WO2005029468A1 - Voice activity detector (vad) -based multiple-microphone acoustic noise suppression - Google Patents

Voice activity detector (vad) -based multiple-microphone acoustic noise suppression Download PDF

Info

Publication number
WO2005029468A1
WO2005029468A1 PCT/US2004/029234 US2004029234W WO2005029468A1 WO 2005029468 A1 WO2005029468 A1 WO 2005029468A1 US 2004029234 W US2004029234 W US 2004029234W WO 2005029468 A1 WO2005029468 A1 WO 2005029468A1
Authority
WO
WIPO (PCT)
Prior art keywords
transfer function
acoustic
signal
noise
voicing
Prior art date
Application number
PCT/US2004/029234
Other languages
French (fr)
Inventor
Gregory C. Burnett
Eric F. Breitfeller
Original Assignee
Aliphcom, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=34375865&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=WO2005029468(A1) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Aliphcom, Inc. filed Critical Aliphcom, Inc.
Publication of WO2005029468A1 publication Critical patent/WO2005029468A1/en

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0272Voice signal separating
    • G10L21/0308Voice signal separating characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0316Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
    • G10L21/0364Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00Details of transducers, loudspeakers or microphones
    • H04R1/46Special adaptations for use as contact microphones, e.g. on musical instrument, on stethoscope
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10KSOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
    • G10K2210/00Details of active noise control [ANC] covered by G10K11/178 but not provided for in any of its subgroups
    • G10K2210/30Means
    • G10K2210/301Computational
    • G10K2210/3023Estimation of noise, e.g. on error signals
    • G10K2210/30232Transfer functions, e.g. impulse response
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10KSOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
    • G10K2210/00Details of active noise control [ANC] covered by G10K11/178 but not provided for in any of its subgroups
    • G10K2210/30Means
    • G10K2210/301Computational
    • G10K2210/3028Filtering, e.g. Kalman filters or special analogue or digital filters
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10KSOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
    • G10K2210/00Details of active noise control [ANC] covered by G10K11/178 but not provided for in any of its subgroups
    • G10K2210/30Means
    • G10K2210/301Computational
    • G10K2210/3045Multiple acoustic inputs, single acoustic output
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L2021/02082Noise filtering the noise being echo, reverberation of the speech
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161Number of inputs available containing the signal or the noise to be suppressed
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02165Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02168Noise filtering characterised by the method used for estimating noise the estimation exclusively taking place during speech pauses
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78Detection of presence or absence of voice signals

Definitions

  • VAD Voice Activity Detector
  • the disclosed embodiments relate to systems and methods for detecting and processing a desired signal in the presence of acoustic noise.
  • VAD Voice Activity Detector
  • voice is generally understood to include human voiced speech, unvoiced speech, or a combination of voiced and unvoiced speech.
  • the VAD has also been used in digital cellular systems. As an example of such a use, see United States Patent Number 6,453,291 of Ashley, where a VAD configuration appropriate to the front-end of a digital cellular system is described.
  • CDMA Code Division Multiple Access
  • GSM Global System for Mobile Communication
  • GSM Global System for Mobile Communication
  • Figure 1 is a block diagram of a denoising system, under an embodiment.
  • Figure 2 is a block diagram including components of a noise removal algorithm, under the denoising system of an embodiment assuming a single noise source and direct paths to the microphones.
  • Figure 3 is a block diagram including front-end components of a noise removal algorithm of an embodiment generalized to n distinct noise sources (these noise sources may be reflections or echoes of one another).
  • Figure 4 is a block diagram including front-end components of a noise removal algorithm of an embodiment in a general case where there are n distinct noise sources and signal reflections.
  • Figure 5 is a flow diagram of a denoising method, under an embodiment.
  • Figure 6 shows results of a noise suppression algorithm of an embodiment for an American English female speaker in the presence of airport terminal noise that includes many other human speakers and public announcements.
  • Figure 7A is a block diagram of a Voice Activity Detector (VAD) system including hardware for use in receiving and processing signals relating to VAD, under an embodiment.
  • Figure 7B is a block diagram of a VAD system using hardware of a coupled noise suppression system for use in receiving VAD information, under an alternative embodiment.
  • Figure 8 is a flow diagram of a method for determining voiced and unvoiced speech using an accelerometer-based VAD, under an embodiment.
  • VAD Voice Activity Detector
  • Figure 9 shows plots including a noisy audio signal (live recording) along with a corresponding accelerometer-based VAD signal, the corresponding accelerometer output signal, and the denoised audio signal following processing by the noise suppression system using the VAD signal, under an embodiment.
  • Figure 10 shows plots including a noisy audio signal (live recording) along with a corresponding SSM-based VAD signal, the corresponding SSM output signal, and the denoised audio signal following processing by the noise suppression system using the VAD signal, under an embodiment.
  • Figure 11 shows plots including a noisy audio signal (live recording) along with a corresponding GEMS-based VAD signal, the corresponding GEMS output signal, and the denoised audio signal following processing by the noise suppression system using the VAD signal, under an embodiment.
  • signal represents any acoustic signal (such as human speech) that is desired
  • noise is any acoustic signal (which may include human speech) that is not desired.
  • An example would be a person talking on a cellular telephone with a radio in the background.
  • acoustic is generally defined as acoustic waves propagating in air. Propagation of acoustic waves in media other than air will be noted as such.
  • References to "speech” or “voice” generally refer to human speech including voiced speech, unvoiced speech, and/or a combination of voiced and unvoiced speech. Unvoiced speech or voiced speech is distinguished where necessary.
  • the term “noise suppression” generally describes any method by which noise is reduced or eliminated in an electronic signal.
  • VAD is generally defined as a vector or array signal, data, or information that in some manner represents the occurrence of speech in the digital or analog domain.
  • a common representation of VAD information is a one-bit digital signal sampled at the same rate as the corresponding acoustic signals, with a zero value representing that no speech has occurred during the corresponding time sample, and a unity value indicating that speech has occurred during the corresponding time sample. While the embodiments described herein are generally described in the digital domain, the descriptions are also valid for the analog domain.
  • Figure 1 is a block diagram of a denoising system 1000 of an embodiment that uses knowledge of when speech is occurring derived from physiological information on voicing activity.
  • the system 1000 includes microphones 10 and sensors 20 that provide signals to at least one processor 30.
  • the processor includes a denoising subsystem or algorithm 40.
  • Figure 2 is a block diagram including components of a noise removal algorithm 200 of an embodiment. A single noise source and a direct path to the microphones are assumed. An operational description of the noise removal algorithm 200 of an embodiment is provided using a single signal source 100 and a single noise source 101, but is not so limited.
  • This algorithm 200 uses two microphones: a "signal" microphone 1 ("MJC1") and a "noise” microphone 2 ("MIC 2”), but is not so limited.
  • MJC1 signal microphone 1
  • MIC 2 "noise” microphone 2” microphone 2
  • the signal microphone MIC 1 is assumed to capture mostly signal with some noise, while MIC 2 captures mostly noise with some signal.
  • the data from the signal source 100 to MIC 1 is denoted by s(n), where s(n) is a discrete sample of the analog signal from the source 100.
  • the data from the signal source 100 to MIC 2 is denoted by s 2 (n).
  • the data from the noise source 101 to MIC 2 is denoted by n(n).
  • the data from the noise source 101 to MIC 1 is denoted by n 2 (n).
  • the data from MIC 1 to noise removal element 205 is denoted by m ⁇ (n), and the data from MIC 2 to noise removal element 205 is denoted by m 2 (n).
  • the noise removal element 205 also receives a signal from a voice activity detection (VAD) element 204.
  • VAD voice activity detection
  • the VAD 204 uses physiological information to determine when a speaker is speaking.
  • the VAD can include at least one of an accelerometer, a skin surface microphone in physical contact with skin of a user, a human tissue vibration detector, a radio frequency (RF) vibration and/or motion detector/device, an electroglottograph, an ultrasound device, an acoustic microphone that is being used to detect acoustic frequency signals that correspond to the user's speech directly from the skin of the user (anywhere on the body), an airflow detector, and a laser vibration detector.
  • the transfer functions from the signal source 100 to MIC 1 and from the noise source 101 to MIC 2 are assumed to be unity.
  • MIC 2 is used to attempt to remove noise from MIC 1.
  • an (generally unspoken) assumption is that the VAD element 204 is never perfect, and thus the denoising must be performed cautiously, so as not to remove too much of the signal along with the noise.
  • the VAD 204 is assumed to be perfect such that it is equal to zero when there is no speech being produced by the user, and equal to one when speech is produced, a substantial improvement in the noise removal can be made.
  • Equation 1 This is the general case for all two microphone systems. In a practical system there is always going to be some leakage of noise into MIC 1, and some leakage of signal into MIC 2. Equation 1 has four unknowns and only two known relationships and therefore cannot be solved explicitly. However, there is another way to solve for some of the unknowns in Equation 1.
  • different inputs are being used (now only the signal is occurring whereas before only the noise was occurring).
  • H (z) the values calculated for H ⁇ (z) are held constant and vice versa.
  • H ⁇ (z) and H (z) are being calculated, the one not being calculated does not change substantially. After calculating H ⁇ (z) and H (z), they are used to remove the noise from the signal.
  • N(z) may be substituted as shown to solve for S(z) as
  • Figure 3 is a block diagram including front-end components 300 of a noise removal algorithm of an embodiment, generalized to n distinct noise sources. These distinct noise sources may be reflections or echoes of one another, but are not so limited. There are several noise sources shown, each with a transfer function, or path, to each microphone. The previously named path H has been relabeled as Ho, so that labeling noise source 2's path to MIC 1 is more convenient.
  • H is analogous to H ⁇ (z) above.
  • H depends only on the noise sources and their respective transfer functions and can be calculated any time there is no signal being transmitted.
  • the "n" subscripts on the microphone inputs denote only that noise is being detected, while an “s” subscript denotes that only signal is being received by the microphones. Examining Equation 4 while assuming an absence of noise produces
  • FIG. 4 is a block diagram including front-end components 400 of a noise removal algorithm of an embodiment in the most general case where there are n distinct noise sources and signal reflections.
  • signal reflections enter both microphones MIC 1 and MIC 2.
  • M,(z) S(z) + S(z)H 0l (z) + N I (z)H l (z) + N 2 (z)H 2 (z) + ...N n (z)H
  • M 2 (z) S(z)H 00 (z) + S(z)H 02 (z) + N l (z)G l (z) + N 2 (z)G 2 (z) + ...N n (z)G z) .
  • Equation 9 reduces to
  • FIG. 5 is a flow diagram 500 of a denoising algorithm, under an embodiment.
  • the acoustic signals are received, at block 502. Further, physiological information associated with human voicing activity is received, at block 504.
  • a first transfer function representative of the acoustic signal is calculated upon determining that voicing information is absent from the acoustic signal for at least one specified period of time, at block 506.
  • a second transfer function representative of the acoustic signal is calculated upon determining that voicing information is present in the acoustic signal for at least one specified period of time, at block 508.
  • Noise is removed from the acoustic signal using at least one combination of the first transfer function and the second transfer function, producing denoised acoustic data streams, at block 510.
  • An algorithm for noise removal, or denoising algorithm is described herein, from the simplest case of a single noise source with a direct path to multiple noise sources with reflections and echoes. The algorithm has been shown herein to be viable under any environmental conditions. The type and amount of noise are inconsequential if a good estimate has been made of H, and H 2 , and if one does not change substantially while the other is calculated. If the user environment is such that echoes are present, they can be compensated for if coming from a noise source.
  • Equation 3 where H 2 (z) is assumed small and therefore H 2 z H/(z « 0, so that Equation 3 reduces to S(z)*M,(z)-M 2 (z)H,(z).
  • the acoustic data was divided into 16 subbands, and the denoising algorithm was then applied to each subband in turn. Finally, the 16 denoised data streams were recombined to yield the denoised acoustic data. This works very well, but any combinations of subbands (i.e., 4, 6, 8, 32, equally spaced, perceptually spaced, etc.) can be used and all have been found to work better than a single subband.
  • the amplitude of the noise was constrained in an embodiment so that the microphones used did not saturate (that is, operate outside a linear response region). It is important that the microphones operate linearly to ensure the best performance.
  • the VAD of an embodiment uses a radio frequency (RF) vibration detector interferometer to detect tissue motion associated with human speech production, but is not so limited.
  • the signal from the RF device is completely acoustic-noise free, and is able to function in any acoustic noise environment.
  • a simple energy measurement of the RF signal can be used to determine if voiced speech is occurring.
  • Unvoiced speech can be determined using conventional acoustic-based methods, by proximity to voiced sections determined using the RF sensor or similar voicing sensors, or through a combination of the above. Since there is much less energy in unvoiced speech, its detection accuracy is not as critical to good noise suppression performance as is voiced speech. With voiced and unvoiced speech detected reliably, the algorithm of an embodiment can be implemented.
  • the noise removal algorithm does not depend on how the VAD is obtained, only that it is accurate, especially for voiced speech. If speech is not detected and training occurs on the speech, the subsequent denoised acoustic data can be distorted. Data was collected in four channels, one for MIC 1, one for MIC 2, and two for the radio frequency sensor that detected the tissue motions associated with voiced speech. The data were sampled simultaneously at 40 kHz, then digitally filtered and decimated down to 8 kHz. The high sampling rate was used to reduce any aliasing that might result from the analog to digital process. A four-channel National Instruments A/D board was used along with Labview to capture and store the data.
  • FIG. 6 shows a denoised audio 602 signal output upon application of the noise suppression algorithm of an embodiment to a dirty acoustic signal 604, under an embodiment.
  • the dirty acoustic signal 604 includes speech of an American English- speaking female in the presence of airport terminal noise where the noise includes many other human speakers and public announcements. The speaker is uttering the numbers "406 5562" in the midst of moderate airport terminal noise.
  • the dirty acoustic signal 604 was denoised 10 milliseconds at a time, and before denoising the 10 milliseconds of data were prefiltered from 50 to 3700 Hz. A reduction in the noise of approximately 17 dB is evident.
  • the VAD signals and methods described herein are processed independently of the noise suppression system, so that the receipt and processing of VAD information is independent from the processing associated with the noise suppression, but the embodiments are not so limited. This independence is attained physically (i.e., different hardware for use in receiving and processing signals relating to the VAD and the noise suppression), but is not so limited.
  • the VAD devices/methods described herein generally include vibration and movement. sensors, but are not so limited.
  • an accelerometer is placed on the skin for use in detecting skin surface vibrations that correlate with human speech. These recorded vibrations are then used to calculate a VAD signal for use with or by an adaptive noise suppression algorithm in suppressing environmental acoustic noise from a simultaneously (within a few milliseconds) recorded acoustic signal that includes both speech and noise.
  • Another embodiment of the VAD devices/methods described herein includes an acoustic microphone modified with a membrane so that the microphone no longer efficiently detects acoustic vibrations in air. The membrane, though, allows the microphone to detect acoustic vibrations in objects with which it is in physical contact (allowing a good mechanical impedance match), such as human skin.
  • the acoustic microphone is modified in some way such that it no longer detects acoustic vibrations in air (where it no longer has a good physical impedance match), but only in objects with which the microphone is in contact.
  • This configures the microphone, like the accelerometer, to detect vibrations of human skin associated with the speech production of that human while not efficiently detecting acoustic environmental noise in the air.
  • the detected vibrations are processed to form a VAD signal for use in a noise suppression system, as detailed below.
  • an electromagnetic vibration sensor such as a radiofrequency vibrometer (RF) or laser vibrometer, which detect skin vibrations.
  • RF radiofrequency vibrometer
  • laser vibrometer which detect skin vibrations.
  • FIG. 7A is a block diagram of a VAD system 702 A including hardware for use in receiving and processing signals relating to VAD, under an embodiment.
  • the VAD system 702A includes a VAD device 730 coupled to provide data to a corresponding VAD algorithm 740.
  • noise suppression systems of alternative embodiments can integrate some or all functions of the VAD algorithm with the noise suppression processing in any manner obvious to those skilled in the art.
  • the voicing sensors 20 include the VAD system 702A, for example, but are not so limited.
  • the VAD includes the VAD system 702A, for example, but is not so limited.
  • Figure 7B is a block diagram of a VAD system 702B using hardware of the associated noise suppression system 701 for use in receiving VAD information 764, under an embodiment.
  • the VAD system 702B includes a VAD algorithm 750 that receives data 764 from MIC 1 and MIC 2, or other components, of the corresponding signal processing system 700.
  • Alternative embodiments of the noise suppression system can integrate some or all functions of the VAD algorithm with the noise suppression processing in any manner obvious to those skilled in the art.
  • the vibration/movement-based VAD devices described herein include the physical hardware devices for use in receiving and processing signals relating to the VAD and the noise suppression. As a speaker or user produces speech, the resulting vibrations propagate through the tissue of the speaker and, therefore can be detected on and beneath the skin using various methods. These vibrations are an excellent source of VAD information, as they are strongly associated with both voiced and unvoiced speech (although the unvoiced speech vibrations are much weaker and more difficult to detect) and generally are only slightly affected by environmental acoustic noise (some devices/methods, for example the electromagnetic vibrometers described below, are not affected by environmental acoustic noise).
  • a VAD system 702A of an embodiment includes an accelerometer-based device 730 providing data of the skin vibrations to an associated algorithm 740.
  • the algorithm 740 of an embodiment uses energy calculation techniques along with a threshold comparison, as described herein, but is not so limited. Note that more complex energy-based methods are available to those skilled in the art.
  • Figure 8 is a flow diagram 800 of a method for determining voiced and unvoiced speech using an accelerometer-based VAD, under an embodiment.
  • the energy is calculated by defining a standard window size over which the calculation is to take place and summing the square of the amplitude over time as
  • operation begins upon receiving accelerometer data, at block 802.
  • the processing associated with the VAD includes filtering the data from the accelerometer to preclude aliasing, and digitizing the filtered data for processing, at block 804.
  • the digitized data is segmented into windows 20 milliseconds (msec) in length, and the data is stepped 8 msec at a time, at block 806.
  • the processing further includes filtering the windowed data, at block 808, to remove spectral information that is corrupted by noise or is otherwise unwanted.
  • the energy in each window is calculated by summing the squares of the amplitudes as described above, at block 810.
  • the calculated energy values can be normalized by dividing the energy values by the window length; however, this involves an extra calculation and is not needed as long as the window length is not varied.
  • the calculated, or normalized, energy values are compared to a threshold, at block 812.
  • the speech corresponding to the accelerometer data is designated as voiced speech when the energy of the accelerometer data is at or above a threshold value, at block 814.
  • the speech corresponding to the accelerometer data is designated as unvoiced speech when the energy of the accelerometer data is below the threshold value, at block 816.
  • Noise suppression systems of alternative embodiments can use multiple threshold values to indicate the relative strength or confidence of the voicing signal, but are not so limited. Multiple subbands may also be processed for increased accuracy.
  • Figure 9 shows plots including a noisy audio signal (live recording) 902 along with a corresponding accelerometer-based VAD signal 904, the corresponding accelerometer output signal 912, and the denoised audio signal 922 following processing by the noise suppression system using the VAD signal 904, under an embodiment.
  • the noise suppression system of this embodiment includes an accelerometer (Model 352A24) from PCB Piezotronics, but is not so limited.
  • the accelerometer data has been bandpass filtered between 500 and 2500 Hz to remove unwanted acoustic noise that can couple to the accelerometer below 500 Hz.
  • the audio signal 902 was recorded using a microphone set and standard accelerometer in a babble noise environment inside a chamber measuring six (6) feet on a side and having a ceiling height of eight (8) feet.
  • the microphone set for example, is available from Aliph, Brisbane, California.
  • the noise suppression system is implemented in real-time, with a delay of approximately 10 msec.
  • the difference in the raw audio signal 902 and the denoised audio signal 922 shows noise suppression approximately in the range of 25-30 dB with little distortion of the desired speech signal.
  • denoising using the accelerometer-based VAD information is very effective.
  • a VAD system 702A of an embodiment includes a SSM VAD device 730 providing data to an associated algorithm 740.
  • the SSM is a conventional microphone modified to prevent airborne acoustic information from coupling with the microphone's detecting elements.
  • a layer of silicone or other covering changes the impedance of the microphone and prevents airborne acoustic information from being detected to a significant degree.
  • this microphone is shielded from airborne acoustic energy but is able to detect acoustic waves traveling in media other than air as long as it maintains physical contact with the media.
  • the silicone or similar material allows the microphone to mechanically couple efficiently with the skin of the user.
  • FIG. 10 shows plots including a noisy audio signal (live recording) 1002 along with a corresponding SSM-based VAD signal 1004, the corresponding SSM output signal 1012, and the denoised audio signal 1022 following processing by the noise suppression system using the VAD signal 1004, under an embodiment.
  • the audio signal 1002 was recorded using an Aliph microphone set and standard accelerometer in a babble noise environment inside a chamber measuring six (6) feet on a side and having a ceiling height of eight (8) feet.
  • the noise suppression system is implemented in real-time, with a delay of approximately 10 msec.
  • the difference in the raw audio signal 1002 and the denoised audio signal 1022 clearly show noise suppression approximately in the range of 20-25 dB with little distortion of the desired speech signal.
  • denoising using the SSM-based VAD information is effective.
  • Electromagnetic (EM) Vibrometer VAD Devices/Methods Returning to Figure 2 and Figure 7A, a VAD system 702A of an embodiment includes an EM vibrometer VAD device 730 providing data to an associated algorithm 740.
  • the EM vibrometer devices also detect tissue vibration, but can do so at a distance and without direct contact of the tissue targeted for measurement. Further, some EM vibrometer devices can detect vibrations of internal tissue of the human body. The EM vibrometers are unaffected by acoustic noise, making them good choices for use in high noise environments.
  • the noise suppression system of an embodiment receives VAD information from EM vibrometers including, but not limited to, RF vibrometers and laser vibrometers, each of which are described in turn below.
  • the RF vibrometer operates in the radio to microwave portion of the electromagnetic spectrum, and is capable of measuring the relative motion of internal human tissue associated with speech production.
  • the internal human tissue includes tissue of the trachea, cheek, jaw, and/or nose/nasal passages, but is not so limited.
  • the RF vibrometer senses movement using low-power radio waves, and data from these devices has been shown to correspond very well with calibrated targets.
  • the VAD system of an embodiment uses signals from these devices to construct a VAD using the energy/threshold method described above with reference to the accelerometer-based VAD and Figure 8.
  • An example of an RF vibrometer is the General Electromagnetic Motion Sensor (GEMS) radiovibrometer available from Aliph, located in Brisbane, California. Other RF vibrometers are described in the Related Applications and by Gregory C.
  • GEMS General Electromagnetic Motion Sensor
  • Figure 11 shows plots including a noisy audio signal (live recording) 1102 along with a corresponding GEMS-based VAD signal 1104, the corresponding GEMS output signal 11 12, and the denoised audio signal 1122 following processing by the noise suppression system using the VAD signal 1 104, under an embodiment.
  • the GEMS- based VAD signal 1104 was received from a trachea-mounted GEMS radiovibrometer from Aliph, Brisbane, California.
  • the audio signal 1 102 was recorded using an Aliph microphone set in a babble noise environment inside a chamber measuring six (6) feet on a side and having a ceiling height of eight (8) feet.
  • the noise suppression system is implemented in real-time, with a delay of approximately 10 msec.
  • the difference in the raw audio signal 1 102 and the denoised audio signal 1 122 clearly show noise suppression approximately in the range of 20-25 dB with little distortion of the desired speech signal.
  • denoising using the GEMS-based VAD information is effective. It is clear that both the VAD signal and the denoising are effective, even though the GEMS is not detecting unvoiced speech. Unvoiced speech is normally low enough in energy that it does not significantly affect the convergence of H ⁇ (z) and therefore the quality of the denoised speech.
  • the method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein includes a method for removing noise from acoustic signals, including: receiving a plurality of acoustic signals; receiving information on the vibration of human tissue associated with human voicing activity; generating at least one first transfer function representative of the plurality of acoustic signals upon determining that voicing information is absent from the plurality of acoustic signals for at least one specified period of time; and removing noise from the plurality of acoustic signals using the first transfer function to produce at least one denoised acoustic data stream.
  • VAD voice activity detector
  • the method of an embodiment further includes: generating at least one second transfer function representative of the plurality of acoustic signals upon determining that voicing information is present in the plurality of acoustic signals for the at least one specified period of time; and removing noise from the plurality of acoustic signals using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream.
  • the plurality of acoustic signals include at least one reflection of at least one associated noise source signal and at least one reflection of at least one acoustic source signal.
  • receiving the plurality of acoustic signals includes receiving using a plurality of independently located microphones.
  • removing noise further includes generating at least one third transfer function using the at least one first transfer function and the at least one second transfer function.
  • generating the at least one first transfer function comprises recalculating the at least one first transfer function during at least one prespecified interval.
  • generating the at least one second transfer function comprises recalculating the at least one second transfer function during at least one prespecified interval.
  • generating the at least one first transfer function comprises use of at least one technique selected from a group consisting of adaptive techniques and recursive techniques.
  • information on the vibration of human tissue is provided by a mechanical sensor in contact with the skin.
  • information on the vibration of human tissue is provided via at least one sensor selected from among at least one of an accelerometer, a skin surface microphone in physical contact with skin of a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector.
  • the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
  • the method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a method for removing noise from electronic signals, including: detecting an absence of voiced information during at least one period, wherein detecting includes measuring the vibration of human tissue; receiving at least one noise source signal during the at least one period; generating at least one transfer function representative of the at least one noise source signal; receiving at least one composite signal comprising acoustic and noise signals; and removing the noise signal from the at least one composite signal using the at least one transfer function to produce at least one denoised acoustic data stream.
  • the at least one noise source signal includes at least one reflection of at least one associated noise source signal.
  • the at least one composite signal includes at least one reflection of at least one associated composite signal.
  • the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
  • detecting includes use of a mechanical sensor in contact with the human tissue.
  • detecting includes use of a sensor selected from among at least one of an accelerometer, a skin surface microphone in physical contact with a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector.
  • RF radio frequency
  • receiving includes receiving the at least one noise source signal using at least one microphone.
  • the at least one microphone includes a plurality of independently located microphones.
  • removing the noise signal from the at least one composite signal using the at least one transfer function includes generating at least one other transfer function using the at least one transfer function.
  • generating at least one transfer function comprises recalculating the at least one transfer function during at least one prespecified interval.
  • generating the at least one transfer function comprises calculating the at least one transfer function using at least one technique selected from a group consisting of adaptive techniques and recursive techniques.
  • the method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein further includes a method for removing noise from electronic signals, including: determining at least one unvoicing period during which voiced information is absent based on vibration of human tissue; receiving at least one noise signal input during the at least one unvoicing period and generating at least one unvoicing transfer function representative of the at least one noise signal; receiving at least one composite signal comprising acoustic and noise signals; and removing the noise signal from the at least one composite signal using the at least unvoicing transfer function to produce at least one denoised acoustic data stream.
  • VAD voice activity detector
  • producing at least one denoised acoustic data stream further includes: determining at least one voicing period during which voiced information is present; receiving at least one acoustic signal input from at least one signal sensing device during the at least one voicing period and generating at least one voicing transfer function representative of the at least one acoustic signal; and removing the noise signal from the at least one composite signal using at least one combination of the at least one unvoicing transfer function and the at least one voicing transfer function to produce the denoised acoustic data stream.
  • the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
  • the method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a system for removing noise from the acoustic signals, the system including: at least one receiver that receives at least one acoustic signal; at least one sensor that receives human tissue vibration information associated with human voicing activity; and at least one processor coupled among the at least one receiver and the at least one sensor that generates a plurality of transfer functions, wherein at least one first transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is absent from the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using the first transfer function to produce at least one denoised acoustic data stream.
  • VAD voice activity detector
  • At least one second transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is present in the at least one acoustic signal for the at least one specified period of time, wherein noise is removed from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce the at least one denoised acoustic data stream.
  • the sensor includes a mechanical sensor in contact with the skin.
  • the sensor includes at least one of an accelerometer, a skin surface microphone in physical contact with skin of a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector.
  • RF radio frequency
  • the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
  • the system of an embodiment further includes: dividing acoustic data of the at least one acoustic signal into a plurality of subbands; removing noise from each of the plurality of subbands using the at least one first transfer function, wherein a plurality of denoised acoustic data streams are generated; and combining the plurality of denoised acoustic data streams to generate the at least one denoised acoustic data stream.
  • the at least one receiver includes a plurality of independently located microphones.
  • the method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a system for removing noise from acoustic signals, including at least one processor coupled among at least one microphone and at least one voicing sensor, wherein the at least one voicing sensor detects human tissue vibration associated with voicing, wherein an absence of voiced information is detected during at least one period using the at least one voicing sensor, wherein at least one noise source signal is received during the at least one period using the at least one microphone, wherein the at least one processor generates at least one transfer function representative of the at least one noise source signal, wherein the at least one microphone receives at least one composite signal comprising acoustic and noise signals, and the at least one processor removes the noise signal from the at least one composite signal using the at least one transfer function to produce at least one denoised acoustic data stream.
  • VAD voice activity detector
  • the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
  • the method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a signal processing system coupled among at least one user and at least one electronic device, wherein the signal processing system includes at least one denoising subsystem for removing noise from acoustic signals, the denoising subsystem comprising at least one processor coupled among at least one receiver and at least one sensor, wherein the at least one receiver is coupled to receive at least one acoustic signal, wherein the at least one sensor detects human tissue vibration associated with human voicing activity, wherein the at least one processor generates a plurality of transfer functions, wherein at least one first transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is absent from the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using the first transfer function to produce at least one denoised acoustic data stream.
  • the signal processing system
  • At least one second transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is present in the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream.
  • at least one electronic device includes at least one of cellular telephones, personal digital assistants, portable communication devices, computers, video cameras, digital cameras, and telematics systems.
  • the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
  • the method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a computer readable medium comprising executable instructions which, when executed in a processing system, remove noise from received acoustic signals by: receiving at least one acoustic signal; receiving human tissue vibration information associated with human voicing activity; generating at least one first transfer function representative of the at least one acoustic signal upon determining that voicing information is absent from the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using the at least one first transfer function to produce at least one denoised acoustic data stream.
  • VAD voice activity detector
  • removing noise from received acoustic signals further includes: generating at least one second transfer function representative of the at least one acoustic signal upon determining that voicing information is present in the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream.
  • the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
  • the method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes an electromagnetic medium comprising executable instructions which, when executed in a processing system, remove noise from received acoustic signals by: receiving at least one acoustic signal; receiving human tissue vibration information associated with human voicing activity; generating at least one first transfer function representative of the at least one acoustic signal upon determining that voicing information is absent from the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using the at least one first transfer function to produce at least one denoised acoustic data stream.
  • VAD voice activity detector
  • removing noise from received acoustic signals further includes: generating at least one second transfer function representative of the at least one acoustic signal upon determining that voicing information is present in the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream.
  • the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the. chest.
  • aspects of the noise suppression system may be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (PLDs), such as field programmable gate arrays (FPGAs), programmable array logic (PAL) devices, electrically programmable logic and memory devices and standard cell- based devices, as well as application specific integrated circuits (ASICs).
  • PLDs programmable logic devices
  • FPGAs field programmable gate arrays
  • PAL programmable array logic
  • ASICs application specific integrated circuits
  • Some other possibilities for implementing aspects of the noise suppression system include: microcontrollers with memory (such as electronically erasable programmable read only memory (EEPROM)), embedded microprocessors, firmware, software, etc. If aspects of the noise suppression system are embodied as software at least one stage during manufacturing (e.g.
  • the software may be carried by any computer readable medium, such as magnetically- or optically-readable disks (fixed or floppy), modulated on a carrier signal or otherwise transmitted, etc.
  • aspects of the noise suppression system may be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types.
  • MOSFET metal-oxide semiconductor field-effect transistor
  • CMOS complementary metal-oxide semiconductor
  • ECL emitter-coupled logic
  • polymer technologies e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures
  • mixed analog and digital etc.
  • noise suppression system can be applied to other processing systems and communication systems, not only for the processing systems described above.
  • the elements and acts of the various embodiments described above can be combined to provide further embodiments. These and other changes can be made to the noise suppression system in light of the above detailed description. All of the above references and United States patent applications are incorporated herein by reference. Aspects of the noise suppression system can be modified, if necessary, to employ the systems, functions and concepts of the various patents and applications described above to provide yet further embodiments of the noise suppression system.
  • the terms used should not be construed to limit the noise suppression system to the specific embodiments disclosed in the specification and the claims, but should be construed to include all processing systems that operate under the claims to provide a method for compressing and decompressing data files or streams. Accordingly, the noise suppression system is not limited by the disclosure, but instead the scope of the noise suppression system is to be determined entirely by the claims. While certain aspects of the noise suppression system are presented below in certain claim forms, the inventors contemplate the various aspects of the noise suppression system in any number of claim forms. For example, while only one aspect of the noise suppression system is recited as embodied in computer-readable medium, other aspects may likewise be embodied in computer-readable medium. Accordingly, the inventors reserve the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the noise suppression system.

Abstract

Acoustic noise suppression is provided in multiple-microphone systems using Voice Activity Detectors (VAD). A host system receives acoustic signals via multiple microphones. The system also receives information on the vibration of human tissue associated with human voicing activity via the VAD. In response, the system generates a transfer function representative of the received acoustic signals upon determining that voicing information is absent from the received acoustic signals during at least one specified period of time. The system removes noise from the received acoustic signals using the transfer function, thereby producing a denoised acoustic data stream.

Description

Voice Activity Detector (VAD) -Based Multiple-Microphone Acoustic Noise Suppression
RELATED APPLICATIONS This patent application is a continuation-in-part of United States Patent Application Number 09/905,361, filed July 12, 2001 , which claims priority from United States Patent Application Number 60/219,297, filed July 19, 2000. This patent application also claims priority from United States Patent Application Number 10/383,162, filed March 5, 2003.
FIELD OF THE INVENTION The disclosed embodiments relate to systems and methods for detecting and processing a desired signal in the presence of acoustic noise.
BACKGROUND Many noise suppression algorithms and techniques have been developed over the years. Most of the noise suppression systems in use today for speech communication systems are based on a single-microphone spectral subtraction technique first develop in the 1970's and described, for example, by S. F. Boll in "Suppression of Acoustic Noise in Speech using Spectral Subtraction," IEEE Trans, on ASSP, pp. 1 13-120, 1979. These techniques have been refined over the years, but the basic principles of operation have remained the same. See, for example, United States Patent Number 5,687,243 of McLaughlin, et al., and United States Patent Number 4,811,404 of Vilmur, et al. Generally, these techniques make use of a microphone-based Voice Activity Detector (VAD) to determine the background noise characteristics, where "voice" is generally understood to include human voiced speech, unvoiced speech, or a combination of voiced and unvoiced speech. The VAD has also been used in digital cellular systems. As an example of such a use, see United States Patent Number 6,453,291 of Ashley, where a VAD configuration appropriate to the front-end of a digital cellular system is described. Further, some Code Division Multiple Access (CDMA) systems utilize a VAD to minimize the effective radio spectrum used, thereby allowing for more system capacity. Also, Global System for Mobile Communication (GSM) systems can include a VAD to reduce co-channel interference and to reduce battery consumption on the client or subscriber device. These typical microphone-based VAD systems are significantly limited in capability as a result of the addition of environmental acoustic noise to the desired speech signal received by the single microphone, wherein the analysis is performed using typical signal processing techniques. In particular, limitations in performance of these microphone-based VAD systems are noted when processing signals having a low signal- to-noise ratio (SNR), and in settings where the background noise varies quickly. Thus, similar limitations are found in noise suppression systems using these microphone-based VADs.
BRIEF DESCRIPTION OF THE FIGURES Figure 1 is a block diagram of a denoising system, under an embodiment. Figure 2 is a block diagram including components of a noise removal algorithm, under the denoising system of an embodiment assuming a single noise source and direct paths to the microphones. Figure 3 is a block diagram including front-end components of a noise removal algorithm of an embodiment generalized to n distinct noise sources (these noise sources may be reflections or echoes of one another). Figure 4 is a block diagram including front-end components of a noise removal algorithm of an embodiment in a general case where there are n distinct noise sources and signal reflections. Figure 5 is a flow diagram of a denoising method, under an embodiment. Figure 6 shows results of a noise suppression algorithm of an embodiment for an American English female speaker in the presence of airport terminal noise that includes many other human speakers and public announcements. Figure 7A is a block diagram of a Voice Activity Detector (VAD) system including hardware for use in receiving and processing signals relating to VAD, under an embodiment. Figure 7B is a block diagram of a VAD system using hardware of a coupled noise suppression system for use in receiving VAD information, under an alternative embodiment. Figure 8 is a flow diagram of a method for determining voiced and unvoiced speech using an accelerometer-based VAD, under an embodiment. Figure 9 shows plots including a noisy audio signal (live recording) along with a corresponding accelerometer-based VAD signal, the corresponding accelerometer output signal, and the denoised audio signal following processing by the noise suppression system using the VAD signal, under an embodiment. Figure 10 shows plots including a noisy audio signal (live recording) along with a corresponding SSM-based VAD signal, the corresponding SSM output signal, and the denoised audio signal following processing by the noise suppression system using the VAD signal, under an embodiment. Figure 11 shows plots including a noisy audio signal (live recording) along with a corresponding GEMS-based VAD signal, the corresponding GEMS output signal, and the denoised audio signal following processing by the noise suppression system using the VAD signal, under an embodiment.
DETAILED DESCRIPTION The following description provides specific details for a thorough understanding of, and enabling description for, embodiments of the noise suppression system. However, one skilled in the art will understand that the invention may be practiced without these details. In other instances, well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the noise suppression system. In the following description, "signal" represents any acoustic signal (such as human speech) that is desired, and "noise" is any acoustic signal (which may include human speech) that is not desired. An example would be a person talking on a cellular telephone with a radio in the background. The person's speech is desired and the acoustic energy from the radio is not desired. In addition, "user" describes a person who is using the device and whose speech is desired to be captured by the system. Also, "acoustic" is generally defined as acoustic waves propagating in air. Propagation of acoustic waves in media other than air will be noted as such. References to "speech" or "voice" generally refer to human speech including voiced speech, unvoiced speech, and/or a combination of voiced and unvoiced speech. Unvoiced speech or voiced speech is distinguished where necessary. The term "noise suppression" generally describes any method by which noise is reduced or eliminated in an electronic signal. Moreover, the term "VAD" is generally defined as a vector or array signal, data, or information that in some manner represents the occurrence of speech in the digital or analog domain. A common representation of VAD information is a one-bit digital signal sampled at the same rate as the corresponding acoustic signals, with a zero value representing that no speech has occurred during the corresponding time sample, and a unity value indicating that speech has occurred during the corresponding time sample. While the embodiments described herein are generally described in the digital domain, the descriptions are also valid for the analog domain. Figure 1 is a block diagram of a denoising system 1000 of an embodiment that uses knowledge of when speech is occurring derived from physiological information on voicing activity. The system 1000 includes microphones 10 and sensors 20 that provide signals to at least one processor 30. The processor includes a denoising subsystem or algorithm 40. Figure 2 is a block diagram including components of a noise removal algorithm 200 of an embodiment. A single noise source and a direct path to the microphones are assumed. An operational description of the noise removal algorithm 200 of an embodiment is provided using a single signal source 100 and a single noise source 101, but is not so limited. This algorithm 200 uses two microphones: a "signal" microphone 1 ("MJC1") and a "noise" microphone 2 ("MIC 2"), but is not so limited. The signal microphone MIC 1 is assumed to capture mostly signal with some noise, while MIC 2 captures mostly noise with some signal. The data from the signal source 100 to MIC 1 is denoted by s(n), where s(n) is a discrete sample of the analog signal from the source 100. The data from the signal source 100 to MIC 2 is denoted by s2(n). The data from the noise source 101 to MIC 2 is denoted by n(n). The data from the noise source 101 to MIC 1 is denoted by n2(n). Similarly, the data from MIC 1 to noise removal element 205 is denoted by mι(n), and the data from MIC 2 to noise removal element 205 is denoted by m2(n). The noise removal element 205 also receives a signal from a voice activity detection (VAD) element 204. The VAD 204 uses physiological information to determine when a speaker is speaking. In various embodiments, the VAD can include at least one of an accelerometer, a skin surface microphone in physical contact with skin of a user, a human tissue vibration detector, a radio frequency (RF) vibration and/or motion detector/device, an electroglottograph, an ultrasound device, an acoustic microphone that is being used to detect acoustic frequency signals that correspond to the user's speech directly from the skin of the user (anywhere on the body), an airflow detector, and a laser vibration detector. The transfer functions from the signal source 100 to MIC 1 and from the noise source 101 to MIC 2 are assumed to be unity. The transfer function from the signal source 100 to MIC 2 is denoted by H2(z), and the transfer function from the noise source 101 to MIC 1 is denoted by Hι(z). The assumption of unity transfer functions does not inhibit the generality of this algorithm, as the actual relations between the signal, noise, and microphones are simply ratios and the ratios are redefined in this manner for simplicity. In conventional two-microphone noise removal systems, the information from
MIC 2 is used to attempt to remove noise from MIC 1. However, an (generally unspoken) assumption is that the VAD element 204 is never perfect, and thus the denoising must be performed cautiously, so as not to remove too much of the signal along with the noise. However, if the VAD 204 is assumed to be perfect such that it is equal to zero when there is no speech being produced by the user, and equal to one when speech is produced, a substantial improvement in the noise removal can be made. In analyzing the single noise source 101 and the direct path to the microphones, with reference to Figure 2, the total acoustic information coming into MIC 1 is denoted by mι(n). The total acoustic information coming into MIC 2 is similarly labeled m2(n). In the z (digital frequency) domain, these are represented as Mι(z) and M2(z). Then,
M,(z) =S(z) + N2 (z) M2 (z) = N(z) + S2(z) with N2(z) = N(z)H,(z) S2(z) = S(z)H2(z), so that M,(z) = S(z) + N(z)H,(z) M2(z) = N(z) + S(z)H2(z) . Eq. 1 This is the general case for all two microphone systems. In a practical system there is always going to be some leakage of noise into MIC 1, and some leakage of signal into MIC 2. Equation 1 has four unknowns and only two known relationships and therefore cannot be solved explicitly. However, there is another way to solve for some of the unknowns in Equation 1.
The analysis starts with an examination of the case where the signal is not being generated, that is, where a signal from the VAD element 204 equals zero and speech is not being produced. In this case, s(n) = S(z) = 0, and Equation 1 reduces to
M z) = N(z)H,(z) M2n(z) = N(z) ,
where the n subscript on the M variables indicate that only noise is being received. This leads to
M z) = MJz)H,(z) MJn(z)
The function Hι(z) can be calculated using any of the available system identification algorithms and the microphone outputs when the system is certain that only noise is being received. The calculation can be done adaptively, so that the system can react to changes in the noise. A solution is now available for one of the unknowns in Equation 1. Another unknown, H (z), can be determined by using the instances where the VAD equals one and speech is being produced. When this is occurring, but the recent (perhaps less than 1 second) history of the microphones indicate low levels of noise, it can be assumed that n(s) = N(z) ~ 0. Then Equation 1 reduces to
M z) = S(z) M z) = S(z)H2(z) , which in turn leads to
M2s(z) = M z)H2(z) H2() = ^ , M z) which is the inverse of the Hι(z) calculation. However, it is noted that different inputs are being used (now only the signal is occurring whereas before only the noise was occurring). While calculating H (z), the values calculated for Hι(z) are held constant and vice versa. Thus, it is assumed that while one of Hι(z) and H (z) are being calculated, the one not being calculated does not change substantially. After calculating Hι(z) and H (z), they are used to remove the noise from the signal. If Equation 1 is rewritten as S(z) = M,(z)-N(z)H,(z) N(z) = M2(z)-S(z)H2(z) S(z)=M,(z)-[M2(z)-S(z)H2(z)]H,(z)' S(z)[l-H2(z)H1(z)] = M,(z)-M2(z)H,(z) ,
then N(z) may be substituted as shown to solve for S(z) as
Ml (z)-M2(z)Hl (z) _ l-H2(z)H,(z) If the transfer functions Hι(z) and H2(z) can be described with sufficient accuracy, then the noise can be completely removed and the original signal recovered. This remains true without respect to the amplitude or spectral characteristics of the noise. The only assumptions made include use of a perfect VAD, sufficiently accurate Hι(z) and H2(z), and that when one of Hι(z) and H2(z) are being calculated the other does not change substantially. In practice these assumptions have proven reasonable. The noise removal algorithm described herein is easily generalized to include any number of noise sources. Figure 3 is a block diagram including front-end components 300 of a noise removal algorithm of an embodiment, generalized to n distinct noise sources. These distinct noise sources may be reflections or echoes of one another, but are not so limited. There are several noise sources shown, each with a transfer function, or path, to each microphone. The previously named path H has been relabeled as Ho, so that labeling noise source 2's path to MIC 1 is more convenient. The outputs of each microphone, when transformed to the z domain, are: M, (z) = S(z) + N, (z)H, (z) + N2 (z)H2 (z) + ...Nn (z)H n (z) M2(z) = S(z)H0(z) + Nl(z)G,(z) + N2(z)G2(z) + ...Nn(z)Gn(z). Eq.4
When there is no signal (VAD = 0), then (suppressing z for clarity) Mln=N,Hl+N2H2+...NnHn M2n=NlGl+N2G2+...NnGn. Eq.5
A new transfer function can now be defined as
H _Mln_NlH, + N3H2+...NnH„ g ' M2n N,G1+N2G2-r...N„Gn ' where H is analogous to Hι(z) above. Thus H depends only on the noise sources and their respective transfer functions and can be calculated any time there is no signal being transmitted. Once again, the "n" subscripts on the microphone inputs denote only that noise is being detected, while an "s" subscript denotes that only signal is being received by the microphones. Examining Equation 4 while assuming an absence of noise produces
M =S M2S=SH0.
Thus, Ho can be solved for as before, using any available transfer function calculating algorithm. Mathematically, then,
Figure imgf000009_0001
Rewriting Equation 4, using H defined in Equation 6, provides,
/?,- M'~S . E,.7 ' M2-SH0
Solving for S yields,
S."--"-?'. Eq.8 1-H.H, which is the same as Equation 3, with Ho taking the place of H2, and H taking the place of Hi. Thus the noise removal algorithm still is mathematically valid for any number of noise sources, including multiple echoes of noise sources. Again, if Ho and H can be estimated to a high enough accuracy, and the above assumption of only one path from the signal to the microphones holds, the noise may be removed completely. The most general case involves multiple noise sources and multiple signal sources. Figure 4 is a block diagram including front-end components 400 of a noise removal algorithm of an embodiment in the most general case where there are n distinct noise sources and signal reflections. Here, signal reflections enter both microphones MIC 1 and MIC 2. This is the most general case, as reflections of the noise source into the microphones MIC 1 and MIC 2 can be modeled accurately as simple additional noise sources. For clarity, the direct path from the signal to MIC 2 is changed from Ho(z) to Hoo(z), and the reflected paths to MIC 1 and MIC 2 are denoted by H0ι(z) and Ho2(z), respectively. The input into the microphones now becomes
M,(z) = S(z) + S(z)H0l(z) + NI(z)Hl(z) + N2(z)H2(z) + ...Nn(z)H„(z) M2(z) = S(z)H00(z) + S(z)H02(z) + Nl(z)Gl(z) + N2(z)G2(z) + ...Nn(z)G z) . Eq. 9
When the VAD = 0, the inputs become (suppressing z again) Mln =NlHl +N2H2 +...N„Hn M2„ =NiG, +N2G2 +...NnGn ,
which is the same as Equation 5. Thus, the calculation of H, in Equation 6 is unchanged, as expected. In examining the situation where there is no noise, Equation 9 reduces to
M =S + SH0l M2i =SH00 +SH02 . This leads to the definition of H2 as
H M = +H» , Eq. 10 Mh 1 + H0I Rewriting Equation 9 again using the definition for H, (as in Equation 7) provides s,= M "! -.-S*(H'„*+"H«„;) . Eq. „ Some algebraic manipulation yields S(1 + H0I ~H,(H00 +H02)) = MI -M2HI
Figure imgf000011_0001
S(l + H0l)[l-HlH3 M, -M2Hl t and finally
S(l + Hυ!) = M' -^LH' . Eq. 12 1-H,H2 Equation 12 is the same as equation 8, with the replacement of Ηo by Η2, and the addition of the (1 + H0ι) factor on the left side. This extra factor (1 + Hoi) means that S cannot be solved for directly in this situation, but a solution can be generated for the signal plus the addition of all of its echoes. This is not such a bad situation, as there are many conventional methods for dealing with echo suppression, and even if the echoes are not suppressed, it is unlikely that they will affect the comprehensibility of the speech to any meaningful extent. The more complex calculation of H2 is needed to account for the signal echoes in MIC 2, which act as noise sources. Figure 5 is a flow diagram 500 of a denoising algorithm, under an embodiment. In operation, the acoustic signals are received, at block 502. Further, physiological information associated with human voicing activity is received, at block 504. A first transfer function representative of the acoustic signal is calculated upon determining that voicing information is absent from the acoustic signal for at least one specified period of time, at block 506. A second transfer function representative of the acoustic signal is calculated upon determining that voicing information is present in the acoustic signal for at least one specified period of time, at block 508. Noise is removed from the acoustic signal using at least one combination of the first transfer function and the second transfer function, producing denoised acoustic data streams, at block 510. An algorithm for noise removal, or denoising algorithm, is described herein, from the simplest case of a single noise source with a direct path to multiple noise sources with reflections and echoes. The algorithm has been shown herein to be viable under any environmental conditions. The type and amount of noise are inconsequential if a good estimate has been made of H, and H2 , and if one does not change substantially while the other is calculated. If the user environment is such that echoes are present, they can be compensated for if coming from a noise source. If signal echoes are also present, they will affect the cleaned signal, but the effect should be negligible in most environments. In operation, the algorithm of an embodiment has shown excellent results in dealing with a variety of noise types, amplitudes, and orientations. However, there are always approximations and adjustments that have to be made when moving from mathematical concepts to engineering applications. One assumption is made in Equation 3, where H2(z) is assumed small and therefore H2 z H/(z « 0, so that Equation 3 reduces to S(z)*M,(z)-M2(z)H,(z).
This means that only Ηι(z) has to be calculated, speeding up the process and reducing the number of computations required considerably. With the proper selection of microphones, this approximation is easily realized. Another approximation involves the filter used in an embodiment. The actual Hι(z) will undoubtedly have both poles and zeros, but for stability and simplicity an all- zero Finite Impulse Response (FIR) filter is used. With enough taps the approximation to the actual Hj(z) can be very good. To further increase the performance of the noise suppression system, the spectrum of interest (generally about 125 to 3700 Hz) is divided into subbands. The wider the range of frequencies over which a transfer function must be calculated, the more difficult it is to calculate it accurately. Therefore the acoustic data was divided into 16 subbands, and the denoising algorithm was then applied to each subband in turn. Finally, the 16 denoised data streams were recombined to yield the denoised acoustic data. This works very well, but any combinations of subbands (i.e., 4, 6, 8, 32, equally spaced, perceptually spaced, etc.) can be used and all have been found to work better than a single subband. The amplitude of the noise was constrained in an embodiment so that the microphones used did not saturate (that is, operate outside a linear response region). It is important that the microphones operate linearly to ensure the best performance. Even with this restriction, very low signal-to-noise ratio (SNR) signals can be denoised (down to -10 dB or less). The calculation of Hι(z) is accomplished every 10 milliseconds using the Least- Mean Squares (LMS) method, a common adaptive transfer function. An explanation may be found in "Adaptive Signal Processing" (1985), by Widrow and Steams, published by Prentice-Hall, ISBN 0-13-004029-0. The LMS was used for demonstration purposes, but many other system identification techniques can be used to identify Hι(z) and H2(z) in Figure 2. The VAD for an embodiment is derived from a radio frequency sensor and the two microphones, yielding very high accuracy (>99%) for both voiced and unvoiced speech. The VAD of an embodiment uses a radio frequency (RF) vibration detector interferometer to detect tissue motion associated with human speech production, but is not so limited. The signal from the RF device is completely acoustic-noise free, and is able to function in any acoustic noise environment. A simple energy measurement of the RF signal can be used to determine if voiced speech is occurring. Unvoiced speech can be determined using conventional acoustic-based methods, by proximity to voiced sections determined using the RF sensor or similar voicing sensors, or through a combination of the above. Since there is much less energy in unvoiced speech, its detection accuracy is not as critical to good noise suppression performance as is voiced speech. With voiced and unvoiced speech detected reliably, the algorithm of an embodiment can be implemented. Once again, it is useful to repeat that the noise removal algorithm does not depend on how the VAD is obtained, only that it is accurate, especially for voiced speech. If speech is not detected and training occurs on the speech, the subsequent denoised acoustic data can be distorted. Data was collected in four channels, one for MIC 1, one for MIC 2, and two for the radio frequency sensor that detected the tissue motions associated with voiced speech. The data were sampled simultaneously at 40 kHz, then digitally filtered and decimated down to 8 kHz. The high sampling rate was used to reduce any aliasing that might result from the analog to digital process. A four-channel National Instruments A/D board was used along with Labview to capture and store the data. The data was then read into a C program and denoised 10 milliseconds at a time. Figure 6 shows a denoised audio 602 signal output upon application of the noise suppression algorithm of an embodiment to a dirty acoustic signal 604, under an embodiment. The dirty acoustic signal 604 includes speech of an American English- speaking female in the presence of airport terminal noise where the noise includes many other human speakers and public announcements. The speaker is uttering the numbers "406 5562" in the midst of moderate airport terminal noise. The dirty acoustic signal 604 was denoised 10 milliseconds at a time, and before denoising the 10 milliseconds of data were prefiltered from 50 to 3700 Hz. A reduction in the noise of approximately 17 dB is evident. No post filtering was done on this sample; thus, all of the noise reduction realized is due to the algorithm of an embodiment. It is clear that the algorithm adjusts to the noise instantly, and is capable of removing the very difficult noise of other human speakers. Many different types of noise have all been tested with similar results, including street noise, helicopters, music, and sine waves. Also, the orientation of the noise can be varied substantially without significantly changing the noise suppression performance. Finally, the distortion of the cleaned speech is very low, ensuring good performance for speech recognition engines and human receivers alike. The noise removal algorithm of an embodiment has been shown to be viable under any environmental conditions. The type and amount of noise are inconsequential if a good estimate has been made of H , and H2. If the user environment is such that echoes are present, they can be compensated for if coming from a noise source. If signal echoes are also present, they will affect the cleaned signal, but the effect should be negligible in most environments. When using the VAD devices and methods described herein with a noise suppression system, the VAD signal is processed independently of the noise suppression system, so that the receipt and processing of VAD information is independent from the processing associated with the noise suppression, but the embodiments are not so limited. This independence is attained physically (i.e., different hardware for use in receiving and processing signals relating to the VAD and the noise suppression), but is not so limited. The VAD devices/methods described herein generally include vibration and movement. sensors, but are not so limited. In one embodiment, an accelerometer is placed on the skin for use in detecting skin surface vibrations that correlate with human speech. These recorded vibrations are then used to calculate a VAD signal for use with or by an adaptive noise suppression algorithm in suppressing environmental acoustic noise from a simultaneously (within a few milliseconds) recorded acoustic signal that includes both speech and noise. Another embodiment of the VAD devices/methods described herein includes an acoustic microphone modified with a membrane so that the microphone no longer efficiently detects acoustic vibrations in air. The membrane, though, allows the microphone to detect acoustic vibrations in objects with which it is in physical contact (allowing a good mechanical impedance match), such as human skin. That is, the acoustic microphone is modified in some way such that it no longer detects acoustic vibrations in air (where it no longer has a good physical impedance match), but only in objects with which the microphone is in contact. This configures the microphone, like the accelerometer, to detect vibrations of human skin associated with the speech production of that human while not efficiently detecting acoustic environmental noise in the air. The detected vibrations are processed to form a VAD signal for use in a noise suppression system, as detailed below. Yet another embodiment of the VAD described herein uses an electromagnetic vibration sensor, such as a radiofrequency vibrometer (RF) or laser vibrometer, which detect skin vibrations. Further, the RF vibrometer detects the movement of tissue within the body, such as the inner surface of the cheek or the tracheal wall. Both the exterior skin and internal tissue vibrations associated with speech production can be used to form a VAD signal for use in a noise suppression system as detailed below. Figure 7A is a block diagram of a VAD system 702 A including hardware for use in receiving and processing signals relating to VAD, under an embodiment. The VAD system 702A includes a VAD device 730 coupled to provide data to a corresponding VAD algorithm 740. Note that noise suppression systems of alternative embodiments can integrate some or all functions of the VAD algorithm with the noise suppression processing in any manner obvious to those skilled in the art. Referring to Figure 1, the voicing sensors 20 include the VAD system 702A, for example, but are not so limited. Referring to Figure 2, the VAD includes the VAD system 702A, for example, but is not so limited. Figure 7B is a block diagram of a VAD system 702B using hardware of the associated noise suppression system 701 for use in receiving VAD information 764, under an embodiment. The VAD system 702B includes a VAD algorithm 750 that receives data 764 from MIC 1 and MIC 2, or other components, of the corresponding signal processing system 700. Alternative embodiments of the noise suppression system can integrate some or all functions of the VAD algorithm with the noise suppression processing in any manner obvious to those skilled in the art. The vibration/movement-based VAD devices described herein include the physical hardware devices for use in receiving and processing signals relating to the VAD and the noise suppression. As a speaker or user produces speech, the resulting vibrations propagate through the tissue of the speaker and, therefore can be detected on and beneath the skin using various methods. These vibrations are an excellent source of VAD information, as they are strongly associated with both voiced and unvoiced speech (although the unvoiced speech vibrations are much weaker and more difficult to detect) and generally are only slightly affected by environmental acoustic noise (some devices/methods, for example the electromagnetic vibrometers described below, are not affected by environmental acoustic noise). These tissue vibrations or movements are detected using a number of VAD devices including, for example, accelerometer-based devices, skin surface microphone (SSM) devices, and electromagnetic (EM) vibrometer devices including both radio frequency (RF) vibrometers and laser vibrometers. Accelerometer-based VAD Devices/Methods Accelerometers can detect skin vibrations associated with speech. As such, and with reference to Figure 2 and Figure 7A, a VAD system 702A of an embodiment includes an accelerometer-based device 730 providing data of the skin vibrations to an associated algorithm 740. The algorithm 740 of an embodiment uses energy calculation techniques along with a threshold comparison, as described herein, but is not so limited. Note that more complex energy-based methods are available to those skilled in the art. Figure 8 is a flow diagram 800 of a method for determining voiced and unvoiced speech using an accelerometer-based VAD, under an embodiment. Generally, the energy is calculated by defining a standard window size over which the calculation is to take place and summing the square of the amplitude over time as
Energy = ∑ x , where i is the digital sample subscript and ranges from the beginning of the window to the end of the window. Referring to Figure 8, operation begins upon receiving accelerometer data, at block 802. The processing associated with the VAD includes filtering the data from the accelerometer to preclude aliasing, and digitizing the filtered data for processing, at block 804. The digitized data is segmented into windows 20 milliseconds (msec) in length, and the data is stepped 8 msec at a time, at block 806. The processing further includes filtering the windowed data, at block 808, to remove spectral information that is corrupted by noise or is otherwise unwanted. The energy in each window is calculated by summing the squares of the amplitudes as described above, at block 810. The calculated energy values can be normalized by dividing the energy values by the window length; however, this involves an extra calculation and is not needed as long as the window length is not varied. The calculated, or normalized, energy values are compared to a threshold, at block 812. The speech corresponding to the accelerometer data is designated as voiced speech when the energy of the accelerometer data is at or above a threshold value, at block 814. Likewise, the speech corresponding to the accelerometer data is designated as unvoiced speech when the energy of the accelerometer data is below the threshold value, at block 816. Noise suppression systems of alternative embodiments can use multiple threshold values to indicate the relative strength or confidence of the voicing signal, but are not so limited. Multiple subbands may also be processed for increased accuracy. Figure 9 shows plots including a noisy audio signal (live recording) 902 along with a corresponding accelerometer-based VAD signal 904, the corresponding accelerometer output signal 912, and the denoised audio signal 922 following processing by the noise suppression system using the VAD signal 904, under an embodiment. The noise suppression system of this embodiment includes an accelerometer (Model 352A24) from PCB Piezotronics, but is not so limited. In this example, the accelerometer data has been bandpass filtered between 500 and 2500 Hz to remove unwanted acoustic noise that can couple to the accelerometer below 500 Hz. The audio signal 902 was recorded using a microphone set and standard accelerometer in a babble noise environment inside a chamber measuring six (6) feet on a side and having a ceiling height of eight (8) feet. The microphone set, for example, is available from Aliph, Brisbane, California. The noise suppression system is implemented in real-time, with a delay of approximately 10 msec. The difference in the raw audio signal 902 and the denoised audio signal 922 shows noise suppression approximately in the range of 25-30 dB with little distortion of the desired speech signal. Thus, denoising using the accelerometer-based VAD information is very effective.
Skin Surface Microphone (SSM) VAD Devices/Methods Referring again to Figure 2 and Figure 7A, a VAD system 702A of an embodiment includes a SSM VAD device 730 providing data to an associated algorithm 740. The SSM is a conventional microphone modified to prevent airborne acoustic information from coupling with the microphone's detecting elements. A layer of silicone or other covering changes the impedance of the microphone and prevents airborne acoustic information from being detected to a significant degree. Thus this microphone is shielded from airborne acoustic energy but is able to detect acoustic waves traveling in media other than air as long as it maintains physical contact with the media. The silicone or similar material allows the microphone to mechanically couple efficiently with the skin of the user. During speech, when the SSM is placed on the cheek or neck, vibrations associated with speech production are easily detected. However, airborne acoustic data is not significantly detected by the SSM. The tissue-borne acoustic signal, upon detection by the SSM, is used to generate the VAD signal in processing and denoising the signal of interest, as described above with reference to the energy/threshold method used with accelerometer-based VAD signal and Figure 8. Figure 10 shows plots including a noisy audio signal (live recording) 1002 along with a corresponding SSM-based VAD signal 1004, the corresponding SSM output signal 1012, and the denoised audio signal 1022 following processing by the noise suppression system using the VAD signal 1004, under an embodiment. The audio signal 1002 was recorded using an Aliph microphone set and standard accelerometer in a babble noise environment inside a chamber measuring six (6) feet on a side and having a ceiling height of eight (8) feet. The noise suppression system is implemented in real-time, with a delay of approximately 10 msec. The difference in the raw audio signal 1002 and the denoised audio signal 1022 clearly show noise suppression approximately in the range of 20-25 dB with little distortion of the desired speech signal. Thus, denoising using the SSM-based VAD information is effective. Electromagnetic (EM) Vibrometer VAD Devices/Methods Returning to Figure 2 and Figure 7A, a VAD system 702A of an embodiment includes an EM vibrometer VAD device 730 providing data to an associated algorithm 740. The EM vibrometer devices also detect tissue vibration, but can do so at a distance and without direct contact of the tissue targeted for measurement. Further, some EM vibrometer devices can detect vibrations of internal tissue of the human body. The EM vibrometers are unaffected by acoustic noise, making them good choices for use in high noise environments. The noise suppression system of an embodiment receives VAD information from EM vibrometers including, but not limited to, RF vibrometers and laser vibrometers, each of which are described in turn below. The RF vibrometer operates in the radio to microwave portion of the electromagnetic spectrum, and is capable of measuring the relative motion of internal human tissue associated with speech production. The internal human tissue includes tissue of the trachea, cheek, jaw, and/or nose/nasal passages, but is not so limited. The RF vibrometer senses movement using low-power radio waves, and data from these devices has been shown to correspond very well with calibrated targets. As a result of the absence of acoustic noise in the RF vibrometer signal, the VAD system of an embodiment uses signals from these devices to construct a VAD using the energy/threshold method described above with reference to the accelerometer-based VAD and Figure 8. An example of an RF vibrometer is the General Electromagnetic Motion Sensor (GEMS) radiovibrometer available from Aliph, located in Brisbane, California. Other RF vibrometers are described in the Related Applications and by Gregory C. Burnett in "The Physiological Basis of Glottal Electromagnetic Micropower Sensors (GEMS) and Their Use in Defining an Excitation Function for the Human Vocal Tract", Ph.D. Thesis, University of California Davis, January 1999. Laser vibrometers operate at or near the visible frequencies of light, and are therefore restricted to surface vibration detection only, similar to the accelerometer and the SSM described above. Like the RF vibrometer, there is no acoustic noise associated with the signal of the laser vibrometers. Therefore, the VAD system of an embodiment uses signals from these devices to construct a VAD using the energy/threshold method described above with reference to the accelerometer-based VAD and Figure 8. Figure 11 shows plots including a noisy audio signal (live recording) 1102 along with a corresponding GEMS-based VAD signal 1104, the corresponding GEMS output signal 11 12, and the denoised audio signal 1122 following processing by the noise suppression system using the VAD signal 1 104, under an embodiment. The GEMS- based VAD signal 1104 was received from a trachea-mounted GEMS radiovibrometer from Aliph, Brisbane, California. The audio signal 1 102 was recorded using an Aliph microphone set in a babble noise environment inside a chamber measuring six (6) feet on a side and having a ceiling height of eight (8) feet. The noise suppression system is implemented in real-time, with a delay of approximately 10 msec. The difference in the raw audio signal 1 102 and the denoised audio signal 1 122 clearly show noise suppression approximately in the range of 20-25 dB with little distortion of the desired speech signal. Thus, denoising using the GEMS-based VAD information is effective. It is clear that both the VAD signal and the denoising are effective, even though the GEMS is not detecting unvoiced speech. Unvoiced speech is normally low enough in energy that it does not significantly affect the convergence of Hι(z) and therefore the quality of the denoised speech. The method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein includes a method for removing noise from acoustic signals, including: receiving a plurality of acoustic signals; receiving information on the vibration of human tissue associated with human voicing activity; generating at least one first transfer function representative of the plurality of acoustic signals upon determining that voicing information is absent from the plurality of acoustic signals for at least one specified period of time; and removing noise from the plurality of acoustic signals using the first transfer function to produce at least one denoised acoustic data stream. The method of an embodiment further includes: generating at least one second transfer function representative of the plurality of acoustic signals upon determining that voicing information is present in the plurality of acoustic signals for the at least one specified period of time; and removing noise from the plurality of acoustic signals using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream. In the method of an embodiment, the plurality of acoustic signals include at least one reflection of at least one associated noise source signal and at least one reflection of at least one acoustic source signal. In the method of an embodiment, receiving the plurality of acoustic signals includes receiving using a plurality of independently located microphones. In the method of an embodiment, removing noise further includes generating at least one third transfer function using the at least one first transfer function and the at least one second transfer function. In the method of an embodiment, generating the at least one first transfer function comprises recalculating the at least one first transfer function during at least one prespecified interval. In the method of an embodiment, generating the at least one second transfer function comprises recalculating the at least one second transfer function during at least one prespecified interval. In the method of an embodiment, generating the at least one first transfer function comprises use of at least one technique selected from a group consisting of adaptive techniques and recursive techniques. In the method of an embodiment, information on the vibration of human tissue is provided by a mechanical sensor in contact with the skin. In the method of an embodiment, information on the vibration of human tissue is provided via at least one sensor selected from among at least one of an accelerometer, a skin surface microphone in physical contact with skin of a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector. In the method of an embodiment, the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest. The method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a method for removing noise from electronic signals, including: detecting an absence of voiced information during at least one period, wherein detecting includes measuring the vibration of human tissue; receiving at least one noise source signal during the at least one period; generating at least one transfer function representative of the at least one noise source signal; receiving at least one composite signal comprising acoustic and noise signals; and removing the noise signal from the at least one composite signal using the at least one transfer function to produce at least one denoised acoustic data stream. In the method of an embodiment, the at least one noise source signal includes at least one reflection of at least one associated noise source signal. In the method of an embodiment, the at least one composite signal includes at least one reflection of at least one associated composite signal. In the method of an embodiment, the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest. In the method of an embodiment, detecting includes use of a mechanical sensor in contact with the human tissue. In the method of an embodiment, detecting includes use of a sensor selected from among at least one of an accelerometer, a skin surface microphone in physical contact with a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector. In the method of an embodiment, receiving includes receiving the at least one noise source signal using at least one microphone. In the method of an embodiment, the at least one microphone includes a plurality of independently located microphones. In the method of an embodiment, removing the noise signal from the at least one composite signal using the at least one transfer function includes generating at least one other transfer function using the at least one transfer function. In the method of an embodiment, generating at least one transfer function comprises recalculating the at least one transfer function during at least one prespecified interval. In the method of an embodiment, generating the at least one transfer function comprises calculating the at least one transfer function using at least one technique selected from a group consisting of adaptive techniques and recursive techniques. The method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein further includes a method for removing noise from electronic signals, including: determining at least one unvoicing period during which voiced information is absent based on vibration of human tissue; receiving at least one noise signal input during the at least one unvoicing period and generating at least one unvoicing transfer function representative of the at least one noise signal; receiving at least one composite signal comprising acoustic and noise signals; and removing the noise signal from the at least one composite signal using the at least unvoicing transfer function to produce at least one denoised acoustic data stream. In the method of an embodiment, producing at least one denoised acoustic data stream further includes: determining at least one voicing period during which voiced information is present; receiving at least one acoustic signal input from at least one signal sensing device during the at least one voicing period and generating at least one voicing transfer function representative of the at least one acoustic signal; and removing the noise signal from the at least one composite signal using at least one combination of the at least one unvoicing transfer function and the at least one voicing transfer function to produce the denoised acoustic data stream. In the method of an embodiment, the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest. The method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a system for removing noise from the acoustic signals, the system including: at least one receiver that receives at least one acoustic signal; at least one sensor that receives human tissue vibration information associated with human voicing activity; and at least one processor coupled among the at least one receiver and the at least one sensor that generates a plurality of transfer functions, wherein at least one first transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is absent from the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using the first transfer function to produce at least one denoised acoustic data stream. In the system of an embodiment, at least one second transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is present in the at least one acoustic signal for the at least one specified period of time, wherein noise is removed from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce the at least one denoised acoustic data stream. In the system of an embodiment, the sensor includes a mechanical sensor in contact with the skin. In the system of an embodiment, the sensor includes at least one of an accelerometer, a skin surface microphone in physical contact with skin of a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector. In the system of an embodiment, the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest. The system of an embodiment further includes: dividing acoustic data of the at least one acoustic signal into a plurality of subbands; removing noise from each of the plurality of subbands using the at least one first transfer function, wherein a plurality of denoised acoustic data streams are generated; and combining the plurality of denoised acoustic data streams to generate the at least one denoised acoustic data stream. In the system of an embodiment, the at least one receiver includes a plurality of independently located microphones. The method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a system for removing noise from acoustic signals, including at least one processor coupled among at least one microphone and at least one voicing sensor, wherein the at least one voicing sensor detects human tissue vibration associated with voicing, wherein an absence of voiced information is detected during at least one period using the at least one voicing sensor, wherein at least one noise source signal is received during the at least one period using the at least one microphone, wherein the at least one processor generates at least one transfer function representative of the at least one noise source signal, wherein the at least one microphone receives at least one composite signal comprising acoustic and noise signals, and the at least one processor removes the noise signal from the at least one composite signal using the at least one transfer function to produce at least one denoised acoustic data stream. In a system of one embodiment, the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest. The method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a signal processing system coupled among at least one user and at least one electronic device, wherein the signal processing system includes at least one denoising subsystem for removing noise from acoustic signals, the denoising subsystem comprising at least one processor coupled among at least one receiver and at least one sensor, wherein the at least one receiver is coupled to receive at least one acoustic signal, wherein the at least one sensor detects human tissue vibration associated with human voicing activity, wherein the at least one processor generates a plurality of transfer functions, wherein at least one first transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is absent from the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using the first transfer function to produce at least one denoised acoustic data stream. In a system of one embodiment, at least one second transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is present in the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream. In a system of one embodiment, at least one electronic device includes at least one of cellular telephones, personal digital assistants, portable communication devices, computers, video cameras, digital cameras, and telematics systems. In a system of one embodiment, the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest. The method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes a computer readable medium comprising executable instructions which, when executed in a processing system, remove noise from received acoustic signals by: receiving at least one acoustic signal; receiving human tissue vibration information associated with human voicing activity; generating at least one first transfer function representative of the at least one acoustic signal upon determining that voicing information is absent from the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using the at least one first transfer function to produce at least one denoised acoustic data stream. In the medium of an embodiment, removing noise from received acoustic signals further includes: generating at least one second transfer function representative of the at least one acoustic signal upon determining that voicing information is present in the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream. In the medium of an embodiment, the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest. The method and apparatus for voice activity detector (VAD) -based multiple- microphone acoustic noise suppression as described herein also includes an electromagnetic medium comprising executable instructions which, when executed in a processing system, remove noise from received acoustic signals by: receiving at least one acoustic signal; receiving human tissue vibration information associated with human voicing activity; generating at least one first transfer function representative of the at least one acoustic signal upon determining that voicing information is absent from the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using the at least one first transfer function to produce at least one denoised acoustic data stream. In the medium of an embodiment, removing noise from received acoustic signals further includes: generating at least one second transfer function representative of the at least one acoustic signal upon determining that voicing information is present in the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream. In the medium of an embodiment, the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the. chest. Aspects of the noise suppression system may be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (PLDs), such as field programmable gate arrays (FPGAs), programmable array logic (PAL) devices, electrically programmable logic and memory devices and standard cell- based devices, as well as application specific integrated circuits (ASICs). Some other possibilities for implementing aspects of the noise suppression system include: microcontrollers with memory (such as electronically erasable programmable read only memory (EEPROM)), embedded microprocessors, firmware, software, etc. If aspects of the noise suppression system are embodied as software at least one stage during manufacturing (e.g. before being embedded in firmware or in a PLD), the software may be carried by any computer readable medium, such as magnetically- or optically-readable disks (fixed or floppy), modulated on a carrier signal or otherwise transmitted, etc. Furthermore, aspects of the noise suppression system may be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types. Of course the underlying device technologies may be provided in a variety of component types, e.g., metal-oxide semiconductor field-effect transistor (MOSFET) technologies like complementary metal-oxide semiconductor (CMOS), bipolar technologies like emitter-coupled logic (ECL), polymer technologies (e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures), mixed analog and digital, etc. Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise," "comprising," and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in a sense of "including, but not limited to." Words using the singular or plural number also include the plural or singular number respectively. Additionally, the words "herein," "hereunder," "above," "below," and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application. When the word "or" is used in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list. The above descriptions of embodiments of the noise suppression system are not intended to be exhaustive or to limit the noise suppression system to the precise forms disclosed. While specific embodiments of, and examples for, the noise suppression system are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the noise suppression system, as those skilled in the relevant art will recognize. The teachings of the noise suppression system provided herein can be applied to other processing systems and communication systems, not only for the processing systems described above. The elements and acts of the various embodiments described above can be combined to provide further embodiments. These and other changes can be made to the noise suppression system in light of the above detailed description. All of the above references and United States patent applications are incorporated herein by reference. Aspects of the noise suppression system can be modified, if necessary, to employ the systems, functions and concepts of the various patents and applications described above to provide yet further embodiments of the noise suppression system. In general, in the following claims, the terms used should not be construed to limit the noise suppression system to the specific embodiments disclosed in the specification and the claims, but should be construed to include all processing systems that operate under the claims to provide a method for compressing and decompressing data files or streams. Accordingly, the noise suppression system is not limited by the disclosure, but instead the scope of the noise suppression system is to be determined entirely by the claims. While certain aspects of the noise suppression system are presented below in certain claim forms, the inventors contemplate the various aspects of the noise suppression system in any number of claim forms. For example, while only one aspect of the noise suppression system is recited as embodied in computer-readable medium, other aspects may likewise be embodied in computer-readable medium. Accordingly, the inventors reserve the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the noise suppression system.

Claims

CLAIMSWhat we claim is:
1. A method for removing noise from acoustic signals, comprising: receiving a plurality of acoustic signals; receiving information on the vibration of human tissue associated with human voicing activity; generating at least one first transfer function representative of the plurality of acoustic signals upon determining that voicing information is absent from the plurality of acoustic signals for at least one specified period of time; and removing noise from the plurality of acoustic signals using the first transfer function to produce at least one denoised acoustic data stream.
2. The method of claim 1, wherein removing noise further comprises: generating at least one second transfer function representative of the plurality of acoustic signals upon determining that voicing information is present in the plurality of acoustic signals for the at least one specified period of time; and removing noise from the plurality of acoustic signals using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream.
3. The method of claim 1, wherein the plurality of acoustic signals include at least one reflection of at least one associated noise source signal and at least one reflection of at least one acoustic source signal.
4. The method of claim 1, wherein receiving the plurality of acoustic signals includes receiving using a plurality of independently located microphones.
5. The method of claim 2, wherein removing noise further includes generating at least one third transfer function using the at least one first transfer function and the at least one second transfer function.
6. The method of claim 1, wherein generating the at least one first transfer function comprises recalculating the at least one first transfer function during at least one prespecified interval.
7. The method of claim 2, wherein generating the at least one second transfer function comprises recalculating the at least one second transfer function during at least one prespecified interval.
8. The method of claim 1, wherein generating the at least one first transfer function comprises use of at least one technique selected from a group consisting of adaptive techniques and recursive techniques.
9. The method of claim 1, wherein information on the vibration of human tissue is provided by a mechanical sensor in contact with the skin.
10. The method of claim 1, wherein information on the vibration of human tissue is provided via at least one sensor selected from among at least one of an accelerometer, a skin surface microphone in physical contact with skin of a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector.
1 1. The method of claim 1 , wherein the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
12. A method for removing noise from electronic signals, comprising: detecting an absence of voiced information during at least one period, wherein detecting includes measuring the vibration of human tissue; receiving at least one noise source signal during the at least one period; generating at least one transfer function representative of the at least one noise source signal; receiving at least one composite signal comprising acoustic and noise signals; and removing the noise signal from the at least one composite signal using the at least one transfer function to produce at least one denoised acoustic data stream.
13. The method of claim 12, wherein the at least one noise source signal includes at least one reflection of at least one associated noise source signal.
14. The method of claim 12, wherein the at least one composite signal includes at least one reflection of at least one associated composite signal.
15. The method of claim 12, wherein the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
16. The method of claim 12, wherein detecting includes use of a mechanical sensor in contact with the human tissue.
17. The system of claim 12, wherein detecting includes use of a sensor selected from among at least one of an accelerometer, a skin surface microphone in physical contact with a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector.
18. The method of claim 12, wherein receiving includes receiving the at least one noise source signal using at least one microphone.
19. The method of claim 18, wherein the at least one microphone includes a plurality of independently located microphones.
20. The method of claim 12, wherein removing the noise signal from the at least one composite signal using the at least one transfer function includes generating at least one other transfer function using the at least one transfer function.
21. The method of claim 12, wherein generating at least one transfer function comprises recalculating the at least one transfer function during at least one prespecified interval.
22. The method of claim 12, wherein generating the at least one transfer function comprises calculating the at least one transfer function using at least one technique selected from a group consisting of adaptive techniques and recursive techniques.
23. A method for removing noise from electronic signals, comprising: determining at least one unvoicing period during which voiced information is absent based on vibration of human tissue; receiving at least one noise signal input during the at least one unvoicing period and generating at least one unvoicing transfer function representative of the at least one noise signal; receiving at least one composite signal comprising acoustic and noise signals; and removing the noise signal from the at least one composite signal using the at least unvoicing transfer function to produce at least one denoised acoustic data stream.
24. The method of claim 23, wherein producing at least one denoised acoustic data stream further includes: determining at least one voicing period during which voiced information is present; receiving at least one acoustic signal input from at least one signal sensing device during the at least one voicing period and generating at least one voicing transfer function representative of the at least one acoustic signal; and removing the noise signal from the at least one composite signal using at least one combination of the at least one unvoicing transfer function and the at least one voicing transfer function to produce the denoised acoustic data stream.
25. The method of claim 23, wherein the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
26. A system for removing noise from the acoustic signals, comprising: at least one receiver that receives at least one acoustic signal; at least one sensor that receives human tissue vibration information associated with human voicing activity; and at least one processor coupled among the at least one receiver and the at least one sensor that generates a plurality of transfer functions, wherein at least one first transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is absent from the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using the first transfer function to produce at least one denoised acoustic data stream.
27. The system of claim 26, wherein at least one second transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is present in the at least one acoustic signal for the at least one specified period of time, wherein noise is removed from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce the at least one denoised acoustic data stream.
28. The system of claim 26, wherein the sensor includes a mechanical sensor in contact with the skin.
29. The system of claim 26, wherein the sensor includes at least one of an accelerometer, a skin surface microphone in physical contact with skin of a user, a human tissue vibration detector, a radio frequency (RF) vibration detector, and a laser vibration detector.
30. The system of claim 26, wherein the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
31. The system of claim 26, further comprising: dividing acoustic data of the at least one acoustic signal into a plurality of subbands; removing noise from each of the plurality of subbands using the at least one first transfer function, wherein a plurality of denoised acoustic data streams are generated; and combining the plurality of denoised acoustic data streams to generate the at least one denoised acoustic data stream.
32. The system of claim 26, wherein the at least one receiver includes a plurality of independently located microphones.
33. A system for removing noise from acoustic signals, comprising at least one processor coupled among at least one microphone and at least one voicing sensor, wherein the at least one voicing sensor detects human tissue vibration associated with voicing, wherein an absence of voiced information is detected during at least one period using the at least one voicing sensor, wherein at least one noise source signal is received during the at least one period using the at least one microphone, wherein the at least one processor generates at least one transfer function representative of the at least one noise source signal, wherein the at least one microphone receives at least one composite signal comprising acoustic and noise signals, and the at least one processor removes the noise signal from the at least one composite signal using the at least one transfer function to produce at least one denoised acoustic data stream.
34. The system of claim 33, wherein the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
35. A signal processing system coupled among at least one user and at least one electronic device, wherein the signal processing system includes at least one denoising subsystem for removing noise from acoustic signals, the denoising subsystem comprising at least one processor coupled among at least one receiver and at least one sensor, wherein the at least one receiver is coupled to receive at least one acoustic signal, wherein the at least one sensor detects human tissue vibration associated with human voicing activity, wherein the at least one processor generates a plurality of transfer functions, wherein at least one first transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is absent from the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using the first transfer function to produce at least one denoised acoustic data stream.
36. The system of claim 35, wherein at least one second transfer function representative of the at least one acoustic signal is generated in response to a determination that voicing information is present in the at least one acoustic signal for at least one specified period of time, wherein noise is removed from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream.
37. The system of claim 35, wherein the at least one electronic device includes at least one of cellular telephones, personal digital assistants, portable communication devices, computers, video cameras, digital cameras, and telematics systems.
38. The system of claim 35, wherein the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
39. A computer readable medium comprising executable instructions which, when executed in a processing system, remove noise from received acoustic signals by: receiving at least one acoustic signal; receiving human tissue vibration information associated with human voicing activity; generating at least one first transfer function representative of the at least one acoustic signal upon determining that voicing information is absent from the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using the at least one first transfer function to produce at least one denoised acoustic data stream.
40. The medium of claim 39, wherein removing noise from received acoustic signals further includes: generating at least one second transfer function representative of the at least one acoustic signal upon determining that voicing information is present in the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream.
41. The medium of claim 39, wherein the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
42. An electromagnetic medium comprising executable instructions which, when executed in a processing system, remove noise from received acoustic signals by: receiving at least one acoustic signal; receiving human tissue vibration information associated with human voicing activity; generating at least one first transfer function representative of the at least one acoustic signal upon determining that voicing information is absent from the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using the at least one first transfer function to produce at least one denoised acoustic data stream.
43. The medium of claim 42, wherein removing noise from received acoustic signals further includes: generating at least one second transfer function representative of the at least one acoustic signal upon determining that voicing information is present in the at least one acoustic signal for at least one specified period of time; and removing noise from the at least one acoustic signal using at least one combination of the at least one first transfer function and the at least one second transfer function to produce at least one denoised acoustic data stream.
44. The medium of claim 42, wherein the human tissue is at least one of on a surface of a head, near the surface of the head, on a surface of a neck, near the surface of the neck, on a surface of a chest, and near the surface of the chest.
PCT/US2004/029234 2003-09-18 2004-09-09 Voice activity detector (vad) -based multiple-microphone acoustic noise suppression WO2005029468A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/667,207 2003-09-18
US10/667,207 US8019091B2 (en) 2000-07-19 2003-09-18 Voice activity detector (VAD) -based multiple-microphone acoustic noise suppression

Publications (1)

Publication Number Publication Date
WO2005029468A1 true WO2005029468A1 (en) 2005-03-31

Family

ID=34375865

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2004/029234 WO2005029468A1 (en) 2003-09-18 2004-09-09 Voice activity detector (vad) -based multiple-microphone acoustic noise suppression

Country Status (3)

Country Link
US (3) US8019091B2 (en)
TW (1) TWI281354B (en)
WO (1) WO2005029468A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2196988A1 (en) * 2008-12-12 2010-06-16 Harman/Becker Automotive Systems GmbH Determination of the coherence of audio signals
EP2621150A1 (en) * 2012-01-30 2013-07-31 Research In Motion Limited Adjusted noise suppression and voice activity detection
EP2736041A1 (en) * 2012-11-21 2014-05-28 Harman International Industries Canada, Ltd. System to selectively modify audio effect parameters of vocal signals
US8831686B2 (en) 2012-01-30 2014-09-09 Blackberry Limited Adjusted noise suppression and voice activity detection
CN112435683A (en) * 2020-07-30 2021-03-02 珠海市杰理科技股份有限公司 Adaptive noise estimation and voice noise reduction method based on T-S fuzzy neural network

Families Citing this family (112)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8280072B2 (en) 2003-03-27 2012-10-02 Aliphcom, Inc. Microphone array with rear venting
US8019091B2 (en) 2000-07-19 2011-09-13 Aliphcom, Inc. Voice activity detector (VAD) -based multiple-microphone acoustic noise suppression
US8452023B2 (en) * 2007-05-25 2013-05-28 Aliphcom Wind suppression/replacement component for use with electronic systems
US8326611B2 (en) * 2007-05-25 2012-12-04 Aliphcom, Inc. Acoustic voice activity detection (AVAD) for electronic systems
EP1443498B1 (en) * 2003-01-24 2008-03-19 Sony Ericsson Mobile Communications AB Noise reduction and audio-visual speech activity detection
US9066186B2 (en) 2003-01-30 2015-06-23 Aliphcom Light-based detection for acoustic applications
US9099094B2 (en) 2003-03-27 2015-08-04 Aliphcom Microphone array with rear venting
KR100705563B1 (en) * 2004-12-07 2007-04-10 삼성전자주식회사 Speech Recognition System capable of Controlling Automatically Inputting Level and Speech Recognition Method using the same
US20070036342A1 (en) * 2005-08-05 2007-02-15 Boillot Marc A Method and system for operation of a voice activity detector
GB2430129B (en) * 2005-09-08 2007-10-31 Motorola Inc Voice activity detector and method of operation therein
US8345890B2 (en) 2006-01-05 2013-01-01 Audience, Inc. System and method for utilizing inter-microphone level differences for speech enhancement
US8744844B2 (en) 2007-07-06 2014-06-03 Audience, Inc. System and method for adaptive intelligent noise suppression
US8194880B2 (en) 2006-01-30 2012-06-05 Audience, Inc. System and method for utilizing omni-directional microphones for speech enhancement
US9185487B2 (en) 2006-01-30 2015-11-10 Audience, Inc. System and method for providing noise suppression utilizing null processing noise subtraction
US8204252B1 (en) 2006-10-10 2012-06-19 Audience, Inc. System and method for providing close microphone adaptive array processing
US20070276658A1 (en) * 2006-05-23 2007-11-29 Barry Grayson Douglass Apparatus and Method for Detecting Speech Using Acoustic Signals Outside the Audible Frequency Range
US8934641B2 (en) 2006-05-25 2015-01-13 Audience, Inc. Systems and methods for reconstructing decomposed audio signals
US8150065B2 (en) 2006-05-25 2012-04-03 Audience, Inc. System and method for processing an audio signal
US8849231B1 (en) 2007-08-08 2014-09-30 Audience, Inc. System and method for adaptive power control
US8204253B1 (en) 2008-06-30 2012-06-19 Audience, Inc. Self calibration of audio device
US8949120B1 (en) 2006-05-25 2015-02-03 Audience, Inc. Adaptive noise cancelation
EP2033489B1 (en) 2006-06-14 2015-10-28 Personics Holdings, LLC. Earguard monitoring system
US20080260169A1 (en) * 2006-11-06 2008-10-23 Plantronics, Inc. Headset Derived Real Time Presence And Communication Systems And Methods
US9591392B2 (en) * 2006-11-06 2017-03-07 Plantronics, Inc. Headset-derived real-time presence and communication systems and methods
WO2008062782A1 (en) * 2006-11-20 2008-05-29 Nec Corporation Speech estimation system, speech estimation method, and speech estimation program
US8254591B2 (en) 2007-02-01 2012-08-28 Personics Holdings Inc. Method and device for audio recording
US8259926B1 (en) 2007-02-23 2012-09-04 Audience, Inc. System and method for 2-channel and 3-channel acoustic echo cancellation
US11683643B2 (en) 2007-05-04 2023-06-20 Staton Techiya Llc Method and device for in ear canal echo suppression
US11856375B2 (en) 2007-05-04 2023-12-26 Staton Techiya Llc Method and device for in-ear echo suppression
US8503686B2 (en) 2007-05-25 2013-08-06 Aliphcom Vibration sensor and acoustic voice activity detection system (VADS) for use with electronic systems
US8321213B2 (en) * 2007-05-25 2012-11-27 Aliphcom, Inc. Acoustic voice activity detection (AVAD) for electronic systems
US8488803B2 (en) * 2007-05-25 2013-07-16 Aliphcom Wind suppression/replacement component for use with electronic systems
US8982744B2 (en) * 2007-06-06 2015-03-17 Broadcom Corporation Method and system for a subband acoustic echo canceller with integrated voice activity detection
US8189766B1 (en) 2007-07-26 2012-05-29 Audience, Inc. System and method for blind subband acoustic echo cancellation postfiltering
US8180064B1 (en) 2007-12-21 2012-05-15 Audience, Inc. System and method for providing voice equalization
US8143620B1 (en) 2007-12-21 2012-03-27 Audience, Inc. System and method for adaptive classification of audio sources
US8554550B2 (en) 2008-01-28 2013-10-08 Qualcomm Incorporated Systems, methods, and apparatus for context processing using multi resolution analysis
US8194882B2 (en) 2008-02-29 2012-06-05 Audience, Inc. System and method for providing single microphone noise suppression fallback
US8355511B2 (en) 2008-03-18 2013-01-15 Audience, Inc. System and method for envelope-based acoustic echo cancellation
US9094764B2 (en) * 2008-04-02 2015-07-28 Plantronics, Inc. Voice activity detection with capacitive touch sense
US8774423B1 (en) 2008-06-30 2014-07-08 Audience, Inc. System and method for controlling adaptivity of signal modification using a phantom coefficient
US8521530B1 (en) 2008-06-30 2013-08-27 Audience, Inc. System and method for enhancing a monaural audio signal
US8600067B2 (en) 2008-09-19 2013-12-03 Personics Holdings Inc. Acoustic sealing analysis system
US8948415B1 (en) 2009-10-26 2015-02-03 Plantronics, Inc. Mobile device with discretionary two microphone noise reduction
US9838784B2 (en) 2009-12-02 2017-12-05 Knowles Electronics, Llc Directional audio capture
US9008329B1 (en) 2010-01-26 2015-04-14 Audience, Inc. Noise reduction using multi-feature cluster tracker
US8798290B1 (en) 2010-04-21 2014-08-05 Audience, Inc. Systems and methods for adaptive signal equalization
US8504629B2 (en) * 2010-07-01 2013-08-06 Plantronics, Inc. Connection device and protocol
US8532987B2 (en) * 2010-08-24 2013-09-10 Lawrence Livermore National Security, Llc Speech masking and cancelling and voice obscuration
US8831937B2 (en) * 2010-11-12 2014-09-09 Audience, Inc. Post-noise suppression processing to improve voice quality
JP5874344B2 (en) * 2010-11-24 2016-03-02 株式会社Jvcケンウッド Voice determination device, voice determination method, and voice determination program
CN102411936B (en) * 2010-11-25 2012-11-14 歌尔声学股份有限公司 Speech enhancement method and device as well as head de-noising communication earphone
US9264553B2 (en) 2011-06-11 2016-02-16 Clearone Communications, Inc. Methods and apparatuses for echo cancelation with beamforming microphone arrays
WO2012176199A1 (en) * 2011-06-22 2012-12-27 Vocalzoom Systems Ltd Method and system for identification of speech segments
US9100756B2 (en) 2012-06-08 2015-08-04 Apple Inc. Microphone occlusion detector
US9966067B2 (en) * 2012-06-08 2018-05-08 Apple Inc. Audio noise estimation and audio noise reduction using multiple microphones
US9094749B2 (en) * 2012-07-25 2015-07-28 Nokia Technologies Oy Head-mounted sound capture device
US9135915B1 (en) 2012-07-26 2015-09-15 Google Inc. Augmenting speech segmentation and recognition using head-mounted vibration and/or motion sensors
US9313572B2 (en) 2012-09-28 2016-04-12 Apple Inc. System and method of detecting a user's voice activity using an accelerometer
US9438985B2 (en) 2012-09-28 2016-09-06 Apple Inc. System and method of detecting a user's voice activity using an accelerometer
US9640194B1 (en) 2012-10-04 2017-05-02 Knowles Electronics, Llc Noise suppression for speech processing based on machine-learning mask estimation
US9363596B2 (en) 2013-03-15 2016-06-07 Apple Inc. System and method of mixing accelerometer and microphone signals to improve voice quality in a mobile device
US9536540B2 (en) 2013-07-19 2017-01-03 Knowles Electronics, Llc Speech signal separation and synthesis based on auditory scene analysis and speech modeling
TWI533289B (en) * 2013-10-04 2016-05-11 晨星半導體股份有限公司 Electronic device and calibrating system for suppressing noise and method thereof
US10043534B2 (en) 2013-12-23 2018-08-07 Staton Techiya, Llc Method and device for spectral expansion for an audio signal
US20150199950A1 (en) * 2014-01-13 2015-07-16 DSP Group Use of microphones with vsensors for wearable devices
US9524735B2 (en) 2014-01-31 2016-12-20 Apple Inc. Threshold adaptation in two-channel noise estimation and voice activity detection
US9530433B2 (en) * 2014-03-17 2016-12-27 Sharp Laboratories Of America, Inc. Voice activity detection for noise-canceling bioacoustic sensor
US9807492B1 (en) 2014-05-01 2017-10-31 Ambarella, Inc. System and/or method for enhancing hearing using a camera module, processor and/or audio input and/or output devices
US9467779B2 (en) 2014-05-13 2016-10-11 Apple Inc. Microphone partial occlusion detector
KR102351061B1 (en) * 2014-07-23 2022-01-13 현대모비스 주식회사 Method and apparatus for voice recognition
CN106797512B (en) 2014-08-28 2019-10-25 美商楼氏电子有限公司 Method, system and the non-transitory computer-readable storage medium of multi-source noise suppressed
US9978388B2 (en) 2014-09-12 2018-05-22 Knowles Electronics, Llc Systems and methods for restoration of speech components
US9830925B2 (en) * 2014-10-22 2017-11-28 GM Global Technology Operations LLC Selective noise suppression during automatic speech recognition
US10163453B2 (en) 2014-10-24 2018-12-25 Staton Techiya, Llc Robust voice activity detector system for use with an earphone
US9685156B2 (en) * 2015-03-12 2017-06-20 Sony Mobile Communications Inc. Low-power voice command detector
US9554207B2 (en) * 2015-04-30 2017-01-24 Shure Acquisition Holdings, Inc. Offset cartridge microphones
US9565493B2 (en) 2015-04-30 2017-02-07 Shure Acquisition Holdings, Inc. Array microphone system and method of assembling the same
US20160379661A1 (en) * 2015-06-26 2016-12-29 Intel IP Corporation Noise reduction for electronic devices
US9691413B2 (en) 2015-10-06 2017-06-27 Microsoft Technology Licensing, Llc Identifying sound from a source of interest based on multiple audio feeds
US20170150254A1 (en) * 2015-11-19 2017-05-25 Vocalzoom Systems Ltd. System, device, and method of sound isolation and signal enhancement
US10616693B2 (en) 2016-01-22 2020-04-07 Staton Techiya Llc System and method for efficiency among devices
US9997173B2 (en) 2016-03-14 2018-06-12 Apple Inc. System and method for performing automatic gain control using an accelerometer in a headset
DE102016003401B4 (en) * 2016-03-19 2021-06-10 Audi Ag Acquisition device and method for acquiring a speech utterance by a speaking person in a motor vehicle
US9820042B1 (en) 2016-05-02 2017-11-14 Knowles Electronics, Llc Stereo separation and directional suppression with omni-directional microphones
US10311219B2 (en) 2016-06-07 2019-06-04 Vocalzoom Systems Ltd. Device, system, and method of user authentication utilizing an optical microphone
US20170365249A1 (en) * 2016-06-21 2017-12-21 Apple Inc. System and method of performing automatic speech recognition using end-pointing markers generated using accelerometer-based voice activity detector
US10482899B2 (en) 2016-08-01 2019-11-19 Apple Inc. Coordination of beamformers for noise estimation and noise suppression
US10566007B2 (en) * 2016-09-08 2020-02-18 The Regents Of The University Of Michigan System and method for authenticating voice commands for a voice assistant
US10433087B2 (en) * 2016-09-15 2019-10-01 Qualcomm Incorporated Systems and methods for reducing vibration noise
US10367948B2 (en) 2017-01-13 2019-07-30 Shure Acquisition Holdings, Inc. Post-mixing acoustic echo cancellation systems and methods
US10845956B2 (en) 2017-05-31 2020-11-24 Snap Inc. Methods and systems for voice driven dynamic menus
US10468020B2 (en) * 2017-06-06 2019-11-05 Cypress Semiconductor Corporation Systems and methods for removing interference for audio pattern recognition
US9998577B1 (en) 2017-06-19 2018-06-12 Motorola Solutions, Inc. Method and apparatus for managing noise levels using push-to-talk event activated vibration microphone
US10339949B1 (en) 2017-12-19 2019-07-02 Apple Inc. Multi-channel speech enhancement
CN107910011B (en) * 2017-12-28 2021-05-04 科大讯飞股份有限公司 Voice noise reduction method and device, server and storage medium
US10951994B2 (en) 2018-04-04 2021-03-16 Staton Techiya, Llc Method to acquire preferred dynamic range function for speech enhancement
EP3804356A1 (en) 2018-06-01 2021-04-14 Shure Acquisition Holdings, Inc. Pattern-forming microphone array
US11297423B2 (en) 2018-06-15 2022-04-05 Shure Acquisition Holdings, Inc. Endfire linear array microphone
WO2020061353A1 (en) 2018-09-20 2020-03-26 Shure Acquisition Holdings, Inc. Adjustable lobe shape for array microphones
EP3942842A1 (en) 2019-03-21 2022-01-26 Shure Acquisition Holdings, Inc. Housings and associated design features for ceiling array microphones
US11558693B2 (en) 2019-03-21 2023-01-17 Shure Acquisition Holdings, Inc. Auto focus, auto focus within regions, and auto placement of beamformed microphone lobes with inhibition and voice activity detection functionality
CN113841421A (en) 2019-03-21 2021-12-24 舒尔获得控股公司 Auto-focus, in-region auto-focus, and auto-configuration of beamforming microphone lobes with suppression
WO2020237206A1 (en) 2019-05-23 2020-11-26 Shure Acquisition Holdings, Inc. Steerable speaker array, system, and method for the same
US11302347B2 (en) 2019-05-31 2022-04-12 Shure Acquisition Holdings, Inc. Low latency automixer integrated with voice and noise activity detection
EP4018680A1 (en) 2019-08-23 2022-06-29 Shure Acquisition Holdings, Inc. Two-dimensional microphone array with improved directivity
US11552611B2 (en) 2020-02-07 2023-01-10 Shure Acquisition Holdings, Inc. System and method for automatic adjustment of reference gain
USD944776S1 (en) 2020-05-05 2022-03-01 Shure Acquisition Holdings, Inc. Audio device
US11706562B2 (en) 2020-05-29 2023-07-18 Shure Acquisition Holdings, Inc. Transducer steering and configuration systems and methods using a local positioning system
EP4285605A1 (en) 2021-01-28 2023-12-06 Shure Acquisition Holdings, Inc. Hybrid audio beamforming system
US11942107B2 (en) 2021-02-23 2024-03-26 Stmicroelectronics S.R.L. Voice activity detection with low-power accelerometer
WO2023028018A1 (en) 2021-08-26 2023-03-02 Dolby Laboratories Licensing Corporation Detecting environmental noise in user-generated content

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002007151A2 (en) * 2000-07-19 2002-01-24 Aliphcom Method and apparatus for removing noise from speech signals
US20030040908A1 (en) * 2001-02-12 2003-02-27 Fortemedia, Inc. Noise suppression for speech signal in an automobile
WO2004056298A1 (en) * 2001-11-21 2004-07-08 Aliphcom Method and apparatus for removing noise from electronic signals

Family Cites Families (171)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US2121779A (en) 1935-02-12 1938-06-28 Ballantine Stuart Sound translating apparatus
US3789166A (en) * 1971-12-16 1974-01-29 Dyna Magnetic Devices Inc Submersion-safe microphone
DE2429045A1 (en) 1974-06-18 1976-01-08 Blasius Speidel BODY SOUND MICROPHONE
US4006318A (en) * 1975-04-21 1977-02-01 Dyna Magnetic Devices, Inc. Inertial microphone system
US4521908A (en) 1982-09-01 1985-06-04 Victor Company Of Japan, Limited Phased-array sound pickup apparatus having no unwanted response pattern
US4607383A (en) 1983-08-18 1986-08-19 Gentex Corporation Throat microphone
US4591668A (en) * 1984-05-08 1986-05-27 Iwata Electric Co., Ltd. Vibration-detecting type microphone
US4777649A (en) 1985-10-22 1988-10-11 Speech Systems, Inc. Acoustic feedback control of microphone positioning and speaking volume
US4653102A (en) 1985-11-05 1987-03-24 Position Orientation Systems Directional microphone system
DE3742929C1 (en) * 1987-12-18 1988-09-29 Daimler Benz Ag Method for improving the reliability of voice controls of functional elements and device for carrying it out
US5276765A (en) 1988-03-11 1994-01-04 British Telecommunications Public Limited Company Voice activity detection
DE3825973A1 (en) 1988-07-29 1990-02-01 Siemens Ag ELECTROACOUSTIC UNIT TRANSFORMER
JPH02149199A (en) * 1988-11-30 1990-06-07 Matsushita Electric Ind Co Ltd Electlet condenser microphone
EP0386765B1 (en) 1989-03-10 1994-08-24 Nippon Telegraph And Telephone Corporation Method of detecting acoustic signal
US5212764A (en) 1989-04-19 1993-05-18 Ricoh Company, Ltd. Noise eliminating apparatus and speech recognition apparatus using the same
EP0453230B1 (en) 1990-04-20 1995-06-21 Matsushita Electric Industrial Co., Ltd. Speaker system
US5205285A (en) 1991-06-14 1993-04-27 Cyberonics, Inc. Voice suppression of vagal stimulation
JP3279612B2 (en) 1991-12-06 2002-04-30 ソニー株式会社 Noise reduction device
FR2687496B1 (en) 1992-02-18 1994-04-01 Alcatel Radiotelephone METHOD FOR REDUCING ACOUSTIC NOISE IN A SPEAKING SIGNAL.
US5353376A (en) 1992-03-20 1994-10-04 Texas Instruments Incorporated System and method for improved speech acquisition for hands-free voice telecommunication in a noisy environment
JP3277398B2 (en) 1992-04-15 2002-04-22 ソニー株式会社 Voiced sound discrimination method
JP3176474B2 (en) * 1992-06-03 2001-06-18 沖電気工業株式会社 Adaptive noise canceller device
US5448637A (en) 1992-10-20 1995-09-05 Pan Communications, Inc. Two-way communications earset
US5732143A (en) 1992-10-29 1998-03-24 Andrea Electronics Corp. Noise cancellation apparatus
US5400409A (en) 1992-12-23 1995-03-21 Daimler-Benz Ag Noise-reduction method for noise-affected voice channels
US5625684A (en) 1993-02-04 1997-04-29 Local Silence, Inc. Active noise suppression system for telephone handsets and method
JPH06318885A (en) * 1993-03-11 1994-11-15 Nec Corp Unknown system identifying method/device using band division adaptive filter
US5459814A (en) 1993-03-26 1995-10-17 Hughes Aircraft Company Voice activity detector for speech signals in variable background noise
US5633935A (en) * 1993-04-13 1997-05-27 Matsushita Electric Industrial Co., Ltd. Stereo ultradirectional microphone apparatus
US5590241A (en) * 1993-04-30 1996-12-31 Motorola Inc. Speech processing system and method for enhancing a speech signal in a noisy environment
US5414776A (en) 1993-05-13 1995-05-09 Lectrosonics, Inc. Adaptive proportional gain audio mixing system
ES2142323T3 (en) * 1993-07-28 2000-04-16 Pan Communications Inc TWO-WAY COMBINED HEADPHONE.
US5406622A (en) * 1993-09-02 1995-04-11 At&T Corp. Outbound noise cancellation for telephonic handset
US5463694A (en) * 1993-11-01 1995-10-31 Motorola Gradient directional microphone system and method therefor
US5473701A (en) * 1993-11-05 1995-12-05 At&T Corp. Adaptive microphone array
US5684460A (en) * 1994-04-22 1997-11-04 The United States Of America As Represented By The Secretary Of The Army Motion and sound monitor and stimulator
US5515865A (en) * 1994-04-22 1996-05-14 The United States Of America As Represented By The Secretary Of The Army Sudden Infant Death Syndrome (SIDS) monitor and stimulator
JPH07298387A (en) 1994-04-28 1995-11-10 Canon Inc Stereophonic audio input device
US5402669A (en) 1994-05-16 1995-04-04 General Electric Company Sensor matching through source modeling and output compensation
US5933506A (en) * 1994-05-18 1999-08-03 Nippon Telegraph And Telephone Corporation Transmitter-receiver having ear-piece type acoustic transducing part
US5815582A (en) 1994-12-02 1998-09-29 Noise Cancellation Technologies, Inc. Active plus selective headset
JPH08181754A (en) 1994-12-21 1996-07-12 Matsushita Electric Ind Co Ltd Handset for communication equipment
JP2758846B2 (en) * 1995-02-27 1998-05-28 埼玉日本電気株式会社 Noise canceller device
US5835608A (en) 1995-07-10 1998-11-10 Applied Acoustic Research Signal separating system
US6000396A (en) * 1995-08-17 1999-12-14 University Of Florida Hybrid microprocessor controlled ventilator unit
US5729694A (en) * 1996-02-06 1998-03-17 The Regents Of The University Of California Speech coding, reconstruction and recognition using acoustics and electromagnetic waves
US6006175A (en) * 1996-02-06 1999-12-21 The Regents Of The University Of California Methods and apparatus for non-acoustic speech characterization and recognition
JP3522954B2 (en) 1996-03-15 2004-04-26 株式会社東芝 Microphone array input type speech recognition apparatus and method
US5853005A (en) * 1996-05-02 1998-12-29 The United States Of America As Represented By The Secretary Of The Army Acoustic monitoring system
US5796842A (en) 1996-06-07 1998-08-18 That Corporation BTSC encoder
JP3297307B2 (en) 1996-06-14 2002-07-02 沖電気工業株式会社 Background noise canceller
DE19635229C2 (en) * 1996-08-30 2001-04-26 Siemens Audiologische Technik Direction sensitive hearing aid
US6408079B1 (en) 1996-10-23 2002-06-18 Matsushita Electric Industrial Co., Ltd. Distortion removal apparatus, method for determining coefficient for the same, and processing speaker system, multi-processor, and amplifier including the same
JP2874679B2 (en) 1997-01-29 1999-03-24 日本電気株式会社 Noise elimination method and apparatus
US6041127A (en) 1997-04-03 2000-03-21 Lucent Technologies Inc. Steerable and variable first-order differential microphone array
US6430295B1 (en) 1997-07-11 2002-08-06 Telefonaktiebolaget Lm Ericsson (Publ) Methods and apparatus for measuring signal level and delay at multiple sensors
FI114422B (en) * 1997-09-04 2004-10-15 Nokia Corp Source speech activity detection
US5986600A (en) 1998-01-22 1999-11-16 Mcewan; Thomas E. Pulsed RF oscillator and radar motion sensor
JP3344647B2 (en) 1998-02-18 2002-11-11 富士通株式会社 Microphone array device
US5966090A (en) 1998-03-16 1999-10-12 Mcewan; Thomas E. Differential pulse radar motion sensor
US6420975B1 (en) 1999-08-25 2002-07-16 Donnelly Corporation Interior rearview mirror sound processing system
US6173059B1 (en) 1998-04-24 2001-01-09 Gentner Communications Corporation Teleconferencing system with visual feedback
KR100474826B1 (en) 1998-05-09 2005-05-16 삼성전자주식회사 Method and apparatus for deteminating multiband voicing levels using frequency shifting method in voice coder
US6717991B1 (en) 1998-05-27 2004-04-06 Telefonaktiebolaget Lm Ericsson (Publ) System and method for dual microphone signal noise reduction using spectral subtraction
JP3955686B2 (en) 1998-08-31 2007-08-08 株式会社オーディオテクニカ Waterproof microphone
EP1145219B1 (en) 1999-01-15 2012-08-15 Fishman Transducers, Inc. Measurement and processing of stringed acoustic instrument signals
US6191724B1 (en) 1999-01-28 2001-02-20 Mcewan Thomas E. Short pulse microwave transceiver
US7146013B1 (en) 1999-04-28 2006-12-05 Alpine Electronics, Inc. Microphone system
JP2000312395A (en) * 1999-04-28 2000-11-07 Alpine Electronics Inc Microphone system
US7120261B1 (en) 1999-11-19 2006-10-10 Gentex Corporation Vehicle accessory microphone
US6473733B1 (en) 1999-12-01 2002-10-29 Research In Motion Limited Signal enhancement for voice coding
JP2001189987A (en) * 1999-12-28 2001-07-10 Pioneer Electronic Corp Narrow directivity microphone unit
US6816469B1 (en) 1999-12-30 2004-11-09 At&T Corp. IP conference call waiting
US6766292B1 (en) 2000-03-28 2004-07-20 Tellabs Operations, Inc. Relative noise ratio weighting techniques for adaptive noise cancellation
US6980092B2 (en) 2000-04-06 2005-12-27 Gentex Corporation Vehicle rearview mirror assembly incorporating a communication system
DE10017646A1 (en) 2000-04-08 2001-10-11 Alcatel Sa Noise suppression in the time domain
DK174402B1 (en) 2000-05-09 2003-02-10 Gn Netcom As communication Unit
US6668062B1 (en) 2000-05-09 2003-12-23 Gn Resound As FFT-based technique for adaptive directionality of dual microphones
FR2808958B1 (en) 2000-05-11 2002-10-25 Sagem PORTABLE TELEPHONE WITH SURROUNDING NOISE MITIGATION
US6771788B1 (en) 2000-05-25 2004-08-03 Harman Becker Automotive Systems-Wavemakers, Inc. Shielded microphone
US8254617B2 (en) 2003-03-27 2012-08-28 Aliphcom, Inc. Microphone array with rear venting
US8280072B2 (en) 2003-03-27 2012-10-02 Aliphcom, Inc. Microphone array with rear venting
US8019091B2 (en) 2000-07-19 2011-09-13 Aliphcom, Inc. Voice activity detector (VAD) -based multiple-microphone acoustic noise suppression
US7246058B2 (en) 2001-05-30 2007-07-17 Aliph, Inc. Detecting voiced and unvoiced speech using both acoustic and nonacoustic sensors
US8682018B2 (en) 2000-07-19 2014-03-25 Aliphcom Microphone array with rear venting
US20070233479A1 (en) 2002-05-30 2007-10-04 Burnett Gregory C Detecting voiced and unvoiced speech using both acoustic and nonacoustic sensors
US20020116187A1 (en) 2000-10-04 2002-08-22 Gamze Erten Speech detection
US6963649B2 (en) 2000-10-24 2005-11-08 Adaptive Technologies, Inc. Noise cancelling microphone
US6889187B2 (en) 2000-12-28 2005-05-03 Nortel Networks Limited Method and apparatus for improved voice activity detection in a packet voice network
US7206418B2 (en) 2001-02-12 2007-04-17 Fortemedia, Inc. Noise suppression for a wireless communication device
AU2002250080A1 (en) 2001-02-14 2002-08-28 Gentex Corporation Vehicle accessory microphone
US7171357B2 (en) 2001-03-21 2007-01-30 Avaya Technology Corp. Voice-activity detection using energy ratios and periodicity
DE10118653C2 (en) 2001-04-14 2003-03-27 Daimler Chrysler Ag Method for noise reduction
US8326611B2 (en) 2007-05-25 2012-12-04 Aliphcom, Inc. Acoustic voice activity detection (AVAD) for electronic systems
US8452023B2 (en) 2007-05-25 2013-05-28 Aliphcom Wind suppression/replacement component for use with electronic systems
US7433484B2 (en) 2003-01-30 2008-10-07 Aliphcom, Inc. Acoustic vibration sensor
WO2002098169A1 (en) 2001-05-30 2002-12-05 Aliphcom Detecting voiced and unvoiced speech using both acoustic and nonacoustic sensors
US6996241B2 (en) 2001-06-22 2006-02-07 Trustees Of Dartmouth College Tuned feedforward LMS filter with feedback control
JP4161547B2 (en) 2001-06-28 2008-10-08 株式会社Sumco Single crystal pulling apparatus, single crystal pulling method, program and recording medium
US7123727B2 (en) 2001-07-18 2006-10-17 Agere Systems Inc. Adaptive close-talking differential microphone array
EP1413169A1 (en) 2001-08-01 2004-04-28 Dashen Fan Cardioid beam with a desired null based acoustic devices, systems and methods
US20030044025A1 (en) 2001-08-29 2003-03-06 Innomedia Pte Ltd. Circuit and method for acoustic source directional pattern determination utilizing two microphones
AU2002365352A1 (en) 2001-11-27 2003-06-10 Corporation For National Research Initiatives A miniature condenser microphone and fabrication method therefor
US7742588B2 (en) 2001-12-31 2010-06-22 Polycom, Inc. Speakerphone establishing and using a second connection of graphics information
US7085715B2 (en) 2002-01-10 2006-08-01 Mitel Networks Corporation Method and apparatus of controlling noise level calculations in a conferencing system
US8942387B2 (en) 2002-02-05 2015-01-27 Mh Acoustics Llc Noise-reducing directional microphone array
US8098844B2 (en) 2002-02-05 2012-01-17 Mh Acoustics, Llc Dual-microphone spatial noise suppression
EP1483591A2 (en) 2002-03-05 2004-12-08 Aliphcom Voice activity detection (vad) devices and methods for use with noise suppression systems
KR20040101373A (en) 2002-03-27 2004-12-02 앨리프컴 Microphone and voice activity detection (vad) configurations for use with communication systems
DE10223544C1 (en) 2002-05-27 2003-07-24 Siemens Audiologische Technik Amplifier device for hearing aid with microphone and pick-up coil inputs, has amplifier provided with separate filters for acoustic and inductive feedback compensation
US7613310B2 (en) 2003-08-27 2009-11-03 Sony Computer Entertainment Inc. Audio input system
US6685638B1 (en) 2002-12-23 2004-02-03 Codman & Shurtleff, Inc. Acoustic monitoring system
US9066186B2 (en) 2003-01-30 2015-06-23 Aliphcom Light-based detection for acoustic applications
US7895036B2 (en) 2003-02-21 2011-02-22 Qnx Software Systems Co. System for suppressing wind noise
US7885420B2 (en) 2003-02-21 2011-02-08 Qnx Software Systems Co. Wind noise suppression system
WO2004077020A2 (en) 2003-02-25 2004-09-10 Spectragenics, Inc. Skin sensing method and apparatus
FR2852779B1 (en) 2003-03-20 2008-08-01 PROCESS FOR PROCESSING AN ELECTRICAL SIGNAL OF SOUND
US8477961B2 (en) 2003-03-27 2013-07-02 Aliphcom, Inc. Microphone array with rear venting
US9099094B2 (en) 2003-03-27 2015-08-04 Aliphcom Microphone array with rear venting
EP1478208B1 (en) 2003-05-13 2009-01-07 Harman Becker Automotive Systems GmbH A method and system for self-compensating for microphone non-uniformities
JP2004361938A (en) 2003-05-15 2004-12-24 Takenaka Komuten Co Ltd Noise reduction device
US7099821B2 (en) 2003-09-12 2006-08-29 Softmax, Inc. Separation of target acoustic signals in a multi-transducer arrangement
US20050071154A1 (en) 2003-09-30 2005-03-31 Walter Etter Method and apparatus for estimating noise in speech signals
SG119199A1 (en) 2003-09-30 2006-02-28 Stmicroelectronics Asia Pacfic Voice activity detector
US7190775B2 (en) 2003-10-29 2007-03-13 Broadcom Corporation High quality audio conferencing with adaptive beamforming
US20070149246A1 (en) 2004-01-09 2007-06-28 Revolabs, Inc. Wireless multi-user audio system
US7243912B2 (en) 2004-02-24 2007-07-17 Siemens Water Technologies Holding Corp. Aeration diffuser membrane slitting pattern
US8644525B2 (en) 2004-06-02 2014-02-04 Clearone Communications, Inc. Virtual microphones in electronic conferencing systems
US7864937B2 (en) 2004-06-02 2011-01-04 Clearone Communications, Inc. Common control of an electronic multi-pod conferencing system
US7916849B2 (en) 2004-06-02 2011-03-29 Clearone Communications, Inc. Systems and methods for managing the gating of microphones in a multi-pod conference system
US7649988B2 (en) 2004-06-15 2010-01-19 Acoustic Technologies, Inc. Comfort noise generator using modified Doblinger noise estimate
GB0415626D0 (en) 2004-07-13 2004-08-18 1 Ltd Directional microphone
US7970151B2 (en) 2004-10-15 2011-06-28 Lifesize Communications, Inc. Hybrid beamforming
EP1810221B1 (en) 2004-10-16 2014-06-25 Identix Incorporated Diffractive imaging system for acquiring an image of skin topology and corresponding method
EP1806030B1 (en) 2004-10-19 2014-10-08 Widex A/S System and method for adaptive microphone matching in a hearing aid
US7778408B2 (en) 2004-12-30 2010-08-17 Texas Instruments Incorporated Method and apparatus for acoustic echo cancellation utilizing dual filters
US7464029B2 (en) 2005-07-22 2008-12-09 Qualcomm Incorporated Robust separation of speech signals in a noisy environment
JP4356670B2 (en) 2005-09-12 2009-11-04 ソニー株式会社 Noise reduction device, noise reduction method, noise reduction program, and sound collection device for electronic device
US7983433B2 (en) 2005-11-08 2011-07-19 Think-A-Move, Ltd. Earset assembly
JP4669041B2 (en) 2006-02-28 2011-04-13 パナソニック株式会社 Wearable terminal
US7970564B2 (en) 2006-05-02 2011-06-28 Qualcomm Incorporated Enhancement techniques for blind source separation (BSS)
US8068619B2 (en) 2006-05-09 2011-11-29 Fortemedia, Inc. Method and apparatus for noise suppression in a small array microphone system
US7761106B2 (en) 2006-05-11 2010-07-20 Alon Konchitsky Voice coder with two microphone system and strategic microphone placement to deter obstruction for a digital communication device
US7995778B2 (en) 2006-08-04 2011-08-09 Bose Corporation Acoustic transducer array signal processing
US7773759B2 (en) * 2006-08-10 2010-08-10 Cambridge Silicon Radio, Ltd. Dual microphone noise reduction for headset application
US7706549B2 (en) 2006-09-14 2010-04-27 Fortemedia, Inc. Broadside small array microphone beamforming apparatus
US20080084831A1 (en) 2006-09-27 2008-04-10 Nortel Networks Limited Active source identification for conference calls
US8321213B2 (en) 2007-05-25 2012-11-27 Aliphcom, Inc. Acoustic voice activity detection (AVAD) for electronic systems
US8488803B2 (en) 2007-05-25 2013-07-16 Aliphcom Wind suppression/replacement component for use with electronic systems
US8503686B2 (en) 2007-05-25 2013-08-06 Aliphcom Vibration sensor and acoustic voice activity detection system (VADS) for use with electronic systems
WO2008157421A1 (en) 2007-06-13 2008-12-24 Aliphcom, Inc. Dual omnidirectional microphone array
WO2009003180A1 (en) 2007-06-27 2008-12-31 Aliphcom, Inc. Microphone array with rear venting
US20090154726A1 (en) 2007-08-22 2009-06-18 Step Labs Inc. System and Method for Noise Activity Detection
US7912020B2 (en) 2007-09-21 2011-03-22 Motorola Mobility, Inc. Methods and devices for dynamic mobile conferencing with automatic pairing
US8954324B2 (en) 2007-09-28 2015-02-10 Qualcomm Incorporated Multiple microphone voice activity detector
US8175291B2 (en) 2007-12-19 2012-05-08 Qualcomm Incorporated Systems, methods, and apparatus for multi-microphone based speech enhancement
US9094764B2 (en) 2008-04-02 2015-07-28 Plantronics, Inc. Voice activity detection with capacitive touch sense
US8457328B2 (en) 2008-04-22 2013-06-04 Nokia Corporation Method, apparatus and computer program product for utilizing spatial information for audio signal enhancement in a distributed network environment
US8321214B2 (en) 2008-06-02 2012-11-27 Qualcomm Incorporated Systems, methods, and apparatus for multichannel signal amplitude balancing
US8731211B2 (en) 2008-06-13 2014-05-20 Aliphcom Calibrated dual omnidirectional microphone array (DOMA)
US8699721B2 (en) 2008-06-13 2014-04-15 Aliphcom Calibrating a dual omnidirectional microphone array (DOMA)
EP2297727B1 (en) 2008-06-30 2016-05-11 Dolby Laboratories Licensing Corporation Multi-microphone voice activity detector
US8218751B2 (en) 2008-09-29 2012-07-10 Avaya Inc. Method and apparatus for identifying and eliminating the source of background noise in multi-party teleconferences
US11627413B2 (en) 2012-11-05 2023-04-11 Jawbone Innovations, Llc Acoustic voice activity detection (AVAD) for electronic systems
AU2009308442A1 (en) 2008-10-24 2010-04-29 Aliphcom, Inc. Acoustic Voice Activity Detection (AVAD) for electronic systems
JP5789199B2 (en) 2009-02-25 2015-10-07 ヴァレンセル,インコーポレイテッド Headset and earbud
CN203086710U (en) 2009-06-29 2013-07-24 艾利佛有限公司 Dual omnidirectional microphone array calibration system
WO2011140110A1 (en) 2010-05-03 2011-11-10 Aliphcom, Inc. Wind suppression/replacement component for use with electronic systems
CN203435060U (en) 2010-07-15 2014-02-12 艾利佛有限公司 Telephone system and telephony gateway for wireless conference call
US20120239469A1 (en) 2011-03-15 2012-09-20 Videodeals.com S.A. System and method for marketing
US20140126743A1 (en) 2012-11-05 2014-05-08 Aliphcom, Inc. Acoustic voice activity detection (avad) for electronic systems

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002007151A2 (en) * 2000-07-19 2002-01-24 Aliphcom Method and apparatus for removing noise from speech signals
US20030040908A1 (en) * 2001-02-12 2003-02-27 Fortemedia, Inc. Noise suppression for speech signal in an automobile
WO2004056298A1 (en) * 2001-11-21 2004-07-08 Aliphcom Method and apparatus for removing noise from electronic signals

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
HOLZRICHTER J F ET AL: "Speech articulator and user gesture measurements using micropower, interferometric em-sensors", IMTC 2001. PROCEEDINGS OF THE 18TH. IEEE INSTRUMENTATION AND MEASUREMENT TECHNOLOGY CONFERENCE. BUDAPEST, HUNGARY, MAY 21 - 23, 2001, IEEE INSTRUMENTATION AND MEASUREMENT TECHNOLOGY CONFERENCE. (IMTC):, NEW YORK, NY : IEEE, US, vol. VOL. 1 OF 3. CONF. 18, 21 May 2001 (2001-05-21), pages 1942 - 1946, XP010547289, ISBN: 0-7803-6646-8 *
NG L C ET AL: "Denoising of human speech using combined acoustic and EM sensor signal processing", 2000 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING. PROCEEDINGS. (ICASSP). ISTANBUL, TURKEY, JUNE 5-9, 2000, IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), NEW YORK, NY : IEEE, US, vol. VOL. 1 OF 6, 5 June 2000 (2000-06-05), pages 229 - 232, XP010507310, ISBN: 0-7803-6294-2 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2196988A1 (en) * 2008-12-12 2010-06-16 Harman/Becker Automotive Systems GmbH Determination of the coherence of audio signals
US8238575B2 (en) 2008-12-12 2012-08-07 Nuance Communications, Inc. Determination of the coherence of audio signals
EP2621150A1 (en) * 2012-01-30 2013-07-31 Research In Motion Limited Adjusted noise suppression and voice activity detection
US8831686B2 (en) 2012-01-30 2014-09-09 Blackberry Limited Adjusted noise suppression and voice activity detection
EP2736041A1 (en) * 2012-11-21 2014-05-28 Harman International Industries Canada, Ltd. System to selectively modify audio effect parameters of vocal signals
CN112435683A (en) * 2020-07-30 2021-03-02 珠海市杰理科技股份有限公司 Adaptive noise estimation and voice noise reduction method based on T-S fuzzy neural network
CN112435683B (en) * 2020-07-30 2023-12-01 珠海市杰理科技股份有限公司 Adaptive noise estimation and voice noise reduction method based on T-S fuzzy neural network

Also Published As

Publication number Publication date
US20120059648A1 (en) 2012-03-08
US9196261B2 (en) 2015-11-24
TW200514456A (en) 2005-04-16
US20040133421A1 (en) 2004-07-08
TWI281354B (en) 2007-05-11
US20160155434A1 (en) 2016-06-02
US8019091B2 (en) 2011-09-13

Similar Documents

Publication Publication Date Title
US8019091B2 (en) Voice activity detector (VAD) -based multiple-microphone acoustic noise suppression
US20030179888A1 (en) Voice activity detection (VAD) devices and methods for use with noise suppression systems
US20020039425A1 (en) Method and apparatus for removing noise from electronic signals
US9165567B2 (en) Systems, methods, and apparatus for speech feature detection
EP2643981B1 (en) A device comprising a plurality of audio sensors and a method of operating the same
CN103026733B (en) For the system of multi-microphone regioselectivity process, method, equipment and computer-readable media
WO2003096031A9 (en) Voice activity detection (vad) devices and methods for use with noise suppression systems
CN104246877A (en) Systems and methods for audio signal processing
CN102893331A (en) Methods, apparatus, and computer - readable media for processing of speech signals using head -mounted microphone pair
Kalgaonkar et al. Ultrasonic doppler sensor for voice activity detection
KR100936093B1 (en) Method and apparatus for removing noise from electronic signals
US9659574B2 (en) Signal noise attenuation
US20030128848A1 (en) Method and apparatus for removing noise from electronic signals
CA2465552A1 (en) Method and apparatus for removing noise from electronic signals
CN115691473A (en) Voice endpoint detection method and device and storage medium

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BW BY BZ CA CH CN CO CR CU CZ DK DM DZ EC EE EG ES FI GB GD GE GM HR HU ID IL IN IS JP KE KG KP KZ LC LK LR LS LT LU LV MA MD MK MN MW MX MZ NA NI NO NZ PG PH PL PT RO RU SC SD SE SG SK SY TJ TM TN TR TT TZ UA UG US UZ VN YU ZA ZM

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SZ TZ UG ZM ZW AM AZ BY KG MD RU TJ TM AT BE BG CH CY DE DK EE ES FI FR GB GR HU IE IT MC NL PL PT RO SE SI SK TR BF CF CG CI CM GA GN GQ GW ML MR SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 69(1) EPC

122 Ep: pct application non-entry in european phase