CA2306095A1

CA2306095A1 - Voice recognition unit for audience measurement system

Info

Publication number: CA2306095A1
Application number: CA002306095A
Authority: CA
Inventors: Daozheng Lu; David H. Harkness; R. Gordon Price
Original assignee: Nielsen Media Research, Inc.; Daozheng Lu; David H. Harkness; R. Gordon Price
Current assignee: TNC US Holdings Inc
Priority date: 1997-11-20
Filing date: 1998-06-09
Publication date: 1999-06-03
Also published as: AU7833898A; EP1032998A1; WO1999027668A1; JP2001524776A; CN1282470A

Abstract

A member of an audience of a program to which a receiver is tuned is identified by a system which includes first and second microphones, a noise cancellation unit, and a voice recognition unit. One of the first and second microphones picks up relatively more of a voice signal from the member of the audience, and the other of the first and second microphones picks up relatively more of a sound signal from the receiver. The noise cancellation unit substantially eliminates the sound signal from the voice signal. The voice recognition unit identifies the member from the member's voice signal.
The first and second microphones, the noise cancellation unit, and the voice recognition unit may be housed in a hand held housing or in a table top housing. The system may also include a passive people meter to passively identify the member, and a program to identify the program or channel to which the receiver is tuned.

Description

VOICE RECOGNITION UNIT
FOR AUDIENCE MEASUREMENT SYSTEM
technical Field of the Invention The present invention relates generally to an audience measurement system that identifies members of an audience of a program to which a receiver, such as a television, radio, or computer, is tuned, and/or that identifies a program to which a receiver, such as a television, radio, or computer, is tuned.
background of the Invention Measuring audiences of programs has long been of interest to program suppliers and advertisers because audience measurements provide the data from which the effectiveness of programs and advertisements may be eval-IS uated. A variety of well known methods have been em-ployed in order to provide an estimate of the total audi-ence to a program, to a portion of a program, and/or to a commercial. In addition to the total audience, these methods provide additional estimates of demographically significant audience segments (e.g., the number of men and/or women between certain ages who watched a selected portion of a given program). These methods involve manu-ally and/or automatically measuring the viewing and/or listening habits of audience members of statistically selected households and then estimating the total audi-ence based on these measurements. Audience members of statistically selected households are usually referred to as panelists.
The measurement of the viewing and/or listening habits of an audience generally requires three separate determinations: (1) a determination of the channels or stations to which the monitored receivers are tuned with-in the statistically selected households; (2) a determi-nation of the programs which were available on the tuned channels at the times during which the receivers were tuned to these channels or stations; and, (3) a determi-nation of the household members and visitors who were present in the area of the receivers during the time that the receivers were tuned to selected programs.
All three determinations can affect the accu-racy of audience measurements. Of the three determina-tions discussed above, determination (3) perhaps most

- 2 -affects the accuracy of audience measurements because the members of an audience of a receiver are usually required to cooperate in the identification process. When members are required to participate in the identification pro-cess, they frequently make errors, or they simply forget to identify either themselves or the other members of the audience. For example, one known system which allows audience members to identify themselves in order to indi-cate their presence is a "People Meter." The "People Meter" may be a remote control and/or a plurality of push buttons activated by the audience members when they enter or exit the monitored area. However, audience members frequently forget to press any button at all, or they sometimes make errors in pressing the right button.
In an attempt to overcome this problem, several measurement systems have been developed. For example, in U.S. Patent No. 3,056,135, Currey et al. describe a sys-tem which utilizes strategically placed switches for counting the number of persons entering, leaving, and re-maining within a monitored area, and a photographic re-corder for periodically recording the composition of the

- 3 -audience. This approach requires that the photographic record be viewed by an operator, which both invades the privacy of the audience members, and imposes an unaccept-able cost on the measurement operation.
These problems led to a variety of suggestions for passive, non-obtrusive methods of counting (but not identifying) viewers, and of tracking their movement about the monitored area. For example, Kiewit, et al., in U.S. Patent No. 4,644,509, describe an ultrasonic sonar system which counts and tracks members of the audi-ence. However, the described system does not identify the members of the audience and, therefore, requires the members of the audience to manually identify themselves.
Other systems, such as that described by Kiewit in U.S. Patent 4,930,011, rely upon electronically active tags which are worn or carried by members of an audience and which uniquely identify corresponding ones of these audience members. These systems respond to the tags whenever the audience members wearing the tags are within the monitored area of a receiver. However, audience

- 4 -members often forget to wear their tags, in which case such audience members are not identified.
Lu, in U.S. Patent No. 4,858,000, and in U.S.
Patent No. 5,031,228, discloses an image recognition system for identifying the members of an audience of a receiver. The system uses a video camera to acquire images of the faces of the audience members, and a com-puter subsystem to recognize the faces in these images by comparing these faces to reference faces stored in a data base. The system also locates and tracks viewers, and includes infrared illuminators that provide a control-lable level of imperceptible illumination of the moni-toyed area for the video camera. However, the system is generally expensive and difficult to install.
The system disclosed by Lu et al., in U.S.
Patent No. 5,550,928 also uses image recognition to iden-tify audience members. This system incorporates algo-rithms for finding bodies and then heads of members of an audience. Once the head is found, a face finding algo-rithm extracts facial image representations that are provided to two face recognition routines, an Eigenface

- 5 -recognition routine and a template matching routine.
This system can identify audience members who are moving about the monitored area and/or who are not facing the cameras. However, it is still complex and expensive.
Determinations (1) and (2) discussed above also can affect the accuracy of audience measurements because one of these determinations alone is normally not suffi-cient to identify the programs to which receivers are tuned. That is, a determination of the channels to which a receiver in a statistically selected household is tuned does not lead to a determination of the tuned programs without also knowing what programs were available on the tuned channels. By the same token, a determination of the programs that were available on the tuned channels does not lead to a determination of the tuned programs without knowing the channels to which the receiver in a statistically selected household is tuned. Thus, both determinations are typically required in order to deter-mine the programs tuned by the audience members.
Acquisition of information about available programs and entering this information into a database for later

- 6 -WO 99!17668 pCTIUS98/I2034 matching with the tuned channels leads to acquisition and data entry errors that adversely affect the accuracy of audience measurements.
Thomas et al., in U.S. Patent No. 5,481,294, addresses these problems with an audience measurement system that detects and records ancillary codes which are embedded in, and identify, the programs to which receiv-ers are tuned. Thus, only one determination needs to be made if an ancillary code is present in the viewed pro-grams. If ancillary codes are not available in the tuned programs, program signatures are extracted from the pro-grams and are compared to reference signatures previously extracted.
The metering apparatus disclosed in this patent may include non-intrusive sensors, such as microphones near the receiver, that detect the ancillary codes in the audio output of the monitored receivers. If microphones are used to detect the ancillary codes, second micro-phones may be provided to pick up background noise. The outputs of the ancillary code sensing microphones and the noise sensing microphones are processed in order to sub-tract the background noise from the signals provided by the ancillary code sensing microphones.
This audience measurement system may also in-clude a "People Meter" as described above for additional backup.
Turner et al., in U.S. Patent No. 4,907,079, disclose a system for monitoring audiences of receivers.
The members of the audience provide control commands (such as channel selections) to a microprocessor through an infrared unit which includes a keyboard. Information collected by the microprocessor may be stored and trans-mitted to a central computer over conventional phone lines. A motion detector coupled to the microprocessor detects when an audience member has entered or exited the monitored area. A character generator coupled to the microprocessor allows communication with the audience members via alphanumeric messages displayed, for example, on a television screen. Also, a speech output unit cou-pled to the microprocessor generates speech signals to communicate to the audience members.
_ g _ The Turner et al. system also includes a speech input unit coupled to the microprocessor to allow audi-ence members to audibly identify themselves. Audible speech from the audience members is detectable by a mi-crophone, is amplified, is digitized, and is provided to the microprocessor. The microprocessor performs a power density spectrum analysis on the signal to derive voice signatures of the particular audience members. These signatures are compared to reference signatures, which were previously stored during initialization of the sys-tem, in order to identify viewers. The microprocessor may also normalize the digital voice data and remove small amplitude samples in order to minimize the effects of background noise. When the system disclosed in this Turner et al. patent is activated, an initial display message is placed on the viewing screen which requests the audience to speak their first and last names in order to provide the reference signatures. Voice recognition by itself does not, however, permit a viable audience measurement system. Moreover, the disclosed signal pro-_ g _ AttOrIley ~?OCKet 28Q49j33953 cessing that minimises the effects of background noise may not be adequate under many conditions.
Kimura, in U.S. Patent No. 5,267,323, disc~.ases a voice operated remote control having a tranamit~ex 10A.
The transmitter l0A includes first and second microphones M3. and M2. The (first micrvphQne M9. serves to input a voice oommaz~d and the second microphone M2 serves to pick up ambient sound or noise. A noise xemover 104 removes the ambiez~.t noise by producing a different~.al signal indicative of the difference between the a~.gnaZs from the t~,rst a,~id eecor~.d microphones M1 and M2.
The present invent~.on is directed to a system which identifies members of an audience 4f a receiver and which salves one or more t~f the probxems rioted above.
Summary of he Tnve~ '-n ~ ion Tn accordance urith a first aspeot o~ the pres-ent invention, a system for identifying a metrtber of an audience of a reoexver comprises first and second micxo-phones, a noise cancellation unit, and a voice recogni-_ ~.0 _ C~'~
~f~~c~0 P

Attorney Docket ~B~Q9~33953 t~.on unit. The first and second microphones axe arxan~ed so that ore of the First anal second Microphones picks up rplati~rely rnvre of a voice szgr~al from the member of the GudienCe and the other of the first and sacdnd micro-phones picks up relatively more of a sound Bigna~. from the receiver. The noise cancellation unit i~3 Gaug7.ed to tr~e first and seco:~d microphones, arid the noise cancel.la-Lion u,n.~.t is arranged to substantially elim~.nate the sound sigrs.a~. ~rom the voice signal. The 'mice xecogr~i-tion .nit is coup3.ed to the anise canceilativh unit, and - x0a -C~O~,O~~

the voice recognition unit is arranged to identify the voice signal from the member of the audience.
In accordance with another aspect of the pres-ent invention, a system comprises a microphone, a voice recognition unit, and a program sensor. The system iden-tifies a member of an audience of a program to which a receiver is tuned. The microphone is located to receive a voice signal from the member. The voice recognition unit is coupled to the microphone and is arranged to identify the member from the voice signal. The program sensor is arranged to identify the program.
In accordance with yet another aspect of the present invention, a system comprises a passive people meter, a microphone, a voice recognition unit, and a con-troller. The system identifies a member of an audience of a program to which a receiver is tuned. The passive people meter is arranged to provide a first identifica-tion of the member by passively identifying the member.
The microphone is located to receive a voice signal from the member. The voice recognition unit is coupled to the microphone and is arranged to provide a second identifi-cation of the member by identifying the member from the voice signal. The controller is arranged to select at least one of the first and second identifications in order to identify the member.
In accordance with still another aspect of the present invention, an audience recognition unit comprises a housing, a microphone supported by the housing, and a voice recognition unit supported by the housing and cou-pled to the microphone. The voice recognition unit is arranged to recognize a member of an audience from an identifier spoken by the member. Accordingly, the audi-ence recognition unit identifies the member of the audi-ence of a program to which a receiver is tuned.

These and other features and advantages of the present invention will become more apparent from a de-tailed consideration of the invention when taken in con-junction with the drawings in which:
Figure 1 illustrates the system of the present invention for use in identifying members of an audience of a program to which a television receiver is tuned;
Figure 2 is a block diagram of an embodiment of the system which is in accordance with the present inven-tion and which includes an audience recognition unit (ARU?;
Figure 3 is a more detailed diagram of the audience recognition unit shown in Figure 2;
Figure 4 is a diagram of a noise cancellation arrangement for use in the audience recognition unit shown in Figure 2;
Figure 5 illustrates a hand held version of the audience recognition unit shown in Figure 3;
Figure 6 illustrates a table top version of the audience recognition unit shown in Figure 3;

Figure 7 is a flow chart depicting a program which may be implemented by the audience recognition unit shown in Figure 3 during a training period; and, Figure 8 is a flow chart depicting a program which may be implemented by the audience recognition unit shown in Figure 3 during voice recognition of audience members.
An audience measurement system 10, which deter-mines the members of an audience to a program to which a monitored receiver is tuned, is shown in Figure 1. The audience measurement system 10 includes an audience recognition unit 12, which is shown in Figure 1 as a hand held unit but which alternatively may be any other type of unit, such as a table top unit. The audience recog-nition unit 12 permits the members of the audience to be identified either passively or actively, and may or may not interact with a receiver 14 during the identification process.

A program sensor 16 cooperates with the re-ceiver 14 in order to determine the programs (and/or channels) to which the receiver 14 is tuned. The program sensor 16 may be any known sensor which determines chan-nets (and/or programs). For example, the program sensor 16 may sense ancillary codes which are embedded in the program signals to which the receiver 14 is tuned and which specifically identify the tuned programs. In addi-tion, or alternatively, the program sensor 16 may include a signature extractor which extracts characterizing sig-natures from the program signals to which the receiver 14 is tuned so that the signatures may be compared to refer-ence signatures in order to identify the programs (and/or channels) to which the receiver 14 is tuned. In addi-tion, or alternatively, the program sensor 16 may be an apparatus which determines the channels to which the receiver 14 is tuned by monitoring the tuner of the re-ceiver 14.
The audience measurement unit 12 and the pro-gram sensor 16 communicate with a base unit 18 which may be either a site unit or a home unit. As is known in the WO 99!27668 PCTIUS98112034 art, a site unit may be provided at each audience area within a household, and all site units within the house-hold may communicate to a home unit over the electrical power lines in the walls of the household. The home unit may then communicate by way of the public telephone sys-tem with a central unit which gathers tuning and audience information from other households so that the information can be combined into reports useful to program suppliers, advertisers, and the like. In households having only one viewing area, the base unit 18 may be arranged to both receive data from the audience measurement unit 12 and the program sensor 16, and may communicate this data by way of the public telephone system to the central unit.
Thus, as shown in Figure 2, the audience recognition unit 12 provides its recognition data to the base unit 18 by way of a communication link 20 which, for example, may be an infrared or RF communication link.
The program sensor 16 provides its program identification data to the base unit 18 by way of a communication link 21 which, for example, may be a hard wire, but which may alternatively be any other communication link, such as an infrared or RF communication link. The base unit 18 collects the audience recognition data from the audience recognition unit 12 and the program identification data from the program sensor 16, and may communicate this data over a public telephone system 22 to a central site 24 which accumulates similar data from other households.
As indicated in Figure 2, the audience measure-ment system 10 may also include a passive people meter 26. For example, the passive people meter 26 may be of the type which passively identifies audience members in a monitored area in accordance with the teachings of U.S.
Patent No. 5,550,928 described above. The passive people meter 26 provides its people identification data to the base unit 18 by way of a communication link 28 which, for example, may be a hard wire, but which may alternatively be any other communication link, such as an infrared or RF communication link.
Accordingly, if the audience measurement system 10 includes both the audience recognition unit 12 and the passive people meter 26, the passive people meter 26 may be used to passively identify audience members in the event that the audience recognition unit 12 is unable to do so. Alternatively, the audience recognition unit 12 may be used to actively or passively identify audience members in the event that the passive people meter 26 is unable to passively identify audience members. As a still further alternative, the outputs of both the audi-ence recognition unit 12 and the passive people meter 26 may be applied simultaneously to a soft decision logic that identifies audience members based upon both of these outputs. In any of these alternatives, confidence is in-creased that audience members can be accurately recog-nized.
The audience recognition unit 12 is shown in more detail in Figure 3. The audience recognition unit 12 includes first and second microphones 30 and 32. The first and second microphones 30 and 32 are positioned in the audience recognition unit 12 so that one of the first and second microphones 30 and 32 tends to pick up rela-tively more of the voice signals from the audience mem-bers, and the other of the first and second microphones and 32 tends to pick up relatively more of the audio output from the monitored receiver. For example, the first and second microphones 30 and 32 may be separated so that, in most orientations of the audience recognition unit 12, one of the first and second microphones 30 and 32 is closer to the audience members and the other of the first and second microphones 30 and 32 is closer to the monitored receiver. A preferred separation between the first and second microphones 30 and 32 is 6.72 inches, although any other suitable separation may be used. In this embodiment of the present invention, the first and second microphones 30 and 32 may be pointed in opposite directions. Alternatively, one or both of the first and second microphones 30 and 32 may be located remotely from the audience recognition unit 12 so that one of the first and second microphones 30 and 32 is closer to the audi-ence members and the other of the first and second micro-phones 30 and 32 is closer to the monitored receiver.
The analog output of the first microphone 30 is amplified by an amplifier 34, and this amplified analog output is converted to a digital signal by an analog to digital converter 36. Similarly, the analog output of WO 99/27668 pCT/US98/12034 the second microphone 32 is amplified by an amplifier 38, and this amplified analog output is converted to a digi-tal signal by an analog to digital converter 40.
Even though the first and second microphones 30 and 32 can be positioned or directed so that one of the first and second microphones 30 and 32 tends to pick up relatively more of the voice signals from the audience members and the other of the first and second microphones 30 and 32 tends to pick up relatively more of the audio output from the monitored receiver, the first and second microphones 30 and 32 pick up both the voice signals from the audience members and the audio output from the re-ceiver 14. Also, both the first and second microphones 30 and 32 pick up background noise other than the audio output from the receiver 14. In order to reduce or elim-mate the effects of background noise and the audio out-put from the receiver 14, the outputs from the analog to digital converters 36 and 40 are provided to a noise cancellation arrangement 42. The noise cancellation arrangement 42 processes the outputs from the analog to digital converters 36 and 40 so as to cancel the audio output of the receiver 14 as well as background noise from the voice signals provided by the audience members, so that these voice signals may be more easily identi-fied.
The noise cancellation arrangement 42 may use signal subtraction in order to subtract background noise and the audio output of the receiver 14 from the voice signals provided by the audience members. For example, if the first microphone 30 is directed toward the audi-ence members, and if the second microphone 32 is directed toward the receiver 14, the signal from the first micro-phone 30 may be designated SP (p meaning people), and the signal from the second microphone 32 may be designated as Sr (r meaning reference). In general, the signal Sp may be given according to the following equation:
Sp = aVp + bVr ( 1 ) where SP is the signal from the first microphone 30, Vp is the contribution to the signal from the first microphone 30 due to the voice signals of the members of the audi-ence, Vr is the contribution to the signal from the first microphone 30 due to the audio output of the receiver 14 and other background noise, a and b are parameters dis-cussed below, and a is greater than b.
Similarly, the signal Sr may be given according to the following equation:
Sr = cVp + dVr ( 2 ) where Sr is the signal from the second microphone 32, c and d are parameters discussed below, and c is less than d. Equations (1) and (2) may be subtracted with a weighting parameter a according to the following equa-tion:
D = eSP - Sr = (ae - c) ~ p + (be - d} ~V (3) where D is the weighted difference between equations (1) and (2), and a is a weighting parameter discussed below.
If a, b, c, and d are fixed parameters deter-mined by the environment and the features of the first and second microphones 30 and 32, the weighting parameter a can be selected so that (be - d) - 0, or so that a =
d/b > 1. Accordingly, if (be - d) - 0, the difference D
will consist only of the signal Vp. In this case, the signal VP from equation (3) is given by the following equation:
D
p ae - c' Because a, e, and c are constants in a given household, these constants may be treated as a single constant k ac-cording to the following equation:
k ael c (5) so that the signal Vp is given by following equation:
VP = kD. (() The weighting parameter a may be determined adaptively during periods when the receiver 14 is on but no member of the audience is talking. During such a period, the audience recognition unit 12 can automati-cally find the weighting parameter a so that D = 0. This weighting parameter a is then inserted into equations (5) and (6) in order to determine VP, and VP is processed by a voice recognition unit 44 as discussed below.
This noise cancellation approach assumes that the first microphone 30 is directed to, or is closer to, the audience than is the second microphone 32, and that the second microphone 32 is directed to, or is closer to, the receiver 14 than is the first microphone 30. For example, if the audience recognition unit 12 is a hand held unit, the first microphone 30 and the second micro-phone 32 may be mounted on the audience recognition unit 12 so that they are separated and so that the user is WO 99127668 PC'T/US98112034 directed to orient the audience recognition unit 12 with the first microphone 30 directed toward the audience and the second microphone 32 directed toward the receiver 14.
Similarly, if the audience recognition unit 12 is a table top unit, the first microphone 30 and the second micro-phone 32 again may be mounted on the audience recognition unit 12 so that they are separated and so that the user is directed to orient the audience recognition unit 12 with the first microphone 30 directed toward the audience and the second microphone 32 directed toward the receiver 14. Alternatively, whether the audience recognition unit 12 is a hand held unit or a table top unit, the first and second microphones 30 and 32 may be connected to the audience recognition unit 12 by lines or other communica-tion media which permit the first microphone 30 to be mounted in close proximity to the audience and the second microphone 32 to be mounted in close proximity to the receiver 14.
Instead of canceling noise as discussed above, the noise cancellation arrangement 42 may use phase cancellation. It is noted that sound travels at approxi-mately 1,120 ft per second in air. Sounds not originat-ing equidistant from both of the first and second micro-phones 30 and 32, which are preferably separated by 6.72 inches, reach these microphones at different times and are canceled or are at least partially canceled. Based on the actual frequencies of the sounds, and the spacing between the first and second microphones 30 and 32, dif-ferent phase angles result. The noise cancellation ar-rangement 42 implementing phase cancellation uses these different phase angles in order to cancel the audio out-put from the receiver 14 and other background noise from voice signals provided by the members of the audience.
In this embodiment of the present invention, the first and second microphones 30 and 32 are preferably direc-tional microphones mounted on the same side of the audi-ence recognition unit 12. The audience recognition unit 12 may have any orientation with respect to the audience and the receiver 14 as long as the first and second mi-crophones 30 and 32 and not pointed directly at the re-ceiver 14.

In this embodiment of the present invention, the noise cancellation arrangement 42 may be of the form shown in Figure 4. According to this embodiment, the noise cancellation arrangement 42 includes an amplifier 60, which amplifies the signal from the first microphone 30, and an amplifier 62, which amplifies the signal from the second microphone 32. Because the amplifiers 60 and 62 may be essentially the same, only one of the amplifi-ers is shown in detail in Figure 4. The outputs of the amplifiers 60 and 62 are summed by a summer 64 so that background noise is substantially eliminated. The output from the summer 64 is detected and converted to a DC
level by a detector 66 and is supplied to a timer 68.
If the output of the summer 64 is above a predetermined threshold, the timer 68 is triggered and provides an output to a switch 70 in order to connect the output of the summer 64 to the voice recognition unit 44.
The voice recognition unit 44 provides a hold signal through a terminal 72 in order to hold the timer 68 in its triggered state until a predetermined time after the voice recognition unit 44 last recognizes a voice.

The voice recognition unit 44 may be an RSC164 voice recognition processor provided by Sensory Inc. The voice recognition unit 12 compares the output from the noise cancellation arrangement 42 to prestored reference voice signals (identifiers) stored in a memory. These prestored reference voice signals (identifiers) may be spoken identifier words or voice features from a speaker-'s frequency spectrum. When matches are found, members of the audience are identified, and the voice recognition unit 44 provides a suitable identification record to a controller 46 for each match. The controller 46 stores the identification record in a memory 48 for later transmission to the base unit 18 by way of a communica-tion arrangement 50. The communication arrangement 50 may use the communication link 20 as previously de-scribed. The controller 46 may also be coupled to a clock 52 so that the times that the members of the audi-ence are identified may also be stored in the memory 48.
A prompt unit 54 may also be coupled to the controller 46. The controller 46 controls the prompt unit 54 in order to prompt audience members to either provide their corresponding voice signals to the first and second microphones 30 and 32, or use a keypad 56 in order to manually identify themselves. The prompt unit 54 may be a voice synthesizer and/or a graphic or video display (such as an LCD or LED) of the audience recog-nition unit 12 arranged to provide audible and/or visible prompts to the members of the audience. Alternatively, the prompt unit 54 may be the speakers and/or the screen of the receiver 14 arranged to provide audible and/or visible prompts to the members of the audience in re-sponse to interaction with the audience recognition unit 12.
The audience recognition unit 12 may be implemented as a hand held unit 90 which is illustrated in Figure 5. The hand held unit 90 includes a housing 92 that houses the elements of the audience recognition unit 12 which are shown in Figure 3. The housing 92 supports an LED display 94 which may perform the functions of the prompt unit 54. As noted above, however, the prompt unit 54 alternatively may be a voice synthesizer, in which case the housing 92 would incorporate an output speaker to project audible voice prompts from the voice synthe-sizer to the members of the audience. The LED display 94 (and/or the voice synthesizer) may perform functions other than prompting. The housing 92 also houses an IR
transmitter 96 for establishing the communication link 20 shown in Figure 2. Alternatively, the communication link 20 may be established by an ultrasonic or RF link, a hard wire, or the like. The housing 92 further supports a keypad 98, which may be the keypad 56 shown in Figure 3, for allowing audience members to enter data. For exam-ple, predetermined ones of the keys in the keypad 98 may be assigned to members and visitors of the audience so that they may identify themselves manually if they cannot be recognized through their voice signals. The housing 92 may contain a rechargeable battery which is arranged to provide power to the audience recognition unit 12 and which may be charged overnight or when the audience rec-ognition unit 12 is not in use. The hand held unit 90 also has a power on/off button 95 in order to turn the hand held unit 90 on and off, and an LED 97 in order to indicate that the hand held unit 90 is ready to identify audience members and visitors.
Alternatively, the audience recognition unit 12 may be implemented as a table top unit 100 which is illustrated in Figure 6. The table top unit 100 includes a housing 102 that houses the elements of the audience recognition unit 12 which are shown in Figure 3. The housing 102 may support an LED display (not shown) in order to implement the functions of the prompt unit 54.
As noted above, however, the prompt unit 54 alternatively may be a voice synthesizer, in which case the housing 102 would incorporate an output speaker to project audible voice prompts from the voice synthesizer to the members of the audience. The LED display or the voice synthe-sizer may perform functions other than prompting. The housing 102 may also house an IR or RF transmitter (not shown) for establishing the communication link 20 shown in Figure 2. Alternatively, the communication link 20 may be established by an ultrasonic link, a hard wire, or the like. The housing 102 supports a keypad 104, which may be the keypad 56 shown in Figure 3, for allowing audience members to enter data. For example, predeter-mined ones of the keys in the keypad 104 may be assigned to members of the audience so that they may identify themselves manually if they cannot be recognized through their voice signals. The table top unit 100 may contain a rechargeable battery which is arranged to provide power to the audience recognition unit 12 and which may be charged overnight or when not in use. Alternatively, the table top unit 100 may have a power cord for providing power to the audience recognition unit 12 from an elec-trical outlet. Also, the table top unit 100 may have a power on indicator light 106 and a voice detected indica-for light 108.
During a training mode, the controller 46 of the audience recognition unit 12 may implement a program 120 depicted in Figure 7 by way of a flow chart. The program 120 is executed during the training mode in order to store identifier words or voice features associated with the members of the audience. During training, an audience member should assume a position at a specified distance from the audience recognition unit 12. This audience member initiates training by pressing an appropriate key or button on the keypad 56 of the audi-ence recognition unit 12.
When the training mode is initiated so that the program 120 is executed, the audience recognition unit 12, at a block 122, determines whether a predetermined amount of time has passed since the last audience member activity. If the audience recognition unit 12 determines that there has been no audience member activity for the predetermined amount of time, the program 120 assumes that the training mode has ended and the program 120 exits. If the predetermined amount of time has not pass-ed since the last audience member activity, the audience recognition unit 12, at a block 124, continues to wait for a wakeup command to be spoken. The wakeup command should be a unique word or word combination not normally present in the audio of the receiver I4. The audience member should speak the wakeup command in a normal voice to the first and second microphones 30 and 32. When the audience recognition unit 12 detects the wakeup command, the audience recognition unit 12, at a block 126, prompts the audience member to say the identifier ward which the viewer has chosen to identify himself or herself. Such an identifier word may be the name of the viewer.
The audience recognition unit 12 then deter-mines at a block 128 whether the audience member has said the identifier. If not, the audience recognition unit 12, at the block 126, again prompts the audience member to say the identifier word. If the audience recognition unit 12 determines at the block 128 that the audience member has said the identifier word, the audience recog-nition unit 12, at a block 130, suitably processes the identifier word and stores the identifier word in the memory 48. Flow returns to the block 122 where the training mode is terminated or another audience member enters a corresponding identifier word.
During an identification mode, the controller 46 of the audience recognition unit 12 may implement a program 140 depicted in Figure 8 by way of a flow chart.
The program 140 is executed during the identification mode in order to identify members of the audience. The identification mode may be initiated by pressing an appropriate key or button on the keypad 56 of the audi-ence recognition unit 12.
When the identification mode is initiated so that the program 140 is executed, the audience recogni-tion unit 12, at a block 142, determines whether an audi-ence member desires to end the identification mode by pressing an appropriate key or button on the keypad 56 of the audience recognition unit 12. If the audience recog-nition unit 12 determines that the audience member de-sires to end the identification mode, the program 140 exits. If the audience recognition unit 12 determines that the audience member does not desire to end the iden-tification mode, the audience recognition unit 12, at a block 144, waits for a wakeup command to be spoken. The audience member should speak the wakeup command in a normal voice to the first and second microphones 30 and 32. When the audience recognition unit 12 detects the wakeup command, the audience recognition unit 12, at a block 146, prompts the audience member to say his or her identifier word.

WO 99/2?668 PCT/US98/12034 The audience recognition unit 12 then deter-mines at a block 148 whether the audience member has said the identifier word. If not, the audience recognition unit 12, at the block 146, continues to prompt the audi-ence member to say the identifier word. If the audience recognition unit 12 determines at the block 148 that the audience member has said the identifier word, the audi-ence recognition unit 12, at a block 150, instructs the voice recognition unit 44 to compare the identifier word to the vocabulary of identifier words stored in the mem-ory 48.
If the audience recognition unit 12 determines at a block 152 that the voice recognition unit 44 detects a match between the spoken identifier word and one of the identifier words stored in the memory 48, the audience recognition unit 12, at a block 154, stores the spoken identifier word and the time of the match as an identification record in the memory 48 for immediate or later communication to the base unit 18. (Alternatively, the audience recognition unit 12, at the block 154, may communicate the spoken identifier word and time directly to the base unit 18.) If the audience recognition unit 12 determines at the block 152 that the voice recognition unit 44 cannot match the spoken identifier word to one of the identifier words stored in the memory 48, the audi-ence recognition unit 12, at a block 156, prompts the audience member to try again or to use another farm of identification. For example, the prompt 140 may prompt the audience member to speak his or her identifier word a predetermined number of times before prompting the audi-ence member to use another form of identification (such as entering the identifier word through use of the keypad 56). Alternatively, or additionally, if the controller 46 determines that the voice recognition unit 44 cannot detect a match at the block 152, the controller 46 at the block 156 may send a message to the base unit 18 indi-Gating that identification should be made from the pas-sive people meter 26. After the audience recognition unit 12, at the block 154, stores the spoken identifier word and the time of the match as an identification re-cord in the memory 48, or after the audience recognition unit 12, at the block 156, prompts the audience member to try again, or to use another form of identification, or to send a suitable message to the base unit 18 to seek an identification from the passive people meter 26, or the like, flow returns to the block 142.
The advantage of using a wakeup command during normal voice recognition is that the comparison process is simplified. That is, during normal audience recogni-tion, each spoken identifier word is compared to every reference identifier word stored in the memory 48. How-ever, the wakeup command only needs to be compared to one reference in the memory 48.
The block 126 of Figure 7 and the block 146 of Figure 8 may be arranged to also prompt each audience member to speak a cue in addition to the member's identi-fier word. The advantage of coupling a cue to an identi-fier word is to minimize the possibility that the audi-ence recognition unit 12 may incorrectly identify a per-son as an audience member when that person's identifier word is contained in the audio output of the receiver 14, but that person is not in the audience. For example, the identifier word may be a name, and the cue may be a word common to all audience members, such as the word "enter,"
or the word "exit," or the like. Thus, a member of the audience, when entering (or leaving) the monitored area, speaks his or her name, as the identifier word, coupled with the cue. For example, the member may say "John enter" or "Mary enter." Alternatively, the cue may be unique to each audience member. Either arrangement reduces the likelihood that a word audibly emitted by the receiver 14 will be mistaken as an identifier word spoken by a member of the audience.
Certain modifications of the present invention have been discussed above. Other modifications will occur to those practicing in the art of the present invention. For example, the invention has been described above in connection with measuring the audiences of tele-vision programs. However, the present invention is use-ful also in measuring the audiences of radio programs and programs of other media.
Also, the program sensor 16 is described above as sensing the program and/or channel to which the re-ceiver 14 is tuned. Instead, the audience recognition unit 12 may be arranged to accept manual entry of program identifications by the audience members through use of the keypad 56.
Moreover, the functions of a remote control may be incorporated into the audience recognition unit 12.
Furthermore, instead of relying upon two micro-phones, such as the first and second microphones 30 and 32, the audience recognition unit 12 may rely upon a single, close range directional microphone. If so, noise cancellation may not be necessary.
In addition, the use of a wakeup command may be eliminated.
Also, as described above, the passive people meter 26 may be of the type which passively identifies audience members in a monitored area in accordance with the teachings of U.S. Patent No. 5,550,928 described above. However, the passive people meter 26 may be any known type of passive device that is capable of passively identifying members of an audience. For example, the passive people meter 26 may use an ultrasonic and/or infrared transducer or an array of such transducers in order to capture an image of the audience area and may use processing algorithms in order to recognize members in the audience.
Moreover, the passive people meter 26, as des-cribed above, passively identifies audience members in a monitored area. If the passive people meter 26 cannot passively identify the members in the audience, it can be arranged to count the members of the audience and to prompt manual identification of the members until the IO number of manually identified members equals the count.
Alternatively, if the passive people meter 26 cannot passively identify every member in the audience, it can be arranged to count the members of the audience and to prompt manual identification of those members which it cannot passively identify. As a still further alterna-tine, a counter may be used instead of the passive people meter 26. The counter can comprise an ultrasonic and/or infrared transducer or an array of such transducers or other device or devices which can detect people in the audience and can count the number of detected people.
Then, the audience measurement system 10 could be ar-WO 99/27668 pCT/US98/12034 ranged to prompt manual identification until the number of people manually identified equals the count.
Furthermore, the audience measurement unit 12, the program sensor 16, and the passive people meter 26 are shown in Figure 2 as being separate elements. In-stead, the audience measurement unit 12 may include the program sensor 16 and/or the passive people meter 26.
Accordingly, the description of the present invention is to be construed as illustrative only and is for the purpose of teaching those skilled in the art the best mode of carrying out the invention. The details may be varied substantially without departing from the spirit of the invention, and the exclusive use of all modifica-tions which are within the scope of the appended claims is reserved.

Claims

WHAT IS CLAIMED IS:

1. A system (10) for identifying a member of an audience of a receiver (14) having a first microphone (30) and a voice recognition unit (44), wherein the first microphone (30) picks up a voice signal from the member of the audience, and wherein the voice recognition unit (44) is arranged to identify the member of the audience from the voice signal, the system BEING CHARACTERIZED IN
THAT:
the first microphone (30) is arranged to pick up relatively more of the voice signal from the member of the audience;
a second microphone (32) is arranged to pick up relatively more at a sound signal from the receiver (14);
and, a noise cancellation unit (42) is coupled to the first and second microphones (30 and 32), wherein the noise cancellation unit (42) is arranged to substantially eliminate the sound signal from the voice signal based upon a phase relationship between the sound signal and the voice signal so as identify the member of the audience from the voice signal.

2. The system of claim 1 further comprising a program sensor (16) arranged to identity a program received by the receiver (14).

3. The system of claim 2 wherein the program sensor (16) comprises a channel sensor.

4. The system of claim 2 wherein the program sensor (16) comprises a code reader arranged to detect a program identifying code in a program to which the receiver is tuned.

5. The system of claim 2 wherein the program sensor (16) comprises a signature extractor arranged to extract uniquely identifiable signatures from a program to which the receiver is tuned.

6. The system of claim 2 wherein the program sensor (16) comprises a keypad arranged to permit the member of the audience to key in a program or channel identity.

7. The system of claim 1 further comprising a clock (52) and a memory (48), wherein the memory (48) stores an identity of the member of the audience based upon the voice signal from the member, and wherein the memory (48) stores a time from the clock (52) when the member is identified.

8. The system of claim 1 wherein the receiver (14) is a television.

9. The system of claim 1 wherein the receiver (14) is a radio.

10. The system of claim 1 wherein the receiver (14) is a multimedia reception device.

11. The system of claim 1 further comprising a keypad (56) arranged to accept an identification input from the member of the audience.

12. The system of claim 1 further comprising a prompting device (54) arranged to prompt the member of the audience to say the voice signal.

13. The system of claim 12 further comprising a sensor (16) to identify a program or channel received by the receiver.

14. The system of claim 12 further comprising a passive people meter (26) arranged to passively identify the member of the audience.

15. The system of claim 14 further comprising a sensor (15) to identify a program or channel received by the receiver.

16. The system of claim 1 wherein the voice signal comprises an identifier of the member of the audience.

17. The system of claim 2 wherein the voice signal comprises an identifier and a cue of the member of the audience.

18. The system of claim 1 further comprising a passive people meter (26) arranged to passively identify the member of the audience.

19. The system of claim 1 further comprising a communication unit (50) arranged to communicate the identity of the member and/or program identity to a remote point.

20. The system of claim 19 wherein the communication unit (50) is arranged to communicate the identity of the member to the remote point by way of infrared signals.

21. The system of claim 7, wherein the first and second microphones (30 and 32), the noise cancellation unit (42), and the voice recognition unit (44) are housed in a hand held housing (92).

22. The system of claim 1 wherein the first and second microphones (30 and 32), the noise cancellation unit (42), and the voice recognition unit (44) are housed in a table top housing (102).

23. The system of claim 1 wherein the first and second microphones (30 and 32) are directionally oriented in substantially opposite sensing directions.

24. The system of claim 1 wherein the first microphone (30) is mounted closer to the audience than is the second microphone (32), and wherein the second microphone (32) is mounted closer to the receiver than is the first microphone (30).

25. The system of claim 1 further comprising a counter, a display, and a manual entry device, wherein the voice recognition unit is arranged to supply prompts to the display in order to prompt manual identification of members of the audience through use of the manual entry device until the members so identified equal a count supplied by the counter.

26. The system of claim 1 wherein the phase relationship is based upon the speed of sound.

27. The system of claim 26 wherein the first and second microphones are separated by a distance of substantially 6.72 inches.

28. A system (10) for identifying a member of an audience of a program to which a receiver (14) is tuned has a microphone (30) and a voice recognition unit (44), the microphone (30) is located to receive a voice signal from the member, and the voice recognition unit (44) is arranged to identify the member from the voice signal, the system BEING CHARACTERIZED IN THAT:
the voice recognition unit (44) is arranged to identify the member from the voice signal which includes an identifier and a cue for the member.

29. The system of claim 28 further comprising a memory (48) arranged to stare reference identification information corresponding to the member, and wherein the voice recognition unit (44) is arranged to identify the member based upon a comparison of the voice signal and the reference identification information.

30. The system of claim 29 wherein the feature is an identification code embedded in the program.

31. The system of claim 29 wherein the feature is an uniquely identifiable signature extracted from the program.

32. The system of claim 28 wherein the receiver (14) is a television.

33. The system of claim 28 wherein the receiver (14) is a radio.

34. The system of claim 28 wherein the receiver (14) is a multimedia reception device.

35. The system of claim 28 further comprising a keypad (56) arranged to accept an identification input from the member.

36. The system of claim 35 wherein the identification input is an identification of the member.

37. The system of claim 35 wherein the identification input is a program or channel identification.

38. The system of claim 28 further comprising a prompting device (54) arranged to prompt the member to say the voice signal.

39. The system of claim 36 further comprising a passive people meter (26) arranged to passively identify the member.

40. The system of claim 28 wherein the voice signal comprises an identifier of the member.

41. The system of claim 28 further comprising a passive people meter (26) arranged to passively identify the member.

42. The system of claim 28 further comprising a communication unit (50) arranged to communicate the identity of the member to a remote point.

43. The system of claim 42 wherein the communication unit (50) is arranged to communicate the identity of the member to the remote point by way of infrared signals.

44. The system of claim 28 wherein the microphone (30) and the voice recognition unit (44) are housed in a hand held housing (92).

45. The system of claim 28 wherein the microphone and the voice recognition unit and housed in a table top housing (102).

46. The system of claim as further comprising a counter, a display, and a manual entry device, wherein the voice recognition unit is arranged to supply prompts to the display in order to prompt manual identification of members of the audience through use of the manual entry devise until the members so identified equal a count supplied by the counter.

47. A system (10) for identifying a member of an audience of a program to which a receiver (14) is tuned having a microphone (30) located to receive a voice signal from the member and a voice recognition unit (44) arranged to provide a first identification of the member by identifying the member from the voice signal, the system BEING CHARACTERIZED IN THAT:
a passive people meter (26) is arranged to provide a second identification of the member by passively identifying the member; and, a controller (45) is arranged to select the first identification or the second identification in order to identify the member.

48. The system of claim 47 wherein the controller (46) includes a prompting unit (54) arranged to prompt the member to say the voice signal.

49. The system of claim 48 further comprising a sensor (16) arranged to identify the program or channel.

50. The system of claim 47 wherein the controller (46) includes a prompting unit (54) arranged to prompt the member to say the voice signal if the passive people meter (26) is unable to provide the first identification.

51. The system of claim 47 wherein the controller (46) is arranged to select the first identification if the voice recognition unit (44) is unable to provide the second identification.

52. The system of claim 47 wherein the voice signal comprises an identifier.

53. The system of claim 47 wherein the voice signal comprises an identifier and a cue.

54. The system of claim 47 further comprising a sensor (16) arranged to identify the program or channel.

55. The system of Claim 47 further comprising a communication unit (50) arranged to communicate the identity of the member to a remote point.

56. The system of claim 55 wherein the communication unit (50) is arranged to communicate the identity of the member to the remote point by way of infrared signals.

57. The system of claim 47 wherein the microphone (30) and the voice recognition unit (44) are housed in a hand held housing (92).

58. The system of claim 47 wherein the microphone and the voice recognition unit are housed in a table top housing (102).

59. The system of claim 47 further comprising a counter, a display, and a manual entry device, wherein the controller is arranged to supply prompts to the display in order to prompt manual identification of members of the audience through use of the manual entry device until the members so identified equal a count supplied by the counter.

60. An audience recognition unit (90 or 100) that identifies a member of an audience of a program to which a receiver is tuned having a microphone (30) and a voice recognition unit (44) arranged to recognize the member from as identifier spoken by the member, the audience recognition unit BEING CHARACTERIZED IN THAT:
a housing (92 or 102) supports the microphone (30) and the voice recognition unit (44); and, a communication unit (50) is supported by the housing (92 or 102), wherein the communication unit (50) is arranged to communicate data related to member recognition through air to a local base unit (18).

61. The audience recognition unit of claim 60 further comprising a prompt unit (54) supported by the housing (92 or 102), wherein the prompt unit (54) is arranged to prompt the member to take an action.

62. The audience recognition unit of claim 61 further comprising a keypad (56) supported by the housing (92 or 102), wherein the keypad (56) is arranged to permit the member to enter data.

63. The audience recognition unit of claim 60 further comprising a keypad (56) supported by the housing (92 or 102), where in the keypad (56) is arranged to permit the member to enter data.

64. The audience recognition unit of claim 60 further comprising a clock (52) and a memory (48) supported by the housing (92 or 102), wherein the memory (48) stores an identity of the member based upon the spoken identifier, and wherein the memory (48) stores a time from the clock (52) when the member is identified.

65. The audience recognition unit of claim 64 wherein the identifier includes a word and a cue.

66. The audience recognition unit of claim 60 wherein the identifier includes a word and a cue.

67. The audience recognition unit of claim 60 wherein the audience recognition unit is a hand held unit (90).

68. The audience recognition unit of claim 60 wherein the audience recognition unit is a table top unit (100).