WO2007008589A2 - Total timing control system for theater - Google Patents

Abstract

Theater control system to control items in the way synchronized with performance on going, including prompt to performers, caption to audience, lighting, and moving facilities. System keeps advancing internal performing time, expressed with such as measure and beat, and modifies it to fit performance on stage. Contents of control are stored with timing data expressed with performing time. Each item is queued when internal performing time comes to time of stored timing data. Both character display and voice playback are available for prompt and caption. For the prompt, multiple display devices are placed around the stage and dynamically assigned to roll players.

Description

TITLE OF INVENTION

Total timing control system for theater.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based on Provisional Application No. 60/697,747, filed on July 8, 2005. Also, part of this invention is related to an invention applied for PCT with US National Submission No. 10/583047 filed on June 15, 2006.

BACKGROUND OF THE INVENTION

This invention is related to better and also automated information and control system for theater, which is used for performances.

Theater performance such as opera, musical, ballet, concert, drama, require staffs other than performers, who work along with timing of performance. They are lighting engineer, audio mixer, stage mechanics controller, prompter, caption display controller and others. Necessary timing for these work, if expressed in physical time by hour, minute and second, vary for each performance day by day. This variance does not allow usage of scheduler or controller programmed with fixed clock time. They must follow progress on the stage with referring music score or script book. This situation makes it difficult to automate the control and ask such staffs tension to follow up score or script sometimes very precisely. Recent electronic keyboard instruments have ability of control lighting using MIDI code. But, usual acoustic instruments have no direct connection to control systems. Following music by means of sound analysis is difficult or unreliable with current state of art.

This invention aims to automate or semi-automate all these timing related tasks.

Caption in this invention means display of translated lines for audience, who are not fluent enough with language sung or spoken in performance. Timing of display is critical and operated by staff that can follow score or script. This invention offers timing control of caption.

Musicians of course must follow up music. They must use their brain sometimes only to count rest bars, which may reaches hundreds bars. Also, to display music on the electronic display, system must provide an input tool for turning over the pages, or preferably system follows up the music to determine the timing of turning over the pages. Timing may be different for each instrument because density of notes differs. The inventor proposed to turn page gradually automatically or semi-automatically in US National submission No. 10/583047.

Prompter teaches singers lines or words to sing a little before the singers sing them from the small box in front of the stage. Prompt helps singers a lot, because singers sometimes must sings huge volume of lines in a full opera in non-mother language. Current style of prompt are requested clearness and perfect timing. So, only limited number of prompters can do it. This invention offers different and automated way of prompt.

BRIEF SUMMARY OF THE INVENTION

The objective of this invention is to centralize the function of following up music or performance, so that many performers and staffs get free from following up music or performance. The objective includes to provide electronic display of music or lines of words and then to eliminate the use of paper music score or lines of words. Related objective of this invention is to provide better way of prompt. Further objective of this invention is to provide multi channel interpretation to audience.

In accordance to this invention, total system includes as many computers as necessary, each of which is accessed from musicians, performers, staffs, or audience through electronic display to see music or lines of words or through earphone to listen to digitally stored voices. All these computers are connected by network. One of connected computer called timing master, which carries the function of following up music or performance, delivers real time timing information to all these computers through the network.

To follow up music with this invention, hierarchical multi means are used. These means are, from low priority to high priority, playback of recorded timing, sound analysis, conductor's tact detection, and direct tapping input. If follow up goes well without tapping input, system is automatic. But, human assistance is necessary until we get robust technique for these automatic following up. Tapping input is operated by a dedicated operator or an alternative orchestra player. With playback of recorded timing, it is not so busy task, because only correction of recorded timing and machine follow up is required. Further more, the invention gives the tool, with which tapping input at only check point marked in the score for this purpose work well. Timing of caption display for audience is pre-designed and stored in caption control computer associated with music timing, expressed by bar and beat numbers. Also, segmented and digitally stored voce of interpretation of lines of words are played back at the pre-designed music timing. Real time music timing used for either caption display or interpretation voice is given from the timing master.

With one of realization of the invention, prompt to the performer is given through computer display with scrolling lines of words. Best place for the display is on the wall of orchestra pit facing to the stage and one side or both sides of the conductor. Because performer looks at the display also can see the conductor, which is necessary. The displays can be placed at plural points. These are easy to see from various position and posture of performers. Scrolling is synchronized with musical timing supplied from the timing master. Lines of words are displayed with assigned color for each roll. Display shows not only words but also timing or rest count.

With another realization of the invention, prompt to the performer is given through earphones, which are connected wireless to computer, which reproduces digitally recorded and stored voices at the timing specified with music timing also stored in the computer.

Those earphones used for prompt and interpretation are designed as both voices sent from computers and sound picked at the ear position are mixed, so that listeners feel stereo attendance.

Control for lighting and stage mechanics can be programmed using musical timing provided from timing master.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is an example of network structure of present invention;

Fig. 2 is a flow schematic of program modules and data in the timing master computer;

Fig. 3 is a view from stage showing places of displays for prompt;

Fig. 4 is an example of contents of prompt display;

Fig. 5 is an electronic schematic of earphone device used in present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBOD YMENT Fig. 1 shows computer network in a preferred embodiment of the theater information system. There is one main LAN (abbreviation of Local Area Network) 1 and many sub LANs 2—13, which are used for each own functionality. Each sub LAN is connected to the main LAN through computer 2a ~ 13 a, works as bridge between LANs. These bridge computers have their specific function.

Bridge computer 2a at sub LAN 2 is named as Timing Master. Only this Timing Master in the whole system carries out the function of following up music or performance. The function was task done by each performers and stuffs. Parts of the function may be carried by computers 2b or 2c, which are connected to sub LAN 2. An operator may attend at console of Timing Computer 2a. The operator assists conductor to manage rehearsal or accident at performance, and has audio-visual connection even places are far from each other. In case operator is not available, one of orchestra players operates using foot pedals as he plays his part. So, sub LAN 2 is connected to few computers 4b for orchestra players.

Timing Master 2a delivers music time to other bridge computers and then bridge computers deliver it to computers connected their sub LANs. Different with physical time expressed by hour, minute, and second, music time is expressed by movement, bar and beat. More detailed timing is expressed with decimal number or fraction number of 1 beat. Time of such as drama, which has not music, is expressed by chapter, section, sentence and phrase numbers. These numbers have to be assigned to each part of script. So, more general term for both music time and time for drama may be performance time. But, for convenience, the words music time is used hereunder. As character of a live performance, music time varies for each performance, day by day. So, follow up of each performance is necessary.

Sub LANs 3~5 are used for showing music to orchestra players and singers. In Fig. 1, orchestra is divided in two groups corresponding to sub LAN 3 and 4. Each one or two players need a display of music. Using tablet PC, each display is part of a computer, which has connected to sub LAN. It has all function in one thin box, and can be set on a stand. Usually singers do not see music at opera performance. But, sub LAN 5 is prepared for singers to see at the rehearsal. Music score is shown on the screens using projector.

Sub LAN 6 and 7 are used for prompt purpose. Several computers for displaying prompt are placed around the stage and are connected to sub LAN 6. Also, several computers for giving voice prompt to each singer through wireless earphone are connected to sub LAN 7. Display and voice are alternative media for prompt. Singers can select their preferable media. Theater has both media available for use.

Sub LAN 8 and 9 are used to provide caption or interpretation to audience. Several computers corresponding to the languages generate character display of translated lines of words, and they are connected to sub LAN 8. Also, several computers corresponding to the languages reproduce digitally recorded voice or synthesize voice, and they are connected to sub LAN 9.

Sub LAN 10 is connected to group of computers for lighting control. Lighting control during performance is programmed using music time. Tasks at the console are automated and change from direct operation to observation. Music score or script is displayed for each stuff.

Sub LAN 11 is connected to group of computers for stage control, including operation of lifts, platforms, revolving stage, curtains and such those. Sound mixing is included in this group. Control during performance is programmed using music time. Tasks at the console are automated and change from direct operation to observation. Music score or script is displayed for each stuff.

Sub LAN 12 is connected to group of computers at attendants' station and shops in lobby and dressing rooms and security office. They show music time and expected time left to take necessary actions.

Sub LAN 13 is connected to group of workstations for preparation and check of music, prompt and interpretation. Voice recording and editing, music score scanning and recognition are included. Work may be done keeping synchronized to the music time or independently.

Conductor's console computer 14 is connected to main LAN 1. It has double touch panel display for display of two pages of music score. Music display is gradually updated according to delivered music time. Touch panel is used for specifying restarting point or place related to comment at rehearsal. These operations are sent to Timing Master. Timing Master stops music time then deliver replaced music time. These operation may be done by an assistant operator following conductor's command at Timing Master 2a, which has the same touch panel displays.

Archive Library computer 15 connected to main LAN 1 stores all data about each music work, which has been edited and compiled at preparation workstations 13b or updated at Timing Master 2a. When work to be performed changes, all data necessary at each computer are provided from this computer.

Fig. 2 shows construction of Timing Master 2a. It is a computer in the embodiment. So, it shows program modules and data and connection between them. Program modules are drawn with rectangular and data are drawn with rhombus.

Before music starts, data are read from data storage 20 and set to each places. Music structure 21 is description of structure of the music work in time scale. It is sequence of pair of rhythm and noted tempo and bar numbers that the pair continues. Also, repeats and notation of moving tempo are included. Follow up means mapping from this noted music structure to time intervals of beat actually performed.

Stored timing 22 is mapped data just mentioned. It is a sequence of interval time between consecutive beats. At first, data is filled at workstations with values calculated from noted tempo. For example, if tempo is 120, value is 0.5 second. Also, notation with words such as Allegro, Andante and Largo are interpreted to numerical value generally used for them.

Music notes 23 are series of every note to be played in the sequence of music time. It is a computerized expression of music.

Check points data 24 is newly introduced idea of this invention. It is a series of music time, which is chosen as turning point of tempo, and is marked in the music display. Candidates of check point are such as points where tempo notation are given and points few bars later than these points where tempo are established, or points where soloist plays in free tempo. Check points may be chosen at workstations 13b.

Operation control program 25 controls start and stop of music time within this computer and also all other computers on LAN 1. It has a console of touch panel display and keyboard to give graphical user interface with an operator, who assists the conductor. The operator specifies music works to be performed or rehearsed so that each data is set up ready to use. In rehearse, operator specify starting point on the score, which is transposed to internal expression of music time and provided to music structure 21 as a reading pointer, also, provided to other computers on LAN 1.

In case operator is absent, conductor can operate on his terminal 14, and through LAN 1 and LAN interface 27 interact with operation control program 25.

Once operation control program 25 sets up starting point and gives start timing, until it gives stop command or music reaches to the end, timer control program 28 sets computer timer with value of the current beat length, which is read from stored timing data 22. When computer timer set off, timer control program 25 sends beat signal to priority handling program 29, and sets computer timer again with the value of next beat length. Priority handling program 29 then updates current music time 30 along with physical time it happened, and call distribution program 31 to send out music time to other computers on LAN 1. Thus with only timer control program, current music time can be updated according to stored timing 22.

Block 32 shows sound analysis, one of music follow up function. Microphone 33 picks up sound and audio input program34 performs analog-digital conversion and buffering of data. Sound analysis program 35 detects start timing and tone of music sound. Matching program 36 starts when sound analysis program sends detected note. Nearest a few notes detected are bundled and matched with notes from music notes data 23 around current music time, with standpoint of interval between notes and height of notes. If matching gets reasonable match, it send out pair of sound start time and music time in data 23 about the last sound in the bundle. If music time is just on beat, priority-handling program 29 handles the event as follows. First, it replaces physical time, kept with current music time 30, by detected value. If timer has not set off, timer is cancelled and is set to next beat length. If timer has set off, timer is reset to beat length again. In both case, delay of detection at sound analysis program 32 is compensated in value for timer setting.

Sound analysis block 32 may be processed in separate computers 2b on LAN 2, because it requires big computing power. There may be plural such computers corresponding to instrument sections or on stage singers to get more accurate matching result. Even this block is processed outside, output link to priority program 29 is realized same way.

Block 37 shows tact detection, another follow up function. There were inventions of how to detect conductor's tact. In this invention, it is a tool to accomplish main objective, and any feasible method can be used here. To explain, video analysis method is described. Video camera 38 picks up motion video image of conductor. And video input program 39 digitizes and buffer it in memory. Then image analysis program 40 detects position of the end of tact. Then beat position detection program 41 detects rebound point with its physical time happened and make a guess about music time. Detected data is sent to priority handling program 29. Priority program handle the event same way as detection by sound analysis block 32.

Also, video image analysis is heavy data processing. Tact detection block 37 may be processed in separate computer 2c on LAN 2.

Both sound analysis block 32 and tact detection block 37 are option. It can be added to the system when robust detection program becomes available.

Tap input is the final reliable robust input means. Operator can use ordinal computer mouse 42, foot switch, or some device like these. Any of these input devices can be connected to mouse input program 43, and then to tap input program 44. There are two modes in input operation, detail mode and macro mode.

Pushing and keep pushing right button of mouse 42 activates detail mode. In this mode, first left button ON action is recorded with physical time, and matched with nearest beat either in past and future. Matched music time and physical time of button ON are used to update current music time 30. And after that every new tap input and its physical time are used to update current music time 30 as time of next beat.

During right button is off, it is in macro mode. Left button input in macro mode further classified in two cases. If there is checkpoint within one bar length, ON input is accepted as timing of the checkpoint, and physical time and music time registered with checkpoint are sent to priority handling program 29. If there is no checkpoint around the timing, on input is accepted as timing of first beat of the bar nearest the timing. It is used, when there are gradual difference between music time of inside computer and actual performance, to correct inside music time and get synchronized.

Priority program 29 update current music time 30 and call distribution program 31. Further it calls stored timing update program 45, when there is tap input. In detail mode, it updates one by one stored timing data23. In macro mode, stored timing update program 45 calculate the ratio of physical time period to summation of beat length of stored timing data22, between now and the nearest past check point. Then the program update data of stored timing data 22 between said two points as well as data between current music time and next check point, to the value multiplied by the ratio calculated. Thus macro mode tap input gets effect to compensate slight difference of tempo around there. When operator does not attend at console 26, designated orchestra player operates two foot pedals taking place of the operator. Input signals come through LAN 2.

Thus, timing master 2a gives updated music time to other computers. These computers on sub LAN 3-5 receive music time and use it to update each music display. It is more than automatic turning page. Using electronic display enables to update music pages gradually, so that music stems corresponding to current playing position, that is music time, and parts front and back with certain range are always displayed on the screen. This means top part of next pages is displayed early enough before playing position reaches the end of page. Also, bottom part of former page remains a little while, when playing position goes top of page. It is not like turn over page, which update whole page at once. This gives players enough time and room to prepare for playing the new page.

In the embodiment of Fig. 1, sub LAN 3 is used for string instruments and sub LAN 4 is used for wind and percussion instruments. Usual number of computers on each LAN is from 20 to 40. It is possible to divide or join sub LAN according to the speed of sub LAN. Music information for each display, whish are different with instruments, are distributed before performance or rehearsal to each computer.

Computer for orchestra player assigned to operation of tap input displays his part and piano score of music, which includes checkpoint marks, and connects to two foot pedals. This computer has two LAN connections, sub LAN 2 and sub LAN 4.

Vocal music displays at sub LAN 5 is used for rehearsal. These are large format display set at the audience seats.

This invention gives two new way of prompt to the performer. With first way, prompt is given through computer display with scrolling lines of words. Scrolling is synchronized with musical timing supplied from the timing master 2. Fig. 3 shows places to set displays for this purpose. It is a view from stage 51 towards audience seats 52. Usual theater has frame 53 and orchestra pit 54. Because orchestra pit is lower than audience floor, there should be a wall 54a. Best place for the prompt display is on the wall of orchestra pit facing to the stage and one side or both sides 58a and 58b of the conductor 55. Because performer looks at the display also can see the conductor 55, whom they must watch frequently, in same time. The displays can be placed at plural points, such as on the same wall of orchestra pit but away from the conductor 59a and 59b, or as shown as 61a, 61b and 61c, placed side by side with foot lights 57 in case there is no orchestra pit, as shown as 60a and 60b, near doors 56a and 56b at wings, or as shown as 62a and 62b back side of the frame. These are easy to see when performers are at various positions not in center of stage and in various postures. Each display may be assigned to performers time by time.

Background color is black, so that display light does not interfere in dark scenes. Fig. 4 shows samples of display contents. 63a is displayed at 58a and 63b is displayed at 58b. Display shows not only words but also timing or rest count. 64a, 64b, 64c and 64d are names of rolls, and 65a, 65b, 65c and 65d are prompt of each roll, and scrolled from right to left. Each line colored by specific color assigned to the roll all the time, even they move between displays or disappears, so that each roll player can find them easily. Lines 66a and 66b show current timing, and stay at same position. 67 show bar marks. 68 show bar numbers left until words appear, and 69 show beat numbers until start of words. Contents of lines of words, notations along with bar numbers are input at workstations 13b. Also assignment of displays to be used for each bars are done at the workstations 13b.

With another realization of the invention, prompt to the performers are given through earphones, which are connected wireless to computers 7b. Each computer can reproduces a few channels of digitally recorded and stored voice segments at the timing specified with music timing stored along with voices. One method of recording prompt voices and timing data is do it at rehearsal with wireless connection between professional prompter and performers.

Projection display device is used for caption. Screen may set above the frame 53. Also, small personal display set at each seat, as seen in cabin of airplane, can be used for multi language translation. Sentences are stored with music time to show. At performance, next stored music time is compared with current music time delivered. When they matched, sentences are displayed. These processes are carried in computers on sub LAN 8.

Other way of giving interpretation is using earphone. Voice signal may be delivered through wireless or wired way. In case wireless, theater rents out devices to audience. When theater has earphone jacks at audience seats wired connection is used. User can tune to preferable channel of language. Voice signal of sentences are digitally recorded and stored with music time. And reproduced at specified music time.

Fig. 5 shows schematic diagram of special earphone used for interpretation for audience and prompt for performer. Receiver box 71 has power switch 72, channel change buttons 73 and volume change buttons 74. Control circuit transforms these inputs to control signals to tuner 76 and demodulation circuit 77. Earphone piece 78 has a microphone 79, which picks up the outer sound, and mixing am 80, which mixes microphone signal and voice signal from receiver box. Output of mixing amp drive speaker 81, so that outer sound can be heard at the same level. User can listen music in natural stere sound and also interpretation or prompt voices at desired level.

Timing of prompt voices should be very precise. So, timing is specified in decimal numbers. Ther is some amount of delay between actual beat timing and timing of corresponding current music tim 30 detected and generated in Timing Master 2a reaches to reproducing program in computer 7b. BuI it comes with physical time of the beat; so physical time corresponding specified music time fo prompt can be estimated using interpolation at computer 7b.

Claims

CLAIMSWhat is claimed is:

1. Theater control system to control items in the way synchronized with performance, which has: memory means to hold series of control contents or information segments to be presented to the participants, each of control contents or information segments is associated to the performing time, first means to keep internal performing time advancing and to adjust it to on-stage performing time, second means to initiate each control action including presentation of said information segment, contents of which is stored in said memory means, when the internal performing time comes to said associated performing time, where performing time is expressed by unit in musical notation such as measure and beat in music-leading performance or unit in script such as line, sentence and phrase number in drama like performance.

2. The theater control system as claimed in claim 1, wherein said information to participants includes prompt for performers.

3. The theater control system as claimed in claim 2, wherein prompt is displayed with electronic display device.

4. The theater control system as claimed in claim 3, wherein plural display devices are placed around the stage.

5. The theater control system as claimed in claim 4, wherein places of display include wall of orchestra pit facing to the stage, and at least one device is placed near side of the conductor.

6. The theater control system as claimed in claim 4, wherein display is assigned to rolls dynamically according to positions or lines of sight of roll players.

7. The theater control system as claimed in claim 2, wherein prompt is displayed in assigned color to each roll.

8. The theater control system as claimed in claim 2, wherein prompt for each roll is presented with voice, segment by segment, stored in said memory means.

9. The theater control system as claimed in claim 8, wherein voice is delivered through wireless earphone, which mixes sound of outside the ear to voice signal to maintain stereo presence.

10. The theater control system as claimed in claim 1, wherein said information to participants includes caption to audience.

11. The theater control system as claimed in claim 10, wherein caption is displayed with electronic display device.

12. The theater control system as claimed in claim 11, wherein captions are presented in multiple languages.

13. The theater control system as claimed in claim 10, wherein caption is presented with voice.

14. The theater control system as claimed in claim 13, wherein multiple channels of voice are delivered and each audience is able to select preferred channel.

15. The theater control system as claimed in claim 13, wherein special earphone is used, which mixes sound of outside the ear to voice signal to maintain stereo presence.

16. The theater control system as claimed in claim 1, wherein control items include but not limited to lighting control, up down of curtains, lifts, and rotation of revolving stage.