US20060176832A1 - Adaptive bit-rate adjustment of multimedia communications channels using transport control protocol - Google Patents
Adaptive bit-rate adjustment of multimedia communications channels using transport control protocol Download PDFInfo
- Publication number
- US20060176832A1 US20060176832A1 US11/051,674 US5167405A US2006176832A1 US 20060176832 A1 US20060176832 A1 US 20060176832A1 US 5167405 A US5167405 A US 5167405A US 2006176832 A1 US2006176832 A1 US 2006176832A1
- Authority
- US
- United States
- Prior art keywords
- estimate
- tcp packets
- data
- receiver
- received
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L1/00—Arrangements for detecting or preventing errors in the information received
- H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
- H04L1/0002—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the transmission rate
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
- H04L65/1101—Session protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2402—Monitoring of the downstream path of the transmission network, e.g. bandwidth available
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44209—Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/637—Control signals issued by the client directed to the server or network components
- H04N21/6377—Control signals issued by the client directed to the server or network components directed to server
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
- H04N21/6582—Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Definitions
- the present invention relates generally to bit rate control for digital data transmission. More particularly, the present invention relates to adaptive bit-rate adjustment of multimedia communications channels using transport control protocol (TCP).
- TCP transport control protocol
- Multimedia communications channels such as those used in Internet videoconferencing generally employ the user datagram protocol (UDP) to transport packets of video data. Because UDP does not support the retransmission of lost packets, it is well-suited to real-time data transmission. The delay required for the retransmission of a lost packet in a real-time multimedia communications channel would produce a noticeable fault at the receiver such as frozen video and clicks in the audio.
- UDP user datagram protocol
- UDP is a connectionless protocol, and so presents a network security issue. Many businesses will not permit UDP connections to traverse their corporate firewalls, and so cannot use UDP videoconferencing systems.
- TCP transmission control protocol
- TCP retransmits lost packets, and so is generally not well-suited for real-time multimedia communications.
- TCP also provides network congestion control by effectively changing the bit rate of the communications channel, lowering the bit rate of each channel on a congested network connection to allow all of the channels to share the network connection. This congestion control can adversely affect multimedia communications. For example, if a videoconferencing application is transmitting at a bit rate greater than that permitted by TCP congestion control, a growing transmission lag will result. If the difference in bit rates is 10%, then at the end of a one-hour videoconference the lag will be 6 minutes, hardly real-time.
- the invention features a videoconferencing apparatus comprising a video codec to generate video data at a video bit rate; a video packetizer to produce Transport Control Protocol (TCP) packets of the video data; an audio codec to generate audio data at an audio bit rate; an audio packetizer to produce packets of the audio data; a transmit circuit to transmit the TCP packets of the video data and the packets of the audio data; a receive circuit to receive a Real-time Transport Control Protocol (RTCP) receiver report representing a number of the TCP packets of the video data received by a receiver of the TCP packets of the video data; and a controller to generate an estimate of a number of the TCP packets of the video data transmitted by the transmit circuit but not yet received by the receiver based on a difference between a number of the TCP packets of the video data transmitted by the transmit circuit and the number of the TCP packets of the video data received by the receiver, and control the video bit rate according to the estimate.
- TCP Transport Control Protocol
- the invention features an apparatus and corresponding method and computer program.
- the apparatus comprises a data generator to generate data at a bit rate; a packetizer to produce Transport Control Protocol (TCP) packets of the data; a transmit circuit to transmit the TCP packets of the data; and a controller to control the bit rate according to an estimate of a number of the TCP packets of the data transmitted by the transmit circuit but not yet received by a receiver of the TCP packets of the data.
- TCP Transport Control Protocol
- the controller generates the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver based on a number of the TCP packets of the data transmitted by the transmit circuit following an initialization event, a number of the TCP packets of the data received by the receiver following the initialization event, and a previous estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver.
- the controller determines a plurality of differences, each at a different time, between a number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and a number of the TCP packets of the data received by the receiver following the initialization event; and generates the estimate according to at least one of the group consisting of a median of the differences, a mean of the differences, and a mode of the differences.
- the controller decreases the bit rate when a first predetermined number of consecutive differences between the number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and the number of the TCP packets of the data received by the receiver following the initialization event fall above a threshold that is a first function of the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver; increases the bit rate when a second predetermined number of consecutive differences between the number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and the number of the TCP packets of the data received by the receiver following the initialization event fall below a threshold that is a second function of the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver; and increases the bit rate when the bit rate has not been increased for a predetermined interval.
- the controller further generates a new estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver; determines whether the new estimate falls outside an estimate window surrounding the estimate; controls the bit rate based on the new estimate when the new estimate falls outside the estimate window surrounding the estimate; and controls the bit rate based on the estimate when the new estimate falls inside the estimate window surrounding the estimate.
- the apparatus further comprises a receive circuit to receive a packet of data representing the number of the TCP packets of the data received by the receiver.
- the packet of data representing the number of the TCP packets of the data received by the receiver comprises a Real-time Transport Control Protocol (RTCP) receiver report packet.
- the data generator comprises at least one of the groups consisting of a video codec to encode video data; and an audio codec to encode audio data.
- a videoconferencing system comprises the apparatus.
- FIG. 1 shows a videoconferencing system in communication with a network such as the Internet.
- FIG. 2 shows an adaptive bit-rate control process for the videoconferencing system of FIG. 1 according to a preferred embodiment of the present invention.
- FIG. 3 shows a process for controlling the bit rate of the video data based on an estimate of the number of the TCP video packets currently in transit.
- Embodiments of the present invention provide adaptive bit-rate adjustment of multimedia communications channels using transport control protocol (TCP).
- TCP transport control protocol
- the techniques disclosed herein are especially useful in real-time two-way applications such as videoconferencing and voice-over-IP telephony, but are also applicable to one-way communications channels, and to communications channels with less stringent latency requirements.
- FIG. 1 shows a videoconferencing system 100 in communication with a network 102 such as the Internet.
- a network 102 such as the Internet.
- Videoconferencing system 100 comprises a video source 104 that provides a video signal, for example from a videoconferencing camera, a video codec 106 to encode the video signal as video data, and a video packetizer 108 to produce TCP packets of the video data.
- Videoconferencing system 100 further comprises an audio source 110 that provides an audio signal, for example from a microphone, an audio codec 112 to encode the audio signal as audio data, and an audio packetizer 114 to produce TCP packets of the audio data.
- Videoconferencing system 100 further comprises one or more transmit circuits 116 such as Ethernet ports to transmit the video and audio TCP packets, one or more receive circuits 118 to receive data and control packets from network 102 , and a controller 120 to control videoconferencing system 100 .
- FIG. 2 shows an adaptive bit-rate control process 200 for videoconferencing system 100 according to a preferred embodiment of the present invention. While process 200 is described with respect to controlling a video data bit rate, it is equally applicable to controlling an audio data bit rate or bit rates for any sort of digital data generated by data generators such as audio and video sources, codecs, and the like. Preferably controller 120 executes two instances of process 200 concurrently. One instance of process 200 controls the video data bit rate while the other instance controls the audio data bit rate.
- a source such as video source 104 , produces a data signal, e.g., a video signal (step 202 ).
- a codec such as video codec 106 , encodes the video to produce video data at a video bit rate (step 204 ).
- a packetizer such as video packetizer 108 , produces TCP packets of the video data (step 206 ). Transmit circuit 116 transmits the TCP packets of video data (step 208 ).
- Controller 120 estimates the number of the TCP packets of video data in transit, that is, the number of the TCP packets of video data transmitted by transmit circuit 116 but not yet received by the receiver of the packets such as another videoconferencing system (step 210 ). An estimate is used because it is not possible to know the exact number of the TCP video packets currently in transit. Controller 120 controls the bit rate of the video data according to the estimate of the number of the TCP packets of video data in transit (step 212 ).
- FIG. 3 shows a process 300 for controlling the bit rate of the video data based on an estimate of the number of the TCP video packets currently in transit.
- Controller 120 determines a difference DIFF between a number of the TCP packets of the video data transmitted by transmit circuit 116 and a number of the TCP packets of the video data received by the receiver (step 302 ), preferably over a predetermined interval.
- the number of the TCP packets of the (video) data received by the receiver is obtained from the receiver, preferably as a Real-time Transport Control Protocol (RTCP) receiver report packet sent by the receiver and received by receive circuit 118 .
- the number of the TCP packets of the video data transmitted by videoconferencing system 100 is obtained from videoconferencing system 100 .
- the RTCP reporting interval is two seconds, and the numbers of packets are counted starting with an initialization event, such as the start of the current videoconferencing session.
- Controller 120 also estimates the number D of transmitted packets of the video data that are in transit over network 102 (step 304 ).
- the estimate D is calculated as the median of the previous 50 values of DIFF, although a different number of values of DIFF can be used, and instead of the median, the mean, the mode or some other function of the values of DIFF can be used.
- DIFF Downlink Preferably the first value of DIFF is used until 7 values of DIFF have been calculated. Then the median of all of the values of DIFF is used until 50 values of DIFF have been calculated. Thereafter the sliding window of 50 values of DIFF is used, as described above.
- the initial video bit rate is preferably initially limited based on the size S of the average packet of video data transmitted by videoconferencing system 100 .
- K can be used.
- Process 300 benefits from the stability of the value of D. Therefore, in a preferred embodiment, when a new value of D is calculated, it is compared to the previous value of D. If the new value of D falls inside an estimate window surrounding the previous value of D, then the new value of D is discarded, and the previous value of D is used.
- the estimate window is D ⁇ one standard deviation of DIFF.
- the standard deviation of DIFF is computed as the median absolute deviation of the previous 50 values of DIFF, although other computation methods can be used.
- Process 300 estimates the standard deviation SDev of the packets of video data in transit (step 306 ).
- the standard deviation SDev is computed as the median absolute deviation of the previous 50 values of DIFF, although other computation methods can be used. However, upon initialization an insufficient number of values of DIFF are available.
- the standard deviation SDev is computed as the average of the highest and lowest values of DIFF until 7 samples of DIFF have been received, although other computation methods can be used. Thereafter the standard deviation SDev is computed as described above.
- Process 300 then controls the bit rate of the video data according to the values of DIFF and D.
- process 300 decreases the bit rate when M consecutive values of DIFF fall above a threshold that is a function of D and increases the bit rate when N consecutive values of DIFF fall below a threshold that is a function of D.
- Multiple thresholds can be used, as described in detail below.
- Process 300 maintains a counter I for each threshold. For four thresholds, process 300 maintains counters I 1 , I 2 , I 3 , and I 4 . Process 300 also preferably maintains a counter I 5 to count the number of receiver reports for which no video bit rate adjustments are made.
- process 300 After changing the video bit rate, and before making another estimate of the number of TCP packets of video data in transit, process 300 waits for a predetermined interval, preferably by skipping 2 RTCP receiver reports (step 318 ). Process 300 also resets all of the counters I 1 , I 2 , I 3 , I 4 , and I 5 after changing the video bit rate (step 318 ). Process 300 then resumes at step 302 .
- step 308 DIFF ⁇ D+2SDev
- counter I 1 is reset to zero (step 320 ) to ensure that counter I 1 counts only consecutive RTCP receiver reports where DIFF>D+2SDev.
- step 322 DIFF ⁇ D+SDev
- counter I 2 is reset to zero (step 328 ) to ensure that counter I 2 counts only consecutive RTCP receiver reports where DIFF>D+SDev.
- step 330 if at step 330 DIFF ⁇ D, counter I 3 is reset to zero (step 336 ) to ensure that counter I 3 counts only consecutive RTCP receiver reports where DIFF>D.
- the increase is 10%, although other values can be used.
- step 338 DIFF ⁇ D
- counter I 4 is reset to zero (step 346 ) to ensure that counter I 4 counts only consecutive RTCP receiver reports where DIFF ⁇ D.
- controller 120 increases the video bit rate.
- process 300 includes a burst detection routine to handle bursts of video data, for example when an i-frame is to be sent when the video includes significant motion.
- controller 120 halves the video bit rate, and maintains that value for 3 RTCP receiver report packets before resuming process 300 .
- the invention can be implemented in digital electronic circuitry, or in hardware, firmware, software, or in combinations of them.
- An apparatus of the invention can be implemented in a computer program product tangibly embodied in a device-readable medium, e.g., storage device, for execution by a programmable processor; and method steps of the invention can be performed by a programmable processor executing a program of instructions to perform functions of the invention by operating on input data and generating output.
- the invention can be implemented advantageously in one or more programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device.
- Each computer program can be implemented in a high-level procedural or object-oriented programming language, or in assembly or machine language if desired; and in any case, the language can be a compiled or interpreted language.
- Suitable processors include, by way of example, both general and special purpose microprocessors.
- a processor will receive instructions and data from a read-only memory and/or a random access memory.
- a computer will include one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks.
- Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM disks. Any of the foregoing can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).
- semiconductor memory devices such as EPROM, EEPROM, and flash memory devices
- magnetic disks such as internal hard disks and removable disks
- magneto-optical disks magneto-optical disks
- CD-ROM disks CD-ROM disks
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A videoconferencing apparatus having a corresponding method and instruction program comprises a video codec to generate video data; a video packetizer to produce TCP packets of the video data; an audio codec to generate audio data; an audio packetizer to produce packets of the audio data; a transmit circuit to transmit the TCP packets; a receive circuit to receive a RTCP receiver report representing a number of the TCP packets of the video data received by a receiver; and a controller to generate an estimate of a number of the TCP packets of the video data transmitted but not yet received based on a difference between a number of the TCP packets of the video data transmitted by the transmit circuit and the number of the TCP packets of the video data received by the receiver, and control the video bit rate according to the estimate.
Description
- The present invention relates generally to bit rate control for digital data transmission. More particularly, the present invention relates to adaptive bit-rate adjustment of multimedia communications channels using transport control protocol (TCP).
- Multimedia communications channels such as those used in Internet videoconferencing generally employ the user datagram protocol (UDP) to transport packets of video data. Because UDP does not support the retransmission of lost packets, it is well-suited to real-time data transmission. The delay required for the retransmission of a lost packet in a real-time multimedia communications channel would produce a noticeable fault at the receiver such as frozen video and clicks in the audio.
- However, UDP is a connectionless protocol, and so presents a network security issue. Many businesses will not permit UDP connections to traverse their corporate firewalls, and so cannot use UDP videoconferencing systems.
- However, another transport protocol is available, namely transmission control protocol (TCP). But TCP retransmits lost packets, and so is generally not well-suited for real-time multimedia communications. TCP also provides network congestion control by effectively changing the bit rate of the communications channel, lowering the bit rate of each channel on a congested network connection to allow all of the channels to share the network connection. This congestion control can adversely affect multimedia communications. For example, if a videoconferencing application is transmitting at a bit rate greater than that permitted by TCP congestion control, a growing transmission lag will result. If the difference in bit rates is 10%, then at the end of a one-hour videoconference the lag will be 6 minutes, hardly real-time.
- In general, in one aspect, the invention features a videoconferencing apparatus comprising a video codec to generate video data at a video bit rate; a video packetizer to produce Transport Control Protocol (TCP) packets of the video data; an audio codec to generate audio data at an audio bit rate; an audio packetizer to produce packets of the audio data; a transmit circuit to transmit the TCP packets of the video data and the packets of the audio data; a receive circuit to receive a Real-time Transport Control Protocol (RTCP) receiver report representing a number of the TCP packets of the video data received by a receiver of the TCP packets of the video data; and a controller to generate an estimate of a number of the TCP packets of the video data transmitted by the transmit circuit but not yet received by the receiver based on a difference between a number of the TCP packets of the video data transmitted by the transmit circuit and the number of the TCP packets of the video data received by the receiver, and control the video bit rate according to the estimate.
- In general, in another aspect, the invention features an apparatus and corresponding method and computer program. The apparatus comprises a data generator to generate data at a bit rate; a packetizer to produce Transport Control Protocol (TCP) packets of the data; a transmit circuit to transmit the TCP packets of the data; and a controller to control the bit rate according to an estimate of a number of the TCP packets of the data transmitted by the transmit circuit but not yet received by a receiver of the TCP packets of the data.
- Particular implementations can include one or more of the following features. The controller generates the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver based on a number of the TCP packets of the data transmitted by the transmit circuit following an initialization event, a number of the TCP packets of the data received by the receiver following the initialization event, and a previous estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver. To generate the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver, the controller determines a plurality of differences, each at a different time, between a number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and a number of the TCP packets of the data received by the receiver following the initialization event; and generates the estimate according to at least one of the group consisting of a median of the differences, a mean of the differences, and a mode of the differences. To control the bit rate according to the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver, the controller decreases the bit rate when a first predetermined number of consecutive differences between the number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and the number of the TCP packets of the data received by the receiver following the initialization event fall above a threshold that is a first function of the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver; increases the bit rate when a second predetermined number of consecutive differences between the number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and the number of the TCP packets of the data received by the receiver following the initialization event fall below a threshold that is a second function of the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver; and increases the bit rate when the bit rate has not been increased for a predetermined interval. To control the bit rate according to the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver, the controller further generates a new estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver; determines whether the new estimate falls outside an estimate window surrounding the estimate; controls the bit rate based on the new estimate when the new estimate falls outside the estimate window surrounding the estimate; and controls the bit rate based on the estimate when the new estimate falls inside the estimate window surrounding the estimate. The apparatus further comprises a receive circuit to receive a packet of data representing the number of the TCP packets of the data received by the receiver. The packet of data representing the number of the TCP packets of the data received by the receiver comprises a Real-time Transport Control Protocol (RTCP) receiver report packet. The data generator comprises at least one of the groups consisting of a video codec to encode video data; and an audio codec to encode audio data. A videoconferencing system comprises the apparatus.
- The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.
-
FIG. 1 shows a videoconferencing system in communication with a network such as the Internet. -
FIG. 2 shows an adaptive bit-rate control process for the videoconferencing system ofFIG. 1 according to a preferred embodiment of the present invention. -
FIG. 3 shows a process for controlling the bit rate of the video data based on an estimate of the number of the TCP video packets currently in transit. - The leading digit(s) of each reference numeral used in this specification indicates the number of the drawing in which the reference numeral first appears.
- Embodiments of the present invention provide adaptive bit-rate adjustment of multimedia communications channels using transport control protocol (TCP). The techniques disclosed herein are especially useful in real-time two-way applications such as videoconferencing and voice-over-IP telephony, but are also applicable to one-way communications channels, and to communications channels with less stringent latency requirements.
-
FIG. 1 shows avideoconferencing system 100 in communication with anetwork 102 such as the Internet. But while embodiments of the present invention are described with respect to network videoconferencing, the techniques disclosed herein are equally applicable to other sorts of one-way and two-way communications applications over networks or direct links. -
Videoconferencing system 100 comprises avideo source 104 that provides a video signal, for example from a videoconferencing camera, avideo codec 106 to encode the video signal as video data, and avideo packetizer 108 to produce TCP packets of the video data.Videoconferencing system 100 further comprises anaudio source 110 that provides an audio signal, for example from a microphone, anaudio codec 112 to encode the audio signal as audio data, and anaudio packetizer 114 to produce TCP packets of the audio data.Videoconferencing system 100 further comprises one or more transmitcircuits 116 such as Ethernet ports to transmit the video and audio TCP packets, one or more receivecircuits 118 to receive data and control packets fromnetwork 102, and acontroller 120 to controlvideoconferencing system 100. -
FIG. 2 shows an adaptive bit-rate control process 200 forvideoconferencing system 100 according to a preferred embodiment of the present invention. Whileprocess 200 is described with respect to controlling a video data bit rate, it is equally applicable to controlling an audio data bit rate or bit rates for any sort of digital data generated by data generators such as audio and video sources, codecs, and the like. Preferablycontroller 120 executes two instances ofprocess 200 concurrently. One instance ofprocess 200 controls the video data bit rate while the other instance controls the audio data bit rate. - During a videoconference, a source, such as
video source 104, produces a data signal, e.g., a video signal (step 202). A codec, such asvideo codec 106, encodes the video to produce video data at a video bit rate (step 204). A packetizer, such asvideo packetizer 108, produces TCP packets of the video data (step 206). Transmitcircuit 116 transmits the TCP packets of video data (step 208). -
Controller 120 estimates the number of the TCP packets of video data in transit, that is, the number of the TCP packets of video data transmitted by transmitcircuit 116 but not yet received by the receiver of the packets such as another videoconferencing system (step 210). An estimate is used because it is not possible to know the exact number of the TCP video packets currently in transit.Controller 120 controls the bit rate of the video data according to the estimate of the number of the TCP packets of video data in transit (step 212). -
FIG. 3 shows aprocess 300 for controlling the bit rate of the video data based on an estimate of the number of the TCP video packets currently in transit.Controller 120 determines a difference DIFF between a number of the TCP packets of the video data transmitted bytransmit circuit 116 and a number of the TCP packets of the video data received by the receiver (step 302), preferably over a predetermined interval. - The number of the TCP packets of the (video) data received by the receiver is obtained from the receiver, preferably as a Real-time Transport Control Protocol (RTCP) receiver report packet sent by the receiver and received by receive
circuit 118. The number of the TCP packets of the video data transmitted byvideoconferencing system 100 is obtained fromvideoconferencing system 100. In a preferred embodiment, the RTCP reporting interval is two seconds, and the numbers of packets are counted starting with an initialization event, such as the start of the current videoconferencing session. -
Controller 120 also estimates the number D of transmitted packets of the video data that are in transit over network 102 (step 304). Preferably the estimate D is calculated as the median of the previous 50 values of DIFF, although a different number of values of DIFF can be used, and instead of the median, the mean, the mode or some other function of the values of DIFF can be used. - However, upon initialization an insufficient number of values of DIFF are available. Preferably the first value of DIFF is used until 7 values of DIFF have been calculated. Then the median of all of the values of DIFF is used until 50 values of DIFF have been calculated. Thereafter the sliding window of 50 values of DIFF is used, as described above.
- If
network 102 is slow, the first few estimates of D might be too large, for example when the initial video bit rate is much greater than the average bit rate ofnetwork 102. Therefore the initial video bit rate is preferably initially limited based on the size S of the average packet of video data transmitted byvideoconferencing system 100. In a preferred embodiment, if the average packet size exceeds K bits, then the bit rate is decreased by K/DS until DS<K, where K=40,000. Of course, other values for K can be used. -
Process 300 benefits from the stability of the value of D. Therefore, in a preferred embodiment, when a new value of D is calculated, it is compared to the previous value of D. If the new value of D falls inside an estimate window surrounding the previous value of D, then the new value of D is discarded, and the previous value of D is used. Preferably the estimate window is D±one standard deviation of DIFF. Preferably the standard deviation of DIFF is computed as the median absolute deviation of the previous 50 values of DIFF, although other computation methods can be used. -
Process 300 estimates the standard deviation SDev of the packets of video data in transit (step 306). Preferably the standard deviation SDev is computed as the median absolute deviation of the previous 50 values of DIFF, although other computation methods can be used. However, upon initialization an insufficient number of values of DIFF are available. Preferably the standard deviation SDev is computed as the average of the highest and lowest values of DIFF until 7 samples of DIFF have been received, although other computation methods can be used. Thereafter the standard deviation SDev is computed as described above. -
Process 300 then controls the bit rate of the video data according to the values of DIFF and D. In particular,process 300 decreases the bit rate when M consecutive values of DIFF fall above a threshold that is a function of D and increases the bit rate when N consecutive values of DIFF fall below a threshold that is a function of D. Multiple thresholds can be used, as described in detail below. -
Process 300 maintains a counter I for each threshold. For four thresholds,process 300 maintains counters I1, I2, I3, and I4.Process 300 also preferably maintains a counter I5 to count the number of receiver reports for which no video bit rate adjustments are made. - If a value of DIFF exceeds the sum of the value of D and twice the standard deviation SDev (step 308), then
controller 120 increments counter I1 (step 310). If I1=3, meaning DIFF>D+2SDev for three consecutive RTCP receiver reports (step 312), thencontroller 120 decreases the video bit rate (step 314). Preferably the decrease is 20%, although other values can be used. - After changing the video bit rate, and before making another estimate of the number of TCP packets of video data in transit,
process 300 waits for a predetermined interval, preferably by skipping 2 RTCP receiver reports (step 318).Process 300 also resets all of the counters I1, I2, I3, I4, and I5 after changing the video bit rate (step 318).Process 300 then resumes atstep 302. - However, if at
step 308 DIFF≦D+2SDev, counter I1 is reset to zero (step 320) to ensure that counter I1 counts only consecutive RTCP receiver reports where DIFF>D+2SDev. - If a value of DIFF exceeds the sum of the value of D and the standard deviation SDev (step 322), then
controller 120 increments counter I2 (step 324). If I2=5, meaning DIFF>D+SDev for five consecutive RTCP receiver reports (step 326), thencontroller 120 decreases the video bit rate (step 314), skips 2 RTCP reports (step 316), and resets counters I (step 318).Process 300 then resumes atstep 302. Preferably the decrease is 20%, although other values can be used. - However, if at
step 322 DIFF≦D+SDev, counter I2 is reset to zero (step 328) to ensure that counter I2 counts only consecutive RTCP receiver reports where DIFF>D+SDev. - If a value of DIFF exceeds the value of D (step 330), then
controller 120 increments counter I3 (step 332). If I3=9, meaning DIFF>D for nine consecutive RTCP receiver reports (step 334), thencontroller 120 decreases the video bit rate (step 314), skips 2 RTCP reports (step 316), and resets counters I (step 318).Process 300 then resumes atstep 302. Preferably the decrease is 20%, although other values can be used. - However, if at
step 330 DIFF≦D, counter I3 is reset to zero (step 336) to ensure that counter I3 counts only consecutive RTCP receiver reports where DIFF>D. - If a value of DIFF is below the value of D (step 338), then
controller 120 increments counter I4 (step 340). If I4=6, meaning DIFF<D for six consecutive RTCP receiver reports (step 342), thencontroller 120 increases the video bit rate (step 344), skips 2 RTCP reports (step 316), and resets counters I (step 318).Process 300 then resumes atstep 302. Preferably the increase is 10%, although other values can be used. - However, if at
step 338 DIFF≧D, counter I4 is reset to zero (step 346) to ensure that counter I4 counts only consecutive RTCP receiver reports where DIFF<D. - To ensure that the video bit rate does not stabilize at an unnecessarily low value, if no changes to the video bit rate are made for J consecutive values of DIFF (that is, for J RTCP receiver report packets), then
controller 120 increases the video bit rate. Preferably J=16 and the increase is 10%, although other values can be used. Therefore when no video bit rate adjustment is made for a RTCP receiver report,process 300 increments counter I5 (step 348). If I5=16, meaning no video bit rate adjustment has been made for 16 consecutive RTCP receiver reports (step 350), thencontroller 120 increases the video bit rate (step 344), skips 2 RTCP reports (step 316), and resets counters I (step 318). Preferably the increase is 10%, although other values can be used. Otherwise process 300 resumes withstep 302. - Preferably
process 300 includes a burst detection routine to handle bursts of video data, for example when an i-frame is to be sent when the video includes significant motion. When such a burst occurs,controller 120 halves the video bit rate, and maintains that value for 3 RTCP receiver report packets before resumingprocess 300. - The invention can be implemented in digital electronic circuitry, or in hardware, firmware, software, or in combinations of them. An apparatus of the invention can be implemented in a computer program product tangibly embodied in a device-readable medium, e.g., storage device, for execution by a programmable processor; and method steps of the invention can be performed by a programmable processor executing a program of instructions to perform functions of the invention by operating on input data and generating output. The invention can be implemented advantageously in one or more programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. Each computer program can be implemented in a high-level procedural or object-oriented programming language, or in assembly or machine language if desired; and in any case, the language can be a compiled or interpreted language. Suitable processors include, by way of example, both general and special purpose microprocessors. Generally, a processor will receive instructions and data from a read-only memory and/or a random access memory. Generally, a computer will include one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM disks. Any of the foregoing can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).
- A number of implementations of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. Accordingly, other implementations are within the scope of the following claims.
Claims (20)
1. A videoconferencing apparatus comprising:
a video codec to generate video data at a video bit rate;
a video packetizer to produce Transport Control Protocol (TCP) packets of the video data;
an audio codec to generate audio data at an audio bit rate;
an audio packetizer to produce packets of the audio data;
a transmit circuit to transmit the TCP packets of the video data and the packets of the audio data;
a receive circuit to receive a Real-time Transport Control Protocol (RTCP) receiver report representing a number of the TCP packets of the video data received by a receiver of the TCP packets of the video data; and
a controller to
generate an estimate of a number of the TCP packets of the video data transmitted by the transmit circuit but not yet received by the receiver based on a difference between a number of the TCP packets of the video data transmitted by the transmit circuit and the number of the TCP packets of the video data received by the receiver, and
control the video bit rate according to the estimate.
2. An apparatus comprising:
a data generator to generate data at a bit rate;
a packetizer to produce Transport Control Protocol (TCP) packets of the data;
a transmit circuit to transmit the TCP packets of the data; and
a controller to control the bit rate according to an estimate of a number of the TCP packets of the data transmitted by the transmit circuit but not yet received by a receiver of the TCP packets of the data.
3. The apparatus of claim 2 , wherein:
the controller generates the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver based on
a number of the TCP packets of the data transmitted by the transmit circuit following an initialization event,
a number of the TCP packets of the data received by the receiver following the initialization event, and
a previous estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver.
4. The apparatus of claim 3 , wherein, to generate the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver, the controller:
determines a plurality of differences, each at a different time, between a number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and a number of the TCP packets of the data received by the receiver following the initialization event; and
generates the estimate according to at least one of the group consisting of
a median of the differences,
a mean of the differences, and
a mode of the differences.
5. The apparatus of claim 2 , wherein, to control the bit rate according to the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver, the controller:
decreases the bit rate when a first predetermined number of consecutive differences between the number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and the number of the TCP packets of the data received by the receiver following the initialization event fall above a threshold that is a first function of the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver;
increases the bit rate when a second predetermined number of consecutive differences between the number of the TCP packets of the data transmitted by the transmit circuit following an initialization event and the number of the TCP packets of the data received by the receiver following the initialization event fall below a threshold that is a second function of the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver; and
increases the bit rate when the bit rate has not been increased for a predetermined interval.
6. The apparatus of claim 5 , wherein, to control the bit rate according to the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver, the controller further:
generates a new estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver;
determines whether the new estimate falls outside an estimate window surrounding the estimate;
controls the bit rate based on the new estimate when the new estimate falls outside the estimate window surrounding the estimate; and
controls the bit rate based on the estimate when the new estimate falls inside the estimate window surrounding the estimate.
7. The apparatus of claim 2 , further comprising:
a receive circuit to receive a packet of data representing the number of the TCP packets of the data received by the receiver.
8. The apparatus of claim 7 , wherein the packet of data representing the number of the TCP packets of the data received by the receiver comprises:
a Real-time Transport Control Protocol (RTCP) receiver report packet.
9. The apparatus of claim 2 , wherein the data generator comprises at least one of the group consisting of:
a video codec to encode video data; and
an audio codec to encode audio data.
10. A videoconferencing system comprising the apparatus of claim 2 .
11. A method comprising:
generating data at a bit rate;
producing Transport Control Protocol (TCP) packets of the data;
transmitting the TCP packets of the data; and
controlling the bit rate according to an estimate of a number of the TCP packets of the data transmitted by the transmit circuit but not yet received by a receiver of the TCP packets of the data.
12. The method of claim 11 , further comprising:
generating the estimate of the number of the TCP packets of the data transmitted by the transmit circuit but not yet received by the receiver based on
a number of the TCP packets of the data transmitted following an initialization event,
a number of the TCP packets of the data received by the receiver following the initialization event, and
a previous estimate of the number of the TCP packets of the data transmitted but not yet received by the receiver.
13. The method of claim 12 , wherein generating the estimate of the number of the TCP packets of the data transmitted but not yet received by the receiver comprises:
determining a plurality of differences, each at a different time, between a number of the TCP packets of the data transmitted following an initialization event and a number of the TCP packets of the data received by the receiver following the initialization event; and
generating the estimate according to at least one of the group consisting of
a median of the differences,
a mean of the differences, and
a mode of the differences.
14. The method of claim 11 , wherein controlling the bit rate according to the estimate of the number of the TCP packets of the data transmitted but not yet received by the receiver comprises:
decreasing the bit rate when a first predetermined number of consecutive differences between the number of the TCP packets of the data transmitted following an initialization event and the number of the TCP packets of the data received by the receiver following the initialization event fall above a threshold that is a first function of the estimate of the number of the TCP packets of the data transmitted but not yet received by the receiver;
increasing the bit rate when a second predetermined number of consecutive differences between the number of the TCP packets of the data transmitted following an initialization event and the number of the TCP packets of the data received by the receiver following the initialization event fall below a threshold that is a second function of the estimate of the number of the TCP packets of the data transmitted but not yet received by the receiver; and
increasing the bit rate when the bit rate has not been increased for a predetermined interval.
15. The method of claim 14 , wherein controlling the bit rate according to the estimate of the number of the TCP packets of the data transmitted but not yet received by the receiver of the TCP packets of the data further comprises:
generating a new estimate of the number of the TCP packets of the data transmitted but not yet received by the receiver;
determining whether the new estimate falls outside an estimate window surrounding the estimate;
controlling the bit rate based on the new estimate when the new estimate falls outside the estimate window surrounding the estimate; and
controlling the bit rate based on the estimate when the new estimate falls inside the estimate window surrounding the estimate.
16. The method of claim 11 , further comprising:
receiving a packet of data representing the number of the TCP packets of the data received by the receiver.
17. The method of claim 16 , wherein the packet of data representing the number of the TCP packets of the data received by the receiver comprises:
a Real-time Transport Control Protocol (RTCP) receiver report packet.
18. The method of claim 11 , wherein the data comprises at least one of video data or audio data.
19. A device-readable medium or waveform containing a program of instructions executable by a device and adapted to perform the method of claim 11 .
20. A device-readable medium or waveform containing a program of instructions executable by a device adapted to perform the method of claim 12.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/051,674 US20060176832A1 (en) | 2005-02-04 | 2005-02-04 | Adaptive bit-rate adjustment of multimedia communications channels using transport control protocol |
JP2006015359A JP4265609B2 (en) | 2005-02-04 | 2006-01-24 | Video conferencing apparatus, apparatus, video conferencing system, method and device readable medium or waveform comprising instruction program adapted to perform the method |
US11/736,451 US7590060B2 (en) | 2005-02-04 | 2007-04-17 | Network congestion recognition and management |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/051,674 US20060176832A1 (en) | 2005-02-04 | 2005-02-04 | Adaptive bit-rate adjustment of multimedia communications channels using transport control protocol |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/736,451 Continuation-In-Part US7590060B2 (en) | 2005-02-04 | 2007-04-17 | Network congestion recognition and management |
Publications (1)
Publication Number | Publication Date |
---|---|
US20060176832A1 true US20060176832A1 (en) | 2006-08-10 |
Family
ID=36779819
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/051,674 Abandoned US20060176832A1 (en) | 2005-02-04 | 2005-02-04 | Adaptive bit-rate adjustment of multimedia communications channels using transport control protocol |
US11/736,451 Expired - Fee Related US7590060B2 (en) | 2005-02-04 | 2007-04-17 | Network congestion recognition and management |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/736,451 Expired - Fee Related US7590060B2 (en) | 2005-02-04 | 2007-04-17 | Network congestion recognition and management |
Country Status (2)
Country | Link |
---|---|
US (2) | US20060176832A1 (en) |
JP (1) | JP4265609B2 (en) |
Cited By (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060255930A1 (en) * | 2005-05-12 | 2006-11-16 | Berkman William H | Power line communications system and method |
US20070002771A1 (en) * | 2005-06-21 | 2007-01-04 | Berkman William H | Power line communication rate limiting system and method |
US20070091800A1 (en) * | 2005-10-21 | 2007-04-26 | Corcoran Kevin F | Power line communication voice over IP system and method |
US20080049660A1 (en) * | 2006-08-25 | 2008-02-28 | Ati Technologies Inc. | Method & Apparatus for Content Delivery to Devices |
DE102006039783A1 (en) * | 2006-08-24 | 2008-03-20 | Siemens Ag | Transmission method for video data formed by audio and video signals, involves processing of data packet formed by audio data-stream which is withheld by unit produced by transmission control protocol packet |
US20080165690A1 (en) * | 2007-01-10 | 2008-07-10 | International Business Machines Corporation | Infiniband Credit-Less Flow Control For Long Distance Links |
US20090187344A1 (en) * | 2008-01-19 | 2009-07-23 | Brancaccio Daniel S | System, Method, and Computer Program Product for Analyzing Power Grid Data |
US20090187284A1 (en) * | 2008-01-21 | 2009-07-23 | Kreiss David G | System and Method for Providing Power Distribution System Information |
US20090184835A1 (en) * | 2008-01-20 | 2009-07-23 | Deaver Sr Brian J | System, Device and Method For Providing Power Outage and Restoration Notification |
US20090289637A1 (en) * | 2007-11-07 | 2009-11-26 | Radtke William O | System and Method for Determining the Impedance of a Medium Voltage Power Line |
US7675897B2 (en) | 2005-09-06 | 2010-03-09 | Current Technologies, Llc | Power line communications system with differentiated data services |
US20100118114A1 (en) * | 2008-11-07 | 2010-05-13 | Magor Communications Corporation | Video rate adaptation for congestion control |
US20100157015A1 (en) * | 2008-12-23 | 2010-06-24 | Christopher Jensen Read | Videoconference Arrangement |
US20140237136A1 (en) * | 2013-02-21 | 2014-08-21 | Fujitsu Limited | Communication system, communication controller, communication control method, and medium |
US9363480B2 (en) * | 2014-08-20 | 2016-06-07 | Cisco Technology, Inc. | Obtaining replay of audio during a conference session |
US20190082145A1 (en) * | 2017-09-11 | 2019-03-14 | Zeller Digital Innovations, Inc. | Videoconferencing calibration systems, controllers and methods for calibrating a videoconferencing system |
EP2600576B1 (en) * | 2007-07-10 | 2019-05-08 | Citrix Systems, Inc. | Adaptive bitrate management for streaming media over packet networks |
Families Citing this family (42)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040162637A1 (en) | 2002-07-25 | 2004-08-19 | Yulun Wang | Medical tele-robotic system with a master remote station with an arbitrator |
US7813836B2 (en) | 2003-12-09 | 2010-10-12 | Intouch Technologies, Inc. | Protocol for a remotely controlled videoconferencing robot |
US8077963B2 (en) | 2004-07-13 | 2011-12-13 | Yulun Wang | Mobile robot with a head-based movement mapping scheme |
US9198728B2 (en) | 2005-09-30 | 2015-12-01 | Intouch Technologies, Inc. | Multi-camera mobile teleconferencing platform |
US8849679B2 (en) | 2006-06-15 | 2014-09-30 | Intouch Technologies, Inc. | Remote controlled robot system that provides medical images |
US9160783B2 (en) | 2007-05-09 | 2015-10-13 | Intouch Technologies, Inc. | Robot system that operates through a network firewall |
US10875182B2 (en) | 2008-03-20 | 2020-12-29 | Teladoc Health, Inc. | Remote presence system mounted to operating room hardware |
US8638653B2 (en) * | 2008-03-27 | 2014-01-28 | Intel Corporation | Adaptive transmissions for optimized application delivery in wireless networks |
US8179418B2 (en) | 2008-04-14 | 2012-05-15 | Intouch Technologies, Inc. | Robotic based health care system |
US8170241B2 (en) | 2008-04-17 | 2012-05-01 | Intouch Technologies, Inc. | Mobile tele-presence system with a microphone system |
US9193065B2 (en) | 2008-07-10 | 2015-11-24 | Intouch Technologies, Inc. | Docking system for a tele-presence robot |
US9842192B2 (en) | 2008-07-11 | 2017-12-12 | Intouch Technologies, Inc. | Tele-presence robot system with multi-cast features |
US8340819B2 (en) | 2008-09-18 | 2012-12-25 | Intouch Technologies, Inc. | Mobile videoconferencing robot system with network adaptive driving |
US8996165B2 (en) | 2008-10-21 | 2015-03-31 | Intouch Technologies, Inc. | Telepresence robot with a camera boom |
US9138891B2 (en) | 2008-11-25 | 2015-09-22 | Intouch Technologies, Inc. | Server connectivity control for tele-presence robot |
US8463435B2 (en) | 2008-11-25 | 2013-06-11 | Intouch Technologies, Inc. | Server connectivity control for tele-presence robot |
US8849680B2 (en) | 2009-01-29 | 2014-09-30 | Intouch Technologies, Inc. | Documentation through a remote presence robot |
US8897920B2 (en) | 2009-04-17 | 2014-11-25 | Intouch Technologies, Inc. | Tele-presence robot system with software modularity, projector and laser pointer |
US8384755B2 (en) | 2009-08-26 | 2013-02-26 | Intouch Technologies, Inc. | Portable remote presence robot |
US11399153B2 (en) | 2009-08-26 | 2022-07-26 | Teladoc Health, Inc. | Portable telepresence apparatus |
US11154981B2 (en) | 2010-02-04 | 2021-10-26 | Teladoc Health, Inc. | Robot user interface for telepresence robot system |
US8670017B2 (en) | 2010-03-04 | 2014-03-11 | Intouch Technologies, Inc. | Remote presence system including a cart that supports a robot face and an overhead camera |
US10343283B2 (en) | 2010-05-24 | 2019-07-09 | Intouch Technologies, Inc. | Telepresence robot system that can be accessed by a cellular phone |
US10808882B2 (en) | 2010-05-26 | 2020-10-20 | Intouch Technologies, Inc. | Tele-robotic system with a robot face placed on a chair |
US8904027B2 (en) * | 2010-06-30 | 2014-12-02 | Cable Television Laboratories, Inc. | Adaptive bit rate for data transmission |
US9264664B2 (en) | 2010-12-03 | 2016-02-16 | Intouch Technologies, Inc. | Systems and methods for dynamic bandwidth allocation |
US9323250B2 (en) | 2011-01-28 | 2016-04-26 | Intouch Technologies, Inc. | Time-dependent navigation of telepresence robots |
WO2012103525A2 (en) | 2011-01-28 | 2012-08-02 | Intouch Technologies, Inc. | Interfacing with a mobile telepresence robot |
US10769739B2 (en) | 2011-04-25 | 2020-09-08 | Intouch Technologies, Inc. | Systems and methods for management of information among medical providers and facilities |
US9098611B2 (en) | 2012-11-26 | 2015-08-04 | Intouch Technologies, Inc. | Enhanced video interaction for a user interface of a telepresence network |
US20140139616A1 (en) | 2012-01-27 | 2014-05-22 | Intouch Technologies, Inc. | Enhanced Diagnostics for a Telepresence Robot |
US8836751B2 (en) | 2011-11-08 | 2014-09-16 | Intouch Technologies, Inc. | Tele-presence system with a user interface that displays different communication links |
US8902278B2 (en) | 2012-04-11 | 2014-12-02 | Intouch Technologies, Inc. | Systems and methods for visualizing and managing telepresence devices in healthcare networks |
US9251313B2 (en) | 2012-04-11 | 2016-02-02 | Intouch Technologies, Inc. | Systems and methods for visualizing and managing telepresence devices in healthcare networks |
US9361021B2 (en) | 2012-05-22 | 2016-06-07 | Irobot Corporation | Graphical user interfaces including touchpad driving interfaces for telemedicine devices |
WO2013176762A1 (en) | 2012-05-22 | 2013-11-28 | Intouch Technologies, Inc. | Social behavior rules for a medical telepresence robot |
US10225761B2 (en) | 2014-11-06 | 2019-03-05 | At&T Intellectual Property I, L.P. | Enhanced network congestion application programming interface |
US10291941B2 (en) | 2017-03-09 | 2019-05-14 | At&T Mobility Ii Llc | Pre-caching video content to devices using LTE broadcast |
US11862302B2 (en) | 2017-04-24 | 2024-01-02 | Teladoc Health, Inc. | Automated transcription and documentation of tele-health encounters |
US10483007B2 (en) | 2017-07-25 | 2019-11-19 | Intouch Technologies, Inc. | Modular telehealth cart with thermal imaging and touch screen user interface |
US11636944B2 (en) | 2017-08-25 | 2023-04-25 | Teladoc Health, Inc. | Connectivity infrastructure for a telehealth platform |
US10617299B2 (en) | 2018-04-27 | 2020-04-14 | Intouch Technologies, Inc. | Telehealth cart that supports a removable tablet with seamless audio/video switching |
Citations (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5541852A (en) * | 1994-04-14 | 1996-07-30 | Motorola, Inc. | Device, method and system for variable bit-rate packet video communications |
US6141324A (en) * | 1998-09-01 | 2000-10-31 | Utah State University | System and method for low latency communication |
US6292834B1 (en) * | 1997-03-14 | 2001-09-18 | Microsoft Corporation | Dynamic bandwidth selection for efficient transmission of multimedia streams in a computer network |
US6310857B1 (en) * | 1997-06-16 | 2001-10-30 | At&T Corp. | Method and apparatus for smoothing and multiplexing video data flows |
US20020156910A1 (en) * | 2001-04-19 | 2002-10-24 | Yuzo Senda | Flow control system and method |
US6483842B1 (en) * | 1997-02-21 | 2002-11-19 | Nortel Networks Limited | Multimedia switching system |
US20030016630A1 (en) * | 2001-06-14 | 2003-01-23 | Microsoft Corporation | Method and system for providing adaptive bandwidth control for real-time communication |
US20030037158A1 (en) * | 1997-08-22 | 2003-02-20 | Koichi Yano | Data communication apparatus and method |
US20030069963A1 (en) * | 2001-09-27 | 2003-04-10 | Nikil Jayant | System and method of quality of service signaling between client and server devices |
US20030074674A1 (en) * | 2001-10-17 | 2003-04-17 | Magliaro Maximilian Matthew | Method and system for dynamically adjusting video bit rates |
US20030093244A1 (en) * | 2001-05-24 | 2003-05-15 | Andrew Corlett | Network metric system |
US20030123464A1 (en) * | 2001-12-27 | 2003-07-03 | Eung-Don Lee | Method for controlling error of internet fax data |
US20040047290A1 (en) * | 2002-04-25 | 2004-03-11 | Sridhar Komandur | Multimedia traffic optimization |
US20040078478A1 (en) * | 2002-10-16 | 2004-04-22 | Nec Corporation | Data transmission rate regulating system, monitor and control apparatus of data transmission rate, and data transmission rate regulating method to be used in the same |
US20050172028A1 (en) * | 2002-03-27 | 2005-08-04 | Nilsson Michael E. | Data streaming system and method |
US20050195843A1 (en) * | 2004-03-03 | 2005-09-08 | Texas Instruments Incorporated | Group based packet scheduling algorithm |
US7072305B1 (en) * | 1999-10-29 | 2006-07-04 | Applied Digital Access, Inc. | Method and apparatus for analyzing a communications network link |
US7197014B2 (en) * | 2004-06-25 | 2007-03-27 | Fujitsu Limited | Recording medium, fault analysis device and fault analysis method |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6075768A (en) | 1995-11-09 | 2000-06-13 | At&T Corporation | Fair bandwidth sharing for video traffic sources using distributed feedback control |
US5748900A (en) | 1996-03-13 | 1998-05-05 | Cray Research, Inc. | Adaptive congestion control mechanism for modular computer networks |
US6144637A (en) * | 1996-12-20 | 2000-11-07 | Cisco Technology, Inc. | Data communications |
US6198724B1 (en) * | 1997-10-02 | 2001-03-06 | Vertex Networks, Inc. | ATM cell scheduling method and apparatus |
CA2237264A1 (en) | 1998-05-08 | 1999-11-08 | Northern Telecom Limited | Receiver based congestion control |
US7023800B1 (en) | 1999-06-01 | 2006-04-04 | Microsoft Corporation | Congestion avoidance within aggregate channels |
JP3705353B2 (en) * | 2001-05-22 | 2005-10-12 | 日本電気株式会社 | Packet scheduling apparatus and method |
US7558197B1 (en) * | 2002-01-17 | 2009-07-07 | Juniper Networks, Inc. | Dequeuing and congestion control systems and methods |
US7110411B2 (en) * | 2002-03-25 | 2006-09-19 | Erlang Technology, Inc. | Method and apparatus for WFQ scheduling using a plurality of scheduling queues to provide fairness, high scalability, and low computation complexity |
-
2005
- 2005-02-04 US US11/051,674 patent/US20060176832A1/en not_active Abandoned
-
2006
- 2006-01-24 JP JP2006015359A patent/JP4265609B2/en not_active Expired - Fee Related
-
2007
- 2007-04-17 US US11/736,451 patent/US7590060B2/en not_active Expired - Fee Related
Patent Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5541852A (en) * | 1994-04-14 | 1996-07-30 | Motorola, Inc. | Device, method and system for variable bit-rate packet video communications |
US6483842B1 (en) * | 1997-02-21 | 2002-11-19 | Nortel Networks Limited | Multimedia switching system |
US6292834B1 (en) * | 1997-03-14 | 2001-09-18 | Microsoft Corporation | Dynamic bandwidth selection for efficient transmission of multimedia streams in a computer network |
US6310857B1 (en) * | 1997-06-16 | 2001-10-30 | At&T Corp. | Method and apparatus for smoothing and multiplexing video data flows |
US6701372B2 (en) * | 1997-08-22 | 2004-03-02 | Canon Kabushiki Kaisha | Data communication apparatus and method |
US20030037158A1 (en) * | 1997-08-22 | 2003-02-20 | Koichi Yano | Data communication apparatus and method |
US6141324A (en) * | 1998-09-01 | 2000-10-31 | Utah State University | System and method for low latency communication |
US7072305B1 (en) * | 1999-10-29 | 2006-07-04 | Applied Digital Access, Inc. | Method and apparatus for analyzing a communications network link |
US20020156910A1 (en) * | 2001-04-19 | 2002-10-24 | Yuzo Senda | Flow control system and method |
US20030093244A1 (en) * | 2001-05-24 | 2003-05-15 | Andrew Corlett | Network metric system |
US20030016630A1 (en) * | 2001-06-14 | 2003-01-23 | Microsoft Corporation | Method and system for providing adaptive bandwidth control for real-time communication |
US20030069963A1 (en) * | 2001-09-27 | 2003-04-10 | Nikil Jayant | System and method of quality of service signaling between client and server devices |
US20030074674A1 (en) * | 2001-10-17 | 2003-04-17 | Magliaro Maximilian Matthew | Method and system for dynamically adjusting video bit rates |
US20030123464A1 (en) * | 2001-12-27 | 2003-07-03 | Eung-Don Lee | Method for controlling error of internet fax data |
US20050172028A1 (en) * | 2002-03-27 | 2005-08-04 | Nilsson Michael E. | Data streaming system and method |
US20040047290A1 (en) * | 2002-04-25 | 2004-03-11 | Sridhar Komandur | Multimedia traffic optimization |
US20040078478A1 (en) * | 2002-10-16 | 2004-04-22 | Nec Corporation | Data transmission rate regulating system, monitor and control apparatus of data transmission rate, and data transmission rate regulating method to be used in the same |
US20050195843A1 (en) * | 2004-03-03 | 2005-09-08 | Texas Instruments Incorporated | Group based packet scheduling algorithm |
US7197014B2 (en) * | 2004-06-25 | 2007-03-27 | Fujitsu Limited | Recording medium, fault analysis device and fault analysis method |
Cited By (38)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060255930A1 (en) * | 2005-05-12 | 2006-11-16 | Berkman William H | Power line communications system and method |
US7558206B2 (en) * | 2005-06-21 | 2009-07-07 | Current Technologies, Llc | Power line communication rate limiting system and method |
US20070002771A1 (en) * | 2005-06-21 | 2007-01-04 | Berkman William H | Power line communication rate limiting system and method |
US7675897B2 (en) | 2005-09-06 | 2010-03-09 | Current Technologies, Llc | Power line communications system with differentiated data services |
US20070091800A1 (en) * | 2005-10-21 | 2007-04-26 | Corcoran Kevin F | Power line communication voice over IP system and method |
US7856007B2 (en) | 2005-10-21 | 2010-12-21 | Current Technologies, Llc | Power line communication voice over IP system and method |
DE102006039783A1 (en) * | 2006-08-24 | 2008-03-20 | Siemens Ag | Transmission method for video data formed by audio and video signals, involves processing of data packet formed by audio data-stream which is withheld by unit produced by transmission control protocol packet |
US7962182B2 (en) * | 2006-08-25 | 2011-06-14 | Qualcomm Incorporated | Method and apparatus for content delivery to devices |
US20080049660A1 (en) * | 2006-08-25 | 2008-02-28 | Ati Technologies Inc. | Method & Apparatus for Content Delivery to Devices |
US20080165690A1 (en) * | 2007-01-10 | 2008-07-10 | International Business Machines Corporation | Infiniband Credit-Less Flow Control For Long Distance Links |
US7952998B2 (en) * | 2007-01-10 | 2011-05-31 | International Business Machines Corporation | InfiniBand credit-less flow control for long distance links |
EP2600576B1 (en) * | 2007-07-10 | 2019-05-08 | Citrix Systems, Inc. | Adaptive bitrate management for streaming media over packet networks |
US20090289637A1 (en) * | 2007-11-07 | 2009-11-26 | Radtke William O | System and Method for Determining the Impedance of a Medium Voltage Power Line |
US20090187344A1 (en) * | 2008-01-19 | 2009-07-23 | Brancaccio Daniel S | System, Method, and Computer Program Product for Analyzing Power Grid Data |
US20090184835A1 (en) * | 2008-01-20 | 2009-07-23 | Deaver Sr Brian J | System, Device and Method For Providing Power Outage and Restoration Notification |
US7965195B2 (en) | 2008-01-20 | 2011-06-21 | Current Technologies, Llc | System, device and method for providing power outage and restoration notification |
US8000913B2 (en) | 2008-01-21 | 2011-08-16 | Current Communications Services, Llc | System and method for providing power distribution system information |
US20090187284A1 (en) * | 2008-01-21 | 2009-07-23 | Kreiss David G | System and Method for Providing Power Distribution System Information |
US8280656B2 (en) | 2008-01-21 | 2012-10-02 | Current Communications Services, Llc | System and method for providing power distribution system information |
US8285500B2 (en) | 2008-01-21 | 2012-10-09 | Current Communications Services, Llc | System and method for providing power distribution system information |
US8290727B2 (en) | 2008-01-21 | 2012-10-16 | Current Communications Services, Llc | System and method for providing power distribution system information |
US20100118114A1 (en) * | 2008-11-07 | 2010-05-13 | Magor Communications Corporation | Video rate adaptation for congestion control |
CN102239690A (en) * | 2008-11-07 | 2011-11-09 | 玛格通讯有限公司 | Stable video rate adaptation for congestion control |
US8446452B2 (en) * | 2008-11-07 | 2013-05-21 | Magor Communications Corporation | Video rate adaptation for congestion control |
US8259156B2 (en) * | 2008-12-23 | 2012-09-04 | Sony Corporation | Videoconference arrangement |
US8817064B2 (en) | 2008-12-23 | 2014-08-26 | Sony Corporation | Videoconference arrangement |
US20100157015A1 (en) * | 2008-12-23 | 2010-06-24 | Christopher Jensen Read | Videoconference Arrangement |
US20140237136A1 (en) * | 2013-02-21 | 2014-08-21 | Fujitsu Limited | Communication system, communication controller, communication control method, and medium |
US9882751B2 (en) * | 2013-02-21 | 2018-01-30 | Fujitsu Limited | Communication system, communication controller, communication control method, and medium |
US9363480B2 (en) * | 2014-08-20 | 2016-06-07 | Cisco Technology, Inc. | Obtaining replay of audio during a conference session |
US20190082145A1 (en) * | 2017-09-11 | 2019-03-14 | Zeller Digital Innovations, Inc. | Videoconferencing calibration systems, controllers and methods for calibrating a videoconferencing system |
US10477146B2 (en) * | 2017-09-11 | 2019-11-12 | Zeller Digital Innovations, Inc. | Videoconferencing calibration systems, controllers and methods for calibrating a videoconferencing system |
US20200068170A1 (en) * | 2017-09-11 | 2020-02-27 | Zeller Digital Innovations, Inc. | Videoconferencing Calibration Systems, Controllers And Methods For Calibrating A Videoconferencing System |
US10715763B2 (en) * | 2017-09-11 | 2020-07-14 | Zeller Digital Innovations, Inc. | Videoconferencing calibration systems, controllers and methods for calibrating a videoconferencing system |
US11134216B2 (en) * | 2017-09-11 | 2021-09-28 | Zeller Digital Innovations, Inc. | Videoconferencing calibration systems, controllers and methods for calibrating a videoconferencing system |
US11539917B2 (en) * | 2017-09-11 | 2022-12-27 | Zeller Digital Innovations, Inc. | Videoconferencing calibration systems, controllers and methods for calibrating a videoconferencing system |
US20230179739A1 (en) * | 2017-09-11 | 2023-06-08 | Zeller Digital Innovations, Inc. | Videoconferencing Calibration Systems, Controllers And Methods For Calibrating A Videoconferencing System |
US11902709B2 (en) * | 2017-09-11 | 2024-02-13 | Zeller Digital Innovations, Inc. | Videoconferencing calibration systems, controllers and methods for calibrating a videoconferencing system |
Also Published As
Publication number | Publication date |
---|---|
JP4265609B2 (en) | 2009-05-20 |
US20070189167A1 (en) | 2007-08-16 |
JP2006217589A (en) | 2006-08-17 |
US7590060B2 (en) | 2009-09-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20060176832A1 (en) | Adaptive bit-rate adjustment of multimedia communications channels using transport control protocol | |
RU2305908C2 (en) | Adaptive method for evaluating multimedia data transmission speed | |
US9609040B2 (en) | Efficient bitrate adaptation in video communications over IP networks | |
US7920492B1 (en) | Devices, softwares and methods for redundantly encoding a data stream for network transmission with adjustable redundant-coding delay | |
US7047308B2 (en) | System and method for simultaneous media playout | |
US6853625B2 (en) | Method of dynamically transmitting data packets using RTP and RTCP protocols | |
US9356869B2 (en) | VoIP bandwidth management | |
US20050021830A1 (en) | Data communications method and system using buffer size to calculate transmission rate for congestion control | |
WO2017000719A1 (en) | Congestion control method and device based on queue delay | |
US8948058B2 (en) | System and method for improving audio quality during web conferences over low-speed network connections | |
WO2008049434A1 (en) | Method and system for firewall friendly mobile real-time communication | |
CA2457193C (en) | Data communications method and system for transmitting multiple data streams calculating available bandwidth per stream and bit stream trade-off | |
US20020075895A1 (en) | Transmission rate controller and transmission rate control method | |
KR100982630B1 (en) | Device and process for adjusting the bit rate of a stream of contents and associated products | |
CN107483990B (en) | Dynamic code rate adjusting method and device for streaming media transmission and transmission system | |
EP1533969A1 (en) | Loss reporting for packet-switched streaming services using loss RLE report blocks | |
Kim et al. | TCP-friendly Internet video with smooth and fast rate adaptation and network-aware error control | |
Narbutt et al. | Adaptive playout buffering for audio/video transmission over the internet | |
KR100931375B1 (en) | Efficient data streaming method using efficien tparameters and data streaming server | |
CN1768509B (en) | Method of ensuring the quality of service in a network | |
CN113612649B (en) | Round trip estimation | |
EP3161983A1 (en) | Method for communicating media data between two devices incorporating effectiveness of error correction strategies and associated computer program, communication quality module and device | |
KR101107325B1 (en) | Method and System for Measuring Quality by Network Interval for Real-time Multimedia Service | |
Begić et al. | Rapid synchronization of RTP multicast sessions using the retransmission server | |
Papadimitriou | An integrated smooth transmission control and temporal scaling scheme for MPEG-4 streaming video |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: EPSON RESEARCH AND DEVELOPMENT, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICELI, SEAN;REEL/FRAME:016257/0993 Effective date: 20050203 |
|
AS | Assignment |
Owner name: SEIKO EPSON CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:EPSON RESEARCH AND DEVELOPMENT, INC.;REEL/FRAME:016185/0459 Effective date: 20050428 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |