WO2015034405A1

WO2015034405A1 - Adaptive multimedia codec parameter adaptation

Info

Publication number: WO2015034405A1
Application number: PCT/SE2013/051032
Authority: WO
Inventors: Markus Andersson; Fredrik Jansson
Original assignee: Telefonaktiebolaget L M Ericsson (Publ)
Priority date: 2013-09-05
Filing date: 2013-09-05
Publication date: 2015-03-12

Abstract

There is provided determining of multimedia codec parameters for a multimedia communications session. A channel quality indicator value representing current throughput characteristics of a communications channel used for the multimedia communications session is acquired. The channel quality indicator value is compared to a previous channel quality indicator value of the communications channel. It is determined, based on the comparing, whether to delay adaptation of values of a set of multimedia codec parameters to be used by a multimedia codec during the multimedia communications session or not, and which values of the set of multimedia codec parameters to adapt.

Description

ADAPTIVE MULTIMEDIA CODEC PARAMETER ADAPTATION

TECHNICAL FIELD

Embodiments presented herein relate to multimedia communications sessions, and particularly to a method, an electronic device, a computer program, and a computer program product for determining multimedia codec parameters for a multimedia communications session.

BACKGROUND

In communication networks, there is always a challenge to obtain good performance and capacity for a given communications protocol, its parameters and the physical environment in which the communication network is deployed.

Further, the area of communications has evolved rapidly over the last years; from traditional person-to-person phone calls to many and more advanced services; such as multimedia sessions involving e.g., multiparty video conferencing. Such services put extensive requirements on the

communications network and when multimedia such as audio and video are transmitted from one device to another over the communications network, it is not uncommon that the capacity of the communications network is lower than what is required by the multimedia session to give the end-user a desired user experience. One non-limiting example is video telephony over a 3G network where on the one hand, High Definition (HD) video transmission with a bit rate of several Mbit/ s could be needed to deliver a high quality end- user experience, whilst on the other hand, such a high bit rate could by the communications network only be supported under benign conditions. As a result of varying network performance, running a multimedia session service with a high fixed bit rate over wireless communications networks (or other communications networks with time-varying throughput

characteristics) may lead to quality problems and unsatisfied end-users. To mitigate this some existing services have implemented mechanisms to cope with temporarily congested networks by means of rate adaptation. Through various techniques the multimedia stream is adapted with the goal to suit present conditions of the communications channel. In general terms, it is common that the end-user equipment and/ or the network nodes of the communications network measures the perceived end-user quality or other Quality of Service (QoS) parameters such as packet loss or jitter that may have a correlation with the perceived end-user quality. These measurements may then be used to control the multimedia bit rate, and thus the bandwidth consumption, used by the electronic device for transmitting the multimedia stream. However, there is still a need for improved control of bandwidth

consumption and/or end-user quality during a multimedia communications session, such as a video conferencing service.

SUMMARY

An object of embodiments herein is to provide improved bandwidth consumption and/ or end-user quality during a multimedia communications session, such as a video conferencing service.

The inventors of the enclosed embodiments have through a combination of practical experimentation and theoretical derivation discovered that it may be advantageous to, instead of having static rules for how to spend available bits, have adaptive rules for how to spend the available bits during a multimedia communications session.

A particular object is therefore to provide improved bandwidth consumption during a multimedia communications session, such as a video conferencing service, wherein the bandwidth consumption is adapted to varying network conditions in an efficient way.

According to a first aspect there is presented a method for determining multimedia codec parameters for a multimedia communications session. The method is performed by an electronic device. The method comprises acquiring a channel quality indicator value representing current throughput characteristics of a communications channel used for the multimedia communications session. The method comprises comparing the channel quality indicator value to a previous channel quality indicator value of the communications channel. The method comprises determining, based on said comparing, whether to delay adaptation of values of a set of multimedia codec parameters to be used by a multimedia codec during said multimedia communications session or not, and which values of said set of multimedia codec parameters to adapt.

Advantageously this provides improved bandwidth consumption during a multimedia communications session. Advantageously this may improve network efficiency for multimedia services.

Advantageously this may improve the Quality of Service, and thus the end- user experience.

Advantageously this may enable network costs to be reduced.

According to a second aspect there is presented an electronic device for determining multimedia codec parameters for a multimedia communications session. The electronic device comprises a processing unit. The processing unit is arranged to acquire a channel quality indicator value representing current throughput characteristics of a communications channel used for the multimedia communications session. The processing unit is arranged to compare the channel quality indicator value to a previous channel quality indicator value of the communications channel. The processing unit is arranged to determine, based on said comparing, whether to delay adaptation of values of a set of multimedia codec parameters to be used by a multimedia codec during said multimedia communications session or not, and which values of said set of multimedia codec parameters to adapt.

According to an embodiment the electronic device is an end-user equipment. According to another embodiment the electronic devices is a server.

According to a third aspect there is presented a vehicle. The vehicle

comprises an electronic device according to the second aspect. According to a fourth aspect there is presented a computer program for determining multimedia codec parameters for a multimedia communications session. The computer program comprises computer program code which, when run on an electronic device, causes the electronic device to perform a method according to the first aspect.

According to a fifth aspect there is presented a computer program product comprising a computer program according to the fourth aspect and a computer readable means on which the computer program is stored.

It is to be noted that any feature of the first, second, third, fourth, and fifth aspects may be applied to any other aspect, wherever appropriate. Likewise, any advantage of the first aspect may equally apply to the second, third, fourth and/or fifth aspect, respectively, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following detailed disclosure, from the attached dependent claims as well as from the drawings.

Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a/an/the element, apparatus, component, means, step, etc." are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

BRIEF DESCRIPTION OF THE DRAWINGS

The inventive concept is now described, by way of example, with reference to the accompanying drawings, in which:

Fig 1 schematically illustrates a communications network according to an embodiment;

Figs 2a and 2b schematically illustrate examples of electronic devices according to embodiments; Fig 3: schematically illustrates a computer program product according to an embodiment;

Fig 4 schematically illustrates a vehicle according to an embodiment;

Fig 5 schematically illustrates a video parameter mapper according to an embodiment;

Fig 6 schematically illustrates a user interface of an electronic device according to an embodiment;

Figs 7 and 8 are flowcharts of methods according to embodiments; and

Figs 9 and 10 schematically illustrate examples of throughput/ quality over time.

DETAILED DESCRIPTION

The inventive concept will now be described more fully hereinafter with reference to the accompanying drawings, in which certain embodiments of the inventive concept are shown. This inventive concept may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept to those skilled in the art. Like numbers refer to like elements throughout the description. In the flowcharts any step indicated by dashed lines should be regarded as optional.

Fig 1 shows a schematic overview of an exemplifying communications network 11 where embodiments presented herein can be applied. The communications network 11 comprises a network node (NN) 13 providing network coverage over cells (not shown). An electronic device (ED) 12a, positioned in a particular cell is thus provided network service by the network node 13 serving that particular cell. As the skilled person understands, the communications network 11 may comprise a plurality of network nodes 13 and a plurality of UEs 12a operatively connected to at least one of the plurality of network nodes 13. The network node 13 is operatively connected to a core network 14. The core network 14 may provide services and data to the electronic device 12a operatively connected to the network node 13 from an external Internet Protocol (IP) packet switched data network 15. At least parts of the communications network 11 may generally comply with any one or a combination of W-CDMA (Wideband Code Division Multiplex), LTE (Long Term Evolution), EDGE (Enhanced Data Rates for GSM Evolution, Enhanced GPRS (General Packet Radio Service)), CDMA2000 (Code

Division Multiple Access 2000), WiFi, microwave radio links, HSPA (High Speed Packet Access), etc., as long as the principles described hereinafter are applicable. An electronic device(ED) 12b may further have a wired connection to the external IP packet switched data network 15. Examples of electronic devices 12a, 12b include, but are not limited to end-user equipment such as mobile phones, tablet computers, laptop computers, and stationary computers. The electronic device 12a, 12b may alternatively be a server. In general terms, an electronic device 12a, 12b as herein disclosed may have either a wireless connection, or a wired connection, or both a wireless connection and a wired connection to the IP packet switched network 15. Hence the communications network 11 may comprise any combinations of purely wirelessly connected electronic devices 12a, purely wired connected electronic devices 12b, and electronic devices 12a, 12b with both wireless and wired connections.

One example of services and data which may be communicated through the communications system 31 is multimedia communications. In multimedia communications multimedia streams are communicated between two electronic devices 12a, 12b (such as from electronic device 12a to electronic device 12b, or vice versa) or between a server of the IP network 15 and at least one electronic device 12a, 12b (such as from the server to at least one electronic device 12a, 12b or from a at least one electronic device 12a, 12b to the server). The multimedia stream may comprise payload data in the form of audio and video content. The audio and video content may be synchronized. As the skilled person understands the multimedia streams may comprise further payload data.

Commonly, a transmitting electronic device 12a transmits a multimedia stream comprising encoded video over a non-ideal channel and a receiving electronic device 12b intended to receive the encoded multimedia stream measures the quality of the received bit stream. Currently, the results of the measurement maybe translated into a bit rate request that is provided to the transmitting electronic device 12a. For example, if the receiving electronic device 12b detects that the bit rate used for transmitting the multimedia stream most likely is too high it will transmit a bit rate request to the transmitting electronic device 12a to lower its bit rate. Common parameters to measure are inter-arrival delay, jitter and packet loss. Alternatively the receiving electronic device 12b may transmit the measurement results to the transmitting electronic device 12a, and the transmitting electronic device 12a may then by itself obtain a corresponding bit rate request from the

measurement results.

Given that the measurements indicate that there is a certain bit rate available for transmitting content of the multimedia session the transmitting electronic device 12a has to decide on how to allocate the available number of bits for a given time slot. For example, the transmitting electronic device 12a maybe arranged to determine how many bits to allocate for audio content and how many bits to allocate for video content. For example, the transmitting electronic device 12a maybe arranged to determine how many bits to allocate for different video and/or audio codec parameters. For example, the video bit rate could be lowered by using a coarser quantization in the video encoder but it can also be lowered by reducing the resolution and/or the video frame rate.

Currently the electronic devices 12a, 12b have fixed rules for how to adapt the parameters of the multimedia codec to produce a bit stream for a given bit rate. Fel! Hittar inte referenskalla.Table 1 shows an example of this. Thus, for example, for a bit rate request of 500 kb/s the resolution of the video is set to 640 by 480 pixels at a frame rate of 25 Hz. However, having such fixed rules may neither be optimal from the end-user experience perspective, nor from the bandwidth consumption of the electronic devices 12a, 12b.

Bit Rate Resolution Frame Rate

100 kb/s 320 X 240 12,5 Hz

300 kb/s 640 X 480 12,5 Hz

500 kb/s 640 X 480 25 Hz

1000 kb/s 1280 X 720 25 Hz

2000 kb/s 1920 X 1080 25 Hz Table 1: Resolution and frame rate as a function of bit rate

The embodiments disclosed herein relate to determining multimedia codec parameters for a multimedia communications session. In order to determine multimedia codec parameters for a multimedia communications session there is provided an electronic device, a method performed by the electronic device, a computer program comprising code, for example in the form of a computer program product, that when run on an electronic device, causes the electronic device to perform the method.

Fig 2a schematically illustrates, in terms of a number of functional modules, the components of an electronic device 12a, 12b according to an embodiment. A processing unit 22 is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate arrays (FPGA) etc., capable of executing software instructions stored in a computer program product 31 (as in Fig 3), e.g. in the form of a storage medium 24. Thus the processing unit 22 is thereby arranged to execute methods as herein disclosed. The storage medium 24 may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, solid state memory or even remotely mounted memory. The electronic device 12a, 12b may further comprise communications interface 23 for receiving and transmitting data to another device. The communications interface 23 may thus comprise one or more transmitters and receivers, comprising analogue and digital components and a suitable number of antennae for radio communication and/ or ports for wired communication. The electronic device 12a, 12b may further comprise a multimedia unit 25. The multimedia unit 25 may be arranged to encode and/ or render a multimedia stream, such as a video stream and/or an audio stream. The processing unit 22 controls the general operation of the electronic device 12a, 12b, e.g. by sending control signals to, and receiving control signals from, the communications interface 23 and the multimedia unit 25, and by retrieving stored data and instructions from the storage medium 24, as well as writing data to the storage medium 24. Other components, as well as the related functionality, of the electronic device 12a, 12b are omitted in order not to obscure the concepts presented herein.

Fig 2b schematically illustrates, in terms of a number of functional units, the components of an electronic device 12a, 12b according to an embodiment. The electronic device 12a, 12b of Fig 2b comprises a number of functional units; an acquiring unit 22a, a comparing unit 22b, and a determining unit 22c. The electronic device 12a, 12b of Fig 2b may further comprises a number of optional functional units, such as an adapting unit 22d. The functionality of each functional unit 22a-d will be further disclosed below in the context of which the functional units may be used. In general terms, each functional unit 22a-d maybe implemented in hardware or in software. The processing unit 22 may thus be arranged to from the storage medium 24 fetch

instructions as provided by a functional unit 22a-d and to execute these instructions, thereby performing any steps as will be disclosed hereinafter.

The electronic device 12a, 12b maybe provided as a standalone device or as a part of a further device. For example, the electronic device 12a, 12b may be provided in a vehicle 41. Fig 4 illustrates a vehicle 41 comprising at least one electronic device 12a, 12b as herein disclosed. The electronic device 12a, 12b maybe provided as an integral part of the vehicle 41. That is, the components of the electronic device 12a, 12b may be integrated with other components of the vehicle 41; some components of the vehicle 41 and the electronic device 12a, 12b maybe shared. For example, if the vehicle 41 as such comprises a processing unit, this processing unit maybe arranged to perform the actions of the processing unit 22 associated with the electronic device 12a, 12b.

Alternatively the electronic device 12a, 12b maybe provided as a separate unit in the vehicle 41. The vehicle 41 may be a vehicle for land transportation, such as a car, a truck, a motorcycle, or the like, a vehicle for water

transportation, such as a boat, a ship, a vessel, or a submarine, or the like, or a vehicle for aerial transportation, such as an aeroplane, a helicopter, or the like. Figs 9 and 10 are flow charts illustrating embodiments of methods for determining multimedia codec parameters for a multimedia communications session. The methods are performed by the electronic device 12a, 12b. The methods are advantageously provided as computer programs 32. Fig 3 shows one example of a computer program product 31 comprising computer readable means 33. On this computer readable means 33, a computer program 32 can be stored, which computer program 33 can cause the processing unit 22 and thereto operatively coupled entities and devices, such as the storage medium 24 , the communications interface 23, and/ or the multimedia unit 25 to execute methods according to embodiments described herein. The computer program 32 and/ or computer program product 31 may thus provide means for performing any steps as herein disclosed.

In the example of Fig 3, the computer program product 31 is illustrated as an optical disc, such as a CD (compact disc) or a DVD (digital versatile disc) or a Blu-Ray disc. The computer program product 31 could also be embodied as a memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM) and more particularly as a non- volatile storage medium of a device in an external memory such as a USB (Universal Serial Bus) memory. Thus, while the computer program 32 is here schematically shown as a track on the depicted optical disk, the computer program 32 can be stored in any way which is suitable for the computer program product 31.

Returning now to Fig 1, the enclosed embodiments relate to an electronic device 12a, 12b (the transmitting electronic device) that is arranged to generate a multimedia stream comprising e.g., audio and video content, such as audio-visual conversational data, and to communicate this multimedia stream over a communications channel to another electronic device 12a, 12b (the receiving electronic device).

A method for determining multimedia codec parameters for a multimedia communications session as performed by an electronic device 12a, 12b will now be disclosed. The processing unit 22 of the electronic device 12a, 12b is arranged to, in a step S106, acquire a channel quality indicator value representing current throughput characteristics of the communications channel. The acquiring maybe performed by executing functionality of the acquiring unit 22a. The computer program 32 and/or computer program product 31 may thus provide means for this acquiring. The thus acquired channel quality indicator value may be associated with a time stamp indicating the point in time at which the channel quality indicator value was indicated and then stored in the storage medium 24. The processing unit 22 of the electronic device 12a, 12b is further arranged to, in a step S108, compare the channel quality indicator value to a previous channel quality indicator value. The comparing may be performed by executing functionality of the comparing unit 22b. The computer program 32 and/or computer program product 31 may thus provide means for this comparing. This enables the electronic device 12a, 12b to determine whether or not (and if so: how much) any characteristics of the communications channel changes over time. The processing unit 22 of the electronic device 12a, 12b is further arranged to, in a step S112, determine whether to delay adaptation of values of a set of multimedia codec parameters to be used by a multimedia codec during the multimedia communications session or not. The processing unit 22 of the electronic device 12a, 12b is further arranged to, also in step S112, determine which values of the set of multimedia codec parameters to adapt. These two determinations are based on a result of the comparison made in step S108. The determining may be performed by executing functionality of the determining unit 22c. The computer program 32 and/or computer program product 31 may thus provide means for this determining.

Hence, the electronic device 12a, 12b is thereby arranged to, based on current and previous channel quality indicator values determine if, when, and how multimedia codec parameters to be used by a multimedia codec during the multimedia communications session are to be adapted. This enables adaptive adaptation of the multimedia codec parameters, and hence of the bit budget allocated to the multimedia communications session.

Embodiments relating to further details of determining multimedia codec parameters for a multimedia communications session will now be disclosed.

In general terms, the multimedia codec parameters represent parameters associated with the multimedia codec for encoding multimedia of the multimedia communications session. For example, according to an

embodiment the multimedia codec parameters represent a total bit budget to be used by the multimedia codec for encoding multimedia of the multimedia communications session. For example, according to an embodiment the multimedia codec parameters represent a frame rate to be used by the multimedia codec for encoding multimedia of the multimedia

communications session. For example, according to an embodiment the multimedia codec parameters represent a resolution to be used by the multimedia codec for encoding multimedia of the multimedia

communications session. According to an embodiment the multimedia codec parameters are a combination of a total bit budget, a frame rate, and a resolution to be used by the multimedia codec for encoding multimedia of the multimedia communications session. Adapted values of the set of multimedia codec parameters may be provided to the multimedia codec used for encoding multimedia of the multimedia communications session.

There may be different ways to determine the channel quality indicator values. Different embodiments relating thereto will now be described in turn. According to an embodiment the channel quality indicator value is based on a bit rate request and/or a measured channel throughput. The bit rate request may be generated and transmitted by the receiving electronic device 12b to the transmitting electronic device 12a. The channel throughput maybe measured and transmitted by the receiving electronic device 12b to the transmitting electronic device 12a.

As mentioned above, currently a bit rate request is generated as a function of measurements on the received data. According to the disclosed embodiments measured parameters, e.g. arrival time and packet loss, may need to be filtered and some type of hysteresis on the generation of the bit rate request may be needed to give a stable behavior. This means that the bit rate request may not only depend on a measurement made on the last received

multimedia packet of the multimedia communications session. The measurement made on the last received multimedia packet maybe regarded as a short-term history of the channel throughput. Thus, according to an embodiment the channel quality indicator value is a short-term channel indicator. There are however situations where the end-user quality can be improved by considering a larger portion of the past channel variations into account. Such measurements may be regarded as a long-term history of the channel throughput. Thus, according to an embodiment the previous channel quality indicator value thus is based on a long-term channel indicator value. According to an embodiment the channel quality indicator value thus is a short-term channel indicator and the previous channel quality indicator value thus is based on a long-term channel indicator value. Hence the short-term channel indicator values represent data in shorter time spans than the long- term channel indicator values. For example, the long-term channel quality indicator value may be based on a plurality of acquired short-term channel quality indicator values. Further, the long-term channel quality indicator may be associated with more multimedia frames, data packets, received bit rate requests, and/or channel throughput measurements pertaining to the multimedia communications session than the short-term channel quality indicator. According to an embodiment the processing unit 22 of the electronic device 12a, 12b is arranged to, in an optional step S102, acquire a long-term channel indicator value representing long-term throughput characteristics of the communications channel; and in an optional step S104 determine the previous channel quality indicator value based on the long- term channel indicator value. The acquiring may be performed by executing functionality of the acquiring unit 22a. The computer program 32 and/or computer program product 31 may thus provide means for this acquiring. The determining may be performed by executing functionality of the determining unit 22c. The computer program 32 and/or computer program product 31 may thus provide means for this determining.

The long-term channel quality indicator value may be based on at least one from a group of estimated, predicted, and occurred behaviour of the communications channel. The influence of the short term history may be handled by conventional signal processing means, for example by low pass filtering the measurement data. The long term history could be handled in the same way. Alternatives regarding how to consider the long term history will be provided below.

There may be different ways to determine if, when, and how multimedia codec parameters to be used by a multimedia codec during the multimedia communications session are to be adapted. Different embodiments relating thereto will now be described in turn.

According to an embodiment the adaptation is delayed if a large positive channel deviation is determined. In more detail, the processing unit 22 of the electronic device 12a, 12b maybe arranged to, in an optional step Sii2a, in a case the channel quality indicator value positively deviates more than a predetermined amount from said previous channel quality indicator value, adapt the values based on the channel quality indicator value once the channel quality indicator value has positively deviated more than the predetermined amount from the previous channel quality indicator value during a pre-determined amount of time. An example of this will be provided below. The adapting may be performed by executing functionality of the adapting unit 22d. The computer program 32 and/or computer program product 31 may thus provide means for this adapting.

According to an embodiment the adaptation is delayed if the channel quality is already above a certain limit. In more detail, the processing unit 22 of the electronic device 12a, 12b maybe arranged to, in an optional step Sii2b, in a case the channel quality indicator value is above a predetermined limit, adapt the values based on the channel quality indicator value once the channel quality indicator value has been above the predetermined limit during a predetermined amount of time. An example of this will be provided below. The adapting may be performed by executing functionality of the adapting unit 22d. The computer program 32 and/or computer program product 31 may thus provide means for this adapting.

According to an embodiment the adaptation is performed directly if a small positive deviation or a negative deviation (small or large) is determined. In more detail, the processing unit 22 of the electronic device 12a, 12b may be arranged to, in an optional step Sii2c, in a case a positive deviation of the channel quality indicator value is within the predetermined amount or in a case the channel quality indicator value is lower than the previous channel quality indicator value, directly adapt values of the set of multimedia codec parameters based on the channel quality indicator value. An example of this will be provided below. The adapting may be performed by executing functionality of the adapting unit 22d. The computer program 32 and/or computer program product 31 may thus provide means for this adapting.

According to an embodiment, if a large negative deviation has been determined the resolution is first kept but the frame rate is lowered directly. In more detail, the processing unit 22 of the electronic device 12a, 12b maybe l6 arranged to, in an optional step Sii2d, in a case the channel quality indicator value negatively deviates more than the predetermined amount from the previous channel quality indicator value, adapt values of the set of

multimedia codec parameters during a pre-determined amount of time. This adaptation may include, but is not limited to, lowering a frame rate and keeping a resolution of the multimedia codec. An example of this will be provided below. The adapting may be performed by executing functionality of the adapting unit 22d. The computer program 32 and/or computer program product 31 may thus provide means for this adapting.

According to an embodiment, if the large negative deviation continues the resolution is lowered and the lowered frame rate is increased again. In more detail, the processing unit 22 of the electronic device 12a, 12b maybe arranged to, in an optional step Sii2e, in a case the channel quality indicator value negatively still deviates more than the predetermined amount from the previous channel quality indicator value after the pre-determined amount of time, adapt values of the set of multimedia codec parameters. This adaptation may include, but is not limited to, increasing the lowered frame rate and lowering the resolution of the multimedia codec. An example of this will be provided below. The adapting may be performed by executing functionality of the adapting unit 22d. The computer program 32 and/or computer program product 31 may thus provide means for this adapting.

According to an embodiment, only limited changes of the video codec parameters are allowed. In more detail, the values of the set of multimedia codec parameters may be adapted less than indicated by the channel indicator value.

The determination of if, when, and how multimedia codec parameters to be used by a multimedia codec during the multimedia communications session are to be adapted may be based on further parameters. Different

embodiments relating thereto will now be described in turn. According to an embodiment the processing unit 22 of the electronic device 12a, 12b is arranged to, in an optional step Siioa, acquire multimedia content information. The multimedia content information is acquired from the multimedia unit 25. The multimedia content information may relate to movement of an object between video coding frames generated by the multimedia unit during the multimedia communications session. Such movement of an object between video coding frames maybe obtained from motion vector parameters. The multimedia content information may relate to speech activity in an audio segment recorded by the multimedia unit during the multimedia communications session. Such speech activity in an audio segment maybe obtained from voice activity detection (VAD) flags. The processing unit 22 of the electronic device 12a, 12b is then arranged to, in an optional step Sii2f, adapt the values of the set of multimedia codec parameters based also on the acquired multimedia content information. The adapting may be performed by executing functionality of the adapting unit 22d. The computer program 32 and/or computer program product 31 may thus provide means for this adapting.

According to an embodiment the processing unit 22 of the electronic device 12a, 12b is arranged to, in an optional step Snob, acquire at least one rendering parameter of the multimedia communications session. The multimedia content information is acquired from the receiving electronic device 12b rendering the received multimedia stream during the multimedia communications session. The processing unit 22 of the electronic device 12a, 12b is then arranged to, in an optional step Sii2g, adapt the values of the set of multimedia codec parameters based also on the acquired at least one rendering parameter. The adapting may be performed by executing functionality of the adapting unit 22d. The computer program 32 and/or computer program product 31 may thus provide means for this adapting.

Fig 5 schematically illustrates a video parameter mapper 51 according to an embodiment. The video parameter mapper 51 may be implemented in hardware, or software, or any combination thereof. Hence, the functionality of the video parameter mapper 51 may be stored in the storage medium 24 or l8 be part of the computer program 33 and be performed by the processing unit 22 implementing functional modules 22a, 22b, 22c, and 22c. The video parameter mapper 51 comprises inputs 52 and one output 53. The video parameter mapper 51 is arranged to as input receive a channel quality indicator value representing current throughput characteristics of the communications channel, a previous channel quality indicator value, and optionally a long-term channel indicator value, multimedia content information, or at least one rendering parameter of the multimedia communications session. The video parameter mapper 51 is arranged to as output provide control signals relating to if, when, and how to adapt values of a set of multimedia codec parameters to a multimedia codec. Hence, the video parameter mapper 51 maybe enabled to receive channel quality indicator values and to combine the channel quality indicator values with a bit rate request, information about rendering as received from the receiving electronic device 12b as well as information from speech and video encoders about activity/movement to create a parameter setting for the multimedia codec.

Table 2 shows an example of a mapping from the input parameters to the frame rate and resolution settings of the encoder. As the skilled person understands, these are just examples of multimedia codec parameters and how these can be adapted.

Output multimedia codec

Input parameters parameters

Video Speech Rendering Bit Rate Used Frame Resolution

Movement Activity Request Bit Rate

Rate

Low Yes Thumbnail 350 250 25 Hz 320X240 kb/s kb/s

High Yes Thumbnail 350 350 25 Hz 320X240 kb/s kb/s

Low No Thumbnail 350 125 12.5 320X240 kb/s kb/s Hz

High No Thumbnail 350 350 25 Hz 320X240 kb/s kb/s

Low Yes Fullscreen 500 350 25 Hz 640X360 kb/s kb/s

High Yes Fullscreen 500 500 25 Hz 640X360 kb/s kb/s

Low No Fullscreen 500 500 12.5 1280X720 kb/s kb/s Hz

High No Fullscreen 500 500 25 Hz 640X360 kb/s kb/s

Low Yes Thumbnail 1500 250 25 Hz 320X240 kb/s kb/s

High Yes Thumbnail 1500 350 25 Hz 320X240 kb/s kb/s

Low No Thumbnail 1500 125 12.5 320X240 kb/s kb/s Hz

High No Thumbnail 1500 350 25 Hz 320X240 kb/s kb/s

Low Yes Fullscreen 1500 800 25 Hz 1280X720 kb/s kb/s

High Yes Fullscreen 1500 1200 25 Hz 1280X720 kb/s kb/s Low No Fullscreen 1500 12.5 1920X1080 kb/s Hz

High No Fullscreen 1500 1200 25 Hz 1280X720 kb/s kb/s

Table 2: Example of input parameters and output multimedia codec parameters

As an example, for low video movement, speech activity, the multimedia stream being rendered as a thumbnail, and a bit rate request of 350 kb/s the multimedia codec is to encode multimedia of the multimedia session at a used bitrate of 250 kb/s, with a frame rate of 25 Hz and a resolution of 320 by 240 pixels.

Fig 6 schematically illustrates an exemplary exterior of an electronic device 12a, 12b according to an embodiment. The exterior of the electronic device 12a, 12b comprises a display screen 62 for displaying visual multimedia components, such video content, images, text, graphics, etc. The exterior of the electronic device 12a, 12b comprises a microphone 63 for recording audio. The exterior of the electronic device 12a, 12b comprises a loudspeaker 64 for playing out audio. On the display screen 62 a first visual multimedia component 65, a second visual multimedia component 66, and a third visual multimedia component 67 are illustrated. As the skilled person understands, these are just examples of visual multimedia components which may be displayed by the display screen 62. On the first visual multimedia component 65 a moving object (representing a mouth) is schematically identified at reference numeral 68. It is for illustrative purposes assumed that the first visual multimedia component 65 represents a fullscreen object, that the second visual multimedia component 66 represents a thumbnail object, and that the third visual multimedia component 67 represents a thumbnail object. It is for illustrative purposes assumed that each of the visual multimedia components represent video content of a multimedia stream. In Fig 6 h is the height of the display screen, 1 is the length of the first visual multimedia component 65, L is the width of the display, and D denotes the diameter of the display 62.

Two non-limiting scenarios where the disclosed embodiments may be applied will now be disclosed. The first scenario, as illustrated in Fig 9, is based on the understanding that short time glimpses of high quality multimedia rendering during a longer multimedia communications session with medium quality multimedia rendering may actually degrade the total end-user experience, i.e., the end- user experience as a whole when considering the complete multimedia communications session.

Fig 9 schematically illustrates a first example of available throughput 91 of the communications channel for the multimedia communications session as a function of time together with the quality 92 of the rendered multimedia content (as a consequence of the multimedia codec using certain multimedia codec values) as a function of time. Also the highest allowable quality of the rendered multimedia content is illustrated at 93.

In Fig 9 the quality is adapted to increase with the throughput if the throughput increases no more than a maximum channel deviation value. Otherwise the quality is only allowed to be increased after an adaptation delay. Further, the quality is only allowed to be increased by a maximum adaptation range. Further, the quality is only allowed to be increased to the highest allowable quality 93 - regardless of the available throughput. In order to determine if, when and how to adapt the corresponding multimedia codec parameters to be used by a multimedia codec during the multimedia communications session, the transmitting electronic device 12a is arranged to perform steps S106, S108 and S112 as outlined above. Further, the

transmitting electronic device 12a is arranged to, in the present exemplary first scenario, perform at least steps Sii2a, Sii2b and Sii2c as outlined above. In the example illustrated in Fig 9 there are three periods of high throughput; one shorter and two longer. If the bit rate of the multimedia session is adapted accordingly (i.e., to closely follow the throughput behavior also for the high throughput) the end-user's will perceive the medium quality present in the rest of the multimedia session as of low quality. In order to mitigate such a negative impact on the end-user's perception of the quality, the quality is only increased if the throughput is increased for a long period (i.e., to make sure that the peak is not just temporary), as in step Sii2a. Further, the quality is adapted so as to never exceed the highest allowable quality 93. The second scenario, as illustrated in Fig 10, is based on the understanding that during short time glimpses of reduced throughput it may, from an end- user perspective be better to lower the frame rate than the resolution.

Fig 10 schematically illustrates a second example of available throughput 101 of the communications channel for the multimedia communications session as a function of time together with the quality 102 of the rendered

multimedia content (as a consequence of the multimedia codec using certain multimedia codec values) as a function of time. Also the highest allowable quality of the rendered multimedia content is illustrated at 103. In order to determine if, when and how to adapt the corresponding multimedia codec parameters to be used by a multimedia codec during the multimedia communications session, the transmitting electronic device 12a is arranged to perform steps S106, S108 and S112 as outlined above. Further, the

transmitting electronic device 12a is arranged to, in the present exemplary second scenario, perform at least steps Sii2c, Sii2d and Sii2e as outlined above.

In the example illustrated in Fig 10 the channel throughput is high

throughout most of the duration of the multimedia communications session. There are however two shorter dip and one longer dips with significantly lower throughput. During these dips the multimedia codec parameters are adapted so as to lower the bit rate and thus the quality of the multimedia stream. A lowering of the video resolution could be a preferably choice if the throughput limitation would last for a longer period. If the lasting of the throughput dip is short it may however be better from a user perception point of view to keep the (high) resolution during the dip by lowering the frame rate, thus accepting a quality degradation due to uneven playback, Switching between resolutions, especially if this leads to a change in the video display size, may from a user experience point of view, be more annoying than temporarily lowering the frame rate. Thus, during the short dips only the frame rate is lowered, as in step Sii2c and step Sii2d, and during the long dip first the frame rate is lowered, and then the frame rate is increased but the resolution is lowered, as in step Sii2e.

In summary, there have been disclosed mechanisms that, instead of having static rules, have adaptive rules for how to spend available bits during communications of a multimedia session. Some non-limiting examples of such adaptive rules which may be suitable for multimedia sessions involving video communications will be summarized next. Firstly, if there is no or little movement in the video and no audio the adaptation may involve lowering the frame rate. The quality degradation of such a bit rate reduction may not be evident to the end-user. Secondly, if the video is not rendered in fullscreen, for example if it is only used as one of the small thumbnails as in Fig 6, the adaptation may involve lowering resolution and keeping the frame rate.

Thirdly, if there is a high level of movement and/ or audio the adaptation may involve keeping a high frame rate but lowering the bit rate by changing the quantization or reducing the resolution. As the skilled person understands, combinations of the non-limiting examples summarized above are also possible. For example, multimedia content rendered as a thumbnail with little movement may by the multimedia codec be encoded with both a low frame rate and a low resolution, depending on the content and/ or context of the multimedia communications session.

The inventive concept has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the inventive concept, as defined by the appended patent claims.

Claims

1. A method for determining multimedia codec parameters for a multimedia communications session, the method being performed by an electronic device (12a, 12b), comprising the steps of:

acquiring (S106) a channel quality indicator value representing current throughput characteristics of a communications channel used for the multimedia communications session;

comparing (S108) the channel quality indicator value to a previous channel quality indicator value of the communications channel; and

determining (S112), based on said comparing, whether to delay adaptation of values of a set of multimedia codec parameters to be used by a multimedia codec during said multimedia communications session or not, and which values of said set of multimedia codec parameters to adapt.

2. The method according to claim 1, wherein the channel quality indicator value is based on at least one from a group of a bit rate request and measured channel throughput.

3. The method according to claim 1 or 2, wherein the channel quality indicator value is a short-term channel indicator, the method further comprising:

acquiring (S102) a long-term channel indicator value representing long- term throughput characteristics of the communications channel; and

determining (S104) the previous channel quality indicator value based on the long-term channel indicator value.

4. The method according to claim 3, wherein the long-term channel quality indicator value is based on at least one from a group of estimated, predicted, and occurred behaviour of the communications channel.

5. The method according to claim 3 or 5, wherein the long-term channel quality indicator value is based on a plurality of acquired short-term channel quality indicator values.

6. The method according to any one of the preceding claims, wherein in a case the channel quality indicator value positively deviates more than a predetermined amount from said previous channel quality indicator value: adapting (Sii2a) said values based on the channel quality indicator value once the channel quality indicator value has positively deviated more than said predetermined amount from the previous channel quality indicator value during a pre-determined amount of time

7. The method according to any one of the preceding claims, wherein in a case the channel quality indicator value is above a predetermined limit:

adapting (Sii2b) said values based on the channel quality indicator value once the channel quality indicator value has been above said predetermined limit during a pre-determined amount of time.

8. The method according to any one of the preceding claims, further comprising:

in a case a positive deviation of the channel quality indicator value is within the predetermined amount or in a case the channel quality indicator value is lower than the previous channel quality indicator value: directly adapting (Sii2c) values of the set of multimedia codec parameters based on the channel quality indicator value.

9. The method according to claim 8, wherein in a case the channel quality indicator value negatively deviates more than the predetermined amount from the previous channel quality indicator value: adapting (Sii2d) values of the set of multimedia codec parameters, such as by lowering a frame rate and keeping a resolution of the multimedia codec, during a pre-determined amount of time.

10. The method according to claim 9, wherein in a case the channel quality indicator value negatively still deviates more than the predetermined amount from the previous channel quality indicator value after the pre-determined amount of time: adapting (Sii2e) values of the set of multimedia codec parameters, such as by increasing the lowered frame rate and lowering the resolution of the multimedia codec.

11. The method according to any one of the preceding claims, wherein the multimedia codec parameters represent at least one from a group of total bit budget, frame rate, and resolution to be used by the multimedia codec for encoding multimedia of the multimedia communications session.

12. The method according to any one of the preceding claims, wherein said values of the set of multimedia codec parameters are adapted less than indicated by the channel indicator value.

13. The method according to any one of the preceding claims, further comprising:

acquiring (Snoa) multimedia content information; and

adapting (Sii2f) said values of the set of multimedia codec parameters based also on the acquired multimedia content information.

14. The method according to any one of the preceding claims, further comprising:

acquiring (Snob) at least one rendering parameter of the multimedia communications session; and

adapting (Sii2g) said values of the set of multimedia codec parameters based also on the acquired at least one rendering parameter.

15. An electronic device (12a, 12b) for determining multimedia codec parameters for a multimedia communications session, the electronic device comprising a processing unit (22) arranged to:

acquire a channel quality indicator value representing current throughput characteristics of a communications channel used for the multimedia communications session;

compare the channel quality indicator value to a previous channel quality indicator value of the communications channel; and

determine, based on said comparing, whether to delay adaptation of values of a set of multimedia codec parameters to be used by a multimedia codec during said multimedia communications session or not, and which values of said set of multimedia codec parameters to adapt.

16. The electronic device according to claim 15, wherein the electronic device is one of an end-user equipment and a server.

17. A vehicle (41) comprising an electronic device according to claim 15 or 16.

18. A computer program (32) for determining multimedia codec

parameters for a multimedia communications session, the computer program comprising computer program code which, when run on an electric device (12a, 12b), causes the electric device to:

acquire (S106) a channel quality indicator value representing current throughput characteristics of a communications channel used for the multimedia communications session;

compare (S108) the channel quality indicator value to a previous channel quality indicator value of the communications channel; and

determine (S112), based on said comparing, whether to delay adaptation of values of a set of multimedia codec parameters to be used by a multimedia codec during said multimedia communications session or not, and which values of said set of multimedia codec parameters to adapt.

19. A computer program product (31) comprising a computer program (32) according to claim 18 and a computer readable means (33) on which the computer program is stored.