WO2009067057A1

WO2009067057A1 - Method and apparatus for assessing service performance

Info

Publication number: WO2009067057A1
Application number: PCT/SE2007/050872
Authority: WO
Inventors: Jan Holm; Per Synnergren
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2007-11-20
Filing date: 2007-11-20
Publication date: 2009-05-28

Abstract

Method and apparatus for enabling assessment of the performance of an IP service consumed or utilised by means of a user terminal (200). When a service trigger is received, the user terminal measures (2:1) a predefined service performance indicator according to a predefined measurement procedure. The performance indicator reflects the user experience when consuming the IP service. When the measurement has been completed, the user terminal reports (2:2) measurement results to a measurement controller (210) in a service message sent from the user terminal in connection with the IP service. Thereby, the performance of an IP service as experienced by the user can be assessed without requiring any network-based measurements.

Description

METHOD AND APPARATUS FOR ASSESSING SERVICE PERFORMANCE.

TECHNICAL FIELD

The present invention relates generally to a method and arrangement for enabling assessment or evaluation of the performance of IP based multimedia services in communication networks based on measured service performance parameters or indicators.

BACKGROUND

A multitude of different types of communication terminals or devices have been developed for packet-based multimedia communication using IP (Internet Protocol) . Multimedia services typically involve the transmission of media in different formats and combinations over IP networks. For example, an IP enabled mobile terminal may exchange media such as visual and/or audio information with another IP enabled terminal, or may download media from a content server over the Internet. In recent times, the general trend in the telecommunication business has been to offer an extensive amount of more or less advanced IP based multimedia services, in addition to the traditional circuit-switched telephony services and basic interface to the Internet. Hence, a network architecture called IMS (IP Multimedia Subsystem) has been developed by the 3^rd Generation Partnership Project (3GPP) as a platform for handling and controlling multimedia services and sessions, commonly referred to as the IMS network. Thus, an IMS network can be used to set up and control multimedia sessions for "IMS enabled" terminals connected to various access networks, regardless of the access technology used. Although conceived primarily to enable multimedia services for mobile IP terminals, the IMS concept can also be used for fixed IP terminals. Examples of services that can be implemented by using IMS are MMTeI (Multimedia Telephony) and PoC (Push-to-talk over Cellular) . Multimedia sessions are handled by specific session control nodes in the IMS network, e.g. the nodes P-CSCF (Proxy Call Session Control Function), S-CSCF (Serving Call Session Control Function), and I-CSCF (Interrogating Call Session Control Function) . Further, a database node HSS (Home Subscriber Server) is used in the IMS network for storing subscriber and authentication data. The IMS network may also include various application servers and/or be connected to external ones, for providing different multimedia services or IP services.

Operators of cellular or mobile access networks, hereafter referred to simply as "mobile networks", typically monitor the network performance with respect to various characteristics and aspects, to generally assess and diagnose their networks. Today, various solutions and systems are known for locating any performance-related shortcomings, and for configuring and optimising different parameters in the radio access and core network parts of a mobile network impacting the general performance. Moreover, the network performance should be monitored also to ensure that various obligations towards subscribers such as service level agreements are met, and that expected levels of QoS (Quality of Service) are fulfilled.

Performance monitoring is typically handled in mobile networks by a function called OSS (Operation and Support System) . Practically all nodes in the mobile network, e.g. Node B and RNC (Radio Network Controller), can be configured to provide more or less performance-related information to the OSS or other similar systems over specific interfaces. Various sensors and counters are thus installed in the different network nodes, in order to collect measurements of relevant performance-related parameters in each node during operation. In this context, a performance-related parameter is often referred to as KPI (Key Performance Indicator) . The data throughput and the success rate for establishing a radio bearer are typical examples of such performance-related parameters or KPIs.

Each sensor or counter in the network nodes is configured to frequently send measurement results, i.e. measured KPI values, to the OSS. A suitable software in the OSS will then derive relevant information from the received measurements, adapted for presentation on a navigator tool or the like to operational staff working with network management, to provide an overview of the current network performance. In practice, a relatively great number of sensors and counters are required in a plurality of nodes in order to achieve an adequate view of the network.

The development towards IP and IMS based multimedia services mentioned above, hereafter referred to simply as "IP services", will most likely require additional functionality for the new services in the performance monitoring systems in order to assess and control the service performance properly. The total performance in such services typically depends on the performance in one or more individual service systems and entities that are used in addition to the access network, such as nodes associated with the IMS system and any utilised application servers.

The performance monitoring systems of today are mainly adapted for monitoring specific nodes, parts and subsystems in access networks such as mobile networks. Sensors and counters frequently report KPI values to the OSS individually from each monitored node. This provides for a relatively accurate assessment of the network performance since the network-related KPIs have often been defined to be measured by using sensors/counters in only a single node, as in the case of data throughput and success rate for establishing a radio bearer. The reported KPI may then be a more or less accurate indication of the performance with respect to the available bandwidth and accessibility in the network .

However, when it comes to multimedia and IP services, it is desirable to define KPIs that are more "service-oriented" in the sense of reflecting the performance or quality of the IP service as perceived by the user, in contrast to the "network-oriented" solutions above. As the performance of these IP services is typically dependent on a combination of networks, nodes, elements and servers, it is a problem that complexity must be introduced in order to aggregate and combine measurements from plural sensors and counters at different locations throughout the networks and nodes involved. Hence, obtaining relevant service-related and user-oriented KPIs for a multitude of different IP services is certainly a difficult task, also requiring considerable amounts of processing and signalling capacity. It is a further problem that even if sensor and counter measurements are collected from plural networks, nodes, elements and servers as described above, the resulting total KPI will typically only give a rough estimation of the actual user experience.

Some examples of relevant service-related KPIs reflecting the user experience for a typical IP service are: - Session set-up latency (i.e. the waiting time before the service is started, ready for use) .

- Session set-up success rate (i.e. the probability of starting the service successfully) .

- Media latency (i.e. the end-to-end latency for the transport of media) .

- Packet loss rate (i.e. the amount of media lost in the transport end-to-end) .

Some IP services, e.g. VoIP (Voice over IP), entail the communication of voice data between two end-users who may, e.g., be connected to different mobile networks. In that case, two mobile networks, two core networks, two service layers and two user terminals are typically involved in the session setup and media transport. Therefore, relevant user-perceived service KPIs including, e.g., the session set-up latency and media latency, cannot be determined by using counters in only one network node, being dependent on the sum of performances in all the above- mentioned subsystems.

This situation is illustrated in Fig. 1 where two mobile terminals IOOA and IOOB are involved in a VoIP session. Terminal IOOA is connected to a mobile network 102A (which includes a radio network part and a core network part) and uses an IMS core 104A and a VoIP application server 106A to initiate and control the session. Likewise, terminal B is connected to a mobile network 102B (also including a radio network part and a core network part) , where an IMS core 104B and corresponding application server 106B controls the session on behalf of terminal IOOB. The actual voice data is communicated over a transport network 108 situated between the mobile networks 102A and 102B.

An OSS node 110 is used for evaluating the performance of different services, including the VoIP service utilized by terminal 10OA. The various dashed arrows in the figure illustrate that KPI measurements are reported to the OSS node 110 from different sensors and counters in different nodes of the mobile network 102A and IMS core 104A, in the application server 106A, and possibly also in the transport network 108. It can be readily understood from this picture that multiple interfaces are required to the OSS node 110, and that a complex assessment logic is needed in OSS node 110 to aggregate the different measurements to provide a relevant view of the service performance. Even though the systems of today are capable of monitoring the performance of all the individual network nodes involved in an IP based service, it is difficult, if not impossible, to derive the actual experienced performance by aggregating different measured statistics into service KPI statistics. It is therefore a further problem that the current performance monitoring systems are not capable of monitoring the actual experienced service performance of IP- based services.

SUMMARY

It is an object of the present invention to address the problems outlined above. Further, it is an object to provide a solution that enables assessment of the performance of an IP service. These objects and others may be obtained by providing a method and apparatus according to the independent claims attached below. According to one aspect, a method is provided for enabling assessment of the performance of an IP service consumed or utilised by means of a user terminal. When receiving a service trigger, the user terminal measures a predefined performance indicator of the IP service according to a predefined measurement procedure, the performance indicator being indicative of the user experience when consuming or utilising the IP service. The user terminal then reports measurement results to a measurement controller in a service message sent from the user terminal in connection with the IP service.

According to another aspect, an apparatus in a user terminal is provided for enabling assessment of the performance of an IP service consumed or utilised by means of the user terminal. The user terminal apparatus comprises a receiving unit adapted to receive a service trigger, a measuring unit adapted to measure a predefined performance indicator associated with the IP service according to a predefined measurement procedure, and a reporting unit adapted to report measurement results to a measurement controller in a service message sent from the user terminal in connection with the IP service.

Different embodiments are possible in the method and apparatus in the user terminal above. For example, the measuring unit may measure the performance indicator in response to receiving a measurement order in a service message sent from the measurement controller in connection with the IP service, e.g. the service trigger. The measuring unit may also measure the performance indicator automatically in response to receiving the service trigger, and report the measurement results to the measurement controller if a measurement order has been received from the measurement controller. In that case, the user terminal may have been configured with an order to measure the performance indicator and report to the measurement controller whenever the IP service is consumed or utilised. The predefined measurement procedure may dictate that the measurement of the performance indicator starts when the service trigger is received and ends when a service or measurement termination trigger is received, or that the measurement of the performance indicator should be performed periodically when the IP service is consumed or utilised, or that the performance indicator should be measured according to certain preconditions. The preconditions may include any of: measure the performance indicator when the user terminal is located in a certain area; measure the performance indicator during a specific time of day, week or season; and measure the performance indicator every n : th time the IP service is consumed or utilised, n being an integer >1. The service message containing the measurement report may be a session terminating message. If SIP or MBCP is used when the IP service is consumed or utilised the measurement results can be provided in an XML document embedded in a SIP or MBCP message, respectively. If RTCP is used when the IP service is consumed or utilised the measurement results can be provided in a binary format in a field of an RTCP message.

The performance indicator may relate to experienced media quality and/or experienced latency/delay. The measuring unit may measure the performance indicator by detecting signals conveyed on an internal interface between an IMS client and a communication unit in the user terminal. Further possible features and benefits of the present invention will be explained in the detailed description below.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be explained in more detail by means of exemplary embodiments and with reference to the accompanying drawings, in which:

Fig. 1 is a schematic view illustrating the monitoring of service performance of a VoIP session for two mobile terminals, according to the prior art.

Fig. 2 is a schematic view illustrating the monitoring of service performance of an IP service utilized by a mobile terminal, in accordance with one embodiment. - Fig. 3 is a flow chart illustrating a procedure for enabling assessment of the performance of an IP service, in accordance with another embodiment.

Fig. 4 is a flow chart illustrating a slightly modified procedure for enabling assessment of the performance of an IP service, in accordance with yet another embodiment.

Fig. 5 is a signalling diagram illustrating a first example of measuring a performance indicator, in accordance with yet another embodiment.

Fig. 6 is a signalling diagram illustrating a second example of measuring a performance indicator, in accordance with yet another embodiment.

Fig. 7 is a signalling diagram illustrating a third example of measuring a performance indicator, in accordance with yet another embodiment. - Fig. 8 is a signalling diagram illustrating a fourth example of measuring a performance indicator, in accordance with yet another embodiment. Fig. 9 is a block diagram illustrating an apparatus in a user terminal in more detail, according to yet another embodiment .

DETAILED DESCRIPTION

Briefly described, the present invention can be used for monitoring, assessing or evaluating the performance of any IP services utilised or consumed by means of a user terminal. One or more service performance parameters or indicators, hereafter referred to as a "performance indicator" for short, are measured by the user terminal, instead of by network nodes as described above. Thereby, the measurement will more closely reflect the user' s experience of the service, without requiring the aggregation of measurements from plural sensors and counters, particularly where plural network nodes and elements influence the measured performance indicator. Further, the measurement results can be provided from the terminal in a simple manner by means of an existing protocol over an existing communication interface.

The terminal initially receives an instruction from a measurement controller or the like in the service provider' s domain, to execute a measurement of a predefined performance indicator when an IP service is initiated. In response thereto, the terminal measures the performance indicator related to the service until the service is terminated, or until a predetermined measurement period or phase is completed. The terminal then reports the measurement results to the measurement controller. Thus, the term "performance indicator" is used in this description to generally represent any measurable performance-related parameter that in some way reflects the user experience when consuming an IP service, such as the above-mentioned KPIs. For example, a performance indicator in this context may relate to any of the above-mentioned session set-up latency, session set-up success rate, media latency, media quality, packet loss rate, or any other possible parameters. Typically, the service performance basically relates to experienced media quality or experienced latency/delay. Even though the following examples and embodiments refer to a single performance indicator, the present invention is not limited thereto and may involve the measurement of any number of performance indicators .

Further, the term "measurement controller" is used here to represent a logic entity or function adapted to order the user terminal to execute and report a performance indicator measurement, and also adapted to receive the measurement results from the terminal. The measurement controller may reside in an application server responsible for the service, or in a node in the IMS network involved in a session for the service, such as the above-mentioned session control nodes.

Fig. 2 illustrates how the performance of an IP service can be monitored, in accordance with one embodiment. In this example, a user terminal 200 connected to an access network 202 utilises or consumes the IP service involving communication of media with another party over network 202 and possibly also over other transport networks, not shown. The access network 202 may be any type of network for wireless or fixed access, such as a mobile network, WLAN (Wireless Local Area Network) , WiMax, or fixed broadband networks including networks using optical access or DSL (Digital Subscriber Line) . The service is offered and controlled by an IMS core 204 and an application server 206 connected thereto, in this case both belonging to a service provider's domain. A service monitoring node 208 is generally configured to monitor the performance of various IP services based on performance indicator measurements, to present the results to operational staff and derive various statistics and conclusions, basically in the manner of the above-described OSS system. The service monitoring node 208 may thus monitor a multitude of different IP services offered from plural application servers, including application server 206, or otherwise. In this example, a measurement controller 210 resides in the application server 206. Alternatively, the measurement controller 210 may reside in a suitable node

(e.g. a call session control function node) of the IMS core 204, as indicated with dashed lines in the figure. Both the application server 206 and the IMS core 204 belong to the IMS service provider' s domain and both are involved to communicate with the user terminal for executing the IP service .

Any signalling messages to and from the terminal 200 when the IP service is utilised/consumed, pass through the radio and core parts of the access network 202, and through the IMS core 204. For services that need specific application logic, such as PoC and MMTeI, the signalling messages also passes through or are even terminated in the application server 206 where the application specific logic is typically implemented. The other nodes in access network 202 and IMS core 204 are commonly used by any implemented IP services in the network. Therefore, the application server 206 is a suitable node to provide a main interface to the service monitoring node 208 for this service. However, the IMS core 204 may also be suitable as the interface to node 208 due to the close connection with the application server 206. It should be noted that already today, application servers and IMS cores generally have interfaces to performance monitoring systems or the like. However, the information sent over these interfaces are measurements executed by the application servers and IMS cores themselves. In this solution, these interfaces can be used to provide measurements of more service-oriented measurements made by user terminals, to enable assessment of the actual service performance.

For this particular IP service, a specific performance indicator has been predefined as a relevant representation of the user' s experience of the service when executed. In addition, a specific measurement method or procedure has been predefined for the performance indicator. It is assumed that terminal 200 has somehow received an order to measure the performance indicator in the predetermined manner. The measurement order may be sent from the measurement controller 210 to terminal 200 in connection with a session setup procedure or a registration procedure, although the terminal may be ordered or instructed in other ways to measure.

For example, terminal 200 may have been configured with an order to measure the performance indicator and report to the measurement controller 210 whenever the IP service is consumed. Moreover, certain preconditions may also have been defined for executing measurements on a specific performance indicator. Exemplary preconditions are that measurements should be executed when terminal 200 is located in certain areas such as within a home network, or during a specific time of day, week or season, or every n : th time the service is consumed, n being an integer >1, etc. When such preconditions are applied, no specific order is needed from the measurement controller 210 to activate each measurement, since terminal 200 is "preconfigured" with a conditioned measurement order.

Furthermore, the terminal 200 may have been ordered to make a series of predefined measurements, e.g. once every hour, minute or second, and send the measurements to the measurement controller 210 in succession until the service is terminated. The measurement periods may then be time-limited. Thus, throughout the description, the phrase "receiving a measurement order" is intended to cover any of the above examples.

In a first shown stage 2:1, the terminal 200 accordingly executes the performance indicator measurement according to the predefined procedure when utilising/ consuming the IP service, either at session setup or during the communication of media, or both. The measurement may be commenced when receiving a service trigger, which may be an input command from the user or the reception of a specific message or media, depending on the nature of the performance indicator. Likewise, the measurement may be terminated when receiving a service termination trigger, or according to a predetermined measurement scheme or the like.

In a second stage 2:2, the terminal 200 reports the measurement results to the measurement controller 210, either located in the application server 206 (according to full arrows) or in the IMS core 204 (according to dashed arrows) . Measurement controller 210 then finally sends the measurement results on to the service monitoring node 208, in a last shown stage 2:3.

A flow chart in Fig. 3 illustrates a procedure for enabling assessment of the performance of an IP service consumed by means of a user terminal, in accordance with another embodiment. In a first step 300, an order to measure a specific performance indicator associated with the IP service and according to a predefined measurement procedure, is received at the terminal. This measurement order can be received in different ways, either from a measurement controller or otherwise, as exemplified above for Fig. 2.

In a next step 302, a "service trigger" is received indicating that the IP service is somehow activated or started. If the terminal is an originating (or calling) party, the service trigger may be an input command from the user such as when pressing a "send" button on the terminal for starting a VoIP call or media session, or when generally requesting the IP service in one way or another. If the terminal is a terminating (or called) party, the service trigger may be the reception of a specific message, e.g. a session invite message, depending on the nature of the service and/or performance indicator to be measured.

In a further step 304, the terminal executes the measurement of the performance indicator accordingly. The predefined measured performance indicator may have been predefined by the service provider or standardised by, e.g., ETSI (European Telecommunications Standards Institute) or 3GPP. Further, a measurement method or scheme and/or any prevailing preconditions for making a measurement, may also have been predefined for the performance indicator, either by the service provider or generally standardised. In a next step 306, a "service or measurement termination trigger" may be received indicating either that the IP service has been terminated in some respect or that the measurement is to be terminated even though the actual service may continue to be used, depending on how the measurement procedure or scheme is defined for the performance indicator.

Finally, the terminal reports the measurement results to the measurement controller, in a last step 308. The measurement results are reported preferably by means of a protocol used anyway when executing the IP service, over an existing interface between the terminal and the node where the measurement controller is implemented, e.g. the application server or an IMS node. The measurement controller can then pass the measurement results on to a service monitoring system or the like. A dashed arrow back to step 304 indicates that the measuring and reporting may continue as long as the service is consumed, if the predefined measurement procedure requires timer-based measurements and periodic measurement reports when the timer expires. In that case, the timer acts as the measurement termination trigger of step 306.

According to different alternatives, the protocols SIP (Session Initiation Protocol) and RTCP (Real Time Control Protocol) may be used to transport the measurement information, e.g. contained in a specific document or otherwise, depending on the nature of the IP service. SIP is generally utilised to initiate and control IMS sessions. Standard SIP messages include the session initiating message "SIP INVITE", the response message "SIP 200 OK". and the session termination message "SIP BYE" For SIP, the measurement information can be sent in a SIP BYE message when terminating an IMS session, i.e. when the IP service has been completed such as when the user enters a service termination command. The measurement information may be accommodated in an XML (extended Markup Language) document that can be embedded in suitable messages of the above protocols, e.g. in the SIP BYE message. For a PoC service using the protocol MBCP (Media Burst Control Protocol), the measurement information can be provided in an MBCP message, e.g. accommodated in an XML document embedded in the MBCP message. For a service using RTCP, the measurement information can be provided in a binary format in a suitable field of an RTCP message. More detailed examples of transporting the measurement information to the measurement controller will be described further below.

A slightly different procedure, in accordance with yet another embodiment, for enabling assessment of the performance of an IP service consumed by means of a user terminal, is illustrated by a flow chart in Fig. 4. Here, it is assumed that the terminal is configured to start measuring a specific performance indicator associated with the IP service regardless of whether an order has been received beforehand.

In some situations, it is desirable to start measuring a performance indicator before any explicit measurement order can be conveyed to the terminal from the measurement controller. Thus, the first opportunity to convey a measurement order to the terminal may not occur until after the service has been started and after a predefined performance indicator measurement should already have been commenced. This situation may occur if the user terminal is an originating party, e.g. when the measurement should start right at the moment of receiving an initial input command from the user, that is before any service messages are received from the application server or IMS core . In a first step 400, a service trigger or the like is received indicating that the IP service is somehow started, as similar to step 302 in Fig. 3. In response thereto, the terminal executes the measurement of the performance indicator accordingly, in a next step 402. At some point during the measurement, a measurement order may be received from the measurement controller, preferably in a suitable message according to the executed IP service. Nevertheless, the terminal continues to measure the performance indicator until a service or measurement termination trigger is received in a further step 304, as similar to step 306 in Fig. 3. It is then determined in a step 406 whether a measurement order has been received or not. If a measurement order has been received, the terminal reports the measurement results to the measurement controller, in a step 408. The measurement controller can then pass the measurement results on to a service monitoring system or the like. However, if no measurement order has been received, the measurement results are simply discarded in a step 410. A user terminal may thus receive an order to measure a specific predetermined performance indicator according to a predetermined measurement method/scheme and possibly also being subject to certain preconditions, according to different possible alternatives. Thus, the measurement order may be received in connection with a registration procedure with the IMS core. The order may then be conveyed to the terminal embedded in a suitable registration message, such as a SIP 200 OK message issued in response to a SIP REGISTER message from the terminal. In that case, the terminal will do the measurement upon receiving a service trigger, when being either an originating party or a terminating party. The terminal will thus start the measurement after receiving the measurement order, as in the procedure of Fig. 3.

Alternatively, the user terminal may receive the measurement order during a session setup procedure. If the terminal is an originating party, the measurement order may be conveyed embedded in a suitable session setup message such as a SIP 200 OK message received in response to a SIP INVITE message from the terminal. In that case, the terminal may be configured to start the measurement before receiving the measurement order, as in the procedure of Fig. 4. If the terminal is a terminating party, the measurement order may be embedded in a session setup message such as a SIP INVITE message conveyed over the IMS core, coming from the originating party. In that case, a measurement controller in the application server or in a session control node of the IMS core can insert the measurement order in the SIP INVITE message sent to the terminal. The terminal will thus start the measurement after receiving the measurement order, as in the procedure of Fig. 3. In fact, for some performance indicators, e.g. the session setup latency, the SIP INVITE message may contain a measurement order and at the same time be a service trigger starting the measurement, such that steps 300 and 302 are executed simultaneously. As mentioned above, the measurement information reported from the terminal may be accommodated in an XML document, embedded in a suitable message from the terminal to the measurement controller in a communication executed when utilising/consuming the service. The XML document can preferably be specified in a standardization body, e.g. according to the OMA (Open Mobile Alliance) . An exemplary XML document could be configured as follows:

<?xml version="l .0" encoding="UTF-8" ?> <qoe_monitoring xmlns="urn : ietf :params : xml : ns :pidf" xmlns :pm="urn : oma : xml :prs :pidf : qoe" entity="sip : someoneΘexample . com"> <tuple id="al232">

<pm:kpil >4.36</op : kpil> <pm: kpi2>success</op : kpi2> <pm:kpi3>0.98</op:kpi2> <pm: kpi4>N/A</op: kpi3>

<pm: service-description>

<pm: service-id>org .3gpp : MMTeK/op : service-id> <pm: version>l .0</op:version> </pm: service-description> <contact>sip : someoneΘexample . com</contact> <timestamp>2005-02-22T10 :25: 01Z</timestamp> </tuple> </qoe monitoring >

In the XML document, the monitored service and the measurement results are specified as "service-id" and "kpi", respectively. In this example, the service is "3GPP MMTeI" and the performance indicators contained in the document are "kpil-kpi4". The performance indicators are also preferably standardized and in the example above, KPIl, KPI2 and KPI3 have been measured, whereas KPI4 has not been measured. In this example, KPIl is the session set-up latency reported as a floating point value "4.36", KPI2 is the success rate of session establishment reported as "success", and KPI3 is the packet reception percentage reported as "98%". KPI4 may be a performance indicator of a service aspect that was not utilised during this particular communication session. For example, KPI4 may be the messaging success rate, and this session may have involved voice communication only. Therefore, KPI4 is reported as "N/A".

Fig. 5 is a signalling diagram illustrating how a performance indicator for an IP service can be measured by a user terminal 500 in a regular service procedure, according to a first example. The performance indicator to be measured relates to session set-up latency, in this case the session setup time for a multimedia telephony session, i.e. the MMTeI service, with an opposite party (not shown) as initiated by the user of terminal 500. Terminal 500 is thus the originating party and the opposite party is the terminating party connected to a terminating network.

The user terminal 500 is logically divided into an IMS client part 500a and a communication unit part 500b, the IMS client 500a acting as an interface towards the user for this particular service. The terminal 500 is connected to an access network 502, while the multimedia telephony session and MMTeI service are handled by an IMS core 504 and an application server 506, respectively.

Performance indicators can generally be measured by detecting signals conveyed on an internal interface "x" between the IMS client 500a and the communication unit 500b, which is a location in terminal 500 where service actions experienced by the user can be detected, such as input commands and various messages or signals to the user. In this example, the session setup time is basically measured by starting a timer when the user initiates the service and stopping the timer when the user is notified that the called party has been alerted. Various arrows in the figure illustrate different steps in the procedure, each of which may involve one or more individual messages back and forth depending on the implementation.

A first step 5:1 illustrates that the user makes an input command to initiate the service at the IMS client 500a, e.g. by pressing a call button or similar. In a next step 5:2, an internal session initiating signal is sent from the IMS client 500a to the communication unit 500b, which is basically a service trigger in the sense described above. Measurement of the performance indicator is started at this point by detecting the internal session initiating signal. In this example, the terminal is configured to measure the performance indicator and report the measurement results if a measurement order has been received, either before or during the process.

Next, the communication unit 500b obtains a connection with the access network 502 for a session setup procedure, in a step 5:3, which typically involves the communication of plural messages between terminal 500 and network 502, also requiring signalling within the network

502 to establish network and radio resources for the session setup. Then, communication unit 500b exchanges various SIP messages with the application server 506 over a session control unit in the IMS core 504, generally indicated as "SIP signalling" in a further step 5:4.

Although not explicitly shown here, this step comprises a SIP INVITE message sent from the terminal towards the opposite party, and a SIP 183 session progress message returned to the terminal from the terminating network. Both messages are conveyed over the IMS core 504 and the application server 506. The terminal 500 then obtains further network and radio resources in network 502 for the session, as indicated in a step 5:5 of "resource reservation" involving further messaging between terminal 500 and network 502.

Further SIP signalling is executed in a step 5:6 including a SIP PRACK message from the terminal 500 to the terminating network, a SIP 200 OK (PRACK) response message back to the terminal, a SIP UPDATE message from the terminal to the terminating network, and a SIP 200 OK (UPDATE) response message back to the terminal, and so forth. These messages are likewise conveyed over the IMS core 504 and the application server 506.

Finally, terminal 500 receives a SIP 180 ringing message from the terminating network, in a next step 5:7, indicating that the called opposite party has been alerted. Alternatively, if the terminating party uses automatic answer, terminal 500 will receive a notification in a SIP 200 OK (INVITE) message that the session has been set-up, instead of the SIP 180 ringing message.

A notification signal or the like is then conveyed in a final step 5:8 over the internal interface x from communication unit 500b to IMS client 500a, to provide an alert or session setup notification to the user. When the notification signal is detected on the internal interface x, the timer is stopped to complete the performance indicator measurement. The notification signal of step 5:8 is thus effectively a service or measurement termination trigger to stop the measurement. Thereby, the total time T has been measured as the performance indicator of session set-up latency. Since the measurement is made in the terminal more or less at the user level, the latency of all underlying network procedures will be included, such as the time to establish the connection to, e.g., a radio access network (step 5:3 connect) and the time to establish additional bearers for media (step 5:5 resource reservation) .

The terminal 500 can then report the measurement results to a measurement controller (not shown) residing either in the application server 506 or in a session control unit of the IMS core 504, in a service message sent in connection with the session at a later point. For example, the measurement results may be delivered in an XML document embedded in a session terminating message, e.g. the SIP BYE message, as described above.

It should be noted that terminal 500 may receive a measurement order from the measurement controller in connection with the IP service at any time during the above- described process, or otherwise. Alternatively, terminal 500 may have been configured to perform and report a measurement of the performance indicator automatically whenever the IP service is consumed or utilised, e.g. according to prevailing preconditions, without requiring a measurement order during the process or session.

Fig. 6 is a signalling diagram illustrating how a performance indicator for an IP service can be measured and reported by a user terminal 600, according to a second example. The performance indicator is again the session set- up latency, in this case for a PoC session with an opposite party. As similar to Fig. 5, the performance indicator is measured by detecting signals conveyed on the internal interface x between an IMS client 600a and a communication unit 600b of terminal 600, being connected to an access network 602. The PoC session is handled by an IMS core 604 and an application server 606, respectively. In a first step 6:1, the PoC session is generally established over a pre-established session, not shown. In a next step 6:2, the user pushes a call button to initiate a talk burst. In a next step 6:3, an internal session initiating signal is sent from the IMS client 600a to the communication unit 600b, as a service trigger initiating the talk burst which is detected on interface x to start measurement of the performance indicator at this point.

Next, the communication unit 600b obtains an uplink connection with the access network 602 for the talk burst, in a further step 6:4, involving an establish uplink request from the terminal 600 and an establish uplink response from the network 602. Thereafter, the communication unit 600b sends a message called "media burst request" to application server 606 according to the protocol MBCP, in a further step 6:5. In response thereto, communication unit 600b receives an MBCP message called "media burst granted" from application server 606, in a following step 6:6.

Finally, a signal "permission to speak" is conveyed over the internal interface x to the IMS client 600a in a step 6:7, resulting in a notification to the user. The permission-to-speak signal is detected at the internal interface x, which is effectively a service or measurement termination trigger to stop the timer, thereby completing the measurement. Thus, the total time T has been measured as the performance indicator of session set-up latency. The latency measurement is thus made in the terminal as precisely perceived by the user, without requiring any network measurements. The terminal 600 can then report the measurement results, i.e. the session set-up latency T, to a measurement controller.

The user terminal is thus somehow ordered to measure a performance indicator and the results are somehow delivered to a measurement controller residing within the operator's network. As mentioned above, one option is to use the protocol SIP, if used for executing the IP service, for both sending a measurement order to the terminal and sending measurement results from the terminal. In that case, a measurement order may be conveyed to the terminal by means of a SIP header called "Supported/Required" according to RFC 3261. A possible encoding of the Supported/Required header would be "performance-measurement" or the like, and the measurement order can additionally be negotiated in a so- called SDP (Service Description Protocol) message if a measurement for a specific media type is required. The "Supported" header is typically included in various SIP requests, e.g. SIP INVITE, SIP REGISTER or SIP REFER, and the "Required" header is typically included in the request response message SIP 200 OK.

Measurement results can be conveyed to the measurement controller either by using a new SIP header, e.g. a private header (P-header) extension to SIP, or by using the so-called "Event package". A possible encoding of the P-header may be "P-Quality-of-Experience-Info" or the like.

For example, the application server or a session control node in the IMS core may send the measurement order at IMS registration, IMS session set-up or at PoC Session establishment . Fig. 7 is a signalling diagram illustrating how a measurement order is conveyed during IMS registration and measurements results are conveyed in a SIP BYE message, according to a third example. A user terminal 700 will thus register with an IMS core and activate an IP service in an application server, the IMS core/application server 702 being illustrated jointly in this figure for simplicity. The measurement results will finally be delivered to an OSS node 704. Further, a measurement controller, not shown, resides in the IMS core/application server 702.

In a first step 7:1, the user makes an input command to basically power on the terminal 700. In a next step 7:2, the terminal 700 sends a SIP REGISTER message to the IMS core/application server 702 and includes the Supported header in the message, indicating that the terminal generally supports performance indicator measurements .

The measurement controller in IMS core/application server 702 then effectively orders the terminal 700 to measure a specific performance indicator according to a predefined measurement procedure and to report the measurement results to the IMS core/application server, by including the Required header in the SIP 200 OK response of a next step 7:3. Upon receiving the measurement order, the terminal 700 enables its measurement procedures accordingly. As mentioned earlier, the terminal may at this step of course be ordered to measure more than one performance indicator, although the examples throughout this description refer to a performance indicator for simplicity. A "ready notification" is then issued at the terminal in a further step 7:4, notifying the user that a session or service can be initiated. In response to another user input at step 7:5 for activating the IP service, a session setup procedure is conducted with IMS core/ application server 702 in a following step 7:6. A session is then executed according to the activated IP service, in a next step 7:7. At any time during the session setup and session execution, the terminal 700 will measure the performance indicator according to the predefined measurement procedure, which is schematically indicated by the dashed two-way arrow "M". In a further user input command in terminal 700 at step 7:8, the user terminates the session. Accordingly, the terminal 700 sends a SIP BYE message to IMS core/application server 702 attaching the measurement results thereto, in a next step 7:9. When the measurement controller in IMS core/application server 702 receives the SIP BYE message, the performance indicator measurements are extracted therefrom and the information is finally sent to the service monitoring OSS node 704, in a step 7:10. A last step 7:11 illustrates that a SIP 200 OK message is sent to terminal 700 in response to the SIP BYE message.

In another example of consuming or utilising an IP service, not shown, the IMS core or application server may send a measurement order to the terminal during a SIP session set-up procedure. The measurement order may then be conveyed from the IMS core or application server in a SIP

200 OK message sent in response to a SIP INVITE message from the terminal .

As described above, the terminal may be configured to always start measuring one or more performance indicators according to their respective associated predefined measurement procedures, e.g. the session setup latency, as soon as the user has activated the IP service, to be able to provide accurate measurement results if a measurement order is received after the measurements were started. As further described above, the measurement results may be provided in an XML document which is included in the SIP BYE message or other suitable SIP message.

It is also possible to convey a performance indicator measurement, e.g. in an XML document, by using the SIP Event Package according to a fourth example illustrated by a signalling diagram in Fig. 8. In this example, a user terminal 800 has registered with an IMS core 802 and will activate an IP service in an application server 804, and at some point perform the measurements, whereas the measurement results will finally be delivered to an OSS node 806. Further, a measurement controller, not shown, resides in the application server 806.

Normally, the SIP Event Package generally provides a mechanism for subscribing to information on clients which is delivered from a server as notifications, according to a set of standard SIP messages. For example, the SIP Event Package is typically used in presence services to provide client data regarding a "Presentity" to a "Watcher".

In a first step 8:1, application server 804 sends a SIP SUBSCRIBE message over the IMS core 802 to the user terminal 800, in order to subscribe to measurement results from the terminal 800 according to a predefined measurement procedure, by means of the SIP Event Package. The SIP SUBSCRIBE message is thus effectively a measurement order from the measurement controller in this context. Terminal 800 then accepts the subscription request by sending a SIP 200 OK message as a response to the application server 804, in a next step 8:2. After a user input command in step 8:3 to activate the IP service and initiate a SIP session, terminal 800 sends a SIP INVITE message to the application server 804, in a further step 8:4. Since the measurement order was received in the subscription request in step 8:1, terminal will duly start to perform the measurement at some point after the input command in step 8:3, as dictated by the predefined measurement procedure. In a next step 8:5, the application server 804 responds by sending another SIP 200 OK message to the terminal 800. The session is then executed, as illustrated by a step 8:6.

In a next step 8:7, another user input command is received at the terminal 800 to terminate the SIP session. Accordingly, the terminal 800 sends a SIP BYE message to the application server 804, in a next step 8:8. In a following step 8:9, application server 804 sends a SIP 200 OK message to terminal 800 in response to the SIP BYE message. The dashed two-way arrow "M" illustrates schematically that the terminal 800 may measure the performance indicator basically at any time between the input commands of step 8:3 and step 8:7, depending on the predefined measurement procedure. A further step 8:10 illustrates that a release notification is issued at the terminal, notifying the user that the session or service has been released. In a next step 8:11, terminal 800 sends a SIP

NOTIFY message including the measurement results, e.g. in an XML document, to the subscribing application server 804, to deliver the measurement results by means of the SIP Event Package. In this context, the SIP NOTIFY message is thus effectively a measurement report to the measurement controller residing in application server 806. The performance indicator measurements are then extracted from the SIP NOTIFY message, and this information is finally sent to the service monitoring OSS node 806, in a step 8:12. A last step 8:13 illustrates that a SIP 200 OK message is sent to terminal 800 in response to the SIP NOTIFY message. Fig. 9 is a block diagram illustrating an apparatus in a user terminal in more detail, according to yet another embodiment. The user terminal apparatus 900 is adapted to enable assessment of the performance of an IP service consumed or utilised by means of the user terminal. Of course, the user terminal comprises various other equipment, not shown, for further functions including radio communication, user applications, and so forth.

According to the present context, the user terminal apparatus 900 comprises a receiving unit 900a adapted to receive a service trigger T, e.g. as a user input command or as a specific message, as described above. A measuring unit 900b is adapted to measure a predefined performance indicator associated with the IP service according to a predefined measurement procedure, the performance indicator being indicative of the user experience when consuming or utilising the IP service. The apparatus 900 further comprises a reporting unit 900c adapted to report measurement results R to a measurement controller MC in a service message sent from the user terminal in connection with the IP service. It should be noted that this figure merely illustrates the various functional units in a logical sense, while the skilled person is free to implement these functions in practice using any suitable software and hardware means. Thus, the present invention is generally not limited to the shown structure of the user terminal apparatus 900. For an IP service like PoC, a "pre-established" (using the OMA terminology for PoC) SIP session may for certain scenarios be used to establish PoC Sessions. In this case, the SIP session is not terminated when a PoC Session is terminated. In the case of a pre-established session, the measurement controller may thus order the terminal at session initiation to measure a performance indicator, and the terminal may then report the measurement results at the end of a PoC Session. In another example, not shown, of consuming or utilising an IP service involving a PoC Session, the terminal initiates a SIP session when the user has activated the IP service. At some point thereafter, the user wants to establish a PoC Session and selects one or more users in his/her phone book. The following procedure may then be used:

1) A measurement order for a performance indicator is negotiated in a SIP INVITE request from the terminal and a SIP 200 OK response from the "network", i.e. an IMS core node or an application server where a measurement controller resides as described above.

2) The terminal sends a SIP REFER request message containing a list of the selected users, and the application server accordingly invites the users in the list to the forthcoming PoC session.

3) The terminal measures the performance indicator at any time during the session setup and/or the actual PoC session, depending on the predefined measurement procedure.

4) When the terminal user inputs a command to terminate the PoC session, the terminal sends a SIP REFER request indicating the BYE method. The terminal includes the measurement results in the SIP REFER request. An alternative solution to the above would be a combination of some earlier examples as follows:

1) A measurement order is negotiated during registration, or during SIP session establishment, or when a PoC Session is established.

2) The Performance measurements are then reported by means of the SIP PUBLISH request to an address that is provisioned to the terminal.

Further, when a PoC service is consumed or utilised, a PoC session may be active for quite a long time. The PoC session may be started in the morning when a person using PoC as a communication means, arrives to his/her working site, and the session is terminated when he/she leaves the working site in the evening. In this traffic scenario, it is desirable to convey the measurement orders and measurement results with other means than the SIP BYE or SIP REFER messages.

As mentioned above, measurement results can be conveyed by means of the MBCP protocol which is used by the terminal in a PoC session. A measurement order can be included either in already existing MBCP messages, e.g. the "MBCP Media Burst Idle" message, or in a new message dedicated for this purpose. Likewise, measurements results can be included either in already existing MBCP messages, e.g. the "MBCP Media Burst Release" or "MBCP Disconnect" messages or in a new message dedicated for this purpose.

Another possibility is that various "filters" can be used to monitor specific parts of a service network, at certain times, certain users or user categories, specific services and certain media types. Since it is an operator- controlled network node that issues the measurement orders to the IMS clients/terminals in the network, the operator can select which users and how many users to make the measurements, as well as in which areas and at what time the service performance monitoring should take place.

For example, another private header (P-header) extension to SIP is the "P-Access-Network-Info" header, which could also be used in this context. This header is normally used by terminals to inform the network about which cell or area the terminal is located in. If this SIP header is used for measurement orders and/or reports, the operator is able to order measurements only in certain cells or areas, or may simply filter out measurements from certain parts of the network.

Legacy cellular networks normally have network- oriented performance monitoring systems implemented already, but the performance of the network nodes and links are monitored in the network nodes. In addition to the benefits evident from the above description, some advantages that can be obtained by the present invention may also include:

- Network operators and service providers can monitor the performance of end-to-end services in a user and service-oriented way without requiring network-based measurements .

- Network operators and service providers can select specific users, specific network areas, specific media types and specific IP services for which measurements are performed.

- A structured way is also provided to monitor service performance in an IMS network by accessing measurements right at the end-points, thereby avoiding the problems that the performance of a service typically depends on the performance of many nodes and links in the networks. While the invention has been described with reference to specific exemplary embodiments, the description is in general only intended to illustrate the inventive concept and should not be taken as limiting the scope of the invention. Although the concepts of IMS, SIP, MMTeI and PoC have been used when describing the above embodiments, any other similar suitable standards, protocols and network elements may basically be used as described herein. The present invention is generally defined by the following independent claims.

Claims

1. A method of enabling assessment of the performance of an IP service consumed or utilised by means of a user terminal, comprising the following steps executed by the user terminal (200):

- receiving a service trigger,

- measuring (2:1) a predefined performance indicator of said IP service according to a predefined measurement procedure, the performance indicator being indicative of the user experience when consuming or utilising the IP service, and

- reporting (2:2) measurement results to a measurement controller (210) in a service message sent from the user terminal in connection with the IP service.

2. A method according to claim 1, wherein said performance indicator is measured in response to receiving a measurement order in a service message sent from the measurement controller in connection with the IP service,

3. A method according to claim 2, wherein the measurement order is received in said service trigger.

4. A method according to claim 1, wherein said performance indicator is measured automatically in response to receiving said service trigger, and the measurement results are reported to the measurement controller if a measurement order has been received from the measurement controller.

5. A method according to claim 4, wherein the user terminal has been configured with an order to measure the performance indicator and report to the measurement controller whenever the IP service is consumed or utilised.

6. A method according to any of claims 1-5, wherein said predefined measurement procedure dictates that the measurement of the performance indicator starts when said service trigger is received and ends when a service or measurement termination trigger is received.

7. A method according to any of claims 1-6, wherein said predefined measurement procedure dictates that the measurement of the performance indicator should be performed periodically when the IP service is consumed or utilised.

8. A method according to any of claims 1-7, wherein said predefined measurement procedure dictates that the performance indicator should be measured according to certain preconditions.

9. A method according to claim 8, wherein said preconditions include any of: measure the performance indicator when the user terminal is located in a certain area; measure the performance indicator during a specific time of day, week or season; and measure the performance indicator every n : th time the IP service is consumed or utilised, n being an integer >1.

10. A method according to any of claims 1-9, wherein said service message is a session terminating message.

11. A method according to any of claims 1-10, wherein if SIP or MBCP is used when the IP service is consumed or utilised said measurement results are provided in an XML document embedded in a SIP or MBCP message, respectively, and if RTCP is used when the IP service is consumed or utilised said measurement results are provided in a binary format in a field of an RTCP message.

12. A method according to any of claims 1-11, wherein the IP service is provided from an application server over an IMS core, and said measurement results are reported to the measurement controller residing in the application server or in a session control function node of the IMS core .

13. A method according to any of claims 1-12, wherein said performance indicator relates to experienced media quality and/or experienced latency/delay.

14. A method according to any of claims 1-13, wherein the performance indicator is measured by detecting signals conveyed on an internal interface between an IMS client and a communication unit in the user terminal.

15.An apparatus in a user terminal for enabling assessment of the performance of an IP service consumed or utilised by means of the user terminal, the user terminal apparatus (900) comprising: - a receiving unit (900a) adapted to receive a service trigger,

- a measuring unit (900b) adapted to measure a predefined performance indicator associated with said IP service according to a predefined measurement procedure, the performance indicator being indicative of the user experience when consuming or utilising said IP service, and

- a reporting unit (900c) adapted to report measurement results to a measurement controller in a service message sent from the user terminal in connection with said IP service .

16.An apparatus according to claim 15, wherein the measuring unit is further adapted to measure said performance indicator in response to receiving a measurement order in a service message sent from the measurement controller in connection with the IP service.

17.An apparatus according to claim 15, wherein the measuring unit is further adapted to measure said performance indicator automatically in response to receiving said service trigger, and the reporting unit is further adapted to report the measurement results to the measurement controller if a measurement order has been received from the measurement controller.

18.An apparatus according to claim 17, wherein the user terminal has been configured with an order to measure the performance indicator and report to the measurement controller whenever the IP service is consumed or utilised.

19.An apparatus according to any of claims 15-18, wherein if SIP or MBCP is used when the IP service is consumed or utilised the reporting unit is further adapted to provide said measurement results in an XML document embedded in a SIP or MBCP message, respectively, and if RTCP is used when the IP service is consumed or utilised the reporting unit is further adapted to provide said measurement results in a binary format in a field of an RTCP message.

20.An apparatus according to any of claims 15-19, wherein the IP service is provided from an application server over an IMS core, and the reporting unit is further adapted to report said measurement results to the measurement controller residing in the application server or in a session control node of the IMS core.

21.An apparatus according to any of claims 15-20, wherein the measuring unit is further adapted to measure the performance indicator by detecting signals conveyed on an internal interface between an IMS client and a communication unit in the user terminal.