WO2014096861A2

WO2014096861A2 - Methods and apparatus for forming image using, and finding positions of, plural pixel devices

Info

Publication number: WO2014096861A2
Application number: PCT/GB2013/053401
Authority: WO
Inventors: James Cobb
Original assignee: Crowd Connected Ltd
Priority date: 2012-12-21
Filing date: 2013-12-20
Publication date: 2014-06-26
Also published as: GB2509202A; GB2531186B; GB201315741D0; GB201223306D0; WO2014096861A3; GB2531186A; GB2509157A; GB2509202B; GB201522595D0

Abstract

Methods and apparatus for forming an image using plural pixel devices are described. Methods and apparatus for determining the positions of a plurality of mobile devices are also described. The method of forming an image comprises forming a communication channel over an Internet Protocol (IP) network between each of the pixel devices (20) and a remote server(40) and calculating at the server a display attribute for each of said pixel devices in accordance with the image to be formed. The display attribute for each pixel device (20) is communicated to just that pixel device. The pixel devices (20) change their display in accordance with the display attribute to forma desired image.

Description

METHODS AND APPARATUS FOR FORMING IMAGE USING. AND FINDING

POSITIONS OF. PLURAL PIXEL DEVICES

In aspects, the present invention relates to a method and apparatus for forming an image using plural pixel devices. In other aspects, the present invention relates to methods and apparatus for determining the positions of a plurality of mobile devices. In preferred embodiment, the methods and apparatus for determining the positions of the mobile devices are used in forming an image on mobile pixel devices.

It is known to form displays over large areas using distributed display units or pixels forming a pixel surface for the image. For example, at the 2012 Olympic stadium 70,500 custom LED paddles were installed on a custom stand at each seat in the 2012 Olympic stadium and used to provide various video effects, named "Landscape Video". A similar technique was used at the opening of the Arab Games in Doha.

In another example, the members of the audience at Coldplay (RTM) concerts have been given so-called "Xylobands" (RTM) which are simple, monochrome LED wristbands that can be remotely triggered to create interactive lighting effects involving the audience.

These techniques have also been used to create art installations and events. The physical extent of the pixel surface means wiring is complex. In the Olympic installation 370km of cable were required to link all of the paddles back to the controller. This took a team of 10 people five weeks to install. The positioning of pixels relies on fixed installation matching a predefined map which the controller uses to control the display. It is also costly to fabricate the custom units and install them. There is also limited opportunity for audience interaction, as the devices are fixed in position and offer no user input opportunity. The fixed installation also makes the devices intrusive. Some prior art has suggested using wireless devices to form a pixel display surface. For example, US7657097 discloses forming an image using

independent pixel devices, where a controller determines based on the locations of the pixel devices within the image space which part of the image the pixel device should reproduce. In some examples, the pixel devices feed back location information to the controller, generated using GPS or by emitting a radio signal which is triangulated by antennas placed around the image space.

EP1870802 discloses a method for making user equipment, such as a mobile phone, part of a display during a performance event. Such systems require a lot of infrastructure to be set up and configured on site and would struggle to cope with increases in the number of pixel elements. Generally these two documents describe methods for locating pixels where, other than GPS, the mobile device acts as a beacon, and optical or RF sensors then triangulate the pixel. For the above reasons, that is not the preferred approach.

More generally, accurately positioning mobile devices including smartphones and tablets is increasingly important for uses such as navigation, location based information and advertising, emergency service response, etc, as well as being usable in distributed displays such as described above. Many devices are now equipped with GPS circuitry to enable them to self-locate to within a few meters. However in urban canyons or indoors, GPS positioning can be inaccurate or impossible. The situation we address is well populated public places where GPS positioning may be incomplete, reaching some devices but not all, or with lower than required accuracy. For instance a sports stadium, shopping centre, high street, conference and exhibition centre, train station, airport etc.

Many techniques have been described for locating mobile devices. Those relating to mobile phones have largely been uncooperative techniques where each device works in isolation. Positioning is attempted using fixed networks of beacons or sensors. GPS is an example of such a system. Where GPS is unavailable, base stations either need to be installed for the purpose, or (for instance using already installed WiFi base stations), calibration is required. The majority of the proposals in the art involve fingerprinting (the creating of a field strength map) - for instance (Chen and Ofek 2010) and (Meunier and Snowdon 2007).

Solutions involving the cooperative localization of mobile phones have largely been examples of self-location, involving passing information directly between devices, and carrying out calculation locally. For instance (Muller, Schill et al. 2010), (Aoyama 2009), (Wymeersch, Win et al. 2009). These systems use different algorithms for inferring position from ranges, but all work by devices collecting location and range information from neighbours, and calculating their position. (Cho, Choi et al. 201 1 ) proposes a central estimation server, but still envisages device to device communication, and data and information storage on mobile devices.

There are various problems with these approaches.

1. The two possible modes of communication available on current

smartphones are WiFi and Bluetooth. Most smart-phones cannot act as both a WiFi access point and a WiFi client simultaneously. This makes connecting smartphones using WiFi in a communicating grid impossible. Bluetooth connections can be made between devices to form a grid, but each connection requires pairing permission from the user which is unfeasible. Additionally Bluetooth places a limit on the number of simultaneous connections. Cooperative self-localization is therefore not currently possible as described in the prior art.

2. The prior art also requires the immediate neighbours of a device to have known position prior to starting a position calculation. Positioning a device that is out of RF range of 'anchors' (known position devices) is therefore very computationally and network expensive, involving positioning every device along the path from the anchors. 3. There is also a privacy problem with any cooperative self-localization system. The user must be content for their device to communicate directly with nearby devices of unknown trustworthiness - via Bluetooth, WiFi, or some other available protocol in the future. This is problematic.

(Marinier 2007) notes that the prior art has not established how to encourage users to participate in cooperative localization. In many scenarios this will not be a problem. Crowds gathered for a single purpose (e.g. a conference, a concert, or a sports event) will be easier to communicate with and can enjoy benefits from cooperative localization that are not present in other scenarios.

In the field of wireless sensor localization, many techniques have been described that utilize estimated distances between devices. However these techniques are generally designed for very different circumstances. They do not consider very large or dense networks of devices (for instance > 100,000 devices, or density > 2 devices per square meter). They assume homogeneity of devices. They require device-device communication in the absence of a central server. And they do not consider very large numbers of neighbours for each device (for instance > 100). Some of the mathematical techniques we propose to use are well known in this field, but have not yet been applied in the relevant prior art. Particularly relevant is ( Luo, Li et al. 2007), which outlines a method of distributed weighted multidimensional scaling with error tracking which can be used in examples of the system disclosed in this document. List Of References

Aoyama, A. (2009). Positioning method using mobile terminal and mobile terminal having positioning function, EP 1 ,206,152.

Chen, B. and E. Ofek (2010). HYBRID MOBILE PHONE GEOPOSITIONING, Google Patents.

Cho, S. Y., Y. Choi and J. Kim (201 1 ). METHOD AND SERVER FOR

ESTIMATING POSITION OF MOBILE NODE, US20120071 170. Jamali-Rad, H. and G. Leus (2012). "Dynamic Multidimensional Scaling for Low- Complexity Mobile Network Tracking." Signal Processing, IEEE Transactions on 60(8): 4485-4491.

Li, X., H. Shi and Y. Shang (2005). A sorted RSSI quantization based algorithm for sensor network localization. Parallel and Distributed Systems, 2005.

Proceedings. 1 1th International Conference on, IEEE.

Liu, J., Y. Zhang and F. Zhao (2006). Robust distributed node localization with error management. Proceedings of the 7th ACM international symposium on Mobile ad hoc networking and computing, ACM.

Luo, H., J. Li, Z. Zhu, W. Yuan, F. Zhao and Q. Lin (2007). Distributed

multidimensional scaling with relative error-based neighborhood selection for node localization in sensor networks. Integration Technology, 2007. ICIT'07. IEEE International Conference on, IEEE.

Marinier, P. (2007). METHOD AND SYSTEM FOR MANAGING COOPERATIVE POSITIONING AMONG WIRELESS TRANSMIT/RECEIVE UNITS, EP

1 ,620,954.

Meunier, J. and D. Snowdon (2007). Mobile device and method for determining location of a mobile device in a WLAN network, EP 1 ,542,492.

Muller, G., D. Schill and W. Hagg (2010). Cooperative positioning, EP 1 ,207,404. Patwari, N. and A. O. Hero Hi (2003). Using proximity and quantized RSS for sensor localization in wireless networks. Proceedings of the 2nd ACM

international conference on Wireless sensor networks and applications, ACM. Shang, Y., W. Ruml, Y. Zhang and M. P. J. Fromherz (2003). Localization from mere connectivity. Proceedings of the 4th ACM international symposium on Mobile ad hoc networking & computing, ACM.

Wymeersch, H., M. Z. Win and J. Lien (2009). COOPERATIVE LOCALIZATION FOR WIRELESS NETWORKS, EP 2,084,553

According to a first aspect of the present invention, there is provided a method of forming an image using plural pixel devices, the method comprising:

forming a communication channel over an Internet Protocol (IP) network between each of the pixel devices and a remote server; calculating at the server a display attribute for each of said pixel devices in accordance with the image to be formed;

for each pixel device, communicating the display attribute for just that pixel device to the pixel devices; and,

changing the display of the pixel devices in accordance with the display attribute such that the plurality of pixel devices together forms the desired image, each pixel device forming a pixel of the image.

A pixel device is a device able to display at least a single pixel, preferably by setting the colour of the pixel. An image is any image or effect comprised of plural display elements. The image can be as simple as turning all of the pixels on, or pulsing the pixels in synchronisation or randomly pulsing the pixels to create visual effects. Alternatively more complicated images can be formed by utilising information about the positions of the pixels. The image can be static or time varying. Where position information is available to the server for the pixel devices, more complicated graphical effects can be achieved.

In preferred embodiments, the pixel device is a mobile electronic device, such as a mobile phone, PDA or tablet computer. Custom devices could also be used. Alternatively, fixed pixel devices can be used. This eliminates the need to provide large numbers of cabling and interconnects and to calibrate and map the pixel devices, and thus makes set up of an installation in say a stadium or other arena quicker. Using an IP network and preferably TCP and HTTP protocols to communicate between the server and the devices is important in allowing the preferred system to be scalable and be easy to install. This could be wide area networks (3G / 4G) or WiFi, or wired Ethernet. In an embodiment push technology is used for communication between the server and the devices. Push technology means the server initiates the communication and sends data to the pixel device when it is ready to do so. This has a small overhead as opposed to a pull based technology/long polling techniques, where the pixel devices would initiates the communication. To achieve low latency control, prior art pull techniques will require very frequent polling (perhaps 10 times per second) in order to achieve satisfactory synchronisation between the pixels. This will lead to a very high number of network requests even when no image is being projected, causing network congestion. Preferably the channel is a full duplex link. Preferably Websocket connections are used to communicate between the pixel devices and the server. These minimise the network overhead compared to prior art schemes and make the system scalable to larger numbers of pixels. A public WAN such as 3G or 4G data networks can be used to access a remote server from the mobile devices. This advantageously means that the system can be implemented with little or no infrastructure installed at the location in preferred embodiments. Websockets is built on top of the HTTP protocol used for websites, so is able to traverse public internet connections. In contrast, proprietary protocols using multi-cast, UDP packets etc will not be able to traverse public internet connections.

In embodiments, the system is envisaged operating using wireless connections between the server and devices, namely WiFi and also with WAN wireless connections (primarily 3G and 4G). Alternatively, wired connections may be preferred in some implementations, e.g. wired Ethernet. With very large numbers of pixels, network congestion is likely to be a problem. The preferred system is therefore designed to minimize network traffic.

Preferably the server is configured to be able to connect with and form an image on a large numbers of pixel devices, e.g. greater than 1000, or even greater than 10,000, or even greater than 100,000. In an embodiment the display attribute sent by the server comprises colour information, the pixel device being arranged to change its display to display that colour. The colour code can be just a byte or two or three of information, meaning that the network traffic is reduced and thus the number of pixel devices that can be accommodated on the network can be increased.

In an embodiment the server only sends the display attribute to the pixel device when the display is changing. This helps reduce network traffic and thus increases the number of pixel devices that can be accommodated on the network.

In an embodiment, the method comprises the server sending a delay attribute to a pixel device, the pixel device thereafter waiting for a period indicated by the delay attribute between each change of its display. Thus, the network overhead is further decreased where attribute changes take place at regular intervals by applying the previous delay to the following attribute change if no new delay is specified.

In an embodiment, the method comprises sending different delay attributes to different pixel devices and/or sending different delay attributes to the same pixel device at different times during the display of the image so as to spatially and/or temporally customise the frame rate. In this way, pixel devices corresponding spatial or temporal parts of image that are changing less frequently can be accordingly given a lower frame rate to reduce network overheads.

In an embodiment, the method comprises caching display attributes at a pixel device if they are received at a faster rate than they are to be displayed. Thus, the attributes are buffered at the pixel device to achieve the desired display rate.

This variable frame rate scheme is capable of real-time low latency control, and fixed frame rate cached changes with only display attributes (e.g. colour attributes) being sent, and of variable rate cached changes with only the minimum of delay information sent from the server, as well as the real-time attribute changes of the previous embodiment

A 'flush' command can be sent from the server to the device in which case the cache is cleared and subsequently received display attributes are displayed as soon as they are received. Thus, the method can proceed in a similar way to the method described above in which the server only send the display attribute to the pixel device when the display is changing, thus achieving the minimum overhead. In an embodiment multiple attribute values and any attached delay values relating to future display changes can be sent together in a single websocket message or other network packet to reduce network congestion.

In an embodiment, the method comprises the server timing the round trip time of a message sent to the pixel devices, and calculating an estimated latency for each pixel device in accordance with the round trip time, compensated for in calculating future delay attributes.

In an embodiment, the method comprises the server sending messages to the devices, optionally within display command messages, requesting timing information, and the devices replying with the timing information, e.g. time elapsed since a predetermined point in time. This allows repeated measures of synchronisation to be made by the server and averaged for each device to obtain an estimate of synchronisation offset for each device, and to send a message to the devices comprising a timing offset to apply to all future colour changes. The server can weight each measure of synchronisation used in obtaining the estimate. For example, the measure can be weighted according to how much confidence there is in the measurement, i.e. its variance. This might be taken to be a function of the round trip time of the messages to the devices requesting timing information and the replies to the server. In an embodiment the server is arranged to add new pixel devices to the image dynamically. Thus, the pixel surface can grow during an event as pixel devices request to join the image and the server incorporates them into the display. In an embodiment the server comprises a peer-to-peer network of plural peer servers, each peer server being arranged to communicate with different pixel devices in order to form the image. In an embodiment, each peer server is arranged to calculate the positions of pixel devices and/or calculate display attributes to be communicated to the pixel devices. Alternatively, positioning and determining the display attributes necessary for forming the image/video on the pixel devices can be calculated by another server in communication with the plurality of peer servers and sent to the peer servers for communication with the pixel devices. In principle any of these can be carried out by peer servers. This helps allow scaling of the display to large numbers of pixel devices.

In an embodiment the method comprises: forming an initial communication channel between a new pixel device and a first server of the plurality of servers; at a later time, allocating the pixel device to a different server and migrating the communication channel from the device to the different server in order to balance the load on the servers. Preferably a pixel connects initially to any one server, and then receives an instruction from that server to connect to another server, in response to which it closes the original connection, then opens a new connection to the new server. This is highly advantageous in making the system truly scalable and fault tolerant.

In an embodiment, at least one peer server is a virtual server, the method comprising dynamically adding a new instance of a virtual server when the load or expected load on the existing servers reaches predetermined level. In other words, one or more of the servers is preferably in "the cloud". Cloud computing is a means of providing location-independent computing, whereby shared servers provide resources, software, and data to computers and other devices on demand. Generally, cloud computing customers do not own the physical infrastructure, instead avoiding capital expenditure by renting usage from a third- party provider. Cloud computing users avoid capital expenditure on hardware, software, and services when they pay a provider only for what they use. New resources can quickly be put on line allowing the resources to be dynamically scaled. This provides flexibility for the user. Examples of cloud computing are the Amazon Elastic Compute Cloud (EC2) and Windows Azure Platform.

In an embodiment, the method comprises: sensing with a sensor of the pixel devices position data indicative of the position of the pixel device; receiving at the server said position data from said pixel devices; determining at the server the position of each pixel device in accordance with said data; calculating the display attribute for each of the pixel devices in accordance with the determined position of that pixel device. This is particularly useful where the pixel devices are devices without a fixed position, such as mobile phones, PDAs and tablets, etc for forming complex visual effects. Preferably the position data sensed by any of the pixel devices can be one or more of Global Positioning System (GPS) data, visual information such as a barcode or other identifier with a predetermined location capable of being scanned by a sensor of the pixel device, Bluetooth or WiFi or audio signals as discussed elsewhere in this document.

In an embodiment, the method comprises sensing in the vicinity of the pixel device one of a plurality of position markers positioned at different predetermined physical locations, the position markers encoding respective unique identifiers; the server calculating the predetermined physical location for that identifier to obtain the position of the pixel device. This provides a simple and inexpensive way of accurately determining position, or determining position to within a chosen accuracy. The server may have a simple look up table for mapping identifiers to positions, or use an algorithm. This can be useful for example in a stadium where the position marker can be placed at predetermined positions in the stadium. For example the position markers can be associated with each seat, row, aisle or entrance/exit gate in the stadium and which the owner of the pixel device can scan with the pixel device at some time prior to joining the display. In an embodiment, the position marker is a barcode and the sensor is a camera associated with the pixel device. This is useful for use with mobile phones, PDAs and tablets and the like which for example are often equipped with a camera. 3D barcodes are suitable as images for encoding the location identifier.

Alternatively, near field technology or RFID tags or Bluetooth Low Energy devices such as i Beacons (RTE) can be used for the position marker.

In an embodiment, the position marker also encodes a web address for server, the pixel device decoding the web address and connecting to the server indicated by the web address to join the display. In a preferred embodiment, the web address will contain a fully qualified domain name so that the pixel device knows which server to connect to, but will also preferably embed information as to which pixel surface (e.g. which event) the device is to be part of, and preferably also position information, as described above.

In an embodiment, the method comprises: providing at least three wireless transmitters at predetermined locations; sensing with a sensor of the mobile device the wireless signal to obtain information indicative of the distance from the mobile device to the wireless transmitters; receiving at the server said information from said mobile device; calculating at the server the position of the mobile device using the distance information for the at least three wireless transmitters. Preferably the device measures the signal strength of the wireless transmitter. Alternatively, time of arrival or hop count information can be used. Various techniques for calculating the position based on three or more distance metrics from points with known positions are known, i.e. trilateration or multilateration calculations.

In an embodiment, the method comprises calculating the error in position of the pixel device from said position data and calculating the display attribute in accordance with the position of the pixel device and the error in the position. This can be used to achieve a number of advantageous schemes to deal with different degrees of certainty in the position of pixel devices. For example, if error is above a predetermined amount, the brightness can be lowered so that if the error is such that the "wrong" display attribute is sent to the pixel element, the resulting artifacts in the image/video are mitigated by the reduced brightness. Another scheme is to average the pixel attributes for neighbouring pixel devices if the error is above a certain amount.

According to a second aspect of the present invention, there is provided a method of determining the positions of a plurality of mobile devices, each mobile device having a transmitter and receiver for a wireless signal, each mobile device being within wireless sensor range of at least one other of said mobile devices, the method comprising:

sensing with a receiver of the mobile device a wireless signal of at least one neighbouring mobile devices to obtain information indicative of the identity of and the distance to said at least one other mobile device;

receiving at a server said information from said mobile devices;

calculating at the server a map of the relative positions of the mobile devices using said distance information.

Bluetooth is a preferred wireless signal to use with this method, as most smartphones are equipped with a Bluetooth transmitter/receiver. The density of devices makes this workable and compensates for the variable TX power and RX sensitivity which makes prior art schemes of cooperative localisation unworkable. Any number of received signal strengths can be used to make an estimate, but generally the more, the better. In preferred embodiments, the device density is at least 1 device per 5 m², and in some embodiments, such as a concert crowd, at least 1 device per m² Importantly no communication is required between devices using the wireless transmitter/receiver, e.g. over the Bluetooth channel. So for Bluetooth the devices must be 'discoverable' (i.e. will respond to other devices' enquiries by confirming their presence and their unique MAC address), but do not initiate pairing, and do not communicate any data. In the case of Bluetooth Low Energy (as opposed to "Classic" Bluetooth), the devices act both as a 'peripheral' (advertising a unique id and preferably TX power), and in central mode, scanning for advertisements. Again no connection is necessary.

Other signals may be used to estimate range. Audio signals are advantageous as the transmission and reception of audio is possible within the browser of most mobile devices, allowing position calculations to be carried out within a web application rather than a native application. Techniques for encoding digital information (e.g. a unique id of the transmitting device) in an audio signal are a well known per se. However prior art techniques have not contemplated the use of audio signals applied to cooperative positioning. In an embodiment, the method comprises a mobile device emitting said wireless signal of discrete duration at a time interval determined by the server. Thus, the server coordinates the pixel devices to avoid interference with neighbouring pixel devices transmitting. Thus, the server can dictate that no mobile device will wirelessly transmit with any other mobile device in a predetermined range at the same time. The range can be selected based on the strength of the wireless signal transmitted and the amount of interference that can be tolerated at the receiving device. This is particularly useful in the case of using audio signals for detecting range to neighbouring devices in avoiding interference from

neighbouring devices.

The requirement to calculate on the server rather than device is driven by the desire not to have devices connect with and communicate with each other directly, but only with the central server. There are both practical reasons and security reasons for this. Most prior art concerned with locating mobile devices have them communicating directly with neighbouring devices. The preferred method can be used to form a display of pixel devices by feeding back the information to the server allowing the mobile pixel device positions to be calculated. This is particularly useful when used in conjunction with the scalable and network traffic minimising techniques described above.

This provides a technique of remote localisation where devices rely on a central service to determine their position. The prior art has largely dismissed the use of centralised localisation on the grounds of scalability, but using distributed data storage and computation and low network overhead techniques, a server grid can provide centralised localisation on a scalable level.

The preferred system allows devices such as smartphones to communicate incomplete or inaccurate positional data to a central server, where an estimation of the devices' position can then be calculated. This estimated position can then be used by the server system, or communicated back to the device, for any required purpose.

In a preferred embodiment, two types of information are communicated. First, devices which are able to self-locate communicate information regarding their absolute position and positional error to the server grid. Secondly, all devices (whether able to self locate or not) communicate metrics regarding wireless signals received from other devices (which can be "beacons" and/or devices with positional error) within range. In the current embodiment the first type of information (absolute positioning information) is provided by GPS. When a GPS fix is achieved, the position and accuracy of that position are transmitted to the server grid. However absolute positioning data may come from any other absolute positioning technique, such as WiFi signal trilateration from known base stations, barcode localization, and/or by position markers, tags, barcodes, etc. used when connecting to the server as described herein. In the current embodiment the second type of information, signal metrics, consists of Bluetooth received signal strength following a Classic Bluetooth enquiry scan or Bluetooth Low Energy scan. Practical limitations of the OS (Operating Systems) on current smartphones (e.g. Android, iOS) mean that WiFi cannot be used for ranging purposes, so that Bluetooth is currently the preferred RF signal available. But the use of other signals for RSS ranging may be practical in the future. For example, audio signals may be used.

Similarly signal strength is the only currently available metric using smartphones, but other metrics (such as those related to time of flight) may be used in the future. The server grid is able to estimate on demand the absolute position of any device from the reported Bluetooth signal strengths from a sufficient number of devices, and from the reported GPS (or other absolute) positions of some but not all devices. The problem of calculating relative or absolute position from inter- device ranges is well known in the field of wireless sensor localization and there are many mathematical techniques for accomplishing this, including centroid and weighted centroid localization, hop counts, maximum likelihood estimates through linear and non-linear least squares techniques, multidimensional scaling, semi-definite programming, Kalman filters and particle filters.

The preferred system works with very little and in some cases zero local infrastructure, and zero calibration, although as described herein in some cases limited infrastructure comprising simple Bluetooth transmitters will be

advantageous and carry low cost. Non-cooperative proposals for positioning mobile devices without GPS either involve extensive transmitting reference base stations, extensive static sensor networks, or extensive surveying and calibration.

The preferred system requires no modification to current mobile telephone hardware or operating systems. Many other proposed cooperative systems for positioning mobile phones are only possible with modified operating systems or hardware.

The user will need to give permission for their device to be 'discoverable' via Classic Bluetooth. But will not have to pair their device in order for any

communication to take place over the channel. Only the MAC address and signal strength of a device are advertised to untrusted devices, considerably reducing privacy fears. The preferred system can scale well to very many devices using a scaleable peer-peer grid of servers, and low overhead network techniques.

The preferred system is capable in theory of accurately absolutely positioning devices that are a considerable distance from the nearest GPS positioned devices, as long as there is a sufficient device density in the vicinity. In a centralized system, positioning is possible in relation to devices that are outside of Bluetooth communication range, by using "hop" count techniques. Position estimates for devices can be provided "on demand", without the need for every device position to be calculated, or for sufficient immediately neighbouring devices to have known position.

Bluetooth RSS can provide a reasonable estimate of distance over short ranges, but in sparser networks can provide up to around 10m range. Range adaptive technique allows for areas of high and low density. In low density areas all neighbours are considered, and range is around 10m. In higher density areas, more distant neighbours will be ignored due to their higher estimated position error, resulting in an effective decrease in range. Most prior art considers the use of fixed transmission range RF only. Iterative estimation techniques are preferably used, so that approximate positions can be available quickly, and more accurate positions after further iterations. Preferably, the error in GPS provided positions and in calculated positions is modeled and stored by the server software. This allows the system to weight more accurate information, and provide better position estimates. This prevents errors from propagating through the network. Many other methods simply consider devices to be of known position, or unknown position. The

consideration of positional error is essential in working with error prone GPS positioning, but without fixed base stations.

In an embodiment, the estimated error of a device's currently estimated error is stored by the system. The corresponding error in each new observation (e.g. GPS absolute position or a range observation) is calculated. The original position and a position newly estimated using the observation are combined in inverse proportion to their estimated variance. In an extension to the embodiment, the timestamp of each observation (either of absolute position e.g. GPS or of ranges) is stored, and the weight given to each observation in calculating estimated positions depends both on the estimated observation error, and also decreases as a function of age. In this way the aims of positioning as accurately as possible both static and moving devices are balanced.

In an embodiment, the method comprises receiving at the server the absolute position of plural of the mobile devices and using this together with the received distance information to determine the absolute map of mobile device positions. Having at least one mobile device in the network provide an estimate of absolute position in advance (e.g. through GPS), allows the absolute positions of all mobile devices to be calculated. The use of some devices with known absolute positions can be particularly advantageous in for example a stadium or arena, where some devices may be able to determine their absolute position for example by using GPS location information (e.g. those in the centre of an open- air stadium) or by scanning a barcode next to their seat (e.g. those in the seating part of the stadium), but where other devices do not have those options for determining their absolute locations (e.g. for those where GPS is blocked, or for those who are not positioned near a barcode). The number of devices used will depend on the number of dimensions. Using just two devices is theoretically possible, but using four devices is preferred in practice.

In an embodiment the absolute position is determined by GPS, scanning a barcode, or detecting a wireless transmitter with a known position.

In an embodiment a position map is calculated which uses no absolute positions for the devices, and the server can only calculate from the reported ranges between devices to find the relative position map. Other information (such as the topology of the device density map) can be used to estimate a transformation between the relative map and absolute and therefore calculate absolute position if it is required. For example a single range at a time algorithm can be used to first calculate device positions in an arbitrary coordinate system. Then at some later point a transform is made into 'real' absolute coordinates. This might be in response to finally receiving geo position data from a device, or receiving a range to a beacon. Or the transform could be calculated by other means - for instance by looking at the 'shape' of the device density map in arbitrary coordinates, and fitting that to the boundaries of the absolute space - e.g. the shape of the room or space the devices are known to be in.

In an embodiment, the method comprises using multidimensional scaling to infer position from said distance information and any prior known absolute positions to determine said map.

In an embodiment, the method comprises seeding the network with at least one device having a predetermined known position. Where there is no available or limited GPS, a number of very simple Bluetooth beacons of predetermined position can be used to determine the positions of the mobile devices. This may be advantageous in for example allowing operation indoors at exhibitions, shows, etc. It also means the method can operate without the system knowing the size / position / layout of a stadium or event in advance. By putting a very simple low cost wireless beacon, such as Bluetooth, in known relative positions (left front of stage, right front of stage, mixing tower, edge of pitch, back of seating, etc.), the system can scale / stretch / skew or otherwise transform the content for the show into any venue.

In an embodiment, the positions of plural mobile pixel devices are used in the method of forming an image using plural mobile pixel devices as described above.

In an embodiment, the method is carried out in a stadium or arena.

According to a third aspect of the present invention, there is provided a system comprising a server and one or more mobile device, the system being constructed and arranged to carry out the method described above.

According to a fourth aspect of the present invention, there is provided a server for forming an image using plural pixel devices, the server being constructed and arranged to:

form a communication channel over an Internet Protocol (IP) network with each of the pixel devices;

calculate a display attribute for each of said pixel devices in accordance with the image to be formed;

communicate the display attribute for just that pixel device to the pixel devices such that the pixel devices can change their display in accordance with the display attribute and together forms the desired image, each pixel device forming a pixel of the image.

According to a fifth aspect of the present invention, there is provided a server for determining the positions of a plurality of mobile devices, each mobile device having a transmitter and receiver for a wireless signal, each mobile device being within wireless sensor range of at least one other of said mobile devices, the server being constructed and arranged to:

receive information from said mobile devices sensed by their wireless sensors, wherein the information is indicative of the identity of and the distance to said at least one other mobile device;

calculate a map of the relative positions of the mobile devices using said distance information.

Any suitable computer hardware can be used for the server. In general the server will include a processor for executing program instructions, memory for holding data and program instructions, storage and network connectors and input/output devices. In preferred embodiments, the server is a scalable grid of servers. According to a sixth aspect of the present invention, there is provided a computer program for a pixel device for forming an image using plural pixel devices executing that program, the computer program comprising program instructions which when run on the pixel device cause it to:

form a communication channel over an Internet Protocol (IP) network with a remote server;

receive a display attribute for just that pixel device from the server in accordance with the image to be formed; and,

changing the display of the pixel device in accordance with the display attribute such that the plurality of pixel devices together forms the desired image, each pixel device forming a pixel of the image.

According to a seventh aspect of the present invention, there is provided a computer program for a mobile device for use in determining the positions of a plurality of mobile devices executing that program, each mobile device having a transmitter and receiver for a wireless signal, each mobile device being within wireless sensor range of at least one other of said mobile devices, the computer program comprising program instructions which when run on the mobile device cause it to:

transmit said information to a server arranged to calculate a map of the relative positions of the mobile devices using said distance information.

The computer program may be recorded on a non-transitory computer readable medium. The mobile device or pixel device may be a smart phone or tablet computer or the like or a custom device. In general the device will include a processor for executing program instructions, memory for holding data and program instructions, and input/output devices and sensors. It will be appreciated that any features expressed herein as being provided "in one example" or as being "preferable" may be provided in combination with any one or more other such features together with any one or more of the aspects of the present invention. Embodiments of the present invention will now be described in detail with reference to the accompanying drawings, in which:

Figure 1 shows an example of a display system in an arena according to an embodiment of the present invention;

Figure 2 shows schematically an example of a display system according to an embodiment of the present invention;

Figure 3 shows a flowchart of communications between a pixel element and a server in the display of Figure 2; Figure 4 shows a flowchart of an example of the operations carried out by a server when RSS data is received from a mobile device according to an embodiment; and, Figure 5 shows a flowchart of an example of the operations carried out by a server when GPS data is received from a mobile device according to an embodiment.

1. Pixel Device Display System

Figure 1 shows an example of a display system 1 in an image space according to an embodiment of the present invention. In this example, the image space is an arena 10 having a stage 1 1 at one end, a seating area 12 covered by a roof around the outside of the arena 10, and an open air section 13 in the middle. For example, the arena 10 could be a sports stadium being used for a concert. The audience sit in the seating area 12 and/or stand in the middle of the stadium 13. Some or preferably most if not all of the audience members each have a pixel device 20 which has a light emitting element 21 which can be controlled to emit light. The pixel devices 20 around the arena 10 form a pixel surface 22 on which an image 23 (in this example, a cross shape) can be displayed by controlling the individual pixel devices 20. The pixel devices 20 communicate via a wireless network 30 using TCP/IP and HTTP protocols with a server 40 from which they receive control signals 85 for controlling their light emitting elements 21.

In the present system, the pixel devices 20 are the audience's own mobile devices, e.g. smartphones or tablet computers or the like, which use their screens as light emitting elements 21 by changing the screen to a particular colour and/or light intensity according to display attributes contained in the control signal 85. The pixel devices 20 need not be identical, e.g. different smartphones can be used. Nonetheless, as will be apparent from the rest of the present description, other pixel devices 20 can be used, such as fixed or custom pixel devices. Referring now to Figure 2, to join the display, a mobile device 20 connects to a Websocket URL (uniform resource locator) using software running on the device 20. In this example, web browser software running on the device 20 is used, although as discussed below, other options such as Apps can be used. This URL is mapped using a DNS mapping service 41 , e.g. Amazon's (RTM) Route53 DNS service, to a server 42. In this preferred example, the server 42 is a scalable group of servers 42a..d provided by a virtual server provider, such as Amazon's elastic compute cloud 43. In response to the mobile device's request 81 , the DNS service 41 returns 82 the IP address of one of the server 42b in the cloud 43. Each server 42a.. d is running identical server software that can establish a duplex communication channel 80 to the mobile device 20. The server 42b creates a websocket endpoint for the connection. The server 42b also provides an HTML5 page to the mobile device 20 to be displayed by the web browser, which comprises the client application. Javascript on this HTML5 page running in the web browser is responsible for initiating the websocket connection with the endpoint server 42b.

The client application (in this case Javascript using HTML5 positioning API (Application Program Interface)) sends the mobile device's position over the websocket 80 to the server 40. Various ways of establishing position are discussed in the present application. One simple way is to rely on GPS positioning 50 where the mobile device 20 has this capability. However it is determined, the position can be mapped to an (x,y) pixel in a video file (or a static image) at the server 40 such that a large number of pixel devices 20 within a user defined geographic area will display the video. As the frames of the video are played, each time a pixel is required to change colour, the device is sent control signal 85 containing a suitable display attribute, e.g. a new colour code, over the websocket connection 80. The client application (e.g. Javascript on an HTML5 page) changes the background colour of the display 21 in response to this control signal 85. The system is not limited to displaying a video or static image. A wide range of visual effects can be created, some of which do not require position to be fed back from the mobile devices 20 to the server 40. For example every device can be made to flash with a colour, independent of position.

It is anticipated that currently a single server 42 instance will be able to handle a large number (thousands) of connected pixels concurrently at 10 fps. If load increases additional servers 42 are automatically started, and existing

connections migrated to the new servers 42 as described in more detail below.

In a preferred embodiment, QR or other barcodes 60 or RFIDs or position markers are scanned by the mobile device 20 and used to easily locate the correct URL (with an event specific ID) for the server 40 and identify themselves to the server 40. Barcodes 60 can also be used to send position information to the server 40 as described in more detail below. The web address will contain a fully qualified domain name so that the pixel device knows which server 40 to connect to, but will also embed information as to which surface (e.g. which event) we wish to be part of, and also position information (described in more detail below in relation to positioning techniques).

So, an example of a URL might be: - http://server1 . biggerpix. net/famousbandshow/id 123

where "server1 .biggerpix.net" identifies the server, "famousbandshow" identifies the event, i.e. a show by "famous band", and "id123" identifies the position of the barcode (and thus the position of the mobile device 20 which scanned it) by reference to a seat number or other location in the arena mapped by the server to the unique position identified. Alternatively, the url might include a "tag" such as: - myseat.is/afj5l3ku

where the tag maps to a 'location' that maps to a surface and/or event. The location tag (a 8 character base32 code in this example) can permanently be associated with a physical location, and the server has a map to associate these physical locations with a surface and/or event to join.

In this example, a HTML5 webpage page running Javascript is used as the software at the mobile device 20 which has the advantage of being simple and platform neutral, but has some limitations. For example, Javascript will not allow access to lower level device operation such as turning up screen brightness to full, stopping the device from powering down, or access to sensors such as Bluetooth signal strength. Therefore native clients (e.g. "Apps" or mobile applications) running on the mobile devices Operating System (e.g. Android, iOS, etc.) may be used instead of Javascript, using the same protocols to

communicate with the server 40, but allowing other functionality to be enabled.

Using a native App client, the system can automatically increase the display brightness to full and achieve other advanced functionality. Native Apps will also be able to control the camera flash source on most smartphones. This could be used to create monochrome effects, but with much greater brightness that will read in daylight conditions. When using a native App, the same method of joining the pixel surface can be used, i.e. scanning a QR code or RFID tag with embedded URL. But the web page returned by the server 40 will now contain an option to download and install or start the App, passing any embedded

parameters such as surface identity and location.

Both the HTML5 web client and App clients will allow features to encourage user engagement, for example warning when pixels are about to activate, sending messages out to users, advertising content, links to websites where images / video of the live effect can be downloaded etc.

The aims of the system (very large pixel numbers over very large areas) make wireless connections most practical in most situations, and make public internet wireless WAN connection (primarily mobile internet 3G and 4G) the preferred way of communicating between devices 20 and the server 40. However other wireless systems can be used, for instance WiFi. In particular a dedicated WiFi mesh can be setup at events, directing any connected devices to the correct URL automatically. This has the advantage that the 3G / 4G base station will not become the limiting factor on pixel numbers. Even wired connections such as Ethernet can be envisaged in the use case of custom pixel devices rather than mobile phones.

With very large numbers of pixels, network congestion is likely to be a problem. The system is therefore designed to minimize network traffic. Many internet based applications are 'pull' systems, even if they appear to be push systems through the use of techniques such as long polling. In other words, any information that is sent from the server to the client is the result of a request from the client to the server. Using pull techniques, very frequent polling (perhaps 10 times per second) will be required in order to achieve satisfactory synchronisation between the pixels whilst achieving low latency control. This will lead to a very high number of network requests even when no image is being projected. Pull techniques are mentioned for example in EP1870802. Such techniques are unlikely to be easily scaleable. Other 'pull' techniques that have been proposed involve a device pulling packets of information containing fixed future attribute changes relating to all devices, and calculating at the device which attributes to display dependent on their ID or position. This does not allow for real time low latency control, and creates large network overhead.

Other 'pull' techniques proposed involve using a push notification from the server to one or to all devices, which triggers a pull of attribute information from the server. This avoids repetitive polling as outlined above, but introduces unnecessary latency into the system and increases network traffic.

As mentioned above, the preferred system 1 uses Websocket connections. This protocol sits on top of the HTTP protocol, so will pass through most networks, routers and proxies that are designed for web traffic. This is key to being able to communicate with remote servers over public WAN. Websockets are also a true push technology. The connection between server and client remains open for long periods with no network overhead. When the server is ready to send a message, it is dispatched with minimal overhead - theoretically only two bytes on top of the payload. Websockets also allow (limited) functionality to be achieved within web browsers as the client application, allowing compatibility with a greater number of different client devices. As an alternative using Websockets, the UDP protocol could be used to communicate between the server and the mobile devices 20. This would also achieve most of the advantages of using Websockets noted above and indeed may have possible lower latencies and lower overhead than Websockets. Various schemes may be implemented for sending the data to the mobile devices 20.

In one example, in order to minimize network traffic, the data sent is just the new display attribute, e.g. the colour the pixel in question is required to display. This means that colour information need only be sent to the client when necessary, not at a regular frame rate. If no pixels are changing, there is no network traffic at all. The system is designed to be able to display live video feeds. It therefore needs to be able to operate in a "real time" mode where colour information is sent at substantially the moment it is to be displayed.

However the transport protocols used mean that sometimes messages reach the device quickly, but sometimes are delayed by as much as 5 seconds or more. In this scenario, some devices will suffer from greater latencies than others, and connection bandwidths may differ too.

In another example, in order to cope with network delays and outages, an extended scheme may be used in which a cache of future attribute changes are stored at the mobile device 20, thereby forming a buffer of the display attributes. This works as follows:

The server 40 can send a command including only a display attribute (e.g. a 3 byte representation of the colour which the screen is required to display). This will immediately be displayed by the mobile device 20.

The server 40 can send a command including a display attribute and a delay attribute. In this case the mobile device 20 will wait for the specified delay before displaying the new colour. This delay is stored and is applied to any future commands containing only a display attribute.

The server 40 can then continue to send commands containing only a display attribute, each will be delayed by the same amount. This is equivalent to a fixed frame rate.

The server 40 can also send a command including a display attribute and a new delay attribute. In which case the new delay is stored and used for this and all future colour changes.

All of these commands can be sent at a faster rate than the required delays, allowing a cache to build up on the client side.

The server 40 can send a "flush" command to the device. In this case, the cache is emptied, any stored delay value is cleared, and the next colour (and future received colours) is displayed immediately. The flush command may be within the stream of attribute/delay values - for example it may include a display attribute and a new delay attribute where the delay is zero indicating it is to be displayed immediately. Alternatively, the flush command can be a separate system message to flush the cache that can be appended to a bundle of attribute messages or send independently to attribute messages.

If it is determined that the frame rate for that device is too high, the minimum allowed delay for that device 20 only may be changed at the server 40, so the frame rate varies by device. If the determination is made at the device 20, this is fed back to the server 40 to adjust the frame rate. The determination that the frame rate is too high can be made for example by identifying that display attribute changes are being received after the required display time at a particular mobile device 20, i.e. that the cache is empty, indicating that the frame rate is too high. Alternatively or additionally the rate of dropped packets at the device 20 can be used. Alternatively or additionally using a websocket protocol allows the server 40 to identify where messages are being sent before the previous message has been received.

The above calculations do not take account of the latency of transmitting commands from the server 40 to the devices 20, which furthermore may vary between devices 20. Synchronisation of the mobile devices 20 may be achieved by requiring each device to reply to certain attribute change commands or other messages from the server 40 with timing information. The server 40 times the round-trip, allowing an estimate to be made of the latency of the message. From this latency estimation and the timing information received a synchronisation error can be calculated and a corresponding offset which can be applied to synchronise the server and devices. The required offset can then be

communicated to the device to be applied immediately to all attribute changes. Alternatively, the offset could be applied at the server to adjust the times at which the commands are sent.

In one example, a 'latency check' flag can be included with a number of selected attribute messages originating from the server 40, or periodically with command messages requesting timing information from the device 20. The device 20 replies with its position (e.g. in milliseconds elapsed) in the current stream of attributes relative to a convenient reference point in the stream, which typically will be the beginning of the stream, e.g. the first attribute in the stream after the last cache reset or the device joined the surface. The server 40 averages over a number of such replies to give a current weighted estimate of synchronisation offset for that device 20. At the device the new offset factor is applied to any future attribute changes. Thus, in detail, the method may proceed as follows:

1 - Server 40 sends the first message in a stream and starts a timer Ts

2 - The message is received at device 20 which starts its timer Td. N.B. The timers are used by the server and devices as the timing reference for sending commands and changing the displays. Due to the finite outward trip time of the message, the two timers are not initially synchronised. The aim of the method is to find an offset to apply to each device timer to synchronise the timers, i.e. synchronise the server and all the devices.

3 - A later message requesting a latency check is sent from the server which starts timer Tm and records the value of Ts as Ts1. N.B. the first message in the stream might request a latency check so the synchronisation process begins as soon as the display of attributes begins.

4 - As soon as this message is received by the device it replies with the value of Td as Td1 to which is added any current offset (initially zero) being applied by the device. N.B. in this example, the device timer is not adjusted per se, but an offset stored and applied to achieve synchronisation. Thus, the time sent back is the current "synchronised" time.

5 - When this response is received at the server, the value of timer Tm as Tm1 is noted which gives the roundtrip message time. If server and device are perfectly synchronised then the sent time Ts1 plus half the round trip (Tm1 / 2) should be equal to the time sent back by the device. The measured offset is then given by the difference Ts1 - Td1 + (Tm1 / 2).

6 - This offset is sent back to the device and used to adjust all attribute changes with immediate effect. N.B. as mentioned above, an offset is stored and applied at the device, rather than the device timer being permanently adjusted per se. Thus the offset sent back is added to the existing offset at the device. Thus, with each request for timing information, the offset is successively refined at the device to eliminate any remaining synchronisation error and achieve

synchronisation between timers. The above calculation assumes that the round trip is symmetric, i.e. outward journey of a message from server equals inward journey of a message from device. However in reality the round trip time can be very asymmetric and the outbound journey (which is estimated at Tm/2 in the above calculation) is uncertain and could be anything between zero and Tm with an average of Tm/2. The error E (standard deviation) in a measured offset is estimated as a function of the round trip time, e.g. Tm/Factor, where the Factor is chosen to give the best results. A factor of sqrt(12) has been found to give good results. Thus, where the round trip time is shorter, the confidence in the measured offset is higher and this is given a higher weighting compared to measured offsets resulting from longer round trips, where there a large amount of possible asymmetry and the confidence in the estimated outward trip time is accordingly lower.

Thus, in detail, the method may proceed as follows to average corrections over a number of measurements for future measured offsets (i.e. where there has been at least one offset calculated already):

7 - Future measured offsets have their own error Em. The measured offset is multiplied by a factor at the server before being sent to the device so that each measurement is weighted in proportion to the inverse square of the error E. To calculate this factor the server can record the previous error (Es) and the previously applied cumulative offset.

8 - At the device the new offset is added to the existing one.

9 - At the server a new value of Es and a new value of the cumulative offset is stored

10 - Future measured offsets are reduced by an error weighting factor based on the current error estimation, the current cumulative offset and the error in the new measurement. The stored error and stored cumulative offset is updated.

Thus the method weighs successive measurements in proportion to confidence to refine the estimate. As would be appreciated by a person skilled in the art, the actual implementation could be done a number of ways and is not necessarily limited to the exact example given. The method allows attributes to be displayed immediately, without waiting for a significant time whilst synchronisation is calculated. Successive requests for timing information can be used to

successively refine the offset used to synchronise the device. By using the information about round trip time to produce a weighted average, all

measurements can be factored into calculating the offset in an appropriate way, which allows the devices to quickly synchronise to the server. These are important advantages of this technique.

In a simpler example, a One time' estimate of latency can be used based on the round trip time of an initial display attribute. For example, the initial display attribute is sent to be displayed immediately (i.e. with no delay parameter) at to with a request to reply. Reply is received by the server 40 at time t1. A simple estimate of the latency is (t1 - to / 2), i.e. half the round trip time. This means that all attribute changes, the original one and any later ones that have already been sent with delays attached are all assumed to be displaying late by this amount. This offset can be communicated to the device and used to adjust future attribute changes. Alternatively the offset can be applied at the server.

This allows a frame rate which can be made to vary at different times during the visual effect (and can even be made to vary every frame if desired) and/or can be made to vary for each device. If transmitting full frame video pictures, the system will behave very much like a fixed frame rate, with low network overhead.

However if other visual effects have pixels that change only rarely, the system will still only push colour information when needed. Thus, a very flexible system is provided by using this caching scheme where the overhead required to implement caching is just the delay values since the last attribute change, which are small and so minimize network traffic. Additionally delays are sent using a variable length encoding scheme such that short delays may be encoded in a single byte, but no practical limit is placed on the maximum delay that can be communicated. This compares well with possible alternative systems where a full timestamp is provided for each message to indicate when they are to be displayed, which has a high associated overhead. At the same time, much more flexibility is provided compared with systems which just utilise a fixed frame rate. An additional advantage of using delays is that no clock synchronisation service (e.g. NTP) is required. The system is therefore capable of running as a web app in a browser as well as in native applications. Synchronisation is provided through the use of relative stream offsets calculated through repeated measurement of timing information contained in responses to attribute collection or other server-sent messages.

The network overhead is further decreased where attribute changes take place at regular intervals by applying the previous delay to the following attribute change if no new delay is specified.

This variable frame rate scheme is capable of real-time low latency control, and fixed frame rate cached changes with only colour attributes being sent, and of variable rate cached changes with only the mimimum of delay information sent from the server, as well as the real-time attribute changes of the previous embodiment.

If a potential network bottleneck has sufficient bandwidth to stream one HD video, then the preferred system will scale to hundreds of thousands of devices 20 over the same connection.

In a simple implementation, the system can operate by having a single frame currently being processed on the server. Once attributes are pushed to devices to fill client side buffers, the currently processed frame was some time in the future, e.g. 5 seconds. Devices joining the surface would then have to fill their buffer for 5 seconds before being able to display any colours. In another implementation, the server may be capable of calculating attributes at an arbitrary future time to allow for devices to join the surface at different times. In this implementation the server is able to read ahead in any frame based media source by an arbitrary amount. This allows newly joined devices to be sent the colour required for display immediately, while other devices are at the same time being sent colours for caching and then display many seconds and even minutes in the future. This allows for much greater client buffer sizes whilst new devices display something immediately. 1.1 Peer-To-Peer Compute / Data Grid

A centralized system running on a single server may be adequate for some implementations, but it is not easily scaleable. A single server may be unable to undertake various tasks such as communication with the devices, calculation of positions, storing required information and calculation of display attributes if there are very many display devices. Therefore in order to allow for very large numbers of pixels (say, in the millions), the use of multiple servers 42a..d to carry out some or all of the tasks is preferred. There are potentially many ways to distribute data and computing across more than one server, for example controller-slave configurations are known. However, with very large device numbers, any controller-slave system may suffer from both network bottlenecks and CPU / memory limitations. The controller must place some limit on the number of pixels, even if this is very large. Therefore a controller-slave configuration is not preferred. Referring again to Figures 2 and 3, the preferred system 1 utilizes a peer-to-peer grid of servers/computers 42. The software on each server 42a.. d, and the tasks undertaken by it, may be identical in each case. In this way the resources available on a single server 42a.. d do not determine the capacity of the overall system 1.

In one example all of the storage and computing is carried out across the multiple servers 42. This includes communication with the mobile devices, storing device state information, calculating display colour attributes, storing device positions, position error and range information, and calculating device positions. This means that no aspect of storage or computing is constrained by the capability of a single server.

In other examples, some storage or computing functions may take place on a cluster of servers, and other functions on a single server. While this may introduce a limit on the scalability of the system, it may be advantageous in other ways.

In a particular example it may be required to stream a live video feed to the display formed by plural pixels. In this case the calculation of display attributes may be carried out by an independent system comprising a single server, and these attributes may be sent to the cluster of servers that carries out the other storage and computing functions, e.g. calculating device positions and/or communicating with devices.

A load balancer is one option that is sometimes used to distribute requests evenly between servers 42a.. d. However the load balancer itself introduces a constraint on system capacity (just like any other controller) and is therefore not the preferred way of distributing requests.

Instead, the system 1 uses DNS round-robin to initially route the Websocket connection 80 of the client (i.e. the mobile device 20) to a single server 42b in the grid 42 (steps 81 to 83 in Figure 3). This server 42b adds details of the new pixel device 20 into a distributed map 44 of the devices stored in a distributed data grid (for example, using a system called Hazelcast), which is stored in memory for speed of access, and distributed evenly across all servers 42a.. d in the grid 42. The Hazlecast system chooses the actual server 42a.. d on which to store the details using an algorithm to apply even distribution of data across the grid.

Once the identity of the actual server 42a for the data is known, a "migrate" instruction is sent to the client (step 86 in Figure 3) so that it opens a new Websocket connection 80 to this server 42a, and closes the original one (step 87 in Figure 3).

In this way the storage of details of pixel devices 20, such as absolute position (calculated for example by GPS, trilateration of beacons, and barcode scanning) and RSS (received signal strength) from neighboring pixel devices is stored across a peer-to-peer grid, and therefore the capacity of this database and the speed of storing and retrieving data, are not limited by any one machine.

Therefore an effectively limitless number of pixel devices 20 can be handled. This is managed automatically by the Hazelcast distributed map 44. Similarly the Websocket connection 80 between the server 42a.. d and the client 20 is distributed evenly over the grid 42, following the physical location of the pixel entry in the map 44. When additional servers 42a.. d join the grid 42 (either automatically scaled when CPU load increases, or when software predicts load will increase, or manually in response to expected load increases) Hazelcast will automatically migrate pixel information from one map member to another in order to equalize data storage load. Migrate instructions are then sent to the client 20 so that the websocket follows the pixel data, ensuring websocket load (the cause of significant CPU load in the system) is also equalized across the grid 42.

With no migration system, the load on the servers has to be anticipated, and sufficient servers 42 started in advance. With migration techniques, the server grid 42 can scale dynamically as load increases. The servers 42 may be implemented as virtual servers located on Amazon's elastic compute cloud 43.

In some situations it may be preferred to use local servers. For example, a local WiFi mesh may be created specifically for an event. As noted elsewhere this may be advantageous both to bypass public WAN bottlenecks and/or to provide fixed beacons for RSS positioning. Where local WiFi networks are used, the use of local servers becomes possible. Local servers can be setup running exactly the same software packaged in exactly the same way. The same public URLs can be used by devices in contacting the system to reduce complexity. Routers will redirect attempts to access the public URL to the local servers. In fact load could easily be shared between local servers and cloud servers if sufficient bandwidth is available.

2 Positioning Techniques

In the example described above, the smartphone's GPS position 50 is used to locate the pixel device 20. However this technique is not always possible or optimal. Reasons for this include the fact that the GPS system 50 will not work indoors or in locations with only a partial view of the sky, for instance inside conference facilities, arenas and stadiums. In addition, the accuracy of GPS positioning is sufficient to create useful effects, but may in places and at times only give resolution of around 10m. Higher accuracy may be desirable to achieve higher resolution images. Therefore systems for positioning pixels more accurately, and for positioning pixels indoors, are desirable.

2.1 Barcode Positioning

In one embodiment, for seated events, a unique barcode 60 is used to position every user in a particular location 61 , e.g. seat. The barcode 60 is scanned by the user of the mobile device 20 prior to the device 20 joining the image and the barcode information sent to the server 40. The mapping between the unique barcodes 60 and their respective locations in the image area is predetermined and stored at the server 60. In practice the server may not hold a complete one to one map, but an algorithm that given sector, row and seat number can calculate position. Thus, the server 40 just needs to look up/calculate the position of the barcode 60 to find the position of the mobile device 20 that scanned it. This provides very high resolution positioning, works indoors, and is low cost. In practice a QR code 60 might be attached to the back of each seat 61 in the arena 10. As described above, in a preferred embodiment barcode information contains a url which identifies not only the location of the barcode (e.g. seat 15) but also the URL used to connect to the server and the event/pixel surface which the device is to join.

Alternatively, the barcode 60 can be placed in other places, such for every row of seats, or for every aisle in the arena or for every entrance/exit. As will be appreciated, where there is not a unique barcode for the location of each user/mobile device 20, it is not possible to distinguish between those mobile devices 20 using just barcodes 60. Accordingly there is a loss of resolution in the image displayable by the pixel surface. However, this can still be capable of achieving some interesting visual effects and may be sufficient in some applications.

RFID tags can be used in a similar way instead of or in addition to using barcodes 60 to determine the position of the mobile device 20. In principle, any position markers which are positioned at predetermined positions in the image area and which can be scanned at close range (e.g. within 10 meters) by a mobile device 20 can be used. As long as the position marker encodes a unique position identifier which can be detected by the mobile device 20, it can be sent to the server 40 and its position looked up or calculated to determine the position of the mobile device 20.

2.2 Positioning by Trilateration of Fixed Stations

WiFi and Bluetooth signal strengths may be used to trilaterate position against fixed absolute markers (e.g. a number of WiFi base stations located around a stadium). For example, a number of WiFi base stations 90 with well known TX strength can be placed in known locations designed to provide the most orthogonal trilateration or multilateration for the whole pixel surface. The mobile devices 20 detect the received signal strength and feed this information back to the server 40. The server 40 knows the locations and preferably the TX strengths of the base stations 90 and can use this information to trilaterate or multilaterate the position of the mobile devices 20. The preferred embodiment is to use WiFi base stations, but Bluetooth signals, UWB, RFID signals, or any other signal detectable by the mobile device 20 could be used. The preferred embodiment uses signal strength to estimate distance, as WiFi RSS (Received Signal Strength) and Bluetooth signal strength are easily available on typical mobile devices 20. However, time of arrival measurements (TOA) could provide more accurate estimates of distance. Currently most available mobile devices 20 are not equipped to easily determine TOA.

However, it is anticipated that in the future mobile devices 20 might be capable of TOA measurements of Bluetooth, UWB or other signals, which might then be a preferred way of estimating distance.

3 Wireless Ranging

There is now described a method and system of determining the position of a mobile device 20 within a dense crowd of mobile devices 70 by reference to a wireless signal (WiFi and Bluetooth or audio or in the future UWB) generated by each device 20. The method has particular application with the pixel display technology described above in allowing the server 40 to find the position of the various pixel devices 20. However, the technique has applicability for any scenario where it is desired for a central computer 40 to find quickly and accurately the positions of remote mobile devices 20 in a dense crowd.

Periodically the mobile device's App will perform a Bluetooth 'enquiry' (or 'scan' for Bluetooth Low Energy) during or after which the OS will provide for the App a list of all mobile devices 70 that are 'discoverable' (or 'advertising' for Bluetooth Low Energy) within Bluetooth range, the data including their unique MAC address (or a generated universally unique identifier for Bluetooth Low Energy), and the RSS (Received Signal Strength). This map (pairs of MAC addresses/unique identifiers and signal strength) will immediately be reported to the server 40 using the open Websocket connection 80. One disadvantage of Classic Bluetooth enquiry is that it can take some time to report a full scan of the neighbourhood (typically 10 seconds). However in the present application devices 20 are largely static. As Bluetooth Low Energy becomes more widely available, the speed of advertising and scanning will increase. An extension to the originally considered embodiment extends to moving devices. As described above the movement of devices can be accounted for by reducing the weight of observations as a function of the observation's age. In an extension to the embodiment, the velocity of each device can be stored alongside the position, and state estimation techniques such as Kalman filtering can be used to estimate the position of moving devices. Alternatively another extension utilizes dynamic multidimensional scaling as described by (Jamali-Rad and Leus 2012).

Each measured RSS is likely to contain a large degree of error due to

shadowing, multipath effects etc. Also the devices in question (smartphones) are heterogeneous, i.e. we do not know a priori their Bluetooth TX power or RX sensitivity. In almost all the wireless sensor literature homogenous devices are assumed. One advantage of using Bluetooth 'enquiry' is that it ensures the transmitter uses the maximum transmit power. This removes power-saving adaptive TX power problems. By assuming all available smartphones have roughly the same maximum TX power, and using an average in inferring distance from RSS, we can consider TX differences to be part of one error term that also includes path- loss differences, multi path effects and shadowing.

For Bluetooth Low Energy ranges, the TX power may be transmitted in the advertising packet to the scanning device. In a very crowded place (for instance a concert or sports event, exhibition or conference) there may be a very large number of "neighbours", as there may be up to 300 people within Bluetooth range, or in the case of Bluetooth Low Energy, thousands of devices. The large number of measured signals and their short range enables a high accuracy to be attained in the calculated position despite the high noise present in individual measurements. In such circumstances it is possible that the number of RSS measurements is undesirably high. More and more ranges to be reported may generate unacceptable network or computational overhead, and provide little incremental positional value. In these cases the system may be range adaptive, reporting and retaining only the most useful information. At a simple level this may involve ignoring the lowest RSS (furthest) neighbours. However in the preferred embodiment it results from ignoring those RSS measurements with the highest range/position error combination.

3.1 Mobile Device Software

Simple software is required to operate on each device 20. In the case of smartphones this will take the form of an 'App' that operates, for instance, on Android or iOS operating systems. Therefore no hardware, OS, or other modifications are required. The app on each smartphone 20 will, on starting, request permission from the user that the device becomes Bluetooth discoverable. On the latest Android platform this can enable discoverability for up to an hour. This allows the App to scan the area and report the RSS (received signal strength) of all neighbouring devices 70. For Bluetooth Low Energy devices, the device will if possible start advertising its unique identifier. On Apple (RTM) devices this has been possible since iOS version 6.

The App will also, on starting, open a websocket connection 80 to the server 40, and report its unique address (MAC address or unique identifier).

Periodically each device 20 will report its position (if known through GPS 50 or some other self-localization method) to the server 40. In an embodiment, GPS absolute position is only reported if its distance from the previously reported position exceeds a threshold. This reduces unnecessary network traffic and server computation. Periodically each device 20 will carry out a Bluetooth enquiry and report any discovered MAC address/unique identifier and RSS pairs to the server 40.

3.2 Server Software

As described above, the server software is preferably run on a scaleable grid of servers 42 in order that it can scale to very large numbers of devices 20.

Information about each pixel device 20 (its currently estimated position, position error, last GPS fix and RSS table) is held in a distributed map 44. In one example, as each new set of RSS is received, the server cluster 43 will use the following algorithm (as shown by Figure 4) to iteratively improve its map of estimated device positions 44.

A set may comprise a number of Received Signal Strengths (RSS) from a completed Bluetooth Inquiry, or may comprise as few as a single Bluetooth RSS, with the algorithm capable of iteratively enhancing position estimates using single ranges at a time (SRAT).

1. The RSS for each immediate neighbour of the reporting device (i.e. for whom the server knows an RSS observation) is retrieved from the distributed ranges map.

2. For each device in range, distance to that device is calculated from RSS based on a generic log normal path loss model, using empirically determined average TX power / path loss.

3. The error in the range estimation to the neighbour is calculated. In the simplest model for Bluetooth, the error is proportional to range.

4. Error in the neighbour's position is looked up from the distributed map 44. 5. The two error terms are combined.

6. If there not are enough neighbouring devices 70 with sufficiently low error, called "anchors" herein, the process moves an additional "hop" to include neighbours of neighbours, and so on until there are sufficient anchor devices identified, i.e. devices with sufficiently low position error.

7. If the reporting device 20 does not currently have a location stored in the distributed cache, the provisional location of the device 20 is estimated as the weighted centroid of the selected anchors.

8. A distributed weighted multidimensional scaling method (dwMDS) is

iteratively used to calculate the new position of the device 20 using selected anchor devices. Each anchor device used is weighted according to its combined position / range error.

9. The error of the device's new position is estimated and stored.

10. This new position is then used to revise neighbouring positions, then

neighbours of neighbours, and so on to propagate the new information through the network.

Although the calculation is performed centrally, the algorithm is distributive and iterative by nature. This means that calculations can be performed in parallel across the server grid 42 greatly increasing speed.

For non-neighbours, distance is most simply estimated by summing the ranges of each hop and combining their errors, but more complex algorithms are possible. Estimating distance from hop-count is a well known technique. See (Shang, Ruml et al. 2003) and (Patwari and Hero lii 2003) and (Li, Shi et al. 2005)

When GPS data or other absolute position data is received from a device 20, the following steps (as shown by Figure 5) are performed:

1. Store the position and error of the GPS observation

2. The observed GPS position is combined with any existing position

estimate weighted according to the reported and estimated errors, and the resultant position stored as the updated position estimate 3. dwMDS iterations are initiated on neighbours of the device 20 to diffuse the new information through the network.

In another example, a positioning algorithm is used based on using a single range at a time (SRAT), as described below. Whilst this does not have all the advantages of the technique described above, it allows very fast processing of large numbers of observations with only a small reduction in accuracy. As each new single RSS is received, the server cluster 43 will use the following algorithm to iteratively improve its map of estimated device positions 44. The algorithm proceeds as follows:

1. If the observed (TX) device has no estimated position, the range is stored for later use in positioning the observing (RX) device. 2. Otherwise, if the reporting device (RX) has no estimated position already, it is assigned the estimated position of the observed device, with an error weighting calculated to be theoretically equivalent to the range being used in traditional dwMDS. 3. Alternatively if the reporting device (RX) has an estimated position, this is revised (along with the error estimate) using the existing position and its error estimate, the observed device's position and its error estimate, and the observed range and its error estimate as inputs. If the TX device has a position stored in the map 44, the same calculation is applied to revise its position, using the position for the RX device prior to its update.

4. If the TX device has no position in the map 44 but the RX device does, it is assigned the position of the RX device, with an error weighting calculated to be theoretically equivalent to the range being used in traditional dwMDS.

5. If the RX device had no previous position estimate but now has one, the algorithm is applied to the stored unused ranges relating to its other neighbours. A series of such calculations is theoretically equivalent to carrying out

conventional dwMDS for one device but in discrete steps. However no initial estimate such as centroid positioning is needed. Each device gets an estimate as soon as it is possible, and that estimate is theoretically as accurate as possible. Importantly each range that is observed gives rise to two position calculations only (one for transmitter and one for receiver), which massively reduces the computational cost of estimating positions. 6. In an extension to the algorithm, if the position estimate of the reporting device (RX) changes by more than a threshold, its neighbours can have their position updated to account for this. This is achieved by 'undoing' any changes that were caused to neighbours' position estimates prior to the update, and 'replacing' the change using the new estimate. This is possible if the 'stress' from the previous MDS calculation is stored along with the range data.

In this embodiment using the SRAT algorithm, as each new GPS or other absolute position observation is reported, the server cluster 43 will use the following algorithm to iteratively improve its map of estimated device positions 44.

1. If the device has no current position estimate in the map 44, the reported location (transformed from Geographic coordinates to x,y values) is stored, along with the RMSE error estimation that is calculated from the reported precision of the GPS observation.

2. If the device has a currently estimated position, the current estimate and the new observations are combined in inverse proportion to their estimated variance to give a new position stored in the map 44. In the above examples, the only factor that is not accounted for is variable device TX strength. In an extension to the envisaged embodiment, variable TX strength can be accounted for in three ways. It can be modeled as an additional dimension in the device state vector, it can be removed from the calculations by considering the difference in RSS (Received Signal Strength) rather than absolute RSS transmitted by an individual device, or it can be estimated from the estimated position and range maps as a separate iterative step following dwMDS positioning.

In the current system a previously estimated value is used for the pathloss exponent used in estimating distance from RSS. It may be possible to use statistical or other methods to dynamically estimate the pathloss exponent for different regions, allowing more accurate results to be obtained.

In order that the system can work without any special infrastructure being installed at a site, the system is envisaged operating using public WAN wireless connections (primarily 3G and 4G). The prior art discounts centralized calculation partly on the grounds of network traffic. The system is therefore designed to minimize network traffic.

3.3 Audio wireless ranging

In another example, audio signals can be used instead of RF signals as the wireless signal to be used in range calculations. Mobile devices commonly have speakers/microphones for transmitting/receiving sounds to/from the user, which can be employed for audio wireless ranging. Ultrasound audio signals can also be used where the mobile device is capable of sending/receiving ultrasound signals.

Audio signals have the advantage that they can be easily send and received by mobile devices even from within a web browser application without requiring native code. However the interference between simultaneous signals requires that they are used differently from RF signals. Rather than the devices all broadcasting and scanning simultaneously, the server is used to coordinate signal transmission. 1. When the server requires the position of a device for which no estimate exists in the map 44, and the device in question is not capable to using other available signals (e.g. Bluetooth), the server sends a command to the device over the websocket connection to emit a unique identifier modulated as audio.

2. The device will use well known techniques such as frequency shift keying (FSK) used in modems to encode its device ID sent by the server into audio emitted by the device's speaker. The device may monitor the frequencies of background sound by using fast fourier transforms (FFT) on the device's microphone output in order to select frequencies for the FSK that are least likely to suffer background interference.

3. Other active devices will be monitoring the device's microphone output for FSK encoded or other signals.

4. On detection of a signal the receiving device will report the id of the TX device and the signal strength (audio amplitude) to the server over the websocket connection. 5. The server will use exactly the same techniques outlined above to process these audio range observations.

3.4 Error fading and anti-aliasing An advantage of storing the estimated error in estimated positions is that the error can be used in deciding whether to send an attribute to a device at all, or to modify the attribute's value.

In examples some effects where all devices are synchronised can result in attributes being sent to all devices. Other effects that require low resolution positioning can result in attributes only being sent to those devices with at least some position estimate. Effects that require high resolution positioning can result in attributes being sent only to devices with low error position estimates.

In examples, the error of the estimated position can be used in calculating the attribute to be sent. This 'error anti-aliasing' would mean for example sending a colour to a device that is the average colour for pixels within the error radius of the device.

In examples 'error fading' can be used to display colours with higher brightness for devices with lower estimated position error.

3.3 Seeding

If a surface (or part of surface, particularly the edges of an arena 10 for instance) has too few well positioned devices, Bluetooth capable devices with known position can be introduced into the surface to allow position to diffuse from them. No changes to the system would be required. A custom device with Bluetooth could be cheaply produced. It would interact with other devices using Bluetooth, with its a priori known position being added to the distributed device map in advance. Generic Bluetooth beacons such as iBeacons (RTM) can be used for just this purpose.

If there are sufficient well positioned devices and/or beacons, useful absolute positions may be calculated for devices without any cooperative positioning being required. In this edge case the methods described herein become a useful system for processing a combination of self-position observations and beacon signal strengths to derive an error-minimised position estimate.

3.4 Coordinate transforms

The server 40 receives position data from devices 20, and transforms this into a surface position. Case 1 : Position Data is a GPS position providing Latitude, Longitude and Elevation together with a position standard deviation error term. This can be transformed using available transforms into Cartesian coordinates, preferably UTM projections. This can then be converted easily to a local constrained Cartesian coordinate system defined by base point and extents for ease.

Case 2: Position Data is an identifying reference number or string read from a barcode 60 e.g. QR code on the back of seat 61 in a stadium 10. A custom transform reads the identifier and converts it using a lookup table (specific to a venue like a stadium) and/or calculation to elevation and azimuth within the stadium.

Although the examples described above are all envisaged as using user-supplied pixels (smartphones and tablets), custom devices could easily be made with the sole function of operating as remote pixels. These could be handheld, or fixed. Fixed units would allow Olympic stadium type effects to be setup and operated much more efficiently than current technologies. Custom devices could control any light source, for instance glowing floating helium balloons making a video picture, or performer costumes incorporating LED or other display elements.

Embodiments of the present invention have been described with particular reference to the examples illustrated. However, it will be appreciated that variations and modifications may be made to the examples described within the scope of the present invention.

Claims

1. A method of forming an image using plural pixel devices, the method comprising:

forming a communication channel over an Internet Protocol (IP) network between each of the pixel devices and a remote server;

calculating at the server a display attribute for each of said pixel devices in accordance with the image to be formed;

2. A method according to claim 1 , wherein push technology is used to communicate between the server and the devices.

3. A method according to claim 1 or claim 2, wherein the display attribute sent by the server comprises colour information, the pixel device being arranged to change its display to display that colour.

4. A method according to claim 3, wherein the server only sends the display attribute to the pixel device when the display is changing.

5. A method according to any of claims 1 to 3, comprising the server sending a delay attribute to a pixel device, the pixel device thereafter waiting for a period indicated by the delay attribute between each change of its display.

6. A method according to claim 5, comprising sending different delay attributes to different pixel devices and/or sending different delay attributes to the same pixel device at different times during the display of the image so as to spatially and/or temporally customise the frame rate.

7. A method according to claim 5 or claim 6, comprising caching display attributes at a pixel device if they are received at a faster rate than they are to be displayed.

8. A method according to any of claims 5 to 7, comprising the server timing the round trip time of a message sent to the pixel devices, and calculating an estimated latency for each pixel device in accordance with the round trip time, compensated for in calculating future delay attributes.

9. A method according to any of claims 1 to 8, wherein the server is arranged to add new pixel devices to the image dynamically.

10. A method according to any of claims 1 to 9, wherein the server comprises a peer-to-peer network of plural peer servers, each peer server being arranged to communicate with different pixel devices in order to form the image.

1 1. A method according to any of claims 1 to 10, wherein each peer server is arranged to calculate the positions of pixel devices and/or calculate display attributes to be communicated to the pixel devices.

12. A method according to claim 10 or claim 1 1 , comprising:

forming an initial communication channel between a new pixel device and a first server of the plurality of servers;

at a later time, allocating the pixel device to a different server and migrating the communication channel from the device to the different server in order to balance the load on the servers.

13. A method according to any of claims 10 to 12, wherein at least one peer server is a virtual server, the method comprising dynamically adding a new instance of a virtual server when the load or expected load on the existing servers reaches predetermined level.

14. A method according to any of claims 1 to 13, comprising:

sensing with a sensor of the pixel devices position data indicative of the position of the pixel device;

receiving at the server said position data from said pixel devices;

determining at the server the position of each pixel device in accordance with said data;

calculating the display attribute for each of the pixel devices in accordance with the determined position of that pixel device.

15. A method according to claim 14, comprising:

sensing in the vicinity of the pixel device one of a plurality of position markers positioned at different predetermined physical locations, the position markers encoding respective unique identifiers;

the server calculating the predetermined physical location for that identifier to obtain the position of the pixel device.

16. A method according to claim 15, wherein the position marker is a barcode and the sensor is a camera associated with the pixel device.

17. A method according to claim 15 or claim 16, wherein the position marker also encodes a web address for server, the pixel device decoding the web address and connecting to the server indicated by the web address to join the display.

18. A method according to claim 14, comprising:

providing at least three wireless transmitters at predetermined locations; sensing with a sensor of the mobile device the wireless signal to obtain information indicative of the distance from the mobile device to the wireless transmitters;

receiving at the server said information from said mobile device; calculating at the server the position of the mobile device using the distance information for the at least three wireless transmitters.

19. A method according to any of claims 14 to 18, comprising calculating the error in position of the pixel device from said position data and calculating the display attribute in accordance with the position of the pixel device and the error in the position.

20. A method of determining the positions of a plurality of mobile devices, each mobile device having a transmitter and receiver for a wireless signal, each mobile device being within wireless sensor range of at least one other of said mobile devices, the method comprising:

receiving at a server said information from said mobile devices;

21. A method according to claim 20, comprising a mobile device emitting said wireless signal of discrete duration at a time interval determined by the server.

22. A method according to claim 20 or claim 21 , comprising receiving at the server the absolute position of plural of the mobile devices and using this together with the received distance information to determine the absolute map of mobile device positions.

23. A method according to claim 22, wherein the absolute position is determined by GPS, scanning a barcode, or detecting a wireless transmitter with a known position.

24. A method according to any of claims 21 to 23, comprising using multidimensional scaling to infer position from said distance information and any prior known absolute positions to determine said map.

25. A method according to any of claims 21 to 24, comprising seeding the network with at least one device having a predetermined known position.

26. A method according to any of claims 21 to 25, wherein the positions of plural mobile pixel devices are used in the method of forming an image using plural mobile pixel devices according to any of claims 1 to 19.

27. A method according to any of claims 1 to 26, wherein the method is carried out in a stadium or arena.

28. A computer program constructed and arranged to carry out the method of any of claims 1 to 27.

29. A system comprising a server and one or more mobile device, the system being constructed and arranged to carry out the method of any of claims 1 to 27.

30. A server for forming an image using plural pixel devices, the server being constructed and arranged to:

31. A server according to claim 30, the server being constructed and arranged to:

receive from the pixel devices sensor data indicative of the position of the pixel device;

determine the position of each pixel device in accordance with said data; calculate the display attribute for each of the pixel devices in accordance with the determined position of that pixel device.

32. A server for determining the positions of a plurality of mobile devices, each mobile device having a transmitter and receiver for a wireless signal, each mobile device being within wireless sensor range of at least one other of said mobile devices, the server being constructed and arranged to:

33. A computer program for a pixel device for forming an image using plural pixel devices executing that program, the computer program comprising program instructions which when run on the pixel device cause it to:

34. A computer program according to claim 33, the computer program comprising program instructions which when run on the pixel device cause it to: sense with a sensor of the pixel devices position data indicative of the position of the pixel device for determining the position of the pixel devices;

send the data to the server for use in calculating the display attribute for each of the pixel devices in accordance with the determined position of that pixel device.

35. A computer program for a mobile device for use in determining the positions of a plurality of mobile devices executing that program, each mobile device having a transmitter and receiver for a wireless signal, each mobile device being within wireless sensor range of at least one other of said mobile devices, the computer program comprising program instructions which when run on the mobile device cause it to: