US20030105880A1

US20030105880A1 - Distributed processing, storage, and transmision of multimedia information

Info

Publication number: US20030105880A1
Application number: US10/011,883
Authority: US
Inventors: Radu Jasinschi
Original assignee: Koninklijke Philips Electronics NV
Current assignee: Koninklijke Philips NV
Priority date: 2001-12-04
Filing date: 2001-12-04
Publication date: 2003-06-05
Also published as: WO2003049447A1; US20030103074A1; US7305618B2; JP2005512214A; EP1459545A1; AU2002348863A1; TW200304073A; KR20040071176A; CN1600031A

Abstract

A system and method for communicating and processing data. A home computer unit engages in two-way communication with a mobile unit over a communication path having a wireless portion. The home computer unit receives a signal from the mobile unit over the communication path. The signal comprises a request by the mobile unit for information from the home computer unit. The home computer unit extracts the requested information from a data source. If the extracted requested information does not satisfy a bandwidth constraint relating to the communication path or does not fit within available memory of the mobile unit, then the home computer unit restructures the requested information into downsized information such that the downsized information satisfies the bandwidth constraint and fits within the available memory of the mobile unit. Then, the home computer unit transmits the downsized information to the mobile unit over the communication path.

Description

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates to a system and method for communicating A system and method for communicating and processing data. A home computer unit engages in two-way communication with a mobile unit over a communication path having a wireless portion. The home computer unit receives a signal from the mobile unit over the communication path. The signal comprises a request by the mobile unit for information from the home computer unit. The home computer unit extracts the requested information from a data source. If the extracted requested information does not satisfy a bandwidth constraint relating to the communication path or does not fit within available memory of the mobile unit, then the home computer unit restructures the requested information into downsized information such that the downsized information satisfies the bandwidth constraint and fits within the available memory of the mobile unit. Then, the home computer unit transmits the downsized information to the mobile unit over the communication path. data, and more particularly to a system and method of communicating information between a home computer unit and a mobile unit.

2. Related Art

Data communication between disparate computer systems may have incompatibilities that cause such data communication to be inefficient, inaccurate, or too slow. Thus, there is a need for a system and method for communicating data between disparate computer systems in such a way that such data communication is efficient, accurate, and not too slow.

SUMMARY OF THE INVENTION

The present invention provides a system for communication and processing of data, comprising a home computer unit,

wherein the home computer unit is adapted to engage in two-way communication with a mobile unit over a communication path,

wherein a portion of the communication path is wireless,

wherein the home computer unit is adapted to receive at least one signal sent by the mobile unit to the home computer unit over the communication path,

wherein the at least one signal comprises a request by the mobile unit for information from the home computer unit,

wherein the home computer unit is adapted to extract the requested information from a data source, and

wherein if the extracted requested information does not satisfy a bandwidth constraint relating to the communication path or does not fit within available memory of the mobile unit, then:

the home computer unit is adapted to effectuate a restructuring of the extracted requested information into downsized information such that the downsized information satisfies the bandwidth constraint and fits within the available memory of the mobile unit, and

the home computer unit is adapted to transmit the downsized information to the mobile unit over the communication path.

The present invention provides a method for communicating and processing data, comprising

providing a home computer unit;

engaging, by the home computer unit, in two-way communication with a mobile unit over a communication path, wherein a portion of the communication path is wireless;

receiving, by the home computer unit, at least one signal sent by the mobile unit to the home computer unit over the communication path such that the at least one signal comprises a request by the mobile unit for information from the home computer unit;

extracting, by the home computer unit, the requested information from a data source, wherein if the extracted requested information does not satisfy a bandwidth constraint relating to the communication path or does not fit within available memory of the mobile unit, then:

restructuring, by the home computer unit, the extracted requested information into downsized information such that the downsized information satisfies the bandwidth constraint and fits within the available memory of the mobile unit, and

transmitting, by the home computer unit, the downsized information to the mobile unit over the communication path.

The present invention provides a multimedia dictionary having reference content entries which are mapped into compact content entries,

wherein the multimedia dictionary is coupled to a home computer unit,

wherein a portion of the communication path is wireless,

wherein the home computer unit is adapted to extract the requested information from a data source,

wherein if the extracted requested information does not satisfy a bandwidth constraint relating to the communication path or does not fit within available memory of the mobile unit, then the home computer unit is adapted to effectuate a restructuring of the extracted requested information into downsized information such that the downsized information satisfies the bandwidth constraint and fits within the available memory of the mobile unit, and

wherein the home computer unit is adapted to transmit the downsized information to the mobile unit over the communication path,

wherein the home computer unit is adapted to determine a modality of the downsized information by use of the multimedia dictionary,

wherein the extracted requested information relates to a first reference content entry of the reference content entries of the multimedia dictionary, and

wherein the downsized information relates a first compact content entry of the compact content entries of the multimedia dictionary such that the first reference content entry is mapped into the first compact content entry.

The present invention provides a system and method for communicating data between disparate computer systems in such a way that such data communication is efficient, accurate, and not too slow.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system for communicating between a home computer unit and a mobile unit, in accordance with embodiments of the present invention. [0034]
FIG. 2 depicts the home computer unit of FIG. 1 exchanging signals with the mobile unit of FIG. 1, in accordance with embodiments of the present invention. [0035]
FIG. 3 depicts extracted information being restructured by the home computer unit of FIG. 1 into downsized information, wherein the extracted information had been previously requested by the mobile unit of FIG. 1, in accordance with embodiments of the present invention. [0036]
FIG. 4A depicts a intra-modality restructuring of a visual modality, in accordance with embodiments of the present invention. [0037]
FIG. 4B depicts a intra-modality restructuring of a audio modality, in accordance with embodiments of the present invention. [0038]
FIG. 4C depicts a intra-modality restructuring of a text modality, in accordance with embodiments of the present invention. [0039]
FIG. 5A depicts a cross-modality restructuring of a visual modality to a text modality, in accordance with embodiments of the present invention. [0040]
FIG. 5B depicts a cross-modality restructuring of an audio modality to a text modality, in accordance with embodiments of the present invention. [0041]
FIG. 6 depicts, in a perspective view, the result of an intra-modality restructuring of a visual modality into a layered mosaic structure, in accordance with embodiments of the present invention. [0042]
FIG. 7A illustrates a region having variations in color as expressed in shades of grey. [0043]
FIG. 7B depicts the region of FIG. 7A after being restructured by being smoothed with respect to its variations in shades of grey, in accordance with embodiments of the present invention. [0044]
FIG. 8A illustrates a region having variations in texture. [0045]
FIG. 8B depicts the region of FIG. 8A after being restructured by being smoothed with respect to its variations in texture, in accordance with embodiments of the present invention. [0046]
FIG. 9 illustrates a television icon, in accordance with embodiments of the present invention. [0047]
FIG. 10 illustrates the functionality of a multimedia dictionary, in accordance with embodiments of the present invention. [0048]
FIG. 11 is a flow chart showing a first method of determining which downsized information to generate by restructuring, in accordance with embodiments of the present invention. [0049]
FIG. 12 is a flow chart showing a second method of determining which downsized information to generate by restructuring, in accordance with embodiments of the present invention. [0050]
FIG. 13 is a flow chart showing a third method of determining which downsized information to generate by restructuring, in accordance with embodiments of the present invention.[0051]

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 is a block diagram of a [0052] system 10 for communicating between a home computer unit 12 and a mobile unit 14 over a communication path 20, in accordance with embodiments of the present invention. The home computer unit 12 comprises any computer or computer system that is fixed and not mobile such as, inter alia, a personal computer, a work station, a mainframe computer, etc. The home computer unit 12 may be positioned in any fixed location such as in, inter alia, a home, an office, etc. The home computer unit 12 is capable of engaging in data processing functions such as, inter alia: computing; storing and retrieving data from a peripheral source; transmitting and receiving data over a network, data path (e.g., the communication path 20), etc. The home computer unit 12 has sufficient computation power for performing “hard” computations on large files such, inter alia, graphics files. If located in a residence, the home computer unit 12 may be capable of performing such tasks as control of ambient temperature, playing of radio, playing of television, turning on or off electric circuits, etc. The home computer unit 12 has hardware that is typically found in computers and computer systems such as memory (e.g., dynamic random access memory, read-only memory, hard disk, optical disk, floppy disk, etc.), input devices (e.g., keyboard, mouse, computer screen, etc.), output devices (e.g., printer, plotter, computer screen, magnetic tape, removable hard disk, floppy disk, etc.), and a processor for executing software, performing computations, controlling the input and output devices, etc.
The [0053] mobile unit 14 comprises any mobile device that is capable of being easily moved. The mobile unit 14 may be hand-held and may be small enough to be inserted in a handbag, pocket, etc. Although the mobile unit 14 is capable of performing computation, logic (e.g., making decisions) and storing of data, the mobile unit 14 has limited computing and data storage capabilities and is thus incapable of doing highly intensive calculations with large graphics files. Additionally, the mobile unit 14 may have limited or no access to external databases, but may have limited access to the Internet. Examples of the mobile unit 14 include, inter alia: cellular phone with display screen, PDA/cell phone (note: PDA stands for personal digital assistant), etc.
The [0054] communication path 20 includes a wireless portion 22 and may also include a wired portion 21. The wireless portion 22 of the communication path 20, which is directly coupled to the mobile unit 14, exists in all embodiments of the present invention. The wired portion 21 of the communication path 20 exist in some, but not all, embodiments of the present invention. For embodiments in which the wired portion 21 is not present, the mobile unit 14 may be analogous to a cellular phone receiving a signal from the home computer unit 12 over the communication path 20 that is totally wireless, but which may require intermediate relay stations between the home computer unit 12 and the mobile unit 14. For embodiments in which the wired portion 21 is present, the mobile unit 14 may be analogous to a wireless phone such that the wired portion 21 may be telephone wire originating at the home computer unit 12 and ending at a base device at the user's residence, wherein the base device has a transmitter that transmits a signal to the mobile unit 14 wireless phone over the wireless portion 22 that is located within a small radius from the base. The wired portion may comprise any wiring used for communicating data such as, inter alia, telephone wiring, cable wiring, etc. If the wired portion 21 exists, then the wireless portion 22 may be less than the wired portion 21 (i.e., the wireless portion 22 is shorter in linear length than is the wired portion 21). Alternatively if the wired portion 21 exists, then the wireless portion 22 may be equal to or greater than the wired portion 21.
The [0055] communication path 20 is a two-way communication path, as illustrated in FIG. 2. FIG. 2 depicts the home computer unit 12 of FIG. 1 exchanging signals with the mobile unit 14 of FIG. 1, in accordance with embodiments of the present invention. Thus, the home computer unit 12 may transmit a signal 26 to the mobile unit 14 via the communication path 20 (see FIG. 1), and the mobile unit 14 may transmit a signal 28 to the home computer unit 12 via the communication path 20.
Without the [0056] home computer unit 12, the mobile unit 14 would have little or no more functionality than an intelligent phone/organizer. Accordingly, the mobile unit 14 is dependent on the home computer unit 12 for receiving information, via the communication path 20, that the home computer unit 12 has access to. As shown in FIG. 1, the home computer unit 12 is coupled to, and thus has access to, inter alia, a database 16, the Internet 18, a source of video data 19, and a multimedia dictionary 24 which will be discussed infra in conjunction with FIG. 10. At any instant of time, the mobile unit 14 may send a request for information to the home computer unit 12, said request for information being sent over the communication path 20. Due to the limited storage capacity (e.g., memory) of the mobile unit 14 and bandwidth constraints relating to the communication path 20, the home computer unit 12 may not be able to satisfy the request as formulated by the mobile unit 14. If the home computer unit 12 does not have knowledge of the available storage capacity of the mobile unit 14 and the bandwidth constraints relating to the communication path 20, then the two-way communication between the home computer unit 12 and the mobile unit 14 may be beneficially exploited. For example, the home computer unit 12 may send a query signal (see, e.g., the signal 26 of FIG. 2) to the mobile unit 14 over the communication path 20, wherein the query signal requests information as to the available storage capacity of the mobile unit 14 and/or bandwidth constraints relating to the communication path 20. The mobile unit 14 may answer the query by transmitting an answer to the home computer unit 12 via the signal 28 (see FIG. 2). Said answer so transmitted by the mobile unit 14 includes a specification of the available memory of the unit 14 and/or bandwidth constraints relating to the communication path 20. Such communication of metadata between the home computer unit 12 and the mobile unit 14 is ancillary to the actual information that had been requested by the mobile unit 14.
The bandwidth constraints relating to the [0057] communication path 20 may take into account the bandwidth (B) of the communication path 20 as expressed in terms of the maximum number of bits per second that can be transported in the communication path 20. If the information requested by the mobile unit 14 encompasses a total size of S bits, then the transport time via the communication path 20 of said information from the home computer unit 12 to the mobile unit 14 is S/B seconds. The bandwidth constraints relating to the communication path 20 may be expressed as any relevant function of B and S/B. For example, the bandwidth constraint may be S/B<60 seconds. As another example, the bandwidth constraint may be a real-time bandwidth constraint such as when the requested information is a real-time feed such as a real-time video feed or a real-time telecast (e.g., a sports event). With a real-time feed of F bits per second of data, the real-time bandwidth constraint may be B≧F.
With the present invention, the [0058] home computer unit 12 receives a request for information from the mobile unit 14, and the home computer unit 12 is informed as to the available storage capacity of the mobile unit 14 and bandwidth constraints relating to the communication path 20 as discussed supra. The home computer unit 12 may extract the requested information from a data source. Examples of applicable data sources comprise the database 16, the Internet 17, and the source of video data 18 (e.g., frames of video data or a feed of frames of video data). If the requested information, as extracted by the home computer unit 12, does not satisfy a bandwidth constraint relating to the communication path 20 or does not fit within available memory of the mobile unit 14, then in accordance with the present invention, the home computer unit 12 restructures the extracted requested information into downsized information such that the downsized information satisfies the bandwidth constraint and fits within the available memory of the mobile unit 14. After said “content restructuring”(called “restructuring” for short), the home computer unit 12 transmits the downsized information to the mobile unit 14 over the communication path 20.
FIG. 3 shows that the downsized [0059] information 32, as restructured 31 from the extracted requested information 30, is “smaller” in bit-size than the extracted requested information 30 (i.e., encompasses fewer bits than the extracted requested information 30) in order to satisfy the bandwidth constraint(s) and fit within the available memory of the mobile unit 14. Logistically, the extracted requested information 30 does not have to be literally “extracted” in its entirety from the data source but may be partially extracted or not extracted at all if the downsized information 32 could be generated without operating directly on the requested data in the data source, such as by use of the multimedia dictionary 24 of FIG. 1 as will be discussed infra in conjunction with FIG. 10. For example, if the requested information 30 comprises a visual image of the White House in Washington and if the multimedia dictionary 24 receives input indicating that the requested information 30 comprises the White House in Washington (e.g., from the words “White House”, from an icon that identifies the White House in Washington, etc.), and if the multimedia dictionary 24 includes information that identifies the White House in Washington and also includes associated text that describes the White House in Washington, then the downsized information 32 could be generated to include such text without the necessity of actually extracting a visual image of the White House in Washington.
If the bandwidth constraint is a real-time bandwidth constraint, then some or all of the restructuring algorithms may be embedded in hardware (e.g., a Fast Fourier Transform algorithm in a semiconductor chip) instead of being implemented in software, in order to minimize the processing time for the restructuring. [0060]
The restructuring of the requested [0061] information 30 into the downsized information 32 depends on the modality of the requested information 30. The “modality” denotes a medium of expression such as a visual modality, an audio modality, and a text modality. The visual modality includes, inter alia, a video modality (e.g., a feed or stream of video frames) and a graphics modality (i.e., synthetic imagery such as any synthetic graphics representation of information such as a graph, a pictorial bit pattern, etc.). The audio modality denotes an expression of information in terms of sound such as, inter alia, speech, music, etc. The text modality denotes an expression of information in terms of written language (e.g., words, sentences, etc.).
The restructuring of the requested [0062] information 30 into the downsized information 32 is such that the requested information 30 is expressed in modality A and the downsized information 32 is expressed in modality B. A restructuring such that modality A is equal to modality B is called an intra-modality restructuring. FIG. 4A depicts an intra-modality restructuring from the visual modality to the visual modality, FIG. 4B depicts an intra-modality restructuring from the audio modality to the audio modality, and FIG. 4C depicts an intra-modality restructuring from the text modality to the text modality. A restructuring such that modality A differs from modality B is called a cross-modality restructuring. FIG. 5A depicts an cross-modality restructuring from the visual modality to the text modality, and FIG. 5B depicts an cross-modality restructuring from the audio modality to the text modality.
As an example of a request for information and consequent restructuring, a real estate agent who has access to the [0063] mobile unit 14 wants to show a client buyer a video of a house that the buyer is interested in. Due to bandwidth constraints relating to the communication path 20 or lack of sufficient available memory at the mobile unit 14, the real estate agent cannot obtain a visual representation of the house from the home computer unit 12. Instead, the home computer unit 12 sends a restructured downsized version of the requested video (e.g., an equivalent textual representation of visual information relating to the house) to the real estate agent at the mobile unit 14. As another example of a request for information and consequent restructuring, an emergency-room doctor at the mobile unit 14 must operate on a patient in 2 minutes, but the patient data is in the database of another hospital. Due to bandwidth constraints relating to the communication path 20 or lack of sufficient available memory at the mobile unit 14, the doctor may not get the X-ray from the home computer unit 12 at the another hospital but instead will get restructured downsized information in the form of a text description of the X-ray. Note that a given mobile unit 14 may be used for different purposes at different times such as being used for real estate purposes during daytime hours and for personal purposes (e.g., shopping) during evening hours.
The intra-modality restructuring of a visual modality into the downsized information shown in FIG. 4A may include various techniques including, inter alia, generation of layered mosaics, cartoonification, and iconification. [0064]
Layered mosaics are planar images extended in a direction that is normal to the planar images. The layered mosaics encode limited information about a pictorial representation (e.g., a sequence of video frames). The mosaics are accumulated in layers according to the associated relative depth of regions within the pictorial representation. The layer depths simulate objects at different depth levels. Generally an object appearing in multiple successive video frames may be represented as a mosaic layer provided that the object does not change in time or changes insignificantly in time over the video frame subset represented by the mosaic layer. This enables many frames (e.g, 300 frames) to be represented as a single mosaic layer, which enables the restructuring to highly reduce the amount of data to be transported over the [0065] communication path 20 as compared with the original sequence of frames.
For illustrative purposes, FIG. 6 depicts, in a perspective view, the result of an intra-modality restructuring of a visual modality (e.g., of a visual image or a portion thereof) into a layered [0066] mosaic structure 40, in accordance with embodiments of the present invention. The mosaic structure 40 comprises mosaic layers 41-45 as shown. Each such mosaic layer in FIG. 6 comprises a discrete object that does not change in time or changes insignificantly in time over the video frame subset represented by the mosaic layer. For example, a front face 46 of the mosaic layer 41 comprises a tree 48 as shown. Although hidden in the perspective view of FIG. 6, each of the other mosaic layers 42-45 comprise one or more discrete object (e.g., a house, a flower bed, etc.) that does not change in time or changes insignificantly in time over the video frame subset represented by the mosaic layer. The representation of numerous video frames in a single mosaic layer facilitates a significant reduction of data in the intra-modality restructuring of the video modality into the downsized information represented by the layered mosaic structure 40.
The mosaics layers are generated incrementally through recursive algorithms. At each step of these recursive algorithms, comparing the mosaic layer generated at the previous instance with the current image of the video sequence generates a new instance of this mosaic. [0067]
Cartoonification denotes a process in which complex features of a natural image, or a portion of the natural image, are simplified, such as by smoothing of motion (e.g., motion as depicted in successive frames of a dynamic video feed) or smoothing of spatial regions. Smoothing of motion restructures the motion to be represented more simply such as being less diverse and more uniform. Smoothing of spatial regions comprises such smoothing techniques as, inter alia, smoothing of color, smoothing of brightness, and smoothing of texture. [0068]
FIG. 7A depicts a [0069] region 50 of a visual image, wherein the region 50 has variations in color as expressed in terms of variation in shades of grey, said region 50 comprising region segments 51-53, each said region segment having a uniform color (i.e., a uniform shade of grey). FIG. 7B depicts the region 50 of FIG. 7A after being restructured into region 55 by being smoothed with respect to its variations in color (i.e., variation in shade of grey) so as to acquire a uniform shade of grey throughout the region 55, in accordance with embodiments of the present invention. The smoothing of color or shades of grey facilitates a reduction in the number of bits needed to represent the region 50, so that fewer bits are needed to represent the downsized information of the smoothed region 55. Similarly, smoothing over variations of brightness in a specified region of a visual image, or of a portion of the natural image, facilitates a reduction in the number of bits needed to represent the downsized region that has been restructured from the specified region.
FIG. 8A depicts a [0070] region 60 of a natural image, wherein the region 60 has variations in texture. The texture in the region 60 is expressed by the box pattern defined by horizontal lines 61-67 and vertical lines 71-77. FIG. 8B depicts the region 60 of FIG. 8A after being restructured into region 62 by being smoothed with respect to its variations in texture, in accordance with embodiments of the present invention. The smoothing comprises removing horizontal lines 62, 64, 66 and vertical lines 72, 74, 76 so that the resulting box pattern of region 62 is coarser than the box pattern of region 60. Accordingly, the smoothing of texture facilitates a reduction in the number of bits needed to represent the region 60, so that fewer bits are needed to represent the downsized information of the smoothed region 62.
Iconification denotes a process in which a symbol or a symbolic image (e.g., a graphic symbol), called an “icon,” represents an object, entity, item, etc. The icon can be any symbolic representation that the user would recognize as representing the object, entity, item, etc (e.g., a stick figure icon representing a person). As an illustrated example, FIG. 9 depicts a television icon [0071] 70, in accordance with embodiments of the present invention. The television icon 70 represents a natural image of a television. The television icon 70 requires fewer bits to be described than does the natural image of the television that the television icon 70 represents. Accordingly, the iconification facilitates a reduction in the number of bits needed to represent the object, entity, item, etc (e.g., the television), so that fewer bits are needed to represent the downsized information (i.e., the icon such as, for example, the television icon 70).
The intra-modality restructuring of an audio modality into the downsized information shown in FIG. 4B may eliminate redundant audio data and may include various techniques including, inter alia: a representation of natural sound by synthetic sound, or any other technique known to a person of ordinary skill in the art. [0072]
The intra-modality restructuring of a text modality into the downsized information shown in FIG. 4C may include various techniques including: inter alia, summarization, natural language processing to take a first text and generate a second text that is an interpretation of the first text, etc. Any method of compactly representing text, as is known to a person of ordinary skill in the art, is encompassed by the present invention. [0073]
The cross-modality restructuring of a visual modality into the downsized information of a text modality, as shown in FIG. 5A, represents pictorial information as text. For example, a visual image of a lady looking at her baby may be restructured to the text: “Lady looking at her baby.” Such restructuring from visual modality to text modality may be implemented in software having a text-based database available to the software. Such software may recognize pictorial information and replace such pictorial information by text that has been stored in the database. For example, a movie of Manhattan from New Jersey may be generated, in part, by panning and the software may recognize some buildings in the movie (e.g., the Chrysler building, the Empire State building, etc.). Having recognized the buildings, the software may replace the pictorial representations of the buildings by such words as “Chrysler building”, “Empire State building”, etc. The text description for cross-modality restructuring of a visual modality to a text could be as limited as desired so as long as the text description is meaningful. Additionally, portions of the pictorial image (as opposed to the whole pictorial image) can be isolated and described by text. [0074]
The cross-modality restructuring of an audio modality into the downsized information of a text modality, as shown in FIG. 5B, represents audio information as text. For example, closed caption text, also called “speech-to-text,” is commonly known in the art (e.g., in speech recognition software”). Said cross-modality restructuring of an audio modality into the text modality significantly reduces bits, since audio files generally require many more data bits to be represented than do analogous text files. [0075]
A hybrid restructuring involving both intra-modality restructuring and cross-modality restructuring is within the scope of the present invention. For example, a layered mosaic can be annotated with a text description, a cartoon can be also be annotated with a text description, etc. Such annotated textual descriptions, if short, would not add a significant number of bits to the layered mosaic, cartoon, etc. [0076]
FIG. 10 illustrates the functionality of a [0077] multimedia dictionary 80, which is analogous to the multimedia dictionary 24 of FIG. 1, in accordance with embodiments of the present invention. The multimedia dictionary 80 can be used to translate content information from one modality to another or within a given modality. The multimedia dictionary 80 has synonyms, acronyms, etc. The multimedia dictionary 80 describes how to restructure from one modality to another. Thus in the multimedia dictionary 80 of FIG. 10, content information in modality A is mapped (i.e., restructured) into modality B within or across modalities. The multimedia dictionary 80 may store input images and describes restructured output associated with the stored input images. Each content object in the multimedia dictionary 80 is given a description that is textual or is based on metadata information. The metadata information can be generated in a proprietary form, or it can be based on common agreed-upon standards (e.g., MPEG-7, TV-Anytime, and/or SIMPTE). The restructuring between elements of different modalities should be done using the descriptions in the multimedia dictionary 80. The restructurings could get into the multimedia dictionary 80 either by recording restructurings as they occur in practice or by pre-planning certain standard restructurings and placing those standard restructurings in the multimedia dictionary 80. Thus, the multimedia dictionary 80 could be used to standardize certain restructurings so that a subsequent table lookup would effectuate a designated restructuring instead of executing a complex algorithm.
In order to effectuate restructuring, the present invention determines which downsized information to generate by the restructuring by any reasonable method such as by, inter alia, the following first method, second method, or third method. [0078]
The first method of determining which downsized information to generate by the restructuring is shown in the flow chart of FIG. 11. In FIG. 11, the [0079] home computer unit 12 ascertains N permissible downsized information possibilities M₁, M₂, . . . , and M_N, wherein N≧1. If N=1 then the home computer unit 12 determines that M₁is the downsized information to generate by the restructuring. If N>1, however, then the home computer unit 12 sends a query signal to the mobile unit 14 over the communication path 20. The query signal comprises a request of a selection by the mobile unit 14 of a preferred downsized information M_k(k=1, 2, . . . , or N) from M₁, M₂, . . . , and M_Nsuch that M_kwill be generated by the restructuring. The home computer unit 12 receives a query return signal from the mobile unit 14 over the communication path 20. The query return signal specifies M_k.
The second method of determining which downsized information to generate by the restructuring is shown in the flow chart of FIG. 12. In FIG. 12, the [0080] home computer unit 12 ascertains N permissible downsized information possibilities M₁, M₂, . . . , and M_Nrespectively having a size in bits of B₁, B₂, . . . , and B_Nwherein N≧1. If N=1 then the home computer unit 12 determines that M₁is the downsized information to generate by the restructuring. If N>1, however, then the home computer unit 12 selects M_kas the downsized information to generate by the restructuring, wherein k is one of 1, 2, . . . , and N such that B_kis a minimum of B₁, B₂, . . . , and B_N.
The third method of determining which downsized information to generate by the restructuring is shown in the flow chart of FIG. 13. In FIG. 13, the [0081] home computer unit 12 ascertains N permissible downsized information possibilities M₁, M₂, . . . , and M_Nrespectively having a size in bits of B₁, B₂, . . . , and B_N, wherein N≧1. If N=1 then the home computer unit 12 determines that M₁is the downsized information to generate by the restructuring. If N>1, however, then the home computer unit 12 selects M_kas the downsized information to generate by the restructuring, wherein k is one of 1, 2, . . . , and N such that B_kis a maximum of B₁, B₂, . . . , and B_N.
While embodiments of the present invention have been described herein for purposes of illustration, many modifications and changes will become apparent to those skilled in the art. Accordingly, the appended claims are intended to encompass all such modifications and changes as fall within the true spirit and scope of this invention. [0082]

Claims

What is claimed is:

1. A system for communication and processing of data, comprising a home computer unit and a mobile unit,

wherein the home computer unit is adapted to engage in two-way communication with the mobile unit over a communication path,

wherein a portion of the communication path is wireless,

2. The system of claim 1, wherein the portion of the communication path that is wireless includes the entire communication path.

3. The system of claim 1, wherein the portion of the communication path that is wireless is shorter than a portion of the communication path that is not wireless.

4. The system of claim 1, wherein the bandwidth constraint comprises a real-time bandwidth constraint.

5. The system of claim 1, wherein the at least one signal comprises a specification of the bandwidth constraint.

6. The system of claim 1, wherein the at least one signal comprises a specification of the available memory of the mobile unit.

7. The system of claim 1, wherein the data source comprises at least one database external to the home computer unit.

8. The system of claim 1, wherein the data source comprises an Internet.

9. The system of claim 1, wherein the data source comprises a source of video data.

10. The system of claim 1, wherein the restructuring comprises an intra-modality restructuring of a modality of the requested information.

11. The system of claim 10, wherein the modality comprises a visual modality that includes a video modality.

12. The system of claim 10, wherein the modality comprises a visual modality that includes a graphics modality.

13. The system of claim 10 wherein the modality includes a visual modality, wherein the visual modality includes an image, and wherein the intra-modality restructuring includes generation of layered mosaics from a portion of the image.

14. The system of claim 10, wherein the modality comprises a visual modality, wherein the visual modality includes an image, and wherein the intra-modality restructuring includes cartoonification of a portion of the image.

15. The system of claim 14, wherein the cartoonification comprises a smoothing of an attribute of the portion of the image, and wherein the attribute is at least one of color, brightness, and texture.

16. The system of claim 10, wherein the modality comprises a visual modality, wherein the visual modality includes an image, and wherein the intra-modality restructuring includes iconification of a portion of the image.

17. The system of claim 10, wherein the modality includes an audio modality.

18. The system of claim 10, wherein the modality includes a text modality

19. The system of claim 1, wherein the restructuring comprises a cross-modality restructuring of a first modality into a second modality.

20. The system of claim 19, wherein the first modality comprises a visual modality, and wherein the second modality comprises a text modality.

21. The system of claim 20, wherein the visual modality includes a video modality.

22. The system of claim 20, wherein the visual modality includes a graphics modality.

23. The system of claim 1, wherein the home computer unit is adapted to determine a modality of the downsized information by use of a multimedia dictionary having a reference content entry that is mapped into a compact content entry, wherein the extracted requested information relates to the reference content entry, and wherein the downsized information relates to the compact content entry.

24. A method for communicating and processing data, comprising

providing a home computer unit;

25. The system of claim 24, wherein the portion of the communication path that is wireless includes the entire communication path.

26. The system of claim 24, wherein the portion of the communication path that is wireless is shorter than a portion of the communication path that is not wireless.

27. The method of claim 24, wherein the bandwidth constraint comprises a real-time bandwidth constraint.

28. The method of claim 24, wherein the at least one signal comprises a specification of the bandwidth constraint.

29. The method of claim 24, wherein the at least one signal comprises a specification of the available memory of the mobile unit.

30. The method of claim 24, wherein the data source comprises at least one database external to the home computer unit.

31. The method of claim 24, wherein the data source comprises an Internet.

32. The method of claim 24, wherein the data source comprises a source of video data.

33. The method of claim 24, wherein the restructuring comprises an intra-modality restructuring of a modality of the requested information.

34. The method of claim 33, wherein the modality comprises a visual modality that includes a video modality.

35. The method of claim 33, wherein the modality comprises a visual modality that includes a graphics modality.

36. The method of claim 33, wherein the modality includes a visual modality, wherein the visual modality includes an image, and wherein the intra-modality restructuring includes generation of layered mosaics from a portion of the image.

37. The method of claim 33, wherein the modality comprises a visual modality, wherein the visual modality includes an image, and wherein the intra-modality restructuring includes cartoonification of a portion of the image.

38. The method of claim 37, wherein the cartoonification comprises a smoothing of an attribute of the portion of the image, and wherein the attribute is at least one of color, brightness, and texture.

39. The method of claim 33, wherein the modality comprises a visual modality, wherein the visual modality includes an image, and wherein the intra-modality restructuring includes iconification of a portion of the image.

40. The method of claim 33, wherein the modality includes an audio modality.

41. The method of claim 33, wherein the modality includes a text modality

42. The method of claim 24, wherein the restructuring comprises a cross-modality restructuring of a first modality into a second modality.

43. The method of claim 42, wherein the first modality comprises a visual modality, and wherein the second modality comprises a text modality.

44. The method of claim 43, wherein the visual modality includes a video modality.

45. The method of claim 43, wherein the visual modality includes a graphics modality.

46. The method of claim 24, wherein the restructuring comprises utilizing a multimedia dictionary having a reference content entry that is mapped into a compact content entry, wherein the extracted requested information relates to the reference content entry, and wherein the downsized information relates to the compact content entry.

47. The method of claim 24, wherein the restructuring comprises determining which downsized information to generate by the restructuring, said determining comprising:

ascertaining, by the home computer unit, N permissible downsized information possibilities, wherein N is at least 1;

if N=1 then determining that the ascertained permissible downsized information possibility is the downsized information to generate by the restructuring;

if N>1 then

sending a query signal by the home computer unit to the mobile unit over the communication path, wherein the query signal comprises a request of a selection by the mobile unit of a preferred downsized information from the N permissible downsized information possibilities such that the preferred downsized information will be generated by the restructuring, and

receiving by the home computer unit a query return signal from the mobile unit over the communication path, wherein the query return signal specifies the preferred downsized information.

48. The system of claim 24, wherein the restructuring comprises determining which downsized information to generate by the restructuring, said determining comprising:

ascertaining, by the home computer unit, N permissible downsized information possibilities M₁, M₂, . . . , and M_Nrespectively having a size in bits of B₁, B₂, . . . , and B_N, wherein N≧1;

if N=1 then determining that M₁is the downsized information to generate by the restructuring;

if N>1 then selecting M_kas the downsized information to generate by the restructuring, wherein k is one of 1, 2, . . . , and N such that B_kis a minimum of B₁, B₂, . . . , and B_N.

49. The system of claim 24, wherein the restructuring comprises determining which downsized information to generate by the restructuring, said determining comprising:

if N>1 then selecting M_kas the downsized information to generate by the restructuring, wherein k is one of 1, 2 , . . . , and N such that B_kis a maximum of B₁, B₂, . . . , and B_N.

50. A multimedia dictionary having reference content entries which are mapped into compact content entries,

wherein the multimedia dictionary is coupled to a home computer unit,

wherein a portion of the communication path is wireless,

51. A system for communication and processing of data, comprising a home computer unit,

wherein a portion of the communication path is wireless,