US20040008214A1 - Tagging repeating images for improved compression - Google Patents

Tagging repeating images for improved compression Download PDF

Info

Publication number
US20040008214A1
US20040008214A1 US10/385,759 US38575903A US2004008214A1 US 20040008214 A1 US20040008214 A1 US 20040008214A1 US 38575903 A US38575903 A US 38575903A US 2004008214 A1 US2004008214 A1 US 2004008214A1
Authority
US
United States
Prior art keywords
tile
image
region
command
drawn
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/385,759
Inventor
Thomas O'Neill
Jordan Slott
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Microsystems Inc
Original Assignee
Sun Microsystems Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US10/194,123 external-priority patent/US20040008213A1/en
Priority claimed from US10/247,907 external-priority patent/US20040008205A1/en
Application filed by Sun Microsystems Inc filed Critical Sun Microsystems Inc
Priority to US10/385,759 priority Critical patent/US20040008214A1/en
Assigned to SUN MICROSYSTEMS, INC. reassignment SUN MICROSYSTEMS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SLOTT, JORDAN M., O'NEILL, THOMAS G.
Priority to US10/754,681 priority patent/US7012612B1/en
Publication of US20040008214A1 publication Critical patent/US20040008214A1/en
Priority to US11/290,926 priority patent/US20060077207A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/08Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/12Replacement control
    • G06F12/121Replacement control using replacement algorithms
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • G06T9/001Model-based coding, e.g. wire frame
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02Addressing or allocation; Relocation
    • G06F12/08Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0875Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches with dedicated cache, e.g. instruction or stack
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00Aspects of the architecture of display systems
    • G09G2360/12Frame memory handling
    • G09G2360/121Frame memory handling using a cache memory
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14Display of multiple viewports

Definitions

  • the present invention relates to the field of computer graphics. More particularly, the present invention relates to the tagging of repeating images in order to improve compression.
  • the X Windows System is a standard that utilizes X client software and X server software to enable the updating of displays at the requests of X client applications.
  • the X server software runs on the host computer.
  • An X client application which may be running on a different computer, communicates with the X server software by utilizing a low-level library of routines known as Xlib.
  • Xlib provides the functions required to connect to display servers, create windows, render graphics, respond to events, etc.
  • the X server software then may interface with display drivers to actually render the graphics on the display.
  • X is frequently used as a “local display application”, where the X-server and display are both on the same computer. That is, the same computer is both the “host” and “display” computer. Examples of this usage include running an X-server on a workstation or on an X-terminal. An X-terminal typically has a computer processor, graphics subsystem and display, but no hard drive. Applications running on other computers use the Xlib routines to communicate with the X-server software running on the X-terminal.
  • client applications make requests of a centralized server or servers (here known collectively as the “host computer”).
  • the host computer then manages one or more “display computers”, which are typically simple terminal devices.
  • the Sun RayTM appliance from Sun Microsystems, Inc. of Palo Alto, Calif. is an example of a thin client which serves as a “display computer” in a remote computing application.
  • a Sun RayTM appliance has a processor, graphics subsystem and display, but no hard drive.
  • a Sun RayTM appliance is a “display” computer and runs its own proprietary software.
  • the Sun RayTM server is the “host” computer and runs the X-server software.
  • the full screen image is maintained both in RAM on the host computer as well as in the frame buffer of the Sun RayTM appliance's video card.
  • the host computer sends screen update information to the Sun RayTM appliance via a network protocol known as NewT.
  • the Sun RayTM appliance uses the protocol commands to update the state of its hardware frame buffer.
  • the root window in many window managers is a tile of a repeating M ⁇ N image, with M indicating horizontal pixels and N indicating vertical. Stipples, which are essentially two-color tiles, are another source of repeating images.
  • Repeating image content in a graphics image may be detected by identifying certain commands, known generally as “tile commands”. If a tile command is detected, the fact that a portion of an image was created with a tile command may be stored along with the portion of the image. This allows for well-informed decision making when transmission of the image is to be performed. For example, this allows for the transmission of a single tile, and subsequent transmission of local copy commands to repeat the single tile. This can be very useful in speeding transmission of background images or other repeating images.
  • FIG. 1 is a block diagram illustrating a remote display application such as a Sun RayTM network.
  • FIG. 2 is a block diagram illustrating a remote display application in accordance with an embodiment of the present invention.
  • FIG. 3 is a diagram illustrating how when one of the tile commands is used on a realized window, the new repeating image may be sent as a display update to the display computer, in accordance with an embodiment of the present invention.
  • FIG. 4 is a diagram illustrate how local copies can be used to paint part or all of the upper-left width*height pixels on the display computer in accordance with another embodiment of the present invention.
  • FIGS. 5 A- 5 C are flow diagrams illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.
  • FIG. 6 is a flow diagram illustrating the sending of the screen update resulting from copying a pixmap to a realized window in accordance with an embodiment of the present invention.
  • FIG. 7 is a flow diagram illustrating a method for subtracting the drawn-to region from the fill and tile linked lists in accordance with an embodiment of the present invention.
  • FIG. 8 is a flow diagram illustrating a method for submitting a region to a fill linked list in accordance with an embodiment of the present invention.
  • FIG. 9 is a flow diagram illustrating a method for submitting a region to a tile linked list in accordance with an embodiment of the present invention.
  • FIG. 10 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “disjoint” embodiment.
  • FIG. 11 is a flow diagram illustrating performing tag copy from a source to a destination in accordance with an embodiment of the present invention.
  • FIG. 12 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “underlay” embodiment.
  • FIG. 13 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.
  • FIG. 14 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention.
  • FIG. 15 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.
  • FIG. 16 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention.
  • the components, process steps, and/or data structures may be implemented using various types of operating systems, computing platforms, computer programs, and/or general purpose machines.
  • devices of a less general purpose nature such as hardwired devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or the like, may also be used without departing from the scope and spirit of the inventive concepts disclosed herein.
  • One example use for the present invention would be to improve compression of images created in pixel-based (as opposed to vector-based) drawing programs.
  • drawing commands circle, filled circle, square, image importation, etc.
  • the Device Dependent X (DDX) layer of the X server may be modified to include possible compression of the screen data and transmission of remote display protocol commands to the display computer.
  • the DDX provides a software interface to a conceptual hardware device.
  • the desired results can be accomplished by modifying the DDX, without changing the DIX.
  • FIG. 1 is a block diagram illustrating a remote display application such as a Sun RayTM network.
  • Client applications 100 a - 100 c send information to and receive information from the host computer 102 over the X Windows Protocol. Some of the information comprises repeating images.
  • the host computer utilizes a device independent layer (DIX) 104 to act as a controlling layer, which handles initialization procedures and all device-independent handling of requests and events.
  • a device dependent layer (DDX) 106 then is used to handle device dependent functionality.
  • the DDX 106 then communicates with display computer 108 using the NewT protocol.
  • the DIX and the DDX essentially make up an X-server.
  • the present invention provides a mechanism to identify repeating images for special treatment. As will be seen, this results in the ability to dramatically reduce CPU and network bandwidth utilization without affecting image quality.
  • the present application will focus on this identification and handling of repeating images, however the identification and handling of multicolor and singlecolor non-repeating images as well as other, miscellaneous images will also be discussed.
  • the solution for identifying and handling repeating images may be used in conjunction with the solutions for identifying and handling single-color, multi-color images and miscellaneous images, but does not have to be. The identification and handling of repeating images alone will still improve CPU and bandwidth utilization and overall efficiency.
  • FIG. 2 is a block diagram illustrating a remote display application in accordance with an embodiment of the present invention.
  • Client applications 200 a - 200 c send information to and receive information from the host computer 202 over the X Windows Protocol. Some of the information is repeating image information.
  • the host computer utilizes a device independent layer (DIX) 204 to act as a controlling layer, which handles initialization procedures and all device-independent handling of requests and events.
  • a device dependent layer (DDX) 206 then is used to handle device dependent functionality.
  • An image compression layer 208 may then be provided which performs the additional image compression techniques discussed in the present document. Alternatively, this image compression layer may simply be part of a modified DDX.
  • the image compression layer 208 then communicates with display computer 210 using an enhanced version of the NewT protocol.
  • a “tile command” may be defined as a drawing command that creates a horizontally and/or vertically repeating image. In the X-server, this is typically accomplished via one of several DDX commands:
  • fillStyle is FillTiled or FillOpaqueStippled, or
  • fillStyle is FillStippled, and the drawing occurs on top of a single-colored region.
  • Commands 1-3 above can be used to create a repeating image directly in a realized (on-screen) window.
  • Commands 4-6 above can be used to draw a repeating image indirectly, by first creating it in the source drawable.
  • the tile regions that are created using one of these tile commands may be tagged.
  • the tagging may be accomplished by attaching a data structure to each pixmap via the pixmap devPrivates facility.
  • the tagging of windows may be accomplished by having a single data structure for the entire screen image.
  • the data structure may include a list of tile entries.
  • a linked list could be used.
  • One entry may exist for each tile region that is not compatible with any other tile region.
  • two regions are compatible if they are drawn with the same tile image and the tile edges occur at the same multiples of x and y.
  • Each linked list entry may contain the following information:
  • a hash value for the tile image (e.g., MD5), to allow quick comparison of whether two tile images are identical
  • X region struct identifying the portion of the pixmap or screen image that contains this tile image.
  • tile regions are compatible if they have the same values of width, height, hash value, x modulo width, and y modulo height.
  • the modulo computations take into account the fact that the image repeats every width pixels horizontally and height pixels vertically. Thus, in one embodiment, it is convenient to store x modulo width and y modulo height instead of just x and y.
  • the tile image may be used in place of the hash value.
  • a pointer to the tile image pixmap could be used instead of the hash.
  • the pixmap's reference count would have to be incremented to ensure that the pixmap does not get deleted while the pointer is still set.
  • the linked list may be associated with the pixmap through the use of the DDX pixmap devPrivates facility, which allows an X extension to attach a private data structure to each X pixmap.
  • a region resource may be used to describe an arbitrarily shaped piece of real estate in XY pixel space. I.e., it can be used to describe an arbitrary collection of pixels by characterizing their XY positions.
  • the positions are characterized by an array of rectangles for which every pixel is in the region. This representation is efficient for regularly-shaped regions, which occur frequently in X, but inefficient for regions with a lot of isolated pixels, which occur much less frequently in X.
  • the rectangle representation is also very convenient for remote display applications such as Sun RayTM, where each protocol command specifies the screen update for a single screen rectangle.
  • X also provides a series of routines for performing set operations (union, subtraction, intersection, etc.) on the region resources, which are very convenient in the practice of the invention.
  • the region resources and routines were originally developed to assist with the computation of clipping and drawn-to regions.
  • the composite clip list for a drawable represents the region that can currently be drawn to. It includes the effects of clipping requested by the client application (the client clip list) and, if the drawable is a window, by other windows that are on top of the window in question.
  • the drawn-to region for a particular DDX command is found by intersecting the composite clip list with the region that the command would render in the absence of clipping. It is this region to which the command actually renders.
  • stipples are often used to tile a region with a two-color tile.
  • the stipple is specified as a bit mask, where a 0 bit represents a pixel with the current background color and a 1 bit represents a pixel with the current foreground color.
  • a 0 bit represents a pixel with the current background color
  • a 1 bit represents a pixel with the current foreground color.
  • the tile entry should contain enough information to distinguish between such incompatible tiles.
  • the hash value of a stipple's tile image may be computed using the resulting full-color image. If the drawn-to region does not include a full copy of the tile image, this would require an additional computational step where the stipple is converted to a full-color image.
  • the hash value may instead be computed directly from the stipple bit mask. In this case, three additional pieces of data may be required in the tile list entry:
  • colorBg background color
  • the host computer computes the repeating image portion of the drawn-to region.
  • the repeating-image portion of the drawn-to region is just the drawn to region, but several commands require special handling: PolyFillRect in the FillStippled mode, CopyArea, CopyWindow, SaveAreas, and RestoreAreas.
  • foreground pixels may be rendered and background pixels left unchanged.
  • a repeating image may be generated if the stipple is drawn on top of a single-color fill region.
  • the repeating-image portion of the drawn-to region may be computed as follows (compare FIG. 12):
  • a repeating-image portion of the source drawable is the region in one of the source drawable's tile linked list entries.
  • This process may then be repeated for each tile linked list entry in the source drawable.
  • the host may check to see if the new repeating image is compatible with any of the existing tile entries in the tile linked list. If a compatible entry is found, then the repeating-image portion of the drawn-to region for the DDX command may be added to the entry's tile region. If no compatible entry is found, a new entry may be created with the parameters of the new repeating image and the region may be set to the repeating-image portion of the drawn-to region. For more details, see FIG. 9 and its corresponding description.
  • an embodiment of the present invention may be used in conjunction with an “RGB region” which identifies the portions of the drawable composed of RGB sub-images.
  • an RGB image may be defined as one that is rendered from a list of pixel values (in contrast to an image rendered from a specification of a geometric figure or text characters to draw).
  • such an image may be an imported JPEG image where the pixel values are computed from the compressed JPEG data.
  • this may be an image drawn with the XPutImage command, where the pixel values are provided directly by the client application.
  • XPutImage copies the image data in an XImage structure into an area of a window or pixmap. This is the typical way to create non-repeating, many-colored images on an X server, as the other drawing primitives are typically used to draw only one or two colors at a time.
  • XPutImage is a specific command in the X Windows System
  • put image command may be used in the present application to refer to any rendering command used to create an RGB image.
  • an embodiment of the present invention may also be used in conjunction with a “fill region”, which identifies the portions of the-drawable composed of single-color regions.
  • the tile images are treated as underlays of the subsequent drawing. This may be termed the “underlay” embodiment.
  • drawing commands which affect large, rectangular portions of the drawable have their drawn-to regions subtracted from the tile regions. Examples of such commands would be PolyFillRect and PutImage.
  • the drawn-to regions of other commands are instead added to the miscellaneous region.
  • the new repeating image may be sent as a display update to the display computer.
  • FIG. 3 is a diagram illustrating this process.
  • each rectangle in the repeating-image portion is sent separately. For each rectangle:
  • a sequence of local copy commands is sent to the display computer to repeat the tile image.
  • the source of the local copy can grow by powers of two each copy step.
  • the source width*height pixels can be copied immediately to the width*height pixels to their right. This is illustrated by the upper-left drawing 304 of FIG. 3, where the hashed rectangle 300 is the source of the local copy command and the bricked rectangle 302 is the destination.
  • the upper-left (2*width)*height pixels can be copied to their right. This is illustrated by the upper-right drawing 306 of FIG. 3. Once the right edge of the rectangle is reached (drawing 308 in FIG.
  • the entire width*height (here width is the width of the rectangle) upper-left pixels can be used as the source for a vertical local copy (drawing 310 in FIG. 3).
  • the vertical copies continue by powers of two until the bottom edge of the rectangle is reached (drawing 312 in FIG. 3).
  • the host computer can use the tile tags of the screen image to determine whether some or all of the tile image already exists on the display computer's screen. If it does, then local copies can be used to paint part or all of the upper-left width*height pixels on the display computer. This is illustrated in FIG. 4, where part (the bricked rectangle 400 ) of the upper-left width*height pixels of a rectangle 402 is drawn using a local copy from a tiled rectangle (hashed rectangle 404 ) in a rectangle 406 with a compatible repeating image that was already on screen.
  • the screen image tile tag may be updated after each rectangle is drawn.
  • subsequent rectangles can take advantage of tile images created in drawing the previous rectangles. That is, the tile image will never need to be sent more than once no matter how many rectangles exist in the repeating-image portion of the drawn-to region.
  • FIG. 4 could illustrate a case where there are no pre-existing tiled images and a single tile command creates two tiled rectangles. In this case, drawing of the second rectangle would take advantage of the fact that the first rectangle contained part of the tile image.
  • the present invention provides for the quick, low-cost identification of repeating images in cases where prior techniques failed to identify (e.g., screen image refreshes and CopyArea from a pixmap with a repeating image).
  • the present invention also provides for the extremely low bandwidth transmission of the first width*height pixels of a repeating-image rectangle in cases where the tile image already exists on screen. This advantage is present even for tiled images drawn directly to a realized window, where prior techniques would recognize the new image as being tiled, but would still have to send the first width*height pixels from scratch.
  • a cache of tile images may be maintained on the display computer.
  • the server may manage the cache entries based on the size of the cache reported by the display computer at the time the host-client connection is formed.
  • the server may tell the display computer to add the new tile image to its cache with a given ID number.
  • Tile images may only be removed from the cache when cache space is needed for newer tile images (as opposed to because the tile no longer appears on the screen).
  • the server may tell the display computer to make the required copies of the appropriate cached tile image.
  • the cache embodiment will yield lower bandwidth and faster performance than the embodiment that tries to use an existing on-screen copy of the tile image as the source for the first width*height pixels.
  • FIGS. 5 A- 5 C are flow diagrams illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. These figures show the processing of a single DDX command in accordance with an embodiment of the present invention. These figures refer specifically to an embodiment where the Sun RayTM server software is modified to practice the invention. One of ordinary skill in the art will recognize that this method may be modified for use with other remote display applications.
  • pixmap and screen sub-images are identified with fill, tile, and miscellaneous region tags using either the underlay or disjoint model. For better results, RGB and text tags may also be used.
  • DDX commands may require special handling.
  • the command is a DestroyPixmap command. It should be noted that one of ordinary skill in the art will recognize that a destroy pixmap command, any command utilized to destroy or remove a pixmap or pixmap-like structure, could be substituted for DestroyPixmap. If it is a DestroyPixmap command, then at 502 , the regions in the pixmap's fill and tile linked lists are uninitialized and the fill and tile linked lists are destroyed at 504 .
  • the miscellaneous region is uninitialized and at 508 , standard DDX pixmap destruction may be performed. If it is not a DestroyPixmap command, then at 510 the standard DDX routine for the command is called.
  • the standard processing can be enhanced using the methods of FIGS. 3 - 4 to send the repeating images.
  • the command is a CopyArea from a pixmap to a realized window. If not, then it is determined at 514 if the command is a RestoreAreas from a backing store pixmap to a realized window. If neither is true, the system may simply perform standard Sun RayTM post-processing of the command at 516 . For example, if the command renders to a realized window, then the Sun RayTM software sends the appropriate screen update information to the display computer.
  • FIG. 6 is a flow diagram illustrating the sending of an update resulting from copying a pixmap to a window in accordance with an embodiment of the present invention.
  • the drawn-to region is computed.
  • the region will contain zero rectangles if empty or more than zero if it is non-empty. Then it is determined if there are any more rectangles in the tile portion at 606 . If so, then the upper left width * height pixels in the rectangle are sent to the display computer at 608 (also shown in FIGS. 3 - 4 ). Then at 610 , a sequence of local copy commands are sent to the display computer to repeat the tile image (also shown in FIG. 3). This repeats until each rectangle is sent for each region defined by each entry in the fill linked list.
  • the next entry's fill region is used to compute the portion of the drawn-to region that contains fills with the color specified in the linked list entry. The region will contain zero rectangles if empty or more than zero if it is non-empty. Then it is determined if there are any more rectangles in the fill portion at 616 . If so, then the 1-color protocol command for the next rectangle is sent, using the color of the linked list entry at 618 . This repeats until each rectangle is sent for each region defined by each entry in the fill linked list.
  • the pixmap's miscellaneous region may be used to compute the miscellaneous portion of the drawn-to region.
  • miscellaneous portion is empty. If not, it may be sent using the ordinary Sun RayTM method at 624 .
  • the command is a CreatePixmap command. It should be noted that one of ordinary skill in the art will recognize that a create pixmap command, any command utilized to create a pixmap or pixmap-like structure, could be substituted for CreatePixmap. If the command is a CreatePixmap command, then at 523 an empty fill linked list and an empty tile linked list are attached to the pixmap and at 524 , the miscellaneous region is attached to the pixmap and initialized to empty.
  • the command is not a CreatePixmap command, then at 526 it is determined if the command is an opaque PolyFllRect to a tagged drawable. This may represent case 3 of the commands requiring special treatment as described above (opaque for PolyFillRect means full planemask and an alu of GXcopy). It should be noted that one of ordinary skill in the art will recognize that an opaque poly fill rectangle command, any command utilized to overwrite a rectangle with a single color or a tiled pattern, could be substituted for PolyFillRect. PolyFillRect commands are frequently used to set a region to the background color or pattern. If the command is an opaque PolyFillRect command, then at 528 the drawn-to region is computed.
  • FIG. 7 is a flow diagram illustrating a method for subtracting the drawn-to region from the fill and tile linked lists in accordance with an embodiment of the present invention.
  • it is determined if there are more fill linked list entries. If not, then the process continues at step 710 . If so, then at 702 , the drawn-to region is subtracted from the next linked list entry's region.
  • it is determined if the entry's region is empty.
  • the entry's region is uninitialized at 706 and removed from the linked list at 708 . Then, at 710 , it is determined if there are more tile linked list entries. If not, then the process ends and it returns to FIG. 5B. If so, then at 712 , the drawn-to region is subtracted from the next entry's region. At 714 , it is determined if the region's entry is empty. If so, the entry's region is uninitialized at 716 and removed from the linked list at 718 .
  • FIG. 8 is a flow diagram illustrating a method for submitting a region to a fill linked list in accordance with an embodiment of the present invention.
  • an existing linked list entry has the same color as the submitted region. If so, then at 802 , the submitted region is added to that linked list entry. If not, a new entry is created at the end of the linked list at 804 . Then at 806 , the color of the new entry is set to the color of the submitted region.
  • the region of the new entry is set to the submitted region.
  • the drawn-to region is subtracted from the entry's region at 532 and added back in at 802 . This may appear somewhat redundant, but is preferred for programming simplicity in that for the implementation described, the processing in 532 does not depend on the fill style or fill color.
  • FIG. 9 is a flow diagram illustrating a method for submitting a region to a tile linked list in accordance with an embodiment of the present invention.
  • the repeating image region is compatible with any existing tile entries in the tile linked list. If so, then at 902 , the repeating image region is added to that linked list entry.
  • a new entry is created at the end of the linked list at 904 . Then at 906 , the parameters of the new entry is set to the parameters of the repeating image region. At 908 , the region of the new entry is set to the repeating image region.
  • FIG. 10 is a flow diagram illustrating performing tag copy from a source to a destination in accordance with an embodiment of the present invention.
  • an unmodified copy of the source drawable's region tags is made. This is necessary if the source and destination drawable are the same, and if the source and destination rectangles overlap.
  • the drawn-to region is computed. Then at 1004 , the drawn-to region is subtracted from the destination drawable miscellaneous region. Then at 1006 , the drawn-to region is subtracted from the destination drawable's fill linked list according to the procedure in FIG. 7.
  • the source drawable's miscellaneous region is used to compute the miscellaneous portion of the drawn-to region.
  • the miscellaneous portion is added to the destination drawable's miscellaneous region.
  • 546 may also be performed if it is determined if that the command is a CopyWindow at 548 (which also represents case 4 of the commands requiring special treatment as described above), a RestoreArea from a backing store pixmap to a tagged window at 550 (which represents case 5 of the commands requiring special treatment as described above), or a SaveAreas from a tagged window to a backing store pixmap at 552 (which represents case 6 of the commands requiring special treatment as described above).
  • the command is a PaintWindowBackground to a tagged window. If so, then at 556 it is determined if the window has backgroundState equal to BackgroundPixmap. If so, then this represents case 1 of the commands requiring special treatment as described above, and at 562 the drawn-to region is computed. Then at 564 , the drawn-to region is subtracted from the miscellaneous region. At 566 , the drawn-to region may be subtracted from the fill and tile linked lists according to the procedure outlined in FIG. 7 and the corresponding text. Then, at 568 , the repeating image region may be submitted to the tile linked list according to the procedure outlined in FIG. 9 and the corresponding text.
  • 562 - 568 may also be performed if at 558 it is determined that the command is a PaintWindowBorder for a tagged window, and at 560 it is determined that the window has borderIsPixel equal to false, which represents case 2 of the command requiring special treatment as described above.
  • FIG. 11 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “underlay” embodiment.
  • the region tags include an RGB region
  • the destination drawable's RGB region is first subtracted from the submitted region. This is included for the convenience of implementers using an RGB region as a component of the region tags. If an RGB region is not being used, 1100 need not be executed.
  • FIG. 12 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “disjoint” embodiment.
  • the submitted region is first subtracted from the fill and tile linked lists according to the procedure of FIG. 7.
  • the destination drawable's RGB region is subtracted from the submitted region.
  • the remainder is added to the destination drawable's miscellaneous region at 1204 .
  • FIG. 13 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.
  • a certain class of image data e.g., tile regions
  • a tile command executed to create a portion of the graphics image in a destination drawable is recorded.
  • Command recording may or may not be limited to commands from the drawing commands identified above.
  • This drawing command may also be one that is executed during the rendering of the graphics image.
  • the graphics image may be created in a destination drawable.
  • the tile command may be used to update a data structure associated with the destination drawable with information. This may include using information about the region drawn to by the drawing command to update the data structure.
  • This data structure may be associated with the destination drawable.
  • a private data field, such as a devPrivates field may be used to attach the data structure to the destination drawable. While use of the X Windows devPrivates facility would be one method of associating a data structure with the drawable, there are other possible choices for the method of associating the two.
  • the information in the data structure may enable determination of what portion of the image contains sub-images of the specified image class.
  • One example data structure would be a linked list identifying regions of repeating images.
  • an X region resource is used to store the region of the destination drawable which contains sub-images of the specified image class.
  • one or more sub-images of the graphics image may be compressed using a compression scheme, each of the sub-images having a location, the location and the compression scheme for each of the sub-images based on the information in the data structure. It should be noted that the compressor does not necessarily need to know exactly which drawing commands were executed. It only needs to know what sub-images of the image should be compressed with a given compression scheme.
  • the compressor may use the information in the data structure to find the location within the image of sub-images of the specified image class.
  • the compressor may compress these sub-images using a compression scheme selected for the specified image class. If the region covered by the sub-images is represented as a set of one or more rectangles, as is the case for the X region resource, each rectangle may be compressed and sent separately.
  • FIG. 14 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention.
  • a tile command executed to create a portion of the graphics image in a destination drawable may be detected. Command detection may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable.
  • a drawn-to region may be computed from the tile command.
  • the drawn-to region may be subtracted from a miscellaneous region.
  • the drawn-to region may be subtracted from one or more fill and/or tile regions.
  • the drawn-to region may be added to one of one or more tile regions, each of the tile regions corresponding to a repeating image.
  • the tile image i.e., the upper width*height pixels in the rectangle
  • the tile image may be transmitted for display or storage.
  • a sequence of local copy commands may be transmitted to repeat the tile image.
  • FIG. 15 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.
  • a certain class of image data e.g., tile regions
  • a tile command recorder 1500 may record a tile command executed to create a portion of the graphics image in a destination drawable. Command recording may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable.
  • a data structure updater 1502 coupled to the tile command recorder 1500 may use the tile command to update a data structure associated with the destination drawable with information. This may include using information about the region drawn to by the drawing command to update the data structure. This data structure may be associated with the destination drawable.
  • a private data field such as a devPrivates field, may be used to attach the data structure to the destination drawable. While use of the X Windows devPrivates facility would be one method of associating a data structure with the drawable, there are other possible choices for the method of associating the two.
  • the information in the data structure may enable determination of what portion of the image contains sub-images of the specified image class.
  • One example data structure would be a linked list identifying regions of repeating images.
  • an X region resource is used to store the region of the destination drawable which contains sub-images of the specified image class.
  • An image compressor 1504 coupled to the data structure updater 1502 may compress one or more sub-images of the graphics image using a compression scheme, each of the sub-images having a location, the location and the compression scheme for each of the sub-images based on the information in the data structure. It should be noted that the compressor does not necessarily need to know exactly which drawing commands were executed. It only needs to know what sub-images of the image should be compressed with a given compression scheme.
  • the compressor may use the information in the data structure to find the location within the image of sub-images of the specified image class.
  • the compressor may compress these sub-images using a compression scheme selected for the specified image class. If the region covered by the sub-images is represented as a set of one or more rectangles, as is the case for the X region resource, each rectangle may be compressed and sent separately.
  • FIG. 16 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention.
  • a tile command detector 1600 may detect a tile command executed to create a portion of the graphics image in a destination drawable. Command detection may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable.
  • a drawn-to region computer 1602 coupled to the tile command detector 1600 may compute a drawn-to region from the tile command.
  • a drawn-to region to miscellaneous region remover 1604 coupled to the tile command detector 1600 and to the drawn-to region computer 1600 may subtract the drawn-to region from a miscellaneous region.
  • a drawn-to region to fill and tile region remover 1606 coupled to the tile command detector may subtract the drawn-to region from one or more fill and/or tile regions.
  • a tile region adder 1608 coupled to the tile command detector 1600 and to the drawn-to region computer 1602 may add the drawn-to region to one of one or more tile regions, each of the tile regions corresponding to a repeating image.
  • a memory or memories 1610 coupled to the tile region adder 1608 , the drawn-to region to miscellaneous region remover 1604 and to the drawn-to region to fill and tile region remover 1606 may store the various regions.
  • a tile image transmitter 1612 coupled to the memory 1610 may transmit the tile image for display or storage.
  • a local copy command transmitter 1614 coupled to the tile image transmitter 1612 may transmit a sequence of local copy commands to repeat the tile image.

Abstract

Repeating image content in a graphics image may be detected by identifying certain commands, known generally as “tile commands”. If a tile command is detected, the fact that a portion of an image was created with a tile command may be stored along with the portion of the image. This allows for well-informed decision making when transmission of the image is to be performed. For example, this allows for the transmission of a single tile, and subsequent transmission of local copy commands to repeat the single tile. This can be very useful in speeding transmission of background images or other repeating images.

Description

    CROSS-REFERENCE TO RELATED APPLICATION
  • The present application is a continuation-in-part of co-pending application Ser. No. 10/247,907, filed on Sep. 20, 2002, by Thomas G. O'Neill and Jordan M. Slott, entitled “TAGGING SINGLE-COLOR IMAGES FOR IMPROVED COMPRESSION”, attorney docket no. SUN-P7083, which is a continuation in part of co-pending application Ser. no. 10/194,123, filed on Jul. 11, 2002, by Thomas G. O'Neill and Jordan M. Slott, entitled “TAGGING MULTICOLOR IMAGES FOR IMPROVED COMPRESSION”, attorney docket no. SUN-P7082.[0001]
  • FIELD OF THE INVENTION
  • The present invention relates to the field of computer graphics. More particularly, the present invention relates to the tagging of repeating images in order to improve compression. [0002]
  • BACKGROUND OF THE INVENTION
  • Remote computing applications where screen information is generated on one computer (a “host”) and transmitted for display on another computer (“a display”) are growing in popularity. Examples of some display computers include multipurpose PCs, thin-clients, and Personal Digital Assistants (PDAs). [0003]
  • The X Windows System is a standard that utilizes X client software and X server software to enable the updating of displays at the requests of X client applications. The X server software runs on the host computer. An X client application, which may be running on a different computer, communicates with the X server software by utilizing a low-level library of routines known as Xlib. Xlib provides the functions required to connect to display servers, create windows, render graphics, respond to events, etc. The X server software then may interface with display drivers to actually render the graphics on the display. [0004]
  • X is frequently used as a “local display application”, where the X-server and display are both on the same computer. That is, the same computer is both the “host” and “display” computer. Examples of this usage include running an X-server on a workstation or on an X-terminal. An X-terminal typically has a computer processor, graphics subsystem and display, but no hard drive. Applications running on other computers use the Xlib routines to communicate with the X-server software running on the X-terminal. [0005]
  • While in some contexts it is advantageous to have the X server and the display on the same computer, this is not necessarily the case in other contexts. One specific context that will be discussed is a remote display application. In such a design, client applications make requests of a centralized server or servers (here known collectively as the “host computer”). The host computer then manages one or more “display computers”, which are typically simple terminal devices. [0006]
  • The Sun Ray™ appliance from Sun Microsystems, Inc. of Palo Alto, Calif. is an example of a thin client which serves as a “display computer” in a remote computing application. A Sun Ray™ appliance has a processor, graphics subsystem and display, but no hard drive. A Sun Ray™ appliance is a “display” computer and runs its own proprietary software. The Sun Ray™ server is the “host” computer and runs the X-server software. The full screen image is maintained both in RAM on the host computer as well as in the frame buffer of the Sun Ray™ appliance's video card. In order to synchronize the displays, the host computer sends screen update information to the Sun Ray™ appliance via a network protocol known as NewT. The Sun Ray™ appliance uses the protocol commands to update the state of its hardware frame buffer. [0007]
  • In remote display applications, an increased burden is placed on the network as more information is transmitted from the host computer to the display computers. It is desirable to reduce network bandwidth used by remote computing applications. Doing so provides shorter transmission times between the host and display computers, reduced load on the network (and resulting improvement in network performance), and the capability to utilize more devices on a single network. [0008]
  • For many typical computing functions such as web browsing, the network bandwidth between the host computer and the display computer is dominated by the transmission of images with a large number of colors, such as photographs or computer generated images which include anti-aliased text or graphics, as well as single-color regions, such as “fill” regions. Furthermore, even once these multi-color and single-color images are addressed, there can still exist a large amount of screen space dedicated to repeating images. These typically include background “tiles”, which are repeated over and over on the screen. [0009]
  • In particular, the root window in many window managers is a tile of a repeating M×N image, with M indicating horizontal pixels and N indicating vertical. Stipples, which are essentially two-color tiles, are another source of repeating images. [0010]
  • Sending multiple copies of the tile image wastes bandwidth, host computer processing power, and display computer processing power. The resulting delay can be noticeable even over a 100 Mbps connection. What is needed is an efficient mechanism for the host computer to identify and utilize the identification of tiled regions so that the tile image may only be sent once. [0011]
  • BRIEF DESCRIPTION
  • Repeating image content in a graphics image may be detected by identifying certain commands, known generally as “tile commands”. If a tile command is detected, the fact that a portion of an image was created with a tile command may be stored along with the portion of the image. This allows for well-informed decision making when transmission of the image is to be performed. For example, this allows for the transmission of a single tile, and subsequent transmission of local copy commands to repeat the single tile. This can be very useful in speeding transmission of background images or other repeating images. [0012]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate one or more embodiments of the present invention and, together with the detailed description, serve to explain the principles and implementations of the invention. [0013]
  • In the drawings: [0014]
  • FIG. 1 is a block diagram illustrating a remote display application such as a Sun Ray™ network. [0015]
  • FIG. 2 is a block diagram illustrating a remote display application in accordance with an embodiment of the present invention. [0016]
  • FIG. 3 is a diagram illustrating how when one of the tile commands is used on a realized window, the new repeating image may be sent as a display update to the display computer, in accordance with an embodiment of the present invention. [0017]
  • FIG. 4 is a diagram illustrate how local copies can be used to paint part or all of the upper-left width*height pixels on the display computer in accordance with another embodiment of the present invention. [0018]
  • FIGS. [0019] 5A-5C are flow diagrams illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.
  • FIG. 6 is a flow diagram illustrating the sending of the screen update resulting from copying a pixmap to a realized window in accordance with an embodiment of the present invention. [0020]
  • FIG. 7 is a flow diagram illustrating a method for subtracting the drawn-to region from the fill and tile linked lists in accordance with an embodiment of the present invention. [0021]
  • FIG. 8 is a flow diagram illustrating a method for submitting a region to a fill linked list in accordance with an embodiment of the present invention. [0022]
  • FIG. 9 is a flow diagram illustrating a method for submitting a region to a tile linked list in accordance with an embodiment of the present invention. [0023]
  • FIG. 10 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “disjoint” embodiment. [0024]
  • FIG. 11 is a flow diagram illustrating performing tag copy from a source to a destination in accordance with an embodiment of the present invention. [0025]
  • FIG. 12 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “underlay” embodiment. [0026]
  • FIG. 13 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. [0027]
  • FIG. 14 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention. [0028]
  • FIG. 15 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. [0029]
  • FIG. 16 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention. [0030]
  • DETAILED DESCRIPTION
  • Embodiments of the present invention are described herein in the context of a system of computers, servers, and software. Those of ordinary skill in the art will realize that the following detailed description of the present invention is illustrative only and is not intended to be in any way limiting. Other embodiments of the present invention will readily suggest themselves to such skilled persons having the benefit of this disclosure. Reference will now be made in detail to implementations of the present invention as illustrated in the accompanying drawings. The same reference indicators will be used throughout the drawings and the following detailed description to refer to the same or like parts. [0031]
  • In the interest of clarity, not all of the routine features of the implementations described herein are shown and described. It will, of course, be appreciated that in the development of any such actual implementation, numerous implementation-specific decisions must be made in order to achieve the developer's specific goals, such as compliance with application- and business-related constraints, and that these specific goals will vary from one implementation to another and from one developer to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking of engineering for those of ordinary skill in the art having the benefit of this disclosure. [0032]
  • In accordance with the present invention, the components, process steps, and/or data structures may be implemented using various types of operating systems, computing platforms, computer programs, and/or general purpose machines. In addition, those of ordinary skill in the art will recognize that devices of a less general purpose nature, such as hardwired devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or the like, may also be used without departing from the scope and spirit of the inventive concepts disclosed herein. [0033]
  • One example use for the present invention would be to improve compression of images created in pixel-based (as opposed to vector-based) drawing programs. Here, one might keep track of the drawing commands (circle, filled circle, square, image importation, etc.) used to draw the different portion of the image and use that information to influence compression decisions when saving the results to a file. [0034]
  • In the present application, an embodiment of the present invention is described in the context of a modification of traditional X Windows technology for use with remote computing applications. However, one of ordinary skill in the art will recognize that other embodiments are possible and the present invention should not be limited to X Windows, Sun Ray™, or remote computing technology. [0035]
  • In remote display applications based on an X server, the Device Dependent X (DDX) layer of the X server may be modified to include possible compression of the screen data and transmission of remote display protocol commands to the display computer. The DDX provides a software interface to a conceptual hardware device. In an embodiment of the present invention, the desired results can be accomplished by modifying the DDX, without changing the DIX. [0036]
  • FIG. 1 is a block diagram illustrating a remote display application such as a Sun Ray™ network. Client applications [0037] 100 a-100 c send information to and receive information from the host computer 102 over the X Windows Protocol. Some of the information comprises repeating images. The host computer utilizes a device independent layer (DIX) 104 to act as a controlling layer, which handles initialization procedures and all device-independent handling of requests and events. A device dependent layer (DDX) 106 then is used to handle device dependent functionality. The DDX 106 then communicates with display computer 108 using the NewT protocol. The DIX and the DDX essentially make up an X-server.
  • The present invention provides a mechanism to identify repeating images for special treatment. As will be seen, this results in the ability to dramatically reduce CPU and network bandwidth utilization without affecting image quality. The present application will focus on this identification and handling of repeating images, however the identification and handling of multicolor and singlecolor non-repeating images as well as other, miscellaneous images will also be discussed. The solution for identifying and handling repeating images may be used in conjunction with the solutions for identifying and handling single-color, multi-color images and miscellaneous images, but does not have to be. The identification and handling of repeating images alone will still improve CPU and bandwidth utilization and overall efficiency. [0038]
  • FIG. 2 is a block diagram illustrating a remote display application in accordance with an embodiment of the present invention. Client applications [0039] 200 a-200 c send information to and receive information from the host computer 202 over the X Windows Protocol. Some of the information is repeating image information. The host computer utilizes a device independent layer (DIX) 204 to act as a controlling layer, which handles initialization procedures and all device-independent handling of requests and events. A device dependent layer (DDX) 206 then is used to handle device dependent functionality. An image compression layer 208 may then be provided which performs the additional image compression techniques discussed in the present document. Alternatively, this image compression layer may simply be part of a modified DDX. The image compression layer 208 then communicates with display computer 210 using an enhanced version of the NewT protocol.
  • For purposes of this application, a “tile command” may be defined as a drawing command that creates a horizontally and/or vertically repeating image. In the X-server, this is typically accomplished via one of several DDX commands: [0040]
  • 1) A PaintWindowBackground for a window with backgroundState==BackgroundPixmap. [0041]
  • 2) A PaintWindowBorder for a window with borderIsPixel==FALSE [0042]
  • 3) A PolyFillRect with a full planemask and an alu of GXcopy, if: [0043]
  • a) fillStyle is FillTiled or FillOpaqueStippled, or [0044]
  • b) fillStyle is FillStippled, and the drawing occurs on top of a single-colored region. [0045]
  • 4) CopyArea or CopyWindow where the source drawable includes a repeating image. [0046]
  • 5) RestoreAreas from a backing store pixmap containing a repeating image. [0047]
  • 6) SaveAreas to a backing store pixmap from a tagged window containing a repeating image. [0048]
  • Commands 1-3 above can be used to create a repeating image directly in a realized (on-screen) window. Commands 4-6 above can be used to draw a repeating image indirectly, by first creating it in the source drawable. [0049]
  • In an embodiment of the present invention, the tile regions that are created using one of these tile commands may be tagged. In an embodiment of the present invention, the tagging may be accomplished by attaching a data structure to each pixmap via the pixmap devPrivates facility. In another embodiment of the present invention, the tagging of windows may be accomplished by having a single data structure for the entire screen image. [0050]
  • In an embodiment of the present invention, the data structure may include a list of tile entries. For example, a linked list could be used. One entry may exist for each tile region that is not compatible with any other tile region. For purposes of the present invention, two regions are compatible if they are drawn with the same tile image and the tile edges occur at the same multiples of x and y. Each linked list entry may contain the following information: [0051]
  • 1) Width of the tile image [0052]
  • 2) Height of the tile image [0053]
  • 3) Origin x of the tile image [0054]
  • 4) Origin y of the tile image [0055]
  • 5) A hash value for the tile image (e.g., MD5), to allow quick comparison of whether two tile images are identical [0056]
  • 6) X region struct identifying the portion of the pixmap or screen image that contains this tile image. [0057]
  • In this embodiment, tile regions are compatible if they have the same values of width, height, hash value, x modulo width, and y modulo height. The modulo computations take into account the fact that the image repeats every width pixels horizontally and height pixels vertically. Thus, in one embodiment, it is convenient to store x modulo width and y modulo height instead of just x and y. [0058]
  • In another embodiment, the tile image may be used in place of the hash value. For example, a pointer to the tile image pixmap could be used instead of the hash. In this case, the pixmap's reference count would have to be incremented to ensure that the pixmap does not get deleted while the pointer is still set. [0059]
  • The linked list may be associated with the pixmap through the use of the DDX pixmap devPrivates facility, which allows an X extension to attach a private data structure to each X pixmap. In X, a region resource may be used to describe an arbitrarily shaped piece of real estate in XY pixel space. I.e., it can be used to describe an arbitrary collection of pixels by characterizing their XY positions. In X, the positions are characterized by an array of rectangles for which every pixel is in the region. This representation is efficient for regularly-shaped regions, which occur frequently in X, but inefficient for regions with a lot of isolated pixels, which occur much less frequently in X. The rectangle representation is also very convenient for remote display applications such as Sun Ray™, where each protocol command specifies the screen update for a single screen rectangle. [0060]
  • X also provides a series of routines for performing set operations (union, subtraction, intersection, etc.) on the region resources, which are very convenient in the practice of the invention. The region resources and routines were originally developed to assist with the computation of clipping and drawn-to regions. The composite clip list for a drawable (window or pixmap) represents the region that can currently be drawn to. It includes the effects of clipping requested by the client application (the client clip list) and, if the drawable is a window, by other windows that are on top of the window in question. The drawn-to region for a particular DDX command is found by intersecting the composite clip list with the region that the command would render in the absence of clipping. It is this region to which the command actually renders. [0061]
  • In X Windows, stipples are often used to tile a region with a two-color tile. The stipple is specified as a bit mask, where a 0 bit represents a pixel with the current background color and a 1 bit represents a pixel with the current foreground color. Thus, the same stipple produces incompatible tile images if either the background or foreground color is changed. [0062]
  • The tile entry should contain enough information to distinguish between such incompatible tiles. In one embodiment, the hash value of a stipple's tile image may be computed using the resulting full-color image. If the drawn-to region does not include a full copy of the tile image, this would require an additional computational step where the stipple is converted to a full-color image. In another embodiment, the hash value may instead be computed directly from the stipple bit mask. In this case, three additional pieces of data may be required in the tile list entry: [0063]
  • 1) bStipple: true if the tile image was drawn as a one-bit-per-pixel stipple, e.g., using PolyFillRect in the FillOpaqueStippled mode; false if it was drawn as a 24-bit-per pixel pixmap. [0064]
  • 2) colorFg: foreground color [0065]
  • 3) colorBg: background color [0066]
  • If bStipple is false, then color Fg and colorBg may be ignored because the tile image contains the complete color information. If bStipple is true, compatibility of tiles requires that the two tiles have the same foreground and background colors, in addition to the aforementioned conditions. [0067]
  • When a tile command creates a repeating image region in a drawable, the host computer computes the repeating image portion of the drawn-to region. For most of the DDX tile commands, the repeating-image portion of the drawn-to region is just the drawn to region, but several commands require special handling: PolyFillRect in the FillStippled mode, CopyArea, CopyWindow, SaveAreas, and RestoreAreas. [0068]
  • For PolyFillRect in the FillStippled mode, foreground pixels may be rendered and background pixels left unchanged. Thus, a repeating image may be generated if the stipple is drawn on top of a single-color fill region. [0069]
  • For CopyArea, CopyWindow, SaveAreas, and RestoreAreas, the repeating-image portion of the drawn-to region may be computed as follows (compare FIG. 12): [0070]
  • 1) A repeating-image portion of the source drawable is the region in one of the source drawable's tile linked list entries. [0071]
  • 2) This portion is translated to the coordinates of the destination drawable. [0072]
  • 3) The translated portion is intersected with the drawn-to region of the DDX command. [0073]
  • This process may then be repeated for each tile linked list entry in the source drawable. [0074]
  • When a tile command creates a repeating-image in a drawable, the host may check to see if the new repeating image is compatible with any of the existing tile entries in the tile linked list. If a compatible entry is found, then the repeating-image portion of the drawn-to region for the DDX command may be added to the entry's tile region. If no compatible entry is found, a new entry may be created with the parameters of the new repeating image and the region may be set to the repeating-image portion of the drawn-to region. For more details, see FIG. 9 and its corresponding description. [0075]
  • An embodiment of the present invention may be used in conjunction with an “RGB region” which identifies the portions of the drawable composed of RGB sub-images. For purposes of this application, an RGB image may be defined as one that is rendered from a list of pixel values (in contrast to an image rendered from a specification of a geometric figure or text characters to draw). In a drawing program, such an image may be an imported JPEG image where the pixel values are computed from the compressed JPEG data. In an X Server, this may be an image drawn with the XPutImage command, where the pixel values are provided directly by the client application. XPutImage copies the image data in an XImage structure into an area of a window or pixmap. This is the typical way to create non-repeating, many-colored images on an X server, as the other drawing primitives are typically used to draw only one or two colors at a time. [0076]
  • While XPutImage is a specific command in the X Windows System, the term put image command may be used in the present application to refer to any rendering command used to create an RGB image. [0077]
  • Additionally, an embodiment of the present invention may also be used in conjunction with a “fill region”, which identifies the portions of the-drawable composed of single-color regions. [0078]
  • Repeating images are often used to create the background portion of an image. Thus it is frequent that other drawing commands create sub-images on top of the repeating image. In one embodiment of the present invention, when drawing occurs on top of an existing tile region, the drawn-to region is subtracted from the tile region. This may be termed the “disjoint” embodiment. If the present invention is used in conjunction with single-color and multi-color region tagging, then the region removed from the tile region should be added to a “miscellaneous” region or to the RGB region. This miscellaneous region tracks the region of the drawable that contains non-RGB image data drawn on top of a fill region and/or tile region. The miscellaneous region should be used when the tile image is low color (e.g. if bStipple==TRUE). Otherwise, the RGB region should be used. When a tile region becomes empty, the corresponding linked list entry can be removed from the tile linked list. [0079]
  • In another embodiment of the present invention, the tile images are treated as underlays of the subsequent drawing. This may be termed the “underlay” embodiment. In this embodiment, only drawing commands which affect large, rectangular portions of the drawable have their drawn-to regions subtracted from the tile regions. Examples of such commands would be PolyFillRect and PutImage. The drawn-to regions of other commands are instead added to the miscellaneous region. When the image in a drawable is sent to the display computer, correct display may be ensured by sending the tile regions first, followed by the fill, miscellaneous and RGB regions. [0080]
  • When one of the tile commands is used on a realized window, the new repeating image may be sent as a display update to the display computer. FIG. 3 is a diagram illustrating this process. In one embodiment of the present invention, each rectangle in the repeating-image portion is sent separately. For each rectangle: [0081]
  • 1) The upper left width*height pixels in the rectangles are sent to the client. This produces the hashed [0082] rectangle 300 show in FIG. 3. Although the left and top edges of these pixels may not correspond to the left and top edges of the original tile image, the image pattern will still repeat every width*height pixels. That is, every dashed rectangle 302 in FIG. 3 should contain the same image after the update is complete. If these pixels contain many colors, then an RGB codec may be used.
  • 2) A sequence of local copy commands is sent to the display computer to repeat the tile image. For best efficiency, the source of the local copy can grow by powers of two each copy step. For example, the source width*height pixels can be copied immediately to the width*height pixels to their right. This is illustrated by the upper-left drawing [0083] 304 of FIG. 3, where the hashed rectangle 300 is the source of the local copy command and the bricked rectangle 302 is the destination. In the next step, the upper-left (2*width)*height pixels can be copied to their right. This is illustrated by the upper-right drawing 306 of FIG. 3. Once the right edge of the rectangle is reached (drawing 308 in FIG. 3), the entire width*height (here width is the width of the rectangle) upper-left pixels can be used as the source for a vertical local copy (drawing 310 in FIG. 3). The vertical copies continue by powers of two until the bottom edge of the rectangle is reached (drawing 312 in FIG. 3).
  • In another embodiment of the present invention, the host computer can use the tile tags of the screen image to determine whether some or all of the tile image already exists on the display computer's screen. If it does, then local copies can be used to paint part or all of the upper-left width*height pixels on the display computer. This is illustrated in FIG. 4, where part (the bricked rectangle [0084] 400) of the upper-left width*height pixels of a rectangle 402 is drawn using a local copy from a tiled rectangle (hashed rectangle 404) in a rectangle 406 with a compatible repeating image that was already on screen.
  • The screen image tile tag may be updated after each rectangle is drawn. Thus, subsequent rectangles can take advantage of tile images created in drawing the previous rectangles. That is, the tile image will never need to be sent more than once no matter how many rectangles exist in the repeating-image portion of the drawn-to region. For example, FIG. 4 could illustrate a case where there are no pre-existing tiled images and a single tile command creates two tiled rectangles. In this case, drawing of the second rectangle would take advantage of the fact that the first rectangle contained part of the tile image. [0085]
  • If the complete tile image already exists on screen, which is quite common, then the entire tile region update will comprise local copies commands that use very little bandwidth. This represents a huge savings in bandwidth over prior techniques. [0086]
  • Thus, the present invention provides for the quick, low-cost identification of repeating images in cases where prior techniques failed to identify (e.g., screen image refreshes and CopyArea from a pixmap with a repeating image). The present invention also provides for the extremely low bandwidth transmission of the first width*height pixels of a repeating-image rectangle in cases where the tile image already exists on screen. This advantage is present even for tiled images drawn directly to a realized window, where prior techniques would recognize the new image as being tiled, but would still have to send the first width*height pixels from scratch. [0087]
  • Additionally, in another embodiment of the present invention, a cache of tile images may be maintained on the display computer. To minimize latency, the server may manage the cache entries based on the size of the cache reported by the display computer at the time the host-client connection is formed. When a repeating image is first drawn to the screen, the server may tell the display computer to add the new tile image to its cache with a given ID number. Tile images may only be removed from the cache when cache space is needed for newer tile images (as opposed to because the tile no longer appears on the screen). [0088]
  • Any time a repeating image is drawn to the screen, the server may tell the display computer to make the required copies of the appropriate cached tile image. For a sufficiently large tile image cache, there will frequently be situations where a cached tile image can be used even though the tile image is not present anywhere on the screen (e.g., when switching between workspaces with two different tile backgrounds). In these situations, the cache embodiment will yield lower bandwidth and faster performance than the embodiment that tries to use an existing on-screen copy of the tile image as the source for the first width*height pixels. [0089]
  • FIGS. [0090] 5A-5C are flow diagrams illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. These figures show the processing of a single DDX command in accordance with an embodiment of the present invention. These figures refer specifically to an embodiment where the Sun Ray™ server software is modified to practice the invention. One of ordinary skill in the art will recognize that this method may be modified for use with other remote display applications. In this embodiment, pixmap and screen sub-images are identified with fill, tile, and miscellaneous region tags using either the underlay or disjoint model. For better results, RGB and text tags may also be used. For simplicity, use of these regions is not shown in the figure, but one of ordinary skill in the art would recognize that they could easily be added. Also in this embodiment, several DDX commands may require special handling. Referring first to FIG. 5A, at 500, it is determined if the command is a DestroyPixmap command. It should be noted that one of ordinary skill in the art will recognize that a destroy pixmap command, any command utilized to destroy or remove a pixmap or pixmap-like structure, could be substituted for DestroyPixmap. If it is a DestroyPixmap command, then at 502, the regions in the pixmap's fill and tile linked lists are uninitialized and the fill and tile linked lists are destroyed at 504. At 506, the miscellaneous region is uninitialized and at 508, standard DDX pixmap destruction may be performed. If it is not a DestroyPixmap command, then at 510 the standard DDX routine for the command is called. For DDX commands that draw repeating images to a realized window (e.g., PaintWindowBackground, PaintWindowBorder, opaque PolyFillRect with fillStyle equal to FillTiled or FillOpaqueStippled), the standard processing can be enhanced using the methods of FIGS. 3-4 to send the repeating images.
  • At [0091] 512 it is determined if the command is a CopyArea from a pixmap to a realized window. If not, then it is determined at 514 if the command is a RestoreAreas from a backing store pixmap to a realized window. If neither is true, the system may simply perform standard Sun Ray™ post-processing of the command at 516. For example, if the command renders to a realized window, then the Sun Ray™ software sends the appropriate screen update information to the display computer.
  • If the command is a CopyArea or RestoreAreas from a pixmap to a realized window, then at [0092] 518 the update resulting from the pixmap to window copy may be sent. This is described in more detail in FIG. 6. FIG. 6 is a flow diagram illustrating the sending of an update resulting from copying a pixmap to a window in accordance with an embodiment of the present invention. At 600, the drawn-to region is computed. At 602, it is determined if there are more entries in the pixmap's tile linked list. If so, then at 604 the next entry's tile region is used to compute the portion of the drawn-to region that contains tiles compatible with that which is specified in the linked list entry. The region will contain zero rectangles if empty or more than zero if it is non-empty. Then it is determined if there are any more rectangles in the tile portion at 606. If so, then the upper left width * height pixels in the rectangle are sent to the display computer at 608 (also shown in FIGS. 3-4). Then at 610, a sequence of local copy commands are sent to the display computer to repeat the tile image (also shown in FIG. 3). This repeats until each rectangle is sent for each region defined by each entry in the fill linked list.
  • Once that is complete, at [0093] 612 it is determined if there are more entries in the pixmap's fill linked list. If so, then at 614 the next entry's fill region is used to compute the portion of the drawn-to region that contains fills with the color specified in the linked list entry. The region will contain zero rectangles if empty or more than zero if it is non-empty. Then it is determined if there are any more rectangles in the fill portion at 616. If so, then the 1-color protocol command for the next rectangle is sent, using the color of the linked list entry at 618. This repeats until each rectangle is sent for each region defined by each entry in the fill linked list. Once that is complete, at 620 the pixmap's miscellaneous region may be used to compute the miscellaneous portion of the drawn-to region.
  • At [0094] 622, it is determined if the miscellaneous portion is empty. If not, it may be sent using the ordinary Sun Ray™ method at 624.
  • Referring back to FIG. 5A, at [0095] 520, it is determined if the command is a CreatePixmap command. It should be noted that one of ordinary skill in the art will recognize that a create pixmap command, any command utilized to create a pixmap or pixmap-like structure, could be substituted for CreatePixmap. If the command is a CreatePixmap command, then at 523 an empty fill linked list and an empty tile linked list are attached to the pixmap and at 524, the miscellaneous region is attached to the pixmap and initialized to empty.
  • If the command is not a CreatePixmap command, then at [0096] 526 it is determined if the command is an opaque PolyFllRect to a tagged drawable. This may represent case 3 of the commands requiring special treatment as described above (opaque for PolyFillRect means full planemask and an alu of GXcopy). It should be noted that one of ordinary skill in the art will recognize that an opaque poly fill rectangle command, any command utilized to overwrite a rectangle with a single color or a tiled pattern, could be substituted for PolyFillRect. PolyFillRect commands are frequently used to set a region to the background color or pattern. If the command is an opaque PolyFillRect command, then at 528 the drawn-to region is computed.
  • Then at [0097] 530, the drawn-to region is subtracted from the miscellaneous region. At 532, the drawn-to region is subtracted from the fill and tile linked lists. This is described in more detail in FIG. 7. FIG. 7 is a flow diagram illustrating a method for subtracting the drawn-to region from the fill and tile linked lists in accordance with an embodiment of the present invention. At 700, it is determined if there are more fill linked list entries. If not, then the process continues at step 710. If so, then at 702, the drawn-to region is subtracted from the next linked list entry's region. At 704, it is determined if the entry's region is empty. If so, the entry's region is uninitialized at 706 and removed from the linked list at 708. Then, at 710, it is determined if there are more tile linked list entries. If not, then the process ends and it returns to FIG. 5B. If so, then at 712, the drawn-to region is subtracted from the next entry's region. At 714, it is determined if the region's entry is empty. If so, the entry's region is uninitialized at 716 and removed from the linked list at 718.
  • Referring back to FIG. 5B, at [0098] 534, it is determined if the fill style is FillSolid. If so, then at 536, the drawn-to region is submitted to the fill linked list. This is described in more detail in FIG. 8. FIG. 8 is a flow diagram illustrating a method for submitting a region to a fill linked list in accordance with an embodiment of the present invention. At 800, it is determined if an existing linked list entry has the same color as the submitted region. If so, then at 802, the submitted region is added to that linked list entry. If not, a new entry is created at the end of the linked list at 804. Then at 806, the color of the new entry is set to the color of the submitted region. At 808, the region of the new entry is set to the submitted region. Consider an opaque PolyFillRect in the FillSolid fill style in the same color as an existing fill linked list entry: the drawn-to region is subtracted from the entry's region at 532 and added back in at 802. This may appear somewhat redundant, but is preferred for programming simplicity in that for the implementation described, the processing in 532 does not depend on the fill style or fill color.
  • Referring back to FIG. 5B, if the fill style is not FillSolid, then at [0099] 538 it is determined if FillStyle is equal to FillTiled ot FillOpaqueStippled. If so, then this represents case 3a of the commands requiring special treatment as described above, and at 540 the drawn-to region is submitted to the tile linked list. This is described in more detail in FIG. 9. FIG. 9 is a flow diagram illustrating a method for submitting a region to a tile linked list in accordance with an embodiment of the present invention. At 900, it is determined if the repeating image region is compatible with any existing tile entries in the tile linked list. If so, then at 902, the repeating image region is added to that linked list entry. If not, a new entry is created at the end of the linked list at 904. Then at 906, the parameters of the new entry is set to the parameters of the repeating image region. At 908, the region of the new entry is set to the repeating image region.
  • Referring back to FIG. 5B, if FillStyle is not equal to either FillTiled or FillOpaqueStippled, then at [0100] 542 the drawn-to region is added to the miscellaneous region. This handles case 3b of the commands requiring special treatment as described above. Alternatively, one could specifically identify FillStippled on top of a single-color fill as a repeating image using the fill tags of the destination drawable. In this case, the drawn-to region intersected with an existing fill tag for the destination drawable yields a single repeating-image region. Each different fill tag with a non-zero intersection results in a different tile tag. For simplicity, this alternative is not illustrated in the figure.
  • If the command is not an opaque PolyFillRect to a tagged drawable, then it may represent case 4 of the commands requiring special treatment as described above, and at [0101] 544 it is determined if the command is an opaque CopyArea from a tagged drawable to a tagged drawable. If so, then at 546, a tag copy from source to destination is performed. This is described in more detail in FIG. 10. FIG. 10 is a flow diagram illustrating performing tag copy from a source to a destination in accordance with an embodiment of the present invention. At 1000, if necessary, an unmodified copy of the source drawable's region tags is made. This is necessary if the source and destination drawable are the same, and if the source and destination rectangles overlap. Use of the unmodified copy in the remainder of the execution prevents confusion resulting from changes to the destination drawable's region tags during the execution of FIG. 10. At 1002, the drawn-to region is computed. Then at 1004, the drawn-to region is subtracted from the destination drawable miscellaneous region. Then at 1006, the drawn-to region is subtracted from the destination drawable's fill linked list according to the procedure in FIG. 7.
  • At [0102] 1008, it is determined if there are more entries in the source drawable's fill linked list. If so, then at 1010, the next entry's fill region is used to compute the fill portion of the drawn-to region for the corresponding fill color. Then at 1012, the fill portion is submitted to the destination drawable's fill linked list according to the procedure in FIG. 8. Then back to 1008 until there are no more entries in the fill linked list. At 1014, it is determined if there are more entries in the source drawable's tile linked list. If so, then at 1016, the next entry's tile region is used to compute the repeating image portion of the drawn-to region for the corresponding tile image. Then at 1018, the tile portion is submitted to the destination drawable's tile linked list according to the procedure in FIG. 9. This is repeated until there are no more entries in the tile linked list.
  • At [0103] 1020, the source drawable's miscellaneous region is used to compute the miscellaneous portion of the drawn-to region. At 1022, the miscellaneous portion is added to the destination drawable's miscellaneous region.
  • Referring back to FIG. 5B, 546 may also be performed if it is determined if that the command is a CopyWindow at [0104] 548 (which also represents case 4 of the commands requiring special treatment as described above), a RestoreArea from a backing store pixmap to a tagged window at 550 (which represents case 5 of the commands requiring special treatment as described above), or a SaveAreas from a tagged window to a backing store pixmap at 552 (which represents case 6 of the commands requiring special treatment as described above).
  • At [0105] 554, it is determined if the command is a PaintWindowBackground to a tagged window. If so, then at 556 it is determined if the window has backgroundState equal to BackgroundPixmap. If so, then this represents case 1 of the commands requiring special treatment as described above, and at 562 the drawn-to region is computed. Then at 564, the drawn-to region is subtracted from the miscellaneous region. At 566, the drawn-to region may be subtracted from the fill and tile linked lists according to the procedure outlined in FIG. 7 and the corresponding text. Then, at 568, the repeating image region may be submitted to the tile linked list according to the procedure outlined in FIG. 9 and the corresponding text. 562-568 may also be performed if at 558 it is determined that the command is a PaintWindowBorder for a tagged window, and at 560 it is determined that the window has borderIsPixel equal to false, which represents case 2 of the command requiring special treatment as described above.
  • At [0106] 570, it is determined if the command draws to a tagged drawable. If so, then at 572, the drawn-to region is computed. Then at 574, the drawn-to region is submitted to the miscellaneous region according to the procedure in FIG. 11 or FIG. 12. FIG. 11 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “underlay” embodiment. At 1100, if the region tags include an RGB region, the destination drawable's RGB region is first subtracted from the submitted region. This is included for the convenience of implementers using an RGB region as a component of the region tags. If an RGB region is not being used, 1100 need not be executed. Then at 1102, the remainder is added to the destination drawable's miscellaneous region. FIG. 12 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “disjoint” embodiment. At 1200, the submitted region is first subtracted from the fill and tile linked lists according to the procedure of FIG. 7. Then at 1202, in implementations including an RGB region, the destination drawable's RGB region is subtracted from the submitted region. Then the remainder is added to the destination drawable's miscellaneous region at 1204.
  • FIG. 13 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. In this embodiment, a certain class of image data (e.g., tile regions) is singled out for special compression, using a compression mode appropriate for the class of image data. A set of one or more drawing commands (e.g. PaintWindowBackground for a window with borderIsPixel==FALSE) may be identified as creating sub-images of this class. [0107]
  • At [0108] 1300, a tile command executed to create a portion of the graphics image in a destination drawable is recorded. Command recording may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable. At 1302, the tile command may be used to update a data structure associated with the destination drawable with information. This may include using information about the region drawn to by the drawing command to update the data structure. This data structure may be associated with the destination drawable. A private data field, such as a devPrivates field, may be used to attach the data structure to the destination drawable. While use of the X Windows devPrivates facility would be one method of associating a data structure with the drawable, there are other possible choices for the method of associating the two.
  • The information in the data structure may enable determination of what portion of the image contains sub-images of the specified image class. One example data structure would be a linked list identifying regions of repeating images. In one embodiment of the present invention, an X region resource is used to store the region of the destination drawable which contains sub-images of the specified image class. At [0109] 1304, one or more sub-images of the graphics image may be compressed using a compression scheme, each of the sub-images having a location, the location and the compression scheme for each of the sub-images based on the information in the data structure. It should be noted that the compressor does not necessarily need to know exactly which drawing commands were executed. It only needs to know what sub-images of the image should be compressed with a given compression scheme.
  • The compressor may use the information in the data structure to find the location within the image of sub-images of the specified image class. The compressor may compress these sub-images using a compression scheme selected for the specified image class. If the region covered by the sub-images is represented as a set of one or more rectangles, as is the case for the X region resource, each rectangle may be compressed and sent separately. [0110]
  • FIG. 14 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention. At [0111] 1400, a tile command executed to create a portion of the graphics image in a destination drawable may be detected. Command detection may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable. At 1402, a drawn-to region may be computed from the tile command. At 1404, the drawn-to region may be subtracted from a miscellaneous region. At 1406, the drawn-to region may be subtracted from one or more fill and/or tile regions. At 1408, the drawn-to region may be added to one of one or more tile regions, each of the tile regions corresponding to a repeating image. At 1410, the tile image (i.e., the upper width*height pixels in the rectangle) may be compressed. At 1412, the tile image may be transmitted for display or storage. At 1414, a sequence of local copy commands may be transmitted to repeat the tile image.
  • FIG. 15 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. In this embodiment, a certain class of image data (e.g., tile regions) is singled out for special compression, using a compression mode appropriate for the class of image data. A set of one or more drawing commands (e.g. PaintWindowBackground for a window with borderIsPixel==FALSE) may be identified as creating sub-images of this class. [0112]
  • A [0113] tile command recorder 1500 may record a tile command executed to create a portion of the graphics image in a destination drawable. Command recording may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable. A data structure updater 1502 coupled to the tile command recorder 1500 may use the tile command to update a data structure associated with the destination drawable with information. This may include using information about the region drawn to by the drawing command to update the data structure. This data structure may be associated with the destination drawable. A private data field, such as a devPrivates field, may be used to attach the data structure to the destination drawable. While use of the X Windows devPrivates facility would be one method of associating a data structure with the drawable, there are other possible choices for the method of associating the two.
  • The information in the data structure may enable determination of what portion of the image contains sub-images of the specified image class. One example data structure would be a linked list identifying regions of repeating images. In one embodiment of the present invention, an X region resource is used to store the region of the destination drawable which contains sub-images of the specified image class. An [0114] image compressor 1504 coupled to the data structure updater 1502 may compress one or more sub-images of the graphics image using a compression scheme, each of the sub-images having a location, the location and the compression scheme for each of the sub-images based on the information in the data structure. It should be noted that the compressor does not necessarily need to know exactly which drawing commands were executed. It only needs to know what sub-images of the image should be compressed with a given compression scheme.
  • The compressor may use the information in the data structure to find the location within the image of sub-images of the specified image class. The compressor may compress these sub-images using a compression scheme selected for the specified image class. If the region covered by the sub-images is represented as a set of one or more rectangles, as is the case for the X region resource, each rectangle may be compressed and sent separately. [0115]
  • FIG. 16 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention. A [0116] tile command detector 1600 may detect a tile command executed to create a portion of the graphics image in a destination drawable. Command detection may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable. A drawn-to region computer 1602 coupled to the tile command detector 1600 may compute a drawn-to region from the tile command. A drawn-to region to miscellaneous region remover 1604 coupled to the tile command detector 1600 and to the drawn-to region computer 1600 may subtract the drawn-to region from a miscellaneous region. A drawn-to region to fill and tile region remover 1606 coupled to the tile command detector may subtract the drawn-to region from one or more fill and/or tile regions. A tile region adder 1608 coupled to the tile command detector 1600 and to the drawn-to region computer 1602 may add the drawn-to region to one of one or more tile regions, each of the tile regions corresponding to a repeating image. A memory or memories 1610 coupled to the tile region adder 1608, the drawn-to region to miscellaneous region remover 1604 and to the drawn-to region to fill and tile region remover 1606 may store the various regions. A tile image transmitter 1612 coupled to the memory 1610 may transmit the tile image for display or storage. A local copy command transmitter 1614 coupled to the tile image transmitter 1612 may transmit a sequence of local copy commands to repeat the tile image.
  • While embodiments and applications of this invention have been shown and described, it would be apparent to those skilled in the art having the benefit of this disclosure that many more modifications than mentioned above are possible without departing from the inventive concepts herein. The invention, therefore, is not to be restricted except in the spirit of the appended claims. [0117]

Claims (49)

What is claimed is:
1. A method for reducing the size of a graphics image in a computer system, comprising:
recording a tile command executed to create a portion of the graphics image in a destination drawable;
using said tile command to update a data structure associated with said destination drawable with information; and
compressing one or more sub-images of the graphics image using a compression scheme, each of said sub-images having a location, said location and said compression scheme for each of said sub-images chosen based on said information in said data structure.
2. The method of claim 1, wherein said data structure associated with said destination drawable identifies the portion of the destination drawable image that was created with a command in a specific class of drawing commands.
3. The method of claim 1, further comprising:
transmitting one or more sub-images of the graphics image created using said tile command, for display or storage; and
transmitting a sequence of local copy commands to repeat said one or more sub-images.
4. The method of claim 1, further comprising:
determining if a region compatible with one or more sub-images of the graphics image created using said tile command already exists on a display; and
utilizing said compatible region to display said one or more sub-images of the graphics image created using said tile command if a compatible region already exists on said display.
5. The method of claim 3, further comprising:
adding said one or more sub-images of the graphics image created using said tile command to a cache.
6. The method of claim 5, further comprising:
determining if an image compatible with one or more sub-images of the graphics image created using said tile command already exists in said cache; and
utilizing said compatible image to display said one or more sub-images of the graphics image created using said tile command if a compatible image already exists in said cache.
7. The method of claim 2, wherein said data structure associated with said destination drawable is a list of tile regions.
8. The method of claim 7, wherein said list of tile regions includes, for each tile region, the height and width of the tile image.
9. The method of claim 8, wherein said list of tile regions further includes, for each tile region, the origin point of the tile image.
10. The method of claim 9, wherein said list of tile regions further includes, for each tile region, a hash value for the tile image.
11. The method of claim 10, wherein said hash value is an MD5 hash value.
12. The method of claim 1, wherein said tile command is a command that creates a repeating image.
13. A method for reducing the size of a graphics image in a computer system, comprising:
detecting a tile command executed to create a portion of the graphics image in a destination drawable;
computing a drawn-to region from said tile command; and
adding the drawn-to region to one of one or more tile regions, each of said tile regions corresponding to a repeating image.
14. The method of claim 13, further comprising:
compressing said tile image;
transmitting said tile image for display or storage; and
transmitting a sequence of local copy commands to repeat said tile image.
15. The method of claim 13, further comprising:
determining if a region compatible with said tile image already exists on a display; and
utilizing said compatible region to display said tile image if a compatible region already exists on said display.
16. The method of claim 14, further comprising:
adding said tile image to a cache.
17. The method of claim 16, further comprising:
determining if an image compatible with said tile image already exists in said cache; and
utilizing said compatible image to display said tile image if a compatible image already exists in said cache.
18. The method of claim 13, further comprising subtracting the drawn-to region from one or more existing fill and/or tile images.
19. A method for reducing the size of a graphics image in a computer system, comprising:
detecting a tile command executed to create a portion of the graphics image in a pixmap;
computing a drawn-to region from said tile command;
subtracting the drawn-to region from one or more existing region tags; and
adding the drawn-to region to one of one or more tile regions, said tile region corresponding to a repeating image drawn by said tile command.
20. The method of claim 19, further comprising:
compressing said tile image;
transmitting said tile image for display or storage; and
transmitting a sequence of local copy commands to repeat said tile image.
21. An apparatus for reducing the size of a graphics image in a computer system, comprising:
a tile command recorder;
a data structure updater coupled to said tile command recorder; and
an image compressor coupled to said data structure updater.
22. An apparatus for reducing the size of a graphics image in a computer system, comprising:
a tile command detector;
a drawn-to region computer coupled to said tile command detector;
a tile region adder coupled to said drawn-to region computer; and
a memory coupled to said tile region adder, said drawn-to region computer, and said tile region adder.
23. The apparatus of claim 22, further comprising a drawn-to region to fill and tile region remover coupled to said tile command detector and to said memory.
24. The apparatus of claim 22, further comprising:
a tile image transmitter coupled to said memory; and
a local copy command transmitter coupled to said tile image transmitter.
25. An apparatus for reducing the size of a graphics image in a computer system, comprising:
a tile command detector;
a drawn-to region computer coupled to said tile command detector;
a tile region adder coupled to said tile command detector and to said drawn-to region computer;
a drawn-to region to fill and tile region remover coupled to said tile command detector and to said drawn-to region computer;
a drawn-to region to miscellaneous region remover coupled to said tile command detector and to said drawn-to region computer; and
a memory coupled to said tile region adder, to said drawn-to region to miscellaneous region remover, and to said drawn-to region to tile region remover.
26. The apparatus of claim 25, further comprising:
a tile image transmitter coupled to said memory; and
a local copy command transmitter coupled to said tile image transmitter.
27. An apparatus for reducing the size of a graphics image in a computer system, comprising:
means for recording a tile command executed to create a portion of the graphics image in a destination drawable;
means for using said tile command to update a data structure associated with said destination drawable with information; and
means for compressing one or more sub-images of the graphics image using a compression scheme, each of said sub-images having a location, said location and said compression scheme for each of said sub-images chosen based on said information in said data structure.
28. The apparatus of claim 27, wherein said data structure associated with said destination drawable identifies the portion of the destination drawable image that was created with a command in a specific class of drawing commands.
29. The apparatus of claim 27, further comprising:
means for transmitting one or more sub-images of the graphics image created using said tile command, for display or storage; and
means for transmitting a sequence of local copy commands to repeat said one or more sub-images.
30. The apparatus of claim 27, further comprising:
means for determining if a region compatible with one or more sub-images of the graphics image created using said tile command already exists on a display; and
means for utilizing said compatible region to display said one or more sub-images of the graphics image created using said tile command if a compatible region already exists on said display.
31. The apparatus of claim 29, further comprising:
means for adding said one or more sub-images of the graphics image created using said tile command to a cache.
32. The apparatus of claim 31, further comprising:
means for determining if an image compatible with one or more sub-images of the graphics image created using said tile command already exists in said cache; and
means for utilizing said compatible image to display said one or more sub-images of the graphics image created using said tile command if a compatible image already exists in said cache.
33. The apparatus of claim 28, wherein said data structure associated with said destination drawable is a list of tile regions.
34. The apparatus of claim 33, wherein said list of tile regions includes, for each tile region, the height and width of the tile image.
35. The apparatus of claim 34, wherein said list of tile regions further includes, for each tile regions, the origin point of the tile image.
36. The apparatus of claim 35, wherein said list of tile regions further includes, for each tile region, a hash value for the tile image.
37. The apparatus of claim 36, wherein said hash value is an MD5 hash value.
38. The apparatus of claim 27, wherein said tile command is a command that creates a repeating image.
39. An apparatus for reducing the size of a graphics image in a computer system, comprising:
means for detecting a tile command executed to create a portion of the graphics image in a destination drawable;
means for computing a drawn-to region from said tile command; and
means for adding the drawn-to region to one of one or more tile regions, each of said tile regions corresponding to a repeating image.
40. The apparatus of claim 39, further comprising:
means for compressing said tile image;
means for transmitting said tile image for display or storage; and
means for transmitting a sequence of local copy commands to repeat said tile image.
41. The apparatus of claim 39, further comprising:
means for determining if a region compatible with said tile image already exists on a display; and
means for utilizing said compatible region to display said tile image if a compatible region already exists on said display.
42. The apparatus of claim 40, further comprising:
means for adding said tile image to a cache.
43. The apparatus of claim 42, further comprising:
means for determining if an image compatible with said tile image already exists in said cache; and
means for utilizing said compatible image to display said tile image if a compatible region already exists in said cache.
44. The apparatus of claim 42, further comprising means for subtracting the drawn-to region from one or more existing fill and/or tile images.
45. An apparatus for reducing the size of a graphics image in a computer system, comprising:
means for detecting a tile command executed to create a portion of the graphics image in a pixmap;
means for computing a drawn-to region from said tile command;
means for subtracting the drawn-to region from one or more existing region tags; and
means for adding the drawn-to region to one of a tile image, said tile corresponding to a repeating image drawn by said tile command.
46. The apparatus of claim 45, further comprising:
means for compressing said tile image;
means for transmitting said tile image for display or storage; and
means for transmitting a sequence of local copy commands to repeat said tile image.
47. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method for reducing the size of a graphics image in a computer system, the method comprising:
recording a tile command executed to create a portion of the graphics image in a destination drawable;
using said tile command to update a data structure associated with said destination drawable with information; and
compressing one or more sub-images of the graphics image using a compression scheme, each of said sub-images having a location, said location and said compression scheme for each of said sub-images chosen based on said information in said data structure.
48. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method for reducing the size of a graphics image in a computer system, the method comprising:
detecting a tile command executed to create a portion of the graphics image in a destination drawable;
computing a drawn-to region from said tile command; and
adding the drawn-to region to one of one or more tile regions, each of said tile regions corresponding to a repeating image.
49. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method for reducing the size of a graphics image in a computer system, the method comprising:
detecting a tile command executed to create a portion of the graphics image in a pixmap;
computing a drawn-to region from said tile command;
subtracting the drawn-to region from one or more existing region tags; and
adding the drawn-to region to one of one or more tile regions, said tile region corresponding to a repeating image drawn by said tile command.
US10/385,759 2002-07-11 2003-03-10 Tagging repeating images for improved compression Abandoned US20040008214A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US10/385,759 US20040008214A1 (en) 2002-07-11 2003-03-10 Tagging repeating images for improved compression
US10/754,681 US7012612B1 (en) 2002-07-11 2004-01-09 Context dependent image caching
US11/290,926 US20060077207A1 (en) 2002-07-11 2005-11-29 Context dependent image caching

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US10/194,123 US20040008213A1 (en) 2002-07-11 2002-07-11 Tagging multicolor images for improved compression
US10/247,907 US20040008205A1 (en) 2002-07-11 2002-09-20 Tagging single-color images for improved compression
US10/385,759 US20040008214A1 (en) 2002-07-11 2003-03-10 Tagging repeating images for improved compression

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US10/247,907 Continuation-In-Part US20040008205A1 (en) 2002-07-11 2002-09-20 Tagging single-color images for improved compression

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US44954003A Continuation-In-Part 2002-07-11 2003-05-29
US62295603A Continuation-In-Part 2002-07-11 2003-07-18

Publications (1)

Publication Number Publication Date
US20040008214A1 true US20040008214A1 (en) 2004-01-15

Family

ID=35998803

Family Applications (3)

Application Number Title Priority Date Filing Date
US10/385,759 Abandoned US20040008214A1 (en) 2002-07-11 2003-03-10 Tagging repeating images for improved compression
US10/754,681 Expired - Lifetime US7012612B1 (en) 2002-07-11 2004-01-09 Context dependent image caching
US11/290,926 Abandoned US20060077207A1 (en) 2002-07-11 2005-11-29 Context dependent image caching

Family Applications After (2)

Application Number Title Priority Date Filing Date
US10/754,681 Expired - Lifetime US7012612B1 (en) 2002-07-11 2004-01-09 Context dependent image caching
US11/290,926 Abandoned US20060077207A1 (en) 2002-07-11 2005-11-29 Context dependent image caching

Country Status (1)

Country Link
US (3) US20040008214A1 (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090208110A1 (en) * 2008-02-14 2009-08-20 Microsoft Corporation Factoring repeated content within and among images
US7747086B1 (en) 2005-07-28 2010-06-29 Teradici Corporation Methods and apparatus for encoding a shared drawing memory
US7822278B1 (en) 2005-09-20 2010-10-26 Teradici Corporation Methods and apparatus for encoding a digital video signal
CN102253818A (en) * 2011-06-27 2011-11-23 深圳市茁壮网络股份有限公司 Background picture display method and device
US8107527B1 (en) 2005-07-28 2012-01-31 Teradici Corporation Progressive block encoding using region analysis
US8108577B1 (en) 2005-03-30 2012-01-31 Teradici Corporation Method and apparatus for providing a low-latency connection between a data processor and a remote graphical user interface over a network
US8345768B1 (en) 2005-07-28 2013-01-01 Teradici Corporation Progressive block encoding using region analysis
US8560753B1 (en) 2005-03-30 2013-10-15 Teradici Corporation Method and apparatus for remote input/output in a computer system
US8855414B1 (en) 2004-06-30 2014-10-07 Teradici Corporation Apparatus and method for encoding an image generated in part by graphical commands
CN104463818A (en) * 2013-09-25 2015-03-25 北大方正集团有限公司 Method and system for fast drawing background
US9026615B1 (en) 2011-09-22 2015-05-05 Teradici Corporation Method and apparatus for caching image data transmitted over a lossy network
WO2017060678A1 (en) * 2015-10-06 2017-04-13 Displaylink (Uk) Limited Managing display data
US20170148134A1 (en) * 2015-11-19 2017-05-25 Raydium Semiconductor Corporation Driving circuit and operating method thereof

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4589308B2 (en) * 2004-04-05 2010-12-01 パナソニック株式会社 Display screen management device
JP4642697B2 (en) * 2006-05-24 2011-03-02 Necディスプレイソリューションズ株式会社 Image display device having image cache memory
US8619877B2 (en) * 2007-10-11 2013-12-31 Microsoft Corporation Optimized key frame caching for remote interface rendering
US8027960B2 (en) * 2009-03-11 2011-09-27 International Business Machines Corporation Intelligent deletion of elements to maintain referential integrity of dynamically assembled components in a content management system
US20100332530A1 (en) * 2009-06-26 2010-12-30 Microsoft Corporation Islands of data
US8321533B2 (en) 2009-08-03 2012-11-27 Limelight Networks, Inc. Systems and methods thereto for acceleration of web pages access using next page optimization, caching and pre-fetching techniques
US8346784B1 (en) 2012-05-29 2013-01-01 Limelight Networks, Inc. Java script reductor
US8495171B1 (en) 2012-05-29 2013-07-23 Limelight Networks, Inc. Indiscriminate virtual containers for prioritized content-object distribution
US9058402B2 (en) * 2012-05-29 2015-06-16 Limelight Networks, Inc. Chronological-progression access prioritization
CN103390090B (en) * 2012-05-07 2017-12-22 腾讯科技(深圳)有限公司 The method and apparatus of image procossing
US9979960B2 (en) 2012-10-01 2018-05-22 Microsoft Technology Licensing, Llc Frame packing and unpacking between frames of chroma sampling formats with different chroma resolutions
US20140108941A1 (en) 2012-10-17 2014-04-17 Christopher Stephen Joel Method and Apparatus for Automatically Optimizing the Loading of Images in a Cloud-Based Proxy Service
US9098477B2 (en) 2013-05-15 2015-08-04 Cloudflare, Inc. Method and apparatus for automatically optimizing the loading of images in a cloud-based proxy service
US9015348B2 (en) 2013-07-19 2015-04-21 Limelight Networks, Inc. Dynamically selecting between acceleration techniques based on content request attributes
US10368080B2 (en) 2016-10-21 2019-07-30 Microsoft Technology Licensing, Llc Selective upsampling or refresh of chroma sample values

Citations (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4555775A (en) * 1982-10-07 1985-11-26 At&T Bell Laboratories Dynamic generation and overlaying of graphic windows for multiple active program storage areas
US4941193A (en) * 1987-10-02 1990-07-10 Iterated Systems, Inc. Methods and apparatus for image compression by iterated function system
US5020003A (en) * 1988-09-29 1991-05-28 At&T Bell Laboratories Graphics controller image creation
US5179651A (en) * 1988-11-08 1993-01-12 Massachusetts General Hospital Apparatus for retrieval and processing of selected archived images for display at workstation terminals
US5212770A (en) * 1989-12-06 1993-05-18 Eastman Kodak Company Data-handling and display system capable of supporting multiple application programs and output devices
US5241625A (en) * 1990-11-27 1993-08-31 Farallon Computing, Inc. Screen image sharing among heterogeneous computers
US5263136A (en) * 1991-04-30 1993-11-16 Optigraphics Corporation System for managing tiled images using multiple resolutions
US5392391A (en) * 1991-10-18 1995-02-21 Lsi Logic Corporation High performance graphics applications controller
US5440686A (en) * 1993-12-22 1995-08-08 International Business Machines Corporation Selecting a data unit candidate to be demoted to a backing store from a front store based upon thresholds individual to each of the data candidates
US5603034A (en) * 1992-09-08 1997-02-11 International Business Machines Corporation Graphical resource editor for software customization
US5757386A (en) * 1995-08-11 1998-05-26 International Business Machines Corporation Method and apparatus for virtualizing off-screen memory of a graphics engine
US5883640A (en) * 1996-08-15 1999-03-16 Hsieh; Paul Computing apparatus and operating method using string caching to improve graphics performance
US5917504A (en) * 1994-04-07 1999-06-29 Sony Corporation Image processing apparatus, switching between images having pixels of first and second numbers of bits
US5936616A (en) * 1996-08-07 1999-08-10 Microsoft Corporation Method and system for accessing and displaying a compressed display image in a computer system
US5986661A (en) * 1991-10-10 1999-11-16 Hewlett-Packard Company Graphics output system with bounded updating
US6006013A (en) * 1994-05-18 1999-12-21 Xerox Corporation Object optimized printing system and method
US6031550A (en) * 1997-11-12 2000-02-29 Cirrus Logic, Inc. Pixel data X striping in a graphics processor
US6049330A (en) * 1997-08-28 2000-04-11 Oak Technology, Inc. Method and apparatus for optimizing storage of compressed images in memory
US6064771A (en) * 1997-06-23 2000-05-16 Real-Time Geometry Corp. System and method for asynchronous, adaptive moving picture compression, and decompression
US6097388A (en) * 1995-08-22 2000-08-01 International Business Machines Corporation Method for managing non-rectangular windows in a raster display
US6300953B1 (en) * 1998-10-15 2001-10-09 Nvidia Apparatus and method for grouping texture cache requests
US6304928B1 (en) * 1995-07-05 2001-10-16 Microsoft Corporation Compressing/decompressing bitmap by performing exclusive- or operation setting differential encoding of first and previous row therewith outputting run-length encoding of row
US20020021455A1 (en) * 2000-08-09 2002-02-21 Hiroshi Ishii Image processing device, image processing method and image forming apparatus
US20020035596A1 (en) * 2000-05-26 2002-03-21 Ruiguo Yang Remote control of a client's off-screen surface
US6366289B1 (en) * 1998-07-17 2002-04-02 Microsoft Corporation Method and system for managing a display image in compressed and uncompressed blocks
US20020180757A1 (en) * 2001-04-04 2002-12-05 Herbert Duerr Optimized access for drawing operations
US20030028728A1 (en) * 2001-07-31 2003-02-06 Mitsubishi Denki Kabushiki Kaisha Cache memory control device
US20030191859A1 (en) * 2002-04-05 2003-10-09 Ramsey Paul R. Fast remote display of images using compressed XPutImage
US6633299B1 (en) * 2000-01-10 2003-10-14 Intel Corporation Method and apparatus for implementing smart allocation policies for a small frame buffer cache serving 3D and 2D streams
US6664976B2 (en) * 2001-04-18 2003-12-16 Digimarc Corporation Image management system and methods using digital watermarks
US20040002327A1 (en) * 2002-06-28 2004-01-01 Kabushiki Kaisha Toshiba Transmission apparatus, method and computer program product
US20040010543A1 (en) * 2002-07-15 2004-01-15 Steven Grobman Cached resource validation without source server contact during validation
US20040059877A1 (en) * 2002-09-20 2004-03-25 International Business Machines Corporation Method and apparatus for implementing cache state as history of read/write shared data
US6721852B2 (en) * 2001-10-17 2004-04-13 Sun Microsystems, Inc. Computer system employing multiple board sets and coherence schemes
US6751356B2 (en) * 2000-02-07 2004-06-15 Canon Kabushiki Kaisha Image processing apparatus and method

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5283667A (en) * 1989-12-18 1994-02-01 Ricoh Company, Ltd. Electronic filing apparatus provided with a multiple processing function when image data is displayed
JPH09284509A (en) * 1996-04-10 1997-10-31 Canon Inc Picture processor
JPH11175287A (en) * 1997-12-16 1999-07-02 Canon Inc Data processor, data processing method for the same and storage medium storing computer readable program
US6415058B2 (en) * 1998-10-27 2002-07-02 Hewlett-Packard Company System for compression of digital images comprising low detail areas
US6963668B2 (en) * 1998-11-13 2005-11-08 Lightsurf Technologies, Inc. Method and system for fast image correction
US6549210B1 (en) * 1999-02-03 2003-04-15 Ati Technologies Inc. Method and apparatus for cache index hashing
US6853466B1 (en) * 1999-11-24 2005-02-08 Canon Kabushiki Kaisha Image processing apparatus and method
JP2001222395A (en) * 2000-02-10 2001-08-17 Canon Inc Information processor, information processing method and storage medium in which printer driver program is stored
US20020093506A1 (en) * 2001-01-16 2002-07-18 Hobson Jay A. Apparatus and method for storing and retrieving images for transmission to an output device
US6879725B2 (en) * 2001-01-26 2005-04-12 International Business Machine Corporation Method, system, and program for decoding a section from compressed data
US20020141657A1 (en) * 2001-03-30 2002-10-03 Robert Novak System and method for a software steerable web Camera
US7397962B2 (en) * 2001-10-25 2008-07-08 Infoprint Solutions Company, Llc Automatic method of identifying image subregions for reuse during datastream transmission
US6862028B2 (en) * 2002-02-14 2005-03-01 Intel Corporation Bin pointer and state caching apparatus and method

Patent Citations (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4555775B1 (en) * 1982-10-07 1995-12-05 Bell Telephone Labor Inc Dynamic generation and overlaying of graphic windows for multiple active program storage areas
US4555775A (en) * 1982-10-07 1985-11-26 At&T Bell Laboratories Dynamic generation and overlaying of graphic windows for multiple active program storage areas
US4941193A (en) * 1987-10-02 1990-07-10 Iterated Systems, Inc. Methods and apparatus for image compression by iterated function system
US5020003A (en) * 1988-09-29 1991-05-28 At&T Bell Laboratories Graphics controller image creation
US5179651A (en) * 1988-11-08 1993-01-12 Massachusetts General Hospital Apparatus for retrieval and processing of selected archived images for display at workstation terminals
US5212770A (en) * 1989-12-06 1993-05-18 Eastman Kodak Company Data-handling and display system capable of supporting multiple application programs and output devices
US5241625A (en) * 1990-11-27 1993-08-31 Farallon Computing, Inc. Screen image sharing among heterogeneous computers
US5263136A (en) * 1991-04-30 1993-11-16 Optigraphics Corporation System for managing tiled images using multiple resolutions
US5986661A (en) * 1991-10-10 1999-11-16 Hewlett-Packard Company Graphics output system with bounded updating
US5392391A (en) * 1991-10-18 1995-02-21 Lsi Logic Corporation High performance graphics applications controller
US5603034A (en) * 1992-09-08 1997-02-11 International Business Machines Corporation Graphical resource editor for software customization
US5440686A (en) * 1993-12-22 1995-08-08 International Business Machines Corporation Selecting a data unit candidate to be demoted to a backing store from a front store based upon thresholds individual to each of the data candidates
US5917504A (en) * 1994-04-07 1999-06-29 Sony Corporation Image processing apparatus, switching between images having pixels of first and second numbers of bits
US6006013A (en) * 1994-05-18 1999-12-21 Xerox Corporation Object optimized printing system and method
US6304928B1 (en) * 1995-07-05 2001-10-16 Microsoft Corporation Compressing/decompressing bitmap by performing exclusive- or operation setting differential encoding of first and previous row therewith outputting run-length encoding of row
US5757386A (en) * 1995-08-11 1998-05-26 International Business Machines Corporation Method and apparatus for virtualizing off-screen memory of a graphics engine
US6097388A (en) * 1995-08-22 2000-08-01 International Business Machines Corporation Method for managing non-rectangular windows in a raster display
US5936616A (en) * 1996-08-07 1999-08-10 Microsoft Corporation Method and system for accessing and displaying a compressed display image in a computer system
US5883640A (en) * 1996-08-15 1999-03-16 Hsieh; Paul Computing apparatus and operating method using string caching to improve graphics performance
US6064771A (en) * 1997-06-23 2000-05-16 Real-Time Geometry Corp. System and method for asynchronous, adaptive moving picture compression, and decompression
US6049330A (en) * 1997-08-28 2000-04-11 Oak Technology, Inc. Method and apparatus for optimizing storage of compressed images in memory
US6031550A (en) * 1997-11-12 2000-02-29 Cirrus Logic, Inc. Pixel data X striping in a graphics processor
US6366289B1 (en) * 1998-07-17 2002-04-02 Microsoft Corporation Method and system for managing a display image in compressed and uncompressed blocks
US6300953B1 (en) * 1998-10-15 2001-10-09 Nvidia Apparatus and method for grouping texture cache requests
US6633299B1 (en) * 2000-01-10 2003-10-14 Intel Corporation Method and apparatus for implementing smart allocation policies for a small frame buffer cache serving 3D and 2D streams
US6751356B2 (en) * 2000-02-07 2004-06-15 Canon Kabushiki Kaisha Image processing apparatus and method
US20020035596A1 (en) * 2000-05-26 2002-03-21 Ruiguo Yang Remote control of a client's off-screen surface
US20020021455A1 (en) * 2000-08-09 2002-02-21 Hiroshi Ishii Image processing device, image processing method and image forming apparatus
US20020180757A1 (en) * 2001-04-04 2002-12-05 Herbert Duerr Optimized access for drawing operations
US6664976B2 (en) * 2001-04-18 2003-12-16 Digimarc Corporation Image management system and methods using digital watermarks
US20030028728A1 (en) * 2001-07-31 2003-02-06 Mitsubishi Denki Kabushiki Kaisha Cache memory control device
US6721852B2 (en) * 2001-10-17 2004-04-13 Sun Microsystems, Inc. Computer system employing multiple board sets and coherence schemes
US20030191859A1 (en) * 2002-04-05 2003-10-09 Ramsey Paul R. Fast remote display of images using compressed XPutImage
US20040002327A1 (en) * 2002-06-28 2004-01-01 Kabushiki Kaisha Toshiba Transmission apparatus, method and computer program product
US20040010543A1 (en) * 2002-07-15 2004-01-15 Steven Grobman Cached resource validation without source server contact during validation
US20040059877A1 (en) * 2002-09-20 2004-03-25 International Business Machines Corporation Method and apparatus for implementing cache state as history of read/write shared data

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8855414B1 (en) 2004-06-30 2014-10-07 Teradici Corporation Apparatus and method for encoding an image generated in part by graphical commands
US8108577B1 (en) 2005-03-30 2012-01-31 Teradici Corporation Method and apparatus for providing a low-latency connection between a data processor and a remote graphical user interface over a network
US8874812B1 (en) 2005-03-30 2014-10-28 Teradici Corporation Method and apparatus for remote input/output in a computer system
US8560753B1 (en) 2005-03-30 2013-10-15 Teradici Corporation Method and apparatus for remote input/output in a computer system
US8345768B1 (en) 2005-07-28 2013-01-01 Teradici Corporation Progressive block encoding using region analysis
US8731314B1 (en) 2005-07-28 2014-05-20 Teradici Corporation Methods for encoding an image
US8107527B1 (en) 2005-07-28 2012-01-31 Teradici Corporation Progressive block encoding using region analysis
US9020045B1 (en) 2005-07-28 2015-04-28 Teradici Corporation Progressive block encoding using region analysis
US7747086B1 (en) 2005-07-28 2010-06-29 Teradici Corporation Methods and apparatus for encoding a shared drawing memory
US8315468B1 (en) 2005-07-28 2012-11-20 Teradici Corporation Apparatus for block-selected encoding of a digital video signal
US8077989B1 (en) 2005-07-28 2011-12-13 Teradici Corporation Methods and apparatus for encoding a digital video signal
US7916956B1 (en) 2005-07-28 2011-03-29 Teradici Corporation Methods and apparatus for encoding a shared drawing memory
US7822278B1 (en) 2005-09-20 2010-10-26 Teradici Corporation Methods and apparatus for encoding a digital video signal
US20090208110A1 (en) * 2008-02-14 2009-08-20 Microsoft Corporation Factoring repeated content within and among images
US8204338B2 (en) 2008-02-14 2012-06-19 Microsoft Corporation Factoring repeated content within and among images
CN102253818A (en) * 2011-06-27 2011-11-23 深圳市茁壮网络股份有限公司 Background picture display method and device
US9026615B1 (en) 2011-09-22 2015-05-05 Teradici Corporation Method and apparatus for caching image data transmitted over a lossy network
CN104463818A (en) * 2013-09-25 2015-03-25 北大方正集团有限公司 Method and system for fast drawing background
WO2017060678A1 (en) * 2015-10-06 2017-04-13 Displaylink (Uk) Limited Managing display data
US10545868B2 (en) 2015-10-06 2020-01-28 Displaylink (Uk) Limited Managing display data
US20170148134A1 (en) * 2015-11-19 2017-05-25 Raydium Semiconductor Corporation Driving circuit and operating method thereof

Also Published As

Publication number Publication date
US20060077207A1 (en) 2006-04-13
US7012612B1 (en) 2006-03-14

Similar Documents

Publication Publication Date Title
US20040008214A1 (en) Tagging repeating images for improved compression
US5388201A (en) Method and apparatus for providing multiple bit depth windows
US20040010622A1 (en) Method and system for buffering image updates in a remote application
JP5264895B2 (en) Display remoting in bitmap format
US5430465A (en) Apparatus and method for managing the assignment of display attribute identification values and multiple hardware color look-up tables
US5757386A (en) Method and apparatus for virtualizing off-screen memory of a graphics engine
US7944451B2 (en) Providing pixels from an update buffer
US20090096810A1 (en) Method for selectively remoting windows
US9875519B2 (en) Overlap aware reordering of rendering operations for efficiency
US6226017B1 (en) Methods and apparatus for improving read/modify/write operations
US6483515B1 (en) Method and apparatus for displaying data patterns in information systems
US20150243257A1 (en) Cross-Platform Rendering Engine
JP2007179554A (en) System and method for duplicating display screen
US20040008205A1 (en) Tagging single-color images for improved compression
US10831985B2 (en) Processing duplicate multimedia content
US6075532A (en) Efficient redrawing of animated windows
US20120127185A1 (en) System and method for an optimized on-the-fly table creation algorithm
US20040008212A1 (en) Reshaping irregularly-shaped images for improved compression
US6429881B1 (en) Method and system for transitioning graphic elements of a network interface description document
US20040008213A1 (en) Tagging multicolor images for improved compression
KR920006746B1 (en) Depth buffer clipping for window management
US7046250B1 (en) Caching fonts for improved bandwidth of transmitted text
JP4229702B2 (en) Local improvement of display information
CN115934383B (en) Multi-graphics card rendering method under Wayland synthesizer
US20080084426A1 (en) Off-screen buffering management device and method

Legal Events

Date Code Title Description
AS Assignment

Owner name: SUN MICROSYSTEMS, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:O'NEILL, THOMAS G.;SLOTT, JORDAN M.;REEL/FRAME:013866/0070;SIGNING DATES FROM 20030228 TO 20030303

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION