SYSTEM AND METHOD FOR CONTROLLING PANNING AND SCROLLING AREA OF DISPLAY
IMAGE Technical Field
This invention relates to the field of digital device interfaces to digital data. Background Art
The recent shift in the consumer electronics industry from an emphasis on analog technology to a preference for digital technology is largely based on the fact that the former generally limits the user to a role of a passive recipient of information, while the latter is interactive and allows the user to control what, when and how he or she receives and manipulates certain information. This shift in focus has resulted in the increasingly widespread use of graphical information and the development of user interfaces that effectively display the desired graphical information. One particular development is the use of digital bitmaps that portray street atlases, nautical charts, political and physical atlases and other mapping information. These bitmaps, many of them developed at a global scale, far exceed the display constraints of conventional personal computer (PC) monitors or other display devices. Another development is the use of computer-aided design and drafting (CADD) and other graphical applications that portray engineering or architectural designs and schematics and other graphical images. These bitmaps may also far exceed the display constraints of conventional devices. Another particular development is the use of graphical information in portable digital devices. A common digital device is generically referred to as a "personal digital assistant" (PDA). PDAs are handheld electronic devices that have portable design features including a small display screen, which, in many embodiments, is also utilized as a touch sensitive pointing device and data input device that can also be used to input keystroke data. PDAs are designed to capture, store, and display information originating from various digital sources. Accordingly, they may be connected directly to a PC or other PDAs with a "cradle" type data linkage, with an infrared link, with an modem type module such as a telephone module or telephone line adapter, including a wireless cellular phone module, or an Ethernet module or Ethernet adapter, including a wireless Ethernet module. PDAs are often described as an electronic alternative to the filofax; however, device such as handheld computers, electronic books, electronic note takers, personal computers, display telephones and personal communicators may also be embodied as PDAs. Regardless of their embodiment or specialized functions, PDA's primary functions are to capture and store information, search the stored information efficiently, access and display the results of the search, update the information and reconcile the information with data stored on a PC. Clearly, the trade off for decreased size and increased portability is a decrease in the capabilities that can be built into a PDA. For example, PDAs have a very small display area that typically is much smaller than the stored document being viewed by the user. It is therefore necessary for PDAs to be equipped with a convenient scrolling mechanism, so that the user is able to view the entire document by scrolling the display up and down or side to side. Most PDAs are equipped with cursor keys for this purpose, with a user being required to press the appropriate key(s) to scroll the screen in the desired direction until the desired portion of the document is displayed on the screen. Alternatively, scroll bars, which are common features of most of the graphical user interfaces (GUIs) available for use on PCs, may be used to manipulate and scroll the display. PCs that are utilized to display large bitmaps such as street atlases, architectural designs and other graphical images (described above) have a similar characteristic of a smaller size display area relative to a larger size of the entire graphical information that is being displayed. Most PC applications, such as street atlas viewers, have scroll bars, pointer buttons, or curser keys that may be used to manipulate and scroll the display. Another scrolling technique used by
both PC applications and PDAs is the "grab screen" technique. In this technique, the viewer uses the pointing device to press on the display area of the screen and "pull" the bitmap, thus moving the screen in the desired direction until the desired portion of the bitmap is displayed on the screen. Yet another scrolling technique, commonly referred to as "panning", is to establish a new center point (with a curser or other marker) within the display area and re-center the display in accordance with the new center point. This technique is used in conjunction with the zoom function to scroll beyond the boundaries of the current display area. Typically, the user zooms out to gain a larger display area, re-centers the image, and zooms back to the previous zoom level to complete the scrolling function.
One problem with the above scrolling techniques is that, in some implementations, they only enable scrolling in either an X or a Y direction, which makes moving about in a large image more difficult and time consuming than would be the case if nonrectilinear scrolling were supported. The grab screen technique enables non-linear scrolling but, like the other implementations, the user lacks the ability to instantly "jump" to the desired location. Accordingly, extensive scrolling of the image area can be cumbersome, time consuming, and therefore inconvenient to the user. Another major problem is that the user, while scrolling, lacks navigational information of the display area relative to the larger image. Some applications offer a smaller size bitmap representation of the larger image with a center point or superimposed rectangle to represent the location of the current display area. A problem with this solution is that the smaller bitmap takes up a portion of the display area, this being a tradeoff for the smaller display area and, in PDA embodiments, a major tradeoff for the very small display area. Another disadvantage is that much of the detail of the larger image is no longer visible in the smaller bitmap representation. Yet another disadvantage is that a very large bitmap, such as a global atlas, lacks borders that can be meaningfully represented in the smaller size bitmap representation. Another problem is that the navigational information is only available after the scrolling inputs are made. Thus, the user lacks navigational or orientation information prior to inputting the scrolling action. The major problem with the panning technique described above is that when the user zooms out, the larger display area necessarily lacks the image detail that was represented at the previous zoom level. Thus, the user is required to speculate as to the location of image features that are no longer visible at the zoom out level and may not display the desired feature when zooming back to the previous level after re-centering the image. Each of the scrolling and panning techniques are, for reasons of the problems they create, counterintuitive, i.e., the movement of the curser keys or the scroll bars or use of the grab screen technique do not intuitively correspond or instantly correlate to the desired location of the display of a larger image, or do not intuitively correspond to the resulting movement of the display.
Another problem is that the above techniques enable scrolling in only the X or Y directions, which makes moving about in a large document more difficult and time consuming than would be the case if nonrectilinear scrolling were supported. For example, assuming the document is organized in columns, similar to a newspaper, a user will likely find it more convenient, as well as more intuitive, to scroll directly to the top of the next column than to scroll over to the next column and then up to the top of the column, or vice versa.
U.S. patent 5,526,481 issued to Parks et al. on June 11, 1996, discloses a display scrolling system for personal digital assistant (PDA). This system includes a display screen disposed on a top surface of the PDA and a mouse integrated into the bottom surface of the PDA. Documents to be displayed on the screen are stored in a memory of the PDA. The PDA is placed on a work surface such that the mouse is manipulated by rolling the PDA across the surface. The work surface is regarded by the PDA as a virtual display of the document to be displayed on the display screen. As the PDA is rolled across the surface, the mouse generates translation information to a
memory controller, which determines the location of the PDA with respect to the virtual document and generates viewport coordinates, which define a portion of the document to be displayed on the display screen. The viewport coordinates are input to a memory controller, which generates memory addresses to the memory indicating the location in the memory of the document portion defined by the viewport coordinates. The addressed document portion is output from the memory to the display screen for display. In this manner, the display screen serves as a window or viewport through which portions of a document corresponding to the virtual display may be viewed.
Although the system described in the '481 patent presents a scrolling mechanism for use with a PDA which is intuitive and enables a user to scroll in nonrectilinear directions, this system requires the use of a mouse, requiring a new piece of hardware to be used with existing PDAs. It is also not applicable to the scrolling problems of PC applications described above given that such a pointing device is not compatible with a conventional PC display device. Furthermore, the typical PC pointing device (i.e. mouse) cannot be utilized in the same manner as the hardware device of this system. The reason is that the X and Y coordinates of this system's scrolling mechanism are directly inputted into the display centering functions of the system without the use of a graphical display pointer or an activation (i.e. "click") button. Thus, if a PC pointing device is utilized as the equivalent hardware device of this system, it will not be able to complete the display pointer and activation button operations of a general PC pointing device.
Therefore, what is needed is a scrolling mechanism for use with a PDA which is completely intuitive and which enables a user to scroll in nonrectilinear directions, but does not require the use of a mouse or other hardware not already included on every PDA apparatus known in then art. Disclosure of the Invention
The present invention is a method for viewing, on a display, an image stored on a digital device. The method includes the steps of first, detecting a selection of a region of the display by a pointer device then computing newX and newY coordinates of selected region. Next, comparing the newX and newY coordinates on the display with corresponding region of bitmap of the image and determining if the newX and newY coordinates are within a threshold range. If the newX and newY coordinates are within a threshold range, then shifting the coordinates by a predetermined value and comparing the newX and newY coordinates with currentX and currentY coordinates stored in memory. If the newX and newY coordinates are different from currentX and currentY coordinates then updating the image on the display to the section of the image bitmap corresponding with the newX and newY coordinates and storing the newX and newY coordinates in memory as currentX and currentY coordinates.
Such a method may include one or more of the following features. Where the pointer device is a stylus, where the pointer device is a mouse, where the digital device is a personal computer, and where the digital device is a personal digital assistant (PDA).
The present invention also includes a method for scrolling, on a display, an image stored on a digital device. The method includes first detecting a scrolling selection on the display by a pointer device. While the scrolling selection continues, at predetermined intervals, computing newX and newY coordinates of current selected region, next, comparing the newX and newY coordinates on the display with corresponding region of bitmap of the image, then determining if the newX and newY coordinates are within a threshold range, next, if the newX and newY coordinates are within a threshold range. Followed by shift the coordinates by a predetermined value and then comparing the newX and newY coordinates with currentX and currentY coordinates stored in memory. If the newX and newY coordinates are different from currentX and currentY coordinates then updating
the image on the display to the section of the image bitmap corresponding with the newX and newY coordinates.
Finally, storing the newX and newY coordinates in memory as currentX and currentY coordinates. Such a method may include one or more of the following features. Where the pointer device is a stylus, where the pointer device is a mouse, where the digital device is a personal computer, and where the digital device is a personal digital assistant (PDA).
These aspects of the invention are not meant to be exclusive and other features, aspects, and advantages of the present invention will be readily apparent to those of ordinary skill in the art when read in conjunction with the appended claims and accompanying drawings. Brief Description of Drawings FIG. 1 is a flow diagram representing the preferred embodiment of the process of translating a pointer device touch to a transmission of the desired data on a personal digital assistant (PDA) of the present invention.
FIG. 2 is a flow diagram of the preferred embodiment of the MainFormHandleEvent function of the present invention. Modes for Carrying out the Invention The embodiments of the invention described herein are implemented as logical operations in a digital device system. The logical operations of the present invention are implemented (1) as a sequence of digital device implemented steps running on the digital device and (2) as interconnected digital modules within the computer system. The implementation is a matter of choice dependent on the performance requirements of the digital device system implementing the invention. Accordingly, the logical operations making up the embodiments of the invention described herein are referred to variously as operations, steps, or modules. The function and process names, as well as the variable names, are used only for description purposes. The preferred embodiment of this invention is not limited to these names.
The present invention is a method for viewing bitmap data with a smaller display area on a PDA with a stylus and touch screen already incorporated into the PDA, PC or alternate digital device with a conventional pointing device (i.e. mouse) allowing for completely intuitive and nonrectilinear directional viewing or instantaneous panning to the desired portion of the bitmap image while maintaining orientation of the location of the current viewing area or intended panning area of the larger bitmap. Although the logical sequences are described for operation on a PDA, through the use of the stylus and touch screen already incorporated into the PDA, they are equally applicable to operations on a PC and PC display device through the use of a conventional mouse or other PC pointing device and activation button. One major distinction is that the PDA embodiment of the present invention acquires the X and Y coordinates from the stylus contact with the touch screen. In the PC embodiment of the present invention, the X and Y coordinates are acquired from the combination of the mouse pointer location on the display area and pressing the activation (i.e. mouse "click") button. Another major distinction is that the display screen size and the large image size for the present invention any be any size specification providing they are in the proportions that enable the basic functionality of the logical processes described herein. The present invention is described herein with reference to the PDA embodiment but may also describe the PC embodiment or any other digital device embodiment with the above noted distinctions. The present invention allows the viewer to quickly view (and in some instances instantly view) the desired part of an image, while maintaining the viewer's sense of orientation for both current views and intended panning and scrolling views. It does this intuitively and without requiring additional visual aids or other hardware components. The present invention works with rectangular images, having the same proportions, or different proportions of the
display screen that is being viewed. The present invention also works with very large (i.e. "global") images or nonrectangular images (such as circular, cylindrical or spherical images) by imposing a rectangular viewing border over the image area that exceeds the display area of the viewer.
When the image file is first opened, the PDA presents a section of the original image. The viewer identifies the general area of the image they wish to see. The viewer then taps the stylus on the touch screen of the PDA over the desired region of the touch screen corresponding to the area of the image in which the user desires to view. Each point on the PDA touch screen corresponds to an area of the original image. Once the user touches the PDA screen, the display moves to the selected part of the original image. For example, if the viewer touches the top right corner of the touch screen, this corresponds to the top right corner of the original image. Although the top right corner of the original image may not be presently displayed, the present invention updates the display to display the top right corner of the original image. The present invention additionally provides motion and zoom functions simultaneously. In other embodiments, the viewer selects the desired part of the image with a mouse, or other pointing device, and the image is displayed on a personal computer or other digital device.
Referring first to FIG. 1, the preferred embodiment of the process of translating a screen touch to a transmission of the desired data onto the screen of a personal digital assistant (PDA) of the present invention is shown. In alternate embodiments, pressing the activation button of the pointing device (i.e. mouse click) when the pointer display is located over the display screen is the equivalent operation of the PDA screen touch function. In yet another embodiment, a smaller image may be used that represents the full size image area of the larger bitmap, however, this smaller image is unnecessary and may be undesirable given that the smaller bitmap takes up a portion of the display area, this being a tradeoff for the smaller display area and, in PDA embodiments, a major tradeoff for the very small display area. Once the current invention is initiated on a PDA, the function MainFormHandleEvent begins when a stylus touch is detected, in step 10. Although in the preferred embodiment, a stylus is used to view and navigate about the PDA screen, in other embodiments, any device may be used which activates the touch screen, including, but not limited to, any hard pointy object, a pen, a pencil, or a human finger. Any means of selecting a location on the touch screen may be used to act as the stylus touch. Next, in step 12, the MainFormHandleEvent function determines the newX and newY variables. The PDA reads the X,Y location on the display, based on where the stylus touches the screen. The variables newX and newY are the positions in X,Y coordinates that correspond to the position in the full size image relative to the stylus position. Next, in step 14, the newX and newY values are converted to negative numbers. The full size image coordinates are always negative values. The next two steps deal with the possibility that the X,Y coordinates are too close to either the right hand or bottom border of the image. Since the largest "chunk" that can be displayed on a PDA is 160 x 160 pixels, the newX has to be at least 160 pixels away from the right hand border and the newY has to be at least 160 pixels above the bottom border. In alternate embodiments, this section is the size of the display of each respective embodiment. In step 16, the function determines if newX and/or newY are too close to their respective borders. If step 16 determines the newX and/or newY are too close to their respective borders, then, in step 18, the
MainFormHandleEvent function will shift the coordinates in order to display a full 160 x 160 "chunk" of the image. Next, in step 20, the function determines whether the X,Y coordinates have changed. The variables placeX (or current X) and placeY (or currentY) store the last set of X,Y coordinates known to the device. If newX is different than placeX or if newY is different than placeY, then the viewable "chunk" has changed. If the viewable "chunk" has changed, the function proceeds to step 22, where lastNewX and lastNewY are just global variables to store the last "known good" X,Y coordinates. Next, in step 24, the new "chunk" of the image is
drawn on the PDA display. Step 24 is accomplished using the steps described below with respect to the function updateDisplay.
In the preferred embodiment, the updateDisplay function is called to accomplish step 24 in FIG. 1. The updateDisplay function updates the display memory to display the section, or combination of sections, of the image corresponding to the newX and newY, or the shifted newX and newY. In the preferred embodiment, the function first defines the center point of the 160 X 160 section of the image bitmap. Wherever the newX and newY coordinates (or shifted newX and newY coordinates) are on the image bitmap, the function call sets that area of the image bitmap into display memory for viewing on the display. Although this function is used in the preferred embodiment, the function used to update the display with the selected section of the image can be any function known to one of ordinary skill in the art.
Referring next to FIG. 2, the preferred embodiment of the MainFormHandleEvent function is shown. The PDA OS or alternate OS is constantly reading the current X,Y location of the stylus. When a stylus movement event occurs, step 100, the process proceeds to step 102, where the MainFormHandleEvent is called. The stylus movement is determined, in step 100, when the OS determines there has been a change in the X,Y location of the stylus. Dragging the stylus across the screen will generate a stream of calls to continually update the display. In alternate embodiments, pressing activation button of the pointing device (i.e. mouse click) when the pointer display is located over the display screen and then moving the pointer across the display area (and therefore dragging the pointer) is the equivalent operation of dragging the stylus across the PDA screen. In yet another embodiment, a smaller image may be used that represents the full size image area of the larger bitmap, however, this smaller image is unnecessary and may be undesirable given that the smaller bitmap takes up a portion of the display area, this being a tradeoff for the smaller display area and, in PDA embodiments, a major tradeoff for the very small display area.
Next, in step 104, the process will determine whether the GrabScroll mode in the preferred embodiment has been enabled. If the GrabScroll mode has been enabled, the process proceeds to step 106, where the GrabScroll mode begins. In alternate embodiments the GrabScroll or equivalent mode is not implemented, therefore, step 104 and step 106 are bypassed in these embodiments. If the GrabScroll mode has not been enabled, then the ScaleScroll mode beings in step 108. Next, in step 110, if a dragging stylus is detected, then in step 112, the function converts the position on the display to the corresponding section of the image. This process is identical to the process described above and shown in FIG. 1. Next, the process continues to step 114, where the image is continually updated until it is determined (in step 110) that the dragging stylus has stopped. Step 114, similar to step 14 shown in FIG. 1, includes assurances that newX and newY are never greater than zero. This function also includes assurances that the section displayed is not smaller than the 160x160 display (similar to step 18 shown in FIG. 1). The MainFormHandleEvent function will not allow the user of the PDA to attempt to navigate off the right and bottom borders of the image. Next, in step 116, if it is determined that the section has moved, then the process proceeds to step 118, where the updateDisplay function is called (described above) to draw the new "chunk" of the image. Following step 118, the process continues to step 110. If the section has not moved, as determined in step 116, the process loops directly back to step 110, to determine whether the dragging stylus continues, or has stopped. The function exits when the function determines that the dragging stylus has stopped. Storage devices suitable for tangibly embodying computer program instructions include all forms of nonvolatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and
flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and
CD-ROM disks. Any of the foregoing may be supplemented by, or incorporated in, specially designed ASICs
(application-specific integrated circuits).
Other embodiments are within the scope of the following claims. For example, depending on the nature of the work items involved, the computer station may be a terminal instead of a personal computer.
The technique (i.e., the methods described above) may be implemented in hardware or software, or a combination of both. Preferably, the technique is implemented in computer programs executing on programmable computers that each include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Program code is applied to data entered using the input device to perform the method described above and to generate output information. The output information is applied to one or more output devices.
Each program is preferably implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Each such computer program is preferably stored on a storage medium or device (e.g., ROM or magnetic diskette) that is readable by a general or special purpose programmable computer for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described in this document. The system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner.
One skilled in the art of computer science will easily be able to combine the software created as described with appropriate general purpose or special purpose computer hardware to create a computer system embodying the invention.
Although the present invention has been described with reference to certain preferred embodiments thereof, other versions are readily apparent to those of ordinary skill in the art. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred embodiments contained herein.