US20080244385A1 - Systems and methods for managing multimodal documents - Google Patents

Systems and methods for managing multimodal documents Download PDF

Info

Publication number
US20080244385A1
US20080244385A1 US11/846,109 US84610907A US2008244385A1 US 20080244385 A1 US20080244385 A1 US 20080244385A1 US 84610907 A US84610907 A US 84610907A US 2008244385 A1 US2008244385 A1 US 2008244385A1
Authority
US
United States
Prior art keywords
management system
content management
multimodal
shredding
context manager
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US11/846,109
Inventor
Deborra J. Zukowski
Eric C. Friedrich
Arthur J. Parkos
Chao Chen
Jean-Hiram Coffy
John F. Braun
Richard W. Heiden
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Pitney Bowes Inc
Original Assignee
Pitney Bowes Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pitney Bowes Inc filed Critical Pitney Bowes Inc
Priority to US11/846,109 priority Critical patent/US20080244385A1/en
Assigned to PITNEY BOWES INC. reassignment PITNEY BOWES INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HEIDEN, RICHARD W., MR., PARKOS, ARTHUR J., MR., CHEN, CHAO, MS., ZUKOWSKI, DEBORRA J., MS., FRIEDRICH, ERIC C., MR., COFFY, JEAN-HIRAM, MR., BRAUN, JOHN F., MR.
Publication of US20080244385A1 publication Critical patent/US20080244385A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/166Editing, e.g. inserting or deleting
    • G06F40/169Annotation, e.g. comment data or footnotes

Definitions

  • the invention relates generally to multimodal documents and more particularly to systems and methods for managing multimodal documents including the shredding of multimodal documents through the deletion of virtual media and the destruction of physical media.
  • Multimedia documents are documents that contain or are associated with different types of content or media.
  • a document may exist which could consist of word-processing content, video annotations, or other types of media. This media although associated with the document, is not necessarily embedded in the document and can be found in multiple locations throughout the environment.
  • U.S. Pat. No. 5,897,643 for SYSTEM FOR MAINTAINING A RECORD OF DOCUMENTS INCLUDING DOCUMENT COPIES discloses a system that maintains a listing of a document and its copies where the electronic record of a copy is deleted when that copy is deleted.
  • U.S. Pat. No. 6,839,707 for WEB-BASED SYSTEM AND METHOD FOR MANAGING LEGAL INFORMATION discloses a web-based application that organizes the storage of patent applications and other legal documents into a hierarchical folder structure. It allows for the deletion of individual documents within a folder and also the deletion of the folders themselves.
  • 6,148,312 for METHOD AND SYSTEM FOR ELECTRONIC ARTICLE MAINTENANCE AND DELETION discloses a system for deleting documents from an electronic document management system. Requests for deletion must be confirmed by the creator of the document or a pre-set reviewer. Multiple delete sites are possible and are responsible for the confirmation and corresponding deletion of the document and its metadata. Additionally, paper entitled “A Multimedia Document Filing System” by Xien Fan, Qianhong Liu, and Peter A. Ng, published 1997 in the International Conference On Multimedia Computing Systems, discloses a system of organization of multimedia documents based on creating and categorizing metadata derived from the content of the document. Known documents are placed into folders as a method of access. These folders can then be linked together to associate their content. A delete operation is defined on a folder where the operation will also delete the document from subordinate folders.
  • While the above systems provide some limited ability to process multimedia documents, it is desirable to have automated systems managing the removal of such documents from the environment where appropriate with no or limited user intervention required to locate and delete physical and electronic content related to the document. Additionally, the above systems have a narrow view of the scope of what constitutes a document to be processed and are essentially closed systems in that they are solely responsible for managing the documents. Moreover, it is desirable to be able to manage the removal and shredding of multimodal documents which, as explained below, may involve all forms of virtual media and physical media.
  • a multimodal document management system embodying the present invention includes a context manager for managing multimodal documents.
  • a virtual media content management system is coupled to the context manager and a physical media content management system is coupled to the context manager.
  • the context manager is operable to issue multimodal document management instructions to the virtual media content management system and issuing instructions to the physical media content management system.
  • the context manager instructions relate to shredding of multimodal documents by deleting virtual media and destroying physical media.
  • a method for managing multimodal documents embodying the present invention includes the steps of receiving a request for the shred of elements in an information and content data file list of virtual media and physical media elements relating to a multimodal document. A determination is made if the request for shred is authorized and, where the request for shred is authorized creating a record of the shred request. Shred requested elements in the element list are grouped based on which are managed by the same content management system. A message is sent to each content management system requesting the shredding of each element of each grouping of elements managed by such same content management system.
  • FIG. 1 is a block diagram of a system for deleting multimodal document management embodying the present invention which manages the process of shredding (deletion and/or destruction, as the case may be) of various forms of related information which can be embedded in virtual media and/or physical media;
  • FIG. 2 is a diagrammatic illustration of an example of the system shown in FIG. 1 involving multiple information management domains;
  • FIG. 3 is a flow chart of the operation of the system shown in FIG. 1 involving the initiation of the process for shredding a multimodal document where the specific document is a physical document (physical media) enhanced with on-line annotation (virtual media);
  • FIG. 4 is a view of a computer screen showing a cover sheet for the physical document referenced in FIG. 3 having a shredding enablement area, here shown as a personal shredding enablement area;
  • FIG. 5 is a flow chart of the operation of the system shown in FIG. 1 involving the shredding process of the specific physical document referenced in FIG. 3 ;
  • FIG. 6 is a flow chart of the operation of the system shown in FIG. 1 for initiating the process for shredding (deletion and destruction) involving the complete set of elements in an information and content data file for another multimodal document, here an exhaustive shredding including all related virtual media and physical media and all use context related to the multimodal document;
  • FIG. 7 is a view of a computer screen having three open windows helpful in an understanding of the operation of the system shown in FIG. 1 where window A illustrates information and content data which list all of the virtual media (virtual document and parts) and all of the physical media (physical document and artifacts) for the multimodal document, window B illustrates a pdf document listed as the second electronic document in window A) and window C illustrates of one of the virtual document parts listed in Window A;
  • FIG. 8 is a complete view of window A shown in FIG. 7 unobstructed by other open computer screen windows where the complete information and content data is shown with the full listing of the all of the virtual media (virtual document and parts) and physical media (physical document and artifacts) for the multimodal document;
  • FIG. 9 is a flow chart of the operation of the system shown in FIG. 1 in the comprehensive shredding process of a multimodal document with the exhaustive deletion and destruction of the complete set of information and content data relating to the multimodal document where all virtual media (virtual documents and parts) and all physical media (physical documents and artifacts) are to be deleted and/or destroyed and all related context removed;
  • FIG. 10 is a more detailed flow chart of the operation of shredding process shown in FIG. 9 , providing further details of the operation of the system where the context manager organizes and initiates the exhaustive deletion and destruction of the multimodal document;
  • FIG. 11 is a more detailed flow chart of the operation of shredding process shown in FIG. 9 , providing further details of the operation of the system where the context manager awaits responses from various multimodal document content management servers and users; and,
  • FIG. 12 is a more detailed flow chart of the operation of shredding process shown in FIG. 9 , providing further details of the operation of the system where the context manager initiates action when the time allotted to receive a response from a content management server and/or user has passed.
  • Multimodal documents include all forms and types of information. While multimodal documents are often physical documents and related virtual information, it also includes both virtual media and physical media.
  • Virtual media for example, include electronic versions of documents, e-mails, and metadata stored in memory of various types, electronic versions of video information stored in various forms, electronic versions of audio information stored in various forms and software stored in various electronic forms.
  • Physical media for example, include printed documents, media containing video information (DVDs, tapes, etc.), media containing audio information (CDs, tapes, etc.), physical models, photographs, and software disks.
  • Multimodal documents content may be rendered, for example, in complete documents, artifacts and parts thereof that can exist in paper or electronic form.
  • Virtual media and physical media are often related in that, for example, a printed document with hand written annotations (physical media) thereon may be electronically stored (virtual media).
  • These media, the physical media and the virtual media are related in that they involve the same or similar information or information that has been modified.
  • an electronic copy of an engineering drawing (virtual media) and the physical embodiment depicted in the engineering drawing (physical media) are both related in the information they convey.
  • Information domains are the various environments in which the virtual media and the physical media exist and are managed.
  • Domains of control are the various systems and/or subsystems, such as a content management server (server and associated software), responsible for control of a certain set of information such as video files, audio files, document metadata, or physical and/or electronic renderings of a document.
  • a content management server responsible for control of a certain set of information such as video files, audio files, document metadata, or physical and/or electronic renderings of a document.
  • Information and content data is the complete set of data including content, use and domains of a multimodal document including virtual media and physical media; essentially a bucket of data pertaining to the multimodal document.
  • the data typically a list of elements, can be stored in a file or a database or otherwise.
  • ICDF information and content data file
  • Shredding includes the deletion of virtual media and the destruction of physical media, as the case may be.
  • the system facilitates and enhances the ability to more totally shred and also to automate the shredding of multimodal documents.
  • the system will verify the requesting entity is allowed to shred the given multimodal document or portions of the multimodal document. If authorized, the system will shred all, or as much as is authorized or requested, of media constituting the multimodal document including media created in the production of the document. Certain pieces of such a multimodal document may be stored on different systems or in different locations. For each piece of the multimodal document, a request or instruction for deletion of that specific piece is issued. The request for deletion can be grouped.
  • the request is communicated to and processed by the system responsible for that particular piece of the multimodal document, and the success or failure of the deletion is forwarded to the user or entity requesting shredding.
  • the system can be indifferent to the types of media to be shredded or treat different types of media differently.
  • the system is operable across various information domains including virtual media domains and physical media domains. Thus, for example, video content is treated the same as audio content, which is also treated the same as a paper document or an annotated paper document.
  • a user accesses, through an input system 2 , a context manager 4 which manages the tracking and shredding of both virtual and physical media.
  • the context manager interacts with one or more system(s) 6 for tracking virtual media and one or more system(s) 8 for tracking physical media of multimodal documents.
  • the systems 6 and 8 are content management systems and, in part, operate based on instructions from the context manager 4 with respect to multimodal documents.
  • Each of these systems can be separate full functioning system which provides a variety of independent operational functions such as printing documents, word processing, video creation and processing, audio creation and processing.
  • These systems can operate in separate domains and be controlled independently by various means such as personal computers, servers, multifunctional devices, smart paper shredders, etc.
  • the virtual media content management system 6 is coupled to a virtual media shredding subsystem 10 which shreds, by deletion, the virtual media.
  • the physical media content management system 8 is connected to a physical shredding subsystem 12 , which shreds by destroying the physical media.
  • the context manager 4 interactively operates with the various virtual media content management system(s) and the physical media content management system(s) to both identify and compile the content of information files for multimodal documents, to issue shredding instructions and to track the completion of the various shredding operations.
  • subsystems 10 and 12 each of these subsystems can be part of the content management system(s) for tracking media to which it is connected.
  • the context manager 4 includes a storage means 14 where the information and content data for multimodal documents (entity and context models) and other related information are stored.
  • the context manager 4 is coupled via any of a large number of communications systems, here shown as messaging space 16 , to various devices and servers.
  • the context manager 4 is connected so that it can span physical locations and various organizations and structures to manage the totality of the multimodal document content and store references to that content in an information and content data file.
  • the multiple domains of use for the multimodal document is such to enable access by the context manager 4 to a plurality of common, potentially associated (such as video and audio), messaging systems and, for example, by shared names for objects that are involved in the various environments. This can include people, devices and documents, both virtual and physical.
  • the domains to which the context manager 4 is connected each incorporate software systems, instrumented devices, spaces and/or procedures that help to create and track multimodal document use.
  • the context manager 4 is coupled via the messaging space to multifunctional devices 18 and 20 .
  • These may be devices that have been enhanced with one or more document identification technologies, such as radio frequency identification device (RFID) readers, bar code/ANOTO Docld capture, document image capture and hashing. These types of technologies, and others, are employed so that they can identify a pre-existing document that is being printed, scanned or copied. Additionally, these multi-functional devices can create document clones that have been uniquely named, using one of more of these technologies.
  • a printer can create a physical clone of an electronic document while a scanner will do the reverse by taking a physical document and creating an electronic document.
  • Shredders 22 and 24 may be normal shredders that have been enhanced with one or more document identification technologies, such as those noted above. Such shredders can operate to inform the context manager 4 , and other subsystems as appropriate, via the messaging space 16 , when a document has been shredded, that is, physically destroyed, and, should the shredder have a memory, the virtual shredding, that is, the deletion of any relevant information in the shredder related to the multimodal document.
  • An example of one type of shredder with enhanced document identification capability is disclosed in U.S. patent application Ser. No. 11/156,127, filed for B. D. Singer et al. on Jun.
  • Active workspaces 26 and 28 may be desks, shelves or other physical places where documents are used that may be enhanced with one or more document identification technologies. These active workspaces 26 and 28 can have additional on-line, active spaces, such as a user's electronic document mailbox, that act similarly to the physical spaces.
  • the audio server 30 manages all of the captured audio media data stream and stores needed information either in its own system storage 32 or on a locally accessed storage.
  • the audio server 30 need not be physically co-located with other subsystems or other domains. Access to the audio server 30 by different users may be implemented through various types of audio input capability, shown at 34 , 36 and 38 .
  • the ink server 40 like the audio server 30 , manages all of the ink media data stream and stores needed information either in its own system storage 42 or through locally accessed storage.
  • the ink server 40 need not be physically co-located with other subsystems or domains being managed by the context manager 4 via the messaging space 16 .
  • the ink server 40 can employ digital pens, such as ANOTO pens 44 , 46 and 48 .
  • the ANOTO pens 44 , 46 and 48 may be on users' personal computing space and interconnected to the ink server 40 .
  • the architecture of the system shown in FIGS. 1 and 2 can be modified, provided it has the functionality of enabling the context manager 4 to communicate directly or indirectly with the various information domains it is expected to manage in the tracking and shredding of multimodal documents.
  • Each information domain has a system(s) which provide a domain of control to manage the set of content for which it is responsible.
  • various additional domains and subsystems may be added.
  • the context manager can operate within a secure environment if that is desired, such as a secure building, as denoted by the line 50 encircling the context manager 4 and all of the various domains of control.
  • the context manager 4 provides multimodal document compilation. It communicates via the messaging space 16 for messages and data emanating from all of the various information domains it interoperates with and derives the use of each managed multimodal document in each managed domain. For example, when a managed multimodal document is physically printed, the context manager 4 is informed via the messaging space of this activity. The context manager 4 uses the various messages communicated to it via the messaging space to record multimodal document use information. This can include the creation of document clones via the multifunctional devices. The context manager 4 can obtain information relating to the physical location of documents, such as via the active workspaces, and the attachment of media clips on virtual or physical documents via the various media servers, such as the audio server 30 and the ink server 40 . There may also be other on-line internet or intranet applications, not shown, that communicate with the context manager 4 , such as word processing document used on a personal computer.
  • the context manager 4 is the single point of control and relies on the various subsystems in the various domains to shred the multimodal documents.
  • each of the various servers manages the use of the various activities related to that server.
  • the audio server 30 manages audio clip information but communicates that management functionality via the messaging space 16 to the context manager 4 .
  • the context manager 4 remains the single point of control for tracking, issuing shred commands and monitoring shredding activity.
  • Other architecture can also be implemented where the single point of control is modified to provide additional or separate points of control for separate types of subsystems or separate categories of multimodal document virtual media and/or physical media.
  • the system mitigates the need for human intervention when shredding a multimodal document in the system.
  • the multimodal document is shredded to the best ability of the system based on available data concerning the multimodal document and the use of the multimodal document.
  • the shredding of virtual media such as computer files existing on a media server in one location and physical media such as printed documents in another location can be brought about by a request from a context manager such as a server in yet another location.
  • the context server may also possess greater knowledge of the context than individual systems and users of the multimodal document because of the span of domains about which the context server stores or can access data. The system uses this knowledge from the span of domain to assist users in completely shredding a multimodal document or portion of such document as desired and authorized.
  • the system is organized for the context manager to communicate with various domains of control related to the multimodal document so that it is able to initiate and, depending on how the system is implemented, monitor and/or confirm, the shredding of a multimodal document including renditions and annotations associated with the document.
  • a domain of control such as a server
  • the domain of control can communicate back to the system the completion of the shred and/or the status of an attempted shred.
  • the system informs affected users the status of the shredding of the multimodal document. If the system has been unable to shred the content to the extent requested, these notifications of shred status from the various domains may function as a request for user intervention. If successful, the notifications instead signal the completion of the shredding process to the user.
  • the system manages documents that exist in both the physical and electronic environment.
  • the shredding process ensures the removal of the document's metadata and all its associated content (annotations and renderings).
  • a rendering is analogous to a copy of the document. Just as copies of a document can exist in a paper form or as an electronic file, renderings have an equal ambivalence to medium.
  • the system proactively manages paper documents, for example, in several ways. First, if a document is deleted, the system will alert users to the deletion event and request that they destroy the paper rendering. Second, the system will monitor user interactions with a paper rendering for an action that represents deletion of that document. On the occurrence of the triggering process, the system will begin a deletion of all content and metadata related to the current document.
  • FIG. 3 An ANOTO-ized document is created at 52 with a document cover sheet.
  • the document is registered at 54 with the context management system.
  • the document is annotated with ink marks, audio and other media stream segments at 56 . This is in accordance with the operation of the various subsystems as described in FIG. 2 .
  • a check mark or indication is placed next or in the shredding enablement box or area.
  • the box is checked for a personal shredding at 58 .
  • the personal shredding enablement may be, for example, as shown in FIG. 4 . Other more extensive shredding enablement boxes or areas for various users can be provided depending on the design of the system.
  • the document cover sheet referenced in block 52 of FIG. 3 may be as is shown in FIG. 4 .
  • the computer view of the cover sheet 60 shows a shredding enablement area shown generally at 62 .
  • the shredding enablement area 62 is a personal shredding enablement area activated by the activation of personal shredding box 64 .
  • the personal shredding depending upon how the system is implemented, can also include shredding of all virtual media and physical media that involve the user's use and/or relationship to the multimodal document.
  • Confirmation of the shredding including physical destruction of any physical media as well as the confirmation of the deletion of all virtual media can also be provided.
  • other more extensive and comprehensive shredding enablement options can be provided, depending on the system. This can be implemented with verification of the authorization of the initiator for each of the different aspects or parts of the document to be shredded.
  • a manager may be allowed and authorized to initiate the shredding of document parts used within the manager's team but not allowed to shred documents outside of the team.
  • An explicit field for example either in an information and content data file or database (ICDF) at the context manager 4 or in a content management system, can be provided which indicates who is authorized to shred and what they are authorized to shred.
  • the field or another field can also be provided to indicate the ownership or custodian of what is to be shredded and where desired provide this information to other systems.
  • FIG. 5 showing the operation of the system involved in the shredding process of the specific physical document referenced in FIG. 3 .
  • a document is allowed to be shredded by a requester, as identified, for example, via the owner (user) of a specific ANOTO pen.
  • a shred request is sent to the ANOTO server at 68 , including the identification that the shred request was registered to the specific document to be shredded.
  • a shred request is sent by the context manager 4 to each of the media servers for which there are media stream segments included in the metadata associated with the document.
  • a countdown timer is started and at block 76 , responses are awaited from each of the servers to which a shred request was sent by the context manager 4 .
  • the requester is notified that there was a problem with that part of the shred operation.
  • the associated metadata from the context memory 4 is removed.
  • a message is sent to the requester with the state of the shred of the multimodal document requested to be shred. The requester then may physically shred the personal copy of the requester's multimodal document.
  • FIG. 6 where the process for initiating the shredding involving the complete set of content, as represented, the entire information and content data file, for a multimodal document is initiated.
  • a information and content data file or database ICDF
  • ICDF information and content data file or database
  • content is created, it is registered and reference is added to the information and content data file or database and appropriately stored by the context manager 4 .
  • the context manager 4 tracks the use of each piece of content at block 90 .
  • an exhaustive shred is initiated, such as by right clicking the information and content data file for the multimodal document at the context manager 4 and selecting Exhaustive Shred shown in FIG. 8 .
  • the tracking the use of each piece of content at 90 may include location tracking of physical items and use tracking, like writing on items, mailing documents (e.g., e-mail sniffers and enhanced postal meters, such as meters with scanners), etc. All tracking information of both physical media and virtual media is sent to and managed by the context manager 4 .
  • the tracking methods can employ any of the methods described above and other well known methods useful in tracking and sensing and image recognition, including by use of digital pens, radio frequency identification (RFID), bar codes and numeric fields, electronic document identification (ID) sensing, image recognition via cameras, and the like.
  • RFID radio frequency identification
  • ID electronic document identification
  • a computer screen 13 exhibits three open windows, window A, window B, and window C. These windows are examples of the type of information that is gathered, managed by and displayed using the context manager 4 .
  • window A an information and content data file illustrates and lists all of the virtual media (virtual documents and parts) and all of the physical media (physical documents and artifacts) for a multimodal document.
  • the information and content data file in window A includes three electronic documents 94 , 96 and 98 . These three electronic documents 94 , 96 and 98 are pdf-generated from the original ANOTO document and include all existing annotations at the time of generation. Two physical ANOTO-ized documents 100 and 102 are also listed.
  • Window B illustrates a pdf document listed as the second electronic version 94 in Window A. This is document is designated as document 102 a in Window B.
  • Window C illustrates one of the virtual document parts listed in Window A, here designated as 104 a .
  • Various types and formats of displays and organization of the various information related to the creation, tracking and shredding of virtual media and physical media can be employed, depending on the nature and design of the system.
  • FIG. 8 provides a complete view of window A shown in FIG. 7 unobstructed by other open screen windows.
  • the complete information and content data file for the multimodal document is shown with a full and complete listing of all of the virtual media (virtual document and parts) and all of the physical media (physical documents and artifacts) at 94 through 112 . Additional types of information that can be further included and listed in the information and content data file. A user could initiate an exhaustive shred of the full information and content listed in the information and content data file by activating box 95 .
  • FIG. 9 shows the operation of the system in the comprehensive shredding process of a multimodal document at a high level and with the exhaustive deletion and destruction of the complete set of content.
  • a shred request is sent to all the servers and administrators that manage the content items of the document, the context manager and content management systems. This would include all of the systems shown in FIGS. 1 and 2 , if that is the particular architecture that is implemented.
  • a countdown timer is started and at block 120 the system awaits responses from each server and/or administrator to which a shred request was sent.
  • the requester is notified that there was a problem with all failed shreds.
  • the associated metadata is removed from the context manager memory, storage means 4 .
  • a message is sent at block 126 to the requester with the state of the shred operation.
  • the requester must explicitly follow up with external parties to ensure that their shred was complete, to the extent that the external parties are not within the domains that are managed by the context manager 2 .
  • the context manager 4 orchestrates a comprehensive shredding operation.
  • the context manager 4 receives a request for the comprehensive (exhaustive) shredding of the virtual media and physical media listed in the information and content data file at block 130 .
  • This request may have been initiated by the user as indicated in FIG. 8 , or may be initiated via some other interface that may be intentionally activated, for example, one similar to the document cover sheet shown in FIG. 4 .
  • a determination is made at block 132 if the requester is authorized to remove the content from the information and content data file. Where the requester is not so authorized, the context manager at block 134 discards the request.
  • the request by an unauthorized requester for shredding may be stored, if desired, in the context manager 4 for subsequent audit and/or other activities.
  • the context manager 4 creates a record of the request at block 133 .
  • all elements of the information and content data file are listed at block 134 if the list has not previously been compiled. This includes clones, artifacts, document parts, and the like.
  • the list of the context created information and content data file includes all virtual media and all physical media if the list has not previously been compiled.
  • a list of all current users is created of all of the virtual media and all of the physical media.
  • the list of elements and users (and any other list) can be created and exist prior to the request for shred being received.
  • the list build can be ongoing and continuous as elements are created or the build can be periodic, that is, hourly, daily, weekly, monthly, etc.
  • the context manager 4 sub-groups various elements in the list. This may be based on the content management systems employed, such as the various media servers, the various element inventory administration, users of the multimodal documents, and the like.
  • a message is sent to each content management system requesting the deletion and destruction of each element; for example, e-mails sent to persons and the like.
  • a response timer is set for each content management system. The response timer and time involved can be unique to the particular content management system and even within subsystems of the particular content management system, depending on the need and applications for the system.
  • a list of all items that have been sent out of the system domain is created.
  • the requester is notified of all such items including recipients of the items, and at block 148 , the requester follows up on those items using traditional methods, e.g., phone mail, e-mail, and other appropriate methods to ensure the deletion or destruction of the virtual media and physical media.
  • the process steps 144 , 146 and 148 are for those portions of multimodal document that may not or are not under the management of the context manager 4 and need separate, independent operations to ensure their deletion and/or destruction.
  • the context manager 4 manages not only systems within the direct or indirect control of the context manager system 4 but also related multimodal document virtual media and physical media related to such other materials that need to be independently processed when that information is available to the context manager 4 .
  • the context manager at block 150 awaits the next response message from the content management servers and/or users. For each element included at block 152 , a determination is made as to whether the requested shred request was accomplished. If the shred request was accomplished, the elements response trigger is disabled at block 154 and the element is removed from the information and content data file list. At block 155 related context is removed. A determination is then made at block 156 if the information and content data file list is empty. Where the information and content data is not empty, the system loops back to block 150 .
  • the shred request was not accomplished, a determination is made at block 158 for the specific element that was not shredded whether the maximum number of retries have been attempted.
  • the number of tries is a matter of design choice and can vary for each type of element to be shredded. Where the maximum number of retries have been attempted, the element is marked as a failed shred attempt at block 160 . Where the maximum number of retries have not been attempted, at block 162 a request is sent to the context manager 4 to initiate another attempt to shred the same element.
  • a message such as an e-mail, is sent to the person who requested the comprehensive deletion/destruction request, the shred request, notifying the requester that the requested shred was successful.
  • an element response trigger fires. This is when the time allotted to receive a response that a shred request has been accomplished has passed.
  • a determination is then made at block 168 if the maximum number of retries have been attempted. Where this is not the case, at block 170 a request is again sent by the context manager 4 to the content server for the element to delete and/or destroy, shred, the element.
  • the element is marked as failed at block 172 . The element is thus marked as an element of the multimodal document that has failed to be shredded on a shred request.

Abstract

A multimodal document management system and method includes a context manager for managing multimodal documents. A virtual media content management system is coupled to the context manager and a physical media content management system is coupled to the context manager. The context manager is operable to issue multimodal document management instructions to the virtual media content management system and issuing instructions to the physical media content management system. The context manager instructions may relate to shredding of multimodal documents by deleting virtual media and destroying physical media. The method for managing multimodal documents may include receiving a request for the shredding of an information and content data file containing a list of virtual media and physical media relating to a multimodal document. A determination is made if the request for shred is authorized and, where said request for shred is authorized, a record of the shred request is created. Elements in the element list are grouped based on elements in the list which are managed by the same content management system. A message is sent to each content management system requesting the shredding of each element for the grouping of elements for the content management system. The content management systems may verify and report the success of the shredding.

Description

    RELATED APPLICATIONS
  • This application claims the benefit under 35 U.S.C. 119(e) of U.S. provisional patent applications: Ser. No. 60/909,281 filed Mar. 30, 2007, and entitled “A METHOD AND SYSTEM FOR ENABLING COLLABORATIVE CAPTURE AND REPLAY OF DIGITAL MEDIA FILES USING PHYSICAL DOCUMENTS”; and Ser. No. 60/909,273 filed Mar. 30, 2007, and entitled “SYSTEMS AND METHODS FOR MANAGING MULTIMODAL DOCUMENTS.” Both provisional patent applications are incorporated herein by reference.
  • FIELD OF THE INVENTION
  • The invention relates generally to multimodal documents and more particularly to systems and methods for managing multimodal documents including the shredding of multimodal documents through the deletion of virtual media and the destruction of physical media.
  • BACKGROUND OF THE INVENTION
  • Document retention and destruction is becoming increasingly important. Government regulations and litigation threats place pressure on companies to properly handle documents that have reached the end of their usefulness. Additionally, documents that are not properly discarded can contribute to corporate espionage and identity theft. The handling of documents is also complicated by the multimedia and distributed nature of documents in our environment. Multimedia documents are documents that contain or are associated with different types of content or media. For example, a document may exist which could consist of word-processing content, video annotations, or other types of media. This media although associated with the document, is not necessarily embedded in the document and can be found in multiple locations throughout the environment.
  • Various systems have been developed for deleting and destroying documents and related content. U.S. Pat. No. 5,897,643 for SYSTEM FOR MAINTAINING A RECORD OF DOCUMENTS INCLUDING DOCUMENT COPIES discloses a system that maintains a listing of a document and its copies where the electronic record of a copy is deleted when that copy is deleted. U.S. Pat. No. 6,839,707 for WEB-BASED SYSTEM AND METHOD FOR MANAGING LEGAL INFORMATION discloses a web-based application that organizes the storage of patent applications and other legal documents into a hierarchical folder structure. It allows for the deletion of individual documents within a folder and also the deletion of the folders themselves. U.S. Pat. No. 6,148,312 for METHOD AND SYSTEM FOR ELECTRONIC ARTICLE MAINTENANCE AND DELETION discloses a system for deleting documents from an electronic document management system. Requests for deletion must be confirmed by the creator of the document or a pre-set reviewer. Multiple delete sites are possible and are responsible for the confirmation and corresponding deletion of the document and its metadata. Additionally, paper entitled “A Multimedia Document Filing System” by Xien Fan, Qianhong Liu, and Peter A. Ng, published 1997 in the International Conference On Multimedia Computing Systems, discloses a system of organization of multimedia documents based on creating and categorizing metadata derived from the content of the document. Known documents are placed into folders as a method of access. These folders can then be linked together to associate their content. A delete operation is defined on a folder where the operation will also delete the document from subordinate folders.
  • While the above systems provide some limited ability to process multimedia documents, it is desirable to have automated systems managing the removal of such documents from the environment where appropriate with no or limited user intervention required to locate and delete physical and electronic content related to the document. Additionally, the above systems have a narrow view of the scope of what constitutes a document to be processed and are essentially closed systems in that they are solely responsible for managing the documents. Moreover, it is desirable to be able to manage the removal and shredding of multimodal documents which, as explained below, may involve all forms of virtual media and physical media.
  • SUMMARY OF THE INVENTION
  • It is an object of the present invention to provide a multimodal document management system that allows the management of multimodal documents across different domains of control.
  • It is a further object of the present invention to enable the management of shredding of multimodal documents including virtual media and physical media relating to a multimodal document.
  • It is yet another objective of the present invention to provide a system to interact with different domains of control to automate, based on tracking information available to the system, the shredding of multimodal documents by the deletion of virtual media and destruction of physical media relating to a multimodal document.
  • A multimodal document management system embodying the present invention includes a context manager for managing multimodal documents. A virtual media content management system is coupled to the context manager and a physical media content management system is coupled to the context manager. The context manager is operable to issue multimodal document management instructions to the virtual media content management system and issuing instructions to the physical media content management system.
  • In accordance with a feature of the present invention, the context manager instructions relate to shredding of multimodal documents by deleting virtual media and destroying physical media.
  • A method for managing multimodal documents embodying the present invention includes the steps of receiving a request for the shred of elements in an information and content data file list of virtual media and physical media elements relating to a multimodal document. A determination is made if the request for shred is authorized and, where the request for shred is authorized creating a record of the shred request. Shred requested elements in the element list are grouped based on which are managed by the same content management system. A message is sent to each content management system requesting the shredding of each element of each grouping of elements managed by such same content management system.
  • DESCRIPTION OF THE DRAWINGS
  • The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate presently preferred embodiments of the invention, and together with the general description given above and the detailed description of the preferred embodiments given below, serve to explain the principles of the invention. As shown throughout the drawings, like reference numerals designate like or corresponding parts.
  • FIG. 1 is a block diagram of a system for deleting multimodal document management embodying the present invention which manages the process of shredding (deletion and/or destruction, as the case may be) of various forms of related information which can be embedded in virtual media and/or physical media;
  • FIG. 2 is a diagrammatic illustration of an example of the system shown in FIG. 1 involving multiple information management domains;
  • FIG. 3 is a flow chart of the operation of the system shown in FIG. 1 involving the initiation of the process for shredding a multimodal document where the specific document is a physical document (physical media) enhanced with on-line annotation (virtual media);
  • FIG. 4 is a view of a computer screen showing a cover sheet for the physical document referenced in FIG. 3 having a shredding enablement area, here shown as a personal shredding enablement area;
  • FIG. 5 is a flow chart of the operation of the system shown in FIG. 1 involving the shredding process of the specific physical document referenced in FIG. 3;
  • FIG. 6 is a flow chart of the operation of the system shown in FIG. 1 for initiating the process for shredding (deletion and destruction) involving the complete set of elements in an information and content data file for another multimodal document, here an exhaustive shredding including all related virtual media and physical media and all use context related to the multimodal document;
  • FIG. 7 is a view of a computer screen having three open windows helpful in an understanding of the operation of the system shown in FIG. 1 where window A illustrates information and content data which list all of the virtual media (virtual document and parts) and all of the physical media (physical document and artifacts) for the multimodal document, window B illustrates a pdf document listed as the second electronic document in window A) and window C illustrates of one of the virtual document parts listed in Window A;
  • FIG. 8 is a complete view of window A shown in FIG. 7 unobstructed by other open computer screen windows where the complete information and content data is shown with the full listing of the all of the virtual media (virtual document and parts) and physical media (physical document and artifacts) for the multimodal document;
  • FIG. 9 is a flow chart of the operation of the system shown in FIG. 1 in the comprehensive shredding process of a multimodal document with the exhaustive deletion and destruction of the complete set of information and content data relating to the multimodal document where all virtual media (virtual documents and parts) and all physical media (physical documents and artifacts) are to be deleted and/or destroyed and all related context removed;
  • FIG. 10 is a more detailed flow chart of the operation of shredding process shown in FIG. 9, providing further details of the operation of the system where the context manager organizes and initiates the exhaustive deletion and destruction of the multimodal document;
  • FIG. 11 is a more detailed flow chart of the operation of shredding process shown in FIG. 9, providing further details of the operation of the system where the context manager awaits responses from various multimodal document content management servers and users; and,
  • FIG. 12 is a more detailed flow chart of the operation of shredding process shown in FIG. 9, providing further details of the operation of the system where the context manager initiates action when the time allotted to receive a response from a content management server and/or user has passed.
  • DETAILED DESCRIPTION OF THE PRESENT INVENTION
  • Various terms used herein have the meaning and usage set out below.
  • Multimodal documents include all forms and types of information. While multimodal documents are often physical documents and related virtual information, it also includes both virtual media and physical media. Virtual media, for example, include electronic versions of documents, e-mails, and metadata stored in memory of various types, electronic versions of video information stored in various forms, electronic versions of audio information stored in various forms and software stored in various electronic forms. Physical media, for example, include printed documents, media containing video information (DVDs, tapes, etc.), media containing audio information (CDs, tapes, etc.), physical models, photographs, and software disks. Multimodal documents content may be rendered, for example, in complete documents, artifacts and parts thereof that can exist in paper or electronic form. Virtual media and physical media are often related in that, for example, a printed document with hand written annotations (physical media) thereon may be electronically stored (virtual media). These media, the physical media and the virtual media, are related in that they involve the same or similar information or information that has been modified. For example, an electronic copy of an engineering drawing (virtual media) and the physical embodiment depicted in the engineering drawing (physical media) are both related in the information they convey.
  • Information domains are the various environments in which the virtual media and the physical media exist and are managed.
  • Domains of control are the various systems and/or subsystems, such as a content management server (server and associated software), responsible for control of a certain set of information such as video files, audio files, document metadata, or physical and/or electronic renderings of a document.
  • Information and content data is the complete set of data including content, use and domains of a multimodal document including virtual media and physical media; essentially a bucket of data pertaining to the multimodal document. The data, typically a list of elements, can be stored in a file or a database or otherwise. Herein, the phrase information and content data file (ICDF) is intended to encompass all of the foregoing.
  • Shredding includes the deletion of virtual media and the destruction of physical media, as the case may be.
  • The following is an overview of the operation of the system shown in FIGS. 1-12. The system facilitates and enhances the ability to more totally shred and also to automate the shredding of multimodal documents. Once a request for shredding is made, the system will verify the requesting entity is allowed to shred the given multimodal document or portions of the multimodal document. If authorized, the system will shred all, or as much as is authorized or requested, of media constituting the multimodal document including media created in the production of the document. Certain pieces of such a multimodal document may be stored on different systems or in different locations. For each piece of the multimodal document, a request or instruction for deletion of that specific piece is issued. The request for deletion can be grouped. The request is communicated to and processed by the system responsible for that particular piece of the multimodal document, and the success or failure of the deletion is forwarded to the user or entity requesting shredding. The system can be indifferent to the types of media to be shredded or treat different types of media differently. The system is operable across various information domains including virtual media domains and physical media domains. Thus, for example, video content is treated the same as audio content, which is also treated the same as a paper document or an annotated paper document.
  • Reference is now made to FIG. 1. A user accesses, through an input system 2, a context manager 4 which manages the tracking and shredding of both virtual and physical media. The context manager interacts with one or more system(s) 6 for tracking virtual media and one or more system(s) 8 for tracking physical media of multimodal documents. The systems 6 and 8 are content management systems and, in part, operate based on instructions from the context manager 4 with respect to multimodal documents. Each of these systems can be separate full functioning system which provides a variety of independent operational functions such as printing documents, word processing, video creation and processing, audio creation and processing. These systems can operate in separate domains and be controlled independently by various means such as personal computers, servers, multifunctional devices, smart paper shredders, etc.
  • The virtual media content management system 6 is coupled to a virtual media shredding subsystem 10 which shreds, by deletion, the virtual media. The physical media content management system 8 is connected to a physical shredding subsystem 12, which shreds by destroying the physical media. The context manager 4 interactively operates with the various virtual media content management system(s) and the physical media content management system(s) to both identify and compile the content of information files for multimodal documents, to issue shredding instructions and to track the completion of the various shredding operations. Although shown as separate shredding systems, subsystems 10 and 12, each of these subsystems can be part of the content management system(s) for tracking media to which it is connected.
  • Reference is now made to FIG. 2. The context manager 4 includes a storage means 14 where the information and content data for multimodal documents (entity and context models) and other related information are stored. The context manager 4 is coupled via any of a large number of communications systems, here shown as messaging space 16, to various devices and servers. The context manager 4 is connected so that it can span physical locations and various organizations and structures to manage the totality of the multimodal document content and store references to that content in an information and content data file. The multiple domains of use for the multimodal document, as shown in FIG. 2, is such to enable access by the context manager 4 to a plurality of common, potentially associated (such as video and audio), messaging systems and, for example, by shared names for objects that are involved in the various environments. This can include people, devices and documents, both virtual and physical. The domains to which the context manager 4 is connected each incorporate software systems, instrumented devices, spaces and/or procedures that help to create and track multimodal document use.
  • The context manager 4 is coupled via the messaging space to multifunctional devices 18 and 20. These may be devices that have been enhanced with one or more document identification technologies, such as radio frequency identification device (RFID) readers, bar code/ANOTO Docld capture, document image capture and hashing. These types of technologies, and others, are employed so that they can identify a pre-existing document that is being printed, scanned or copied. Additionally, these multi-functional devices can create document clones that have been uniquely named, using one of more of these technologies. A printer can create a physical clone of an electronic document while a scanner will do the reverse by taking a physical document and creating an electronic document.
  • Shredders 22 and 24 may be normal shredders that have been enhanced with one or more document identification technologies, such as those noted above. Such shredders can operate to inform the context manager 4, and other subsystems as appropriate, via the messaging space 16, when a document has been shredded, that is, physically destroyed, and, should the shredder have a memory, the virtual shredding, that is, the deletion of any relevant information in the shredder related to the multimodal document. An example of one type of shredder with enhanced document identification capability is disclosed in U.S. patent application Ser. No. 11/156,127, filed for B. D. Singer et al. on Jun. 17, 2006, entitled SYSTEM AND METHOD FOR CONTROLLING THE STORAGE AND DESTRUCTION OF DOCUMENTS and assigned to Pitney Bowes Inc. (Pitney Bowes Docket No. F-933). Active workspaces 26 and 28 may be desks, shelves or other physical places where documents are used that may be enhanced with one or more document identification technologies. These active workspaces 26 and 28 can have additional on-line, active spaces, such as a user's electronic document mailbox, that act similarly to the physical spaces.
  • There are two primary types of software systems (audio server and ink server) shown in FIG. 2. These systems provide media services and a system that senses and represents reports events associated with the content, that is, document use within the environment. The audio server 30 manages all of the captured audio media data stream and stores needed information either in its own system storage 32 or on a locally accessed storage. The audio server 30 need not be physically co-located with other subsystems or other domains. Access to the audio server 30 by different users may be implemented through various types of audio input capability, shown at 34, 36 and 38. The ink server 40, like the audio server 30, manages all of the ink media data stream and stores needed information either in its own system storage 42 or through locally accessed storage. The ink server 40 need not be physically co-located with other subsystems or domains being managed by the context manager 4 via the messaging space 16. The ink server 40 can employ digital pens, such as ANOTO pens 44, 46 and 48. The ANOTO pens 44, 46 and 48 may be on users' personal computing space and interconnected to the ink server 40.
  • There can, of course, be other domains, including other media servers such as those relating to video domains. The architecture of the system shown in FIGS. 1 and 2 can be modified, provided it has the functionality of enabling the context manager 4 to communicate directly or indirectly with the various information domains it is expected to manage in the tracking and shredding of multimodal documents. Each information domain has a system(s) which provide a domain of control to manage the set of content for which it is responsible. For the architecture shown, various additional domains and subsystems may be added. Moreover, the context manager can operate within a secure environment if that is desired, such as a secure building, as denoted by the line 50 encircling the context manager 4 and all of the various domains of control.
  • As can be seen above, the context manager 4 provides multimodal document compilation. It communicates via the messaging space 16 for messages and data emanating from all of the various information domains it interoperates with and derives the use of each managed multimodal document in each managed domain. For example, when a managed multimodal document is physically printed, the context manager 4 is informed via the messaging space of this activity. The context manager 4 uses the various messages communicated to it via the messaging space to record multimodal document use information. This can include the creation of document clones via the multifunctional devices. The context manager 4 can obtain information relating to the physical location of documents, such as via the active workspaces, and the attachment of media clips on virtual or physical documents via the various media servers, such as the audio server 30 and the ink server 40. There may also be other on-line internet or intranet applications, not shown, that communicate with the context manager 4, such as word processing document used on a personal computer.
  • For the architecture shown in FIG. 2, the context manager 4 is the single point of control and relies on the various subsystems in the various domains to shred the multimodal documents. As shown in FIG. 2, each of the various servers manages the use of the various activities related to that server. For example, the audio server 30 manages audio clip information but communicates that management functionality via the messaging space 16 to the context manager 4. Thus, the context manager 4 remains the single point of control for tracking, issuing shred commands and monitoring shredding activity. Other architecture can also be implemented where the single point of control is modified to provide additional or separate points of control for separate types of subsystems or separate categories of multimodal document virtual media and/or physical media.
  • The system mitigates the need for human intervention when shredding a multimodal document in the system. The multimodal document is shredded to the best ability of the system based on available data concerning the multimodal document and the use of the multimodal document. The shredding of virtual media such as computer files existing on a media server in one location and physical media such as printed documents in another location can be brought about by a request from a context manager such as a server in yet another location. The context server may also possess greater knowledge of the context than individual systems and users of the multimodal document because of the span of domains about which the context server stores or can access data. The system uses this knowledge from the span of domain to assist users in completely shredding a multimodal document or portion of such document as desired and authorized.
  • The system is organized for the context manager to communicate with various domains of control related to the multimodal document so that it is able to initiate and, depending on how the system is implemented, monitor and/or confirm, the shredding of a multimodal document including renditions and annotations associated with the document. Upon receiving a request or instruction for shredding, a domain of control, such as a server, will attempt to shred the portions of the multimodal document which it processes or controls. The domain of control can communicate back to the system the completion of the shred and/or the status of an attempted shred. After receiving responses to various requests, or expiration of a timeout period, the system informs affected users the status of the shredding of the multimodal document. If the system has been unable to shred the content to the extent requested, these notifications of shred status from the various domains may function as a request for user intervention. If successful, the notifications instead signal the completion of the shredding process to the user.
  • Thus, the system manages documents that exist in both the physical and electronic environment. For any given document managed by the context manager, the shredding process ensures the removal of the document's metadata and all its associated content (annotations and renderings). A rendering is analogous to a copy of the document. Just as copies of a document can exist in a paper form or as an electronic file, renderings have an equal ambivalence to medium. The system proactively manages paper documents, for example, in several ways. First, if a document is deleted, the system will alert users to the deletion event and request that they destroy the paper rendering. Second, the system will monitor user interactions with a paper rendering for an action that represents deletion of that document. On the occurrence of the triggering process, the system will begin a deletion of all content and metadata related to the current document.
  • Reference is now made to FIG. 3. An ANOTO-ized document is created at 52 with a document cover sheet. The document is registered at 54 with the context management system. The document is annotated with ink marks, audio and other media stream segments at 56. This is in accordance with the operation of the various subsystems as described in FIG. 2. When a shredding operation is desired, a check mark or indication is placed next or in the shredding enablement box or area. Here, the box is checked for a personal shredding at 58. The personal shredding enablement may be, for example, as shown in FIG. 4. Other more extensive shredding enablement boxes or areas for various users can be provided depending on the design of the system.
  • Reference is now made to FIG. 4. The document cover sheet referenced in block 52 of FIG. 3 may be as is shown in FIG. 4. The computer view of the cover sheet 60 shows a shredding enablement area shown generally at 62. The shredding enablement area 62 is a personal shredding enablement area activated by the activation of personal shredding box 64. When the box 64 is checked, all electronic references to the user's copy of the document will be deleted and the user will receive a confirmation e-mail once the shredding process is completed. The personal shredding, depending upon how the system is implemented, can also include shredding of all virtual media and physical media that involve the user's use and/or relationship to the multimodal document. Confirmation of the shredding, including physical destruction of any physical media as well as the confirmation of the deletion of all virtual media can also be provided. As previously noted, other more extensive and comprehensive shredding enablement options can be provided, depending on the system. This can be implemented with verification of the authorization of the initiator for each of the different aspects or parts of the document to be shredded. For example, a manager may be allowed and authorized to initiate the shredding of document parts used within the manager's team but not allowed to shred documents outside of the team. An explicit field, for example either in an information and content data file or database (ICDF) at the context manager 4 or in a content management system, can be provided which indicates who is authorized to shred and what they are authorized to shred. The field or another field can also be provided to indicate the ownership or custodian of what is to be shredded and where desired provide this information to other systems.
  • Reference is now made to FIG. 5, showing the operation of the system involved in the shredding process of the specific physical document referenced in FIG. 3. At 66, a document is allowed to be shredded by a requester, as identified, for example, via the owner (user) of a specific ANOTO pen. A shred request is sent to the ANOTO server at 68, including the identification that the shred request was registered to the specific document to be shredded. At block 72, a shred request is sent by the context manager 4 to each of the media servers for which there are media stream segments included in the metadata associated with the document. At block 74, a countdown timer is started and at block 76, responses are awaited from each of the servers to which a shred request was sent by the context manager 4. At block 78, for all failed shreds, that is, a shred that has not been implemented or accomplished, then the requester is notified that there was a problem with that part of the shred operation. At block 80, for all successful shred operations, the associated metadata from the context memory 4 is removed. At block 82, a message is sent to the requester with the state of the shred of the multimodal document requested to be shred. The requester then may physically shred the personal copy of the requester's multimodal document.
  • Reference is now made to FIG. 6, where the process for initiating the shredding involving the complete set of content, as represented, the entire information and content data file, for a multimodal document is initiated. At block 86, a information and content data file or database (ICDF) is created. At block 88, as content is created, it is registered and reference is added to the information and content data file or database and appropriately stored by the context manager 4. The context manager 4 tracks the use of each piece of content at block 90. At block 92, an exhaustive shred is initiated, such as by right clicking the information and content data file for the multimodal document at the context manager 4 and selecting Exhaustive Shred shown in FIG. 8.
  • The tracking the use of each piece of content at 90 may include location tracking of physical items and use tracking, like writing on items, mailing documents (e.g., e-mail sniffers and enhanced postal meters, such as meters with scanners), etc. All tracking information of both physical media and virtual media is sent to and managed by the context manager 4. The tracking methods can employ any of the methods described above and other well known methods useful in tracking and sensing and image recognition, including by use of digital pens, radio frequency identification (RFID), bar codes and numeric fields, electronic document identification (ID) sensing, image recognition via cameras, and the like.
  • Reference is now made to FIG. 7. A computer screen 13 exhibits three open windows, window A, window B, and window C. These windows are examples of the type of information that is gathered, managed by and displayed using the context manager 4. In window A, an information and content data file illustrates and lists all of the virtual media (virtual documents and parts) and all of the physical media (physical documents and artifacts) for a multimodal document. The information and content data file in window A includes three electronic documents 94, 96 and 98. These three electronic documents 94, 96 and 98 are pdf-generated from the original ANOTO document and include all existing annotations at the time of generation. Two physical ANOTO- ized documents 100 and 102 are also listed. These documents are Deb's copy and Jill's copy. Also listed are five virtual copy parts of multimodal document. These copy parts are shown as 104, 106, 108, 110 and 112. It should be recognized that depending upon the nature and use of the multimodal document, other types of listings from various subsystems may exist, such as audio clips and video clips, etc.
  • Window B illustrates a pdf document listed as the second electronic version 94 in Window A. This is document is designated as document 102 a in Window B. Window C illustrates one of the virtual document parts listed in Window A, here designated as 104 a. Various types and formats of displays and organization of the various information related to the creation, tracking and shredding of virtual media and physical media can be employed, depending on the nature and design of the system.
  • Reference is now made to FIG. 8, which provides a complete view of window A shown in FIG. 7 unobstructed by other open screen windows. Here, the complete information and content data file for the multimodal document is shown with a full and complete listing of all of the virtual media (virtual document and parts) and all of the physical media (physical documents and artifacts) at 94 through 112. Additional types of information that can be further included and listed in the information and content data file. A user could initiate an exhaustive shred of the full information and content listed in the information and content data file by activating box 95.
  • Reference is now made to FIG. 9 which shows the operation of the system in the comprehensive shredding process of a multimodal document at a high level and with the exhaustive deletion and destruction of the complete set of content. At block 114, if the content is allowed to be shredded by the requester, as identified via the ownership of the information and content data file. At block 116, a shred request is sent to all the servers and administrators that manage the content items of the document, the context manager and content management systems. This would include all of the systems shown in FIGS. 1 and 2, if that is the particular architecture that is implemented. At block 118, a countdown timer is started and at block 120 the system awaits responses from each server and/or administrator to which a shred request was sent.
  • At block 122, the requester is notified that there was a problem with all failed shreds. For all successful shred operations at block 124, the associated metadata is removed from the context manager memory, storage means 4. A message is sent at block 126 to the requester with the state of the shred operation. At block 128, the requester must explicitly follow up with external parties to ensure that their shred was complete, to the extent that the external parties are not within the domains that are managed by the context manager 2.
  • Reference is now made to FIG. 10, where the context manager 4 orchestrates a comprehensive shredding operation. The context manager 4 receives a request for the comprehensive (exhaustive) shredding of the virtual media and physical media listed in the information and content data file at block 130. This request may have been initiated by the user as indicated in FIG. 8, or may be initiated via some other interface that may be intentionally activated, for example, one similar to the document cover sheet shown in FIG. 4. A determination is made at block 132 if the requester is authorized to remove the content from the information and content data file. Where the requester is not so authorized, the context manager at block 134 discards the request. The request by an unauthorized requester for shredding may be stored, if desired, in the context manager 4 for subsequent audit and/or other activities. Where the requester is authorized to remove the content from the information and content data file at block 132, the context manager 4 creates a record of the request at block 133. Thereafter, all elements of the information and content data file are listed at block 134 if the list has not previously been compiled. This includes clones, artifacts, document parts, and the like. The list of the context created information and content data file includes all virtual media and all physical media if the list has not previously been compiled. At block 136, a list of all current users is created of all of the virtual media and all of the physical media. As noted, the list of elements and users (and any other list) can be created and exist prior to the request for shred being received. The list build can be ongoing and continuous as elements are created or the build can be periodic, that is, hourly, daily, weekly, monthly, etc.
  • At block 138, the context manager 4 sub-groups various elements in the list. This may be based on the content management systems employed, such as the various media servers, the various element inventory administration, users of the multimodal documents, and the like. At block 140, a message is sent to each content management system requesting the deletion and destruction of each element; for example, e-mails sent to persons and the like. At block 142, a response timer is set for each content management system. The response timer and time involved can be unique to the particular content management system and even within subsystems of the particular content management system, depending on the need and applications for the system. At block 144 a list of all items that have been sent out of the system domain is created. At block 146, the requester is notified of all such items including recipients of the items, and at block 148, the requester follows up on those items using traditional methods, e.g., phone mail, e-mail, and other appropriate methods to ensure the deletion or destruction of the virtual media and physical media. The process steps 144, 146 and 148 are for those portions of multimodal document that may not or are not under the management of the context manager 4 and need separate, independent operations to ensure their deletion and/or destruction. Thus, the context manager 4 manages not only systems within the direct or indirect control of the context manager system 4 but also related multimodal document virtual media and physical media related to such other materials that need to be independently processed when that information is available to the context manager 4.
  • Reference is now made to FIG. 11, where the context manager 4 is awaiting a response from the various subsystems for which a shred request was issued. The context manager at block 150 awaits the next response message from the content management servers and/or users. For each element included at block 152, a determination is made as to whether the requested shred request was accomplished. If the shred request was accomplished, the elements response trigger is disabled at block 154 and the element is removed from the information and content data file list. At block 155 related context is removed. A determination is then made at block 156 if the information and content data file list is empty. Where the information and content data is not empty, the system loops back to block 150. Where, at block 152, the shred request was not accomplished, a determination is made at block 158 for the specific element that was not shredded whether the maximum number of retries have been attempted. The number of tries is a matter of design choice and can vary for each type of element to be shredded. Where the maximum number of retries have been attempted, the element is marked as a failed shred attempt at block 160. Where the maximum number of retries have not been attempted, at block 162 a request is sent to the context manager 4 to initiate another attempt to shred the same element. Where at block 156 a determination is made that the information and content data file is empty, at block 164 a message, such as an e-mail, is sent to the person who requested the comprehensive deletion/destruction request, the shred request, notifying the requester that the requested shred was successful.
  • Reference is now made to FIG. 12, where the absence of an expected response from each content management system which was issued a shred instruction is tracked by the context manager 4. At block 166, an element response trigger fires. This is when the time allotted to receive a response that a shred request has been accomplished has passed. A determination is then made at block 168 if the maximum number of retries have been attempted. Where this is not the case, at block 170 a request is again sent by the context manager 4 to the content server for the element to delete and/or destroy, shred, the element. When the maximum number of retries have been attempted, the element is marked as failed at block 172. The element is thus marked as an element of the multimodal document that has failed to be shredded on a shred request. A determination is then made at block 174 whether the list contains only failed items. Where the list does contain only failed items, a message is sent at block 176 to the person who requested the comprehensive shredding, deletion and/or destruction request, notifying that the request was not successful, including the list of elements that failed. Where the list does not contain only failed items, the system loops back to block 166.
  • While the present invention has been disclosed and described with reference to a single embodiment thereof, it will be apparent, as noted above that variations and modifications may be made therein. It is, thus, intended in the following claims to cover each variation and modification that falls within the true spirit and scope of the present invention.

Claims (25)

1. A multimodal document management system, comprising:
a context manager for managing multimodal documents;
a virtual media content management system, said virtual media content management system coupled to said context manager;
a physical media content management system, said physical media content management system coupled to said context manager; and,
said context manager operable to issue multimodal document management instructions to said virtual media content management system and to said physical media content management system.
2. A multimodal document management system as defined in claim 1, further comprising:
a second virtual media content management system, said virtual media content management system coupled to said context manager; and,
said first and said second virtual media content management systems each operating in different domains of control.
3. A multimodal document management system as defined in claim 1, further comprising:
a second physical media content management system, said physical media content management system coupled to said context manager; and,
said first and said second physical media content management systems each operating in different domains of control.
4. A multimodal document management system as defined in claim 1, further comprising:
a second virtual media content management system, said virtual media content management system coupled to said context manager;
said first and said second virtual media content management systems each operating in different domains of control;
a second physical media content management system, said physical media content management system coupled to said context manager; and,
said first and said second physical media content management systems each operating in different domains of control.
5. A system multimodal document management as defined in claim 1, where said virtual media content management system is interactively coupled to said context manager and said physical media content management system is interactively coupled to said context manager such that said virtual media content management system and said physical media content management system can interactively communicate with said context manager.
6. A multimodal document management system as defined in claim 5, further comprising:
a virtual media shredding subsystem coupled to said virtual media content management system and a physical media shredding subsystem coupled to said physical media content management system.
7. A multimodal document management system as defined in claim 6, wherein said virtual media shredding subsystem is operable under control of said virtual media content management system to delete virtual media upon instructions from said context manager to said virtual media content management system and wherein said physical media shredding subsystem is operable under control of said physical media content management system to destroy physical media upon instructions from said context manager to said physical media content management system.
8. A multimodal document management system as defined in claim 7, wherein said context manager receives communications from said virtual media tracking system concerning the status of virtual media related to multimodal documents and receives communications from said physical media tracking system concerning the status of physical media related to said multimodal documents.
9. A multimodal document management system as defined in claim 8, wherein said context manager creates information and content data files concerning multimodal documents.
10. A multimodal document management system as defined in claim 9, wherein said context manager information and content data files for a multimodal document includes a list of virtual media content relating to said multimodal document and of physical media content relating to said multimodal document.
11. A multimodal document management system as defined in claim 10, wherein said information and content data file is employed in identifying elements of said multimodal document in issuing shredding instructions.
12. The multimodal document management system as defined in claim 11, wherein said shredding instructions issued by said context manager to virtual media content management system and to said physical media content management system include personal shredding instructions involving the shredding of all multimodal document element content in said information and content data file relating to a particular user.
13. The multimodal document management system as defined in claim 11, wherein said shredding instructions involving exhaustive shredding of all multimodal document element content in said information and content data file.
14. A multimodal document management system as defined in claim 12, wherein said context manager subsequent to the shredding of all said personal virtual media relating to a multimodal document and all said personal physical media relating to a multimodal document deletes all personal virtual media data relating to said multimodal document stored in memory means for said context manager.
15. A multimodal document management system as defined in claim 13, wherein said context manager subsequent to the exhaustive shredding of all said virtual media relating to a multimodal document and all said physical media relating to a multimodal document deletes all virtual media data relating to said multimodal document stored in memory means for said context manager.
16. A multimodal document management system as defined in claim 11, wherein said multimodal document includes physical media including physical documents and virtual media including virtual documents relating to said physical documents.
17. A multimodal document management system as defined in claim 16, wherein said multimodal document virtual media further includes audio clip media relating to said physical documents.
18. A method for managing multimodal documents, comprising the steps of:
receiving a request for a shred of elements in an information and content data file containing a list of virtual media and physical media elements relating to a multimodal document;
determining if the request for shred is authorized and, where said request for shred is authorized, creating a record of said shred request;
grouping said shred requested elements in said element list based on each element in said which are managed by the same content management system; and,
sending a message to each content management system requesting the shredding of each element in each grouping of elements managed by said same content management system.
19. A method for managing multimodal documents as defined in claim 18, comprising the further step of setting a response timer for receiving a response to said shredding request message from each content management system to which a shredding request is sent.
20. A method for managing multimodal documents as defined in claim 18, comprising the further step of:
awaiting a response from said content management systems;
determining if the shred request was accomplished from each content management system response;
disabling an element response trigger and removing each element from said list where a response is received from said content management system that said shred request was accomplished, where said element response trigger is triggered by a failure to receive said response; and,
if said information and content data file list elements have been successfully shredded, sending a message to said requester that said shred request was successful.
21. A method for managing multimodal documents as defined in claim 20, comprising the further step of repeating said steps beginning with said awaiting a response from said content management systems if said information and content data file have not been successfully shredded.
22. A method for managing multimodal documents as defined in claim 21, comprising the further steps of,
determining if a predetermined maximum number of retries have been attempted for shred requested elements where said element response trigger has fired when a response period has passed for a content management server to respond; and,
where said maximum number of retries attempts have been implemented for shred requested elements, marking said element as having failed to be shredded; and,
notifying said requester that said element has failed to be shredded.
23. A method for managing multimodal documents as defined in claim 18 wherein said request for a shred of elements in an information and content data file containing a list of virtual media and physical media elements relating to a multimodal document is a request for a comprehensive shred of all said elements of virtual media and physical in said information and content data file list.
24. A multimodal document management system, comprising:
a context manager for managing multimodal documents;
a first and a second virtual media content management system, said first and said second virtual media content management system coupled to said context manager, said first and said second virtual media content management systems each operating in different domains of control;
a first and a second physical media content management system, said first and said second physical media content management system coupled to said context manager, said first and said second physical media content management systems each operating in different domains of control;
said context manager operable to issue multimodal document management shredding instructions to said first and said second virtual media content management system and to issue shredding instructions to said first and said second physical media content management system; and,
said first and said second virtual media content management system, in response to shredding instruction received from said context manager, operable to initiate the process of shredding of virtual media; and,
said first and said second physical media tracking system, in response to shredding instruction received from said context manager, operable to initiate the process of shredding of physical media.
25. A multimodal document management system as defined in claim 24, wherein said context manager receives communications from said first and said second virtual media tracking system concerning the status of virtual media shredding related to multimodal documents and receives communications from said first and said second physical media tracking system concerning the status of physical media shredding related to said multimodal documents.
US11/846,109 2007-03-30 2007-08-28 Systems and methods for managing multimodal documents Abandoned US20080244385A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US11/846,109 US20080244385A1 (en) 2007-03-30 2007-08-28 Systems and methods for managing multimodal documents

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US90927307P 2007-03-30 2007-03-30
US90928107P 2007-03-30 2007-03-30
US11/846,109 US20080244385A1 (en) 2007-03-30 2007-08-28 Systems and methods for managing multimodal documents

Publications (1)

Publication Number Publication Date
US20080244385A1 true US20080244385A1 (en) 2008-10-02

Family

ID=39792316

Family Applications (2)

Application Number Title Priority Date Filing Date
US11/846,109 Abandoned US20080244385A1 (en) 2007-03-30 2007-08-28 Systems and methods for managing multimodal documents
US11/846,160 Expired - Fee Related US9275027B2 (en) 2007-03-30 2007-08-28 Method and system for collaborative capture and replay of digital media files using multimodal documents

Family Applications After (1)

Application Number Title Priority Date Filing Date
US11/846,160 Expired - Fee Related US9275027B2 (en) 2007-03-30 2007-08-28 Method and system for collaborative capture and replay of digital media files using multimodal documents

Country Status (1)

Country Link
US (2) US20080244385A1 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080059212A1 (en) * 2006-08-31 2008-03-06 Andrei Obrea System and method for assembling complex document sets from geographically disparate sources
US20090113351A1 (en) * 2007-10-29 2009-04-30 Kabushiki Kaisha Toshiba Document management system, document management method and document management program
US20090205043A1 (en) * 2008-02-12 2009-08-13 Fuji Xerox Co., Ltd. Information processing system, information processing method, and computer readable medium
US20160026263A1 (en) * 2014-07-25 2016-01-28 Rovio Entertainment Ltd Physical surface interaction
US9325672B2 (en) * 2014-04-25 2016-04-26 Cellco Partnership Digital encryption shredder and document cube rebuilder
US20180077313A1 (en) * 2016-09-15 2018-03-15 Accenture Global Solutions Limited Document data processing including image-based tokenization
US10061980B2 (en) 2015-08-20 2018-08-28 Accenture Global Services Limited Digital verification of modified documents
US11409892B2 (en) * 2018-08-30 2022-08-09 International Business Machines Corporation Enhancing security during access and retrieval of data with multi-cloud storage

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008150919A1 (en) * 2007-05-29 2008-12-11 Livescribe, Inc. Electronic annotation of documents with preexisting content
US8438489B2 (en) * 2008-01-24 2013-05-07 Paulo Barthelmess System and method for document markup
US8510646B1 (en) * 2008-07-01 2013-08-13 Google Inc. Method and system for contextually placed chat-like annotations
US9350817B2 (en) * 2009-07-22 2016-05-24 Cisco Technology, Inc. Recording a hyper text transfer protocol (HTTP) session for playback
US8504624B2 (en) * 2009-09-08 2013-08-06 Ricoh Co., Ltd. Stroke and image aggregation and analytics
US9524282B2 (en) * 2013-02-07 2016-12-20 Cherif Algreatly Data augmentation with real-time annotations
US9430141B1 (en) * 2014-07-01 2016-08-30 Amazon Technologies, Inc. Adaptive annotations
US10228775B2 (en) * 2016-01-22 2019-03-12 Microsoft Technology Licensing, Llc Cross application digital ink repository
US10817169B2 (en) 2016-10-14 2020-10-27 Microsoft Technology Licensing, Llc Time-correlated ink
US11048864B2 (en) * 2019-04-01 2021-06-29 Adobe Inc. Digital annotation and digital content linking techniques

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5243149A (en) * 1992-04-10 1993-09-07 International Business Machines Corp. Method and apparatus for improving the paper interface to computing systems
US5897643A (en) * 1995-04-20 1999-04-27 Fuji Xerox Co., Ltd. System for maintaining a record of documents including document copies
US6027026A (en) * 1997-09-18 2000-02-22 Husain; Abbas M. Digital audio recording with coordinated handwritten notes
US6148312A (en) * 1997-02-12 2000-11-14 Synopsys, Inc. Method and system for electronic article maintenance and deletion
US6590837B1 (en) * 1999-05-21 2003-07-08 Lawrence M. Kaplan Apparatus and method for annotating an object with an audio message
US20040221014A1 (en) * 2002-11-26 2004-11-04 Tomkow Terrence A. System for, and method of, authenticating an electronic message to a recipient
US6839707B2 (en) * 2001-01-17 2005-01-04 General Electric Company Web-based system and method for managing legal information
US20050181819A1 (en) * 2003-12-18 2005-08-18 Satoshi Washio Communication system, communication terminal device, and information storage module
US7227872B1 (en) * 2002-06-28 2007-06-05 Cisco Technology, Inc. Mechanisms for providing stateful NAT support in redundant and asymetric routing environments
US20070168870A1 (en) * 2005-11-15 2007-07-19 Kabushiki Kaisha Toshiba Content management system and content management method
US20080163364A1 (en) * 2006-12-27 2008-07-03 Andrew Rodney Ferlitsch Security method for controlled documents
US20080276168A1 (en) * 2006-10-13 2008-11-06 Philip Andrew Mansfield Method, device, and graphical user interface for dialing with a click wheel

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7609862B2 (en) * 2000-01-24 2009-10-27 Pen-One Inc. Method for identity verification
US7551300B2 (en) 2005-06-17 2009-06-23 Pitney Bowes Inc. System and method for controlling the storage and destruction of documents

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5243149A (en) * 1992-04-10 1993-09-07 International Business Machines Corp. Method and apparatus for improving the paper interface to computing systems
US5897643A (en) * 1995-04-20 1999-04-27 Fuji Xerox Co., Ltd. System for maintaining a record of documents including document copies
US6148312A (en) * 1997-02-12 2000-11-14 Synopsys, Inc. Method and system for electronic article maintenance and deletion
US6027026A (en) * 1997-09-18 2000-02-22 Husain; Abbas M. Digital audio recording with coordinated handwritten notes
US6590837B1 (en) * 1999-05-21 2003-07-08 Lawrence M. Kaplan Apparatus and method for annotating an object with an audio message
US6839707B2 (en) * 2001-01-17 2005-01-04 General Electric Company Web-based system and method for managing legal information
US7227872B1 (en) * 2002-06-28 2007-06-05 Cisco Technology, Inc. Mechanisms for providing stateful NAT support in redundant and asymetric routing environments
US20040221014A1 (en) * 2002-11-26 2004-11-04 Tomkow Terrence A. System for, and method of, authenticating an electronic message to a recipient
US20050181819A1 (en) * 2003-12-18 2005-08-18 Satoshi Washio Communication system, communication terminal device, and information storage module
US20070168870A1 (en) * 2005-11-15 2007-07-19 Kabushiki Kaisha Toshiba Content management system and content management method
US20080276168A1 (en) * 2006-10-13 2008-11-06 Philip Andrew Mansfield Method, device, and graphical user interface for dialing with a click wheel
US20080163364A1 (en) * 2006-12-27 2008-07-03 Andrew Rodney Ferlitsch Security method for controlled documents

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Microsoft Computer Dictionary: Fifth Edition," Microsoft Press, 2002. *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080059212A1 (en) * 2006-08-31 2008-03-06 Andrei Obrea System and method for assembling complex document sets from geographically disparate sources
US20090113351A1 (en) * 2007-10-29 2009-04-30 Kabushiki Kaisha Toshiba Document management system, document management method and document management program
US8230365B2 (en) * 2007-10-29 2012-07-24 Kabushiki Kaisha Kaisha Document management system, document management method and document management program
US20090205043A1 (en) * 2008-02-12 2009-08-13 Fuji Xerox Co., Ltd. Information processing system, information processing method, and computer readable medium
US9325672B2 (en) * 2014-04-25 2016-04-26 Cellco Partnership Digital encryption shredder and document cube rebuilder
US20160026263A1 (en) * 2014-07-25 2016-01-28 Rovio Entertainment Ltd Physical surface interaction
US9639159B2 (en) * 2014-07-25 2017-05-02 Rovio Entertainment Ltd Physical surface interaction
US10061980B2 (en) 2015-08-20 2018-08-28 Accenture Global Services Limited Digital verification of modified documents
US20180077313A1 (en) * 2016-09-15 2018-03-15 Accenture Global Solutions Limited Document data processing including image-based tokenization
US10116830B2 (en) * 2016-09-15 2018-10-30 Accenture Global Solutions Limited Document data processing including image-based tokenization
US11409892B2 (en) * 2018-08-30 2022-08-09 International Business Machines Corporation Enhancing security during access and retrieval of data with multi-cloud storage

Also Published As

Publication number Publication date
US20080236904A1 (en) 2008-10-02
US9275027B2 (en) 2016-03-01

Similar Documents

Publication Publication Date Title
US20080244385A1 (en) Systems and methods for managing multimodal documents
US20200356529A1 (en) Information Source Agent Systems and Methods for Distributed Data Storage and Management Using Content Signatures
US6804687B2 (en) File system management with user-definable functional attributes stored in a token action log
CA2705379C (en) Systems and methods for creating copies of data, such as archive copies
Smetters et al. How users use access control
CN101554010B (en) Systems and methods for document control using public key encryption
US20100306176A1 (en) Deduplication of files
US20120185505A1 (en) Methods and computer program products for accelerated web browsing
US20080294899A1 (en) Secure management of document in a client-server environment
WO2003040963A1 (en) Knowledge management system
JP2011081842A (en) Managing data object in dynamic, distributed and collaborative context
EP1066561A4 (en) Document management extension software
JP6601772B2 (en) Personal information management system
Mallery Secure file deletion: Fact or fiction?
Phillips Do electronic objects create business risk?
JP2006318037A (en) Life cycle management system
Plagman et al. Image processing
Stephens The Daily Management of Records and Information
JP2007094476A (en) Electronic article collection system
Seward The Debtor's Digital Autopsy or Where's the Money!
Kahn et al. If the Slate Is Wiped Clean: Spoliation: What It Can Mean for Your Case
Lubarsky Deposition Questions for an IT Person
BIGLER EDPACS
Tennant et al. NYSBA
JP2009003558A (en) Settlement information management device and program

Legal Events

Date Code Title Description
AS Assignment

Owner name: PITNEY BOWES INC., CONNECTICUT

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZUKOWSKI, DEBORRA J., MS.;FRIEDRICH, ERIC C., MR.;PARKOS, ARTHUR J., MR.;AND OTHERS;REEL/FRAME:019756/0925;SIGNING DATES FROM 20070802 TO 20070820

STCB Information on status: application discontinuation

Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION