WO1994001824A1

WO1994001824A1 - A single chip integrated circuit system architecture for video-instruction-set-computing

Info

Publication number: WO1994001824A1
Application number: PCT/US1993/005863
Authority: WO
Inventors: Venson M. Shaw; Steven M. Shaw
Original assignee: Shaw Venson M; Shaw Steven M
Priority date: 1992-07-06
Filing date: 1993-06-17
Publication date: 1994-01-20
Also published as: US5457780A; GB2284525B; GB9500137D0; CA2139660C; AU4768693A; KR950702723A; GB2284525A; AU677791B2

Abstract

The present invention pertains to integrated circuit system based on novel architecture of Video-Instruction-Set-Computing (VISC). The integrated circuit comprises a plurality of functional units to independently execute the tasks of remote communication, bandwidth adaptation, application control, multimedia management, and universal video encoding. The integrated circuit is also comprised of scalable formatter element connecting to the functional units which can inter-operate arbitrary external video formats and intelligently adapt to selective internal format depending upon the system throughput and configuration. Additionally, there is a smart memory element connecting to the functional units and scalable formatter, which can access, store, and transfer blocks of video data based on selective internal format. In the preferred embodiment, the integrated circuit is also comprised of an embedded RISC or CISC co-processor element in order to execute DOS, Windows, NT, Macintosh, OS2, or UNIX applications. In a more preferred embodiment, the integrated circuit includes a real time object oriented operation system element wherein concurrent execution of the application program and real time VISC based video instruction sets can be performed. The present invention is designed to sustain the evolution of a plurality of generations of the VISC microprocessors. These novel VISC microprocessors can be efficiently used to perform wide range of real time distributed video signal processing functions for applications such as interactive video, HDTV, and multimedia communications.

Description

A SINGLE CHIP INTEGRATED CIRCUIT SYSTEM ARCHITECTURE

FOR VIDEO-INSTRUCTION-SET-COMPUTING

FIELD OF THE INVENTION

The present invention is related to an integrated circuit system based on a novel architecture entitled Video-Instruction-Set -Computing (VISC) . More specifically, the present invention not only provides the core functions for a initial single chip realiz¬ ation, it can also be evolved into several generations of scalable high performance microprocessors. In particular, these novel VISC microprocessors can efficiently perform broad range of real time distributed video signal processing functions for applications such as interactive video, HDTV , and multimedia communications.

BACKGROUND OF THE INVENTION

Video signal processing, of motion and still image informa¬ tion, represents a critical functional component for many emerging computing systems. All of the prior video signal processing techni¬ ques, that have been proposed or implemented, employ a single or plurality of special purpose coprocessors based on the more tradi¬ tional CISC or RISC computing principle. Consequently, these CISC/ RISC host coprocessors can only partially improve the performance of specific subsystems, such as video encoding for multiple algori-

SUBSTITUTE SHEET th s, high speed frame memory retrieval, and dynamic display management. The significant advantage is however, their ability to adapt with all existing DOS, WINDOW, or UNIX program data structur

Though practical, the speed and performance of these techniques are severely limited by the system throughput, the mismatch of data types, and the rigid CISC and RISC processor and memory system architecture, which have been designed to optimize the performance for text or graphics data types, but inefficient for real time interactive video processing. For example, see U.S. Patent 4,777,620 to Shimoni, U.S. Patent 4,772,946 to Hammer, U.S. Patent 4,7227,589 to Hirose, and U.S. Patent 4,398,256 to Nussmier. Typically, these special purpose host coprocessors would be imple¬ mented either as digital signal processing or custom application specific integrated circuit (custom ASICs) .

While the aforesaid patents teach various methods and apparatus for compressing and decompressing video data, improve frame memory subsystem performance, and enhancing the image quality of the display data. None of the aforesaid patents have ever directed themselves to the concept and structure of an effective and generalized system architecture, which would prioritize the complex video data types, and optimize performance for video signal processing, while the traditional CISC or RISC application programs can still be efficiently performed.

This novel method and apparatus would interconnect all data processors among consumer, communications, and desktop

SUBSTITUTE SHEET computing, allows for individuals to select and convey multiple forms of information such as sound, image, graphics,- data, and live video, automatically adjust to the available bandwidth, and capable of communicating in multiple bandwidths.

More specifically, although prior arts have shown CISC and RISC can be extremely suitable for dedicated desktop computing in processing the traditional text and graphics data types. None of the aforesaid patents have directed themselves to the concept and structure of further broadening the scope, and to develop a new computing platform. This new platform would not only interconnect the traditional desktop data processors such as computers and workstations, but it would also interconnect television , VCR's, CD player, cameras, multimedia sensors, or any other consumer and communications data processors in a totally integrated environment. Consequently, in this novel integrated computing environment, complex video data types declare much higher priority, and require much higher run-time performance as comparing to the traditional text and graphics data.

OBJECTS OF THE INVENTION

An object of the present invention is to define a integrated computing architecture which can accommodate communica¬ tions, both transmission and retrieval, of all digitally-coded or algorithmic complex video data types.

Another object of the invention is to provide a novel

SUBSTITUTESHEEi integrated system architecture which is flexible and allows the control and communications among television, VCR's, CD player, cameras, sensors, or any other consumer and communications data processors, as well as the desktop data processors such as computer and workstations.

A still further object of the present invention is to provide for a novel process architecture which not only allows for digital coding techniques, but also can interface with traditional analog storage or transmission techniques.

A still further object of the present invention is to provide for a novel process architecture which allows the human users to interface with application program, and to select the appropriate media combination either before or during the communication session.

A still further object of the present invention is to provide for a novel process architecture which not only allows for an optimized system performance for complex video data types, but also can directly execute traditional desktop application programs using a CISC or RISC application coprocessor.

A still further object of the present invention is to provide for a novel process architecture which allows for an object-oriented real time operating system for complex video data types, and would accommodate traditional UNIX, DOS, or other tradi¬ tional desktop operating systems.

SUBSTITUTESHEET Further objects and advantages of the present invention will become apparent from a consideration of the drawings and ensuing description of it.

SUMMARY OF THE INVENTION

Our present invention, VISC architecture, offers a totally new computing platform. Distinguish from all prior arts which have employed the traditional CISC or RISC computing discipline, VISC provides new methods and apparatus to organize a plurality of complex video data types, VISC also streamline, optimize and preschedule the video instruction clusters, and provide parallel or pipeline execution for these instructions. VISC also facilitates a CISC or RISC application coprocessor to conveniently process the traditional DOS or UNIX applications. Finally, VISC provides a distributed object oriented operating system facilities which can provide concurrent execution of traditional DOS or UNIX operating systems with the real time VISC video signal processing functions.

FIG. 2 shows the architecture principle of VISC (Video- Instruction-Set-Computing) . Contrary to a traditional RISC or CISC architecture, the data processors and memory system are completely optimized to facilitate block oriented data instead of the tradi¬ tional bit-oriented data streams. To be more specific, data infor¬ mation are organized according to selective internally reconfigur- able block format, wherein these internal format can accommodate various processor configurations as well as data throughput.

SUBSTITUTESHEET A scalable smart memory system architecture and memory management unit also provide programmable data block addressing, frame memory management, block data manipulation, and associative block search.

In addition, VISC instruction sets can be readily compiled into a set of system look-up tables (SLUT's) . Based upon the run time bandwidth constraints caused by either network congestion or application request, the VISC instruction processors can prefetch a group of such VISC instructions, and designate them with various functional units for parallel or pipeline execution. Comparing to the more traditional cache memory techniques widely used in the existing RISC or CISC computer, The SLUT technique employes intelligent fast associative search schemes, and is able to perform scheduling, compilation, assembling, and simultaneously issuing instructions for execution and memory management.

In the VISC architecture, a smart memory system is connected to the functional units and scalable formatter, which can access, store, and transfer blocks of video data based on the selective internal format.

In FIG. 4, VISC architecture also illustrate an embedded RISC or CISC coprocessor (COP) element in order to directly execute the bit oriented application programs in DOS, Window, NT, Macintosh, OS2, or UNIX. In a more preferred embodiment, a VISC can include a real time object oriented operation system wherein concurrent execution of the application program and real time VISC based video instruction sets can be performed.

SUBSTITUTE SHEET VISC architecture provides a single computing platform to perform a plurality of complex data types including motion video, voice, data, still image, and animated graphics. Consequent¬ ly, it becomes feasible to integrate digital television, computer, cameras, 104 with human users and traditional application program.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. l shows the pictorial illustration of a video instruction set computing (VISC) system environment.

FIG. 2 shows the core VISC integrated circuit system architecture in accordance with the present invention.

FIG. 3 illustrates the major functional operations for a VISC (video instruction set computing) integrated circuit in accordance with the present invention.

FIG. 4 is a detailed block diagram illustrating a single chip VTSC integrated circuit according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

1. GENERAL DISCUSSION

Referring now to the drawings wherein like reference numerals refers to similar or identical parts throughout the

TUTE SHEET several views, and more specifically to FIG.l thereof, there is shown a pictorial representation of an novel integrated circuit VISC 112 (video instruction set computing) system apparatus, there is also shown a pictorial illustration depicting most of the popular electronic apparatus relating to computer 108, communica¬ tions, and consumers, presently available for the homes or offices. These include a VCR 102, CD Player, 104, television , personal computer 108, and fax machine.

It is Applicant's intention to disclose a unified system method towards integrated design of all future generations'inter¬ active television, video communicator, and visual computing system.

It is also Applicant's intention to illustrate the architecture design of the VISC apparatus 112 according to this unified system design method.

Furthermore, The VISC 112 system apparatus allows for compatibility with all existing electronic apparatus. VISC user/ operator can control, complement, and utilize the functions of each electronic apparatus by means of the VISC 112 system apparatus. The VISC apparatus 112, being of compact size and shape, similar to that of a VCR 102, notebook PC, remote controller 116, or smaller, can locally integrate all existing electronic apparatus, and permit them to function complimentary with each other.

It is yet another Applicant's intention to substantiate a distributed system architecture for VISC 112, in which a plurality

SUBSTITUTESHEET of VISC's apparatus 112 can remotely communicate with each other and can also communicate with other non-VISC apparatus, regardless of whether other apparatus re analog, digital or algorithmic, and to encode or decode automatically to the available bandwidth, in a totally integrated system environment.

2. GENERAL INTRODUCTION OF VISC

FIG.2 illustrates the core system architecture, operation, and methodology for a single chip design and implementation of VISC 112 integrated circuit . This VISC 112 microprocessor apparatus would make it possible to exchange a multitude of different forms of video articles over a wide range of communications networks. Prior arts have shown methods and apparatus to improve compression and decompression techniques for individual video coding algorithm and individual bandwidth ranges. However, since video coding algo¬ rithms are intrinsically incompatible with each other, there is a need for an apparatus to provide common interface whereby incompat¬ ible equipment can freely exchange video objects through interfac¬ ing with such apparatus.

The diagrammatic representation illustrated in FIG.2 comprises the following major system components. They are a preprocessing/motion processor (PREM) 202, a postprocessor (POST) 222, a bandwidth processor (BAND) 204, a formatting processor (FORM) 206, a encoding processor (ENC) 208, a packet processor (PACK) 210, a smart memory (SMART) 216, a transmission processor (TX) 212, a receiving processor (RX) 214, a decoding processor

SUBSTITUTE SHEET (DEC) 220, and a system controller (CON) 218.

The PREM 202 (preprocessing/motion processing) integrated circuit is able to capture, preprocess, differentiate, and generate a motion vector 302 signal either for a sequential input frames of live motion video, still photo image, or animated bit¬ mapped graphic files. Referring to FIG.2, The PREM 202 integrated circuit is further comprised of a video input capturer and a graphics input bit-mapper adapting elements. The video input capturer produces a digital video signal corresponding to any external video input conforming to RS-170, NTSC 246, PAL, or SCAM video formats. The graphics input bit-mapper, on the other hands, produces a digital signal corresponding to any animated graphics or still image input files conforming to PCX, GIF, EPS 248, TIFF, or alike popular file formats. The PREM 202 is also comprised of a memory element connecting to these input adaptors for receiving, storing, and transferring these digital video or bit-mapped graphics input signal. In a preferred embodiment, the PREM 202 integrated circuit is further comprised of a processor element which produces a differential frame signal and a motion vector signal corresponding to the sequential input frames of motion video, still image, or animated graphic files, the preprocessor element produces a digital signal blocks conforming to the CCITT CIF or SIF video formatting standard. Both the processor element and the memory element can be specifically designed to optimize the performance of transferring, storage, retrieval, and processing of these CCITT CIF/SIF compatible digital signal blocks. In a more preferred embodiment, the PREM 202 is further comprised of a single

SUBSTITUTE SHEE I or plurality of integrated sensor element, the sensor produce the required inputs corresponding to the energy it received, the processor and memory element can efficiently perform similar analog and digital functions corresponding to other non-VTSC foreign data inputs including but not limited to text, data, and audio.

The BAND 204 (bandwidth processor) integrated circuit is able to compute the required communication bandwidth for a local or remote digital video signal and generate a list of run-time attributes for the appropriate compression ratio, frame rate, and display resolution. The BAND 204 integrated circuit is also able to sensitize run-time networking traffic conditions, and dynamically reconfigurate the aforementioned run-time attributes corresponding to the available run-time communication bandwidth. The BAND 204 integrated circuit is further able to sensitize user input or application-specific requirements 356 and interactive update the aforementioned run-time attributes. Preferably, the BAND 204 integrated circuit is further able to exchange a variety of digital encoded input and output foreign video signals corresponding to intrinsically incompatible video coding algorithms whereby incompatible transmission, storage, retrieval, and display apparatus can inter-operate through such interface.

The BAND 204 integrated circuit is comprised of a look- ahead-pipeline processor element connected to the PREM 202 and RX (receiving processor) integrated circuit, which receives a local or remote inbound differential video signal and motion vector 302, it then calculate and produce a corresponding run-time attributes

SUBSTITUTESHEE1 signal. The BAND 204 integrated circuit is also comprised of a controller element connected to the FORM 206 (format processor) ,

ENC 208 (encoding processor) and CON 218 (system controller) integrated circuits, which receives a set of initial run-time attributes according to the algorithmically pre-determined default parameters retaining within the SLUT 234 (system look-up-table) .

Provided said video signal is requested by the CON 218 for outbound transmission. The BAND 204 integrated circuit is further comprised of a sensitizing 342 circuit connected to the TX 212 (transmission processor) , which can intelligently analyze a plurality of networking traffic conditions, and dynamically reconfigurate the run-time attributes corresponding to the available communication bandwidth. The aforesaid sensitizing 342 circuit will first initiate a request signal sending to the TX 212 along with the required communication bandwidth data, it will then either receive a grant signal from the TX 212 provided the network condition is sufficient, or TX 212 will issue a run-time bandwidth allowance signal to BAND 204 integrated circuit corresponding to the realistic network traffic condition, the BAND 204 integrated circuit will further produce a request for reset signal to the FORM

206, ENC 208, and CON 218 integrated circuits in order to reset the

SLUT 234 table, and to reconfigurate the appropriate run-time attributes. In a more preferred embodiment, the BAND 204 integrated circuit is further comprised of a user/application interface element connected to the FORM 206, ENC 208, and CON 218 integrated circuit which receives a plurality of signals regarding user/ operator preference or application-specific requirements 356, it then send these signals to the processor element and dynamically

SUBSTITUTE SHEET produce a plurality of optional run-time attributes, the BAND 204 integrated circuit will further interact with the CON 218, FORM

206, and ENC 208 integrated circuits and to finalize the run-time attributes corresponding to the available communication bandwidth.

In a further preferred embodiment, the BAND 204 integrated circuit will comprised of a interoperating circuit connected to the

RX 214 and TX 212 integrated circuit in receiving a inbound video signal from RX 214 according to a SLUT 234 encoded video coding format, it then reset the SLUT 234 parameters and translate the inbound signal into a outbound signal according to another video coding standard, it further reform the outbound signal and send to TX 212 for further transmission.

The FORM 206 (formatting processor) integrated circuit is able to statically compute the processing and storage bandwidth requirement corresponding to the run-time attribute lists of a digital input video signal, and translate them into an unified internal data format structure according to the run-time integrated circuit processing capabilities. The FORM 206 integrated circuit is also able to statically schedule, optimize, and produce the appropriate instruction and task sequences, then dynamically parse and partition them into a plurality of continuous signaling bitstreams for the fine grained pipeline or parallel encoding or decoding operations of the input video signal. Preferably, the FORM 206 integrated circuit is further able to provide multi¬ dimensional rotation, shifting 360, preprocessing, and retrieval of the CIF or SIF compatible input signal blocks. The FORM 206 is further able to dynamically invoke system calls and look-up and

HEET reconstruct a run-time internal format corresponding to a plurality of system clock and SLUT 234 parameters options, external network¬ ing conditions, and interactive user application requirement 356.

The FORM 206 integrated circuit is comprised of a interpo¬ lating and extrapolating processor element connected to the BAND 204 integrated circuit, which receives list of run-time attributes corresponding to a local or remote differential video signal and motion vector 302, and statically look-up and formulate an internal data structure according to the pre-assigned system clock and SLUT 234 parameters. The FORM 206 integrated circuit is also comprised of a prescheduler connected to the ENC 208 and DEC 220 integrated circuit, which optimize, partition, and produce a coherent flow of instruction and task bitstreams for the required fine grained pipeline or parallel encoding and decoding operations. In a more preferable embodiment, the FORM 206 is further comprised of a dynamic data format processor, which receives incoming data signals and perform multi-dimensional access, retrieval, rotating 358, shifting 360, and preprocessing according to the internally formatted data, the BAND 204 is further comprised of a dynamic program scheduler and optimizer connected to BAND 204 integrated circuit, which receives alert signal and further reform the data signals corresponding to user application inputs and external networking conditions, in a further preferable embodiment, the FORM 206 is further comprised of a scaling circuit connected to the SMART 216 (smart memory) integrated circuit, which invoke system calls and dynamically adjust system clock rate, aspect ratio, and SLUT 234 parameters in providing a linearly scalable VISC system.

SUBSTITUTESHEE1 The ENC 208 (encoding processor) integrated circuit is able to encode a sequence of internally formatted input still or motion video signal and translate them into a bitstream of tokens corresponding to a plurality of pixel or frequency domain encoding algorithms. The ENC 208 is also able to encode the selective pixel or frequency domain algorithm at either macroblock, group of block, partial frame, or whole frame image level in order to achieve the selective VISC 112 system throughput. Preferably, the ENC 208 is further able to encode the algorithms employing additional external coprocessor elements. The ENC 208 integrated circuit is comprised of a instruction or task queuing circuit connected to the FORM 206, which receive and decode the prescheduled instruction task sequences for an entire encoding operation. The ENC 208 integrated circuit is also comprised of a pixel domain encoder and a frequency domain encoder circuits connected to the FORM 206 integrated circuit, which received the internally formatted still image or motion video signal, and produces bit streams of encoded tokens corresponding to a plurality of externally selective image coding algorithms. The ENC 208 integrated circuit is further comprised of a pipeline buffer circuit connected to the SMART 216, and PACK 210 integrated circuits, which either transfer the encoded tokens to SMART 216 for internal storage or ship to the PACK 210 for outbound transmission. In a more preferable embodiment. The ENC 208 is also comprised of a interface circuit which can pipeline, cascade, or parallelize a plurality of external encoding processor elements, and encode pixel and frequency domain algorithm at macroblock, group of block, partial frame, or whole frame level.

SUBSTITUTE SHEE1 The PACK 210 (packet processor) integrated circuit is able to transcode or format video signals in CIF or SIF compatible macroblocks 306, it is also able to store, retrieve, or relay the transcoded CIF or SIF video signal in a single or plurality of packets or ATM (asynchronous transmission mode) cells for an inbound or outbound communication session. Preferably, The PACK 210 is further able to correlate the CIF or SIF macroblocks 306 into VISC 112 internal format, and transcode voice, data, graphics, and other non-video 240 data types in CIF or SIF macroblock-based packets or cells.

The PACK 210 is comprised of a protocol processor element connected to the ENC 208 and TX 212 integrated circuit, which receives an encoded CIF or SIF compatible video signal macroblocks 306 from ENC 208, and generate and outbound a single or plurality of data, control, and maintenance packets to TX 212. The PACK 210 protocol processor element also connected to the DEC 220 and RX 214 integrated circuit, which receives an inbound video packet or cells from RX 214, and generates and relay a plurality of CIF or SIF compatible macroblocks 306 to DEC 220 for further decoding. The PACK 210 integrated circuit is also comprised of a pipeline buffer element connected to the TX 212 (transmission processor) and RX 214 (receiving processor) integrated circuit, which transmit and receive video signal in CIF or SIF compatible packets or ATM cells. The PACK 210 is further comprised of a address generation circuit connect to pipeline buffer element and F and SMART 216 integrated circuit, which access and transfer a CIF or SIF packets or cells

SUBSTITUTESHEET into VISC 112 internal format via F, and then systematical enable the pipeline buffer circuit to download and store the reformatted CIF or SIF packets or cells in SMART 216. Reversely, the pipeline buffer element further retrieve directly the internal formatted packets and cells from SMART 216, and translate into CIF or SIF compatible macroblocks 306 via F. In a more preferred embodiment, the protocol control processor connected via TX 212 and RX 214 to a plurality of external PACK'S 210, which establish, maintain, and terminate point-to-point and point-to-multipoint networking sessions, it further parse, assemble, or disassemble a video signal representation in CIF or SIF packets or ATM cells forms, according to specific TX 212 or RX 214 request and run-time networking conditions. In a further preferred embodiment, the protocol control processor element can transcode layered signalling data structure according to OSI protocol architecture, which corresponds a plurality of user preference, application requirement, session control, transmission set-up, network control, and logical or physical link setup and termination, and the alike, and transcode non-video 240 data types in VISC 112 internal format.

The SMART 216 (smart memory) integrated circuit is able to provide an optimized article-oriented architecture to address, store, and retrieve a single or plurality quantities of background still image and foreground motion video articles in accordance with the selected VISC 112 internally optimized format. The SMART 216 is also able to provide a set of run-time variables correspond with user, application, and networking conditions, and dynamically reconfigurate itself to address, store, and retrieve these most-

SUBSTITUTE SHEET recently-optimized video articles. Preferably, The SMART 216 is further able to provide a set of alternative referencing parameters in order to dynamically move, overlay, rotate, enlarge, or reduce a single or plurality of motion video articles at run-time without physically modifying or moving their address and data.

The SMART 216 integrated circuit is comprised of single or plurality of memory cells and their associated sensing, control, management, and interface circuits connected to the F integrated circuit, which receives a single or plurality of VISC 's 112 internally modified CIF or SIF macroblocks 306, and produce an optimized data structure to perform an article-oriented addressing 308, storage, and retrieval of the still and motion video articles. The SMART 216 is also comprised of a run-time adaptive decision- logic circuit connected to the CON 218, BAND 204, and ENC 208 integrated circuit, which receives a set of run-time variables correspond with user, application, and networking conditions, and produces a run-time executable configuration in order to address, store, and retrieve these most-recently-optimized video articles. In a more preferred embodiment, the SMART 216 is further comprised of a pointer manipulation circuit connected to the ENC 208, DEC 220, CON 218, POST 222, and PREM 202 integrated circuits, which receives run-time requests to move, overlay, rotate, enlarge, or reduce a single or plurality of these stored video articles, and produces the appropriate alternative referencing parameters to dynamically manipulate these articles without physically modifying or moving their address and data. In further preferred embodiment, the SMART 216 is comprised of pipeline self-synchronization circuit

SUBSTITUTE SHEE^ connected to the CON 218, PREM 202, BAND 204, FORM 206, ENC 208, PACK 210, and DEC 220 integrated circuits, which sensitize and register the abnormal instances of PREM 202, BAND 204, F, ENC 208, DEC 220, and PACK 210 subsystem's pipeline operations, and send a system alert signal to CON 218 in requesting further fine-tuning of the system look-up-table, the CON 218 would then examine the bottlenecked subsystems, identify their overload causes, then authorize and adjust the SLUT 234 parameters to properly scale down the specific subsystem processing and still to maintain the overall system performance, the SMART 216 will then issue selective set of control signals including, but not limited to task redirection, data reformatting, NOP 374 insertion, thread¬ ing, or delay branching to adjust the specific overloaded sub¬ system, and to resume overall system pipeline synchronization.

The TX 212 (transmission processor) and RX 214 (receiving processor) integrated circuit is able to transceive video signal according to a plurality of networking environments and a plurality of run-time bandwidth conditions. The TX 212 and RX 214 comprised of a single or plurality pairs of transceivers corresponding to a plurality of analog or digital networks including, but not limited to: broadband ISDN 130, FDDI, MAN, DS3, twisted-pair or coaxial LAN, switch or dedicated Tl, primary rate ISDN 130, fractional Tl, frame relay, ISDN 130 switched HI, single or dual channel basic rate ISDN 130, digital switched or private PSDN, analog twist-pair, basic rate ISDN 130 D channel, or analog twist-pair. The TX 212 and RX 214 are also comprised of sensitizing 342 circuit connected to the BAND 204 integrated circuit which

SUBSTITUTE SHEET sensitizes a plurality of run-time networking conditions and generate run-time bandwidth allowance which includes, but not limited to: 150 Mbs, 100 Mbs, 45 Mbs, 10 Mbs, 2.048 Mbs, 1.544 Mbs,

384 Mbs, 128 Mbs, 64 Kbs, Px 64 Kbs, 56 Kbs, 19.2 Kbs, or 9.6 Kbs.

In a more preferred embodiment, the TX 212 and RX 214 is further comprised of a self-configuration circuit connected to the CON 218 and ENC 208 integrated circuit, which receives application requirements 356 and user preference, and generate the appropriate run-time bandwidth signal.

The DEC 220 (decoding processor) integrated circuit is able to correspond a bitstream of encoded tokens, and decode them into still or motion video signals according to a plurality selections of pixel or frequency domain coding algorithms. The DEC 220 is also able to decode the selective pixel or frequency domain algorithm at either macroblock, group of block, partial frame, or whole frame image level in order to achieve the selective VISC 112 system throughput. Preferably, the DEC 220 is further able to selectively decode the algorithms with either software program, internal hardware, or employing additional external coprocessor elements. The DEC 220 integrated circuit is comprised of a pixel domain and a frequency domain decoder circuits connected to the SMART 216 and PACK 210 integrated circuits, which received a bit stream of encoded tokens, and produce a sequence of fully decoded still image or motion video signal corresponding to a plurality of externally selectable image coding algorithms. The DEC 220 integrated circuit is also comprised of a instruction or task queuing circuit, which stores and produces a sequence of

SUBSTITUTE SHEET prescheduled instruction tasks for the entire decoding procedures. The DEC 220 integrated circuit is further comprised of a pipeline buffer circuit connected to the SMART 216, and PACK 210 integrated circuits, which either receives internally or externally encoded tokens from SMART 216 or PACK 210 for further decoding, transfer the decoded signals to SMART 216 for internal storage, or relay to the decoded signal to PACK 210 for outbound transmission. In a more preferable embodiment. The DEC 220 is also comprised of a interface circuit which can pipeline, cascade, or parallelize a plurality of external decoding processor elements, or simply employ internal hardware or software program to decode pixel and frequency domain algorithm at macroblock, group of block, partial frame, or whole frame level.

The POST 222 (postprocessing) integrated circuit is able to generate a selective set of live video, sequential stills, or bit¬ mapped graphics image output signal corresponding to the VISC 112 internal signal blocks it received from the decoder (DEC) 220. As the concept of VISC 112 internal signal formatting and decoder integrated circuit design should become more apparent after the later disclosure of other integrated circuit elements. The POST 222 is further comprised of a display adaptor (Da) , and a print/fax adaptor (Pf) output circuits. Both the Da and Pf output adapting circuits are connected to the decoder (DEC) 220 integrated circuit for receiving a VISC 112 compatible internal processing signal. The Da output adapting circuit produces a VGA 242, RGB 244, or NTSC 246 compatible output signal for further displaying live video, animated graphics, or sequential image output.

SUBSTITUTE SHEET The Pf adaptor output circuit produces a G3, G4, or encapsulated postscript (EPS) compatible output file for further printing or faxing a freeze frame of motion video, still image, or still graphics. In a preferred embodiment, the POST 222 integrated circuit is also comprised of a buffer element for receiving the processing signal and storing the processing signal from the decoder integrated circuit. The POST

222 can also efficiently printout, fax , or display output other non-VISC foreign data streams.

The CON 218 (system controller) integrated circuit is able to control and coordinate individual subsystem pipeline operations, to maintain and manage overall video data throughput, and to sensitize and adapt external application, user, networking conditional branches. The CON 218 integrated circuit is also able to initiate, maintain, and update a set of run-time executable system look-up-tables (SLUT'S) 234, and to facilitate static task prescheduling and look-ahead 338 instruction presequencing according to the specific input video data types. The CON 218 is further able to provide run-time coordination among individual subsystems and to reconfigurate each subsystem's functionalities in maintaining a scalable overall system pipeline performance.

The CON 218 integrated circuit is comprised of a dynamic storage and retrieval data structure for implementing a system look-up-table connected to PREM 202, which receives a differential frame bit-map corresponding with specific input video data types,

SUBSTITUTESHEE1 the SLUT 234 then cross reference and generate a set of run-time parameters including, but not limited to: compression ratio, frame rate, and display resolution. The CON 218 is also comprised of a task prescheduler circuit connected to SLUT 234 and every

VISC 112 pipeline subsystems, which further translate the selective run-time SLUT 234 parameters into a specific set of run-time encoding, decoding, and frame updating instruction sequences for the pipeline ENC 208, DEC 220, and POST 222 integrated circuits. In a more preferred embodiment, the CON 218 further analyze each subsystem's pipeline operations requirement, and further generate an appropriate scalable internal data format in accommodating the available hardware processing and memory throughput. In a further preferred embodiment, the CON 218 is further comprised of a dynamic reconfiguration circuit, which sensitize and adapt with system's run-time critical paths, subsystem bottlenecks, networking conditions, user preference, and application's requirements 356, and to insert conditional branch instructions, and to relieve specific overloaded subsystem's functional requirement in maintaining overall system throughput.

3. OPERATIONS OF VISC'S

FIG.3 illustrates the major functional operations for a VISC 112 integrated circuit system, which includes analog and digital preprocessing (PREM 202, API 232, COP 230), look-ahead 338 bandwidth and instruction management (BAND 204, SLUT 234), run-time application priority interface (API 232, BOLUT 236, COP 230), dynamic network communication management (TX 212, RX 214, BAND 204, FORM 206) scalable self-reconfigurable processing (FORM 206) , unified packet control processing (PACK) 210, a scalable memory array reconfigurable architecture (SMART) 216, and analog and digital postprocessing (POST) 222.

It is worth mentioning that instead of focusing on specific implementations and apparatus, we intend to address all the generic critical methods which are required for VISC 112 system realizations. These methods can be implemented in a plurality of integrated circuit technologies.

A. ANALOG AND DIGITAL PREPROCESSING (PREM 202, COP 230, API 232)

The PREM 202 is comprised of a plurality of analog and digital preprocessing steps to capture or receive an analog or video input, to analyze and identify the foreground motion video objects, to generate their associated linear motion vector 302, and produces a sequence of digital video frame data and their corresponding differential frame bit-maps 304. The PREM 202 is further comprised of functional steps to accept, preprocess, and prepare traditional non-VISC data types, and to forward to internal VISC 112 circuits for further processing.

The PREM 202 is comprised of an ANALOG INPUT CAPTURER circuit, which receives analog input conforming to a plurality of video formats including but not limited to NTSC 246, PAL, RS170, SCAM.

HEET The PREM 202 is also comprised of X-Y ADDRESSING 308, SAMPLE 370 and HOLD 372 circuits which receives a continuous sequence of analog live video input and either produces a set corresponding analog video signals, reference to a selective sets of random video frames, or extract a specific set of REGION OF INTEREST (ROI) input subimage, for example, in MACROBLOCK form.

The PREM 202 is further comprised of DELAY 364, COMPARE 366, REGISTRATION 368, and ADC 310 (analog-to-digital conversion) circuit, which references a plurality of input video frames and produces a corresponding set of digital frame data and their associated DIFFERENTIAL FRAME BIT-MAPs.

The PREM 202 is further comprised of X-Y ADDRESSING 308, DELAY 364, COMPARE 366, REGISTRATION 368 circuit connected to the API 232, CON 218 and BOLUT 236, which receives a set of priority requests for application-specific objects, and identify and generate corresponding ROI (REGION OF INTEREST) for each object.

The PREM 202 is further comprised of ADC 310, DELAY LOOP 364, CORRELATE 370, SEARCH, and MATCH 372 circuits for calculating and producing digital motion video object profiles and their corresponding linear MOTION VECTORS. The MOTION VECTORS are forwarded to SMART 216 or other temporary storage.

Preferably, PREM 202 is also comprised of a preprocess ing circuit 350 connected to the COP 230, which can receive

SUBSTITUTESHEE1 traditional non-VISC 240 data types, and reorganize and produce an appropriate internal data format for VISC 112 processing.

B. LOOK-AHEAD BANDWIDTH AND INSTRUCTION MANAGEMENT (BAND 204, SLUT 234)

The BAND 204 integrated circuit receives a sequence of frame differential bit-maps 304 corresponding to each input video signal, it then reference each bit-map 304 to the SLUT 234 in deriving the required run-time attributes includes compression ratio, frame rate, and display resolution. The run-time attributes will further translate into the required run-time transmission and processing bandwidth, BAND 204 will generate and prefetch 340 a group of predefined VISC 112 pipeline instructions for execution.

BAND 204 is comprised of BWCAP 330 circuit connected to ENC 208 and SMART 216, which receives processing, storage, and transmission bandwidth availability signals as soon as the system functions are initialized (SETMODE 376) . The BAND 204 is also comprised of a BWREQ 332 circuit connected to PREM 202, which receives FRAME DIFFERENTIAL BIT-MAP 304 and produces a BWREQ 332 (bandwidth requisition) signal. The BAND 204 will compare BWCAP 330 signal with BWREQ 332 signal in determining whether the run-time communication and processing bandwidth is sufficient.

Provided the bandwidth is sufficient, BAND 204 is further comprised of a SLUT 234 (system look-up-table) , INSTRUCTION LOOK- AHEAD 338, AND INSTRUCTION GROUP FETCH 340 circuits, hich can

SUBSTITUTE SHEET receives FRAME DIFFERENTIAL BIT-MAP 304 signal and prefetch 340 a group of VISC 112 instructions for further processing of the video data.

Provided the bandwidth is not sufficient, BAND 204 will issue a CONDITIONAL BRANCH 336 and transfer the video data to FORM 206 for further bandwidth reduction. FORM 206 is further comprised of OBJECT ID 346, PRIORITY ASSIGN 344, and X-Y INTERPOLATE 348 circuits, which receives full frame or subframe of digital video data form BAND 204, reduce BWREQ 332 signal for lower priority objects through further horizontal and vertical interpolations 348.

C. RUN-TIME APPLICATION PRIORITY INTERFACE (API 232, BOLUT 236, FORM 206, COP 230)

The API 232 (application programming interface) comprises a PRIORITY ASSIGN 344 circuit connected to the BOLUT 236, which receives a set of application-specific input priority requirements 356, and generate a corresponding set of media data types conform¬ ing to the run-time bandwidth availability. The PRIORITY ASSIGN 344 circuit is also connected to the FORM 206 integrated circuit, which produces a PRIORITY REASSIGN 344 signal provided run-time bandwidth is not available, the FORM 206 will in turn produces a modified internal format corresponding to the reassigned priority list, and associate each PRIORITY REASSIGN 344 signal with the corresponding video OBJECT ID 346 during run-time.

n_! ιnr*τ!7[ _Tj -icfci The COP's 230 are connected to the PREM 202, which receive preprocessed 350 non-VISC data types 240, and provide traditional

RISC or CISC like application processing. The COP's 230 is further connected to the API 232, which receives PRIORITY REASSIGN 344 signal and generates a BWREQ 332 signal to BOLUT 236 for further assurance of the run-time bandwidth availability.

D. DYNAMIC NETWORKING MANAGEMENT (TX 212, RX 214, BAND 204, FORM 206)

The BAND 204 integrated circuit is comprised of a SENSITIZING 342 circuit connected to the RX 214 and TX 212, which constantly look out to the network and generate a ALERT 352 signal to the VISC 112 internal system when it receives a change of run¬ time communication bandwidth signal. The VISC 112 internal system will generate a NETWORK CONGEST 354 query and compare against the required run-time outbound transmission bandwidth during that time period. Provided the outbound transmission bandwidth is more demanding, BAND 204 will transfer the outbound transmitting signal to FORM 206 for reduction of the bandwidth prior to transmission.

E. SCALABLE SELF-CONFIGURED PROCESSING (FORM 206)

The FORM 206 is comprised of OBJECT ID 346, PRIORITY ASSIGN 344, and X-Y INTERPOLATE 348 circuits, which can receives a sequence of digital video signal blocks from PREM 202. It will then analyze the application requirements 356, run-time bandwidth

SUBSTITUTE SHEET limitations, and internal storage and processing constraints, assign 344 appropriate priority level for each video object, and generate an appropriate internal format for further encoding, storage, and decoding of the said video signal blocks.

The FORM 206 is further comprised of X-Y EXTRAPOLATE circuits which can reconfigurate the internal signal blocks to external format prior to postprocessing and outbound transmission.

F. UNIFIED PACKET CONTROL PROCESSING (PACK 210)

The PACK 210 provides an unified method for internal storage and external transmission of the video signal blocks. The PACK 210 further partition all encoded digital video signal blocks into sequence of packets. Each macroblock 306 is further comprised of a plurality of packets. Besides the default and bit- parallel packet transfer mode for MACRBIΛCK data, the internal VISC bus 402 architecture can also be reconfigured to support a plurality of circuit transfer modes, which includes burst FRAME transfer, burst SUBFRAME transfer, burst ROI OBJECT transfer, burst BOLUT 236 table transfer, burst FRAME DIFFERENTIAL BIT-MAP transfer, and burst SLUT table transfer.

As a result, contrary to the traditional CISC and RISC systems treats bit as a smallest data entity. VISC 112 treats packet as the smallest entity, and in turn assemble and group individual data bits together for non-VISC data processing.

SUBSTITUTE SHEET G. SCALABLE MEMORY ARRAY RECONFIGURABLE ARCHITECTURE

(SMART 216)

The SMART 216 is comprised of ROTATE 358, LINEAR SHIFT 360, and ZOOM/REDUCE 362 circuits, which can manipulate the motion video object data blocks referencing from the SMART 216. The SMART 216 can further send request to FORM 206 in further reducing the storage requirements for internal video signals. The SMART 216 can further send request to FORM 206 in further resuming the video data conforming to external data format requirements.

SMART 216 can also respond to a plurality of run-time application or networking queries, and reference the internal stored packet data, then reformat and update them according to the run-time priority changes.

SMART 216 can further partition and store the foreground moving objects and background still images into separate memory segments. According to the run-time networking and application requirements 356, SMART 216 can work with FORM 206 in dynamically adapting to the priority levels, quality requirements 356, and networking constraints.

H. ANALOG AND DIGITAL POSTPROCESSING (POST 222)

POST 222 is comprised of a plurality of output circuits connected to DEC 220, which can receive a decoded video signals. It then generates and translate them into a plurality of reformatted video signals for further displaying, faxing 110, or printing out the decoded video signal.

POST 222 is comprised of a plurality of DISPLAY circuits, which includes but not limit to VGA 242, RGB 244, NTSC 246. POST 222 also comprised of a PRINT/FAX circuit, which receive a internal video data and generate a output conforming to EPS 248 data format.

4. VISC ARCHITECTURE AND ORGANIZATION

FIG.4 illustrates the design example of a single chip VISC 112 integrated circuit. This provides a unified VISC 112 micro¬ processor solution suitable for all major system applications related to digital 106, videophone 104, and multimedia computing.

The diagrammatic representation illustrated in FIG.4 comprises the following major system components. They are a preprocessing/motion processor (PREM) 202, a system look-up-table (SLUT 234), a bandwidth controller (BAND 204), a formatting processor (FORM) 206, a encoding processor (ENC) 208, a packet processor (PACK) 210, a scalable memory array reconfigured architecture (SMART) 216, a application priority interface (API 232), a bandwidth overlay look-up-table (BOLUT) 236, an application coprocessor (COP) 230, a decoding processor (DEC) 220, a postprocessor (POST 222) , a transmission processor (TX 212) , a receiving processor (RX) 214, and a system controller (CON) 218. The system components are interconnected internally through a SYSTEM BUS 402 and a CONTROL BUS 404. The CONTROL BUS 404 allows the CON 218, BAND 204, and SLUT 234 to control, access, and communicate with other system components such as BAND 204, PREM 202, FORM 206, ENC 208, DEC 220, COP 230, API 232, PACK 210, and SMART 216. The SYSTEM BUS comprises a plurality of BUS TRANSFER control modes. The standard default mode allows the SYSTEM BUS 402 to perform packet switched transportation of data, control, and operation maintenance packets among various system components. In addition, the system bus 402 can be reconfigured into a plurality of selective circuit switch modes, in order to allow burst transportation of full frame, partial frame, live motion objects, graphics overlay, or other non-VISC data types. Both SYSTEM BUS 402 and CONTROL BUS 404 are bidirectional parallel bus.

Due to the real time performance requirement for high speed video frame processing, three system-wide pipeline interconnections are implemented. The first is the local video pipeline consist of a direct interconnection between PREM 202, FORM 206, ENC 208, PACK 210, SMART 216, RX 214/TX 212. The second system interconnect is a application priority pipeline consist of a direct interconnection between the API 232/COP 230, BOLUT 236, BAND 204, SLUT 234, FORM 206, and PACK 210. The third system interconnect is a remote video communication pipeline consist of a direct interconnection between the RX 214, PACK 210, SMART 216, FORM 206, ENC 208/DEC 220, POST 222/TX 212. In order to facilitate these pipeline interconnect¬ ions, first-in-first-out memory devices are inserted where

SUBSTITUTESHEE1 appropriate. Other Private buses are provided between TX 212, RX 214, and PACK 210, FORM 206 and ENC 208, ENC 208 and DEC 220, DEC 220 and COP 230, COP 230 and API 232, DEC 220 and POST 222, PREM 202 and FORM 206, PREM 202 and SLUT 234, PACK 210 and SMART 216, and SLUT 234 and BAND 204.

The diagrammatic representation illustrated in FIG.4 comprises the following input and output pins. They are 32 bit video input pins, 32 bit video output pins, 32 bit application parallel input/output (PIO) pins, 24 bit tag/control input/output pins, and 8 bit serial communication input and output (SIO) pins.

The first roup of 32 video input pins connected to PREM 202, which receives four set of individual eight-bit digital color input data, i.e., red (R) , green (G) , blue (B) , and arbitrary (X) . These input pins can be multiplexed to receive external address, status, tag, control, and interface signals, and to transport program and data directly from external memory.

A second group of 32 video output pins is connected to POST 222, which can generate four sets of eight-bit output signals for external display, printing, or fax 110. These output pins can be also multiplexed to transmit internal address, status, tag, control, and interface signals, and to transport program and data directly to external memory. The input and output data ports can further be multiplexed to form a single group of 32 bidirectional input/output pins in a baseline implementation. Furthermore, additional input and output pins can be included for transporting

SUBSTITUTESHEET address, tag, control, status analog video, and other non-VISC data signals.

There is a third group of 32 bi-directional parallel input/output (PIO) pins connected to API 232, which is multiplexed to import and export plurality of control and data types including but not limited to: application and system control commands, priority requests, application requirements 356, communication and status messages, and program and data sequences. For a more baseline implementation, PIO can be reduced to 8 or 16 pins for lower post packaging. Reversely, PIO can also be expanded to 64 or higher pins for higher performance implementation. PIO can further be connected to a time-division-multiplex (TDM) , packet-division- multiplex (PDM) local bus interface (LBI) , or small computer system interface (SCSI 110) circuits in order to facilitate the high speed burst data transportation.

There is further a fourth group of 24 tag and control input and output pins connected to the SLUT 234 and CON 218, which provide the required power, ground, clocking, status, control and tag signals. The pins numbers can be modified according to system performance requirements.

There is further a fifth group of 8 serial communication input and output (SIO) pins connected to the RX 214 and TX 212, which simultaneously receives and transmits four channels of full- duplex video signals. The SIO pins numbers can be modified to meet application requirements 356.

SUBSTITUTE SHEET The PREM 202 integrated circuit is comprised of video preprocessors and motion processor, which receives analog or digital video, graphics, or still images and generate FRAME DIFFERENTIAL BIT MAP 304, MOTION VECTOR 302, and MACROBLOCK profile for each input frame. The PREM 202 is also comprised of a PREPROCESSOR circuit which receives other non-VISC data types, and preprocess 350 them into VISC 112 executable formats.

The CON 218 is comprised of SLUT 234 circuit which receives FRAME DIFFERENTIAL BIT MAP 304 from PREM 202, and generates a run-time bandwidth profile for each input frame including compression ratio, frame rate, and display resolution. The CON 218 is also comprised of a INSTRUCTION LOOK-AHEAD 338 circuit, which receives a GRANT 335 signal from BAND 204 and translate the run time bandwidth profile into a group of predefined instruction address pointers. The CON 218 is further comprised of a GROUP INSTRUCTION PREFETCH 340 circuit which receives the instruction pointers and generate a sequence of run-time executable instructions for the input frame.

The BAND 204 integrated circuit is comprised of a BWREQ 332 circuit which receives the FRAME DIFFERENTIAL BIT-MAP 304 from PREM 202 and generate a run-time bandwidth requirement (BWREQ 332) signal for each input frame. The BAND 204 is also comprised of BWCAP 330 circuit which during the VISC 112 system initialization (SETMODE 376) , receives a set of bandwidth constraints relating to internal processing and storage and external communication.

SUBSTITUTESHEE] The BAND 204 is further comprised of a BWALLOW 334 circuit which compare the system bandwidth constraints with the BWREQ 332 signal and determine whether the VISC 112 system has sufficient run-time bandwidth. Provided the bandwidth is sufficient, the BAND

204 will generate a GRANT 335 signal to CON 218 in order to enable the SLUT 234 to search and prefetch 340 the run-time instruction sequences. Provided the bandwidth is insufficient, the BAND 204 will issue a CONDITIONAL BRANCH 336 and request the

FORM 206 to reformat the input frame and further reduce the run-time bandwidth requirement.

The FORM 206 is comprised of a OBJECT ID 346 circuit, which receives standard MACROBLOCK subimages 306 from PREM 202, and generate a list of moving objects for each input frame. The FORM 206 is also comprised of a PRIORITY ASSIGN 344 circuit, which evaluate application requirement from API 232 and communication constraint from BAND 204, and assign 344 the appropriate run-time priority level to each moving object and MACROBLOCK subimage. Provided the run-time bandwidth is insufficient, The FORM 206 is further comprised of an X-Y INTERPOLATE 348 circuit, which receives MACROBLOCK subimages, moving objects, and their associated run-time priority levels and produces a reformatted MACROBLOCK subimage and moving objects. Once the run-time bandwidth is proven to be sufficient, the FORM 206 is further comprised of DATA TRANSFER circuit which will transfer the formatted macroblock 306 subimage to ENC 208 and PACK 210 for further processing.

The ENC 208 is comprised of a programmable ENCODING circuit

SUBSTITUTE SHEET which receives input macrblocks and generate JPEG, MPEG, H.261, VQ, or alike encoded video tokens depending on the application requirement. The ENC 208 circuit can perform all pixel domain and transform domain encoding functions.

The PACK 210 is comprised of a PACKETIZER circuit, which receives internally reformatted macroblock 306 subimage, and further partition into a plurality of data packets. The PACK 210 is also comprised of a PACKET CONTROL circuit, which generates the required control messages for both internal and external data movement, error detection and correction, and flow control manage¬ ment functions. The PACK 210 is further comprised of a PACKET/ CIRCUIT BUS INTERFACE circuit, which can program the internal bus into a packet switch mode, or a selective high speed burst circuit modes for the transportation of moving objects, macroblock 306 subimages, or full frame images.

The SMART 216 is comprised of a SCALABLE MEMORY ARRAY, which receives data packet from PACK 210, and automatically self- configurate into a plurality of internally formatted entities for temporary storage. The SMART 216 is also comprised of a REFORMAT circuit, which receives run-time bandwidth constraint from BAND 204, or receives a run-time application priority request from API 232, and self configured to the appropriate data storage format conforming to these run-time networking and application require¬ ments 356. The SMART 216 is further comprised of a FOREGROUND PROCESSOR circuit, which transceive, prioritize, manipulate, and store foreground data objects. The SMART 216 is further comprised

SUBSTITUTESHEE of a BACKGROUND PROCESSOR circuit, which transceive, prioritize, manipulate, and store background data macroblocks 306. The SMART

216 is further comprised of a DATA MANIPULATION circuit, which generates the address pointer reference list to rotate 358, linear shift 360, enlarge, or reduce 362 the specified data object.

The SMART 216 is further comprised of a SIMULATOR circuit which generate instruction sequences for visualizing artificial knowledge or object data movement.

The TX 212 and RX 214 are comprised of a single or plurality pair of TRANSCEIVERS circuits, which can import and export video and non-VISC data. The TX 212 is also comprised of an QUERY circuit, which can generate a REQUEST signal to inquire about external networking and bandwidth stability conditions. The RX 214 is also comprised of an INTERPRETER circuit, which can receive and decode external networking and bandwidth control information and generate a ALERT 352 signal to BAND 204 in case abnormal networking conditions are happening.

The DEC 220 is comprised of a programmable ENCODING circuit, which receives encoded macrblock tokens and generate JPEG, MPEG, H.261, VQ, or alike macroblock 306 pixel images depending on the application requirement. The DEC 220 circuit can perform all pixel domain and transform domain decoding functions.

The POST 222 is comprised of a plurality of analog and digital postprocessors, which receives digital decoded video pixel runs and generate the appropriate formatted signal for

SUBSTITUTESHEE TSC 246, PAL, SCAM, RS-170, G3, G4, or EPS 248.

The COP 230 is comprised of a RISC or CISC circuit, which receives preprocessed 350 non-VISC data types from PREM 202 and perform traditional data processing tasks.

The API 232 is also comprised of a INTERFACE circuit, which receives application priority requirements 356 and generate the appropriate message signal to FORM 206 in further assigning 344 the run-time priority levels 344 for each object and subimage. API 232 is further comprised of DRIVER circuit, which allow API 232 to directly interface with a plurality of external commercial DOS, UNIX, WINDOW, NT, or OS2 application programs. API 232 is also comprised of a programmable BIOS circuit, which allow the VISC 112 to be executed under a commercial UNIX, DOS, NT, WINDOW operating system.

The BOLUT 236 is comprised of a set of system LOOK-UP-TABLE circuits, which can be dynamically generated to provide a plurality options for multiple data type selections. The BOLUT 236 is also comprised of a PRIORITY SENSITIZE circuit, which receives RUN-TIME APPLICATION PRIORITY request signal from API 232, and produces an appropriate set of run-time media profile for further execution.

5. SUMMARY AND ADVANTAGES OF VISC

Each of the above aspects of the invention, when taken together, result in a significant integrated circuit system

SUBSTITUTESHEE:^r optimized for video instruction set computing. The present invention pertains to integrated circuit system based on the novel architecture of Video-Instruction-Set-Computing (VISC 112) . The integrated circuit comprises a plurality of functional units to independently execute the tasks of remote communication (RX 214, TX 212), bandwidth adaptation (BAND) 204, application control (API) 232, multimedia management (PACK) 210, and universal video encoding (ENC 208, DEC 220). The integrated circuit is also comprised of a scalable formatter element (FORM) 206 connecting to the functional units which can inter-operate arbitrary external video formats and intelligently adapt to selective internal format depending upon the system throughput and configuration. Additionally, there is a smart memory element (SMART) 216 connect¬ ing to the functional units and scalable formatter (FORM) 206, which can access, store, and transfer blocks of video data based on the selective internal format. In the preferred embodiment, the integrated circuit is also comprised of an embedded RISC or CISC co-processor element (COP) 230 in order to execute DOS, Window, NT, Macintosh, OS2, or UNIX applications. In a more preferred embodiment, the integrated circuit includes a real time object oriented operation system element wherein concurrent execution of the application program and real time VISC 112 based video instruction sets can be performed.

The present invention is designed to sustain the evolution of a plurality generations of the VISC 112 microprocessors. These novel VISC 112 microprocessors can be efficiently used to perform wide range of real time distributed video signal processing

BSTITUTESHEET functions for applications such as interactive video, HDTV 106, and multimedia communications.

In summary, our present invention can provide the following advantages:

a. It provides a unified microprocessor integrated circuit for digital TV 106, videophone 104, and multimedia computer 108.

b. It provided the most optimum performance for video specific application tasks.

c. It is equally capable of traditional data processing of non-video 240 data types.

d. It can conveniently interface with traditional DOS, UNIX, and WINDOW applications.

e. It can be executed under traditional DOS, WINDOW, NT, OS2 or UNIX operating system environments.

f. It can apply traditional RISC and CISC microprocessors as coprocessors.

g. It can provide Internal formatting and smart memory allocation for scalable performance.

h. It can provide run-time bandwidth management.

SUBSTITUTE SHEET i. It can conform with application-specific priority requirements 356.

j. It can provide instruction Look-ahead, superscalar, and superpipelining for enhancing system performance.

While our above description contains many specifications, these should not be constructed as limitations on the scope of the invention, but rather as an exemplification of one preferred embodiment thereof. Many other variations are possible. For example, a single chip implementation may further contains other still image processing circuits, or a parallel processor implementation can be provided to a ultra-high performance supercomputing system. Furthermore, many exotic, untested embodiments and minor variations can be provided to further improve the processing speed for ENC 208, DEC 220, PACK 210, TX 212, and RX 214. However, our intention has always been in the design of a low cost and yet upgradable integrated circuit solution for massive consumer, communications, and desktop or embedded computing applications.

Accordingly, scope of the invention should be determined not by the embodiment(s) illustrated, but by the appended claims and their legal claims.

SUBSTITUTESHΞF

Claims

CLAIMSWe claim:

1. An integrated circuit system apparatus optimized for video-instruction-set-computing executing a plurality of functions for computation, communication, interoperation, storage and retri¬ eval of video related data types including live motion video, animated graphics, still image sequence, and non-video related data types including voice, high quality audio, text, and data, for unifying the design of interactive digital television, video telephone, and multimedia systems, selectively providing a plurality of functions including retrieving, interoperating, run-time bandwidth management, application object prioritization, instruction look-ahead, instruction group-fetching, run-time conditional-branch look-ahead, scalable formatting, programmable encoding, self-configurable packaging and storage, programmable decoding, preprocessing, motion processing, postprocessing, and adaptive transceiving and retrieval, whereby said system is compa¬ tible with multiple bandwidths and algorithmic signals, said system comprising: first data processing means for selective receipt of an origination signal or remote signal, and means for image prepro¬ cessing, motion processing, and producing frame differential bit map, macroblock subimages and motion vector signals of said origi¬ nation or remote signal for storage processing, or transmission; first controller means for receipt of application object

SUBSTITUTESHEE! priority request signal, and means for producing run-time object priority assignment signal in accordance with said macroblock subimage; second controller means for producing run-time bandwidth requirement signal in accordance with said frame differential bit¬ map and said run-time priority assignment signal; third controller means for receipt of the available transmission, processing, and storage bandwidth capability signals, and means for producing overall system bandwidth capability signal; fourth controller means for producing and prefetching look-ahead group instruction sequences for run-time execution of said preprocessed macroblocks and motion vector signals provided said overall system bandwidth capability signal is sufficient; second data processor means for scalable data formatting, selective reformatting said format of origination or remote signal to a compatible internal format provided said overall system band¬ width capability signal is not sufficient, said formatter means further comprising means for object identification, priority reassignment, horizontal-vertical interpolation and extrapolation. fifth controller means for dynamic bandwidth management, in communication with said formatter means and said application object prioritization means for continuously monitoring and updat¬ ing run-time bandwidth requirement signal for selective formatting said external format to said internal format in compliance with overall system bandwidth capability signal range; sixth controller means for run-time bandwidth detection, continuously monitoring application priority changes and irregular external networking conditions, and correcting said selective

-,»—>!—:—

formatting of said internal format to said external format based on overall system bandwidth availability; seventh controller means for run-time scheduler, detecting look-ahead-conditional branches and providing parallel, pre-emptive or earlier execution of predefined instruction sequences corresponding to these run-time execution condition changes in accordance with said networking condition and said application object priority; third data processing means for encoding said internally formatted macroblocks in accordance with said prefetched instruction look-ahead sequences, said means selectively comprising a programmable pixel domain encoder and a transform domain encoder; fourth data processing means for packaging said encoded macroblocks in accordance with said prefetched instruction-look- ahead sequences, said means selectively comprising a packet-switch¬ ed data controller and a burst circuit-switched data controller; a scalable and reconfigurable data memory means for receipt of said packaged data and automatically self-confguring into a plurality of internally εtorable entities, said means also comprises a self-configurator, a foreground processor, a background processor, and a video object manipulator means in compliance with the said run-time application priority and networking requirement; fifth data processing means for transmitting and receiv¬ ing remote data, transporting inbound and outbound video or non- video related signals, said means further selectively comprising a query and interpreter circuits continuously requesting and respond¬ ing external networking conditions; sixth data processing means for decoding said internally

STITUTE SHEET re¬ formatted encoded macro blocks in accordance with said prefetched instruction-look-ahead sequences, said means selectively comprising a transform domain decoder and a pixel domain decoder; seventh data processing means for analog or digital postprocessing of said decoded signal in accordance with said prefetched instruction-look-ahead sequences, said means selectively comprises a plurality of display, fax, or printer adaptors.

2. A system apparatus in accordance with Claim 1 wherein said first data processing means for capturing, preprocess¬ ing, differencing, and producing a frame differential bit map, macroblock subimages, and motion vector signals of said remote or origination signal selectively comprises: a video input capturer; a graphics input bit-map adaptor; a retrieval and storage register means for said input signal; a frame differentiation processing means; a motion processing means, a foreground video object processing means, a background image processing means, a single or plurality of integrated sensor means; and analog or digital prepro¬ cessor and storage means for non-video related data input.

3. A system apparatus in accordance with Claim 1 wherein said first controller means for receipt of application- specific input priority requests and producing run-time application priority assignment list selectively comprises a priority assign¬ ment circuit, a priority reassignment circuit, and an object identification processor for producing run-time media object data types conforming to run-time bandwidth availability.

SUBSTITUTE SHEE^T

4. A system apparatus in accordance with Claim 1 wherein said second controller means for receipt of run-time application object priority list and frame differential bit map and producing run-time bandwidth requirement, selectively comprising a run-time attributes processor for producing run-time compression ratio, frame rate, and display resolution.

5. A system apparatus in accordance with Claim 1 wherein said third controller means selectively comprises a controller means and a system look-up-table means for receipt of the run-time available transmission, processing, and storage band¬ width signals, and producing a overall system bandwidth capability signal, and a set of run-time attribute list according to the algorithmically pre-deter ined default parameters table retaining within the system look-up-table.

6. A system apparatus in accordance with Claim 1 wherein said fourth controller means selectively comprises a look-ahead-pipeline processor element for receiving said inbound differential video signal including bit-map, macroblock, and motion vector, and producing a group of predefined instruction sequence for run-time execution.

7. A system apparatus in accordance with Claim 1 wherein said second data processor means selectively comprises interpolating and extrapolating processing means, a instruction and task prescheduler means, a dynamic data formatter, a dynamic program scheduler and optimizer means and a scaling circuit means,

SUBSTITUTE SHEET for reformatting said inbound signal to an appropriate internal format according to the system bandwidth availability, producing coherent instruction bit stream for encoding processing, and controller means for invoking system calls for adjusting system clock rate, and resetting other system parameters.

8. A system apparatus in accordance with Claim 1 wherein said fifth controller means further comprises a run-time bandwidth manager means for continuously communicating with said first controller means and said second data processing means for monitoring and updating run-time bandwidth requirement changes in accordance with overall system bandwidth capability,

9. A system apparatus in accordance with Claim 1 wherein said six controller means selectively comprises a query circuit, a sensor circuit, and a responding and analyzer circuit for continuously sensitizing run-time network traffic conditions, run-time user or application-specific input require¬ ments, and dynamically reconfiguring and updating the aforementioned run-time attributes corresponding to the available run-time communication bandwidth.

10. A system apparatus in accordance with Claim 1 wherein said seventh controller means selectively comprises a run-time sensor and a controller and for receiving run-time condition changes, issuing conditional branches, and retrieving concurrent or preemptive pre-defined instruction bit streams for avoiding run-time execution discontinuity for said inbound signal.

SUBSTITUTE SHE r FETrT

11. A system apparatus in accordance with Claim 1 wherein said third data processing means selectively comprises a programmable encoder, a task queue, a pipeline buffer, and a interface circuit for selective on-board or off-board encoding of a internally formatted input still or motion video signal and to translate them into a bitstream of tokens in accordance with a selective one of a plurality pixel or frequency domain encoding algorithms at selectively macroblock, group of block, partial frame, or whole frame image level in achieving the selective system throughput.

12. A system apparatus in accordance with Claim 1 wherein said fourth data processing means selectively comprises a multiple layer protocol processing and control means, a pipeline buffer and register means, an address generation means, and an external data interface circuit for transcoding said video signals in full or quarter CIF or SIF compatible macroblocks, and storing, retrieving, or relaying said transcoded CIF or SIF video signal in a single or plurality of packets or asynchronous transmission mode cells for inbound or outbound communication session, trans¬ porting to or from said smart memory means, and coprocessing with off-board external packet processors.

13. A system apparatus in accordance with Claim 1 wherein said scalable smart memory means is comprised of a single or plurality of memory cells and their associated sensing, register, control, management, and interface circuits for

I UTE SHEET receiving a single or plurality of internally modified CIF or SIF macroblocks, and producing optimized storage data structure for performing an object-oriented addressing, storage, and retrieval of these still and motion video objects.

14. A system apparatus in accordance with Claim 13 wherein said scalable smart memory means is also comprised of run¬ time adaptive decision-making logic means for receiving a set of run-time variables correspond to user, application, and networking conditions, and producing a run-time executable data storage configuration in order to address, store, and retrieve these most- recently-optimized run-time video articles or objects.

15. A system apparatus in accordance with Claim 14 wherein said scalable memory means is further comprised of a pointer manipulation circuit circuits for receiving run-time requests to move, overlay, rotate, enlarge, or reduce a single or plurality of these stored video articles, and producing the appropriate alternative referencing parameters for run-time manipulation of these articles without physically modifying or moving their address or data.

16. A system apparatus in accordance with Claim 15 wherein said scalable smart memory means is further comprised of a controller means for pipeline self-synchronization, sensitizing and registering the abnormal instances of subsystem's pipeline operations, and sending a system alert signal to system controller in requesting further fine-tuning of the system look-up-table,

SUBSTITUTE SHEET requesting the system controller for examining the bottlenecked subsystems, identifying their overload causes, and authorizing and adjusting the system-look-up-table parameters to properly scale down the specific subsystem performance and still maintain¬ ing overall system throughput, the smart memory means further comprises controller means for issuing selective set of control signals including but not limited to task redirection, data reformatting, NOP insertion, threading, or delay branching for adjusting the specific overloaded subsystems, and for resuming overall system pipeline synchronization.

17. A system apparatus in accordance with Claim 1 wherein said fifth data processing means further comprises a sensitizing circuit for receiving said run-time bandwidth request signal, said transmission processor further comprises a query circuit for identifying whether run-time networking condition is sufficient, and receiving processor further comprises a interpreter means for analyzing and responding to system internal provided networking condition is not sufficient.

18. A system apparatus in accordance with Claim 1 wherein said sixth data processor means selectively comprises a programmable decoder, a task queue, a pipeline register and buffer, and a external data interface circuit for selective hardware or software on-b-^ard or off-board decoding of encoded tokens for input still or motion video in accordance with a plurality pixel or frequency domain decoding algorithms at macroblock, group of block, partial frame, or whole frame image level for achieving selective system throughput.

19. A system apparatus in accordance with Claim 1 wherein said seventh data processing means further comprises a display adaptor, a print/fax adaptor output circuits for selective display, printout, and fax of the live video, sequential stills, or bit-mapped graphics image output signal corresponding to the VISC internal signal blocks it received from the decoder.

20. A system apparatus in accordance with Claim 1 wherein said first, fifth, and seventh data processing means further comprises preprocessing, postprocessing, and transceiving means for selective remote or origination non-video data signal, said combination means permits said optimized video data processor system apparatus for processing traditional non-video data types.

21. An system apparatus improvement for capturing, preprocessing, differencing, and producing a frame differential bit-map, macroblock subimage, motion vectors, and still and moving object profile signals further comprises a single or plurality of programmable processor array reconfigured as n x n cellular logic array where n is an arbitrary integer and is typically of value in a range of between 4 and 64 wherein said cellular logic array processor can be employed to form a plurality of block processor operations for said n x n pixel blocks of subimage, said processor array further comprising: first means for selective digitizing origination or

TE SHEET remote video-related input signal; second on-board or off-board means for selectively retrieving origination remote video-related input signal; third means for frame differencing operation between current and previously displayed frames whereby a group of n x n difference blocks are identified, registered, and stored; fourth means for interframe motion estimation and process¬ ing for producing motion vectors for moving objects; fifth means for interframe or intraframe image and video preprocessing including template matching, edge detection, and feature extraction for producing a object reference profile for still and moving objects, wherein said object reference object profile can be used for representing as alternative motion vector provided said motion processing can not identify the appropriate motion vectors, and said reference profile can also be used for producing run-time execution routines, and for simulating and annealing and animating of said moving objects without physical transmission and receiving of the moving objects; and sixth means for detecting edges for each of said n x n blocks of said subimage and means for enhancing edges for each of decompressed subimage blocks prior to display; seventh means for preprocessing connected to application coprocessor for receiving traditional non-VISC data types, and reorganizing and producing an appropriate internal data format for video-instruction-set-computing.

22. An system apparatus in accordance with Claim 21 wherein said first means for selective digitizing origination or

BSTITUTE SHEET remotely retrieved video signal further comprising: means for capturing selective said analog video input conforming to RS-170, NTSC, PAL, SCAM and the alike formats, and producing corresponding digital input video signal; means for adapting selective said graphics bit-mapped input conforming to PCX, GIF, EPS, TIFF, or alike, and producing corres¬ ponding digital input signal; and register and associated memory means connecting to said input capturer and adaptor means for receiving, storing, and transferring said digital video or bit-mapped input signal.

23. An system apparatus in accordance with Claim 21 wherein said second means for on-board or off-board image sensing and preprocessing further comprises: first single or plurality of integrated sensing means for producing the required input signal corresponding to the energy it received; second single or plurality of integrated analog processor and memory means connected to said sensing means for receiving and remembering incoming energy corresponding to current or previous frame of subimages, and producing selective interframe or intraframe correlation and difference signal; third single or plurality of integrated analog tracking and hold circuit means, and its programmable control means connect¬ ed between each said sensor and said processor means for performing selective delay functions for random referencing from current to selective previous frame; fourth single or plurality of interconnection or shift

*v-» -*<-' I I i (_,- l *__. Ul ιt_t__. i register means connected in between each memory element and its surrounding neighbors for retrieving neighboring subimage signal values and performing a plurality of n x n interframe or interframe n x n nearest-neighborhood operations; fifth single or plurality of analog horizontal and vertical interpolater or extrapolater circuit connected to said sensor means for selectively enlarging or reduction of said pixel size.

24. An analog system apparatus in accordance with Claim 21 wherein said third means for video frame differencing and preprocessing further comprises: a first filter means for identifying and estimating randomly generated signal noise embedded within the origination or remote input image; a second preprocessor means for subtracting said randomly generated signal noise from said input image by means of first sensor and a second sensor, said first sensor sensitizing noise for comparison of said sensor and subtraction of said noise signal; a third tracking and holding means for receiving signals from sensors and selectively referencing previous frames from current frame; a fourth means for horizontal and vertical addressing and sample and holding circuit, for receiving continuous sequence of live video input and either producing corresponding analog video signals, referring to selective sets of random video frames, or extracting specific set of region of interest subimage, i.e., in macroblock form, from said input;

SUBSTITUTE SHE- a fifth means for horizontal and vertical addressing, delay, comparing, and registration circuit connected to application priority interface, system controller, and bandwidth-overlay- look-up-table, which receives priority requests for application- specific articles, and identify and generate corresponding region of interest subimage for each article; a sixth preprocessor means for subtracting said previous frame signal from said current frame input signal by means of first sensor and a second sensor, said first sensor sensitizing and hold¬ ing previous frame for comparison of said second sensor sensitizing and holding current frame and performing subtraction between two holding signal for deriving frame differencing signal; a seventh shift register means for sequential registering frame difference signals, and a eighth analog-to-digital conversion and storage means for digitizing and buffering frame differencing signal;

25. An system apparatus in accordance with Claim 24 wherein said fourth means for parallel image sensing, frame differencing, and preprocessing further comprising: first means for performing an integrated parallel sensor and storage array whereby a plurality of input frames can be sensitized, digitized, and stored; and second means for performing parallel image sensitizing and comparison of a plurality of interframe or intrafra e input images, and producing a plurality of corresponding frame differ¬ encing bit-maps and associated frame differencing data objects, whereby only said frame differencing data objects are required for

SUBSTITUTESHEE^" further interframe or intraframe processing including motion estimation, discrete cosine transform, quantization, and huffman¬ like variable and run length coding.

26. An system apparatus in accordance with Claim 21 wherein said fourth means for motion processing further comprises parallel processor means for performing pixel level operations for an n x n pixel block subimage, said means comprising: first m x m array of said parallel processor means wherein n an m are integers > 2, and said parallel processor means can retrieve, decode, and execute instructions for pixel input data from internal memory or neighboring parallel processor internal memory; second interconnection means for selective communication between each of said parallel processors whereby said pixel data inputs and incoming control messages can be retrieved from said selective parallel processors and said pixel data outputs and out¬ going messages can be broadcasted to said selected parallel processor neighbors; and third processor means further comprising parallel array of comparators applying to each of said n x n subimage blocks for comparison of said previously displayed frame and said current frame, said n x n subimage performing matching functions at said frame differencing locations for generating horizontal and vertical distance for representing said motion vector.

27. An system apparatus in accordance with Claim 26 wherein said means for motion processing further comprises means

E SHEET for recognizing patterns, estimating and compensating motion for said image moving objects, comprising: first means for extracting each of the n x n luminance pixel blocks from said previous reference frame and for forming template matching operations of said n x n reference blocks within m x m window of surrounding pixel blocks residing in said current frame wherein m and n are integers and m is greater than n, said template matching producing a motion vector for said pixel block; second means for determining and adjusting said m x m window size; and third means for performing n x n pattern recognition.

28. An system apparatus in accordance with Claim 21 wherein said fifth means for object reference profile wherein a motion video object can be applied with selective one from a plurality of pattern searching and matching algorithms to derive alternative reference motion vectors wherein said motion video profile can forward or backward reference said motion video object during said interframe sequencing utilizing a combinatorial vector comprising a plurality of detected edges whereby said motion video object can be characterized by a symbolic representation of said object for further transmission, storage, and display.

29. An system apparatus in accordance with Claim 28 wherein said fifth means for image preprocessing and motion pro¬ cessing further comprises: a means for motion estimation and motion compensation operations of moving video objects comprising:

TITUTE means for performing edge detection operations for each of said frame differencing data objects whereby a plurality of detected edge sub-objects are derived to identify changes in motion video objects during motion estimation and compensation operation; means for performing feature extraction operations for each of said frame differencing objects whereby said extracted specific features from said frame differencing data objects can be identified for said motion video objects during said motion estimation process; and means for extracting features and comparing of said previously displayed frames with said current frame.

30. An system apparatus in accordance with Claim 29 wherein said motion video object including a plurality of moving foreground pixels overlaying with still image background pixels whereby said still image background pixels require single trans¬ mission of a first frame among a plurality of interframe transmis¬ sion sequence whereby said small moving foreground pixels can be detected using said interframe and intraframe differencing tech¬ niques as described in Claim 25, whereby said moving foreground pixels are characterized employing said frame differencing or a plurality of said edges and a plurality of said extracted features.

31. An system apparatus in accordance with Claim 30 wherein said means for template matching operation for each of said n x n pixel blocks of said sub-image extracting from said input image and comparing said selected n x n reference data

SUBSTITUTE SHEET stored previously in memory, further comprising: first means for identifying significant objects with said frame; second means for prioritizing said significant objects of said frame; third means for retaining color representation of each significant objects of said frame; fourth means for storing said color representation; fifth means for retrieving said color representation and performing pel domain transform operations; and sixth means for converting said color representation from said pel domain to said frequency domain for performing additional frequency domain operations such as discrete cosine transform and fast fourier transform.

32. A system apparatus improvement for adaptive computing of communication, storage, and processing bandwidth requirement for a origination or remote digital video signal in compliance with run-time networking conditions and run-time application- specific priority input requirements, and producing run-time attributes list for said signal including compression ratio, frame rate, and display resolution comprises integrated circuit further comprising: first means for look-ahead-pipeline controller element connected to preprocessor and receiving processor for receiving local or remote inbound differential video signal and motion vector and calculating and producing corresponding run-time attributes signal;

UTE SHEEt second means for controller element connected to formatting processor, encoding processor, scalable smart memory and system controller for receiving initial run-time attributes in accordance with algorithmically pre-determined default parameters retaining within system-look-up-table for producing run-time storage and run-time processing bandwidth signal; third means for generating transmission request in accord¬ ance with said bandwidth requirement for said outbound transmission of said signal; fourth means for detecting run-time bandwidth constraints wherein said detection means comprises sensitizing circuit means connected to the transmission processor for responding outbound transmission request for said video signal, said means adoptively analyzes a plurality of networking traffic conditions, and dynamically reconfigure and produce run-time attributes correspond¬ ing to the available run-time communication bandwidth; and fifth means for continuous run-time managing and control of network traffic conditions, maintaining constant bit-rate communication, and adapting bandwidth availability for transmitting said signal; sixth means for continuous run-time managing transmission buffer availability , maintaining constant bit-rate communication session, and adapting bandwidth availability for transmitting said signal; seventh means for continuous run-time managing and control of run-time attributes in compliance with run-time application- specific priority input requirement, and adapting bandwidth avail¬ ability for transmitting said signal.

SUBSTITUTE SHEET

33. A system apparatus in accordance with Claim 32 wherein said fourth means for detecting run-time bandwidth con¬ straint, and said fifth means for continuous run-time managing and control of network traffic conditions and adapting bandwidth availability for transmitting said signal further comprising: means for initiating and sending request signal to transmission processor comprising communication bandwidth require¬ ment data; means for receiving grant signal from transmission processor when run-time network condition is sufficient; or means for receiving run-time bandwidth allowance signal from transmission processor for corresponding to realistic run¬ time network traffic condition when run-time network condition is not sufficient; means for comparing said bandwidth allowance with said bandwidth requirement data; means for adjusting said bandwidth requirement in conformance with said bandwidth availability; means for compressing or decompressing said outbound transmitting signal in conformance with run-time attributes; means for continuous monitoring of said outbound transmi¬ ssion and said bandwidth constraints and selectively adjusting said outbound transmission bandwidth to accommodate availability of said bandwidth during said transmission; and means for further producing and sending request signal to the formatting processor, encoding processor, and system controller for resetting the system-look-up-table and reconfiguring the

SUBSTITUTE SHEET appropriate run-time attributes.

34. A system apparatus in accordance with Claim 32 wherein said integrated circuit further sensitizes user input or application-specific requirements and interactively updates the run-time attributes further comprising: means for user/application interface element connected to the formatting processor, encoding processor, and system con¬ troller for receiving a plurality of signal regarding user/operator preference or application-specific requirements; means for sending said signals to the encoding processor and dynamically producing a plurality of optional run-time attri¬ butes; and means for further interacting with the system controller, formatting processor, and encoding processor for determining final run-time attributes corresponding to the available communication bandwidth.

35. A system apparatus in accordance with Claim 32 wherein said integrated circuit further exchanges a variety of digitally encoded input and output foreign video signals corresponding to intrinsically incompatible video coding algorithms whereby incompatible transmission, storage, retrieval, and display apparatus can inter-operate through such interface, further comprising: means for interoperating circuit connected to the trans¬ mission processor and receiving processor for receiving inbound video signal from receiving processor according to a system-

SUBS{ [i U: L orsE look-up-table encoded video coding format; means for resetting the system-look-up-table parameters and translating inbound signal into outbound signal according to different video coding algorithm; and means for further reformatting and sending outbound signal to transmission processor for further transmission.

36. An apparatus improvement for adaptive continuous managing and control of run-time network traffic condition, main¬ taining constant bit-rate transmission session, adapting to application-specific priority input requirement, and controlling bandwidth availability for communicating video related data includ¬ ing motion video, animated graphics, and still images, and non- video related data including voice, audio, text, and ASCI, whereby said apparatus is compatible with multiple analog or digital trans¬ mission standards including analog voice grade line, public switch¬ ed digital network, basic rate ISDN, HI rate ISDN, primate rate ISDN, Tl, local area networks, metropolitan area network , broad¬ band ISDN and fiber-optics digital data interface, said apparatus comprising: means for producing transmission request signal including bandwidth requirement data for said outbound data signal; means for producing application priority request signal from local or remote destination application coprocessor including run-time application priority input bandwidth requirement; means for producing run-time attributes in compliance with application priority requests and said outbound data types; means for receiving transmission grant signal from local

Si J-. ^' . IJ**-^** or remote destination receiving processor or transmission buffer including bandwidth availability; means for relaying said bandwidth availability signal to bandwidth processor for comparison of requirement signal and availability signal; means for producing alternative run-time attributes for said transmitting signal in compliance with bandwidth availability; means for implementing alternative run-time attributes through resetting system controller's system look-up-table and means for encoding processor for further compressing said signal in compliance with said bandwidth availability; means for transmitting said signal according to said bandwidth availability; and means for monitoring and adjusting said outbound trans¬ mission session in compliance with run-time bandwidth availability.

37. An apparatus improvement in accordance with claim 36 wherein said means continuous managing and controlling bandwidth availability for producing transmission request signal and alternative run-time attributes further comprising: means for sensitizing outbound line condition changes or available remaining transmission buffer size, whereby outbound transmission bit rate is adjusted to provide sufficient composition of video-related data, said means for sensitizing said outbound line condition and maintaining constant bit rate transmission comprises selection of a plurality of quality levels for digitally compressed audio, a selection of a plurality of quality levels for digitally compressed still image data, a selection of a plurality

SUBSTITUTE Sht ___!_- t of quality levels for digitally compressed motion video data, sel¬ ection of a plurality of quality levels for compressed bit mapped or vector graphics data, and a selection of a plurality of quality levels for digitally compressed text and ASCI data, said means further comprises a selection of a plurality of options for forming composite of said digital audio, still image, video, and animated graphics; and means for selectively monitoring and adjusting said rate requirement for said outbound transmission.

38. An apparatus in accordance with Claim 36 for continuous managing and control of bandwidth availability in accordance with claim 36 further comprising: means for receiving object reference profile from preprocessor; means for receiving run-time attributes from bandwidth pipeline controller; means for receiving run-time processing and storage bandwidth availability data from system controller, encoding processor, and scalable smart memory; and means for receiving run-time application-specific input priority request and related bandwidth requirement data from application coprocessor and system controller; means for producing a plurality of predefined program sequence in conformance with a plurality of line conditions whereby each of said program sequence comprises specific selected group of video related or non-video related data objects, predefined for said bandwidth availability.

Sue

39. A method improvement for continuous monitoring network traffic conditions, maintaining constant bit-rate transmission session, adapting run-time application priority input, and managing bandwidth availability for communicating video related data including motion video, animated graphics, and still images, and non-video related data including voice, audio, text, and ASCI, whereby said apparatus is compatible with multiple analog or digital transmission standards including analog voice grade line, public switched digital network, basic rate ISDN, HI rate ISDN, primate rate ISDN, Tl, local area networks, metropolitan area network , broadband ISDN and fiber-optics digital data interface, said method comprising:

(a) producing transmission request signal including bandwidth requirement data for said outbound data signal;

(b) producing second bandwidth requirement data during run-time in compliance with the dynamic occurrence of application priority input requirement;

(c) comparing and selecting minimum of the two as effective run-time bandwidth requirement signal;

(d) repeating steps (a) through (c) as required;

(e) receiving transmission grant signal from local or remote destination receiving processor indicating bandwidth availability;

(f) dynamically receiving second bandwidth availability signal from transmission buffer during run-time transmission session indicating available remaining buffer size;

(g) comparing and selecting minimum of the two as effective run-time transmission availability signal;

BSTITUTE SHEE1 (h) repeating steps (e) through (g) as required;

(i) relaying said bandwidth availability signal to bandwidth processor for comparison with said requirement signal;

(j) producing alternative run-time attributes for said transmitting signal in compliance with bandwidth availability when network condition is not sufficient;

(k) relaying said run-time attributes to system controller;

(1) resetting system controller's system look-up-table in accordance with alternative run-time attributes;

(m) encoding, decoding, or postprocessing for further compressing said signal for meeting said bandwidth availability;

(n) transmitting said signal in conformance with said bandwidth availability;

(o) monitoring said bandwidth availability and said transmission conditions during said outbound transmission session;

(p) generating alternative run-time attributes during transmission to adapt to change in said bandwidth availability;

(q) repeating steps (a) through (p) as required.

40. An system apparatus improvement for statically receiving and determining processing and storage bandwidth require¬ ment for run-time attributes of digital input video signal, and producing internal data format structure and its corresponding instruction and task sequences according to the run-time integrated circuit processing capability, comprising: means for interpolating and extrapolating processor element connected to the bandwidth processor, which receives run-time attributes corresponding to a local or remote frame differential video signal and motion vector, and statically look-up and formulate an internal data structure according to pre-assigned system clock and system-look-up-table parameters; and means for horizontal and vertical extrapolation circuit which reconfigure internal signal blocks according to external format prior to postprocessing and outbound transmission. means for object identification, priority assignment, and horizontal and vertical interpolation circuits, said means receives digital video signal blocks from preprocessor, it will then analyze application requirements, run-time bandwidth limitations, and internal storage and processing constraints, and assign appropriate priority level for each video object, and generate appropriate internal format for further encoding, storage, and decoding of the said video signal blocks; means for static prescheduling connected to encoding processor and decoding processor for optimizing, partitioning, and scheduling coherent flow of instruction and task sequences; and means for dynamic parsing and partitioning of said instruction sequences and producing a plurality of continuous signaling bit streams for the required fine grained pipeline or parallel encoding or decoding operations of said input signal.

41. A system apparatus in accordance with Claim 40 wherein said integrated circuit further provides multi-dimensional rotation, shifting, preprocessing, and retrieval of said macroblock subimage, and dynamically invoke system calls and look-up and reconstruct run-time internal format corresponding to a plurality

- • K--s, » s" £ !-*^* Si-*:^■— _- I selections of system clock and SLUT parameters options, external networking conditions, and application priority input requirement, further comprising: means for dynamic data format processing which receives preprocessed input data signals and perform multi-dimensional access, retrieval, rotating, shifting, and preprocessing according to the internal data formatting requirement; and means for dynamic scheduling and optimizing control program connected to bandwidth processor for receiving alert signal and further reformatting said data signal macroblocks in compliance with run-time application priority input requirement and external networking conditions; and means for scaling circuit connected to the scalable smart memory for invoking system calls and dynamically adjusting system clock rate, aspect ratio, and system look-up-table parameters for providing a linearly scalable video-instruction-set-computing system.

42. A apparatus improvement for encoding internally formatted input still or motion video signal at either macroblock, group of block, partial frame, or whole frame image level in compliance with selective system throughput, or encoding selected pixel or frequency domain algorithms employing off-board copro¬ cessor elements, further comprising: first controller means for queuing instruction or task sequence connected to formatting processor, which receives and decodes prescheduled instruction task sequences for the entire encoding operation;

second means for a pixel domain encoder and a frequency domain encoder connected to formatter processor, which receives internal formatted signal, and produces bit streams of encoded tokens corresponding to a selected one of a plurality coding algorithms; third means for pipeline buffering circuit connected to scalable smart memory and packaging processor, which transfer encoded tokens to scalable smart memory for internal storage, or transfer to packaging processor for outbound transmission; fourth means for off-board data interfacing circuit, which pipeline, cascade, or parallelize a plurality of external encoding processor, and selectively encode pixel and frequency domain algorithm at macroblock, group of block, partial frame, or whole frame level.

43. A system apparatus in accordance with Claim 42 wherein encoding and decoding processor further comprises a single or plurality of processor array reconfigured as n x n cellular logic array where n is typically of value in a range of between 8 and 32 wherein said cellular logic array processor can be employed to form a plurality of block processor operations for said n x n pixel blocks of subimage, said processor array further comprising: hadamard transform means for each of said n x n blocks of said subimage whereby each of said n x n blocks is represented by its pixel value multiplied by the n x n hadamard coefficient matrix thereby reducing transmission bandwidth through pixel domain compression;

£1IRςj! i i __ s l lt— . i discrete cosine transform means whereby each of an

8 x 8 frame differencing subimage image block multiplied by an 8 x 8 coefficient block derives a 8 x 8 blocks of frame differences identified by frequency domain transform coefficients; and programmable processor means for performing other pixel domain or frequency domain forward or reverse transform operations.

44. A system apparatus improvement for transcoding or packaging video signals in CIF or SIF compatible macroblocks, and storing, retrieving, or relaying said transcoded CIF or SIF video signal in a single or plurality of packets or asynchronous transmission mode cells for inbound or outbound transmission session, further comprising: first means for protocol processing connected to encoding processor and transmission processor for receiving encoded CIF or SIF compatible video signal macroblocks from encoding processor, and producing a single or plurality of data, control, and main¬ tenance packets to transmission processor; second means for connecting said protocol processor to decoding processor and receiving processor for receiving inbound video packet or cells from receiving processor, and producing and relaying a plurality of CIF or SIF compatible macroblocks to decoding processor for further decoding; third means for pipeline buffering connected to trans¬ mission processor and receiving processor for transmitting and receiving video signals in CIF or SIF compatible packets or asynchronous transmission mode cell format;

SUB ^o _ truT S ^■J^HWPaST fourth means for message address generation circuit connected to said pipeline buffer element, formatting processor, and scalable smart memory for accessing and transferring CIF or SIF packets or cell formatted messages into video-instruction-set- computing internal formatted message via formatting processor, fifth controller means for enabling pipeline buffering circuit to systematically download and store the reformatted CIF or SIF messages in scalable smart memory; sixth reversed controller means for enabling pipeline buffer to directly retrieve internal formatted packets and cells from scalable smart memory; and seventh means for translating internal formatted messages into CIF or SIF compatible macroblocks via formatting processor. eighth means for providing a reconfigurable internal bus architecture for supporting bit-parallel packet switched mode for macroblock or packet data transfer, and including a plurality of high speed circuit switched transfer modes, which includes burst frame transfer, burst subframe transfer, burst region of interest transfer, burst object transfer, burst bandwidth-overlay-look-up-table transfer, burst frame differential bit-map transfer, and burst system look-up-table transfer; ninth means for providing packet as the smallest entity, and assembling and grouping individual data bits together for non-video data processing.

45. A apparatus improvement in accordance with Claim 44 wherein said first means further comprising: means for transcoding multiple layered signalling

SUBSTITUTE SHEET data structure according to OSI protocol architecture, which corresponds a plurality of user preference, application requirement, session control, transmission set-up, network control, and logical or physical link setup and termination, and the alike, means for transcoding non-video data types into video-instruction-set-computing internal format. means for interconnecting off-board to a plurality of external packaging processors connected via transmission processor and receiving processor, which establish, maintain, and terminate point-to-point and point-to-multipoint networking sessions, means for further parsing, assembling, or disassembling a video signal representation in CIF or SIF packets or ATM cells forms according to run-time networking conditions and specific request from transmission processor or receiving processor; means for correlating CIF or SIF macroblocks into VISC internal format; and means for transcoding voice, text, ASCI data, and other non-video data types in CIF or SIF macroblock-based packets or cells.

46. A system apparatus in accordance with Claim 44 wherein said controller means communicating with external devices including remote-controlled means or remote host processor for selective exchanging commands, data, status, and control information with said external devices, whereby external devices perform selective video control application for said controller,

SUBSTITUTE SHEET said controller means between said controlling means and said external devices comprising: means for providing command layer protocol control functions in the form of predefined commands selectively interpreted by said controller; means for updating and rearranging said predefined commands; means for prioritizing said predefined commands; means for providing transport layer protocol control functions; means for transferring data and tracking data in accordance with said command layer protocol; and means for providing physical layer protocol control functions for implementation of termination and setup for transmission processor and receiving processor.

47. A system apparatus improvement for providing optimized control of article-oriented memory architecture to address, store, and retrieve a single or plurality quantities of background still image and foreground motion video articles in accordance with selected internal video-instruction-set- computing optimized format, whereby said apparatus further provides set of run-time variables correspond with user, appli¬ cation, and networking conditions, and dynamically self-configure to address, store, and retrieve most-recently-optimized video articles, further comprising: first controller means for selecting a default internal file format and size based upon processing and memory system

SUBSTITUTE SHEET throughput during system initialization; second controller means for reformatting and resizing a single or plurality of memory cells and their associated sensing, control, addressing, management, and interface circuits connected to formatting processor and packaging processor for reading or writing single or plurality of articles according to said selected internally modified CIF or SIF macroblocks; means for sending request signal to said formatting processor for further reducing storage requirements for internal video signal or further resuming video data signal conforming to external data format requirements; and means for responding to a plurality of run-time application or networking queries, and referencing internal stored packet data, then reformatting and updating them according to run-time priority changes; third controller means and preprocessing means for segregating foreground motion video articles from background still image articles during interframe receiving mode; fourth controller means for producing optimized data structure for segregating article-oriented addressing, storage, and retrieval of still background and motion foreground video articles. fifth controller means for defining an optimized article- oriented instruction set including run-time priority identification tag, linear motion vector, rotation factor, reduction or enlarge¬ ment ratio, sign post, run-time parameters, class referencing parameters, and application-specific parameters; sixth controller means for receiving and decoding optimized article-oriented instruction sent from said sixth controller;

SUBSTITUTE SHEET seventh controller means for generating pointer data structure for manipulating, addressing, storing, and retrieving said still and motion video articles without physically relocating or modifying data from its memory location; eighth controller means for run-time adaptive decision logic circuit connected to system controller, bandwidth processor, and encoding processors for receiving set of run-time variables correspond with user, application, and networking conditions, and producing run-time executable configuration; ninth controller means for producing a fast run-time memory buffer data structure for temporary storage and retrieval of most-recently-optimized video articles.

48. A system apparatus in accordance with Claim 47 wherein said scalable smart memory provides set of alternative referencing parameters for dynamic moving, overlaying, rotating, enlarging, or reducing a single or plurality of motion video articles at run-time without physically modifying or moving their address and data, further comprising: tenth controller means for pointer manipulation circuit connected to encoding processor, decoding processor, system cont¬ roller, preprocessor, and postprocessor for receiving run-time requests to move, overlay, rotate, enlarge, or reduce a single or plurality of these stored video articles; eleventh controller means for producing appropriate alter¬ native referencing parameters to dynamically manipulate articles without physically modifying or moving their address and data; twelfth controller means for self-synchronizing pipeline

SUBSTITUTE SHEET circuit connected to system controller, preprocessor, bandwidth processor, formatting processor, encoding processor, packaging processor, and decoding processor for coordinating system synchronization; thirteen controller means for sensitizing and registering a plurality of abnormal run-time instances for subsystem pipeline operations, including preprocessor, bandwidth processor, formatting processor, encoding processor, decoding processor, and packaging processor; fourteenth means for sending alert signal to system con¬ troller in requesting further fine-tuning of system look-up-table; fifteenth means for system controller examining bottle- necked subsystems, identifying their overload causes, and then authorizing and adjusting system-look-up-table parameters to properly scale down selective subsystem processing requirement without degrading overall system performance; sixteenth controller means for selective issuing set of video-instruction-set-computing system control signals including, but not limited to task redirection, data reformatting, NOP insertion, threading, or delay branching; and seventeenth controller means for adjusting specific overloaded subsystem, and resuming overall system pipeline synchronization. eighteen means for interfacing with formatting processor for dynamically adapting the priority levels, quality requirements, and networking constraints in accordance with run-time networking and application requirements.

SUBSTITUTE SHEET

49. A preprocessing and postprocessing apparatus in accordance with Claim 47 wherein said processors selectively reconfiguring scalable smart memory comprising: means for selecting a default internal file format and size based upon encoding processor and frame memory system throughput during system initialization; means for retrieving external algorithmic coded input signal, said means comprising a receiving processor; means for receipt of local origination signal, said means comprising a capturing processor; means for identifying and receiving input article's selected types and file size during said receiving and capturing stage, said means comprising a image preprocessor, a system controller, and their interface to said capturing processor and said receiving processor; means for providing adequate downsampling ratio, said receiving article can be conformed and reduced to said predefined internal file format and size, said means comprising a reconfiguration processor and a sealer circuit; means for generating article-oriented instruction format from said system controller and packaging processor; means for decoding said article-oriented instruction; and means for manipulating said internal file article; means for self-adjusting said article's internal file format and size in conformance with bandwidth availability, said means comprising a bandwidth management processor in communication with said sealer circuit, reconfiguration processor, and said system controller;

SUBSTITUTE SHEET means for continuous adjusting said internal file format and size during run-time interframe coding mode in accordance with application-specific priority reassignment or run-time network conditions changes; means for dynamic reformatting and resizing motion video foreground articles and still background articles individually to proper size, wherein sequence of static background articles can be burst transmitted during prior intraframe coding mode, said means comprising a frame differencing processor, transmission processor, and motion processor interface with said encoding processor; and means for providing upsampling ratio during transmission, ■ wherein said transmitting article can be conformed and expanded to a selectively-desired file format and size, said means comprising said reconfiguration processor, and said sealer circuit interfacing with said bandwidth management processor.

50. A frame memory apparatus for executing a plurality of preprocessing and postprocessing functions for the communication and storage of video-related articles including motion video, still image, animated graphics, and non video- related articles including audio, voice, text, and ASCI data, said memory apparatus compatible with multiple standard and custom coding algorithmic signals such as H.261, MPEG, JPEG, EDTV, HDTV, said apparatus comprising: means for definition of a default internal file format and size based upon processing and frame memory system throughput; means for receipt of an external algorithmic signal,

SUBSTITUTESHEET said means comprising a receiving processor; means for receipt of a locally-originated signal, said means comprising a capture processor; means for downsampling wherein said input article can be conformed and reduced to a predetermined internal file format and size, said means comprising a reconfiguration processor; means for manipulation of said internal file format and size, said means comprising a system controller and encoding processor; means for adjustment of said internal file format and size to conform to bandwidth availability, said means comprising a bandwidth control processor in communication with said sealer circuit, said reconfiguration processor, and said system controller; means for adjustment of said internal file format and size during interframe coding, wherein motion video foreground article and still background articles are differentiated and separately transmitted, said means comprising a frame differencing processor and motion processor interfacing with said encoding processor; means for providing upsampling during transmission wherein said transmission of said video-related or non-video related article can be formatted and sized, said means comprising said reconfiguration processor, said sealer circuit interfacing with bandwidth control processor.

51. A frame memory apparatus for executing a plurality of preprocessing and postprocessing functions in accordance

SUBSTITUTE SHEET with Claim 50, further comprising: an integrated parallel processor and storage array means for sensitizing, digitizing, and storing a plurality frames of said input image signal; a parallel image preprocessor array means for performing real-time image encoding operations, said preprocessing array including a frame differencing operation whereby during image encoding operations, said input images can be sensitized, and compared to develop frame differencing articles whereby said frame differencing articles can be further processed for motion estimation, discrete cosine transform, quantization, and huffman variable and run-length coding whereby said motion estimation and motion compensation operation comprise an edge detection operation for each of said frame differencing articles whereby a plurality of detected edges articles are derived to identify said motion video articles during motion estimation and compensation operations; and a feature extraction operation means whereby extracted specific features for each of said frame differencing articles can be identified.

52. A frame memory apparatus for executing a plurality of preprocessing and postprocessing functions in accordance with Claim 51 wherein said input image comprising moving foreground pixels overlaying still image background pixels whereby said background pixels can be characterized during interframe coding, requiring single transmission, and said foreground pixels are detected utilizing said interframe differencing technique

SUBSTITUTE SHEET and said motion estimation and compensation techniques whereby said parallel processor comprising a m x m array of parallel processor elements whereby n and m are integers of equal or unequal values.

53. A frame memory apparatus for executing a plurality of preprocessing and postprocessing functions in accordance with Claim 52 wherein said parallel processor elements can retrieve, decode and execute instructions for said pixel input data in its internal buffer or in neighboring parallel processor elements, said pixel output data can be stored internally or in neighboring parallel processors in a interconnection means programming each of said processor to a selective group of neighboring processors whereby pixel data inputs and control messages can be received from said selected neighboring processors and pixel data output and outbound messages can be transmitted to said selected group of processor neighbors.

54. A frame memory apparatus for executing a plurality of preprocessing and postprocessing functions in accordance with

Claim 53 wherein said parallel processors can be arrayed and reconfigured as a n x n cellular logic array processor wherein n is a value between 4 and 64 wherein said cellular logic array processor can perform processing operations for said n x n pixel blocks of subimages including said frame differencing operations between current frames and previously displayed frames whereby a group of n x n differencing blocks are identified, registered, and stored; template matching operations for each of said n x n pixel blocks of subimages, said template matching operation extracting an input image and comparing with a previously stored input image thereby deriving a motion vector; pattern matching; motion estimation apply to each of said n x n subimage residing within said previously displayed frame; edge detection for each of said n x n blocks, feature extraction; Hadmard transform for each of said n x n blocks of subimage whereby each of said n x n blocks can be represented by pixel domain transform coefficient and a hadamard coefficient matrix, and a discrete cosine transform operation.

55. A scalable reconfigurable smart memory array method for executing a plurality of preprocessing and postprocessing functions for the communication and storage of video-related articles including motion video, still image, animated graphics, and non video-related articles including audio, voice, text, and ASCI data, whereby said method is compatible with multiple standard and custom coding algorithmic signals such as H.261, MPEG, JPEG, EDTV, HDTV, said apparatus comprises:

(a) defining a default internal file format and size based upon processing and frame memory system throughput;

(b) selectively receiving an external algorithmic coded signal or locally-originated signal;

(c) downεampling said received article to a predetermined internal file format and size;

(d) manipulating said internal file format and size to conform to bandwidth availability;

(e) distinguishing between motion video foreground article and still background articles during interframe coding and intraframe coding mode;

(f) pre-transmitting still image background article during intraframe coding mode;

(g) transmitting motion video foreground article during interframe coding mode;

(h) upsampling said internal file format to selectively-desired format and size for transmission in accordance with bandwidth availability.

(i) selectively repeating steps (a) through (h) as required.

56. A scalable reconfigurable memory array method in accordance with Claim 55 wherein said method for manipulating said internal file format to conform to bandwidth availability and distinguishing between motion video foreground article and still background articles during interframe coding and intraframe coding mode further comprises:

(a) differentiating between still image background and moving foreground pixels;

(b) performing frame differencing techniques on a n x n pixel block subimage by means of a m x m array of parallel processors whereby n and m are integers of equal or unequal values;

(c) performing pattern matching, motion estimation, edge detection, feature extraction and Hadamard transform. for each of said n x n blocks of subimage;

(d) representing each of said n x n blocks of subimage by a pixel domain transform coefficient, hadamard coefficient matrix and discrete cosine transform.

(e) selectively repeating steps (a) through (d) as required.

57. A method for the application of image preprocessing techniques for interframe coding of video and non-video related data types between transmitting processor and receiving processors whereby said method can be optionally integrated with image capture circuit and bandwidth control processor circuit, said method comprising:

(a) capturing a local origination still image or live video sequence in accordance with frame rate updating requirement set by system controller;

(b) capturing and buffering selective frames of image analog signal or digital data during intraframe coding mode;

(c) referencing selective previous frame image and comparing against current frame image in producing frame differ¬ encing signal in blocks or macroblocks;

(d) storing said frame differencing signal between said current frame and said previous frame in blocks or macroblocks;

(e) registering and coding said frame differencing blocks or frame differencing macroblocks into a frame differencing bit-map;

(f) modifying referencing number to previous frame, and repeating (c) through (e) as required; (g) converting color space from analog RGB or NTSC format into digital RGB, YUV, or alike format;

(h) retrieving luminance (Y) macroblocks according to said frame differencing bit map;

(i) setting search window size by system controller or motion processor;

(j) deriving appropriate linear motion vector for each of said macroblocks through comparing and selecting of the minimum distortion among adjacent neighboring macroblocks within said search window;

(k) modify search window size when currently selected search window can not determine a motion vector;

(1) repeating (i) through (k) as required;

(m) performing edge detection and feature extraction in determining alternative motion vector when (i) through (k) can not determine the appropriate motion vector;

(n) reducing from external file format to a internally defined format in accordance with downsampling ratio established by said system controller, said reconfiguration processor, and said sealer circuit;

(o) translating said frame differencing bit map in conformance with said internally defined format;

(p) performing selective pixel domain or frequency domain encoding or decoding operations for said frame differencing macroblocks individually for all Y, U, and V format;

(q) enlarging from said internal format to an external format according to an appropriate downsampling ratio as establish¬ ed by said system controller, reconfiguration processor, and scaler;

(r) adjusting said downsampling ratio during run-time in accordance with run-time network condition, application-specific input, priority reassignment, and maintenance of constant bit-rate transmission, said downsampling ratio is established by bandwidth processor, application priority interface, transmission processor, system controller, reconfiguration processor, and sealer circuit;

(s) performing run-time pointer manipulation operation in accordance with application priority request, whereby macroblock articles can be referenced from internal frame memory for perform¬ ing rotation, linear 4, zooming, or reduction of macroblock articles without physical movement or modification of the internal stored data;

(t) adding said frame differencing signal to said previous frame for displaying and updating current frame;

(u) repeating steps (a) through (t) as required.

58. A method for the application of frame differencing techniques, pixel interpolation techniques, and motion video simulation and annealing techniques, for the compensation of live video, still image, audio, or animated graphics article, said compensation method comprising:

(a) capturing the scene of motion video article through frame differencing techniques;

(b) representing the motion video profile of said article in a article-oriented format including significant extracted features, edges, and colors;

(c) defining said motion video profile of said motion video article by individual article and relative movement assoc¬ iation relating to still background;

(d) augmenting said motion video profile of said motion video article by means of rotation, shifting, and shuffling operation;

(e) reformatting said motion video profile of said motion video article to a internal file format;

(f) enlarging or reducing said motion video article by means of upsampling or downsampling operations;

(g) overlaying said motion video profile of said motion video article with said voice, audio, text, still back¬ ground, or said animated graphics data to construct a complete multimedia presentation;

(h) defining said motion video profile of said motion video article;

(i) interpoating said motion video profile of said motion video article through regeneration of absent pixels to further refine said motion video articles;

(j) simulating said motion video profile of said motion video article through capturing and recording selective non-destructive frame sequences;

(k) annealing said motion video profile of said motion video article through reconstruction of said absent pixels, absent moving objects, or absent frames employing graphics modeling and annealing techniques;

(1) selectively repeating (a) through (k) as required.

59. A system apparatus improvement for transceiving video signal according to a plurality of networking environments and a plurality of run-time bandwidth conditions comprising: means for a single or plurality pairs of transceivers corresponding to a plurality of analog or digital networks including, but not limited to broadband ISDN, FDDI, MAN, DS3, twisted-pair or coaxial LAN, switch or dedicated Tl, primary rate ISDN, fractional Tl, frame relay, ISDN switched HI, single or dual channel basic rate ISDN, digital switched or private PSDN, analog twist-pair, basic rate ISDN D channel, or analog twist-pair; means for bandwidth processing further comprises sens¬ ing circuit connected to transmission processor and receiving processor for constant monitoring of run-time networking condition and generate "alert" signal to system controller when it receives "change of run-time communication bandwidth" signal; means for system controller generating "network congest" query and comparing against required run-time outbound transmission bandwidth during that time period; When outbound transmission bandwidth is more demanding: means for bandwidth processor transferring outbound transmitting signal to formatting processor for further reduction of bandwidth prior to transmission. means for generating run-time bandwidth allowance which includes, but not limited to: 150 Mbs, 100 Mbs, 45 Mbs, 10 Mbs, 2.048 Mbs, 1.544 Mbs, 384 Mbs, 128 Mbs, 64 Kbs, Px 64 Kbs, 56 Kbs, 19.2 Kbs, or 9.6 Kbs. means for self-configuration circuit connected to system controller and encoding processor for receives application requirements and user preference, and generating the appropriate run-time bandwidth signal.

60. A system apparatus improvement for corresponding a bitstream of encoded tokens, and decoding them into still or motion video signals according to a plurality selections of pixel or frequency domain coding algorithms, said apparatus further perform selective pixel or frequency domain algorithm at either macroblock, group of block, partial frame, or whole frame image level, and employ either software program, internal hardware, or off-board external coprocessor for achieving selective system throughput, further comprising: means for programmable decoding connected to scalable smart memory and packaging processor for receiving bit stream of encoded tokens, and producing sequence of fully decoded still image or motion video signal corresponding to a plurality of externally selective image coding algorithms; means for queuing instruction or task for storing sequence of preseheduled instruction tasks for the entire decoding procedures; means for pipeline buffer circuit connected to the scalable smart memory and packaging processor for receiving internally or externally encoded tokens from scalable memory or packaging processor for further decoding, transferring decoded signals to scalable memory for internal storage, or relaying to transmission processor for outbound transmission; means for off-board interfacing interface circuit, which can pipeline, cascade, or parallelize a plurality of external decoding processor elements, or simply employ internal hardware or software program to decode pixel and frequency domain algorithm at macroblock, group of block, partial frame, or whole frame level.

61. A system apparatus improvement for producing selective set of live video, sequential stills, or bit-mapped graphics image output signal corresponding to the video-instruc¬ tion set-computing internal signal blocks further comprising: means for displaying adaptor connected to decoding processor for receiving video-instruction-set-computing compatible internal processing signal, and producing VGA, RGB, or NTSC compatible output signal for further displaying live video, animated graphics, or sequential image output; means for printing and faxing adaptor connected to decod¬ ing processor for producing G3, G4, or encapsulated postscript compatible output file for further printing or faxing a freeze frame of motion video, still image, or still graphics; means for pipeline data buffering for receiving processing signal from decoding processor; and means for postprocessing non-video related data streams.

62. A system improvement for controlling and coordinating individual subsystem pipeline operations, maintaining and managing data throughput, sensitizing and adapting external application, user, and networking conditions, maintaining and updating run-time executable system look-up-tables, and providing static task preseheduling and look-ahead instruction presequencing, supervising run-time coordination among subsystems, and reconfigur- ing subsystem's function in maintaining scalable system throughput, further comprising: means for dynamic storage and retrieval data structure for implementing system look-up-table connected to preprocessor for receiving frame differential bit-map corresponding to specific input video data type, said means further cross reference said bt-map and generate a set of run-time attributes including, but not limited to: compression ratio, frame rate, and display resolution; means for task prescheduler circuit connected to system- look-up-table and all video-instruction-set-computing pipeline subsystems for translating selective run-time atributes into specific set of run-time encoding, decoding, and frame updating instruction seguences for pipeline encoding processor, decoding processor, and post processing circuits; means for analyzing each subsystem's pipeline operations requirement, and generating appropriate scalable internal data format in accommodating the available hardware processing and memory throughput; means for dynamic reconfiguration processor circuit for sensitizing and adapting with system's run-time critical paths, subsystem bottlenecks, networking conditions, user preference, and application's requirements, and inserting conditional branch instructions, and relieving specific overloaded subsystem's func¬ tional requirement in maintaining the overall system throughput.

63. A system apparatus in accordance with Claim 62 wherein improvement for look-ahead bandwidth and instruction management for receiving and referencing frame differential bit-maps to system-look-up-table, deriving run-time attributes and run-time transmission and processing bandwidth, and prefetching pipeline instructions for execution, further comprising: means for bandwidth capability circuit connected to encoding processor and scalable smart memory for receiving process¬ ing, storage, and transmission bandwidth availability signals as system initialized; means for bandwidth requisition circuit connected to pre¬ processor for receiving frame differential bit map and producing bandwidth requisition signal; means for comparing bandwidth capability signal and bandwidth requisition signal in determining whether run-time communication and processing bandwidth is sufficient; when bandwidth is available: means for a system look-up-table, instruction-look-ahead, and instruction-group-fetch circuit for receiving said frame differential bit-map signal, and prefetching prefetch group of ^'video-instruction-set-computing instructions, when bandwidth is not available: means for issuing conditional branch, transferring video data to formatting processor for further bandwidth reduction; and formatting processor means comprising object identifi¬ cation, priority assignment, and horizontal and vertical interpolation circuits for receiving full frame or subframe of digital video data, reducing bandwidth requisition signal for lower priority objects through horizontal and vertical interpolations.

64. A system apparatus in accordance with Claim 62 wherein improvement for run-time application priority management further comprising: means for priority assignment circuit connected to bandwidth-look-up-table for receiving set of application-specific input priority requirements, and generating corresponding set of media data types conforming to the run-time bandwidth availability; when run-time bandwidth is not available: means for interfacing priority assignment circuit to formatting processor for producing bandwidth reassignment signal; means for formatting processor producing alternative internal format corresponding to the reassigned priority list, and associate each priority reassignment signal with corresponding run-time object-identification number; means for application processor interconnected to prepro¬ cessor for receiving preprocessed non-video data types, and preforming traditional RISC or CISC like application processing; and means for said application coprocessor interconnected to application priority interface, for receiving priority reassignment signal and producing bandwidth requisition signal to bandwidth-overlay-look-up-table for further assurance of run-time bandwidth availability.

65. A single-chip integrated circuit apparatus improvement optimized for video-instruction-set-computing for providing unified microprocessor solution for all video-related system application including digital TV, videophone, and multimedia computing comprises a plurality of independent functional units for executing tasks including remote communication, bandwidth adapta¬ tion, application control, multimedia management, and universal video encoding, whereby said apparatus achieves most optimum performance for video-related data types through run-time scalable, reformatable and adaptable processor and memory, run-time bandwidth management, instruction look-ahead, group-instruction-prefetching, superscaling, and superpipelining, said apparatus is equally capable of processing traditional non-video related data types and maintains compatibility with traditional application programs and operating system environments including DOS, WINDOW, NT, OS2, UNIX, or alike, said apparatus comprising: first means for bandwidth adaptation including a bandwidth controller for continuous monitoring, detecting, and management of run-time networking conditions, said controller means comprises circuit for receiving frame differential bit-map from preprocessor and generating run-time bandwidth requirement signal for each input frame signal, said controller means also comprises circuit for receiving processor, storage, and communication bandwidth constraints during system initialization, said controller means further comparing said constraint signal with said require¬ ment signal in determining whether system has sufficient run-time bandwidth, when run-time bandwidth is sufficient, said controller means will further generate a grant signal to system controller means and enable system-look-up-table controller circuit for searching and prefetching run-time instruction sequences, when run-time bandwidth is not sufficient, said controller means will issue a conditional branch and request formatting processor means for reformatting input frame signal in further reducing said run-time bandwidth requirement signal; second means for application control including application priority interface controller for managing run-time application specific input requirements, said controller means comprises inter¬ facing circuit for receiving application priority requirements and generating message signal to formatting processor means in further assigning run-time priority level to each input object or subimage, said controller means also comprises driver circuit for direct interfacing with off-board commercial application program entities, said controller means further comprises programmable BIOS circuit for direct execution under commercial operating system, said controller means further comprises bit-parallel input and output for external microprocessor interface, said controller means further comprises means for direct control and manipulation of motion video articles through external application program structures and commands, wherein said means further comprises object-oriented instruction means which provides file descriptors for foreground live or animated motion video objects and background still image or graphics objects, said means further includes but not limit to the following object descriptors: linear motion vector, rotation factor, enlargement or reduction ratio, priority assignment level, sign post, classification reference pointer, address reference pointer, and alike; third means for multimedia management including bandwidth overlay look-up-table controller for run-time multimedia data management, said controller means comprises fast look-up-table circuit for dynamic searching and said circuit generating a plural¬ ity options for selecting desirable run-time data types, said con¬ troller means also comprises priority sensitizing circuit for rece¬ iving run-time application priority request from said application controller means and producing selective run-time media profile for further execution; fourth means for multimedia processing including packaging processor for high speed run-time multimedia data processing, said processing means comprises packetization circuit for receiving internally reformatted macroblock subimage and partitioning into a plurality of data packets, said processing means also comprises packet controller circuit for producing control messages for internal and off-board data movement, error detection and correct¬ ion, flow control, and maintenance functions, said processor means further comprises layered protocol controller in conformance with open system interconnection international standard for com¬ municating with remote multimedia processor means or other protocol controller means for performing point-to-point or point-to-multi¬ point multimedia communication sessions, said processing means further comprises internal or external bus controller means for further programmable controlling of system-wide data bus means and control bus means, said bus controller means comprises a plurality of bus transfer controlling modes: standard default mode allows said bus means to perform packet switched transportation of data, control, and operation maintenance packets among various system component means, said bus controller means also comprises selective optional bus transfer modes, wherein said bus means can be reconfigured into a plurality of selective circuit switched modes for allowing burst transportation of full frame, partial frame, live motion objects, graphics overlay, or other non-video related data types, said bus controller means further comprises means for reconfiguring said bus means to selective bit-serial, bit-parallel, unidirectional, or bidirectional modes; fifth means for universal video encoding and decoding including programmable signal processor for selective execution of a plurality pixel domain or frequency domain coding algorithms, said means comprises programmable encoding circuit for receiving input macrblocks and generating selective JPEG, MPEG, H.261, VQ, or alike encoded video tokens in accordance with application input requirement, said means also comprise programmable decoding circuit for receiving encoded macroblock tokens and generating selective JPEG, MPEG, H.261, VQ, or alike macroblock pixel data; sixth means for scalable formatting comprises programmable processor connected to all functional units for inter-operating arbitrary external video formats and adapting to selective internal format depending upon system throughput and configuration, said processor means comprise object identification circuit for receiving standard macroblock subimages from preprocessor means and producing moving object lists for each input frame, said processor means also comprises priority assignment circuit for receiving application requirements from said application controller means and receiving all bandwidth constraints from said bandwidth controller means, said means then evaluate and assign run-time priority level to each moving object and macroblock subimage, when run-time bandwidth is not sufficient, said processor means further comprise horizontal and vertical interpolation circuit for receiving macroblock subimages, moving objects, and correspond¬ ing run-time priority levels and producing reformatted macroblock subimage and moving objects, after run-time bandwidth is modified to become sufficient, said processor means further comprise trans¬ ferring circuit for transporting said formatted macroblock subimage to said encoding means and said packaging means for processing, said processor means further comprises scalable power management, and scalable clock rate adaptation circuit for automatically reconfiguring said interpolation and said extrapolation requirement in accordance with an internally stored system-look-up-table controller means for meeting a variety of system application, environmental, and operating conditions; seventh means for scalable smart memory connected to all functional units and scalable formatter for accessing, storage, and transferring video related data blocks according to selective internal format; said memory means comprise a scalable memory array for receiving data packets from said packaging means and automatic¬ ally self-configuring into a plurality of internally structured entities for temporary storage, said memory means also comprises reformatting circuit for either receiving run-time bandwidth cons¬ traints from said bandwidth controller means or receiving run-time application priority request signal from said application control¬ ler means and self configuring to selective data storage format conforming to said run-time networking and application requirement, said memory means further comprise foreground processing circuit for transceiving, prioritizing, manipulating, and storing fore¬ ground moving live or animated data objects, said memory means further comprises background processing circuit for transceiving, prioritizing, manipulating, and storing still background data macroblock, said memory means comprise data manipulating circuit for modifying and producing address pointer reference list to rotate, linear shift, enlarge, or reduce designated data object, said memory means further comprises simulating circuit for executing predefined instruction sequences for visualizing artificial knowledge or animated object data movement; eighth means for application coprocessing comprising a sin¬ gle or plurality embedded on-board or off-board coprocessors for executing commercial application software programs including DOS, UNIX, or alike, said coprocessor means comprise RISC or CISC processor circuit for receiving preprocessed non-VISC data types from preprocessor means and executing traditional data processing tasks. ninth means for interfacing controller for real-time sched¬ uling, synchronization, and simultaneous execution of non-real-time application program sequences and real-time video-instruction- set-computing; said interface controller in conjunction with said application controller means jointly maintain compatibility of a video-instruction-set-computing system with commercial application programs and provide system-independent execution environment in conforming with all commercial operation systems; tenth means for preprocessing comprising video preprocessor and motion processor circuit for receiving analog or digital video, graphics, or still image input signal and generating frame differential bit-map, motion vectors, and macroblock subimages for each input frame signal, said preprocessor means also comprises preprocessing circuit for receiving other non-video related data types including audio, text, and ASCI data, and preprocessing them into video-instruction-set-computing executable formats; eleventh means for system controlling for run-time overall system management of video-instruction-set-computing apparatus, said means comprising system look-up-table controlling circuit for receiving frame differential bit-map from said preprocessor means and producing run-time attributes for each input frame signal including compression ratio, frame rate, and display resolution, said controller means also comprise instruction-look-ahead circuit for receiving grant signal from said bandwidth controller means and translating said run-time attributes into a group of predefined instruction address pointers, said controller means further comprise group instruction prefetching circuit for said instruction pointers and producing predefined run-time executable instruction sequence for each input frame signal; twelfth means for transmission and retrieval processing for run-time network communications, said processor means provides control interface to a plurality of telecommunication, microwave, radio, satellite, or data communication networks, said processor means comprise a single or plurality pairs of transceiving circuits for importing and exporting video or non-video related data, said means comprises transmission processor which further comprise query circuit for generating outbound request signal to inquire external networking and bandwidth stability conditions, said means also comprises receiving processor which further comprise interpre¬ ting circuit for receiving and decoding inbound acknowledgment signal in determining and decode external networking and bandwidth condition, when abnormal networking conditions occurs, said processor will further generate alert signal and send to said bandwidth controller means for further conditional branching; thirteenth means for high speed control bus for internal high speed interconnection of system components for communicating control and maintenance signals, said bus means comprising a single or plurality of system-wide interconnections among all system components, and a plurality of private interconnections among designated group of system components, said control bus means allows all controller means including system controller, bandwidth controller, application priority controller, and system- look-up-table controller to control, access, and communicate with remaining data processing means including bandwidth processor, preprocessor and motion processor, postprocessor, formatting proce¬ ssor, encoding processor, decoding processor, application coproce¬ ssor, packaging processor, and scalable smart memory; fourteenth means for high speed data bus for internal high speed interconnection of system components for communicating video or non-video related data signals, said bus means comprising a single or plurality of system-wide interconnections among all system components, and a plurality of private bus interconnections between designated system component means for facilitating high speed pipeline data transportation; fifteenth means for post processing comprising a single or plurality of analog and digital postprocessors for receiving analog or digitally decoded video pixel run data and producing properly formatted signal for NTSC, PAL, SCAM, RS-170, G3, G4, EPS, TIFF, GIS, or alike; sixteenth means for other miscellaneous functional means, hardware means, and instruction means including but not limit to: data manipulation, data movement, program manipulation, program status manipulation, clock generation and distribution, power mana¬ gement, voltage regulation, instruction cache, parallel instruction scheduler, multiple ported register file, address ALU, bus request queue, multiple channel DMA controller, interrupt controller, data RAM, graphics unit, bus and cache controller unit, data cache, memory management unit, integer unit, floating-point unit, sequen¬ cer, data unit, and alike;

66. A integrated circuit apparatus in accordance with Claim 65 wherein said fourteen means for system-wide pipelining operations and corresponding dedicated private interconnections for high speed transportation, real-time data processing, and internal data communication of single or selective group of video-related articles corresponding to local origination data, application priority data, and remote communication data, said pipelining means comprising: first multiple stage pipelining bus means for fast local origination video data article transportation comprising direct interconnection in between each following system component means: said preprocessor means, said scalable processing means, said encoding processing means, said multimedia packaging and processing means, said scalable smart memory means, and said transmission or post processing means, first-in-first-out memory or register means are inserted where appropriate for further facilitating said pipeline interconnections, said pipeline means is presently selected as a six stage pipeline, which can be easily redesigned for other implementations to meet system and application requirement; second multiple stage pipelining means for high speed run¬ time application priority data transportation comprising direct interconnection in between each following system component means: said application priority controller or said application coprocess¬ or means, said bandwidth overlay look-up-table controller means, said bandwidth controller means, system-look-up-table controller means, scalable formatting processor means, and said multimedia processing and packaging processing means; first-in-first-out memory or register means are inserted where appropriate for further facilitating said pipeline interconnections, said pipeline means is presently selected as a six stage pipeline, which can be easily redesigned for other implementations to meet system and application requirement; third multi-stage pipelining means for high speed run-time remote video communication data transportation comprising direct interconnection in between each following system component means: said receiving processor means, said multimedia packaging and processor means, said scalable smart memory means, said scalable formatting processor means, said encoding or decoding processor means, and said post processing or transmission processing means, first-in-first-out memory or register means are inserted where appropriate for facilitating said pipeline interconnections, said pipeline means is presently selected as a six stage pipeline, which can be redesigned for other implementations to meet system and application requirement; fourth private bus means for high speed run-time point-to-point data transportation including, but not limit to direct interconnection between the following subsystem means: a. between said multimedia packaging and processor means and said transmission or receiving processor means, b. between said scalable formatting processor means and said encoding processor means, c. between said encoding processor means and said decoding processor means, d. between said decoding processor means and said application coprocessor means, e. between said application coprocessor means and said application priority controller means, f. between said decoding processor means and said post processor means, g. between said preprocessor means and said scalable formatting processor means, h. between said preprocessor means and said syste - -look-up-table controller means, i. between said multimedia packaging and processing means and said scalable smart memory means, and j. between said system-look-up-table controller means and said bandwidth controller means;

67. An integrated circuit apparatus improvement for data communication and signaling transportation among video- instruction-set-computing's internal functional units and off-board external functional units means comprises a plurality of input and output signal pin connectors for bit-parallel transport¬ ation of inbound or outbound video or non-video data and address signals, bit-parallel transportation of control, address, and data signals for external microprocessor and application program, bit parallel transportation of control and tag input and output signals, bit-serial communication of inbound and outbound data, control, and maintenance signal, and bit serial communication for inbound or outbound analog video or non-video data signals, said means further comprising: first means for thirty two (32) unidirectional video input pins, said means connected to preprocessor means for receiving four set of individual eight-bit digital color input data, i.e., red (R) , green (G) , blue (B) , and auxiliary (X) , said input pins means can be further multiplexed for receiving external address, status, tag, control, and interface signals, and transporting program and data directly from external memory; second means for thirty two (32) unidirectional video output pins, said means connected to postprocessor means for producing four sets of eight-bit output signals for external display, printing, or fax, said output pins means can be also time multiplexed for transmitting internal address, status, tag, control, and interface signals, and directly transporting program and data to external memory; third means for combining said first data input means and said second data output means to conform a single group of thirty two (32) bit bidirectional input/output pins for a baseline implementation through further time multiplexing of input and output signals; fourth means for optionally including additional input and output pins for transporting address, tag, control, status, analog video, and other non-VISC data signals; fifth means for thirty-two (32) bidirectional parallel input and output (PIO) pins for interfacing with external micro¬ processor and application programs, said means connected to application priority controller means, which is multiplexed to import and export a plurality of control and data types, including but not limited to: application and system control commands, priority requests, application requirements, communication and status messages, and program and data seguences; sixth means for further reduction of said fifth parallel input and output means from a thirty two (32) pin implementation to eight (8) bit or sixteen bit (16) implementation for more economical baseline implementation; seventh means for further expansion of said fifth parallel input and output means from a thirty two (32) pin implementation to sixty four (64) or higher pins for higher performance implementation; eighth means for further connecting said fifth means to time-division-multiplex (TDM) , packet-division-multiplex (PDM) , local bus interface (LBI) , or small computer system interface (SCSI) circuits in order to facilitate the high speed burst data transportation; ninth means for twenty four (24) bit combination of bidirectional or unidirectional tag/control input/output pins. said means connected to the system-look-up-table controller means and system controller means for providing the required power, ground, clocking, status, control and tag signals; tenth means for eight (8) bit unidirectional serial communication input and output (SIO) pins, said means connected to transmission processor means and receiving processor means for simultaneously receiving and transmitting four channels of full- duplex video signals, said pins numbers can be further modified according to application requirements; eleventh means for further modification of said input and output pins for any of said previous means in conformance with application, system, or performance requirement; twelve means for including additional input and output pins for miscellaneous functional, control, or data signal requirement.