WO1988004861A1

WO1988004861A1 - Audible or visual digital waveform generating system

Info

Publication number: WO1988004861A1
Application number: PCT/US1987/003363
Authority: WO
Inventors: Joseph Charles Lyons; Franklin Hugh Schwartz
Original assignee: Joseph Charles Lyons; Franklin Hugh Schwartz
Priority date: 1986-12-23
Filing date: 1987-12-17
Publication date: 1988-06-30

Abstract

A method and apparatus are used for selectively manipulating data with or among computing machines. Graphical (440) and/or sound (450) data is represented in terms of categories and objects. Categories (400) are used to define the characteristics and attributes of an object (410). The interactions (420, 460) of categories determine the synchronization, manipulation, animation and sequencing of an object. Specific interactions between categories are effected (430) through adding, subtracting, multiplying or dividing discrete digital elements of the respective categories. Such discrete digital elements may be extracted from a user waveform (300, 310, 320), or alternatively, obtained from a library of categories (400).

Description

"AUDIBLE OR VISUAL DIGITAL WAVEFORM GENERATING SYSTEM".

BACKGROUND OP THE INVENTION

This invention relates generally to systems for manipu lating data, and more particularly, to a system for efficien real time synchronization, manipulation, animation and seque cing of graphical and sound data.

Conventional computerized systems, such as those syste which display an image on a display screen, generally are required to perform a large amount of high order mathematica operations, memory store and memory retrieve steps to achiev for example, displacement of the image on the screen. Often times, such mathematics is performed on a pixel by pixel and frame by frame basis. In essence, an algorithm having embed therein the formula which describes the desired displacement applied to each pixel of the freuse individually to calculate the new position of each such pixel in the subsequent frame. number of such pixel by pixel calculations are assembled to form a single frame. Each frame therefore represents a smal displacement of the image with respect to its location in th immediately prior frame.

In the conventional computerized graphical system described hereinabove, processing time required includes an enormous number of calculations'and computer memory operatio which define image displacement on the screen, such as motio along a path, rotation, scaling in size of an image and the like. The number of calculations which must be accommodated a programmer increases significantly with increased resolution and number and complexity of images; of course, with these increases, the processing time required for the displacement o images increases accordingly.

One significant drawback of such a conventional computer ized graphical system is that, in situations where it is desired to monitor or change the video image at some point in the future of the progression of the image on the display, processing of the display is generally halted to permit, the introduction into the controlling algorithm of a new formula o parameter which will govern image displacement. One generally cannot in such a system, while the image is being processed, ask to see the condition or location of an object in said display at a future moment in time. Furthermore, one cannot generally modify the conditions or parameters involved in the execution of said controlling algorithm in any real time sense This inability on the part of a user of a conventional compu¬ terized graphical system to easily monitor and/or modify futur and/or present states of the display in real time, i.e., while the computer is processing prior and/or present frames, carrie over to almost all computer uses, and is not limited to the processing of data which is intended for display only. More particularly, the above-mentioned difficulties encountered in computerized graphical systems pose similar problems in computerized sound systems such as digital musical synthesizer which create, sample, reproduce, modify and output sound data as well as in combination computerized graphical and sound systems. Conventional digital musical synthesizers generally perform a large amount of mathematical operations, memory stor and memory retrieve steps to create or modify a sound. Oftentimes, such mathematics is performed on the audio equivalent of a graphical pixel by pixel or frame by frame basis. In essence, an algorithm having embedded therein the formula which describes the desired manipulation is individu- ally applied to each digital element contained within a seri of successive memory locations, or waveform table, in order calculate new manipulated digital samples which are input to new waveform table. A waveform table contains discrete digi elements which are associated with a specific waveform and m be processed to produce sound. The new waveform table thus contains the information necessary to output the modified sound. A large number of such digital elements contained within the waveform table are assembled and processed by a Digital Signal Processor (DSP) which outputs a signal to an audible output device such as a speaker which produces a meaningful sound. Alternatively, a simple digital to analog (D/A) converter may be substituted for the DSP. Illustrativ of such a DSP are devices available from Intel, Motorola and Texas Instruments. Device TMS 320C30 from Texas Instruments specifically such a device.

A conventional computerized sound system such as a digital musical synthesizer creates a simple sound having a single frequency in the following illustrative manner. The synthesizer calculates numerous individual values, V, from th equation:

V = Sin X (10) by having "X" range from 0' to 360* or beyond. The set of calculated values , V, represents discrete digital elements o a waveform represented by equation 10. As is apparent, such series of calculations may be performed for increments in "X" of 1/2", 1*, 5* or any practical number of degrees, thus providing a corresponding number of values, V. The number of values, V, for a waveform of given duration will determine th quality of sound eventually produced. The calculated values, V, are successively input to a waveform table which thus contains discrete digital points, or elements, of the wavefor represented by equation 10. This waveform table is then inpu to a DSP which in turn outputs an audio signal capable of bei heard over conventional audio speakers. In the conventional digital musical synthesizer, modification of the sound created in the above manner require mathematical manipulation of the elements contained in the waveform table, i.e., series of successive memory locations, prior to inputting such elements into the DSP. For example, increase the volume of the waveform stored in a first wavefor table, a second waveform table containing volume information combined with the first waveform table. More specifically, various discrete digital elements of the first waveform table are individually added to corresponding various discrete digital elements of the second waveform table, thus producing third waveform table having various discrete digital elements which represent the sound represented in the first waveform table at a different volume. Alternatively, an equation similar to equation 10 but having a multiplier or scaling factor k may be directly utilized to produce values to be inp to a waveform table, which is subsequently input to the DSP. Modulation of a first waveform by a second waveform is accomplished by combining corresponding elements of a first a second waveform table and providing the result to a third waveform table which is input to a DSP. Unfortunately, such modulation is characterized by extremely complex mathematics conventional systems. Furthermore, the number of calculations and operations which must be performed increases significantly with increase sound quality; of course, with these increases, the processin time required for the manipulation of sound data increases accordingly. A programmer of conventional systems, as described abov is required to use many commands and iterations to control th discrete digital elements representing waveforms when dynamic situations are represented by a sequence of differing static states. This is particularly true where each such static sta is produced by modifying an immediately prior static state. However, any number of such commands cannot overcome the fac that execution in real time is often prohibited by the time taken to execute or modify the controlling algorithms.

SUMMARY OF THE INVENTION The present invention comprises a method and apparatus for selectively manipulating data within a computing machine real time. The present invention utilizes two schemes for describ the nature of data, namely "category" and "object".

A category is a manipulating function which is used to define the characteristics and attributes of an object. The interactions of categories determine the real time synchroni zation, manipulation, animation and sequencing of an object. An object is thus a data structure created by the interactionaction of categories and has characteristics of i component categories.

Interaction between categories is effected through adding, substracting, multiplying or dividing discrete digit elements of the respective categories. Such discrete digita elements may be extracted from a user drawn waveform, or obtained from a library of categories.

Thus, in the present invention, for example, a categor of the shape of an airplane and the interaction of additiona categories such as a flight path and scaling creates the obj of an "airplane in motion". In other words, through the interaction of the categories of airplane shape, flight path and scale, the object "airplane in motion" could be made to appear to be moving across a screen while simultaneously changing in size.

To change the nature of a graphical display such as the path of the airplane, one simply changes the path category without changing any of the other component categories. Thus an object is defined by the interaction of all of its categories and modification, deletion or addition of any component category generally xesults in modification of the object. Therefore, the nature of an object and/or its interaction with other objects can be modified with ease by changing any of its component categories. Additionally, objects can be nested vithin -each other to produce more comple objects.

Similarly, in -the present invention, for example, a category of pitch interacted with a category of amplitude creates an object of sound- having a specific pitch and a specific amplitude defined by the corresponding pitch and amplitude categories.

Such logic is extended "to the formation of an object fro any practical number of categories. For example, a first category may comprise a plurality of sounds produced by the interaction of a plurality of pitches and a plurality of amplitudes, while a secondary category may comprise transposi¬ tion. An object formed from the interaction of these first an second categories would result in a plurality of sounds having amplitudes defined by the first category and altered pitches defined by the pitches of the first category as altered by the second category.

To change the nature of a sound, one simply changes a component category without changing any of the other component categories. Alternatively, one can add or delete further categories. Thus, an object is defined by the interaction of all of its categories and modification, deletion or addition o any component category generally results in modification of th object. Therefore, "the nature of an object and/or its interaction with other objects can be modified with ease by changing any of its component categories. Additionally, objects can be nested within each other to produce more comple objects.

In the presently preferred embodiment, graphical and/or musical categories and/or objects are selectively user manipu¬ lated by causing interaction with further categories and/or objects. Illustrative of such categories, which may be vie as manipulating functions, are shape, path, rotation, scale, distortion and time-speed, which generally relate to graphi and pitch, amplitude, volume, rhythm, transposition, and ti speed which generally relate to sound, or music.

Shape relates to the boundaries of any graphical repr sentation desired to be manipulated and is typically user defined by drawing the boundaries of the desired shape utilizing an input device such as a light pen/pad, mouse or similar device. Shape also relates to text such as alphanu meric characters and similar symbols which may be entered v keyboard. Manipulation essentially results from applying categories to a shape. Path is a category which relates to desired course of travel and is generally used in conjuncti with a shape to define motion of the shape. Interaction between a shape and a category comprising a path will resul an object wherein the shape is viewed in motion travelling along the path. For example, a display of an airplane flyi nay be produced by forming a first shape by drawing the plan using an input device, assigning this first shape to a firs category, then drawing the desired flight path using the inp device, assigning this path to a second category, and subse¬ quently interacting the two categories to produce the object an airplane in motion along the path. The interaction of categories resulting in the airplane flying may be stored in object memory. Additionally the two categories may individu ally be stored in category memory.

In addition to causing a shape to travel along a path, shape may be rotated any number of degrees, scaled in size o selectively distorted. Such shapes may be arbitrarily compl limited essentially by computing power and display device resolution. Furthermore, a user may select a time-speed category to override a default time-speed value which is generally associated with each path, rotation, scale, distor tion and the like. Sound is synthesized and manipulated in a manner similar to graphics. The presently preferred embodiment forms musical categories such as pitch, amplitude, volume, rhythm, transposi tion and time speed. A pitch as well as the interaction of on or more musical categories operating upon a pitch or pitches i classified as an object. Additionally, a pitch(es) is also classified as a category. Through control over the individual musical categories, a user has much control over the creation and modification of musical categories and/or objects and the like. For example, a plurality of pitches which comprise an object may be individually modified by a musical category such as amplitude, volume, transposition or rhythm, heard in any desired order or heard with any desired time base by applying user selected time-speed category to override a default time- speed value.

Furthermore, musical categories and/or objects can be interacted with graphical categories and/or objects, as disclosed in the parent applications. More specifically, a 0 user may create an object by interacting a graphical category with a musical category, thus, for example, assigning six musical pitches to six vertices on a hexagon and subsequently graphically distorting the hexagon, producing six musically altered pitches, or vice versa. 5 It is, therefore, a primary feature of this^'invention to provide a relatively simple and inexpensive system for control ling large amounts of data with few commands with^'a view towards real time and future conditions, even when correlating graphical and/or musical events that would otherwise be inde- _Q pendent.

It is a further feature of this invention to provide a dynamic system wherein graphical, musical or other events are represented by dynamic states.

It is another feature of this invention to provide a ₅ system wherein only limited types of data are required to define states of events and instructions for their activities. It is also a feature of this invention to provide a universal operating system wherein a wide variety of input d forms and input devices can be used. It is an additional feature of this invention to provi a system wherein a high level of data processing efficiency i achieved through proper data representation.

It is a still further feature of this invention to provide improved image control while significantly decreasing the amount of control data.

It is an additional feature of this invention to improv the processing and generation of audio signals.

It is another feature of this invention to provide a computerized system where data which is displayed need not be calculated or modified on a pixel by pixel and/or frame by frame basis to achieve creation or displacement of an image.

It is also another feature of this invention to provide computerized system where data which is heard need not be calculated or modified through complex mathematics to achieve creation or modification of a sound.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects, features and advantages of the invention will become more readily apparent with reference to the following description of the invention in which:

Fig. 1 is a vector representation of a rectangle comprising vectors V1-V4;

Fig. 2 is a vertice representation of a rectangle comprising four vertices;

Fig. 3 is a vertice representation of a rectangle undergoing rotation;

Fig. 4 is a vertice representation of a rectangle undergoing scaling;

Fig. 5 depicts pixel locations on a graphical screen through which a user drawn waveform passes; Fig. 6 graphically depicts the minimum number of vertic which adequately define the waveform of Fig. 5;

Fig. 7 graphically depicts the discrete digital points, or elements which are stored in a waveform table and which represent the sine wave of Fig. 5;

Fig. 8 depicts a category formed from transposition;

Fig. 9 graphically depicts an object produced by the interaction of categories of musical pitch and musical transposition;

Figs. 10a, 10b, 11a and lib represent the interactions the categories of musical amplitude and graphical scale;

Figs. 12a, 12b, 13a, 13b, 14a and 14b represent the interaction of the categories of musical transposition and graphical distortion;

Fig. 15 represents an object comprising amplitude; and

Fig. 16 depicts, in block diagram form, the apparatus o the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to graphical and musical manipulation of data as well as a combination thereof.

The present invention, as applied to graphics, relates the creation and manipulation of graphical data and specifi¬ cally comprises modifying an image, such as an image represen ted by a shape category, wherein such modification comprises altering the position of points which represent the shape so to produce motion. The present invention, as applied to music, relates to the creation and manipulation of sound data and specifically comprises modulating a first waveform (category) by interacti it with a second waveform (category) to produce a third waveform (object) which may be output as sound. GRAPHICS

The term "graphical category" as used herein with resp 5 to graphical representation and manipulation refers to any shape, path, rotation, scale, distortion, time-speed functio or the like, or any combination thereof.

The presently preferred embodiment recognizes categori and objects formed from the above categories which are 0 preferably interacted or combined with each other, thereby providing additional categories and/or objects. Such categories and/or objects are preferably selectively viewed a computer screen under user control in the form of animated pictoral representations. 5 More particularly, categories such as shape, path, rotation, scale, distortion and time-speed are user selectib to form desired objects to be viewed and/or stored in memory. For example, an object comprising a single shape may be form by a user drawing the desired shape onto a screen by using a 0 light pen. Additional input devices such as a tablet, trackball, joystick, touch panel, valuator and keyboard may also be used to form the desired shape. Objects are formed from any number of shapes and/or categories.

To provide for efficient storage and manipulation of shapes and the like, a vector scheme and/or a vertice scheme provided. In the vector scheme, a number of connected vecto magnitude and direction components corresponding to vectors drawn between strategic points on the shape(s) are stored as opposed to storing the location of each and every pixel throu _Q which the shape passes. Fig. 1 represents the illustrative case of a shape comprising a rectangular box wherein only fou vectors V1-V4, the head of any vector being attached to the tail of a subsequent vector, need be stored. In the case of more irregular shape, a number of additional vectors adequate to define the shape may have to be stored. In the vertice scheme, a number of strategic cartesian co-ordinates through which the shape(s) essentially passes ar chosen and stored to define the shape(s) . Fig. 2 represents the illustrative case of a shape comprising a rectangular box with a vertice at the origin, wherein four strategic points (illustratively, 0,0; 10,0; 10,5; 0,5) need be stored to defin the shape. As in the vector scheme, a number of additional cartesian co-ordinates adequate to define a more irregular shape may have to be stored. Such vector and vertice schemes are not limited to shape representation, but are also applicable to representation of any category. Furthermore, a scheme combining aspects of vertice as well as vector representation may advantageously be used for efficiently representing the shape and/or other categories of an object.

In addition to forming a category comprising a shape, a user may form a category which comprises a path. For example, a spiral may be drawn using an input device and designated a path as opposed to a shape. In such a case, a first category associated with the shape may be caused to interact with a second category associated with the path, thereby providing an object comprising the shape of the first category travelling along the path defined by the*second category. This specific interaction of categories comprising shape and path results in an object having motion.

The motion produced by such interaction between a shape and a path is accomplished with a minimal amount of computing power as follows. In the preferable case of vertice represent ation of shape and path, a shape is moved from a first positio to a displaced second position by the repetitive addition of a single set of cartesian points extracted from the vertices of the path, representing an offset value to the x-direction and y-direction distances between any point on the shape in the first position and the corresponding point on the shape in the second position. Such repetitive addition is not performed o each and every pixel through which the shape passes, but rat only on the minimum number of vertice points determined to adequately represent the shape. In the illustrative case of rectangle, the minimum number of vertice points is four. Of course, such repetitive addition may be performed on more th the minimum number of vertice points. Once such addition is performed, a line generation algorithm is applied to the vertice points of the shape in the second position. More specifically, an algorithm for connecting these vertice point is executed to visually join these vertice points, thus providing the original shape in a slightly displaced position By the similar repetitive addition of another single set of cartesian points extracted from the vertices of the path and the application of the line generation algorithm, the shape i moved from the second position to a third position and so on until the shape has moved through the entire path as defined the second category. In order to produce a line between two given points on raster type display having individual pixels as opposed to calculating the position of each and every pixel connecting t two given points, a line or arc generation algorithm such as that disclosed in Bresenham, J.E., "Algorithm For Computer Control of a Digital Plotter", IBM Systems Journal, Vol. 4, N 1, pp. 25-30 (1965), incorporated herein by reference, is utilized. Additional arc generation algorithms which may be used with the present invention to produce an arc between two given points are disclosed in Pitteway, M.L.V., "Algorithm fo Drawing Ellipses or Hyperbolae with a Digital Plotter", The Computer Journal, Vol. 10, No. 3, pp. 282-289 (November 1967) and Jordan, B.W., et al., "An Improved Algorithm for the Generation of Nonparametriσ Curves", IEEE Transactions on Computers, Vol. C-22, No. 12, pp. 1052-1060 (December 1973) also incorporated herein by reference. Rotation of a shape, unlike motion of a shape along a path, does not affect all of the vertice points of the shape with a constant offset value. In rotation, offset values to the vertice points of the shape are proportional to the distances of the vertice points from the origin point of the rotation. Each of these distances is representative of a radius from the origin point. As in the above-described motio of a shape along a path, numerous intermediate steps are necessary to display a smooth transition from a first position to a second position.

Referring to Fig. 3, rotation is performed on an object formed from a rectangular shape as follows. The rectangular object is initially defined by vertices 110, 115, 120 and 125, which represent cartesian co-ordinates (illustratively 0,0; 10,0; 5,10; 0,5) and is to be rotated about origin point 110 initially to a position defined by vertices 110, 115', 120' an 125', and then to a position defined by vertices 110, 115", 120" and 125". Each of the minimum number of vertices adequat to define the shape are individually rotated the same number o degrees in order to preserve the shape of the rectangular object. For example, vertice 115 is rotated towards vertice 115' through the same angle as vertice 120 is rotated towards vertice 120' and as vertice 125 is rotated towards vertice 125'. The radius from the origin point to a specific vertice before rotation is identical to the radius from the origin point to the corresponding vertice subsequent to rotation, in order to preserve shape. For example, the distance from original point 110 to vertice 115 (r.) is identical to the distance from origin point 110 to vertice 115' (r.'). Likewise, the distance from origin point 110 to vertice 120 (r_) is identical to the distance from origin point 110 to vertice 120' (r,'), as is the distance from origin point 110 t vertice 125 (r_) identical to the distance from origin point 110 to vertice 125' (r-'). Rotation of an individual point, whether relating to a shape or any category, is accomplished as follows. Rotation 5 from an original position to a final position generally requires numerous individual rotations, each such individual rotation being of a predetermined number of degrees. The number of degrees in such an individual rotation is highly dependent on the resolution of the viewing screen and the 0 associated hardware. High resolution screens may be utilize with individual rotations of one degree or less. Generally, individual rotations of approximately five to twenty five degrees may be used, with 11.25 degrees being preferable in presently preferred embodiment. Although each individual 5 rotation preferably is 11.25 degrees, the actual distance a point traverses during this individual rotation is proportio to its radius from an origin point about which the point rotates. To produce traversal of such a distance, a series iterations through the above-mentioned line or arc generatio 0 algorithms are necessary. As a single iteration produces an arc or portion thereof of a known length, (illustratively, a single pixel) an arc of a given length can be produced by performing a known number of iterations. Therefore, knowing the radius relating to a point which is to be rotated a know 5 number of degrees about an origin point, the number of iterations necessary to produce the resultant arc may be calculated or, preferably, read from a look-up table. Once such iterations are performed, the point is thereby rotated through the desired individual rotation. Such a scheme of _Q performing a number of arc generation algorithm iterations which depends on the radius of the point to be rotated is applied to each of the minimum number of vertices which adequately define the shape, thereby providing all such vertices in a slightly rotated position. By performing a 5 number of such individual rotations each of the same number of degrees, a smooth transition from an original position to a final rotated position is accomplished.

Upon rotation of the minimum number of vertices to adequately define a shape from a first position to a second position, a line generation algorithm similar to the line generation algorithm discussed in conjunction with the shape- path interaction is applied to the vertices in the second position, thereby connecting the vertices in such manner as to form the original rectangular shape, rotated a given number of degrees. Once the rectangular shape has been displayed in thi second position, an additional similar rotation is performed o vertices 115', 120' and 125' to produce vertices 115", 120" an 125". - The line generation algorithm is then applied to these twice displaced vertices, forming the original rectangular shape in a third further rotated position. Alternatively, the original vertices can be rotated through an increased angle to provide vertices in the third position.

By performing a number of such minute individual rotations and displaying the rotated shape after each rotation the illusion of smooth movement is seen by a viewer. By providing individual rotations of a smaller number of degrees, a smoother transition between positions can be accomplished, limited by the resolution of the display device.

As will be apparent to one skilled in the art, the above process involving rotation in the two dimensional x-y plane is equally applicable to other axes, thus providing for three dimensional rotation. For example, the illusion of three dimensional movement, i.e., motion along a z-axis, may be effected by scaling a shape in.the x-y plane. Furthermore, rotation about an origin point which is not located on the boundary shape to be rotated may also be accomplished. In addition to causing a category associated with a sha to interact with a category associated with a path and/or rotation, the category associated with the shape may be cause to interact with a category associated with a scale, thereby producing a shape which changes in size.

Referring to Fig. 4, scaling is performed on an object formed from a rectangular shape as follows. The rectangular object is initially defined by vertices 210, 215, 220 and 225 which represent cartesian co-ordinates (illustratively 0,0; 10,0; 10,5; 0,5) which are the minimum vertices necessary to define the shape. A scaling factor is applied to the line represented by the distance from origin point 210 to vertice 215, resulting in the enlarged line represented by the distan from origin point 210 to vertice 215'. The angle associated with any specific line, vector or the like before scaling is identical to the angle associated with the specific line, vector or the like after scaling. The same scaling factor as is applied to a line represented by the distance between vertices 210, 215 to produce a scaled line represented by the distance between vertices 210, 215' is similarly applied to a diagonal line represented by the distance between vertices 210, 220 to produce a scaled line represented by the distance between vertices 210, 220'. Scaling does not change the ratios of a shape, as is apparent from the equality between the angle formed by "lines" 210-215 and 210-220 (before scaling) and th angle formed by "lines" 210-215' and 210-220' (after scaling) In a similar manner, the identical scaling factor is applied "line" 210-225 to produce a scaled line 210-225'. Once the vertices 215', 220' and 225' have been calculated by the abov method, a line generation algorithm such as that used in conjunction with the shape-path interaction is applied to the four vertices 210, 215', 220' and 225' to produce an enlarged rectangle. Once this enlarged rectangle 210-215'-220'-225' is generated and displayed, the above procedure of applying a scaling factor to lines represented by distances between strategic vertices is again similarly applied, although now th scaling function is preferably applied to the enlarged rectangle 210-215'-220'-225' to produce a further enlarged rectangle 210-215"-220"-225". Alternatively, a scaling factor of larger magnitude may be directly applied to the original rectangle 210-215-220-225 to produce the further enlarged rectangle 210-215"-220"-225".

By performing such scaling a number of times and displaying the scaled shape after each scaling, the illusion o smooth movement is seen by a viewer. As in the case of interacting a category comprising a shape and/or categories comprising path and/or rotation, by providing individual scaling factors of magnitude approximately but not equal to one, a smoother transition between positions can be accom¬ plished, limited by the resolution of the display device and associated hardware.

As will be apparent to one skilled in the art, scaling can occur to produce a shape of decreased as well as increased size. Furthermore, the scaling can be looped to provide for a display wherein an object continuously increases in size and decreases in size. Such looping can be applied to any category, and is preferably applied to all categories.

In addition to forming an object which comprises a shape path, rotation and/or scale, a user may form an object which comprises a distortion. A category to which distortion is assigned may be caused to interact with a category comprising shape, resulting in a physical distortion of the shape. Advantageously, a user selects the magnitude of the distorting force as well as the position with respect to the shape to apply the force by the placement of an origin point from which the distorting force radiates. The length of a line drawn fro the origin point determines the magnitude of the distorting force. The magnitude of the distorting force preferably decreases over the distance it radiates, providing for a very 5 selective distortion of the shape. Through such user control of distortion, the leaves of a tree, for example, may be caus to sway, while the trunk of the tree remains essentially stationary.

More specifically, distortion involves use of a look-up 0 table comprising magnitude, direction and positional informa¬ tion of the distorting force desired to be applied to a shape Such a table is formed from the information contained within the distorting force input to the present invention by the user. 5 Information contained in this distortion look-up table utilized in conjunction with the line and/or arc generation algorithm to connect vertices. More specifically, a position value produced by the line generation algorithm and serving t partially join two vertices is added to a displacement value _Q contained within the distortion look up table. Such a displacement value is greatest in magnitude for points produc by the line generation algorithm which are closest in positio to the distorting force. The actual point on the resultant distorted line is calculated by summing the point produced by 5 the line generation algorithm with the displacement value corresponding to that point.

The dynamic categories of path, rotation, scale and distortion generally require the association of time-speed categories to provide a meaningful object display. More _Q specifically, a category comprising a shape may be caused to interact with a category comprising a path, rotation, scale, distortion, or similar dynamic category to provide an object display having motion. The speed of such motion is preferabl defined by a default time-speed value associated with each su ₅ dynamic category. Such association generally occurs at the time the object is formed from a dynamic category. In addition to the above-mentioned categories, a user selectible time-speed category exists in order to override a default time-speed value. Such a time-speed category may result in time expansion or compression during a period of the object display which would be viewed as the object display having an increased or decreased rate of movement. For example, a category comprising a shape of an automobile may be interacted with a category comprising a path wherein the default time-speed value associated with the path has been overridden by a user to a time-speed category having an initia time expansion and a subsequent time compression. The categor produced by such interaction would comprise an automobile 5 accelerating and decelerating along the given path.

As one skilled in the art will appreciate, an object may comprise any number of individual categories, each with a common time-speed value or individual time-speed categories. Similarly, an object may comprise any number of objects. _Q Objects, and therefore motion of such objects may be arbitrar¬ ily complex, limited essentially by display resolution and computing power.

The more complex a shape or category is, the greater the minimum number of vertices necessary to adequately represent ₅ that shape or category. In addition to objects formed by the user, a library of given shapes, objects and/or categories is preferably provided. Illustratively, a single such category may comprise a static shape of a house or it may comprise the dynamic combination of a house having smoke bellowing from a _Q chimney, trees swaying in the wind, a car rapidly approaching along a winding road from the distance and a windmill having rotating vanes.

5 MUSIC

As one skilled in the art will appreciate, the above teachings of graphics and the concepts related to graphical manipulation discussed therein are equally applicable to sou and sound manipulation. In the case of sound, the term "musical category" as used herein with respect to sound refe to any pitch, amplitude, volume, rhythm, transposition, time speed or the like, or any combination thereof.

The presently preferred embodiment recognizes objects formed from the above categories which are preferably manipu lated and interacted with each other, thereby providing additional categories and/or objects. Such categories are preferably selectively viewed in the form of graphical representations of sound and/or heard in the form of sound.

More particularly, categories such as pitch, amplitude, volume, rhythm, transposition, and time-speed are user selectible to form desired sounds to be heard, interacted wi graphical categories, and/or stored in memory. For example, user may synthesize a desired sound by forming a category comprising a single pitch or a plurality of pitches, and interacting that pitch category with other musical categorie Categories are generally formed by a user drawing a waveform a graphical input device and extracting discrete digital elements therefrom. For example, a user drawn single sine w may represent a single pitch having a specific frequency whic depends on a default time-speed value associated with the display, and more specifically, with the X-axis of the displa upon which the sine wave is drawn. Categories may also be resident in a category library in the form of waveforms or waveform tables. A waveform table contains discrete digital elements of a waveform; such a waveform is generally drawn by user. The waveform table may be viewed as containing the discrete digital elements produced by converting the analog input waveform into discrete digital elements. Preferably, pitch and amplitude define a sound.

More specifically, creation of a sound, i.e., formation of a category or object comprising pitch and amplitude, in accordance with the present invention is accomplished by digitizing a first waveform which is preferably input to the present invention by a user drawing the first waveform on a graphical input device and associating the digital waveform with an amplitude. Although numerous co-ordinate systems may be used for the representation of such waveforms, the present invention will described in terms of a cartesian co-ordinate system having a horizontal X-axis. Such digitizing of the waveform results in a plurality of discrete digital samples, elements, which represent the waveform. Illustratively, the discrete digital points are the Y-values of the waveform. Separate X-values generally need not be stored in the wavefor table since the Digital Signal Processor (DSP) which eventual produces an audible sound from the waveform table recognizes that each successive Y-value from the waveform table is associated with a successive X-value of known relative magnitude. However, the discrete digital elements may each contain an X-value and a Y-value. Either an arbitrary defaul amplitude may be assigned to the pitch, i.e., to each of the discrete digital samples, or the pitch may be interacted with category comprising amplitude.

The process by which the discrete digital points which are to be stored in a waveform table and which represent a waveform drawn by a user are chosen is as follows. A user initially inputs the waveform by. drawing it on a graphical input device, such as a CRT. This method produces a pluralit of pixel locations which are continuous and clearly define th waveform, as depicted in Fig. 5 in which the X-axis represent time and the Y-axis represents amplitude. However, it is not necessary to store in the waveform table the location of eac and every pixel through which the user drawn waveform passes order to subsequently recreate the analog waveform from its discrete digital elements stored in the waveform table. Advantageously, an algorithm is utilized to select the minim number of vertices or points on the user drawn waveform whic adequately define the waveform and store such points in memo In the case of a straight line, only two points are necessar to define the straight line, while in the case of a sine wav a greater number of points are necessary. Such algorithms a known in the art and will be apparent to those skilled in th art. In the illustrative case of the sine wave depicted in Fig. .5, application of the algorithm to select the minimum number of points to adequately define the waveform results i the points depicted in Fig. 6. As is apparent from Fig. 6, more points are generally required in areas of sharp curves than in areas of straight lines. A line/arc generation algorithm of the type previously identified under the headin "Graphics", is applied to these minimum number of vertices determined to adequately define the waveform, in order to produce the discrete digital elements which are input to the waveform table for subsequent input into the DSP and eventua output of audible sound. Such elements are graphically depicted in Fig. 7. The waveform table relating to Fig. 7 would thus contain the Y-values of the discrete points in Fig 7. The number of such elements (Fig. 7) is generally less th the number of pixels through which the user drawn waveform passes (Fig. 5) and greater than the minimum number of points which adequately define the waveform (Fig. 6) . Illustrative the line/arc generation algorithm required to produce the points depicted in Fig. 7 and resident in the waveform table from the points depicted in Fig. 6 is a Bresenham type arc generation algorithm. The waveform table thus contains identifying information which determines the sound output. More specifically, the waveform table contains discrete digital elements, each of which may be viewed as having a specific magnitude. A DSP converts these discrete digital elements into a signal capable of directly driving an audible output device. For an input of a given waveform table, different DSPs from different manufac- tures generally produce approximately the same sound output.

In addition to the creation of a sound through the above described process wherein a category comprising a single pitch was formed, a sound previously created may be modified in accordance with the present invention. Such modification is accomplished through the interaction of musical categories. These musical categories are each represented by information stored in a waveform table. Alternatively, such a musical category may be represented by waveforms or information stored in memory, wherein such information is of the same type as the information stored in the waveform table. Such interaction generally comprises the modulation of information associated with a first waveform by information associated with a second waveform.

More specifically, such.modulation comprises adding, subtracting, multiplying and/or dividing discrete digital points or elements representing the first waveform by elements representing the second waveform, thus resulting in elements representing a third waveform which are input to a waveform table which is input to the DSP for subsequent production of sound.

A wide variety of types of sound modification results from adding, subtracting, multiplying and/or dividing X-values of the first waveform with X-vaϊues of the second waveform, X- values of the first waveform with Y-values of the second waveform, Y-values of the first waveform with Y-values of the second waveform, and Y-values of the first waveform with X- values of the second waveform. The shape of the two wavefor affects the shape of the third waveform, thus producing user 5 controlled variations in sound modification.

A category formed from elements of a first waveform whi is interacted, i.e., modulated, with a category formed from elements of a second waveform produces a plurality of element which is input to a waveform table. This waveform table is 0 then input to a DSP which outputs a signal capable of driving speaker, thus producing an object of sound. Through addition and/or multiplication of X-values and/or Y-values of two categories formed from elements of waveforms, modification in pitch, amplitude, volume, rhythm, transposition and time-spee 5 are achieved.

Pitch essentially relates to the average number of vibrations of a given pattern in a given unit of time. Therefore, a sine wave of essentially a single frequency draw by a user relates to a single pitch. However, the above 0 teachings relating to the formation of a category comprising single pitch may be logically extended to the formation of a category comprising a plurality of pitches, in which case a user would draw a more complex waveform. Furthermore, an audible sound may be reduced to discrete digital elements, th 5 digital elements being placed in a waveform table and subsequently modified or output.

In addition to pitch, a category may be formed from a desired amplitude. While a user drawn waveform relating to pitch is associated with a time (X-axis) versus magnitude (Y- _Q axis) variation scheme, a user drawn waveform relating to amplitude is associated with a single value (Y-axis) over tim (X-axis) . As such, amplitude relates to the relative loudnes and/or softness of pitch. For example, a first category may formed of a single amplitude and caused to interact with, for 5 example, a second category comprising a single pitch, in orde to produce a sound having amplitude defined by the first category and pitch defined by the second category. In this specific case, the first category of amplitude comprises a single value over a specified time, such time generally being the same as the duration of the pitch.

More specifically, a category comprising a single pitch is interacted with a category comprising an amplitude by addi the single Y-value of the user drawn amplitude waveform to ea discrete digital element (Y-value) representing the user draw pitch. The resultant discrete digital elements are then inpu to a waveform table for subsequent output as sound via the DS Alternatively, the discrete digital elements (Y-values) of th pitch category may be multiplied by the single Y-value of the amplitude category. Similarly, a first category of amplitude may comprise a first value over a first time period and a second value over a subsequent second time period. Interacti this first category with a second category comprising two pitches having duration corresponding to the duration of the first and second amplitude values products a sound output of first frequency having a first amplitude and then a second frequency having a second amplitude. An object may comprise solely.a pitch, or a pitch having given amplitude or any combination of musical categories. An object comprising sol a pitch must generally interact with, at. minimum, an amplitu in order to become audible.

While amplitude is generally applied to manipulate a single pitch or a series of pitches, volume is used to manipulate the overall loudness or softness of a musical or sound composition, having an effect similar to that produced rotation of a volume control device on a typical audio syste such as a radio. In the present embodiment, volume relates t the overall loudness and/or softness of any combination of amplitude and pitch categories. As a result, a category form from a volume may be interacted with a category formed from a plurality of pitches and amplitudes, in much the same manner a category formed from a single pitch is interacted with a 5 category formed from a single amplitude.

Rhythm is a category which relates to the duration of a pitch. Advantageously, a user may thus selectively shorten o lengthen the duration of a single pitch. Since the present invention provides looping, a single pitch and amplitude

10 combination may be constantly repeated, thus providing a soun of steady frequency and amplitude and having indefinite duration. Alternatively, a series of pitch and amplitude combinations may be repeated. Silence may be represented by pitch of zero kHz and zero amplitude. By interacting a

15 category or object comprising a pitch and amplitude with a category formed from rhythm, a series of sounds is produced, with each sound having the same frequency but a duration defined by the rhythm category. Of course, the duration of t silent periods between successive sounds is similarly defined

20 by the rhythm category. For example, to produce a series of three sounds having the following characteristics:

- Pitch 10kHz, amplitude 5 units, duration 3 seconds; 6 seconds silence;

- Pitch 10kHz, amplitude 5 units, duration 25 12 seconds; 3 seconds silence;

- Pitch 10kHz, amplitude 5 units, duration 9 seconds, one would interact a first category or object comprising discrete digital elements relating to a sound having a pitch

3 10kHz and amplitude 5 units and a pitch of zero kHz and zero amplitude with a second category illustratively having discre digital elements relating to a magnitude of 1 yielding 3 seconds of sound, magnitude 2 yielding 6 seconds of silence, magnitude 4 yielding 12 seconds of sound, magnitude 1 yieldin

35 3 seconds of silence, and magnitude 3 yielding 9 seconds of sound. Of course, the 10kHz, 5 unit amplitude and zero kHz, zero amplitude sound associated with the first category is looped in order to provide the above output totalling 33 seconds. Within this scheme, each magnitude of rhythm corresponds to 3 seconds and equal time-bases are applied to the categories.

In general, computing machines assign a numerical value to represent an audible pitch. Transpositions in pitch are conventionally determined by a high order mathematical recalculation of that numerical value on the audio equivalent of a pixel by pixel basis.

Transposition relates to the addition of an offset facto or factors to a pitch or series of pitches. In the presently preferred embodiment, pitch is ordered in a numerical scale proportioned to the magnitude of the vibrational rate of sound For example, a pitch of 10kHz may be stored and manipulated in terms of the pitch value "10" while a pitch of 14 kHz may be stored and manipulated in terms of the pitch value "14". Transposition of a pitch having a pitch value results in a pitch having a new pitch value, i.e., a pitch with a different number of vibrations per unit of time.

The preferred embodiment of the present invention adds o subtracts a single offset value to/from the initial numerical pitch value to provide a new numerical pitch value. This new numerical pitch value is utilized to extract a new pitch from look-up table, thus eliminating the need for significant high order mathematical calculations.

Transposition also relates to essentially continuous changes and variations in pitch. For example, a user may interact a first category formed from a single pitch (such as that in Fig. 7) having a specific single frequency and amplitude with a second category formed from a user drawn waveform such as that depicted in Fig. 8 in which the X-axis represents time and the Y-axis represents frequency. The result of such an interaction is depicted graphically in Fig. 9. More specifically, such an interaction comprises redu discrete digital elements (X-values) of the pitch of Fig. 7 an amount derived from the discrete digital elements (Y-val of the transposition of Fig. 8. Such reducing may take the form of subtraction or division, resulting in the sound represented graphically in Fig. 9. This sound has a constan amplitude but continuously increases in frequency. X-values and Y-values which are added, subtracted, multiplied or. divi are calculated by the line/arc generation algorithm and may stored in memory.

The presently preferred embodiment of the invention is capable of interacting musical categories and/or objects wit graphical categories and/or objects.

More specifically. Figs. 10a and 10b relate to the interaction of categories of musical amplitude and graphical scale. Fig. 10a represents a musical category comprising si musical pitches 1-6 having an increased amplitude in standar musical notation. Such a category of six musical pitches ma be formed by a user choosing six pitches from a library of pitches, as opposed to drawing six waveforms. Fig. 10b represents a corresponding graphical category wherein the si musical pitches of Fig. 10a having increased amplitude are assigned to six vertices 1-6 of a hexagon.

Similarly, Figs. 11a and lib relate to the interaction categories of musical amplitude and graphical scale. More specifically. Fig. 11a represents a musical category compris the identical six musical pitches 1-6 of Fig. 10a but having decreased amplitude in standard musical notation. Fig. lib represents a corresponding graphical category wherein the si musical pitches of Fig. 11a having decreased musical amplitu are assigned to six vertices 1-6 of a hexagon.

In the practice of the invention, a decrease in the musical amplitude of the pitches by a user, in the interacti of a category or object comprising musical amplitude and a category or object comprising graphical scale, generally results in a downward scaling of the shape to which the six pitches are assigned. This decrease in musical amplitude is seen in the transition from Fig. 10a (louder) to Fig. 11a (softer) , while the corresponding decrease in the size (scaling) of the hexagon shape is seen in the corresponding transition from Fig. 10b to Fig. lib. The above discussion relates to a user directly changing musical amplitude thereby producing a corresponding scaling change in a graphical shape. However, the user may alternativ ely directly downward scale the graphical shape to which musical pitches have been assigned to produce a corresponding decrease in musical amplitude.

Figs. 12a and 12b relate to the interaction of categorie of musical transposition and graphical distortion. More specifically, Fig. 12a represents a musical category comprisi six musical pitches 1-6 having specific amplitudes in standar musical notation. Fig. 12b represents a corresponding graphi cal category wherein the six musical pitches of Fig. 12a of specific amplitudes are assigned to six vertices 1-6 of a hexagon.

Figs. 13a and 13b relate to the interaction of categori of musical transposition and graphical distortion. However, Fig. 13a depicts the six pitches 1-6 of Fig. 12a subsequent t a user induced partial upward transposition in which pitches 3-6 are each transposed upward in pitch. In addition to a resulting audio difference, a graphical distortion is applied to the hexagon 1-6 to produce the distorted hexagon 1-2-3'-4' 5'-6'. Such a graphical distortion is individually applied t produce only the transposed vertices 3'-6' and generally not the non-transposed vertices 1, 2. Referring to Figs. 12a and 13a, an offset value of thre is added to pitches 3-6 to provide pitches 3'-6'. The same offset value of three is added as graphical distortion, as depicted in Figs. 12b and 13b.

Similarly, Figs. 14a and 14b relate to the interaction categories of musical transpositions and graphical distortion Fig. 14a depicts the six pitches 1-6 of Fig. 12a subsequent t a user induced partial downward transposition in which pitche 3-6 are each transposed downward in pitch. In addition to a resulting audio difference, a graphical distortion is applied to the hexagon 1-6 to produce the distorted hexagon 1-2-3"-4" 5"-6". Such a graphical distortion is individually applied t produce only the transposed vertices 3"-6" and generally not the non-transposed vertices 1,2.

The above discussion relates to a user directly trans¬ posing musical pitches and thereby producing a corresponding distortion in a graphical shape. However, the user may alternatively directly distort a graphical shape to which musical pitches have been assigned and thereby produce a musical transposition of pitches.

A user is preferably supplied with a given library of various objects and categories such that he can efficiently synthesize and manipulate a wide variety of sounds.

As in the above discussion under the heading "Graphics" a user selectible time-speed category as well as a default time-speed value exists. Categories may be further manipulat by interacting them with time-speed categories. In such a case, the time base applied to, for example, a series of pitches and amplitudes, is altered so as to produce a sound with increased tempo, i.e., increased rate of execution.

The presently preferred embodiment of the invention provides looping wherein any object or category may be continually and/or selectively repeated, whether relating to graphics and/or sound.. Furthermore, a user may modify object or categories in real time as looping occurs. Illustrative o such looping is the situation where an object is produced by 5 assigning a plurality of pitches to a formed image such as a circle and subsequently causing that object to interact with category comprising an amplitude. Such amplitude illustrativ ely may comprise a first portion 300 having a constant ampli¬ tude, a second portion 310 having a linearly increasing 0 amplitude and a third portion 320 having an exponentially decreasing amplitude, as depicted in Fig. 15.

Fig. 16 depicts, in block diagram form, an apparatus of the present invention comprising, a random access memory (RAM 400, a random access memory (RAM) 410, a read only memory (RO

15 420, an interactor 430, a translator 440, a translator 450 an an object/category organizer 460.

More specifically, RAM 400 is provided with information relating to categories of the present invention. RAM 410 is similarly provided with information relating to objects of th

20 present invention. ROM 420 is provided with information relating to the rules of interaction of the present invention such as rules and algorithms (i.e., line or arc generation algorithms) associated with the categories as well as look up tables and the like.

25 Interactor 430 is a processing unit preferably provided with arithmetic logic unit(s) , general registers, bus handler timing circuitry and associated control circuitry.

Translator 440 serves to review objects associated with RAM 410. Translator 440 selectively provides a video output

30 relating to objects having characteristics suitable for visua representation.

Translator 450 similarly serves to review objects associated with RAM 410. Translator 450 selectively provides an audio output relating to objects having characteristics

35 suitable for audio representation. Object/category organizer 460 serves to organize information contained within RAMS 400, 410 so as to provide translators 440, 450 with easily identifiable data.

In the practice of the apparatus of Fig. 11, ROM 420 provides interactor 430 with rules of interaction in order t produce an output of objects to RAM 410 from an input of categories from RAM 400. Object/category organizer 460 organizes the object information contained within RAMS 400, so as to provide translators 440,450 and interactor 430 with easily identifiable data. Translators 440, 450 then review organized objects associated with RAM 410 and provide a vide output and/or an audio output, respectively, depending on th existence of video and audio characteristics of the organize objects.

A particular application of the present invention is i the transmission of information from computing machine to computing machine, i.e., typical modem communication. The application of the object and category schemes of the presen invention significantly reduces the amount of transmitted da For example, it is only necessary to transmit category information to change the nature of a remote object. Such a remote communication scheme may be summarized as a communica tions protocol.

As will be apparent to one skilled in the art, the abo discussions relating to graphics and sound, though presented separately for purposes of clarity, are interelated and may viewed and heard simultaneously. An object may be formed fr the graphic categories of shape, path, rotation, scale and distortion as well as from the sound categories of pitch, amplitude, volume, rhythm and transposition, in addition to numerous time-speed categories. Furthermore, an object or category relating to sound may be interacted with a category relating to graphics or other objects or categories, and vic versa. Thus it can be seen that there is provided a method whereby graphical and sound data is manipulated in real time without the need for large amounts of computing power. In addition, it is to be understood that the invention can be carried out by specifically different equipment and devices an that various modifications, both as to equipment details and operating procedures can be effected without departing from th spirit and scope of the claimed invention. More specifically, the present invention as claimed can be applied with respect t other co-ordinate systems such as a polar representation. Additionally, further categories and objects may be provided. Although the present invention was described as it relates to graphics and also distinctly as it relates to music for the sake of clarity, graphical categories may be interacte with musical categories. Furthermore, teachings under the heading "Graphics" including but not limited to use of vector and vertice representation and line/arc generation algorithms are equally applicable to teachings under the heading "Music" and vice versa.

Moreover, the X-axis and Y-axis relating to any waveform or shape may represent time, frequency, amplitude or any other variable or parameter.

Claims

1. A method of selectively manipulating data within a computing machine, the method comprising the steps of: forming a first data category comprising a first 0 plurality of discrete digital elements, said first plurality discrete digital elements representing a plurality of points a first waveform; forming a second data category comprising a second plurality of discrete digital elements, said second plurality 5 of discrete digital elements representing a plurality of poin on a second waveform; combining individual corresponding elements of sai first plurality of discrete digital elements with said second plurality of discrete digital elements, providing a third 0 plurality of discrete digital elements; and inputting said third plurality of discrete digital elements to a waveform table.

2. The method of claim 1 further comprising the step providing said third plurality of discrete digital elements o 5 said waveform table to a Digital Signal Processor (DSP) , said DSP driving an audible output device to produce a data object of sound.

3. The method of claim 1 wherein said combining comprises adding, subtracting, multiplying or dividing. _Q

4. The method of claim 1 wherein said forming compris the steps of: drawing a waveform on a graphical input device; calculating a minimum number of points which adequately define said waveform and through which said wavefo 5 essentially passes; and applying a line/arc generation algorithm to said minimum number of points which adequately define said waveform, thus producing said plurality of discrete digital elements.

5. The method of claim 4 wherein said line/arc generation algorithm is a Bresenham type arc generation algorithm.

6. The method of claim 1 wherein said forming comprises the step of retrieving a set of a plurality of discrete, digita elements from a library of such sets.

7. The method of claim 1 wherein said first plurality of discrete digital elements are associated with an X-value of said first waveform and said second plurality of discrete digital elements are associated with an X-value of said second waveform.

8. The method of claim 1 wherein said first plurality of discrete digital elements are associated with an X-value of said first waveform and said second plurality of discrete digital elements are associated with a Y-value of said second waveform.

9. The method of claim 1 wherein said first plurality of discrete digital elements are associated with a Y-value' of said first waveform and said second plurality of discrete digital elements are associated with a Y-value of said second waveform.

10. The method of claim 1 wherein said first plurality of discrete digital elements are associated with a Y-value of said first waveform and said second plurality of discrete _Q digital elements are associated with an X-value of said second waveform.

11. The method of claim 1 wherein said first data category is pitch and said first plurality of discrete digital elements represents a change in magnitude of said first 5 waveform over time, said first waveform having a specific pitch.

12. The method of claim 11 wherein said second data category is amplitude, and said second plurality of discrete digital elements comprises a specific amplitude over time.

13. The method of claim 12 wherein said combining comprises adding.

14. The method of claim 13 further comprising the ste of providing said third plurality of discrete digital elemen of said waveform table to a Digital Signal Processor (DSP) .

15. The method of claim 14 wherein said DSP drives an audible output device to produce a data object of sound comprising said specific pitch and said specific amplitude.

16. The method of claim 1 wherein said first data category comprises a number of specific pitches each having associated therewith a specific amplitude, and said first plurality of discrete digital elements represents a change i magnitude of said first waveform over time.

17. The method of claim 16 wherein said second data category comprises volume and said second plurality of discre digital elements represent a desired volume of said pitches i said first data category.

18. The method of claim 17 wherein said combining comprises adding.

19. The method of claim 18 further comprising the step of providing said third plurality of discrete digital element of said waveform table to a Digital Signal Processor (DSP) .

20. The method of claim 19 wherein said DSP drives an audible output device to produce a data object of sound comprising said number of specific pitches and specific amplitude at said desired volume.

21. The method of claim 16 wherein said second data category comprises rhythm, and said second plurality of discrete digital elements represent a desired rhythm of said pitches in said first data category.

22. The method of claim 21 further comprising the step of providing said third plurality of discrete digital elements

5 of said waveform table to a Digital Signal Processor (DSP) .

23. The method of claim 22 wherein said DSP drives an audible output device to produce a data object of sound having said desired rhythm and comprising a subset of said number of specific pitches each having associated therewith said specifi 0 amplitude.

24. The method of claim 16 wherein said second data category comprises transposition and said second plurality of discrete digital elements represent a desired change in frequency of said pitches in said first category. 5

25. The method of claim 24 wherein said combining comprises dividing.

26. The method of claim 24 further comprising the step of providing said third plurality of discrete digital element of said waveform table to a Digital Signal Processor (DSP) . 0

27. The method of claim 26 wherein said DSP drives an audible output device to produce a data object of sound comprising said number of specific pitches and specific amplitudes characterized in that each such pitch has associat therewith a new desired frequency. 5

28. The method of claim 1 wherein said second data category comprises time-speed and said plurality of discrete digital elements represent a desired time base.

29. The method of claim 1 wherein looping is provided, said looping resulting in a repetitive and dynamic data objec _Q

30. A method of manipulating sound by modulating a fir digitized waveform with a second digitized waveform wherein a first plurality of discrete digital elements associated with said first waveform is individually combined with a second plurality of discrete digital elements associated with said g- second waveform, producing a third plurality of discrete digital elements which defines sound.

31. The method of claim 30 wherein said first pluralit of discrete digital elements are associated with an X-value o

5 said first waveform and said second plurality of discrete digital elements are associated with an X-value of said secon waveform.

32. The method of claim 30 wherein said first pluralit of discrete digital elements are associated with an X-value o

10 said first waveform and said second plurality of discrete digital elements are associated with a Y-value of said second waveform.

33. The method of claim 30 wherein said first pluralit of discrete digital elements are associated with a Y-value of

15 said first waveform and said second plurality of discrete digital elements are associated with a Y-value of said second waveform.

34. The method of claim 30 wherein said first pluralit of discrete digital elements are associated with a Y-value of

₌₂o said first waveform and said second plurality of discrete digital elements are associated with an X-value of said secon waveform.

35. The method of claim 30 wherein said third pluralit of discrete digital elements is input to a Digital Signal

25 Processor (DSP) , said DSP outputting a signal capable of driving an audible output device.

36. The method of claim 30 wherein looping is provided, said looping resulting in a repetitive and dynamic data objec

37. A method of selectively manipulating data within a 3_Q computing machine, the method comprising the steps of: forming a first data category and a second data category, each comprising a plurality of discrete digital elements representing one or more of graphical shape, graphica path, graphical rotation, graphical scale, graphical

35 distortion, graphical time-speed, musical pitch, musical amplitude, musical volume, musical rhythm, musical transposition and musical time-speed; and interacting said first data category with said second data category to produce a data object having characteristics of both said first data category and said second data category.

38. The method of claim 37 wherein said interacting comprises combining said plurality of discrete digital element of said first data category with said plurality of discrete digital elements of said second data category, providing a third plurality of discrete digital elements.

39. The method of claim 38 further comprising the step of inputting said third plurality of discrete digital elements to a waveform table.

40. The method of claim 39 further comprising the step of inputting said third plurality of discrete digital elements of said waveform table to a Digital Signal Processor (DSP) .

41. The method of claim 37 wherein said forming comprises the step of retrieving a set of a plurality of discrete digital elements from a library of such sets.

42. The method of claim 37 wherein said forming comprises the steps of: drawing a waveform on a graphical input device; calculating a minimum number of points which adequately define said waveform and through which said wavefor essentially passes; and applying a line/arc generation algorithm to said minimum number of points which adequately define said waveform thus producing said plurality of discrete digital elements.

43. The method of claim 42 wherein said plurality of discrete digital elements of said first data category are associated with an X-value of a first waveform and said plurality of discrete digital elements of said second data category are associated with an X-value of a second waveform.

44. The method of claim 42 wherein said plurality of discrete digital elements of said first data category are associated with an X-value of a first waveform and said plurality of discrete digital elements of said second data category are associated with a Y-value of a second waveform.

45. The method of claim 42 wherein said plurality of discrete digital elements of said first data category are associated with a Y-value of a first waveform and said plurality of discrete digital elements of said second data category are associated with a Y-value of a second waveform.

46. The method of claim 42 wherein said plurality of discrete digital elements of said first data category are associated with a Y-value of a first waveform and said plurality of discrete digital elements of said second data category are associated with an X-value of a second waveform.

47. The method of claim 37 wherein looping is provided said looping resulting in a repetitive and dynamic data objec

48. A method of selectively manipulating data within a computing machine, the method comprising the steps of: forming a first data category and a second data category, each having data representing one or more of graphical shape, graphical path, graphical rotation-, graphic scale, graphical distortion, graphical time-speed, musical pitch, musical amplitude, musical volume, musical rhythm, musical transposition and musical time-speed, or another graphical or musical category; and interacting said first data category^"with said seco data category to produce a data object having characteristic of both said first data category and said second data category, said data object in a form capable of being output to an output device.

49. The method of claim 48 in which said interacting said first data category with said second data category produces said data object wherein said data object is viewed as a graphical representation in motion on a visual output device.

50. The method of claim 49 wherein a rate of motion of said data object is controlled by said graphical time-speed category.

51. The method of claim 49 wherein a line/arc generati algorithm is used to produce motion.

52. The method of claim 48 in which said interacting said first data category with said second data category produces said data object wherein said data object is heard a sound on an audible output device.

53. The method of claim 52 wherein a tempo of sound of said data object is controlled by said musical time-speed category.

54. The method of claim 48 in which said interacting said first data category with said second data category produces said data object wherein said data object is viewed as a graphical representation of sound.

55. The method of claim 48 wherein said first data category is said graphical shape and said second data catego is said graphical path.

56. The method of claim 55 wherein said interacting comprises adding an identical offset value to each of a minimum number of points determined to adequately represent said graphical shape in a first position to define said minimum number of points in a second displaced position.

57. The method of claim 56 wherein said offset value comprises an x-direction distance and a y-direction distance said distances representing a positional difference of a poi of said graphical shape in said first position and a corresponding point of said graphical shape in said second displaced position.

58. The method of claim 56 wherein said interacting further comprises applying a line/arc generation algorithm t said minimum number of points of said graphical shape in sai second displaced position to visually join said minimum numb of points to define said graphical shape in said second displaced position.

59. The method of claim 56 wherein said interacting further comprises repetitively performing additional additio of said identical offset value and applying a line/arc generation algorithm.

60. The method of claim 58 wherein said line/arc generation algorithm is a Bresenham line generation algorith

61. The method of claim 48 wherein said first data category is said graphical shape and said second data catego is said graphical rotation.

62. The method of claim 61 wherein said interacting comprises adding an offset value to each of a minimum number of points determined to adequately represent said graphical shape in a first position to define said minimum number of points in a second rotated position.

63. The method of claim 62 wherein each of said minimum number of points is rotated through an identical number of degrees about an origin point to preserve said graphical shape.

64. The method of claim 63 wherein said number of degrees is within a range of approximately one to approximately twenty-five.

65. The method of claim 63 wherein said number of degrees is approximately eleven and one-quarter.

66. The method of claim 62 wherein said offset value corresponding to a specific point in said first position is proportional to a distance between said specific point and an origin point about which said minimum number of points is to be rotated.

67. The method of claim 66 wherein rotation of said specific point in said first position to said second rotated position is accomplished by performing a certain number of iterations through a line/arc generation algorithm.

68. The method of claim 67 wherein said certain number of iterations is proportional to said distance between said specific point and said origin point.

69. The method of claim 68 wherein said certain number of iterations is read from a look-up table.

70. The method of claim 67 wherein a single iteration through said line/arc generation algorithm defines rotation through a known distance.

71. The method of claim 70 wherein said distance is a length of a single pixel of a visual output device.

72. The method of claim 62 wherein said interacting further comprises applying a line/arc generation algorithm to said minimum number of points of said graphical shape in said second rotated position to visually join said minimum numbe of points to define said graphical shape in said second 5 rotated position.

73. The method of claim 62 wherein said interacting further comprises repetitively performing additional additi of said offset value and applying a line/arc generation algorithm. 0

74. The method of claim 48 wherein said first data category is said graphical shape and said second data categ is said graphical scale.

75. The method of claim 74 wherein said interacting comprises applying a scaling factor to each line from an 5 origin point to each of a minimum number of points determin to adequately represent said graphical shape in a first position to define said minimum number of points in a secon scaled position.

76. The method of claim 75 wherein an angle associate o with each of said lines prior to applying said scaling fact is identical to an angle associated with each of said lines subsequent to applying said scaling factor.

77. The method of claim 75 wherein said interacting further comprises applying a line/arc generation algorithm 5 said minimum number of points of said graphical shape in sa second scaled position to visually join said minimum number points to define said graphical shape in said second displa position.

78. The method of claim 75 wherein said interacting 0 further comprises repetitively performing additional applications of said scaling factor and applying a line/arc generation algorithm.

79. The method of claim 77 wherein said line/arc generation algorithm is a Bresenham line generation algorit 5

80. The method of claim 48 wherein said first data category is said graphical shape and said second data catego is said graphical distortion.

81. The method of claim 80 wherein said interacting comprises applying a distorting force to said graphical shape to produce a distorted shape.

82. The method of claim 81 wherein said interacting comprises inputting a distorting force to said computing machine, said distorting force having a direction relative t said graphical shape and a magnitude.

83. The method of claim 82 wherein said .distorting for is represented by a line drawn by a user on an input device and said magnitude is proportional to a length of said line and said direction relates to a direction of said line.

84. The method of claim 83 wherein a look-up table is formed by said computing machine, said look-up table containing said magnitude of said distorting force, said direction of said distorting force and positional informatio of said distorting force relative to said graphical shape.

85. The method of claim 84 wherein a line/arc generati algorithm is utilized to join a minimum number of points determined to adequately represent said graphical shape, thu defining additional points.

86. The method of claim 85 wherein at least some of sa additional points are displaced in position by the addition a displacement value from said look-up table.

87. The method of claim 86 wherein said displacement value is greatest in magnitude for said additional points which are closest in position to said distorting force.

88. The method of claim 48 wherein said first data category is said musical pitch and said second data category is said musical amplitude.

89. The method of claim 88 wherein said interacting comprises providing a specific musical pitch with a specific musical amplitude, thereby defining a data object of sound.

90. The method of claim 89 wherein said data object of sound is interacted with said musical rhythm to define a duration of said data object of sound.

91. The method of claim 89 wherein said data object of sound is interacted with said musical transposition to defin a data object of sound having an altered pitch.

92. The method of claim 91 comprising the steps of: adding a single pitch offset value to a pitch value associated with said specific musical pitch to provide a new pitch value; and extracting a new musical pitch from a look-up table

93. The method of claim 89 wherein said data object of sound is output on an audible output device.

94. The method of claim 88 wherein said interacting comprises providing a plurality of musical pitches with a corresponding plurality of musical amplitudes, thereby defining a data object of a corresponding plurality of sound

95. The method of claim 94 wherein said data object of plurality of sounds is interacted with said musical volume t define a new plurality of sounds each haying an equally altered amplitude.

96. A method for operating a computing machine in whic a plurality of data categories are interacted to produce a data object, said data categories being selected from a grou of graphical shape, graphical path, graphical rotation, graphical scale, graphical distortion, graphical time-speed, musical pitch, musical amplitude, musical volume, musical rhythm, musical transposition and musical time-speed.

97. The method of claim 96 wherein said data object comprises a graphical representation in motion.

98. The method of claim 96 wherein said data object comprises a sound.

99. The method of claim 96 wherein said data object comprises a graphical representation of a sound.

100. The method of claim 96 wherein looping is provided, said looping resulting in a repetitive and dynamic data object.

101. An operating system for operating a computing machine wherein a first data category is interacted with an additional data category to produce a data object.

102. The operating system of claim 101 wherein said first data category is a graphical shape comprising a graphical representation of data.

103. The operating system of claim 102 wherein said additional data category is at least one selected from a grou of graphical path, graphical rotation, graphical scale, graphical distortion, graphical time-speed, musical pitch, musical amplitude, musical volume, musical rhythm, musical transposition and musical time-speed.

104. The operating system of claim 102 wherein said graphical shape is drawn by a user onto an input device operatively associated with said computing machine.

105. The operating system of claim 101 wherein said first data category comprises a musical pitch.

106. The operating system of claim 105 wherein said first data category further comprises an amplitude.

107. A computerized arrangement for manipulating data comprising: first memory means for storing category information; second memory means for storing object information; third memory means for storing control information; interactingacting means having an input from said first memory means and an input from said third memory means and an output to said second memory means wherein obj information is produced by interaction of category information; first translator means coupled to said secon memory means for outputting video data; and second translator means coupled to said seco memory means for outputting audio data.

108. A method of selectively manipulating data within computing machine, the method comprising the steps of: forming a first data category and a second data category, each having data representing one or more of graphical shape, graphical path, graphical rotation, graphi scale and graphical distortion; and interacting said first data category with said sec data category to produce a data object having characteristi of both said first data category and said second data category, wherein said interacting comprises applying a line/arc generation algorithm to at least some of said data said first data category and said second data category in order to produce motion of said data object.

109. The method of claim 108 wherein said line/arc generation algorithm is a Bresenham line generation algorit

110. A method of selectively manipulating graphical shape data within a computing machine, said graphical shape data representing a shape which is capable of output to a visual output device, wherein said graphical shape data is operated upon by a line/arc generation algorithm in order to produce motion of said shape.

111. The method of claim 110 wherein said motion is movement of said shape along a path.

112. The method of claim 110 wherein said motion is rotation of said shape.

113. The method of claim 110 wherein said motion is scaling of said shape.

114. The method of claim 110 wherein said motion is distortion of said shape.

115. A method of selectively manipulating data within a computing machine, the method comprising the steps of: forming a first category; and interracting said first category with a second category to produce an object.

116. The method of claim 115 wherein said category is least one selected from the group of shape, path, rotation, time-speed, scale and distortion.

117. The method of claim 115 wherein said category is least one selected from the group of pitch, amplitude, volume rhythm, timbre, distortion, transposition and time-speed.