WO1990010977A1

WO1990010977A1 - Synaptic element and array

Info

Publication number: WO1990010977A1
Application number: PCT/US1990/001116
Authority: WO
Inventors: Carver Andress Mead; Frederico Faggin; Timothy Peter Allen; Janeen Drene Williams Anderson
Original assignee: Synaptics, Inc.
Priority date: 1989-03-10
Filing date: 1990-03-09
Publication date: 1990-09-20
Also published as: EP0414888A4; EP0414888A1

Abstract

An electronic circuit is disclosed having a sample/hold amplifier connected to an adaptive amplifier (10). A plurality of such electronic circuits (10a...10d) may be configured in an array of rows (30, 32) and columns (34, 36). An input voltage vector may be compared with an analog voltage vector (Vi) stored in a row or column of the array and the stored vector closest to the applied input vector may be identified and further processed.

Description

DESCRIPTION

Synaptic Element And Array

1. Field Of The Invention

The present invention relates to artificial neural networks and synapses. More particularly, the present invention relates to a single adaptable synapses and an array of adaptable synapses.

2. The Prior Art

Several schemes for using a matrix of electronic devices for neuron network applications have been proposed. To date, all such schemes involve using "weigh- ts" to control the amount of current injected into an electrical node "neuron". These weights were set by controlling the value of a resistor or the saturation current of a transistor. The limitation of any such scheme is that the value of any parameter of an electronic device in an integrated circuit is not well controlled. For example, the saturation currents of two MOS transistors of the same size can be different by a factor of 2 if these devices are operated in the sub-.threshold regime. The "training" mechanism that adjusts the weights must take these uncertainties into account by iterating and testing the outcome of the weight-adjustment process. It is therefore desireable that an adaptive mechanism be found whereby the matrix element adjusts itself to any uncertainty in device parameters, as part of the training process.

Brief Description Of The Invention

In a first aspect of the present invention, an adaptable CMOS inverter circuit, as disclosed in a co- pending application Serial No. 282,176, filed December 9, 1988, entitled Subthreshold CMOS Amplifier With Offset Adaptation, which application is expressly incorporated herein by reference, is cascaded with a sample/hold circuit. In a preferred embodiment such a sample/hold circuit includes a capacitor connected to an input through a pass gate. The adaptable CMOS inverter may be adapted to a selected voltage placed on the capacitor of the sample/hold circuit. The voltage on the input of the inverter is set by the sample/hold circuit, and the circuit is then adapted by illuminating the aperture located over the feedback capacitor in the amplifier with ultraviolet (U.V.) light as disclosed in co-pending application Serial No. 282,176. After the circuit has been adapted, the current flowing through it will be maximized when the voltage at the input of the circuit equals the voltage to which the circuit has been adapted. In a second aspect of the present invention, a plurality of adaptable CMOS amplifiers and associated sample/hold circuits are placed in a m by n array. The sample/hold inputs of all sample/hold amplifiers in one column are commonly connected to one of a plurality of write lines. The write lines are activated by a write decoder. The voltage input lines to each row of amplifiers in the matrix are commonly connected to a voltage input line. The current supplied to all amplifiers in a column is commonly provided by a sense line.

In order to adapt the matrix of the present invention, the voltages to which a given column of the matrix is to be adapted are placed onto the input voltage lines and into the sample/hold circuits of that column by activating the appropriate write decoder. The input voltages to successive columns are entered sequentially into their sample/hold circuits. The sequential update of the sample/hold voltages is continued, and the entire array is exposed to ultraviolet light for a period of time long enough to adapt the circuits. Brief Description Of The Drawings

FIG. la is a schematic diagram of an adaptive CMOS inverting amplifier circuit according to a preferred embodiment of the present invention. FIG. lb is a plot of output current as a function of input voltage for the circuit of FIG. la.

FIG. 2 is a circuit diagram of a matrix containing a plurality of the circuits in FIG. 1.

FIG. 3a is a schematic diagram of one embodiment of a combination of a sample/hold circuit with a CMOS adaptive subthreshold amplifier circuit.

FIG. 3b is a schematic diagram of a presently- preferred embodiment of a sample/hold amplifier for use with the present invention. FIG. 4 is a schematic diagram of a matrix of circuits similar to those shown in FIG. 3, but using only a single transistor for the sample/hold circuit.

FIG. 5 is a block diagram of the matrix of FIG. 4 and the preprocessing and postprocessing circuits associated therewith.

FIG. 6 is a schematic diagram of a sense amplifier for use in the postprocessing circuits of the present invention.

FIG. 7 is a schematic diagram of a "winner-take-all" circuit for use with the postprocessing circuits of the present invention.

FIG. 8a is a schematic diagram of an alternative embodiment of a CMOS inverting amplifier and error current generating circuit for use with the present invention. FIG. 8b is a graph showing the output current of the circuit of FIG. 8a.

FIG. 9 is a "loser-take-all" circuit for use with the postprocessing circuits of FIG. 8a.

Detailed Description Of A Preferred Embodiment Referring first to FIG. la, an adaptive CMOS in¬ verting amplifier 10 includes an N-channel MOS transistor 12 having its source connected to a source of negative voltage and its drain connected to the drain of a P-channel MOS transistor 14. The gates of N-channel MOS transistor 12 and P-channel MOS transistor 14 are con- nected together to an input node 16. The source of

P-channel MOS transistor 14 is connected to a source of sense voltage V_sense. A first capacitor C. is connected between the input voltage source V,- and input node 16. A second capacitor C₂ is connected between input node 16 and the output of inverting amplifier 10, the common con¬ nection of the drain of N-channel MOS transistor 12 and the drain of P-channel MOS transistor 14. An area above C₂ in the integrated circuit die containing the circuit of FIG. 1 is transparent to U.V. light. The theory and operation of amplifier 10 is fully disclosed in co-pending application Serial No. 282,176, expressly incorporated herein by reference.

The amplifiers disclosed in this specification, and the amplifiers disclosed in co-pending application serial no. 282,176 shall, for the purposes of the specification and claims herein, be referred to as "adaptive" amplifiers.

Referring now to FIG. lb, curves 20, 22, and 24 show the current drawn by the inverting CMOS amplifier 10 as a function of input voltage for three cases in which the circuit has been adapted to different voltages VI, V2, and V3 respectively. A characteristic of the CMOS inverting amplifier 10 of FIG. la is that when the input voltage is equal to the voltage to which the circuit has been adapted, the output current is maximized as shown by curves 20, 22, and 24.

Referring now to FIG. 2, a matrix of four CMOS inverting amplifiers is shown. The first amplifier 10a and second amplifier 10b have their inputs connected together to a input voltage-line V,. Amplifiers 10c and lOd have their inputs connected together to a second input voltage line V₂. The sources of the P-channel transistors in amplifiers 10a and 10c are connected to a common sense line 34. The source of the P-channel MOS transistors in amplifiers 10b and lOd are connected to sense line 36.

One drawback of the array of FIG. 2 is that each amplifier on a common input voltage line is constrained to be adapted to the same voltage since the problem of selectively exposing the feedback capacitors of different amplifiers to ultraviolet light is significant.

One solution to this dilemma is the embodiment shown in FIG. 3a. Referring now to FIG. 3a, subthreshold CMOS inverting amplifier 10 has its input driven by a sample hold circuit comprising a pass gate including an N-channel

MOS transistor 40 and P-channel MOS transistor 42.

Complementary signals SEL and SEL are used to control the pass gate. When the pass gate is activated, the voltage V_j appearing at the input of the pass gate is stored on capacitor 44. Because different voltages may be stored on the individual capacitors in the sample/hold circuits, individual amplifiers in the same row in an array may be adapted to different voltages.

Referring now to FIG. 3b, a presently-preferred embodiment of a sample/hold circuit for use with the present invention is disclosed. A first P-channel transistor channel 46 has its source connected to a source of positive voltage Vdd, shown at reference numeral 48, and its drain connected to the drain of a first N-channel MOS transistor 50. The gate of first P-channel MOS transistor 46 is connected to the drain of first N-channel MOS transistor 50. The source of first N-channel MOS transistor 50 is connected to the drain of second N- channel MOS transistor 52. The source of first N-channel. transistor 52 is connected to a source of negative voltage Vss, shown at reference numeral 54. The gate of first N- channel MOS transistor 50 is connected to an input node V_in.

A second P-channel MOS transistor 56 has its source connected to the source of positive voltage 48, its drain connected to the drain of third N-channel MOS transistor 58, and its gate connected to the gate of first P-channel MOS transistor 46. The source of third N-channel MOS transistor 58 is connected to the source of first N- channel MOS transistor 50 and the drain of second N- channel MOS transistor 52. The gate of third N-channel MOS transistor 58 is connected to its drain, and to one plate of a holding capacitor 44. The other plate of holding capacitor 44 is connected to a source of fixed voltage, preferably Vss, shown at reference numeral 54. The output node 60 of the sample/hold circuit of FIG. 3b, is the common connection of the drain and gate of third N- channel MOS transistor 58 and the first plate of hold capacitor 44. The circuit of FIG. 3b, known to those of ordinary skill in the art as a 5 transistor transconductance amplifier, is superior to the circuit of FIG. 3a because the output V_hold is not disturbed when the select input, the gate of second N-chanriel MOS transistor 52, goes low, because any transient current is divided equally between first N-channel MOS transistor 50 and third N-channel MOS transistor 58. Therefore the effect on V_hold cancels to the first order.

Referring now to FIG. 4, an illustrative embodiment of an array of individual adaptable CMOS inverting amplifiers is shown. A pass gate consisting of a single N-channel MOS transistor 62 and a capacitor 44 form the sample/hold circuit in each of the CMOS inverting amplifiers of the array of FIG. 4. A write decoder -64 has a plurality of output lines which drive the select lines of the sample/hold circuits. Two such lines, 66 and 68 are shown driving the first and second columns in the array, respectively. Write decoder 64 is a high active one of n decoder. Such decoders are well known in the art. In a presently-preferred embodi¬ ment, write decoder 64 will be integrated with the matrix and a CMOS decoder is preferred. Input voltage lines 70 and 72 are connected to all of the amplifiers in the first and second rows of the array, respectively. Current sense lines 74 and 76 are connected to the sources of the P-channel transistors in the first and second columns of the array respectively.

The array of FIG. 4 may be programmed one column at a time by addressing a column of the array by placing a positive voltage on the select line (shown in FIG. 4 as either reference numeral 66 or 68) and placing the input voltage to which the individual amplifiers in the addressed column are to be adapted on each voltage input line (shown in FIG. 4 ^'as reference numerals 70 and 72). After all of the input voltages to which all amplifiers are to be adapted have been placed in the sample/hold circuits the entire array is exposed to a source of ultraviolet light.

The circuit may then be used to associate a series of input vectors with the series of vectors stored as the adapted input voltages. All select lines are simul- taneously brought to a positive level and the voltage vector to be associated is placed on the voltage input lines. The input vector will be closest to the vector stored in one particular column. The input will -cause the current in the current sense line associated with that column to be the highest value.

Referring now to FIG. 5, a matrix M_j, of elements similar to those shown in FIG. 3a, shown at reference numeral 78, is connected to a plurality of inputs through a preprocessing circuit 80 and to a set of outputs through a post processing circuit 82.

Preprocessing circuit 80 may be used for the purpose of normalizing the inputs to the array. In real-world applications these inputs may have peak voltage values which vary over a wide range of voltages. The preprocessing circuit 80 may be used to normalize those voltages so that the magnitude of the resultant voltage vector is constant. An example of such a circuit is disclosed in B. Gilbert, A Monolithic 16-channel Analog Array Normalizer, IEEE Journal of Solid State Circuits, Vol. SC-19, No. 6, p. 956, December 1984, which is expressly incorporated herein by reference. The sense lines associated with each of the columns in the array of FIG. 4 may each be connected to a sense amplifier circuit such as the one shown in FIG. 6. In the circuit of FIG. 6, a sense amplifier 84 includes an operational amplifier 86 having its noninverting input connected to V_sense, and its inverting input connected to the sense line of a particular column. An N-channel MOS transistor 88 has its drain connected to V_0D, a source of positive voltage, its source connected to the inverting input of operational amplifier 86 and its gate connected to the output of operational amplifier 86. A second N- channel MOS transistor 90 has its gate connected to the output of operational amplifier 86 and its source connected to V_sense.

The current flowing into the drain of N-channel MOS transistor 90, shown in FIG. 6 as I¹, will be related to the current flowing in the sense line of the array of FIG. 4 with which the circuit of FIG. 6 is associated. The circuit of FIG. 6 holds the sense line at the voltage V_sense while making a replica of the current I¹., for use in the post processing circuits.

Referring now to FIG. 7, a component of the post processing circuits 82 of FIG. 5, known as a "winner-take- all" circuit is shown. This circuit is completely dis¬ closed in co-pending application Serial No. 277,795, filed November 30, 1988, which is expressly incorporated herein by reference.

In FIG. 7, winner-take-all circuit 92 includes a plurality of sections, two of which are shown. In the first section, a current mirror consisting of P-channel current mirror MOS transistor 94 and P-channel MOS current mirror transistor 96. The drain of P-channel current mirror transistor 96 is connected to the drain of N- channel MOS transistor 98.- The source of N-channel MOS transistor 98 is connected to a source of negative voltage, shown in FIG. 7 as V_ss or ground.

Another N-channel MOS transistor 100 has its source connected to a common gate line 102, its drain connected to a source of positive voltage and its gate connected to the common connection of the drain of N-channel MOS transistor 98 and the drain of P-channel MOS current mirror transistor 96. The node to which the gate of N- channel MOS transistor 100 is connected is the output current node for the column of the array of FIG. 6 associated with that section, shown in FIG. 7 as OU^.

The second section of the circuit for the second column includes a current mirror comprising P-channel MOS current mirror transistors 104 and 106 and N-channel MOS transistors 108 and 110, connected in exactly the same manner as are the transistors for the first section. Common gate line 102 is connected to the drain of N- channel MOS transistor 112 whose purpose is to place a bias on gate line 102.

While the operation of the winner take all circuit in FIG. 7 is fully described in co-pending application Serial No. 277,795 incorporated herein by reference, briefly, follower transistors 100, 110, etc., pull the common gate line 102 up to a voltage at which the current through the common gate pulldown devices 98, 108, etc., is equal to the maximum input current. This turns off all sections in the circuit except for the one having the largest current flowing through it. An alternate embodiment of the synaptic element of FIG. 3a is shown in FIGS. 8a and 8b.

Referring now to FIG. 8a, a CMOS inverting amplifier 10 is driven by a sample/hold circuit including the pass gate comprising N-channel transistor 40 and P-channel transistor 42 driven by SEL and SEL, charging capacitor 44. In the circuit of FIG. 8a, the source of the P- channel transistor in amplifier 10 is connected to a source of reference voltage V_ref and its output is taken from the common connection between the drains of the P- channel and the N-channel transistors.

However, unlike the embodiment of FIGS. 3a and 3b, the embodiment of FIG. 8a drives two follower transistors, N-channel transistor 114 and P-channel transistor 116. N- channel transistor 114 is biased by an error voltage +E_RR and P-channel transistor 116 is biased by an error voltage -E_RR. The error lines, reference numerals 118 and 120 respectively, are biased such that if the output voltage of the amplifier 10 moves away from the center point, the current drawn starts to increase quadratically.

The total output current of the circuit of FIG. 8a is shown in FIG. 8b. The width of the curve in FIG. 8b is determined by C₂ and by the choice of the + and - error voltages which are used to bias lines 118 and 120. In both this circuit and that of FIG. 3a, the error current is a monotonic function of the unsigned difference between the amplifier output voltage and the output voltage at which the amplifier was adapted.

Referring now to FIG. 9, a post processing circuit for use with an array of circuit such as that shown in FIG. 8a is disclosed. This circuit may be referred to as a "loser-take-all circuit" 122 which causes the column having the lowest output current to be passed to the output. One section of a plurality of sections is shown.

In FIG. 9, a current mirror composed of P-channel MOS current mirror transistors 124 and 126 has its first P- channel MOS current mirror transistor 124 connected to the I'_colx node and has its second P-channel MOS transistor connect in series with an N-channel MOS transistor 128. The N-channel MOS transistor 128 has its source connected to a source of negative voltage, shown in FIG. 9 as ground or Vss, and its gate connected to a common gate line 130 for N-channel transistors for all other legs driving all the other columns from the matrix. A third P-channel transistor 132 has its gate connected to the common connection of the drains of N-channel transistor 128 and P-channel transistor 126, its drain connected to ground, and its source connected to the common gate line 130 of the post processing circuit. A bias transistor 134 supplies a bias to common gate line 130.

The loser-take-all circuit of FIG. 9 operates as follows. A bias current is injected into the common gate line 130 by bias transistor 134. That current is just balanced by the current out of the node V_out passing through N-channel MOS transistor 128 of the section whose V_out is the least. The lowest V_out will correspond to the smallest input current in the current mirror, shown in FIG. 9 as 124 and 126, in the section having that least current. In operation, the voltage on node 130 will rise until the N- channel MOS transistor, shown associated with the first section as transistor 128, of one section is able to pull its V_out low. This single low output will drain the bias current from node 130, and its voltage will stabilize such that the drain current of N-channel . transistor 128 just balances in the input current. All sections with higher input currents will have V_out high. Hence, this circuit functions as an encoder of the stage with the minimum input current.

When used with an array of cells with the type shown in FIG. 8a, the circuit of FIG. 9 encodes the column which has the minimum error current, and therefore whose stored vector is closest to the input vector.

While a presently-preferred embodiment of the invention has been disclosed, those of ordinary skill in the art will, from an examination of the within disclosure and drawings be able to configure other embodiments of the invention. These other embodiments are intended to fall within the scope of the present invention which is to be limited only by the scope of the appended claims.

Claims

1. An electronic circuit including in combination: an adaptive amplifier having an input node and an output node, said adaptive amplifier including means for adaptation by ultraviolet light to a preselected voltage placed upon said input node, a sample/hold circuit having an input, a select input, and an output, said output connected to the input node of said adaptive amplifier.

2. The electronic circuit of claim 1 wherein the sample/hold circuit is a five-transistor transconductance amplifier having a capacitor connected between said output and a source of fixed voltage.

3. The electronic circuit of claim 1 wherein said sample/hold circuit comprises a pass-transistor having a gate, a source and a drain, and a capacitor, said source of said pass-transistor being said input of said sample/hold circuit, said drain of said pass-transistor being said output of said sample/hold circuit, said capacitor connected between said output and a source of fixed voltage, and said gate of said pass-transistor being the select input of said sample/hold circuit.

4. An electronic circuit including in combination: an adaptive amplifier having an input node and an output node, said adaptive amplifier including means for adaptation by ultraviolet light to a preselected voltage placed upon said input node, sample/hold circuit means, connected to the input node of said adaptive amplifier, for selectively storing an analog voltage at the input node of said adaptive amplifier.

5. An electronic circuit including in combination: an adaptive amplifier having an input node and an output node, said adaptive amplifier including means for adaptation by ultraviolet light to a preselected voltage placed upon said input node, a sample/hold circuit having an input, a select input, and an output, said output connected to the input node of said adaptive amplifier, and means within said adaptive amplifier for generating an error current which changes monotonically as a function of the voltage difference between the actual voltage on said output node of said adaptive amplifier and said preselected voltage.

6. The electronic circuit of claim 5 wherein said adaptive amplifier is a CMOS inverter, and said error current is the current drawn by said inverter.

7. The electronic circuit of claim 6 wherein the sample/hold circuit is a five-transistor transconductance amplifier having a capacitor connected between said output and a source of fixed voltage.

8. The electronic circuit of claim 6 wherein said sample/hold circuit comprises a pass-transistor having a gate, a source and a drain, and a capacitor, said source of said pass-transistor being the input of said sample/hold circuit, said capacitor connected between said output and a source of fixed voltage, said drain of said pass-transistor being the output of said sample/hold circuit, and said gate of said pass-transistor being the select input of said sample/hold circuit.

9. An array of electronic circuits arranged in rows and columns, each of said electronic circuits including an adaptive amplifier having an input node and an output node and means for generating an error current, and a sample/hold circuit having an input, an output, and a select input, the output of said sample/hold circuit connected to the input node of said adaptive amplifier, the select inputs of all sample/hold circuits associated with the electronic circuits in a given column of said array connected in common to one of a plurality of write lines, and the error current generated from each adaptive amplifier in a column connected in common to one of a plurality of output sense lines, and the inputs of all the sample/hold circuits in a row of said array connected in common to one of a plurality of input voltage lines.

10. The array of claim 9 wherein said means for generating an error current generates an error current decreasing monotonically with error and further including: write decoder means connected to said plurality of write lines for selectively activating one of said write lines at a time, a plurality of sense amplifiers, one of said sense amplifiers having its input connected to each of said sense lines, and means connected to the outputs of said sense amplifiers for indicating which one of said plurality of output sense lines is drawing more current than the other ones of said plurality of output sense lines.

11. The array of claim 9 wherein said means for generating an error current generates an error current increasing monotonically with error and further including: write decoder means connected to said plurality of write lines for selectively activating one of said write lines at a time, a plurality of sense amplifiers, one of said sense amplifiers having its input connected to each of said sense lines, and means connected to the outputs of said sense amplifiers for sensing which one of said plurality of output sense lines is drawing less current than the other ones of said plurality of output sense lines.