CA1073553A - Error recovery and control in a mass storage system - Google Patents

Abstract

ERROR RECOVERY AND CONTROL IN A MASS STORAGE SYSTEM
Abstract A virtually addressed multilevel mass storage system (MSS) has error recovery and definition procedures and apparatus for controlling and enabling recovery from error conditions in an upper storage level. A plurality of possible error conditions in an upper level gives rise to errors in destaging data signals to a lower level, plus possible overwriting good data with data in error.
A first such error condition is a data error detected during a destage. A second such error condition is repeated upper-level equipment (not data) errors. Both errors make data integrity of the lower-level suspect.
Corrective action for a plurality of errors includes co-ordination with a host computer, reconfiguration, destaging data in error after precautionary steps, and preserving data in error at the failing upper level unit which is used in virtual mode except for the portion yielding the error condition.

Description

Documents Relevant to the Invention Burke et al U.S. Patent No. 2,941,738 shows the fundamental concept of an automated record tape storage and retrieval apparatus for attachment to a computer.
Beach et al U.S. Patent No. 3,831,197 shows the enhanced concepts of record media automatic storage and retrieval and particularly the connections in a multi-computer environment.

,~3'' . -. : .
. - ~, . : ':"
., , - . ,, . . . , . . . . - . -,. . , : . . :. : : ::

-. - ::. - -: . - : . .
. . .. . . - .. - : : : :: . :
. . . : . .- . . .

1~73553 1 A mass storage control (MSC) can be con-

2 structed generally in accordance with u. S. Patent

3 3,400,372; specific constructional features are set

4 forth in the description.
Glossary 6 A A-bus, an input to ALU (supra).
7 AASM A-bus assembly circuits for ALU.
8 A-Box DASD controller 15.
9 ACR Address Compare Register.
Active Mass storage volume stored in MSF and Volume 11 available for mounting.
12 ADDRO Address Out.
13 ADR Address.
14 A~U Arithmetic Logic Unit.
AMR Address Mask Register.
16 ASQ Active Schedule Queue.
17 B B-Bus, input to ALU (supra).
18 BASM B-Bus Assembly circuits for ALU.
19 BLK Block.
BOPAR Bus Out Parity.
21 BR Branch Register in computer 20; contains 22 signals on which conditional branches in 23 microprograms can be based.
24 BTRDY Byte Ready.
CA An instruction word field.
26 CAR Cylinder Address Register.
27 CARRY Field of microinstruction related to 28 branch on high.

.j , BO97502~ -3-.

.. . - , - . , - . -. - . - .

~73SS~ ~

1 CB An instruction word field.
2 CC Cylinder identification in a disk spindle.
3 CCHH Cylinder and head identification in a disk 4 - spindle.
CD An instruction word field.
6 CE An instruction word field.
7 CH Field of microinstruction related to ~, 8 branch on high.
9 CHANB Channel.
I
CHK Check.
11 CHL Channel.
12 CHR Cylinder, Head, Record identification.
13 CK Multipurpose field of microinstruction;
14 can be a constant.
CL Field of microinstruction related to 16 branch on low.
17 COMMO Communication.
18 CPV Central Processing Unit.
19 CTOC Cartridge Table Of Contents.
CS An instructlon word field.
21 CU Control Unit.
22 CUEND Control Unit End.
23 CV An instruction word field.
24 CW, An instruction word field.
CX An instruction word field.
26 CYL Cyli/nder; all circular record tracks in a 27 DASD unit having the same radius.
28 D D-Bus, output of ALU (supra).

,~ .. j .
~0975026 -4-', - 1~73S53 1 DAR Data Address Register in computer 20. ~ -2 DASD Direct Access Storage Device; a disk - 3 storage unit as shown in Goddard et al 4 U. S. Patent 3,503,060.
DASDER~SE ~n attribute of a mass storage volume 6 that causes binary zeroes to be written 7 on staging drive after data from mass 8 storage volume has been destaged.
Cartridge Storage medium of MSS. ~
DE Device End. ~;
11 DEQ Dequeue.
12 DIR Director.
13 Director Direct ~ccess Storage Device DASD control 14 unit in the mass storage system that con-trols the transfer of data during staging 16 and destaging operations.
17 DISP Dispatcher.
18 DNSTZl Field of microinstruction related to 19 branch on high.
DR Destage Read; a message from MSC 17 to 21 direct,or 16 to read from a DASD spindle 14.
22 DRC Data Recording Control; component of MSF
23 that controls DRD's, encodes and decodes 24 data, and assists with error recovery.
DRD Data Recording Device; unit in ~ISF which 26 reads and writes data on the cartridge tape.
27 DRV Drive/driver.

Bo975026 ~5-' , ' ' .''''.

- . :
... . .

11~73S.~3 l DSQ DASD Schedule Queue; list of pages to be 2 moved from DASD to MSF.
3 DSTG Destage; move data from a stage drive to 4 - mass storage volume.
DXFER Data Transfer.
6 ECB Event Control Block.
7 ECC Error Correction Code.
8 ENQ Enqueue.
9 EO Emergency Off.
ERP Error Recovery Procedure.
11 ÆXT External.
12 FCN Function.
13 FF Maximum two-digit hexadecimal value. l 14 FM Microprogram instruction word field. I
FMT Format 16 F/S Fetch/Store.
17 FTCH Fetch.
18 GA ~icroprogram general purpose register.
l9 GB Microprogram general purpose register.
GC Microprogram general Furpose register.
21 GD Microprogram general purpose register.
22 GE Microprogram general purpose register.
23 GF Microprogram general purpose register.
24 GP General Purpose.
Group Staging drive group.
26 HDR Header; a set of control signals.
27 HI High 28 HLTIO Halt Input Output; command that stops 29 I/O operations.
' ~3553 1 IAL Lower-ordered byte portion of IAR.
2 IAR Instruction Address Register in computer 3 20 (two bytes).
4 ID Identification.
ILXEQ In-Line Execute.
6 IML Initial Microprogram Load; action of 7 loading a microprogram into control 17 8 or director 16.
9 IMPL Initial Microprogram Load.
INDEX Offset/pointer.
11 INLI~ In-Line.
12 I/O Input/Output.
13 IOC Inout/Output Control - a program.
14 IORB Input/Output Request Block.
IORC Input/Output Return Code.
16 IOS Input/Output System - a program.
17 JL Job List (intra); part of scheduler 18 which contains ID of all "jobs" or 19 programs to be invoked.
JLS Job List Scheduler.
21 Data Set Mass storage volume control journal data 22 set.
23 Journaling Recording transactions against a data set 24 so that the data set can be reconstructed by applying the transactions in the journal 26 against a previous verslon of the data set.
27 K Constant.
28 KK Microprogram instruction word field.

.

"'''.' ' , '' .': .' ' ' '~ :. . . . '' -. -~: : . - : ,. -~o~ 3 1 LIB Mass storage facility 10 2 LRU Least Recently Used; an algorithm that 3 determines the order in which active 4 staged pages must be destaged. The al-gorithm makes sure that the staging drive - 6 group always has the amount of allocatable 7 space defined by the space manager.
8 LTR Logical To Real; a program table.
9 LUA Logical Unit Address; an address of a DASD
spindle 14 used by host computer 19 which 11 has a virtual address translated by MSS to 12 the actual units.
13 LXYZ Address in MSF; L is tape library number, 14 X is horizontal storage cell address, Y is vertical storage cell address, and Z is 16 the storage wall number in a library.
17 MA Microprogram general purpose register.
18 Mass Storage A direct access storage volume residing on Volume 19 two associated data cartridges of MSF 10.
MB Microprogram general purpose register.
21 MC Microprogram general purpose register.
22 MD Microprogram general purpose register.
23 ME Microprogramming register.
24 MF Microprogram general purpose register.
25 MH An instruction word field.
26 MINI Miniheader; a set of control signals.
27 ML An instruction word field.
28 MPL Microprogram Load.

~073553 1 MSC Mass Storage Control; a microprogrammed 2 portion of the mass storage facility that 3 passes information to the accessor con-4 trol and controls data and space on staging drives.
6 MSF Mass Storage Facility the component of a 7 mass storage system that contains the 8 storage media and the facilities for 9 accessing it.
MSG Message.
11 MSS Mass Storage System; the name for the 12 entire storage system consisting of the 13 mass storage facility and all devices that 14 are defined to the mass storage control.
MSSC Mass Storage System Communicator; a program 16 in a CPU that handles communication between 17 system csntrol programs (VS/370) and the 18 mass storage control. The mass storage 19 volume control functions are an integral part of the mass storage system communicator.
21 MVT Mount Volume Table; a set of control regis-22 ters in control 17 identifying volumes 23 logically mounted on DASD.
24 NA Microprogram general purpose register.
N~ Microprogram general purpose register.
26 NC Microprogram general purpose register.
27 ND Microprogram general purpose register.
28 NL Instruction word field.
., , i i }~0975026 ' .

~073553 1 Donistaging Real Drive.
2 OP Operation code of an instruction word.
3 OS/VS Control program in a host that operates 4 in a virtual mode.
Page Eight radially contiguous DASD cylindèrs.
6 PARMS Parameters.
7 PASANO Program module to pass control from ser-8 vice module to service module.
9 Path Hardware connection known to the operating system that permits the movement of data 11 signals within the hardware.
12 PGM Program.
13 PH - Primary Host.
14 PLO Phase Lock Oscillator.
15 Primary CPU The CPU in a multi-CPU system configura-16 tion that has the responsibility of pro-17 cessing unsolicited messages from the 18 MSC.
19 PST Page Status Table in director 16.
PTR Pointer; a set of control signals identi-21 fying location of signals "pointed to".
22 The pointer may be a memory address, off-23 set of a table in memory, register loca-24 tion, etc.
QCB Queue Control Block; a set of control sig-26 nals necessary for storage control 17 to i ; .
27 execute a queue of tasks or functions.

, .

1 RAS Reliability, Availability, and Service-2 ability.
3 Real Drive A drive attached to director 16.
4 REG Register.
REL Release.
6 RP Real Page.
7 RSG# Real Spindle Group Number; address of 8 DASD controller 15.
9 R/W Read/Write.
SA Depending on context - a microprogram 11 general purpose register or Staging 12 Adapter, a portion of director 16 for 13 automatically and independently moving 14 data signals between DASD units 14 and 15 and DRD's.
16 SB Microprogram general purpose register.
17 S/B Sense Byte.
18 SC Microprogram general purpose register.
19 SCHED Scheduler.
SD Microprogram general purpose register.
21 SDG Staging Drive Group; a collection of 22 staging spindles for space management 23 and recovery.
24 SECTR Sector.
SELTD Selected.
26 SERDES Serializer-Deserializer.
27 SNS Sense.

. ,., ~ .

:
.

~073553 1 Solicited A message from the Mass Storage Control Message 2 to the CPU that is expected by the CPU.
3 Sp op Special Operation.
4 SSID Subsystem Identification; identification on each device or unit in the MSS.
6 ST Status register in computer 20.
7 Stage To move data from a data cartridge to a 8 staging DASD spindle.
9 Staging A DASD spindle designated to receive data Drive from a Mass Storage Facility.
11 Staging Disk pack that has been initialized to Pack 12 receive data from a Mass Storage Facility.
13 SUPPO Suppress Out.
14 SW Switch.
SYMP Sympton; an error definition.
16 TA Microprogram general purpose register.
17 TB Microprogram general purpose register.
18 TC Microprogram general purpose register.
19 TD Microprogram general purpose register.
TE Microprogram general purpose register.
21 TF Microprogram general purpose register. ', 22 ~G Microprogram general purpose register.
23 Trace A monitor in the MSC that records data 24 about MSS activity and staging and de-staging; the data aescribes completed MSS
26 functions from the activity schedule 27 queues plu8 time stamps.
28 UCB Unit Control Block.

. ., .j .

i : . .

~373553 `

1 UDEP Unsolicited Device End Proces~or.
2 Unsolicited A message from the MSC to the primary CPU
Message 3 that is not requested or expected by the 4 primary CPU.
VFO Variable Frequency Oscillator.
6 Virtual A direct access storage device that does Drive 7 not physically exist; it exists logically 8 on one or more staging drives.
9 VOLID Volume Identification.
VP Virtual Page; also see Page.
11 VUA Virtual Unit Address; an address for a 12 virtual drive that consists of the channel 13 address, the Staging Adapter address, and 14 the device address. The virtual unit ad-dress can be assigned to any staging drive 16 group. Each staging drive can have more 17 than one virtual unit address, but only one 13 real unit address (RUA).
19 W Virtual Volume; the data from a mass storage volume while it is located on a virtual 21 staging drive.
22 W A Virtual Volume AddresR.
23 W IT Virtual Volume Identification Table.
24 WD Word of four bytes.
25~ XFER Transfer.
26 XREF Cros reference.

, . . . - - -- . : . . . . .
.. . . ~, . , ~ .

1 Background of the Invention 2 The invention relates to mass memory 3 apparatus of the staging data type and more particularly 4 to error control apparatus and methods for use therein.
Direct access storage such as disk storage 6 devices (DASD) has many advantages when used in a 7 data processing system. For example, it enables rapid 8 access to a data record as opposed to moving a record 9 tape to scan long sequential files. It is usually on-line when one needs it. It is reliable. But such 11 direct access storage is expensive. Also, the number 12 of disk drives attachable to a host CPU is usually 13 limited. It is also inefficient because the amount 14 of data in use on a single disk device or drive at oné time is usually small.
16 On the other hand, tape storage has 17 many advantages when used in a data processing system.
18 Large quantities of data can be stored in a tape library.
19 It is reliable and relatively inexpensive. But an entire tape file, perhaps of several reels, must be 21 read and rewritten to obtain a few records that are 22 needed for data processing. Processing must be sequen-23 tlal, which requires transaction files to be sorted 24 before updating a master file. Time can be wasted in finding the proper reel to mount. Mounting the 26 wrong reel of a multi-reel tape file causes rerun 27 problem9. Also, maintaining a tape library can be 28 expensive.

,;, ' ~ 735S3 1 To enhance data processing, a staging 2 mass storage system (MSS) combines the better features 3 of disk storage with the economy of tape storage.
4 The storage capacity equals that of a large tape library.
Data can be processed in a tape-like sequential manner 6 or in the efficient direct access disk manner. Most 7 important in an operating environment, the data is 8 available to the processing system without the delays 9 associated with the finding of the tape reel, mounting it, and returning it to the tape library after use.
ll ~ddressing such apparatus is in a so-called "virtual 12 direct access storage" mode as described in U. S. Patent 13 3,670,307 as implemented using the International Business 14 Machines Corporation 3330 disk storage virtual volume addressing scheme. This scheme defines a logical 16 address space as containing 100,000,000 data bytes--17 the storage capacity of one IBM*3330-type disk pack.
18 Usage of this addressing scheme will become apparent.
19 ~uring the time the data is being processed, it is on-line on 3330 disk drives. When the data is not 21 in use, it is stored on tape in a Mass Storage Facility 22 (MSF) 23 ~ ~lOUNT virtual volume message given 24 to an MSC initiates transfer of data from tape to disk. The MSC searches its tables, finds the location 26 in the MSF where a data cartridge containing that 27 data is stored, finds space on an available disk drive, 28 reads the data from the data cartridge, and wri-tes 29 it on the disk dri~e. -*Registered Trade Mark of International Business Machines Corporation ~073553 .
1 The disk packs to which the data stored 2 in the data cartridges is written are called "staging 3 packs," and the process of copying the data from the 4 data cartridge onto the disk pack i9 called "staging."
Data must be staged ~efore it can be processed by 6 a host. The data needs only be staged once for multiple 7 concurrent uses.
8 The process of writing the disk cylinders 9 containing changed data back to the data cartridge is called "destaging." Since all the original data 11 is still on the data cartridge, writing the changed 12 data back results in the data cartridge having a complete 13 updated data set. Data signals stored in disk storage 14 that i8 not altered are never destaged.
Staging packs are divided into pages 16 of storage. Each page consists of eight cylinders.
17 There are 51 pages of staging space on one staging 18 pack. When data is staged, it is written on whichever 19 pages of space are available at the time. The data from a single data set does not necessarily go on 21 consecutive pages of a staging pack, nor does it neces-22 sarily use only pages on a single staging disk drive.
23 When host computers to the MSS are IBM
24 370 type, the MSS ~esponds to the program operat~ng system OS/~S of such 370 host machines in the virtual 26 direct access storage mode. That is, MSS looks like 27 a lot of disk drives to the ho~ts. This means th~t 28 the known 370 OS/VS programs for operating with the E09750~6 -16-`

- - -:

~C~73553 1 3330 virtual volu~es also operate with MSS. In this 2 mode, OS/'~S assigns a disk virtual voluMe to a system 3 "unit." When a virtual volume is mounted in the .~SS, ~ it is also assigned to a unit address. Since, in MSS, a virtual volume can be as small as one page, 6 a complete staging pack could mount 51 virtual volumes 7 and therefore need 51 unit addresses. Because of 8 this, the old idea of the unit address being a com-9 bination of channel, control unit, and device is modi-fied. MSS uses a "virtual unit address" to designate 11 the logical address of each virtual volume. Each 12 virtual volume is assigned a virtual unit address 13 to be used by MSS in staging data and in locating 14 it on a staging pack. A group of virtual unit addresses is assigned to each group of real disk drives, termed 16 "staging drive group".
17 In operation, these virtual unit addresses 18 are varied on-line and off-line just like other system 19 units and real units are varied on and off.
~n an MSS destaging operation, data signals 21 read from DASD units go through a buffer in a director 22 16 into the tape units DRD for a recording or write 23 operation. The format of the data, as recorded on DASD, 24 is imaged on the tape; that is, the data format includes count, key, and data widely used on DASD. If a write 26 occurred on DASD, a host CPU updated the data which 27 means that the count, key, and data are all changed.
28 Further, control signals recorded at the beginning . ~

. .. , , . - : :-............................. :
-, - - -- ' ' . ' , 1 of a cylinder of data are probably also 2 changed. In summary, the entire data format after a , 3 recording operation on DASD is entirely different from 4 the data format prior to such writing operation. It is revised or newly formatted data signals which have 6 to be accurately recorded on the tape. , ', 7 Each DASD cylinder includes a plurality of 8 record tracks, one track on a recording surface. For 9 example, in one DASD unit, 17 tracks constitute a cylin-der, all of the tracks being at the same radial position 11 on the respective record surfaces. When transferring 12 data from the DASD cylinder to the tape, an error may 13 occur anywhere within the cylinder. Generally, such 14 errors occur only on one track. At this point in time, ~, the DASD reading operation is aborted using known pro-16 cedures. As such, the signals recorded on the tape 17 which correspond to the data signals supposedly recorded 18 in the DASD cylinder contain partly the newly formatted 19 data signals, plus a remainder (unknown amount) of the old forma,tted tape signals. Since the control'signals 21 are alway~ recorded at the beginning of the cylinder, 22 the control information defining the signals following 23 the error has already been destroyed; i.e., the data 24 which has been destaged for the DASD read error has overwritten the old formatted data. Hence, on tape at 26 the on3et of a DASD read error, the tape has a portion 27 of the newly formatted data plus an unknown portion of 28 the old data which has had its control information co~-, . ~, . ,.,j .
¦ BO975026 -18-I! .

, .. , . : ~ . . :

` - ~ 1073553 1 pletely obliterated. It is extremely important that the destaged data signals be in one format; otherwise, all of the data recorded on the tape iil that particular portion ~ecomes substantially meaningless.

The foregoing and o~her objects, features, and advantages of the invention will become apparent from the following more particular description of the preferred embodiment of the invention, as illus-trated in the accompanying drawing, The Drawing FIGURE 1 is a diagrammatic showing of a -mass storage system of the single address field dual- -level type, FIGURE 2 is a diagrammatic showing of storage director apparatus usable with the FIGURE 1 illustrated mass storage system, FIGURE 3 is a diagrammatic showing of micro-codable processor apparatus usable with the FIGURES . :~
1 and 2 illustrated apparatus. --FIGURE 4 is a diagram of a symbolic repre- ;;
sentation of a microcode instruction word usa~le with ;", the FIGURE 3 illustrated apparatus.
E'IGURE S is an overall flow diagram of one embodiment of tlle invention.
FIGURE 6 is a flow diagram of error recovery microprogram steps for detecting an attempted destage with error.
FIGURE 7 is a flow diagram of queue control ~
block modifying microprogram steps related to a lower ~;
storage level.

BO9-75-026 ~ -19- ;.

B~ ` :
.. :~
... . . .. .
, ~- - - - :.

1073553 : `
1 FIGURE 8 is a flow diagram of destage restart microprogram steps.
FIGUI~ 9 is a flow diagram of mark page in error microprogram steps.
FIGURE 10 is a flow diagrarn of D~SD or upper t level queue control block restart microprogram steps. ç
FIGURE 11 is a diagramrnatic showing of an alternate embodiment of the invention.
FIGU~ES 12-73 are flowcharts showing instruc~
tion sequences implementing the flowchart steps of FIG~RES 6~10; the numeric designations on the flow-chart steps also identify the respective instruction sequence flowcllart.

!~
i~ ~

BO9-75-026 -19a-~ ' ' '.
T--1 The present invention is most advantageously used with apparatus referred to above and as shown in FIGURE 1. ~n MSS apparatus includes an MSF 10 having a tape cartridge store such as shown in Beach et al, supra. MSF 10 also includes a plurality of data recording devices tDRD) 12 (tape recorders) and associated data recording controls (DRC's) 13 (tape recorder controls) all constructed in accordance with the documents incorporated by reference. MSF 10 con-stitutes the d~ta base memory portion of the MSS.
An intermediate storage level of MSS consists of a plurality of disk storage units (D~SD) 14, asso-ciated D~SD controllers 15, and storage controls or directors 16. Each director 16 includes a staging adapter portion for automatically moving data siynals between MSF 10 and DASD 14 and 15. Moving data signals E
from MSF 10 to DASD 14 and 15 is termed "staging" (data ¦
promotion to a higher storage level), while moving data signals from DASD 14 and 15 to MSF 10 is termed ¦;
"destaging" (data demotion to a lower storage level).
~n MSC 17, a programmable computer, supervises and directs operations of MSS as will become more clear.
One programmable host computer is a so-celled "primery" host 18. This computer, in a limited ' . .

~ . . .

- .: . - . - - : . : .. .. -: - . - . : .- . . - : -- . . .

1~73553 1 manner, supervises operation of MSS on behalf of all 2 other connected host computers 19. ~ach host com-3 puter 19 has at least one ehannel eonnection to a 4 storage director 16; such channel connections are in aeeordance with Patent 3,400,372. Additionally, 6 primary host 18 has a ehannel eonnection to MSC 17 7 for issuing eommands and reeeiving MSS status signals, 8 as will beeome more apparent. The ~ISC 17 acts as a 9 eontrol unit to primary host 18, all in aecordanee with Patent 3,400,372. MSC 17 connections to MSF
11 10 eontroller 21 and to storage direetors 16 are also 12 in aecordance with Patent 3,400,372, wherein MSC 17 13 is a "host" or "CPU" and units 16 and 17 are the control 14 units of 3,400,372. Controller 21 is as described in Beach et al, supra, and Carter et al T921,023, 16 dated April 16, 1974.
17 As described above, a problem presented 18 in operating a multi-level or hierarchal MSS during 19 destaging or data demotion from the DASD upper storage level to MSF 10 lower storage level is handling and 21 reeovery from D~SD read errors. Each llost CPU must 22 have an opportunity to take reeovery aetions before 23 sueh data is destaged to MSF 10 with an error. Re-24 covery from DASD read errors is 99.5% sueeessful by manually moving a disk paek from one disk drive to 26 another disk drive. That is, 99.5~ of the time the 27 seeond disk drive successfully reads the moved disk 28 paek. In a virtual addressing environment during ~ ' .

- : . :

1~7;~553 1 MSS operations, moving disk packs can destroy address-2 ability--the data cannot be accessed by any host.
3 When moving disk packs from one drive to another, the 4 same channel address can be maintained even though the ~ -pack is on a different drive. In this manner, address-ability is maintained.
7 In a real addressed system, storage equipment 8 errors or checks are not readily propagated as data 9 errors to data in other storage equipment at the same storage level. In a virtually addressed MSS, one 11 storage unit may contain data from many diverse sources;
12 hence, one storage unit having error conditions can 13 result in widespread data sets with increased catastrophic 14 effects over real addressed storage. Such a situation should have early detection and correction.
16 Sununary of the Invention 17 ~ccordingly, it is desired to provide apparatus 18 and methods for enablinq error recovery in a virtual 19 address MSS. -I`his invention enables a vir~ually addressed 21 MSS to recognize DASD (upper level) read errors, reserves 22 the D~S~ spindle in error to an MSC, notifies the 23 appropriate host of the error, accepts the host action 24 causing the failing destage operation to be restarted, and if still in error to destage the data in error 26 with suitable precautions.
27 In a best mode of the invention, the procedures 28 of the invention are initiated and monitored by program-29 ming in MSC 17 and directors 16.

~.

1C~73553 1 ~lSC 17:
2 1. Determines if the DASD read error is 3 a part of a destage operation and if not continues 4 normal processing; i:e., the in~ention is not invoked.; ~
2. Converts the cylinder-in-error address ~ -6 to a mini head page list element index, a program 7 tool to initiate procedures of the invention.
8 3. Determines if this is first or second 9 error on this cylinder and sets flags accordingly.
Then, MSC 17 operates on Queue Control Blocks 11 (QCB1: -12 1. Updates page list entries preceding the 13 falling entry (DASD read error), plus the successful -14 cylinders of the failing entry.
2. Schedules a task Destage Restart for 16 execution; a program will restart the aborted destaging 17 operation.
18 The Destage Restart function is by a program-19 which:
1. Determines if this is first or second 21 error.
22 2. If first error, then program issues 23 a status information message to the host processor 24 and then exits.
3. If this is second error, it issues a 26 message to the host processor that a destage with 27 error is occurring.

~ , . . .; ~' ' ~
.. . . . ... ... .. .. . _ __ _ :

~73553 1 4. A program step will Mark Page In Error 2 to identiEy the eight cylinders associated with the 3 D~SD read error.
4 5. Calls DASD QCB Restart; program, see below.
6 6. Initiates Destage with Error.
7 7. Releases the reserved disk drives for 8 use by directors 16.
9 The UDEP (Unsolicited Device End Processor) function is an expected action by the user to move 11 the disk pack to another disk drive. In readying 12 the pack on the new drive, a device end interrupt ~ -13 is presented to the MSC.
14 Mark Page in Error (used only when a data error occurs) marks the page (eight D~SD cylinclers) 16 containiny the error as not available for use:
17 1. On recognizing a DASD device end, inter-18 rupt calls D~SD QCB Restart.
19 D~SD QCB Restart:
1. Searches all QCB's that have been marked 21 in error for D~SD and sets those executable that have 22 Staging Drive Group and Unit equal to that presented 23 by UDEP or Destage Restart.
24 2. On findin~ a QCB to be reset, it follows all chains and resets all appropriate QCB's.
26 3. Destaging is reinitiated.
27 ~ similar procedure is followed for plural 28 equipment errors even though no data error is detected.

BO~75026 -24- ~ -:' ' . -,~ , .
.... .

~C~73S53 1Such plural equipment errors indicate onset of possi-ble data errors. Mark page in error is not used.

Detailed Description .
Referring more particularly to the drawing, like numerals indicate like parts and structural features in the various diagrams. The present invention enhances the apparatus described with respect to E~IGUR~ 1 in that errors detected in DASD spindles 14 operations 10are controlled such that the errors are not propagated undetected to MSF 10. Without the present invention, the FIGURE 1 illustrated apparatus, during a destaging operation, ~oves data in error from DASD spindles 14 through DASD controllers 15, thence through the staging adapter portion (later described) of directors 16, thence to the DRC's 13, DRD's 12 of ~lSF 10.
In accordance with the invention, such an unsupervised operation, that is, primary host 18 and ~`~?
~ . ' - ~

1(~735S3 1 other hosts 19 have no direct intimate control over 2 the operation of the MSS, cannot always control error 3 propagation. Such hosts operate under the assumption 4 that data in MSF 10, DASD spindles 14, and data trans-mitted therebetween is always correct. Under unusual 6 circumstances, data errors can occur. By controlling 7 the propagation of such errors and appropriately sig-8 naling a host 18, 19, error propagation can be controlled 9 with the data in error preserved for later analysis and recovery beyond the scope of the disclosure.
11 Both MSC 17 and directors 16 are micropro-12 grammable processors with directors 16 additionally 13 having special circuits for exchanging data signals 14 between a host 18, 19 and DASD spindles 14, or between DASD spindles 14 and MSF 10. Since a portion of the 16 invention is initiated under microprogram control, 17 the two microprocessors are described in diagrammatic 18 form. FIGURE 2 illustrates the MSC 17, while FIGURE
19 3 illustrates the storage director with staging adapter 16. Intercommunicatlon between processors is in 21 accordance with Patent 3,400,372. That is, primary 22 hosts and other hosts 19 communicate with MSC 17 wherein 23 MSC 17 is a control unit of the I/O descriptive portion 24 of 3,400,372. Similarly, directors 16 are also control units as described in ~hat patent. Additionally, 26 MSC 17 acts as a host computer with the directors 27 being a control unit to MSC 17~ That is, MSC 17 is 28 a microprocessor logically interposed between primary .j., ~ .

- . , - .

16~73SS3 1 host 18, other hosts 19, and the directors 16.
2 Directors 16 respond to channel commands from MSC
3 17 in the same manner as they respond to prlmary host 4 18 and other hosts 19. Mass storage commands are uniquely received by MSC 17. Since general operation 6 of MSS is not a part of the invention, those particular 7 commands are not further described. For convenience, 8 directors 16 have a four-channel interface; i.e., 9 each director 16 can connect up to four hosts. MSC
17 appears as one of the hosts, always on the same 11 interface connection. If the four connections are 12 labeled A, B, C, and D, host MSC 17 is connected to 13 interface A; while primary ho~t 18 and some of the 14 other hosts 19 are connected to the other three inter-faces. Primary host 18 needs not be connected to 16 all storage directors 16, while MSC 17 should be connected 17 to all directors 16.
18 MSC 17 command~ director 16 to exchange 19 data signals between DASD spindles 14 and MSF 10.
Additionally, MSC 17 supplies MSF 10 operating commands 21 to library controller 21, the details of which are 22 beyond the scope of the present description.
23 Both MSC 17 and directors 16 have the same 24 constructed microprocessor circuits. That is, the calculator portion of directors 16 and MSC 17 are 26 identical. Such calculators are also shown in Figure 27 4 of U. S. Patent 3,716,837 and in the DASD Director 28 Model 3830 produced by International Business Machines .
. .

1 Corporation, Armonk, New York. For a better under-2 standing of the microprogramming involved with the 3 present invention, the calculators and their relation-4 ship to the directors 16 are set forth in sufficient detail in combined flowchart and object code form 6 to facilitate a ready understanding of what is 7 achieved.
8 FIGU~E 2 diagrammatically shows MSC 17.
9 The core of the control is a set of computer circuits 20 constructed ao shown in FIGURE 3. These computer 11 circuits communicate via ~nown byte multiplexor channel 12 circuits 21 to a primary host which has the general 13 responsibility for supervising operation of the entire 14 data processing installation, as well as to selected other host computer~. Such computers can be constructed 16 in accordance with Patent 3,400,372. Typical byte 17 multiplexor circuits 21 are those used by the Interna-18 tional 3usiness Machine~ Corporation in their IBM
19 360/370 so-called "channel" circuits. Circuits 20 receive channel commands from the primary host and 21 the other hosts via circuits 21. Circu~ts 20 include 22 microprograms beyond the ~cope of the present descrip-23 tion for interpreting those commands for executing 24 memory functions such as those executed by the Inter-national Business Machines Corporation Model 3850 26 Mass Storage System. Additionally, control 17 includes 27 a plurality of channel circuits 22 for communicating 28 gtorage orders to dlrectors 16 and MSF 10. These , ~0975026 -28-.
- ., ~ - ~ , , . ~ ' , 1 circuits 22 are constructed as the channel circuit 2 in Patent 3,400,372, whereas the directors 16 and 3 MSF 10 appear as control units to those channel cir-4 cuits. Communication between the computer circuits 20, directors 16, and MSF 10 is in accordance with 6 Patent 3,400,372. Additionally, computer circuits 7 20 include memory circuits for storage control signals 8 incidental to the operation of the MSS. These include 9 a plurality of addressable registers CHR, LRU, UCB, LTR, MVT, W IT, W A-W , VVM, and QCB. In a most pre-11 ferred embodiment, these control registers form a 12 part of a random açcess storage device. Control 13 registers CHR contain control signals for cylinder 14 head records yielding upward level, or DASD, physical locations of control signals usa~le by computer circuits 16 20 in operating MSS. Each register con~ains four 17 bytes for identifying the CCHH address, as is well 18 known in DASD storage technology. The first two bytes 19 can be a vi~tual address, while the second two bytes can be a real address. Information contained in these 21 registers respectively i9 set forth in the table below.
22 Register No. Content 23 01-14 MVT Data Control 24 14-18 Staging Drive Group ID's 18-lB Mounted Volume Names 26 lC-lF Scratch Cartridge List 27 20-23 Virtual Volume Inventory 28 24-27 Transient Volume List .; ~

., ' .

- .. , , :
: . , . . - - : . .~ ~

1~73553 l 28-2B Cross-Reference W A-VOLID
2 2C-2F Configuration Data 3 30-33 Page Data Stable 4 34-37 MSF l0 Cell Map 6 3C-3F Journal 7 40-43 Schedule Queue 8 44-47 Diagnostic Data 9 48-4B Me 5 sage Buffer The LRU control registers contain the least ll recently used data information in connection with 12 the automatic destaging from DASD spindles 14 to MSF
13 l0 for that data which has not been recently used.
14 These registers identify those data locations most eligible, i.e., oldest, for automatic data destaging 16 to make room for data signals to be staged from MSF l0.
17 UCB control registers contain unit control 18 blocks for controlling operations of the DASD and l9 MSF l0 units beyond the scope of the present description.
The LTR contro} registerc are a cross-reference 21 table identifying real registers in DASD with logically 22 named registers from the host in accordance with the 23 sequence of logical names. Hence, computer circuits 24 201 by accessing control registers LTR based upon a logical name, can quickly identify the actual physical 26 location of the data indicated by the logical name.
27 Control reqisters MVT contain a mounted 28 volume table, which is a list of all so-called "DASD
.

,,. :
E~09750~F -30-' . . . ~ ~ ~ , . , i - .

~o735s3 1 data set ~olumes" having activity with respect to the 2 DASD spindles 14. E~en though host computers may 3 indicate that such volumes can be removed to MSF 10 4 by destaging signals, such volumes are continued to be identified in control register MVT until all DASD
6 space of spindle 14 has been allocated to other volumes.
7 The informational content of control registers MVT
8 is not pertinent to a practice of the invention, hence, 9 will not be further described.
Control registers W IT contain a volume 11 inventory table which has identifying and locational 12 information of all data signals stored in MSF 10.
13 Each VOLID (volume identification) includes the attributes 14 of the volume and the physical location of the MSF
assigned to each volume. Also, since this is incidental 16 to a practice of the invention, it is not further ~17 described.
18 W A-W control register is a cross-reference 19 index between a reference number W A assigned to a virtual volume within ~SS and the longer VOLID. In 21 other words, W A is a shorthand notation for identi-22 fying VOLID when any VOLID data signals have been 23 staged to DASD. As an example, W A is one byte long 24 and VOLID i9 six bytes long.
VDM control registers contain a virtual 26 ' device map relating real devices to virtual names 27 for devices. That is, each host computer has a virtual 28 unit address for a virtual device, i.e., a logical name.

.i ~073553 :

1 There i5 not a one-to-one mapping of virtual devices 2 to real physical de~ices; i.e., one virtual device 3 may actually physically consist of several real devices, 4 a part of each real device being assigned to the logi-cally named virtual device. Hence, control registers 6 VDM are an index for relating the logical name units 7 of the host to the actual physical unit of MSS.
8 Control registers QCB contain a so-called 9 "queue control block". These registers are four-byte registers identified in the table below entitled 11 "QCB Control Registers".
12 QCB Control Registers 13Register 0 1 2 3 14 Block Error BFl BF2 Flags Pointer Level 1 Level 2 1 DIR-l DIR-2 W Al W A2 16 2 QCB0 QCBl QCB2 OCB3 6 Trace 2 Trace 2 Trace 3 Trace 3 21 7ST SAVE BR SAVE JCP ___ 22 8 Error Error Error Error 26 Since the QCB control registers contain 27 information relatable to the present invention, they 28 are described in more detail. The byte 0 block flags ~073553 1 of register 00 are decoded as follows. Bits 0 and 2 1, if 00, indicate annulity; 01 indicates ready for 3 delete or reread 01 indicates valid, but the queue 4 is not started; and 11 indicates valid and active in active stage queue (ASQ). That is, staging opera-6 tions have been requested; and a separate queue (not 7 shown) in directors 16 has listed the action from 8 the QCB for staging.
9 Bit 2, when 1, indicates it is a nonstaging adapter type of operation. That is, signals are being 11 exchanged directly between the host 19 and a DASD
12 controller 15.
13 Bit 3, when 1, indicates two directors 16 14 are connected to a DASD spindle.
Bit 4, being active, indicates a DRD 12 16 allocation is not required for work in connection 17 with the control block.
18 Bit 4, when 1, indicates a DRD 12 is assigned 19 via the second director 16.
Bits 6 and 7 are reserved for alternate 21 path retries beyond the scope of the present descrip-22 tion.
23 ~ Byte 1 of register 0 contains the register 24 number of a so-called "mini header" currently in use or active.
26 Block flag 1 (BFl) fox level 1, found in 27 byte 2 of register 0, is interpreted as follows. Bit 28 O~as used herein must be a 0; when it is a 1, it is , .
~0975026 -33-.

1~735S3 1 used for purposes beyond the scope of the present 2 description. Bit 1 being a 1 indicates an error con-3 dition in addressing MSF 10. Bit 2 being a 1 indicates 4 that the message buffer overflowed with respect to this control block. Bit 3 being a 1 indicates that 6 data si~nals indicated by this control block are in 7 error and waiting for job assignment by the scheduler 8 of computer circuits 20. The schedule of computer 9 circuits 20 is an operating supervisor which assigns tasks to various programs for execution, as is wel 11 known in the data processing arts. Bit 5 being a 12 1 inhibits reading the cartridge table of contents 13 (CTOC) such that it cannot be transferred to a host 14 computer. Bits 6 and 7 identify portions of MSF 10.
Block flag 2 (BF2) for level 2 is found 16 in byte 3 of register 0. Bit 0 being a 1 indicates 17 that CTOC has been read. CTOC is from the storage i8 articles of MSF 10. Bits 1 and 2 are not used. Bits 19 3-7 identify the group of DASD spindles 14 associatable with the control block. Such identification can be 21 either in the virtual, real, or physical mode. In 22 register 1, bytes 0 and 1 contain the numerical identi-23 fication of directors 16 (DIR-l and DIR-2). That 24 can be used to access a DRD 12 which has been allocated for processing data signals in connection with the 26 control block.
27 Bytes 2 and 3 of register 1, W Al and W A2, 28 contain the virtual volum~ addresses used by directors 16 B~975026 ~34~

- . . :

1073~S3 1 for accessing data in connection with this control 2 block. It is remembered that ~he WA is a shorthand 3 notation for identifying a VOLID.
4 Registers 2 and 3 contain QCB's 0-B which have the signal contents of the queue control block 6 associatable with a particular schedule queue block.
7 Byte DRD in byte 2 of register 3 is the 8 address or identification of the DRD 12 which has ~ been allocated for use in connection with this queue block.
11 Register 3, byte 3, contains QCBP which 12 is a pointer or an address indicating a word in the 13 QCB registers associatable with this control block.
14 Registers 4 and 5 contain VOLID, the name of a volume associated with the control block. Hence, 16 it i~ seen that the QCB control registers also contain 17 a translation from the W A to the VOLID.
18 Also in register 5, in bytes 2 and 3, the 19 LXYZ storage cell addresses in the MSF 10 for locating magnetic storage articles containing data signals 21 associatable with this control block.
22 Register 6 has performance data in connection 23 with allocation of a DRD 12 and the deallocation of 24 DRD as the CTOC has been read from the storage article of MSF 10. ST and BR SAVE, found in register 7, indicate 26 that the contents of the ST and BR registers of computer 27 circuits 20, as later described, have been imaged 28 in this register for use when the control block is .; .
BO975026 ~35~

.

1 referenced. Register 8 and subsequent registers contain 2 error pointing data.
3 The mini headers (sets of control signals) 4 also stored in the QCB control register contain a sequence byte indicating the status of the sequence. ', 6 This includes bit O indicating verification of an MSF
7 10 addressing, bit 1 indicating that a stage/destage 8 operation is requested, bit 2 indicating check point, 9 bit 3 is unused, bit 4 updates a first director 16 table, bit 5 indicates update of a second director 11 16 table (two directors 16 can access a given DRD
12 - and a given DASD spindle 14), and bits 6 and 7 are ', 13 unused. As the functions are performed, the bits 14 are turned to 0.
A second byte in the mini header is a so-16 called "mini flag byte" wherein bits 0 and 1 are combined 17 to indicate a destage for 00, a nonstaging operation 18 for 01, a,staging operation for 10, and a nonstaging 19 operation for 11. Bit 3 indicates that a second mini header pointer i8 valid. Bit 4 indicates a no-operation, 21 while bit 5 indicates an IORC type of operation. Bit 22 6 indicate~ that an IORB is associated with a second 23 type of director 16. An additional byte is provided 24 to'point to the next mini header. It is a low-order address byte with the schedule sector queue being 26 ,the high-order address byte. The last mini header 27 of the QCB ha9 00. Trace T2 byte indicates the time 28 the particular mini header was inserted into the Bo975026 -36-- : , . . : . .: .. ..... . : . .

~073553 1 schedule queue block. Trace T4 byte is the time that 2 a stage or destage operation was initiated and is 3 associatable with the particular mini header. Trace 4 T5 indicates the time of completion for such stage/
destage. Hence, the three trace bytes indicate per-6 formance. The order ID byte indicates the operation 7 to be performed, i.e., stage/destage, etc. Another 8 byte indicates the hosts associatable with the par-9 ticular function to be performed in the order ID byte.
Additionally, there is a so-called "directors 16 flag 11 byte". Bits 0 and 1, when equal to 00, indicate that 12 a staging operation to cylinders CC is to be performed.
13 Bits 2-4 are all O's. Bit 5 indicates an error has 14 occurred on the stag~ng operation; i.e., the DASD
data is in error. Bit 6 equalling 1 indicates a stage 16 to real DASD cylinders CC and that the page status 17 table PST in directors 16 must be updated. This bit 18 is on whenever a 1 DASD spindle 14 has its signal 19 content transferred to another DASD spindle 14. Bit 7 is a 0. When bits 0 and 1 are a 10, a destage operation 21 from cylinders CC is indicated. Again, bits 2-4 are 22 0. Bit 5 indicates an error on destaging (pertinent 23 to the present invention), and bit 6 being a 1 indicates 24 a destage from a DASD spindle to a storage article in MSF 10. There is also a staging table access flag 26 byte as opposed to the previously described move data 27 flag byte. When bits 0 and 1 of the table access 28 flag byte are 00, the directors 16 PST has cylinder j , .
BO975026 ~37~

~07355~ ~

1 valid bits set active with the PST being updated with 2 the W A and VP. Bit 2 being equal to 1 indicates 3 an unsuppressible device end pending for the VUA (vir- I
4 tual unit address) after cylinder processing. The function of this bit is for other than operation with 6 the present invention. Bit 3 being equal to one in-7 dicates that the WA has to be updated. Bit 4 being 8 equal to one indlcates that suppressible device end 9 pending must be set for a given VUA after CC process-ing. This is a so-called "virtual pack change"; l.e., 11 the host CPU's indicate that the W A is being changed 12 even though the physical location of the data in MSS
13 is unchanged. Bit 5 being equal to 1 means to reset 14 the message buffer. Bit 6 is a 0. Bit 7 indicates that an unsuppressible device end, plus a unit check, 16 is pending for a W A. Unit check indicates an equip-17 ment error. If bits 0 and 1 are a 01, then bit 2 18 indicates that the W A is reserved to storage control 19 17 in connection with a W IT operation. Bit 3 releases the bit 2 indicated reserve. Bit 4 indicates that 21 the W IT must be updated for a given WA. Bit 5 indicates 22 the PST must be updated in connection with an operation 23 not pertinent to the present invention. Bits 6 and 24 7 are similar. When bits 0 and 1 are a 10, this indicates ~. :
that the PST of directors 16 must turn off the cylinder-26 written bits; that is, the PST indicates data has 27 been altered in DASD'requiring destaging operations.
28 With the cylinder-written bits turned off, such DASD

r~

.

: ' ' - ~ ~ . - .
. -. .

1~73553 1 data will not be destaged since a duplicat~ copy is 2 contained in a storage article of MSF 10. Bits 2 3 and 3 are 0. Bit 4 indicates that the DASD CC address 4 must be erased. Bits 5-7 are not used. If bits 0 and 1 are a 11, then bit 2 indicates that a pack change 6 interrupt should be sent to the host CPU for a given 7 W A and that the W A has a one-for-one correspondence 8 with a real physical spindle. Bit 3 resets the PST
9 removing the one-for-one correspondence of the W A
to a real unit. Bit 4 indicates that the LRU values 11 of the LRU control registers have to be replaced.
12 Bit 5 indicates that the staging drive group values 13 for DRC 13 have to be updated. Bit 6 indicates a 14 message of bytes 1-4 of the LTR control registers has to be transferred. Bit 7 indicates that the table 16 of content of LTR must be sent to a program accessible 17 memory portion of computer circuits 20. A count byte 18 indicates the number of pages in the page list asso-19 ciated with the particular mini header. The VUA byte indicates the virtual unit address of the host used 21 in connection with operations with the mini header.
22 The WA contains the W A associated with the data.
23 In addition, the QCB registers may contain other informa-24 tion not associated with the operation of a storage director 16 having a staging adapter.
26 The computer circuits 20 are generally shown 27 in FIGURE 4 of Waddell Patent 3,716,838, as well as 28 in FIGURE 3 herein~ Directors 16 also use identical BO975026 -39~

~073553 1 computer circuits 20 and additionally have a serializer/
2 /deserializer or S~RDES 30 which receives parallel 3 data signals from computer circuits 20, supplies serial 4 signals to DASD controller 15, and receives serial signals from controller 15 to convert same to parallel 6 signals for transmittal to computer circuits 20.
7 Modulation/demodulation of DASD data is accomplished 8 in SERDES in accordance with known techniques and 9 as used on the Model 3830-II Director manufactured by International Business Machines Corporation. Com-11 puter circuits 20 contain a set of instruction steps 12 in program store 31. Because computer circuits 20 13 are microprogrammed, a plurality of instruction word 14 formats are employed. Each instruction has four bytes, 15 with the fields being identified in the table below.
16 Instruction Word Formats 17 Type Byte 0 Byte 1 Byte 2 Byte 3 18 A CK (CA, CV) OP, CB/CD FMT CX CH CL
19 B CD, CV OP, CB FMT CX CH CL
C CW, CV OP, CB FMT CX CH CL
21 D CK/CA, CS OP**, CBjCD FMT CX CH CL
22 E CA, CS OP**, CB/CD FMT ~
23 F OP, CS --**, OP FMT CX CH CL
24 , la CK, CS ML**, CB/CD FMT CX CH CL
lb CK, CS ML**, CB/CD FMT CX CH CL
26 lc CR, CS ML**, CB/CD FMT CX CH CL
27 2a MK/CK, CS ML**, CB FMT CX CH CL

.

., . . . . I
- . -:. - -~, ,-- , : . :- . ~ .
- - :., -. ~ , . - . : , ~73553 ;

1 2b MH/CK, CS ML**, CB FMT CX CH CL
2 3 NH/NK, CS ML**, NL FMT CX CH CL
3 Notes: * Gated to A bus bits 4-7.
. 4 ** Format bit ME in lefthand digit.
OP Is always three bits, remainder fields are 6 five bits.
7 Format Coding in Instruction Words 8 Type ME Bit Coding in Format 10 . A -- 1 X * *
11 . B -- 1 0 13 D 0 0 X * *

F 0 0 1 1 . 1 16 la 1 0 1 0 0 ¦
17 lb 1- 0 1 0 18 lc 1 0 0 0 0 19 2a 1 0 1 1 0 2b 1 0 22Notes: X = 1 or 0 23 * = ND, NB ~ 11 1 24 Each of the fields identified in the table above are micro orders interpreted as set forth in 26 the tables below, respectively identified by the field 27 name. In the tables below, the edge character next 28 to the micro order indicates a vertical po~ition in . - . . .: . .. . . - ~- , .. ~. ,.

10~35S3 1 the FIGU æ 5 illustrated instruction word tabulation 2 identifying the respecti~e fields.
Field CS Micro Orders Miclo Order Edge Function DNST21 C Set ST rcg. bi1 2 if the D-bus i5 non-zero 0-ST0 C Reset ST r~g. bit 0 0-STI C Rese~ ST ,eg i~i~ 1 0-572 C Reiet ST reg. bit 2--0-ST3C C Qeset ST reg. bit 3 0-ST4 C Reso~ ST rcg. bit 4 0-ST5 C Reset ST reg. oit 5 0--ST6 C Reset ST reg. bit o --ST7 C Rr~let ST rey. bit 7 1--STû C Sct ST reg. bit 0 to 1 1--STI C Set ST reg. bit 1 to 1 I--ST3C C Se~ ST reg bi~ 3 to l This bit hkes valu~ o~
~arry-out of ALU ;f C h odded to D-bus stoten ont 1-STS C Se~ ST rog b;~ 5 to 1 I--ST6 C Set ST reg ~;t o to ) ¦
1--S 7 _ _ C Se~ ST !e8 o;~ _ t ~

Field CA Micro Orders !

GA 1001 ~ GP reg. w;th oufgotel to the A-bu~
GB 10000 A GP reg. w;th OutJoteS 1O ~he A-bus GC 0001 A GP rog wlth OUtgotel to A-bul MA 1111 A GP reg. usuolly u~ed ~ dovtr~e~reùd/wr;te dota MB 1110 A GP reg. u~uolly u~ed for di~placennent r-glster--MC Ol l i A GP re~. ulually wed for ECC Eow count MD 0110 A GP reg wlth outgotes to 1he A-i~u~ usuolly usod ~or chonnol 8u~-ln NA 1011 A GP reg. usually used for CU oddr-~ ond channel cond;tior~ls NB 1010 A GP reg. wlth outgotes to th- A-bus NC 0011 A GP reg.
ND 0010 A GP reg. outgote~ to tho A-buJ used ~or CUDI bus ond tog TA 1101 A GP reg. u~uolly used for CUDI bus-out b;ts T9 110 A GP r-~. u~uolly us~d for wr;te control ond ECC
control and CUDI gotes TC 0101 A GP ro~. u~uolly tn d ~or SERDES control~ ~nd channel controh TD 0100 A GP reg. u~wlly used for CUDI tags ond check _ Indlcailonr EiO9?5026 -42-Field CB and CD Micro Orders I

_ ~ 1. , t~icro Order Edgr) Eunction _ ' .
MA 1011 A GP reg. with outgotes to tho B-bus; ingotes from the D-bus --B 1000 A GP r~g. w9h outgole~ ~o ~he B-bu~; ingo~es from the D-bus ~C 1111 A GP reg. wirh outgo~s ~o ~ho a-bus; ingrJtes trom the D-bus I~AD 1100 A GP r~g. wi~h outgotes to ~he B-bus; insotes from the D-bus NA 0111 A (`.P reg. with outgr~es to ~he B-bus; ingotes From the D-bus -NB 10110 A GP reg. w;~h outga~es ~o the B-bus; ingo~es from ~h~ D-bu~
NC 10101 A GP reg. with outgot~s to the B-bus; 7nga~e~ from ~he D-bus ND 10100 A GP reg. with outgo~es to the B-bus; Ingates from ~he D-bus SAOOOO A GP rr~g. wilh outgote~ ~o ~he B-bus; Ingotes from ~h~ D-bus - --SB 0001 A GP reg. with outgohs to the B-bus; ingr~es from th- D-bus 5COO10 A G. reg. wlth out~ates to the B-bus; Ingot-s from th- D-bus.
SD 0011 A GP r-g. wlth outgotes to tho B-bus; ;ngahs from ~ho D-bu~
ST 10001 A GP rea. usually used for StO~UI indlcotions TA1001 A GPreg. with outgotes to the B-bus; ingotes from ~he D-bus TûOI10 A GP reo. w;th outootes to the fi-bus; Ingotes Irom ~h~ D-bu~ --TC 10111 A GPreg. wllh outgat-s to the B-bus; Inptes from the D-ous TD 1010 A GP rea. wlth ou~go~es to ~ho B-bus; ;ngo~os from the D-buJ
rG I OOt I A GP re~;. wlth outgol-~ to the B-bu~; inoot-s from ~h- D-bus O 10000 A Fore~s oll-O s ~o B -ah~e~
75R 111~ ~ ~i~lil~oohs to the B-bus~ Inoahs trom the D-bu-GA 0101 A GP re~;. wlth outgote~ to Jhe i3-bus~ Ingo~n from the D-bus GB 0100 A GP r-g. with outoot-s to the li-bu~; Inoates from ~h- D-bu~
GC 1101 A GP r-~ wlth outgd s to Ihe B-bus; Inootes from ~he D-bu~
GD l(iO10 A GP reg. wlth outgot-s to th- B-bus; Ingot-~ from ~ D-bus ___ ' ' ~, ' .

.
~ ~i BO97.5026 . -43- . , .

~C~73553 Field CH Micro Orders Micro Order Edge Function ADDRO B Sel iAR bl~ i2 to i if Address oul deleclei BOPAR/ B Sol IAR bi~ 12 ~ot if Bus oul Porily checkor IN-Lin I LACT Act;ve BRO B Set IAR bil 12 1o l if BR reg. bi~ O is o on BR2 ¦ B S ~ IAR bit 12 to l If BR reg. bit 2 is r~ one BR4 1 B Sel IAR bit 121o l it BR reg. bil 4 is o ono flR6 B Sol IAR bil 12 1o l if BR reg. bi~ o is o one CARRY B Set IAR bit 12 lo I if Ihere wos o corry out of ALU in previous microblock CHK-2 B Set IAR bil 121o l if Ch-ck 2 error detected.
INLIN B During INI INE modo aR-4 reploced by INLIN
oflor SPEC Of 07 CO~O B Set 1AR bit 12 lo 1 if op-7n ond commond-out ore ocHve EXT bch cond 12 -SECTR B During IMPL liR2 Replaced by SECTR ~Seclor) SUPPO B S-t IAR ol~ 12 h I if scl-cled suppress-oul dolecl-d STO B Set IAR b;t 121o l if ST rtg. b;t O ;s o one--ST2 B S t IAR b;t 12 to 1 ;f ST reg. bit 2 is o ono ST~ -- B Sel IAR bil 12 1o l if ST reg. bit 4 is o ono--ST6 B Sol IAR bit 1210 l if ST reg. biJ o is o ono O OOvO-- B Sel IAR bit 12 lo zero I 0001 B Sot IAR blt 1 2 to I

Field CL Micro Orders Micro C~rder Edge Function 1, B Sol IAR blt 13 to i if BR rt~. bil ï Is I
BRI B Sol IAR bil 13 to 1 if BR rog. bil 3 is IBR3 B Sot IAR bi~ 13 to l If BR reg. blt 5 is IBR5 BR7 B SeJ IAR bil 13 1o 1 if BR reg. bil 7 is I
BTRDY B During IMPL bR3 reploced by BTRDY (byle Reody~
BFRDY B So~ IAR bil 13 to l If Conlrol Unil End lolch Is on CUEND-- or Buffor full ts doloclod CHANB B Sol IAR blJ 13 to I ;f chonn-l B sol-cJod D~O B Sol IAR bll 13 ~o 1 ;f ~h D-bu~ Is eqval ~o 20ro--HLTIO/ B Sot IAR bit 13 to I if Holl 1/0 lotch or ~ron~for XFER lotch is sol _ ILXEQ B Dur;ng INLINES BRS roploced by ILXEQ (Inllne oxo uto swllch lolch1 ~
INDEX-- B Sol IAR bll 13 1o l ;~ STI~I ond INDEX is ~STI ) d-loclod SELTD B Sol IAR bil 13 ~o I If CU s-loc~-d SERVO/ B Sel IAR blt 13 tol If Serv1c--ruJ or roulti-1ol1 swiJch MULTI ~ (l~vo-charmol s~;lch hoturo) orc octivo ST3 B i Sot IAR bit 13 lo l if ST ~o~. bit 3 is o on-ST5 B ; 5-1 IAR bil 13 1o 1 if ST -9. bi~ 5 is o ono--ST7 B Sot IAR bil 13 1o I if ST reg. biJ 7 ;s o ono O B . Sot IAR biJ 13 ~o 2-ro ~: : _ B SotJAR blt 13 Jo ono , , .
.. ', BO975026 . -44-.
.,' . . . ~ , : ' . t .

' . . . ' " ': . ' . .. ' . ' . ' - . ' . . ~ ' ' ' ' , .. . . " ' ' . ' ' . ' ~07~553 1 The bit assignments of the micro-instruction 2 words for the various formats ar~ set forth in the 3 ~ tabLes bel- .

' BO975026 , -45-.

- . .. ~. ' ' - , .- ~

¦ O--N ~ 'tl ' ~ U C~
r _ ~ ~ _ ~;; ~ ,,Z za~ ' ID U U U U U U c n: O _ ~ C U ~ Itl ~ vl U tl! c ~ ~ U U U

D¦u¦u¦u¦u¦T ¦~ 5F ~ F ~ --r _ _ _ _ o c~ F F .

.¦J _ ~, ~, _ _ ~ ~
i~! ~ ~ o o o z o ~ I ~q J ~-E ~I!q ~ J~a o o _ o o Z
_ c _ ~, o ~lq JO E-O ~I!q ~U!J~a ,

5------- i ~ ~ v .~ ~ u ~ _ _ __.

.~ 1~ C~ ~ . ~> ~;~X C~D ~ ~2D~
m . ~ 1~ ~ ~ ~ ~
~ ~ ~0 0~' Zzvz ~ Z'Z ~ 1, ~ r _ 2 2 u juuo---_ _ _ _ _ _ ~ ~t ~t < < ~ ~
0~ ~ ~ ~ r~
~ _ _ _ _ . _ _ _ _ ~ ~ ~ z' o uz ~ ~Sz z e5a ~ ,~
~ O_N ~`01~ 0`~ Q(UIL ¦ ' :~y .~' ' ' ~ ' ' .

. '' ' ' 1~735S3 o ~ 'D ~ 6 ~,lu c, r r ~ ; _ ~ ?U ~ ~ z ~1 1~ C ~ U ~ r7 ~1~ _ ~ uJ > 0 D U ~ T ~D T

r T C = -- 9 ~ r Y ~ U t ~ r r Q
_ _ ~ _ o--u 5 v~ 9 N U ~ ~A Q _ _ _ e _ ,~ x ~i , , ~cO~c~,c r~ _ al ~q r~ o Z
~; F ~
~ ~ t sg o~oc ~lu ~zz~
.~ ~ ~ ~+~+

~ 1~i u~ .

~ u -b ~ f ~ X ~ 5 c _ ' lo ii~! ZZz~ yjc~zlzu 1~
rl --N r ~ ~ ~ -~ ~ ~ . u t, u~ ~, .

~073553 ¦ o r~ ~ ~ `10 ~ ~ , w ~

~ Idl e_~ O _ 0~ ¦ z ~ ~
r _ ~ . r~ ~Ooe~ ,~

a 0 ~ ~a ~Y o. ,S~ ~ C ~ è
~ o C~ e~ c c e ~ ~ ~ ~ ~

m ~r - ~ O~ qz~2yp~i~zz~ yc , .
e I ¦~ ~Q~Q~
E .. _ 'C<~t" . '' .
~ - -`1;

~ arla~ uOu~u~
~I

I' ' " ' ~ ' - . : - .. . `

~LO73553 O N 'ql r ~ O r~l(D O t tDIU O U~ U
_ ~ Z

r r r ~ r r~ o~ C~U~Q ~

to . D ' ~ u<~ U ;? C.C,~,Cn,~CP
_ _ l., _ _ _ ~! ,o ~ D ~ ~, D 'r _ ~ o O o O O O
' i~ ~ 1 - - Z
,~ ~ O O _ O O _ I~qJ~ q~ J~a IU _ - o c _ _ _. Y O J~q J E-O Jl!q '~U!Jaa ~ o o o lo o o ~ ~ Jo ~9 ~ JOU ~ OJ JyJ ~9!~ dS
~d _ _ _ . .. _ _ _ _ _ w ~ S _ 9 ~ 9 l o t~ v~ X 5 c~ ~ tD Z jlt ~ ~ 1~ ~D ~

t r _ _ _ _ _ _ ~ _ Ou~ ---. 9 ~ 9 _ c 8 3 c 1> ~ u ~J O _ _ ~ ~Pt~t c _ _ ul u _ ~ 1~ z ~1 Y ~ ~ ~ z z 5 r~ 0 ~ ~4 O~-~ .

- - - -` `:
.

1~735~3 , ~ Ul o . , ' .
_ ~D ~ . .

L
o~ ~v v~ ~v v~
_ i~ l~t ~ ~ t l ~ t t t t r ~ 4 o~2Y e~ e~2~ e5 !~ o - r~ ~ ~ .o ~ < a , , . . : , - ~, , :

.

¦ O_N~ aU~

c ~¦ o u~ ~r>

u . x ~, æ~ ~
~o g., , Z

8 r _ m _ a) .~ O ~, .~ .. . ~:~3 ~ 1~ 3~ I t~ ooo~llO
7rlo-~

.

BO975026 ' -51-, ~
. . .: . , ~Q735S~

_ u u u ~ < ~ O e z ~ 5 ~ ~ t 5 -~; ~ ~ -' - u ~ z~l~ "~ , 7~7 "
O IJ _ _ _ ~ U U
C

.. .j . , .. .. .. . ... .. ..

~07~553 ~ ~ Y~ _ uy,~ ~:~a ~ z z y "
~ _ _ ~ ~, O O

l u ¦r ¦~ o ~ ~ sls s s ~ r O C~
~o _ C~ D ~ ~ ~ .

~! o a .
O r~ _ _ z ; C
v ~S

r _ u ~ t~ ~ E ~ ~ ~E`
~ ~, ~. ~ ttt~ t~t~ t~t c ~ _ t _~I_ ____ ~ooo oooo :~ r~ _ _ o _ ~ r~ o r~ e~ < m~

.e .
~. ' ' ' ~
~ BO975026 ~53~

, .. . . .. . . . . . . . .

¦ O--N t~ u ~ lU ~ ¦

~J~ C
ooo o ~Q~ o o O "~ Z
;~ ~
_ ~ ~ ~ .
. ~ I ~t~z~o~77tb~ u~7iPi O _ e I 1 1 Z , j! ~ .
_ rlo_N~

d' - .. . . . , -, 1(~735S3 1 In addition to the above-listed operations, 2 directors 16 and controls 17 ha~e special operations;
3 that is, the same micro order performs different 4 functions in the various machines in accordance with hardware connections which use register assignments

6 also defined below. In the two tables below, the

7 special operations used in connection with the FIGURE

8 1 illustrated apparatus are set forth.

.: . . ~ . . . , - .

~0735S3 ~li ~1 ~
'`I ~ ~ ~

_ o ~ ¦ r c _ _ _ ~ c ~ --1~`1~1 o~l ol ~

, . ~. . . ~ . , . . - - ~

1~73553 1 The special operations for directors 16 is 2 similar. A main difference is the special operations 3 for controlling the staging adapter.
4- Each instruction word has an operation code OP in connection with the arithmetic logic unit ALU 32.
6 The ALU operations are set forth in the table below.

ALU Operations Mnemonic OP Micro Order Edge Function An~--D 0000 A A-r~gister OR~d wi~h B-regisler ond tl1~ r~sulls ploced on D-bus A-B--D ; 0001 A A-register ANDed wi~h B-regis~er ond Ihe r~sulh placcd on D-bus A~-D 0010 A A-reg. EXCLUSIVE ORed wi~h Ihe B-reg snd ~he res~ s ploced or\ ~h~ D-bus A~9--D 0011 A A-reg. ADDed ~o Ihe ~-reg. ~nd ~ht rosults ploced on ~he D-bus A+3-C--DC 0100 A A-rtg. ~nd B-reg. ond presen~ condl~ion of Carry-ln (ST3C) ADDod A-~+C-DC 0101 A ADD Complem~n~ Corry Ai B--DC 0110 A A ~md ~ r~gi~lers or~ odded ond result~ pl~ced on D-bus. S~l ST r~g. bil 3 lo 1 if cony occurrod.
A-BI 1--D : 0111 A ADD ~ comDlem~n~ (Su~roc~) 7 Addltionally, certain registers have greater 8 significance in instructions words than others. These

9 include the data address register DAR IAL, the lower byte of the instruction address register used to drive 11 program store; and IAR, the instruction address register.
12 ~ Returning now to FIGURE 3, the program store 13 31 supplies the instruction word to an instruction 14 holding register 35, the outputs of which drive decode circuit 36 to supply a set of micro orders to all 16 the units. Such decoder is constructed as is well 17 known in the arts and serves to sequence operation 18 of all ~f the units. A portion of the instruction word BOg75026 -57-~t 073553 1 is fed back to IAR in the W and X registers for program 2 branching and accessing the next instruction word 3 from program store 31, as will become apparent.
4 Computing circuits 20 operations all center about ALU 32. It has two inputs, the A register and 6 the B register. These two registers receive signals -7 from funnels or assembly circuits AASM and BASM, res-8 pectively. The assembly circuits merely take the 9 signals from a plurality of signal buses and gate same to the A and B registers under control of decode 11 36. Two of the input buses to the AASM and BASM are ~12 the A bus 36, which transfers signals from a set of 13 microprogram registers 37 to the A register, and the 14 B bus 38 which transfers signals to the B register from selected ones of the microprogram registers 37 16 and from the instruction word holding register 35.
17 An important aspect of microprogramming computer circuits 18 20 i9 the assignment and construction of microprogram 19 registers 37. A general arrangement of the micro-program registers is shown in the table below.
21 ` Microprogram Registers 37 ;-22 General Purpose Registers:
.

TA TB TC TD
26 M~ MB MC MD

.

.

~.

. . : , ~ . ~: . -.

~0735S3 1 Special Purpose Registers:
2 SA, SB, SC, SD: Inputs = Control store, D bus, 3 external 4 Outputs = Control store, B bus, external 6 BR tBranch Register): Input = D bus 7 Output = B bus, control 8 store address 9 decode ST tStatus Register): Input = D bus, condition 11 sense 12 Output = B bus, external 13 Not only must the assignment of the micro-14 program registers to the general classes be understood, but also the specific application of the registers 16 37 to execution of the microprogram sequences must 17 be known. As used in directors 16 and control 17, 18 the utilization of the microprogram registers 37 is 19 set forth in the table below. The term "control unit"
refers to the circuits associated with decode 36 which 21 includes an oscillator and sets of registers (not 22 shown) for controlling the sequencing of the microprogram, 23 all in accordance with known computer techonology 24 therefore not further described.

Register Assignments ..
Frorn RegisJer M aning To PoJiHon MA O SERDES
MA I
MMA 3 .
Control MA 4 Wri~o Do~a To Dovico Un7~ MA 5 MA P
MA O Cont~ol Uni~

D-vlce MMA 43 Reod Dota From Devico MMAA 'o MA P _ _ MB O P O Low no~ equal ~o P 3 or pority unequol.
MB I P O Low not oquol to P 3.
MB 2 P O Low not oguol h P 2 or pority unoquol.
MB 3 P O Low nor equol to P 2.
ECC MB 4 P O Low no~ equùl ~o P 1 or pority unequal. Con~rol Uni~
MB S P O Low not equul to P 1.
MB o P O Low no~ quol ~o O.
MB 7 P O Hi not ~quol to O.
MB P Genao~ed Porlty.
MDO POBitlô
MD I P O Bi~ 17 MD 2 P O Bit lo MD3 POiiit 15 ECC MD 4 P O Bit 14 Control Unit MD 5 P O B;t 13 .
MD6 POBit 12 MD 7 P O Bi~ 11 MD P Genera~ed Pori~y IAD O Bu~ In O
MD I Bu~ In I
MD 2 Bu~ In 2 MD 3 Bu~ In 3 Control MD 4 Bu~ In 4 Solectod Unlt MD 5 Bu~ In 5 Chonnel MD 6 Bus In 6 MD7 Busln7 MD P Bu~ in P
NA O Bu~ Out O
NA I Bu~ Out I
NA 2 Bu~ Out 2 S~lec~d i lA 3 Bu~ Ou~ 3 Control Unit Chonnol NNAA 5 BurJ Out 5 (Not Sp Op 14) NA o Bu~ Out o NA 7 Bus Out 7 NA P Bus Out P
Chonnel NA O Chon. A/B Buffor Pority Check Chonnol NA I Chonn-l A In~orfoce ChrJck Chonnol i ~A 2 Chonnol 3 In~rtoce Ch ck Channol NA 3 Da~a Tron~er Ch ck Corltrol Unit 5ERDES NA 4 SERDES~CUDI^ECC Chock (r;oted oy SERDES NA 5 PLO Chock Sp. Op. 14) NA 6 S e~or Counl-r Chock Channol NA 7 NA P Gonorat d Pity BO975û26 -60-CUDI ¦ ND 0-7 ¦ TD 01 = 00 ¦ Conlrol Un;t ND 0 CUDI Bus In 0 ND l CUDI Bus In I
ND 2 CUDI Bus In 2 ND 3 CUDI Bus In 3 ND 4 CUDI Bus In 4 ND S CUDI Bus In 5 ND 6 CUDI Bus In 6 ND 7 CUDI Bus In 7 ND P CUDI Bus In P
TD 01 = 01 ND 0 Driv~ Sel Failuro ND I Tog Involid ND 2 Device Check ND 3 CUDI Bus Oul Check ND 4 CUDI Bus In Check ND 5 CUDI Tog Bus Check ND 7 Not Used ND P Gen~raled Porlly TD 01 = 10 ND 0 CU Phy~iccl Addres ND l CU Physlccl Addross ND 2 Selechd Module 3/6 0 ND 3 S~lcc~ed Module 3/6 1 ND 4 Selected Modu~e 3/6 2 ND 5 Sclected Mcdule 3/o 3 ND 6 Sel~cted h~odule 3/o 4 ND 7 Selecled Modulo 3/6 5 ND P Generot~d Porlty SC 0 ECC Error - No input doto r~celved SC I ECC Error - P 0 or W rilo SC 2 ECC Error - P I or P 3 SC 3 ECC Error - P 2 ECC SC 4 Not Us~d Control Un t SC 5 Not Used (aated by SC 6 Not Used Sp Op 15) SC 7 No~ Used SC P Generaied Parily CUDI SD 0 CUDI Unsofe (us- TD decod- 01) SERDES SD l Wrlte Pori~y Chock SERDES SD 2 R-ad Parily Ch-ck SERDES SD 3 Blt Rlng Ch-ck Conlrol Unll SERDES SD 4 Wrlle Comp-nsation Ch-ck (gal-d by ECC SD S Error 2 In ECC (ECC Ch~ck) Sp Op 13) SERDES SD 6 VFO Ch-ck - rnl~J7ng PLO pul~e~
SERbES 5D 7 VFO Phas- Ch-ck }
SD P Gen ra~ed Porlly TA 0 CUDI Bus Out 0 TA I CUDI Bus Oul I
TA 2 CUDI Bus Ou~ 2 TA 3 CUDI 8u~ Out 3 Conlrol TA 4 CUDI Bus Out 4 Tag modifi-r~
Unl~ TA S CUDI Bu~ Oul S To CUDI
TA 6 CUDI Bus Oul 6 TA 7 CUDI Bu~ Out 7 __ TAP ~ ~O~P .

BO975026 -61- .

_ .~ .., . .. _.
T9 0 Module S loc~ Goh CUDI
TB 1 Tag Go~e CUDI
TE; 2 Enoble Tog Valid Check CUDI
C T9 3 Lonq Select CU
on~rol TB 4 Pro~ec! NA regis~er CU
Unit TB S ECC Con~rol A ECC
TB o ECC Conirol B ECC
TB 7 ECC Conirol C \ ECC
TB P ~_ TC 0 00 No~ Used ond _ 01 Chonnel Read Con~rol ~
TC 1 10 Channel Wrile Con~rol . 11 Freeze 7ransfer Con~rol TC 2 Las~ by~e request Channel ~ CC CONTROL DECODES
Unit TC 34 AdrJress in A ~ C Meaning TC 5 S~o~us In 0 0 0 Rese~
TC o Non-suppressible Reques~ In - Chonnel A 0 0 I Reod TC 7 Suppressible Reques~ In - Channel A 0 1 0 Wri~e Doh Oi-CUDi Check Condi~ions CUDI -- i 0 0 No~ Used TD l ¦ lOe3 Of 6 Cod and CU Physlcal CUDI j i 0 Noi Used TD 3 Address 1 1 1 Allow Decode- WriiG
Conlrol TD 4 Decode B Bil DRV
Unll TD 5 D code 4 Bi~ CUDI DRV
TD 6 Decode 2 Bt~ Tag DRV
TD 7 Decode I B;t Bu~ DRV
TD P Porlly for Tag Bu~
TG ol ~ 01=Lock VFO To f LO SERDES
and _ 10=Lock VFO To Dah SEflDES
TG I . I l Fo~i PLO Lock In SERDES
TG 2 Separa~ Da~o Sync (wrih odcirG~ rrlork DRV
when wrib goie on ) .
TG 3 (0) Disable CU end A - Enoble CU end B SERDES
Canlrol TG 3 (1) Enabl CU nd A - D;soblo DU onri B Chann l Unl~ TG 4 block SW h Channel A
TG 5 Eloci SW ~o Chonn l B Chann l TG 6 Nan-~uppros~ibl Requos~ in ChannGI B Chann l TG 7 Suppr ~ible Requ ~ in Chann l B Channcl TG P
_~
TG û Engage MPL File Hopd TG I Movc MPL One Track In TG 2 Mavo MPL One Track Ou~
CU TG 3 MPL Flle S~ori Reod MPL File (aaled T G 4 byMPL TGS
lalch) TTG 76 T G P

E~0975026 -62-.
-, . .

Sp Op O PGt~-SIOP Sh~ement Clock Sp Op 2 CHK-STOP Sto~en~ent Clock Sp Op 3 Error 2 R~set to User CHL-SERDES
Sp Op 4 Gote Re~d Error P~ttern ~o NB C~ri Sto~
Sp Op S Rese~ S~orage Error Regis~er C~l S~or Sp Op 6 Set Add~ Comp ~rom SA ~ SS CE Controls Sp Op 7 INLIN (In-iino) Br~nch in CE Mode Br ControlsSp Op 7 ILXEQ ~Do~a En~ry) Bronch in CE Modo Br. Con~roh Sp Op ô S~op IAPL Operotions MPL File Sp Op 9 Shrt MPL Operations MPL File Sp Op 10 Gc)te h~PL Drlta to D Bus MPL File Sp Op 13 Gote SERDES Errors ~o SD SERDES
Sp Op 14 Control Errors ~o NA Chonnel CU Sp Op 15 Gale ECC Errors ~o SC ECC
Sp Op 16 Shift P û (if ECC decod~ ~ Allow Decode Write, olso shifts P I P 2 ond P 3) SpOpl7 ShiftP1 P2 ~ndP3 Sp Op 18 Enoble BOPAR ~o CH 15 Bronch Sp Op 19 Set INLINE Activ~ Lotch CE -Sp Op 2û Reset INLINE Ac~ive Lotch CE
Sp Op 21 Unfreeie Ch~nnel Sw;~ch - Allow Dis~ble Chonnel Sp Op 22 Allow Disable A
Sp Op 23 All~w Dis~ble B
Sp Op 24 Lood Sec~or Counter ~o SA on~ SB
520P27 ~
Sp Op 31 _ ~

l Another factor is the accessing of the program 2 store, particularly through the address register IAR.
3 In the table below, the term "latched" means it is 4 held over from the previous instruction word, i.e., not changed. The program store has a two-byte instruc-6 tion address register IAR having the bit assignments 7 set forth in the table below.
8 Program Store Address Register Inputs 9 Byte 0 (Type Instruction D-3 Latched, Bit 0 = 0) Type Bits Input 12 1-3 Latched 13 4-7 Latched/CV

.
, . .
. . . . . . . . . .. . . . . .

~73553 2 1-4 Latched 7 Byte 1:

. 13 CL
11 14, 15 0 12 E All CB
13 The data address register, which accesses 14 the control registers, is set forth below.
Data Address Register (Control Registers~
16 Type BitsInput 17 la-lc 0-4 18 5_7 ML

2a-2b 0 21 1-4. MH

.

1 As set forth above, the microprogram regis-2 ters 37 supply signals to ALU 32 and also receive 3 signals from ALU 32 via D bus 40. Registers 37 also 4 supply signals in directors 16 to SERDES 30 and in control 17 to the channels 22. Hence, the data loop 6 in computer circuits 20 includes ALU 32, D hus 40, 7 microprogram registers 37, A and B buses 36 and 38, 8 and the A and B registers. It should be noted that 9 the inputs from program store 31 to ALU 32 is via the B bus. All communications in the computer circuits 11 flow through ALU 32. For example, D bus 40 also goes 12 to the IG register which then supplies signals to channel 13 circuits 41 which, in turn, are connected to onè of 14 the hosts 19. Cbannel circuits 41 are those channel lS circuits defined as a control unit in Patent 3,400,372, 16 supra, and are those used in connection with the Type 17 370 computer manufactured by International Business 18 Machines Corporation. Further examination of FIGURE
19 3, along with the charts above, will clearly show 2~ the data flow of computer circuits 20, it being understood 21 that this arrangement is essentially the same as that 22 shown and used by International Business Machines 23 Corporation in the 3803 Model II Director Unit.
24 Director 16 has what is termed a "staging adapter", that is, a buffer system which connects 26 DRD's and DRC's 13 to the DASD controller 15. Because 27 tape units operate at different rates than disk storage 28 spindles 14, a buffer memory 42 of the count-up/count-.

~073553 1 down type is interposed between the SERDES 30 which 2 is a connection to the DASD units, as previously 3 explained, and buffer memory 42. Buffer memory 42 4 includes independent control circuits known in the buffering art for automatically transferring signals 6 to the DRC 13 and for receiving signals from DRC 13 7 while ~imultaneously exchanging signals with DASD
8 controllers via SERDES 30. All of this operation 9 is independent of computer circuits 20 of director 16. It should be noted that computer circuits 20 are 11 involved in transfer of signals from DASD spindle 12 14 to the host computer 19 via the SERDES as was prac-13 ticed in the International Business Machines Corpora-14 tion manufactured 3803 Model II Director Unit. Buffer memory 42 has two parts which are alternately used.
16 A first part receives signals from DRC 13, while the 17 second part simultaneously supplies signals to SERDES
18 30. When the transfers are completed, the operations 19 are switched ~uch that the first ~art is then supplying signals to SERDES 30; while the second part receives 21 signals from DRC 13 for providing an uninterrupted 22 exchange o data signals. The same operation is pro-23 vided in moving data from the RASD spindle 14 to DRD 12.
24 Since such buffer memory operations are well known, they are no* further described.
26 Microprogram Instruction Representation 27 Referring to FIGURE 5, the tabular representa-28 tion of instruction words, as defined above, is shown in , . ,. .

. , .

:. ............ . ~. ~

, . .

1 diagrammatic form. The format is amenable to computer 2 printout for identifying the instruction layout in 3 program store 31. Generally, the left side is the 4 input side; and the right side is the output side.
That is, entry into an instruction execution is repre-6 sented at the left side. The movement to the next 7 instruction is represented at the right side. First 8 discussing the input side, the leg identifier means 9 entry from another instruction execution when certain branch conditions h~ve been satisfied. That is, if 11 there are a plurality of instructions which can be 12 executed from a four-way branch, then there will be 13 four blocks as shown in FIGURE 5, each with a diffe-14 rent leg identifier, respectively, indicating branch conditions required fox entry into the block. Note 16 the relationship to the instruction address register.
17 The second line refers to the K fields of the instruc-18 tion word. When the field is CK, the numerical signal 19 contents of that field are used directly by the instruc-tion. That is, this constant is supplied to the A
21 bus and may be decimal or binary. The third line 22 Of the instruction block ha~ the edge identifier A
23 referred to above in the ~arious tables. In ~, the 24 ALU statement, as defined above, is inserted. This includes the operatlon code with the exceptions noted 26 for the various types of instruction words. The fourth 27 line is identified by edge D. The fifth line, identi-28 fied by edge C, has the bit control for the status .

1~73553 1 register ST of registers 37. The sixth line is identi-2 fied by edge B and contains a so-called "high branch"
3 corresponding to the CH field of the instruction word.
4 The bottom line is merely a geographical representation S of where the box is located on the printout and tables, 6 as will become apparent in the micro code tables used 7 below.
8 Going to the upper side of the block symbol, 9 the high branch satisfied corresponds to the CH field and the low branch condition corresponds to the CL
11 field.
12 Going to the output side, or the righthand 13 side, in row 1, the hexaddress represents the content 14 of the instruction address register IAR used to access the instruction word represented by the tabulation.
16 In the second line, the mode indicates the format 17 code as set forth in the tables above. In the fourth 18 line, the data statement or DAR addressing is a con-19 tinuati~n of the edge D field. The data control in the next row indicates whether there should be a fetch 21 or store of data. In the sixth line, the righthand 22 side of the branch row contains the low branch which 23 i3 specified by the CL field. The F field is the 24 feature code; that is, certain particular functions may be added to a microprocessor with these additional 26 functions called "features-". In the bottom line is 27 the box serial number ~alphanumeric), plus a leg selec-28 tor (branch condition) for going to the next instruction ~, . . . - : : ~ .

1(~73SS3 1 word. The lines interconnect the graphical tabulations 2 for showing sequence flow of the microprogram. Further 3 on in the detailed description, this will be more 4 clearly pointed out.
Microprogram Flow 6 Only that portion of the microprogram in 7 control 17 that pertains to the function of the present 8 invention is described in detail. All of the other 9 micro code is resident in machines manufactured by International Business Machines Corporation known 11 as the 3803 Model III (control 17) and the 3803 Model 12 II with staging adapter feature for the director 16.
13 The microprocessor computer circuits 20 contain a 14 supervisory program called a known "scheduler" (as at 50 of FIGURE 5) which provides multitasking and 16 multiprocessing within the computer circuits.
17 The scheduler invokes a microprogram which la initiates a destage operation; i.e., orders are sent 19 from MSC 17 to director 16 to read signals from a specific set of tracks from a specific DASD spindle 21 14. MSC 17 also orders a DRC, DRD to receive signals 22 from the DASD spindle 14 to record same on tape. The 23 buffer in director 16 is activated to automatically 24 transfer the signals being destaged. The actual programs in MSC 17 for initiating and supervising these opera- -26 tion~ are termed IOS/IOC of box 53. Detailed under-27 standing is not pertinent to the invention, it being 28 understood that operational control as defined above 29 is initiated and maintained by MSC 17.
.~.. , ' ~.

~-...

.

1 Assume a destage operation has been initiated 2 by MSC 17 and a DASD read error has occurred during 3 a destage. Such error stops the read operation as 4 in all prior DASD apparatus. That is, the error loca-tion ~DASD spindle address and CCHHR) are recorded 6 in a memory of director 16 plus a host 19; and error 7 recovery procedures (ERP) are automatically invoked, 8 such as at 52. The ERP is also a set of microprograms 9 or defining recovery steps. The ERP also has a sche-duler for selecting which program is to be executed.
11 For brevity, the detailed description and FIGURE 6 12 showing is limited to that program particularly per-13 tinent to the invention; any known method for invoking 14 a program execution may be employed. The portion of the present invention relating generally to an ERP
16 i8 saving the CHR (cylinder, head, and record) asso-17 ciated with the DASD read error; i.e., location of 18 the error in MSC 17. Note old DASD ERP 's in director 19 16 already have saved the CHR for a host 19.
Steps 54, 55, 56, and 57 show the ~resent 21 invention. Step 54 of FIGURE 7 changes the queue 22 control block ~QCB) of MSC 17 as well as releasing 23 ~ a so-called "ASQ" (later described) in MSC 17 for 24 use by other MSS operations. These actions prepare MSC 17 to enter step 55 of FIGURE 8 wherein messages 26 are sent to primary host 17 and director 16 to MSS
27 for restarting destaqing after a DASD read error.
28 This includes an order to swap disk packs between 1 spindles 14 for recovery. Corrective control action 2 for ~ASD is taken in step 56 (FIGURE 9) wherein the 3 page (eight DASD cylinders) having an error is 4 controlled, as will become apparent. Finally, at 57 ~FIGURE 10), destaging is put in a QCB for destaging 6 with a DASD read error to record all data in the same 7 format on tape by DRD. Swapping packs may incidentally 8 remove the read error -- which means all data is now 9 correct.
FIGURES 6-10 are simplified flowcharts showing 11 the micro code used to initiate and control the operation 12 of the MSS for implementing the methodology of the 13 present invention. Each of the flowchart boxes correspond 14 to the table number indicated in the box. For example, QV495 of FIGURE 6 corresponds to table QV495 which 16 is a tabulation of the code which shows the enter 17 and error sensing of the ERP portion of the code for 18 error recovery. The numeral 8380 indicates a memory 19 address (hexidecimal) of the first instruction word of table QV495. In a similar manner, numeral 83A8 21 indicates the memory address of the instruction exiting 22 the table QV495 which is directed to enter the flowchart 23 box QV496 at address 83A4. Also, numerals 8398, 83B0, 24 and 8360 are alternate exit points for entering flowchart box QV499. It is possible in certain flowchart steps 26 that only one instruction is involved in a flowchart 2i box. QV497 of FIGURE 6 has an entrance instruction 28 location of 8738, with the same instruction exiting 29 at 8738 to QV498 at 8748.
.

. . -... . . . . . . .. . ~ - - .- .

1 ERP Entrance to Destage With Error 2 - As mentioned above, the ERP micro code has 3 its own scheduler. One of the programs invoked by 4 the scheduler which operates in a normal manner is the destage with error programs. The ERP destage 6 with error programs are divided into five flowchart 7 steps shown in FIGURE 6. Flowchart steps QV495-QV499 8 are respectively detailed at the instruction level in 9 FIGURES 12-16.
Referring to FI&URE 12, QV495, the first ll instruction 8380 fetches the ASQ pointer for finding 12 the mini header. This pointer is set into external 13 latches at 83F4 with an ASQ scan starting at 83FC.
14 Then, at 8384, the ASQ pointer is moved to the mini header, to the fetch and store register. The mini 16 header is obtained at 839C, and instruction at 8388 17 determines whether or not the destaging operation 18 was involved when an error occurred. If the error l9 ocaurred by other than a destage, the program is short-circuited via instruction at 8398 which enters flowchart 21 step QV499 (FIGURE 16) for bypassing the flowchart 22 steps QV496, QV497, and QV498, all associated with 23 destage with error.
24 When a destage is detected, instruction 25, 8390 abtalns sense data from the sense buffer. This 26 data includes format definition~ for various types 27 of formats~as may be used in MSS. Instructions 83A0, 28 83B8, and 83BC test for format types, i.e., is it .,.. ~ ~

i073553 1 DASD or other? If it is not DASD format, then the 2 error did not occur during a DASD read and, hence, 3 QV4~9 is then entered for bypassing destage with error 4 programming. Assuming destage with error has been sensed, the cylinder in error of the DASD unit is 6 identified at 83B4 and 83A8. Then, flowchart step 7 QV496 (FIGURE 13) is entered for transferring the 8 identification of the real page RP to the ASQ list.
9 ~he sole entry to QV496 is at 83A4 wherein the cylinder address register (CAR) flags or signals 11 are sent to the test register identified by the micro 12 code. At 83EC, constants are read into the-program 13 for identifying the character of the DASD. By way 14 of explanation, different sized spindles 14 may be employed in MSS. QV496 can handle a base size plus 16 a double size. In accordance with this arrangement, 17 the rest of the instructions on QV496 are dedicated 18 for converting the double size to the single size.
19 For example, at memory locations 2DC8 and ?DCC, masks for converting cylinders to a bit map are stored.
21 Additionally, at 8378 and 83C0, an arithmetic conver-22 sion from the actual cylinder address of the double-23 sized DASD spindle 14 to a reference-sized DASD spindle 24 is performed. At 8370, conversion is complete. In the line of instruction words, beginning at 8358 and 26 extending through 83D8, tests are made to find whether 27 or not there are addressing anomalies. In the next 28 line of the table, beginning at memory location 8354 ~0975026 -73-' , ' .

.,- . - . - . . - -. -, .. -... ... - , ., ~ ~

1~73553 1 through 83E8, corrections are made for such anomalies.
2 In the last line, beginning at memory location 837C
3 through 83EO, the bit maps are transferred; and the 4 cylinder masks, as used in DASD, are fetched from the microprogramming register SA with flowchart step 6 QV497 (FIGURE 14) being entered at 8700.
7 Flowchart step QV497 scans the ASQ list 8 for finding the appropriate ASQ entry associatable 9 with the present destaging with error. In fact, QV497 and QV498 (FIGURE 15) have coordi~ated action for 11 finding the ASQ entry in error. Since there are a 12 plurality of entries, i.e., there may be a plurality 1.3 of errors, two flowchart steps provide an ASQ error 14 scan path, as will become apparent.
~V497 (FIGURE 14) first obtains the SSID
16 unit number for the f~iling spindle 14 from the sense 17 register and moves it to a work area in memory, as ~ -18 indicated at 8704. Next, the ASQ list must be scanned, 19 such as beginning at 8714. For example, the controller 15 address was found to match at 872C indicating the 21 scan is in the right spindle group. The branch instruc-22 ` tion at 8738 will branch to QV498 (FIGURE 15) at address 23 8748 to compare the LUA's for virtual unit address 24 ~atching. If there is no match, status register ST
at bit 2 is set to a 1 at instruction 8750. If there 26 is no match, as determined at 8758, the scan must 27 continue. 875C adjusts the scan to the next page 28 (set of eight cylinders) and returns to QV497. Returning .,.,j . ~

, . .... . , ~, . . . ~ .~ ..

1 to QV497, instruction sequences at locations 8740 through 2 877C are also entered from 8738 and are used only 3 at the beginning of the ASQ list.
4 QV498 has some terminal processing whenever a match is made. This occurs in instructions 8754 6 through 876C (second row). That is, the 8758 branch 7 instruction will branch to this sequence whenever 8 there is a match of LUA's. Following the terminal 9 processing in QV498, QV499 (FIGURE 16) is entered at 87AO. , 11 QV499 (FIGURE 16) is a flowchart step which 12 saves all of the accumulated data with respect to destage 13 with error for use later in the destage with error 14 flowchart steps, particularly step 54 of FIGVRE 5.
QV499 can be entered either at 87A0 from the destage 16 with error flowchart steps QV497 and QV498 or by the 17 bypass not destage with error at address 87D0. Exami-18 nation of the instruction list and table will show 19 that the error symptoms associated with destaging 20 ~ are saved and, in the event of entry from QV495, only 21 instruction 87D0 is executed for returning to the 22 ERP dispatcher. In the table, the term 3330 refers 23 to spindle 14, while 3333 refers to the combination 24 of controller 15 and spindle 14. The term "format"
refers to the data format identification; i.e., is 26 it DASD, tape, or otherwise.

.. . , . . . ~ . .

~73553 1 QCB Cleanup 2 The functions of the microprograms repre-3 sented in FIGURE 7 and in the instruction flowcharts 4 of FIGURES 17-33 change the signal content of the queue control blocks (QCB) in preparation for a de-6 stage restart for enabling a destage with error. The 7 first FIGURE 7 flowchart step QS960 (FIGURE 17) is 8 entered at 2018.
9 The main entry at instruction address 2018 is from the ERP scheduler which scans the register 11 including 2018 during an error condition. At 2010, 12 t~e error is tested (D=~, and the of~set, i.e., the 13 relative memory address of the save registers, is set.
14 If the error is other than IOS, i.e., it is not an external error, therefore, not a destage error, then 16 the sequence is exited at 2008. Assuming it is an 17 IOS error, including a destaging error, instructions 18 200C through 22B4 prepare MSC 17 for execution of the 19 other flowchart steps in FIGURE 7. Also, FIGURE 17 lists the microprogram and status bits used for control l;
21 in connection with the FIGURE 7 illustrated program.
22 For example, register NC stores the QCB pointer, ND
23 stores the pointer to the next mini header, etc.
24 QS960 is exited at 224B for entering QS963 j 25 (FIGURE 18) at 202C. If this is an initial entry of 26 the destage restart rather than a re-entry, then the 27 parameters have to be fetched from the MSC 17 job 28 list, such as at 2034. A job list is a program deter-i . .,~ . . .
' Bo975026 -76-. ,j .

~073553 1 mined set of registers (in memory) containing signals 2 necessary for program execution of a task or job. Also 3 upon initial entry, the sequence of instruction words 4 in the second row of instruction word symbols beginning with 203C is executed. Note that the destage restart 6 coding is manipulated in this sequence of instructions.
7 If the entry is a re-entry, i.e., not an initial entry, 8 then the sequence of instructions, beginning with 9 2038, is executed with the code symbol GC bein~ inserted into the lower-byte portion of the instruction address 11 register I~R by instruction 2040. This is a so-called 12 E-word branch for re-entry into the FIGURE 7 program.
13 Table QS963 (FIGURE 18) also stores the 14 indications of the entry points and re-entry points.
For example, initial entry always comes from QS960;
16 while a re-entry can come from several indicated instruc-17 tions identified by the table numbers. Suffixes indicate 18 the branch conditions. GC is a microprogram register 19 addressable for branching to points in accordance with known microprogramming techniques.
21 In addition to storing the branch indicator 22 at GC, QS963 also has exits to QS964 (FIGURE 19) for 23 continuing the initial entry processing. Note that 24 after executing instruction 2040, the MSC 17 returns to the FRP scheduler.
26 Flowchart step QS964 is used only upon initial 27 entry for transferring a message to a host computer 28 19 informing same that a destage with error has been .,. . ~

:

~073553 1 started. This includes the instruction words 2094 2 through 2538 of QS964 with the program exiting at 3 2538 to enter QS980 tFIGURE 26), which sets the para-4 meters for the message. QS980, later described, then exits at 2280 for entering QS970 (FIGURE 22) at 218C
6 for making a call to a program (invokes a program) 7 for transferring the assembled message to primary 8 host 18.
9 Once the message has been handled by the message handler (a program of known structure), as 11 by an exit from QS970 at instruction 21A0 (later 12 - described), re-entry into QS964 (FIGURE l9jis by 13 scanning an index register having index 98 to fetch 14 instruction 2098. Also note the off-page entry 98 is merged with entry from QS985 (FIGURE 28) as later 16 described. The instructions 20E4 through 24F4 set 17 up parameters for continuing the destage with error 18 processing. QS964, in this instance, is exited at 19 instruction 24F4 for entering QS993 (FIGURE 31) for a read/write DSQ operation. That is, the signals 21 for the DSQ block are transferred from a DASD spindle 22 14 into MSC 17. From QS993, at instruction 2498, 23 a program is invoked for the read/write subfunction, 24 i.e., transfer the DSQ block signals from DASD (spindle 14) into the internal memory of MSC 17.
26 Execution of the read/write subfunction 27 causes an index to be read pointing to register 20B8 28 (QS964) termed "B8", which is a re-entry from reading i' , . j, .

~, . . . . , ~073553 1 the DASD for obtaining the ASQ mini header pointer.
2 Then, instructions 20BC through 20FC are ~xecuted 3 for storing the fetched signals in the internal 4 memory of MSC 17. Finally, the next flcwchart step QS965 (FIGURE 20) is entered at 209C for checking 6 the header and device end (DE) status. Note that 7 if MSC 17 had a sufficiently large internal random 8 access memory, the read/write subfunction would be 9 replaced by a mere memory reference.
QS965 (FIGURE 20) checks to see if a destaging 11 with error operation is occurring and that a device 12 end (DE) has not been sent to primary host 18. Exami-~
13 nation of FIGURE 20 will show the functions as they 14 are performed. Only the exits and entries will be described to facilitate an understanding of the 16 program execution.
17 Exit from QS965 i~ at 2134 and 212C to QS973 18 (FIGURE 23) for verifying that the buffer has been 19 filled properly. Exits at 2140 or 2144 are to the next sequential flowchart step QS968 (FIGURE 21) for 21 building the IOR8 for device end (DE). The alternative 22 entries to QS965 are at 2210 and 22B8 for returning 28 from QS973 (FIGURE 23). Additionally, an off-page 2~ entry 9C enters at instruction 209C.
Flowchart block QS968 (FIGURE 21~ builds 26 the input/output request block (IORB) for DE. Entry 27 is only at 214C wherein the staging adapter flag, 28 word counts, W A, and W A are fetched. The page list .

- , . ~ :
., . .,: , - , 1 is then zeroed. Other housekeeping functions are 2 performed in QS968. IORB consists of two words of 3 four bytes each. Finally, register ST has its bit 2 4 position set for indicating to IOS program that there is work to do; i.e., there is going to be a message 6 sent to primary host 18. The QS968 is exited at 7 21CC to enter QS970 (FIGURE 22) at 218C for making the 8 message handler and IOS calls.
9 QS970 is entered at 218C from either QS968 (FIGURE 21) after the IORB is built or from QS980 11 (FIGURE 26) which had set the parameters for a host 12 message. The instruction words of QS970 are house-13 keeping functions in preparation for invoking IOS
14 for a DASD operation and preparation of the message handler for sending a message to primary host 18.
16 IOS invokes functions in director 16 for reading or 17 writing on DASD spindles 14 in the same manner ~hat 18 a host 18/19 operates with a director 16. Such opera-19 tions are set forth in U. S. Patent 3,400,371, supra.
The return from IOS or message can either be a re-21 entry in QS960 or into QS973 via E0 or 84 off-page 22 connectors. QS973 (FIGURE 23) verifies buffer contents 23 and determines whether or not more mini headers have 24 to be fetched.
The verification and checking operations 26 in QS973 are apparent from examination of the instruc-27 tions. A delayed response message to the host simply 28 means that a DASD spindle 14 had some function to ,'~
,.

.
- . - : . :

1 perform and that the host 19 would wait before proceed-2 ing on that particular job until a response has been 3 returned. A typical response is a device end (DE) as 4 explained in U. S. Patent 3,400,371. Exit from step QS973 at instruction 2200 can be to either QS965 6 (FIGURE 20, supra) for further header and device end 7 checks or to QS985 (FIGURE 28) for restoring QCB and 8 testing ASQ. Instruction word 2200 causes the routine 9 to go to QS965 if there are more mini headers and ', to QS985 if everything is alright up to this point in 11 destaging with error.
12 The exit of QS973 at 220C to QS998 (FIGURE 33, 13 infra) occurs only when the buffer is incomplete or 14 an error condition has been detected. The buffer then i3 re-read by QS998 via QS993 (FIGURE 31), which 16 initiates the read/write subfunction; i.e., the infor-17 mation is again read from DASD spindle 14 for insertion 18 into MSC 17. Hence, QS973 flowchart step is a checking 19 function for ensuring high-quality operation of destage with error.
21 Flowchart step QS975 ~FIGURE 24) sets up 22 a destage read ~DR) message to be sent to a director 23 16 for causing it to read signals from an addressed 24 portion of an addressed spindle 14. Entry to QS975 i 25 can be from ~S965 at instruction 2118. This instruc-26 tion also includes control functions with respect 27 to a nonstaging adapter type of operation. However, 28 if it is a destage, then instruction 21BC in Column B

. ~
. .
-

10~35S3 1 loads microprogramming register SB with nonzeroes;
2 and QS973, supra, is entered for verifying buffer, 3 etc. Exit 21EO returns the program to QS965 after 4 manipulating the staging adapter flags, mini header flags, for verifying the updated information. At 6 21E4 of Column C, if the secondary pointer is 7 invalid, QS983 (FIGURE 27) is entered and functions 8 performed as later described.
9 Off-page entry 14 (14 was stored in a micro-programming register) occurs after the DSQ block was

11 written in MSC 17 internal memory. At 2014, a destage

12 error is verified. If it is a destage error, at 20B4,

13 register DN is forced to zero; and QS973 is entered

14 at 213C. The sequence of instructions of QS975 beginning at 20B0 sets up message controls in preparation for 16 entering QS978 (FIGURE 25, Sheet 14) for moving the 17 message for director 16 to a job list, i.e., in pre-18 paration for transferring the message to director 19 16 for the destage with error operation beginning in QS978. Flowc~art step QS97~ merely moves the messa~e 21 ~ for director 16 into the job list of MSC 17. The 22 sole entry of QS978 is at 2240 with the remaining 23 instructions ensuring that the entire message is stored 24 in the MSC 17 buffer area. When the message is stored, the branch at 2244 moves program execution to 224C to 26 post the message, and the posting code (location of 27 posting) is stored with the message. At 225C, the 28 parameters for the message are ready to be set by 29 QS980 ~FIGURE 26).

.... .

~?73553 QS980 (FIGURE 26) can be entered either 2 from QS978 (FIGURE 25) or from QS964 (FIGURE 19), 3 which also sends a message to primary host 18. Entry 4 from QS978 is at 2260 which sets up messages so that the MSC 17 stage scheduler can identify the source 6 of the message and that it should go to an addressed 7 director 16. For a message to primary host 18, entry 8 from QS964is at 226C. QS980 merely offsets the mes-9 sage into the job list; i.e., the messages for primary host 18 or director 16 are intermingled in the job il list. In the FIGURE 26 second row of instructions 12 beginning with 2270, the parameters mentioned above 13 are stored, and the message handler is called from 14 2280. Note that instruction 227C (top row of instruc-tions) sets re-entry point EO for re-entry of the 16 FIGURE 7 illustrated program into QS960 after the 17 messaqe has been sent to either a host 18, 19 or to 18 a director 16. Exit from QS980 at 2280 of moves the 19 program to QS970 (FIGURE 22) for making the message call or the~IOS call as previously described.
21 Flowchart step QS983 (FIGURE 27) is an error 22 handling subroutine. Any secondary pointer error 23 is referred to QS983 which includes setting bit 5 24 of microprogramming register ST by instruction 21B0.
A link pointer is set by instruction 21EC with offset 26 being established by instruction 2324. The 2334 27 instruction word is a branch instruction whic~ includes 28 referring to the QCB entry and bit 4 of the ST micro-.

.

- . .
- . - . ...
.,-- ..

1 program~ing register for exiting to QS988 (FIGURE 29) 2 for calling a job list for further action or exiting 3 to QS985 (FIGURE 28~ yia either instruction 2374 or 4 2350 or restoration of the QCB in an error recovery action.
6 The flowchart step QS985 (FIGURE 28) restores 7 the QCB entry and tests for the ASQ entry having a 8 destage error. At instruction 23B0, the micro code 9 checks to see whether or not the pointer was of the secondary type; if yes, the ALU is dummied at 21C4 11 and QS973 (FIGURE 23, supra) is entered. If not, 12 instruction at 21C0 requires a re-read of the original 13 QCB entry. It sets bits 4 of register ST at 21C8 14 and goes to QS983 for error and recovery functions.
The entry at 2080 from QS963 accumulates the block 16 and mini headers for the ASQ in error, tests for destage 17 (20A8), and at 20A4 sets bit 0 of register S~ to 1 18 signifying a destage in error with no DE to a host 19 18, 19. QS964 is then entered for sending a DE signal to a host 19. Instruction 20AC is executed whenever 21 the operation is not a destage with error. From here, 22 QS964 is entered for sending a message to a host 19 23 signifying inter alia no destage with error.
24 Flowchart step QS988 (FIGURE 29) calls the job list for ensuring that the destage with error 26 continues. The single entry to QS988 is at instruction 27 location 2348. QS988 resets the QCB cleanup bit indi-28 cating that the program of FIGURE 7 will be exited.

~ -"3 80975026 -8~- ~

.. . . ..

~073553 1 A test is initiated at 2358 if the pointer is valid;
Z and at 2360 (top row), a link pointer is fetched which 3 was stored in a previous routine. Then, after the link 4 pointer has been appropriately processed, QS983 is entered. If there is no job list to wake up, then 6 the branch at 2368 causes the program to execute instruc-7 tion 2364 which in turn branches to either store the 8 updated QCB entry, as at 2068, and then enter QS995 9 (for detecting more cleanup work, i.e., updating program control bits) or enters the sequence of instructions 11 beginning at 23A0 which includes storing the updated 12 QCB entry for later use and sets the job list wake up 13 by instruction at location 239C. In this latter 14 instance, the scheduler is re-entered using normal microprogramming techniques.
16 Program flowchart step QS990 (FIGURE 30) 17 transfers the block of ASQ indicating signals to a 18 buffer ~portion of internal memory) in MSC 17. This 19 action prepares the MSS for recording the ASQ updated to the tables contained on DASD. The sole entry is 21 at 2298 from either QS975 or QS965. The instructions 22 merely show the techniques used to move the signals 23 of the ASQ block to the buffer for operation by QS993.
24 QS993 (FIGURE 31) read/write DSQ block merely initiates the read/write subfunction associated with 26 DASD spindles 14. MSC 17 initiates the action as a 27 host to director 16 which then establishes a data 28 path (u~ing known techniques) between MSC 17 and the .

~073S53 1 DASD controller 15 and spindle 14. All of the instruc-2 tions in QS993 save the appropriate control signals 3 such that director 16 and MSC 17 can cooperate in trans-4 ferring the data signals, as above described. The only exit from QS993 is at 2498 which branches to 6 MSC scheduler while inserting control signals in MSC
7 17 storage for initiating the read/write subfunction.
8 Such read/write subfunction (not shown or described 9 in this application?, can be compared to access method services and input/output systems of present-day IBM
11 360/370 computers. After executing instruction 249B, , 12 the program returns to the MSC 17 scheduler.
13 Flowchart step QS995 (FIGURE 32) includes 14 instructions for determining whether or not more updating for destaging with error is required. Entry is from 16 either QS988 (FIGURE 29) directly or via the scheduler 17 from off-page entry 88. Examination of the instructions 18 shows that QCB entries are fetched and compared to 19 determine whether or not all of the job listings have been completed. If not, QS998 (FIGURE 33) is entered 21 from 24D4 from 995 for more QCB operations. If the 22 cleanup i3 completed, the MSC 17 scheduler is re-23 entered from instruction 2428 of QS995.
24 Flowchart step QS998 (FIGURE 33) sets register ME to "3C", zeroes bit 7 of register SC, and enters 26 QS993 (FIGURE 31) for reading the ASQ block. Such 27 action is in preparation for executing the programming 28 illustrated in FIGURE 8.

, . ~, ; , .

~, . ' . '.
I ... . . . . . ~
-. . ; , . , . ... . . ~ - " -- . . ... - - . , . . -.. .,. ., - : - :~ - - .

~073553 1 Destage Restart 2 Referring to FIGURE 8, the program for ini-3 tiatihg a destage in a destage with error is shown 4 in flowchart form. This program sets all the control signals in MSS for facilitating reading DASD after 6 a DASD destage read error and after the disk pack 7 has been moved from one spindle 14 to a second spindle 8 14 without altering the logical address and, hence, 9 without altering any of the control signals within MSS. That is, each pack contains an electrically 11 sensed address card that determines the unit address 12 of the pack, hence, the address of spindle 14 on which 13 the pack is mounted. Such apparatus is included in 14 the IBM 3330 disk storage apparatus. A portion of the programming initiates a vary-off, such as a~t QC930 16 (FIGURE 39) which includes a vary-off neutral in which 17 the disk pack is physically connected but logically 18 disconnected, as will become more apparent.
19 The general entry to the FIGURE 8 illustrated programming is to QC800 (FIGURE 34) which includes 21 the initial entry from the MSC 17 scheduler.
22 In addition to the program steps, PIGURE 34 23 lists the general entry points for a destage restart, 24 as well as the entry point sequence. The format of the save area at memory address 4800 is also shown.
26 In addition to being entered from the MSC 17 scheduler, 27 restart entry also occurs from QC830 (FIGuRE 40). The 28 MSC 17 scheduler entry is from a flowchart step (not .~. ,j . .

.:

' . ~ - ' ' ' ' ' -- ' , ' 1 shown) enumerated ZB021 of the ~SC 17 scheduler. The 2 sole exit is at 18FC, which connects control with the 3 proper linking set of signals to QC810 (FIGURE 35).
4 The signal content of microprogramming register GC
goes to IAR for fetching an instructlon word entering 6 QC810, as next described.
7 QC810 (FIGURE 35) flowchart step is entry 8 error processing, i.e., it adjusts the program sta~us 9 signal for executing destage restart. Messages 95 and 96 are the messages to the primary host 18 CPU to 11 inform it of (9S) equipment checks or (96) data 12 checks. Included is the information a~d destage with 13 error. Concerning the VOLID of the virtual volume being 14 processed, entry to QC810 at 1844 occurs from writing the mes~age 96, while entry at 1824 occurs from writing 16 the message 95. Exit i9 at 171C which returns to the 17 error recovery procedures (not shown or described) 18 referred to briefly with respect to FIGURE 6. In 19 this in~tance, control of MSC 17 in the ERP is at register F204. The qode recorded at F204 is GC. Another entry 21 to QC810 18 at 1830, which returns from the vary 22 neutral fundtion. This action makes the disk pack 23 of the spindle in error unavailable to any host 18, 24 19. Thl~ lnhibition is achieved by reserving the ~pindle tc MSC 17. If there i8, in fact, no error, 26 instruction 18A4 provides reference to a function 27 ~FCN) and then tran~fer~ content of GC to IAL, the 28 off-page aonnector shown in FIGURE 8. If there i8 an .

. - . . ~ .. . - . - . : . . . . - - - - . . . . : . . - , - .
- - - ............ . ~ . . : , .
.

1~73~53 1 error, then in Column B, 18AC, the condition code is 2 obtained from the vary neutral operation o the ERP.
3 Instruction 18A0 is merely an address filler. The 4 instruction se~uence following 18A0 is program housekeep-ing related to particular condition codes which is 6 apparent from inspection of the instructions.
7 When there is an initial entry without error 8 instruction at 1800, the following instruction sequence ~ operates with DSQ's (as shown) for a pure DASD function not pertinent to the present invention. If there 11 is a restart of the scan of DSQ's, the instruction 18E0 12 is entered which leads into instruction 1700 for 13 indexing to the condition code as indicated.
14 More than one director 16 may be connected to a given controller 15 and spindle 14. In such 16 instances, both directors must have information con-17 cerning the destage with error, even though only one 18 of the directors 16 is directly involved. In such 19 an instance, the reset written on the first director 16 labeled "SAl" is done in instruction 1818 (bottom 21 row of FIGURE 35); while for the second, labeled "SA2", 22 is done at 1820. The release for the two directors 16 23 is in the two subsequent instruction words 184C and 24 1850. Then, there is an index to the IORB for obtain-ing the return code, i.e., link address. Then, the 26 IOS return code is sent to the test register of MSC
27 17 and the routine branches to the proper exit instruc-28 tion via branch instruction 1700.

.
; i . .

1073553 ~-:

1 Another function performed by the FIGURE
2 8 illustrated programs is to find the QCB in error.
3 QC820 (FIGURE 36) performs this function. A11 of the 4 entries to QC820 are from instruction l9EC of QC810.
The entry at l9E0 takes the QCB pointer and supplies 6 same to the sector register of DASD, i.e., the regis-7 ter of MSC 17 which identifies the sector having the 8 QCB pointer. Instructions lB68 and lB6C modify the 9 QCB pointer for use with DASD. Then, the instruction at lB14 sets bit 2 of ST register to 1 if it is, in 11 fact, a DASD restart. Exit can be to QC830 (FIGURE 40), 12 later described, or the branch can be to more internal 13 program code of QC820. If it is not a DASD restart, 14 instruction lB10 indicates that the QCB in error count is zero such that at lB00 the QCB in error counter 16 i~ incremented (bumped) to the next QCB. If it is 17 at the end of the sector of DASD, then bit 3 of regis-18 ter ST is set to a one. In this manner, the QCB in 19 error is found by the indication of a DASD restart.
Note the QCB is not in error if there is no DASD restart.
21 From lBlC the program moves to lA34 which saves the 22 SSID for step QC830. Instruction 1834 is a four-23 way branch which goes to an appropriate instruction 24 for moving an appropriate address to the data address register DAR. Instruction lA38 moves the MVT pointer 26 error flag to test register BR. Bit 5 is 1 if the 27 scan is over, i.e., no QCB in error was found. If so, 28 instruction lB7C exits the program to QC900 (FIGURE 38), ~:)73553 1 as later described. On the other hand, lB78 is exe-2 cuted if the QCB iS marked as being in error. Then, 3 instructions lB00 and lBlC can be re-executed in the 4 illustrated loop for finding the QCB in error. Finally, previously described instruction lB14 is executed;
6 and QC830 (FIGURE 40) is entered at lB18.
7 Another entry to QC820 is at 1900 which 8 indexes the program to error count in the job list.
g The QCB scan count is obtained at 1904, saved at 1908, and a QCB scan loop being entered through instruction 11 l9E8.
12 Before going into more of the detail of 13 destage restart, other entry flowchart steps are first 14 de~cribed, specifically, QC890 (FIGURE 37), QC900 (FIGURE 38), and QC930 (FIGURE 39). Off-page entry 16 50 moves the program to 1950 of QC890 to build SSID for 17 the QCB wake-up (follow-up). The QC890 entry is from 18 l9EC of QC810. The SSID for the controller 15 and 19 spindle in error is stored in a program-defined para-meter area of MSC 17. The next entry of the FIGURE
21 8 progr~mming is then set by lBDC (next entry from 22 the block l9EC, QC810) and then QC830 (FIGURE 40) is 23 entered for exit from the FIGURE 8 illustrated program.
24 Turning now to program step QC900, destage restart scanning i~ re-initiated. This can be done 26 by entry from QC820. Instruction l9E4 checks for 27 the D-bus being zero. If it is zero, all of the scan-28 ning is completed and instruction at l9DC returns 1 to the scheduler of MSC 17 indicating function complete.
2 On the other hand. If D is not zero, restart is 3 initiated via l9D8 to enter QC830 at lB5C.
4 Entry to QC900 can be from the vary-off neutral which was initiated by QC930 (FIGURE 39).
6 It must be remembered that vary-off neutral logically 7 removes a controller 15 and ~pindle 14 from MSS while 8 allowing it to be physically attached.
9 QC930 calls a program termed a message handler, and also intiates vary-off neutral. Once the a~ove 11 programs have detected that the DASD read error occurred 12 during a destage and that the DSQ in error has been 13 found, and all of the control registers in MSC 17 have 14 been set up as described for r~starting the destage lS for recovery purposes, the offending spindle 14 can 16 be varied off. Off-page connector 24 allows the program 17 sequence to enter QC930 at address 1924, which is 18 a return from sending a message 95. The subsequent 19 instructions 17F4 et seq set up MSC 17 for varying the offending spindle 14 off. The three instruction 21 loop 17EC~ 17D8, and 17D4 finish setting up MSC 17 22 for the vary-off operation. Once the appropriate 23 number of entries have been made, the branch instruc-24 tion 17EC directs the program to 17DC, which sets the next entry for flowchart step QC830 (FIGU~E 40), 26 which in turn, exits via 18DO (FIGURE 40) to the JLS
27 function, i.e., callR the vary-off programming. The 28 latter programming is the same as used in the IBM
2g 3830-II Director.
.
~0975026 -92-!

.

~(~73553 1 Prior to this time, the DSQ flags have been 2 stored in microprogramming register NA. Entry into 3 QC930 from destage restart which is from QC800 moves 4 the DSQ flags from NA into branch register BR. Both flags are checked at 17F8 (FIGURE 39) which determines 6 whether or not a vary-off neutral is to occur. If 7 BR4=1, it is an equipment check; then, instruction 8 17E8 and l9BC are executed to enter QC850 (FIGURE 45) 9 for sending the message to director 16 for recovering from the equipment check. However, if a data check 11 is detected (BR4=0), then the next entry is set to 12 destage with error for recovery from that DASD read 13 error (data check). QC830 is entered, as above 14 described, for the vary-off operation.
QC830 flowchart step provides the exit routines 16 in the read and write of the DSQ with respect to DASD
17 spindle 14 stored control tables of MSC 17.
18 QC830 (FIGURE 40) is divided into three 19 sequences. A first portion begins with instruction at lB18. This first sequence of instructions through 21 lB40 does program housekeeping functions required 22 after QC820 has found a QCB in error. Exit from this 23 first sequence of instructions is to the second sequence 24 of instructions beginning at lB44. Entrance at lB44 also i8 f~om QC840 (FIGURE 41), which is a part of 26 scanning for the error in DSQ. Finally, the third 27 se~uence, beginning with instruction lBSC, includes 28 entries not only from the first and second se~uences .j. j ~073553 1 of Q~830, but also from a plurality of other flowchart 2 steps. This third sequence provides general processing 3 required by the programs for the plurality of exits 4 at 18D0, 18D4, 18D8, and 18DC, which respectively are the job list function and subfunction call, the 6 pasano call, restart call to QC800, and the IOS call 7 via the job list function.
8 QC840 (FIGURE 41) scans for the error DSQ
9 and then.the program exits to either build the IORB
at QC880 (FIGU~E 48), build a reset for writing data 11 at QC910 (FIGURE 42), or exits the write to DSQ to 12 QC830, supra. An entry from QC925 enables the program 13 to patch QC925 to QC830 for writing DSQ. Instruction 14 lB44 only is used in that patch.
Entry to QC840 at 1910 enables the program 16 to index for the DSQ read function and subfunction.
17 Assuming that the DSQ has not been processed, then 18 the instruction sequence, beginning at l9B0, is exe-19 cuted for indexing for the DSQ. Upon completion of processing, the entry branch instruction at l9A0 branches 21 to either l9B4, DSQ processed, or to l9B8 for building 22 the reset write data. Assuming the processing is 23 completed, the program proceeds through l9B4, 1960, 24 thence lBA0, thence to QC880 to build the IORB. If it is a non-SA type, QC880 is also entered.
26 If the DSQ was already processed, the in-27 structlon 1968 and lBA0 direct the program for building 28 the IORB at QC880 (FIGURE 48). On the other hand, .

.j......... .
BO~75026 -94-., . . . . ' , .' ': -. .

1 the destage with error can require that the DSQ be 2 written. In this case, the branch from instruction 3 1974 is to l9B8 wherein general control of the destage 4 with error flag is turned off. The destage with error flag in director 16 (SA flags) is set at lBB8 and, 6 then, the QC840 is exited by the instruction lBB4.
7 This exit goes to QC830 as previously described.
8 QC910 (FIGURE 42), which is only entered 9 from QC840, builds resets for written data in the destage with error process. This action identifies 11 the pages in error to ensure that the director 16 12 (also abbreviated as SA) will not process data from 1~ the page in error. The SA flags received from director 14 16 are first pointed to in the DSQ. The DSQ moves to the MSC 17 buffer area of its main storage in the 16 small loop beginning with instruction 1610. Upon such 17 data being moved, the branc~ at 1608 fetches the flags 18 and adjusts same for reset written. Then the buffer 19 is invalidated at 1630 and the error page list pointer is adjusted at 162C (page list is a list of pages 21 in DASD spindles 14 to be staged/destaged). Then, 22 at 1640, the program determines which of the cylinders 23 in a page have been destaged. $hat is, program tests 24 for which of the cylinders of DASD were destaged with-out error. This function is achieved by inverting 26 the mask for the page, there being one bit for each of 27 the eight cylinders in each page. A one normally 28 indicates that data has been written in the cylinder.

.. -- - .: ... ... - .......... . . - - : -,: , .

~L0735S3 1 By inverting the mask, the l's will indicate the cylin-2 ders have been destaged. It should be remembered 3 that when a cylinder is destaged and the data has 4 been successfully recorded in MSF, the cylinder mask is altered to reflect a zero indicating that destaging 6 is not required.
7 The last instruction loop, beginning at 8 1648, is for program housekeeping to control the page 9 list entries and accessing. The sole exit from QC910 occurs after the page list processing has been com-11 pleted; it is to QC920 (FIGURE 43~ via branch 1650.
12 QC920 (FIGURE 43) follows QC910 to reset 13 cylinder valid bit and reset the written data. '-14 Resetting the cylinder valid bit provides an indica-tion to director 16 that the cylinder in spindle 14 16 does not contain data. That is, when director 16 17 looks at the cylinder valid map, it appears to director 18 16 that data from MSF has not been staged to that 19 particular cylinder. In this manner, director 16 will not access the cylinder in error thereby protecting 21 the error condltion from being inadvertently accessed 22 by a host 18l 19.
23 In QC920, microprogramming register NC of 24 the computer circuits has the destage restart flagi while register GA points to the mini header in MSC
26 17~ ~torage. Preliminary processing portion of QC920, 27 at 165C et seq, determines which cylinders have been 28 destaged without error. At 1674, the program polnts 29 to the mini header and exits to QC925 (FIGURE 44).

.,., ~ .

1 Flowchart step QC925 merely resets the destage flag 2 and sets same with the error flags in director 16 SA
3 flag byte for the DSQ. The program immediately returns 4 to QC840, previously described.
A second portion of QC920 sets the entry 6 for updating the alternate or second director 1~ (each 7 spindle 14 may be connected to two directors 16).
8 This is done by obtaining the data for the director 9 16 (SA data) and then building a command to such director 16 in flowchart step QC870.
11 The flowchart steps QC850 (FIGURE 45), QC860 12 (FIGURE 46), and QC880 (FIGURE 48) all relate to con-13 structing commands and messages in connection with 14 destage restart. Because this is an addressing func-tion, QC850 identifies the SSID of spindle 14 in error 16 and puts the information in message form, and then 17 saved. QC860 takes the information which has been ~8 previously assembled in various locations and formats 19 same into a message for transmittal to the receiving unit such as director 16. QC870 has two functions.
21 In MSC 17 control tables, it marks the page in error, 22 i.e., set of eight cylinders containing the DASD read 23 error which occurred during a destaging operation.
24 It then builds a release command to the director 16.
QC880 builds the release IORB for releasing director 26 16 later in the destage with error functions. The 27 destage with err~r operation has been completed to a 28 point enabling marking the page in error on the , ; i ~ BO975026 ~97~

.
.
. . . -.
- ~ .. .
.

:10735S3 1 offending spindle 14.
2 Mark Page In Error 3 The destage restart having set the control 4 signals in MSS for enabling destaging the destaging the data in error, the FIGURE 9 illustrated program 6 effects a control in MSS identifying the page having 7 the permanent DASD read error in such a manner that -8 the page will never be allocated again for receiving 9 staged data signals from MSF. The instruction sequence charts are in FIGURES 49-59. This marking of the DASD
11 page prevents that portion of the spindle 14 pack from 12 being reused. This action facilitates reclaiming the 13 data residing on the damaged page, as well as prevent 14 ing repeated errors caused by reuse of that portion of spindle 14. The error flag pointing to the page in 16 error, i.e., the marking, isturned off when the pack 17 is varied off and turned back on when the pack is 18 varied back on.
19 The program has two sets of interface require-ments depending on whether or not the MNT entry for 21 the virtual pages to be marked in error are in the 22 MSC 17 buffer at the time the mark page in error program 23 ^is invoked. ~f the MVT entry is not in the buffer, 24 then the job list index is looked at to find VOLID.
The virtual page to be marked has to be identified.
26 The requirement i8 that the VOLID must be on a word 27 boundary and the ~ob li8t at offset FF identifies the 28 caller. A "caller" is a program request for action.
29 On the other hand, if the appropriate MVT entry is - . . . - .:

1 already in the MSC 17 buffer, identified as buffer 2 32, a plurality of pages of the same W can be marked 3 in error by one program execution. A temporary flag 4 in the MVT entry, bit 3 of the flag device byte, has to be set to a 1 for the pages to be marked in error.
6 After the pages are marked, these bits are set to 7 a 0.
8 Output of the program depends on one of four 9 sets of action taken by mark page in error. These, in turn, depend on the attributes of the page which 11 is identified as being in errox. The first is a real 12 page in a virtual page entry of MVT being a 0. The 13 program assumes that the error occurred on a destage 1~ scheduled by a clear volume command. A clear volume command is from the host CPU preparatory to varying 16 5pindle 14 off-line. Since this process will auto-17 matically zero the page in error flag, no action need 18 be taken. In the second instance, the error occurred 19 on a page belonging to data staged in DASD which is not permitted to be destaged; i.e., a host 19 has 21 commanded MSS to bind the volume or the data set on 22 DASD such that it is always accessible. In this instance, 23 mark page in error marks the page in error flag, but 24 the page is not deallocated; i.e., allocation still remain~ to the bound volume. The third instance occurs 26 when there is an inactive page not belonging to a 27 bound volume or for a bound volume which has been 28 demounted; i.e., the host CPU has said to MSS: Take .
, ~0975026 -99-,! , ~' ' .

~, .

1 the data and put it in MSF. In this instance r the 2 page in error flag is set, and the real page iden-3 tifier in the MVT entry is zeroed. Inactive page 4 count is then decremented. In the fourth instance, the error occurs in an active page that is not bound 6 to D~SD. In this instance, the page in error flag 7 is set, the real page entry in MVT is zeroed, the 8 page is deallocated from assignment to force any active - 9 cylinders off DASD, and the program decrements the - 10 inactive page count by 1. The decrementing of inactive 11 pages means that the page being marked in error is 12 not active; hence, when it is made inactive, the page 13 count would be incremented to keep an accurate count.
14 Marking the page requires that the inactive page count be decremented.
16 In some instances, a page may be detected 17 as being in error by a host without the error being 18 detected by MSS. In this instance, the director 16 19 identifies the program in error to M5C. The micro-program registérs used in this program are MA con-21 taining the virtual page in error identification, 22 MB containing the real page in error identification, 23 MD identifying the virtual page index location in 24 the MVT, and MF containing the constant OC. Upon termination of the program, the contents of MA, MB, 26 and MD are invalidated. The ST register assignments 27 include bit 1 indicating deallocation required, bit 28 2 indicating destage with error, bit 3 looking for ; i i i ,. , ~ - . .

1 multiple errors, bit 5 indicating MVT write is required, 2 bit 6 indicating 0 real page entry in MVT, and bit 3 7 indicating a loop control.
4 The program instruction sequence details for effecting the above-described functions of FIGURE 9 6 flowchart sheets QC701-QC718 are shown in FIGURES 49-59, 7 one figure per FIGURE 9 flowchart step as hereinafter 8 set forth.
9 FIGURE 49 shows QC701 which performs preli-minary program functions incident to program entry.
11 QC704 (FIGURE 50) finds the page with error 12 and identifies the contxoller 15 to which the spindle 13 in error 14 is connected. A read operation is set 14 up.
QC706 (FIGURE 51) sets the page in error 16 flag in the SDG entry and prepares for writing in 17 the SDG tables while reading the MVT.
18 QC702 (FIGURE 52) entered from QC706 (FIGURE
19 51), as well as from non-deallocate entry at 3000, sets up MSC to read the MVT table. In the constructed 21 embodiment, the MVT table is stored in DASD accessible 22 to MSC 17 via a director 16 and controller 15.
23 QC708 ~FIGURE 53) is also entered from QC706 24 (FIGURE 51), as well as off-page entry OC. This step verifies whether or not there is a deallocation request 26 generated by the previous programs. If so, deallocation 27 parameters are set up in the program control signals.
28 Notes indicate other functions of interest. QC710 ..
BO975026 -101- ~

- - ,. .- ~ . .

1 (FIGURE 54) is merely a continuation of QC708.
2 From QC710, after the parameters have been 3 set, QC712 (FIGURE 56) is executing for setting the 4 flags of the virtual page-for deallocation and then verifying whether or not the program is finished.
6 From QC712 (FIGURE 55), there can be a deallocation 7 of the page in error, i.e., move the status of the ~ page from an active status to inactive and then marking 9 it in error for making it unavailable to any host 18, 19. QC712 can also return to QC704 (FIGURE 50) for 11 finding an error if the program is not yet finished.
12 Also, QC713 (FIGURE 56) is entered for zeroing 13 the real page and turning off the temporary error 14 flag. "Zeroing the real page" means the entry in MVT is zeroed to indicate that it is not available.
16 Details of these functions are clearly set forth in 17 the figure.
18 QC714 (FIGURE 57) is entered from QC713 19 (FIGURE 56) for executing a first portion which indexes the cyllnder-head-record in error and then enters 21 a second portion which is also independently enterable 22 from QC702, QC706, and QC704. The second portion 23 sets the parameters for writing in the MVT. It should 24 be remembered that the MVT is stored in DASD under control of MSC 17. Exit is to a return calling a 26 so-called "table module" which enables writing signals 27 in the tables of MSC 17, as well as updating the tables 28 in directors 16. The MSC 17 tables can be in registers 29 of DASD spindleq 14.

~0975026 -102-~073553 1 QC716 (FIGURE 58) merely establishes and 2 transfers a message to a host 19 or primary host 18.
3 This function is generally well known but is still 4 detailed in FIGURE 58.
From QC716, the only exit is to QC718 6 (FIGURE 59) which calls the message handler and handles 7 the return from the message handler, the message handler 8 being a separate program in MSC 17 of the type commonly 9 found in CPU's necessary for transferring messages over an I/O channel to another ccmputer. By combining 11 FIGURE 9 with FIGURES 49-59, the entire program func-12 tions become apparent.
13 DASD QCB Restart , 14 This program (flowchart of FIGURE 10) sets up a queue control block QCB of DASD for restarting 16 the destaging for completing the destage with an error 17 function of MSS. The previous programs pass to this 18 program the SSID of the DASD spindle 14 creating the 19 error such that the present program scans the queue contxol blocks looking for a given QCB that is marked 21 in error, identified as being a DASD error and having 22 a dependency counter not valid. Once a QCB has the 23 three criteria set forth above, the program resets 24 the marked in error bit to 0, makes the DASD error byte all 0's, tracks all chained QCB's, and marks 26 each available for execution. The chained entry table 27 consists of pointers pointing to the next entry in 28 the chain and is a well-known programming technique.

. i ~.. . . . . . . .

~073553 1 Entries to this program (FIGURE 10) include an original 2 entry at QT610 (FIGURE 61) from QC605 (FIGURE 60), a 3 return from reading the QCB DSQ at QT640 (FIGURE 65), 4 return from an MVT access enqueue into QT670 (FIGURE 70), return from reading a pointed-to DSQ at QT680 (FIGURE 71), 6 and return from an MVT access dequeue at QT690 (FIGURE
7 72). Some of the special microprogramming register 8 assignments include register MA for identifying the 9 MSF (MSS may have plural MSF units) to which the destaged data is to go, MB for the second byte of 11 such identificat$on, MD has the saved QCB address, 12 ME has the AMR information, MF has the QCB address, 13 NA contains the SSID unit number (DASD), NB identifies 14 the first entry in the QCB list, NC identifies the last entry in the QCB list, ND identifies the SSID
16 controller 15 number, TA contains a code which has 17 the saved ME, ST is a general status usage, and BR
18 has scan loop control and work to do indicators.
19 QT605 (FIGURE 60) flowchart block controls the entry and re-entry to the program. This flowchart 21 block performs QCB list control and fetches pointers, 22 etc., for program execution. The exit includes branch-23 ing to entry addresses as indicated in FIGURE 60.
24 The original entry block QT610 (FIGURE 61) includes fetching SSID and other initializing infor-26 mation necessary for restarting the destage with error.
27 The only exit is to QT620 (FIGURE 62) for the QCB
28 error scan. The program when executing in the QCB ERP

r j ' 1073s53 1 scan path is seeking the QCB which identifies a DASD
2 destaging read error.
3 QT625 (FIGURE 63~ and QT630 (FIGURE 64) 4 cooperate to call a subfunction for reading a DSQ.
QT630 is also entered from other flowchart slips, as 6 will become apparent. Upon exiting QT630, MSC starts 7 a subfunction to perform a DASD-related operation 8 such as reading the DSQ as commanded by QT625, re-9 cording on DASD, etc. Such operations are well known as input/output operations of a programmed processor.
11 Re-entry into the program after the QT630 12 called subfunction is to QT640 (FIGURE 65) from the 13 scheduler of MSC 17. In destage with error, QT640 14 scans the DSQ acquired from DASD spindle 14 for the appropriate address (SDG) of a controller 15. The 16 combination of the controller 15 and DASD spindles 17 14 i9 called a "staging drive group", with the address 18 being signified by SDG.
i9 QT645 (FIGURE 66~ is entered only from QT640.
QT645 scans the mini header (referenced above) or the 21 appropriate logical unit address (LUA). That is, 22 the virtual addressing for spindle 14 must be matched 23 with that requested. If there is no match, the scan 24 continues by re-entering QT620 which requires a second execution of the subfunction called by QT630, supra.
26 If there is a match, i.e., the appropriate 27 LUA has been found, QT650 (FIGURE 67) then scans the i i .. . .

- . ,.. ~ . . . - -. . . , . . ~

l acquired mini header for the destaging DASD read error 2 indication. If the QCB has no destaging error indica-3 tion, QT620 is entered again. If a destaging error 4 is found in a given QCB, QT650 is exited by the instruc-tion at 12C0 for entering QT655 (FIGURE 68).
6 QT655 readys the Q~B for initiating the 7 ~ASD destage with error. If there is no QCB pointer, 8 QT670 (FIGURE 70) is entered for calling an enqueue.
9 Otherwise, QT655 is exited to QT660 ~FIGURE 69) which stores the QCB pointer.
ll After the DSQ has been read, such as initiated 12 by QT630, QT680 (FIGURE 71) checks the DSQ pointers.
13 If a pointer is found, QT655 (FIGURE 68) is entered 14 for storing the pointer. If a pointer is not found, QT630 is entered for calling a subfunction to again 16 read a DSQ from a DASD spindle 14.
17 QT690 (FIGURE 72) scans the fetched QCB
18 list for another QCB to be fetched.
l9 Finally, QT695 (FIGURE 73) adjusts the QCB
list to ensure the scan has been completed. One exit 21 is to QT620 (FIGURE 62) for scanning the last QCB, 22 while the other exit is a finish or all-done which 23 returns the programming to the scheduler of MSC 17.
24 ~fter the MSC 17 scheduler has been entered, the actual destaging or movement of data signals from DASD with 26 the error proceeds using programs and apparatus in 27 con~unction with the I~M equipment mentioned above.
' ' , .j.. j .

.

1 A Partial Hardware Embodiment 2 FIGURE 11 illustrates a speed-up of certain 3 functions of the present invention when additional 4 hardware circuits are added to the FIGURE 2 illustrated MSC 17 circuits. The description of FIGURE 11 is coordi-6 nated with that of FIGURE 5 such that the interaction 7 of the added hardware with certain ones of the above-8 described microprograms will become apparent. In 9 FIGURE 5, steps 50, 52, and 53 are assumed to have been executed as above described; that is, a destaging 11 DASD read error has been detected and recorded that 12 the CHR has been saved. Also, the QCB cleanup functions 13 and the freeing of the ASQ have been achieved in step 14 54. To enable a destage restart, using the FIGURE
11 illustrated circuits, an SSID memory 100 is attached 16 to the MSC 17 and accessed as shown in FIGURB 11.
17 The D-bus of FIGURE 2 includes AND circuits (not shown) 18 connecting it to the D*-bus, together with a tag line ~19 T* which indicates write or read. The D-bus will normally contain memory addresses for the SSID memory 21 100. Such memory addresses constitute the SSID assigned 22 to given RUA's. Once a memory address has been captured 23 in SSI~ memory 100, subsquent transfers are data siqnal~
24 to be recorded in the SSID memory. Such memory signals include error flags; i.e., the given SSID has an error 26 during destage--there is one register in memory 100 27 for each DASD spindle 14. Also in each register, 28 the signal content includes the address code RUA and .~.. ,; .

.

l lists of real pages which have had a data check, provided 2 the data check flag in that register is set to the 3 active condition. Additionally, the well-known cylinder 4 masks are stored in each of the registers. Memory lO0 enables MSC 17 to compare the SSID with the RUA
6 such that when the primary host 18 refers to a DASD
7 spindle 14 by the SSID, it merely reads memory 100 8 to obtain the RUA for accessing same via director 16.
~9 Read-out of memory lO0 is over cable lOl, together with the read-out tag signal on line 103 to buffer 11 register 102. When register 102 is filled from memory 12 100, bit F sends a flag or interrupt signal over line 13 104 interrupting MSC 17 to receive signals on the -14 A-bus via cable A* from register 102. In FIGURE ll, MSC 17* indicates that the FIGURE 2 illustrated circuits 16 have been modified to receive additional signals over 17 the A-bus while supplying additional signals over 18 the D*-bus and over cable llO, later described.
19 So far, memory 100 appears to MSC 17* as an external memory. In destage restart, SSID having 21 an error flag with either a data check or no data 22 check (equipment checks) must be found. To this end, 23 a set of signals is supplied over cable 110 indicating 24 the RUA to be searched for. Inserted into register ll in re ponse to a received PCI, MSC 17* supplies 26 a PCI* signal over line 112 which inserts the RUA
27 in register 111 in preparation for a search compare 28 from memory 100. Simultaneously, the signal on .,~,j . .

.. . .
... . ~ .~ . ~ - .

~0735S3 1 line 112 activates counter sequencer 113 to scan all 2 registers in memory 100 in a known manner. As counter 3 sequencer 113 scans memory 100, a sequence of RUA's 4 is supplied over cable 101 to be inserted into register 115. Each insertion into register 115 causes a compare 6 in circuit 116 with the signal contents of r~gister 7 111. With no compare, a signal on line 117 goes to 8 counter sequencer 113, incrementing same to the next 9 SSID number of memory 100, resulting in the next RUA
to be read out. This occurs until a successful compare 11 is detected by circuit 116, at which time the signal 12 on line 117 stops counter sequencer 113 and simultaneously 13 supplies an activating signal over line 120 to register 14 115. This line 120 signal activates register 115 to supply the signal contents of the SSID register 16 over cable 121 to register 102 for insertion into 17 MSC 17*. Counter sequencer 113 now contains the SSID
18 number of the error-causing DASD spindle 14. In this 19 regard, SSID memory 100 supplies the SSID number along with the other signals to register 115 such that MSC
21 17* receives both SSID and the RUA for the device 22 in question. At this point in time, DASD QCB restart 23 or mark page in error flowchart can be entered as appro-24 priate. Note that without a data error, mark page in error i9 omitted; and DASD QCB restart is entered 26 correctly, Note that the PCI from the DASD spindle 27 14 initiates the action for recovering from the destaging 28 error.

,....
~09~5026 -109-~073553 1 In the event counter sequencer 113 sequences 2 through all of the registers of memory 100, a scan 3 complete signal supplied over line 125 sets a first 4 I bit of register 102. This supplies the interruption signal to MSC 17* indicating no error was found. Simi-6 larly, when a successful compare 116 is detected, 7 the signal on line 121 sets a second I bit of register 8 102 signifying that a successful comparison has found 9 an SSID error entry enabling mark page in error or DASD QCB restart to be effected.
11 From the above description, it is seen that 12 the registers, compare, and counter of FIGURE 11 replace 13 a portion of the destage restart function 55. It 14 should be noted that the messages to the primary host lS 18, forwarded as described with respect to FIGURE
16 8, axe still under program control and that the FIGURE
17 11 illustrated circuits speed up the scan for finding 18 the spindle in error after a PCI has indicated a DASD
19 pack has been swapped and that destage with error can ensue.
21 While the invention has been particularly 22 shown and described with reference to a preferred 23 embodiment thereof, it will be understood by those 2q skilled in the art that various changes in form and detail may be made therein without departing from 26 the spirit and scope of the invention.
27 What is claimed is:

~0975026 -110-:, . .,. - . =, . .

Claims (7)

The embodiments of the invention in which an exclusive property or privilege is claimed are defined as follows:

1. A data storage apparatus having a first storage level with a multiplicity of addressable storage loca-tions, second storage level with a plurality of ad-dressable storage locations, data signal moving means for selectively exchanging data signals between respective given addressed portions of said storage levels, con-trol means maintaining signal records of where data sig-nals are stored in said storage levels including said given portions, the improvement including in combination:
means monitoring operation of said second storage level and capable of indicating errors in said given portion of said second level; means for counting said errors; and indicator means responsive to said counting means for indicating data signals in said given portion of said first level as containing data errors.

2. The data storage apparatus set forth in Claim 1 further including in combination: means responsive to said counting means to supply a first indication that less than a given number of errors have been counted; vary-off means responsive to said first indication to indicate that a manual storage apparatus reconfiguration is to be effected; reinitiate means responsive to a storage appara-tus reconfiguration indicated by said vary-off means to reinitiate said moving means; error control means respon-sive to said reinitiate means and said monitoring means indicating no errors to erase said error count; means responsive to said reinitiate means and said monitoring means indicating an error to advance said counting means;
threshold means responsive to said counting means reaching said given number to actuate said indicating means; and destage with error means responsive to said threshold means to actuate said data signal moving means to move data signals with errors to said given portion of said first storage level.

3. Data storage apparatus set forth in Claim 2 further including in combination: demark means responsive to said threshold means to indicate said given portion of said second storage level as not available for data sig-nal storage while maintaining storage of said data sig-nals in error therein.

4. Data storage apparatus set forth in Claim 3 further including data director means for connecting said second storage level to a plurality of host units for exchanging data signals therebetween based solely on an address field unique to each host; table means for converting said host address space to real address space of said second level; and means in said monitoring means for erasing said table means with respect to said given portion of said second storage level for making data signals therein unavailable to any of said host units.

5. A program-controlled data storage apparatus having a first storage level having a multiplicity of address-able first storage locations, a given plurality of said first storage locations addressable as a unit to consti-tute a given portion of said first level, a second stor-age level having a plurality of greater than said given plurality of second addressable storage locations; means having a program memory portion and memory means con-taining recorded control signals indicating data sig-nals in said given portions are associatable; computer means responsive to signals from a control means to per-form predetermined functions; signal transfer means for exchanging signals between said levels and responsive to said recorded control signals to exchange signals between said given portions; bistable means indicating that said signal exchange between said given portions is a signal transfer to said first given portion from said second level given portion; the improvement including in com-bination: error means in said second storage level to detect errors therein including errors relatable to said given portions of said second storage level; a destage with error program stored in a memory portion signifying to said computer to execute predetermined functions when-ever a second level error is detected, said computer being responsive to said destage with error program to: (a) upon detecting an error, saving the address of said given por-tion of said second level with an indication that such error occurred during movement of data signals from said second level to said first level; (b) sending status sig-nals to a primary host indicating errors and error loca-tions only to said primary host and adjusting storage apparatus operation to prevent host access to said given portion of said second level; (c) detecting whether or not a detected error being processed is the first or second occurrence, and, if a first occurrence, effecting a re-configuration by moving a volume having said given por-tion in error to another location within said second storage level, and, if a second occurrence, initiating a data movement from said second storage level to said first storage level; (d) upon completion of said reconfiguration, initiating a destage with error sending status signals thereof to said primary host and making said given por-tion of said second storage level inaccessible to said hosts; and (e) initiating moving said data signals from said given portion of said second storage level to said first storage level with said errors.

6. A program-controlled data storage apparatus having a first storage level having a multiplicity of addressable first storage locations, a given plurality of said first storage locations addressable as a unit to constitute a given portion of said first level, a second storage level having a plurality of greater than said given plurality of second addressable storage locations, volumes of said second addressable storage locations being on movable media which are interchangeably locatable within said second level, programmable control means, means for connecting said data storage apparatus to a plurality of host pro-grammable units; means having a program memory portion and memory means containing recorded control signals indi-cating associatable data signals in said portions including associating said given portions of said first and second storage levels; computer means in said programmable con-trol means responsive to signals from said program memory portion to perform predetermined functions; signals trans-fer means for exchanging signals between said levels and responsive to said recorded control signals via a com-puter program in said programmable control means to ex-change signals between said given portions; bistable means indicating that said signal exchange between said given portions is a signal transfer to said first given portion from said second given portion; the improvement including in combination: error means in said second storage level to detect errors therein including errors relatable to said given portion of said second storage level; a destage with error program stored in said program memory portion signifying to said computer to execute predetermined func-tions whenever a second level error is detected; said destage with error program comprising instruction word indicia in said program memory portion representing data storage apparatus operations for: (a) upon detect-ing an error, saving the address of said given portion of said second level with an indication that such error occurred during movement of data signals from said second level to said first level; (b) sending status signals to a primary host indicating errors and error locations only to said primary host and adjusting stor-age apparatus operation to prevent host access to said given portion of said second level; (c) detecting whether or not a detected error being processed is the first or second occurrence and, if a first occurrence, effecting a reconfiguration by moving one of said volumes having said given portion in error to another location within said second storage level and, if a second occurrence, initiating a data movement from said second storage level to said first storage level; (d) upon completion of said reconfiguration, initiating a destage with error sending status signals thereof to said primary host and making said given portion of said second storage level inaccessible to said hosts; and (e) initiating moving said data signals from said given portion of said second storage level to said first storage level with said errors.

7. A data storage apparatus having second storage level consisting of a plurality of independent storage units, each of said independent storage units having a removable storage medium affixed thereon, each said storage medium capable of being operated upon by any of said independent storage units, control means for accessing addressable portions of each said storage media including addressing said units in accordance with address indicia associatable with said record media, a first storage level having a plurality of addressable locations for containing data signals wherein a given addressable portion of said second storage level is selectively associatable with a corresponding given portion of said first storage level, a programmable control means operatively connected to both said storage levels for controlling and operating same including assigning relationships between address-able portions in said second level with addressable portions in said first level, external connection means for connecting the storage apparatus to a plurality of hosts including a primary host and for exchanging data signals with each said host independent of each and every other host, means in said storage apparatus for exchang-ing signals between said levels under control of said programmable control means, means in said programmable control means for receiving address indicia from each of said hosts in an independent manner and translating said independently received addresses to physical ad-dresses of said second storage level, the improvement including the method having the steps of: monitoring data signal transfers from said second storage level to said first storage level for detecting errors in said second storage level occurring during such a transfer;
memorizing a detected error; stopping the transfer of data signals and supplying status signals to said pri-mary host indicating that such internal transfer of data signals has been stopped; checking for a previously memorized error signal associatable with a transfer of signals from a given one of said portions of said second storage level to a corresponding given portion in said first storage level; and, upon detecting coincidence of a previously memorized error signal, signifying same to said primary host; making said given portion of said second storage level inaccessible for data operations to any of said hosts and then moving data signals ir-respective of error conditions from said given one por-tion of said second storage level to the corresponding given portion of said first storage level; upon detect-ing no previous memorized error condition, interchanging a given one of said storage media having said given one portion with a second one of said storage media; and reinitiating movement of data signals from said given one portion of said second storage level and monitoring such transfer for an error, if no error is detected erasing said memorized error signal.