CA2053692A1

CA2053692A1 - On-line reconstruction of a failed redundant array system

Info

Publication number: CA2053692A1
Application number: CA002053692A
Authority: CA
Inventors: David Charles Stallmo
Original assignee: David Charles Stallmo; Array Technology Corporation; Emc Corporation
Current assignee: EMC Corp
Priority date: 1990-10-23
Filing date: 1991-10-18
Publication date: 1992-04-24
Also published as: US5390187A; EP0482819A2; EP0482819B1; DE69130279T2; JP2959901B2; US5208813A; EP0831400A2; AU8590491A; DE69130279D1; EP0831400A3; JPH05143471A; EP0482819A3

Abstract

ABSTRACT
A method for on-line reconstruction of a failed storage unit in a redundant array system, After providing a replacement storage unit for the failed storage unit,reconstruction begins for each data stripe in the array. General reconstructionconsists of applying an error-correction operation (such as an XOR operation on data blocks and a corresponding parity block) to the data blocks from the remain-in storage units in the redundancy group, and storing the result in the corre-sponding block of the replacement storage unit. If a Read operation is requested by the CPU for a data block on the replacement storage unite, then a concurrent Read task is executed which reconstructs the stripe containing the requested data block. If a Read operation is requested by the CPU for a data block not on the replacement storage unit, a concurrent Read task is executed which performs a normal Read. If a Write operation is requested for any data block, then a concurrent Write task is executed which performs a Read-Modify-Write sequence in the general case (the Read operation being performed in accordance with the above rules).

Description

Pl:) 0807SD

S~N-UME RECO?~lg i~ ~10N OF A FAII~I~ REIDUNDAN~ AF~RAY SYSTEM

~AcK~Rouplla OF ~ INYENTK~N

1. ~le~d ef a,e In~ntlon Thl~ InvenUon r~ to ompul~r 6ys~em ~a ~s~or~E~e, ~nd more p~rtla,~larly to a methocl ~or on-lin0 r~constru~lun ~ ~ taU0d s~r~r~go unlt In a re~unaant ~rrAy m.

2. L1~scn'ptlon o~ flol~ted hn .~A typl~l d~t~ proce3sln~ ~y~larn ~ener~ly Involw~ ono or rrK~re stor~o ~nit4 whlch ~r~ conn~ct0cl to ~ Cenlral Proeeagor Unil ~CPU) elth0r dlrocUy or throu~h~ ~antr~l unlt an~ a ctlarmel; Tno nJnc~on o~ ora~e unr~ 1~ fo sfore drlfa and pro~rams whlch the CPU u~ In pcrrormln~ p~rUcul~ da~a proce~lnE~ k9.

Variou~ type o~ stor~e unl~a are u~ed In ourr~n~ data processln~ syst3rns. A
typleal ~y~tom may Includo ono or mora lar~u capaclty 1ape unll~ ~nd/or dlsk drlves (ma~neUo, optlcal, or s~rnlconductor) conn~c~ed to the systern 1nrou~h re~pc~ctlvd oontrel unltc h~r 6torln~ dat;L

However, r~ problem exl~ts U one of the lat~e capaclty stora~e units ~8118 auch ~hat Informatlon cont~ln~d In that unK 18 no lon~r available ~o the systern. GanGrally, uch ~ l~llure will shut down th~ ~ntlre computor syslem.

mO prlor art h3s su~ested eeveral ways ol 601vln~ the probl0m o~ provi~in~
rotl~ble dat~ stora~e. In systems where recorda are relathHly small, 11 Is posslble to u~n error correctln~ codes whlcl~ ~netate ECC syndrome blle that ere appended to each dsta record wlthln n slora~e unll. Wlth ~uch codes, It le po331blo to Gorrect ~ srnall ~mount o1 data that rnay bo read erroneously.
Howo~Jer, ~uch co~es ue ~enerell~y no~ ~ultabb lor correctlr~ or reueat~ lon~
r~cords \,vhlch ~re In error, and provlde no remedy ~t all H ~ oomplete stora~e unlt f~ . Therefore, H n0~d exl~s ~or provldln~ data rellabalty axl~rnal to indhldual~toru~e units, -2- PAT~
PD~1807SD

Olh~F ~ppro~ch~ to such ~exl~rn~D roll~bil~y haive b~r~ describ~ In th0 ~n. A
re~0arch ~roup r~ Unlv~r~ C~ orrlla, ~erkcl~y, h a pap3r entitl~ "A Ca~e 10t RedundRn~ Array~ of Inexpen~lv0 Dlsks (R~D)-, PaHetson, 0t ~1., Pr~. ACM
SJ(;~OD, Jun~ l~Br~l h~s o~al49u~ ~ nurn~er of ~IHwenl ~pproaahn~ ~or psovlclin~ 3UCtl r~llablllty when ~slr~ ~Isk drlv~q ~5 slorn~ unll6. Arraya o1 dl~l~
drh/e~ aro charact~rl~ In or~ ol 7ivo ~rchltccture~, und~r thc acranyrn ~AID~
(~ R~urldQnt Arr~y~ o~ Inexpen~lve Dl~

A FiAID ~ archltecture Involve6 providin~ ~ duplicsle ~el of "mlrrorn eter~ge unlts ~ncl keepln~ ~ dupllcai~ copy ot all da~Q on oach p~lt ol 6tor~0 uni~, Whlle such ~ solutlon sc~ h0 r~ blllty probbm, It doubl0s the cost o7 stora~e. A
numb~r 3~ Irnplementatlone o? ~slD 1 arc~tecture~ have ba4n mado, In puticulsr by T~nderrl CorporaUon A RAID 2 ar~hltacture etore~ eaoh ~t ot eash word o~ ~t~, plus Error D~toc~on ~d Çorroctlon (EDC) bKs tor e~oh wotd, on ~para~e disk drlvos (thla la ~Iw 1~ known as ~blt ~trlppln~"). For ex~mpl0, u.S. Patent No. 4,~22,0~510 Flor~
dlsc~oss~ 8 dl~k dr~ve rnemory usiny ~ ~ura~y of r~lathrely ~mall, Ind~p~ndentlyoper~tln0 dlsk subsy3tcms t~ hmcthn a8 a Inr~o, hlgh capadly dl~k drlve h~vln~
an ~nus~lly hl~h ~a-4t tolQrAncq ~nd a very hl~h data tran6rer banrJwldth. A data organlz~r add~ 7 EDC blts (determlned uslng the w~ kno~Hn Hctnrnin3 codo) to ~Q e~ch 32-blt data word to provWe crtor delection and error cor~otlon c~ablllty.
Th~ resull~n~ 3~1S word Is wrltt~n, ona bl~ pat dlsk drlve, on to 39 dlsk drlvcs.
Il or~ o1 tl~ 39 dlsk ddvo~ , th~ rarnslnln~ 98 blts ot ~ch r~torad 3~bH word ~n bo u~ed to recorwtr~ct e~ch 32-bH da~ word on a word-by~ord ba~i~ o~
~ch data w~rd Is re~d Irorn Ihe dbk driv~, thereby obl~nin~ fault toler~nce.

2~ An obvloL~s dtswbsck of suoh a systcm i3 th~ l~r~ numbet o~ dl~k drlvos roqulrcd ~or ~ mlnlmun sysl~rn ~slnoe most l~r~e compul8rs use a 32 blt word), and the rr~latlvely h~h r~lo of drlves requfred to dor~ thh FDC blls (7 clrlvcs out ol 3~.
A further llmltatlon ol ~ ~AID 2 dlsk drlva memory sysl3rn i~ l lhs indlvldu~l dl3k ~ctu~tors Jre op~r~ted In unlson to wrlt~ e~ch data block, the blt~ ot whlch aredl~lrlbutod over all ol the dl~k drlves. Thls arran0~m~nt nas a hi~h d~la lr~ns~r -~ PATI~NT
PD~7Sr~

banslvvld~h, ~Ino~ o~cll indhldl~r~l diek tr~laf~ part oS ~ blrx~ of dal~, tt~ net ~ect bsln~ t~)a~ ntlr~ block 18 ~lablo to ~he computer sy~l~m much ~a6~er ~hs~n 11~ 81r~1a drlv~ w~3r~ rlc:aa861nt7 IhO b!~?ck. ThlC 18 a~van~a~o~ or l~r~a d~t~ !o~. ~lev~r~Y~r, Ihle arran~emar~ o e~lecUve~ provldr~s ~nly ~ sln~la te~twtlla hr~d ach~ator 1Or Iho entlra 3~e9E~ llnn. Thl~ ~dv~t~oty ~ t3 1ha r~ndom aoCra~ perlo~man~0 ~f ~h0 ~r~ arrar~ n r~l~ s ~re small, ~ince ~Iy on~ nl~ al a tlme cen be aac~s~sd by thu '~In~lo" ~Ictu~tor. ThUs, RAID
2 ~t~rn~ ara ~ensrrllly not oonsld~red t~ i~a sult~bha ~r c~mput0r sy~tcm~
deel~r~ ~or On-Une Tran~sacSion Proc~slng (OLTP3, ~uch ~ in banklng, t o ~ncl~l, ~d r~er~lon ~ys~mll, w~re a lar~o numb~r ot rundorn accessos to .~ rna~y err ~11 do~ fllos r,~rnprls~s ~he bulk ot d4t~l g~ot~a ~ Ir~ns(er ~poraUons.

A FtAlD 3 archlt0clure 1~ i~aso~ on the concopt 1h~t ~ch dl~k drlve s~ora~c unitha~ Internal rn~ans 70r det0ctln~ ~ huit or data ~rror. Thcrator~, n h not necsss~y to etore exlra lnfottnatlon to detect the locatlon ol an ~rror; a slrnplor fo~n~ of parlty-basod error corrocUon can thus be used. In thls epproach, th~
oontents of all ~lora~e unlls sublrlct to ~ilure are ~Excluslve OR~U (XOR d) to yon~r~te psrlty InlormaUon. The resulUna parlq,~ Intorrnatlon Is stor~d In ~ 91n~19 r~d~ndAnl otortA~a unl~ tor~go unn ~al~ Ula d~ta on th~t unlt c~;n be r~con~tructed on to a te~!~ceme-t stora~o un~t by XORln~ Ula deto from lhs remalnlng StOrQ~e unl~ e pnrl~r In~onnatlon. Such an ~rrang~rnerlt h~s the adv~nts~a over ~he n~irrored dl~k RAID 1 archH~cture In lhn1 only one ad0Uonal stora~o unlt ~ requlred for ~ torago unrts. A furthor ~poct ot th- ~AID 3 arehiteotw~ 19 th~t the dl~k drhleQ ate opor~ h ~I cc~ d m nner ~kr~ to ~ ~ID 2 systorn and ~ oln~b dlsk drlve le desl~nal~d s~r thlr p4rlty unlt.

2~ One ImplernontaUon of n RAID 3 erchltecture 13 Ule Mlcropoll~ Corporatlon Parallel Drlvo Arr~y, Model 1BO~ SCSI, that U808 four p~rallel, synchronlz~d di~k drlvr~sand one redundsnt psrlty drlv4. Ih~ ~allure ot ono ot the four data dlsk drlves c~n ba rern~dled by the u8~ o~ tho p~rlty blts stor~d on th~ parlty di~k drlve. AnoUl~r example oi a RAID 3 systQrn 1~ deocri~rJ in U.S. Patont No, 4,092,732 ~o Ouchl.

~~ PAT~
PD~7SD

A ~If~ Isk drh~0 m~rnory 3y3t0m h~ a m~l lo~er ratlo oi r~dundan~r unlts to ~at~ unlt~ ~han a R~ID ~ ~ysl~m. ~lowsver, a RAID 3 syss~m hs~a 1ho c~m~
p~r~rm~ncs llmllr~tl~n ~s a R~ID ~ ~y~m, in Ihat ~ho indh1~ual dls~ actuator~ ~;re ~up1~d, s;~p~r~tln~ h un~on. Thb ~v~rso~ arrec~ ~e r~om r~cco~6 pa~ n¢e o~ dr~ rr~y ~h~ dat~ Sil0c ~r~ sm~l, ~hc0 ~ on~ d~ nie tlrn~ ~n b~ Ir~ tor. Thu8, FUUD 3 8y~
n~r~1y not ~nci~er0d t~ ~ ~ult~bl4 ~ ~omputor ~yst~m~ d0~i~n~ci for OLTP
qs~

A ~ID ~ arr hll~cl~ra uses the ~arn~ p~rity ~rror corre~ion eonccpt o~ tho i~AiiD

3 ai!chlte~llre, but knprove8 orl th0 ~rlorm~ o~ a ~iD 3 sysl~m ~ h r0~j,~ct to ranciorn t~adlrl~ o~ smai~ filr~ by ~uncouplln~" Ul~ operatir~n o~ the Ir~iYiciual di3k drlve ~tuator~ i r~4dln0 an~i wrlnn~ r~cr mlnlmum unount cf dst~
~typlwlly, ~ dl~k s~or) to ~ach ~IQk (~hb b al~o known as bl~ck vtrlp~ ). A
lurth~r ~sp~ thr~ RAICI ~ uchitecturrs 1~ that 0 slngle dora~e unit 1~ d~;natod thx p~rlty llnlt.

A llrni~Uon ol ~ ~D 4 ~6tem i8 Ulat WrlthD a dat~ bhck on ~ny o~ tha Indcp~nd~ntiy operalln~ dtila slors~;~ unlts also requlre~ wrltln~; a now pamy i~lock on th~ parlty unlt. rh~ parity ~nhrn~Horl ~tued on the parity unit must ba read and XOR'd wl~h tha dd dat~ (to ~remove~ Ihs inrorm~lon oonlont ol tile old dala), and the resultln~ sum must then be XOi rd w~ the n~w ct~to (to i~ro~td~ n~w p~rlty In~ormation). Bo~h the d~ta nnd the parlty rocord~ ~hen nNJ~t ico r~ten to lhe dl~k d~l~ mis ptoc~sa i5 cornmonly re~rr~d to as a ~Read-Modity-Wrlte"
s~qu~rK~e.

Thu5, ~ Ro~d ~nd a Wrlte a- the shgie par;ty unil oc~urs each tlrne a record 1 2~ ctlangeci on any ot the data storaSic unit~ covor c by the parity rocord on the patity unlt. ll~e p~rlty unlt becomo~ ~ bottlof~eck to data ~ritinSi oper~tlon~ since ti~ number et chan5;e~ to reoorcls whlch c~n be m~ie p~f unit Gt tlrne Is a function o~ the ~ ee r~te of tile parlty unit, r~ opposed to the f~tsr acc~ss rate provicl~d by parailel operation ol the rnultiplo data stor~ge unlts. 8ecaus4 ol tt~is llmlt~ n, a flAID 4 ~y~tem is ~nsraily not con~idorod to b~ sullablo lor cornputor A~r PD~8078CI

~y~t~rno ~l~d ~or OLTP p~rpo~a~. ~n~e~d, it appe~r~ Ih~ a ~ 4 æy~em h~ not b~n lmplem~3nted ~ ~ny oomrnerci~l purpose.

A FWD 5 archlt~re U~Q~ t~ g~me p~y errot corrscalon conCopt o~ the FU~ID

4 ~chlt~et~o ~nd Ind~pen~ tors, bu~ lmproves orl ~e wrl~ln~ p~rlorm-~!lnC~D 0161 ~D 4 ~y31em by ~IstrlbuUng ~he ~ata ~nd parily ~formatlon aCro~S eJI
0~ ? evall~bl~ dlsk drh~. l'yplo~lly, ~iN ~ 1~ stor~e unll~ ~n a ~ot (~Igo h~owna ~edurldan~y ~roup'r) ~r~ dvid~ ir~o a plur~lity o7 0qu~11y ~k~d aWre88 ar~ 5 r~rted ~0 ~ blod~. EaC7~ stor~Elr~ unl~ ~n~rt~ cont~n~ tho ~ame num~r oi block~, ~bck~ ~rom e~h ~ora~ uni~ In ~ ro~undancy aroup h~n~
.. th~ ~em0 unlt addra~s ran~es ~re re~err~cl to ~ t~lp~. E~ ~po har N
block~ ol ~ , ph~ ono p~ y block on on~ ~tor~g~ devlco conlalnln9 p~ity lor th~ r0m~lnd~ o~ th9 gtrlpe. ~:urlh~t ~tr~ e~ch h~vo ~ p4rlty block 1h~ parlty blooks belr~ dl~rlbutad on ~ner~nt 6~ra~ unlls. Parlty updstln~ activily a~o~l~ted ~Am every modff~ on ot data In t~ rodundancy gr~up 1~ Ulerororo 16 dlstrlbuted oYor ~he dm~rent gtorage uni~. No ~h~l~ un~ l6 burd~ned wlth all ol U~ parlty updat~ actlvlty.

For ex~mple, In ~ RAID 5 ~ystern compt;sin~ 5 disk dtihre-, the p~r Inform~ltion~or Ule flr~t strlpo o~ bi~cks m~ aq ~ e flnh drlve; lhe parHy htormallon ~or the ~acon~ st~i~a ot block~ m~y be uni~tton to th0 iourth drive; Ul~ pasity ~0 Inf~rrn~Uon for the Ullrd slrlpe o~ blocks may b~ wrltt~n to U~ lrd drivo; 6tc. Tho p~rl~r biock ~or su~c00dlng stripa~ typloally ~'prec0sses~ around the disk drives in a h~llc~l pattem (ai-nouEih oUler patterns may b~ uaeci).

Thu~, no sbn~le dl~k drlve 18 u~ or slorln~ ths pcrlty InlorrnaUon, ~nd tile ootti~e nock o~ e RAID 4 archil~otur~ is 011mlna~ad. An ~xsmple o~ a i~AiD 6 syst0rn la de~crlbed In U.S. i~0nt No. ~ to Clsrk et ~/.
~ 7~7~ 7~
A~ In a ~AID 4 ~y6~ , a llrnlt~tlon o~ a iWD 5 ~y~tem i5 tha1 a cJl~n~e In ~ data block r6qulre~ a ReadModHy-Wfito s~quonc~ comprl~in~ oad and two Writo oper~tlons: tine old parrty bl~ck an~ oid data block rnu~t bo read snd XOR'd, and the resulUng sum mu~ en bo XOR'd wrh the now dala. Both Ih4 d~ta ~nd PATE~
Pi3~07St~

~o p~rity blockE~ rl mu~ b~ rowritt~n 20 lno dlsk ~riV~g. 1~110 Iho lwo i~ad oporaSlon~ r~y b0 dono In p~ali-el, ~ n the h~o Wrlte opcratlon6, mo~ifi~bn ot a block o~ ~ia2~ ln ~ RAID 4 or ~ D ~ rsys~rn ~till ~k~ sub~t~nli~lly lon~er ~n lho ~m~ op~r~Uon on ~ convonlion~ k. ~, oonvonlio~3ai di3k doo~; not ~ulr~ tho pr~llmlnary iR~sd op~ralion, ~nci th~ doss hav~ to watt for Iho dbk sirlve~ to rota~e back ~o th~ prevrJu~ posHion In order to i~ri~orm tiho Writ0 op~r~lon, Tho ro~thn~l 18t~1i~y Um~ ~lo~ c n ~mr3un~ ~O ~t 50% c~ o time r0gulreci 70r ~ typi-cal da lncauon o~raUon. Fu~hor, two dlck sto~-o ~It~
ar~ Invo~ed tor ihe dur~lon of eaeh d~ modHlc~Uon op~ra~ion, limltlng ~e thrs:~u~hput o~ tha ~ystem ~8 a w~ol~

~ e~pi~ Ihe Wrlto por~rmz~nc~ penalty, R~ID ~ Iype 3ystam~ hava b~comc Increa31n~1y popular, ~Ince ~hey provl~o hi~in ~t~ rcliabllity with ~ low overhead c~et ~or rcclundancy, rgood P~d perforrnanco, and rair Write i~ormance.

A RAID 5 ~rchl~ecture ha~ prlr~lcular utl1ity In OLTP cennput~r sy~tem~. Many OLTP syst~ms must be hlgh-avallabUity systerns, meanlng that complcle ~llura o~
y~tem hac a lu~-probeblll~v. H10h ~v~ilabillty csn ~ ach0~ 1 by usln~ hi~h-rellablllty compon~n~, hnvln~ a tault-tol0rant de~lgn wiul ~ low mean-Ume-to-repalr (MTTiR), an~ do~l3nln0 t~r ~ag~i" d radaUon, where the 1allurc ct 8 component may r~d~-co cy6tem ~p~blllty but wllhoul c~usln~ total sy~lem tsllure.
Aithou~h a prlnclp~l teatute o~ 8 ~ID system 1~ ~ault-tohrance, such capabillty alonr~ does not guarantee a hbh-avallabUlty systenL 11 a stot~e unlt falls, 5~0nusl ~y3t rn operatlon ~sannot contln~J~ until tho hll~d ~torage unlt i-~ r0plac~d and the Intorrn~Uon on It 1~ restored. When a ~tor~ unlt taUs In a R.IUD tirchlt0clwo, thc ~n teaches that a r~plaoement 6tora~e unn 18 subsUtuted ~Of th~ falled stara~0 unlt (elther manu~lly er el~clronlcslly 8w1tched In ~rom a set ot onQ or more ~p~r~), and th~ Ylost~ data 1~ reconstructod on tho topbc~m~ttt slora~ unlt by XOR'lng each parlty biwk wlth all corr~spondln~ dala block~ Irom U~e r0malnln~
stor~e unlt drlv~s in th~ redundancy ~roup. miS oporation l8 strai~forward 1 th~ ontlre r~dundancy 3tOup can bo dedlcated to reconstructin~ Ihe lallGd stor~eunlt. However, 11 appear~ that the art t~aches Ot OUÇI~J08t6 r95toting the d~ta on PD~07SD

Ih~ r~pl~comont storag~ unlt as a ~epara~e procsdure be~oro ~naral ~ystern operatlon oan b~ recommenc~d. S~ch a mo~hod for rooonntnJcUon do~ not provld~ ~r hi~h av~ilabllity.

Thu~, ~o far as known, Ih~ an doe8 not ~r~oh roeons~ tin6i a ~aileci 3tor~ge unit S In A RAID uchltectut~ whlle con~ulng to provlde on~lna a~Ce~ls lo the enUre reciundancy ~roup.

n thue ~w be hl~hly d~sirat~le to havs ~ RAlt) archltecturo ~y~tcrn h ~vhkh reoon~truction o~ n f~ d slora~0 un~ 4n be cond~i ~on~iino~, whlle genernl ~ ~yst~m operaUon conUmles In a norrr~ bshlon. K 1~ c deslrsbie to ho ah h ~ r ~ power f~ilure with the rr~construction opor~ion tn progross 9~ never pt00ent b~d data ~ ~ooc d~4 to the CPI.I.

Tha preson; hvDnUon provlc~r~ euch a oystem.

PATE~
P~)~807SD

~UMMARY ~F ~}lg~ IIMVE~I~N

~he pr~nt InvenUon provld~ ~ msthod ~r on-l~no r~nstruotion ol ~1 lallad ~1~3 d~a s~or~de unlt. All0r proYldln~ ~ rap~ac~ rl~ g~sr~e unil lor the hllad ~or~e ~1~, ~ r~constr~on l~sk bs~ln~ ~or th0 ~rr~y. 13~neral r4t ~n~trwUon con~l6t~ ol ~pplyln~ ~n 0tror~Drrs~on 4p~r~0n ~s~ch ~ an XOR opor-tbn on dat~ blocke ~nd ~ ~rr~spondln~ parlly block, or ~ Read SolorrK~n co~) to tll~
block~ ~rom th~ r~m~lnln~ slor~a unl~ In lht~ radunclnncy ~roup, ~nd ~torln~ rs3ul~ In the corte~pondln~ ~lock o~ Ule replacement ~tor~e unlt.

d oper~Uon 18 r3qu~3t~d by Ulc CPU 70r P da~ block on th~ replacemon~
3~0raga unlt, then ~ concurr~nt Read ta~k 18 executad ~h reconl~buds the oont~lnln~ re~u~st~d d~l~ block.

U ~ R~ad op~ratlon 1~ requo~t~d by 1h~ CPU for a d~ta block not on the r~pl~cement ~torage unll, ~h0n 6 c~:>ncurrent ~ead t~k 1~ ~x~ d whlch p~rtorrn~
a norn~l R~ed.

16 111 ~ Wrlte op~ration i~ rcq~ostod ~or any ~ata block, ~hen a concurrent Wrlla task lo ex~cut0d whlch perforrns a Recd-M~ Wrlt~ ~qu~n~e In the g~naral cas~
(ths flead opsratlon belr~ perforrned In accord~nce wllh the above rulos).

AKi~ou~h d~t~ block~ on Ihe r0pl~c3ment stora~e un/l "~bove~ the reoonstruction ~rlpo ara ~alld, ~rKi could bo raad clrcctly, the above procadute h easl ~nd ~0 simpler to Irnplemsnt, slnce thore b no nead to koop u~ck of whether or not n rcciueet0d data block 18 ~bove~ th~ r~con~tructlon drlpe. However, ln an aKett~tlv~r embodlmr~nt d the Ir~entlon, the po~ition ~f t~ reques~ed data t~lOCk with respect to 1h~ reconstruction etrlpe 18 tracked, and reconstruotion 1~ onlyperforrnoc for d~ blocks th~t ~e no~ above the ~econstructlon slripe.

Thu8, t~e general ~ies of operatlon In the pre~erred enl~oclment aro lh~t ~il R~ad operatlons trom the replacernent storage unlt raqu~e on-the-fJy raconstructbn olall ol the data blocks request~d, r~3ardl~ ot wheth~r any ol tho~0 data blocks ~~ ~ PATEtJT
PC)~D

havo In ~et been recon~r~ by the ~nsral r~cons~rucUon proCcs~. A~ wnte oper~tlon~ proc~ed In a Resd-ModHy YVrilo ~equcnce in ~he ~eneral case, with ~e Rs~ pAtt of U~u op~aon boln~ per~ d In laoeord~nce ~th 1he ~bove rulos.

Tho pts~ent Inv~n~lon provld~ 4 tellable meth~d or tecor#~ructin3 all o~ U10 data ot al l~llgd ~tora~ unll ontc~ ~ rcpl~cemenl unil, wllUe contlnuin~ tO pl~rmit R9~1n~ and Wrltlcl~ opqraUon-Q t~ 8~0ra~e uni~ ~rray ~s e whole, ~md to Uu~
roplaoement ~toraae untt ~n psnr~l~. U~ ot a flao permlt8 detecUon Ot en abnermt~l tennlnallon o~ eneral rooonetruc~on procegs. The rocon~ruoffon m&y to~um~ ollher at the InlUsl recon~rucllon st~ln0 polnt, w, optlona~ t the .~ polnt ot obnorm~l lotmlnaUon.

Tho ~etillb o1 ~he prelelred embodlment ot the presenl InvenUon ~re set 1sr~n Inth0 ~ccompanyln~ ~r~wlng~ ond the dflswlptlon below. Once the detall- d tho Invsn~lon tlre known, nurnerou~ eddKlonal Inno~allon~ t~nd ch~n~es w111 become obvb~ to one 6kl~ed In the ~

- 1~ PA~ENT

Bi~lEF ~ESC~IPTIION CIIP T~E l)RAWINS35 F1~3lJRE 1 18 biook dla~r~m o~ a ~ llzod FUU01 ~y~lam In t~rd~nce wffll the pt~s~nt Inv~nllon.

FitlUi~E 2/1~18 a d~rBm ot ~ ~d01 RP~ID 5 ~ m, 3howin~ an Initial ~a~.

6 FI~UF/E ~B 1~ ~ dl~tarn al a r~l RAIO 6 ~y6t~rn. showlny a lali0d 3tora~-ie unit.

FI~URE 2C ~ a dla~rrlrn ot ~ mocl~l RAit~ 5 ~y~tem, ~ing an hlt~-lz~d .~rephc~ nt stor~e unit.

Flt~lURE 2D Is ~ dl~gram ol r~ mod~l RAID 5 ~yst~m, showin~ n p~Sia~iy r~con~ d r~plac~ment elora~ ur~

F1~3lJRE 2E h a dla~r~m ot ~ moci~l RAID 6 systorn, ~howir~ ~ compl~y roconetl!uded replacsrnenl ~l~ra~e unl~

Fl~lJRE 3 ie a ~iowch~rt ropr~wnlln~ the ~ener~ reconstsucllon proce~ Oot lhe pre~erred embr~lment o~ the ptesent Invsntbn.

Flt IURE 4 1~ ~ llowch~rt representir~ ~ Re~d op~ratlon tor the preterred embocil-rn~nt ~ the pre~ent Inventlon.

FIQURE 5 18 ~ llowc~lart repru~en~n~ a Wtite opsr~Uon ~or tite preferred embodl-m~t o1 tho present In!~rentlon.

Uk6 re~sronce numbers and ~signaUons in the drawlng~ re~er to liko ol~nts.

PATE~
PC~B07SD

~'~AI~ PTlO~ nrlO~3 rhr~u~hou~ Uhl~ d~3crlpllon, Ul~ pro~orro~ em~oclrr~nl and ~xamples shown ~houlci l}~ oonsld~r~ Xem~ls, r~h~ than llmi~al;on~ ~n ~ m~lhod ~1 ~e ~es~nt In~onil~n.

~3 Flt:llURE~ loc~ cllr.~ram of ~3 ~cnsr~ ys~m in accofd~nc~ e pr~ n~ invenUon. Shown ~r~ ~ CPU 1 s:~uplod l~y a ~ o ~n array corllrdlr~r 3. The ~rr~y oonbroll~r 3 1~ coupled to ~aeh o~ Ih~3 plurall~y ol s~orc~ un~ S1-s~ (nY~3 b~ln~ ~hown by w~y o1 ex~mple snly) by an 1/0 bu0 (e.~ 3(~31 bu5).
rr~y contrull0r 3~r~bly Includes a ~;epar~t~ pro~rammablo, multl-t~sklng proc~aor (~W 0~ampla, th0 MIPS R3000 RIS~ pr~o~s~or, made by MIPS C~ o~, of Sunnyv~10, ~all~ornls,) whloh ORn ~ct Indep~nden21y o~ tha CPU 1 to contr~l t~lo ~tor~e unlls.

The ~tor~g3 unlt~ S1~5 c~n be ~roupod Into on~ or mor~ ledundancy ~roups.
In U~ Illu~trat0d oxampl~s d~gcr~ elow, ffle redundanoy ~roup ~.-nprl~ ~l ~5 of th~ ~torago unl~3 S1~6, 10r s~plcity of sxplana~n.

Th~ present hvenUon i~ pref~r~bly Implsm0nted as 1 m~N~-ta~klng oompular pto~ram cxocutod by the conlrollor 3. Flt3URES ~A-2E are d agrammatlc mr del~
018 RAID ~ sy~tem. FIGUR~ 319 a hl~n-hvel flowchar~ tepresentin~ U~e steps o~
the ~ner~l reconstrwtion prooe~ ~or th0 prererred embodlmont or the invanUon.
~0 FIGURE 413 a Ill~h~Q~rQI 11awchart reprcsenthg the 8t~p8 0~ o~d op~rohbn fo~
tho proterr~d embodlment ol th~ Irvention. fl~;URE 5 is c hi~h-iovol ftowch~t ropretor~in~ the 8top5 01 a Wrlle opsr~tlon tor 1he pre~err~d embo~lmern o1 tt~
Inv~nUon. ~he steps re1erenced In FIGURES ~5 r~ro reler~nced bolow.

(3~n~ral Fieconst~ on 2~ FIGURE 2A Is a ~ia~ram o~ a moclel RAII) y~t~m, showin~ an iniUal s~t~. Th3 Illu3tr~ted arr~y comprl~ fh~o stora~e unlts, Sl-S5. Esch tow A-H i~ ~ strlpa.
Purlty In~ormaffon 1~ Indicated by clrcled num~srs, and i~ ~pr~d throughout tho array Allho~gh on~ p~tt~m of parlty Inlorrneti~n dblrlbutlon 13 showr~, lho ~D-~7~

Inv~nllon ~ncomp~ ny dl~rlblJthn. {:n~ b,it "~ock~ ~r~ s~own ~r e~ch ~tor~ unlt In ~ ~trlpfl lor ~impllcJ~ ch blOCk coul~ In~e~ci b~3 ~ny ot~0r uni~
~7 ~Iz~, a~ch ~ ~ byt~, ~e~tor, ~ ~roup o~ 20r3. In Ihe pre~eneel ~n~o~m~n~, ~oh bl~lt 1~ at l~a~1 on~ ~ctor In ~t2e.

~IQUF~E ~B ~how~ ~h~ ~m~ m~r~0i ~ FK3 URE ZA, ~ut ~Ih ~ra~D unlt S~
h~ ta d (Ih~ cl~h4~ repr0~ntln~ comipbDci dat~ ote ~t ~ XOF( a~n ol ~oh ~brlp~ tor e~r~e un~ 2~ ~s~ J6 th0 lo~ da~e ~7 ~tora~e urlil Sl ~wn in Fl(~URE 2A.

.FI~31URE 2G show~ th~ ~rno ~ID mod~l ~3 FIQURE 2~, bu~ ~ilh I~lied ~toru,~o Wllt Sl havln~ bei n r0p~ce~ r~p~c0ment 6tora~e uni~ S~'. The r~pla~em~nl may b~ on a physlc~l b~3 (I.~., a n wv ~or~o unlt b~in~ FYny81ca~ lltut~d tor ~o f~ unlt), or on ~ lo~ cal bs~l~ Q.o., ~ 6pare ~tor~e unll h oommunicn-~1~ wlth th~ ~:ontroll~r 3 beln~ elsctronlc~ wltch~d into opcratlon). In ~H
ted 0mbodlmont, th~ repl~cem~nt 3torage unlt Sl' ~a~ nol b~en Ir~lz0cl 1~ lo any par~culw data pattern, a~ Indlc~1ed by tho x'~, b~ut tho urit could h~ rmy p~tt~rn ol In~ormaUon on 11.

Ono~ repl~cem~nS ~lora~e unlt S1 ' 18 substltut~d lor the !a!!sd otorage ut~t S1, n~ral r~ nlctlon prooes~ commonco~a a~ ~ task (FIGIUFIE 3, St~p ~O).
Tho roplr~cement e~ora~o unlt S1' 18 mark~ int~rnally to Indlc~to that U~e ~torrd~o ~0 unlt b "Under F(econstnurtlon~ and ~eh~ r~}construct~d (Stop 31).

FK3URE ZD Illuslra~ r~n Interm~dhte ~ta~e ot rccons~ruction, wlUl stripe D, shown hl~hll~l~t~, S5 tha rr cor~truc11On strlpQ. In ~e pr~lerrecl ~mbodlmcnt, a WtNo oporstlon 1hat requlr~s recon~tructlan o~ a slrlpe wnnot t)e m~e to Ihe rccon~
sb~uc~lon ~trlp~ b~auyo that strlpe 18 "lodcod~, In known ~a~hlon, to pr~nt ~6 ~Iteretlon ~Nhile under~dr~ rec~r7~tructlon (Step æ). Th~relore, the Wrlte operatlon Is delay~d Im~U th~ stripe 1~ reeonstructecl.

In th0 pre~errecl smbodlment, the data and parny ~locka are Raad trom stora~o unR~ S2~3 ~or cach strip- ~H ~n thQ array ~nd XOR'd, with tho rc9ult from ~ach ~ 3 PATENT
PD~7SD

~trlpo b~ln~ ~lofod In 11~ oo~rc~porldln~ bl~ck ~f llhs rap~c~me~1 ~tOTa~9 urilt 51' (St~ 33). In ~ 4xP~mple ~ihown In FIGIJ~E 2, ~hl~ pr~s bo~ns nt ~p~ ~4 ~nd pr~ods to ths ~nd o~ lhe ~lor~g~ unlts, ~ ~trlpe ~. How~vcr, any ~ ng poln~ rney b0 u~d, wl~h any p~ m o~ recon~b~ctlon (9.~ Itorr~tln~ ~ip9s).

1~ con2r~7~r 3 U~sn unlocks ~ r~con3~rwUnn ~Iripe (S~op 34), and adv~nco8 tho rooonstr~n ~trlp~ (~3t~p 35). 1~ rer,~ uucilon b 7ncompl~ S~op 33), Ule ~nor~l reconettuc~on prooGss con11nuea (S~ep 32~ unUI U~e en~ire repL~ment ~tora~e unll S~' 13 reconakuc~d, a~ shown In FIGURE 2E. Wh~n the ~eneral ~econ~lrucUon proco~s k cornpleted (5t~,p 36), the ropl~earnenl etot~e l~nlt S1'la mar~ a~ ~No~ Und~r R~construollo7r (Stop 37) ~nd ~11 opor~te normal~ ~Step 3~).

N Ihe reconslluctlon ,woce~ 18 Into~p1ed dw to a power fdlur~ or any other re~on, the lAst st~p o~ m~rkln~ tho roplacemant ~ora~ un~ St' as aNot Und~r PlqconslrLK:tlon'' wlll r~t ba complet~. It ~ thcro~ore po~lole to d0te~t such ~n 16 ~bnorm~l terminatlon. In woh a or~0, the conlrollc~r 3 may resbrt the rer;on6truc-tlon ptoc~ from the ~e~lnnln~ without lo~lny any data. Thl~ may r0~ult in cortoctln~ ~trlp~ that h0ve alr~dy been wrroct~d, but thla ~pproacl~ ~uarRnt~s Unal ~e array s~syst~m wNI never Indlcst4 that dala i~ aood ~hen It h~ not bo~n r~construot~d.

In an alternaffve ombodlrner~t, after each seoUon of the replacement ~tora~o Unit tc.~., o~ch cylind-r ot a disk drive~ le recon~ucI~d, a na~ n0l~ In the replacemen~
~tora~e unlt S1' (or elsewh~e, It deslred~ b marked int~mslly lo Indkale ~a I~st t h~ be~n recor~tructed. AJter an abnorm01 tarminatbn, 1~
reconatructbn process rnay be restarted aftu the 1ast indlca~ed sectl~n, Ulefeby2B oevlng roconottuc~'on tlmo, Conc~lr~nt 1/0 T~k~
Durln~ ttle ~en2r~1 locons~Notion proc~ss, proG~sin~ by Ihe CPU 1 can contlnuo, bu~ wlth some poss ble p~norrnanc~ dagradab'~n ~OI th~ compu1er ~ystcm. ~3 d~wlbod bolow, the general recans~tuclhn proc~ss rn~y be Interrupled to perrnK

PATENI
I~D~7~;D

Fl~d or Wrlt~ op~(atlorl r~c U~t6Ki by the CPU I ~ ps 33, 3~). Such Int~rrupllon~ ar~ pr~ bly Impl~rnq~n~ec~ by rne~n5 ~ urr~nt ~sks h woU-hrlo~n i~hlon. ~ Y~r, th~ InYflnUon ~suornpf~r~Fs E3ny means ol oencunr ~nd/or In~rrupt-dr~n 1/0 op~raUon. F~l~win~3 ~ a d~crlpUon s)f ~0 prnferlad emi~dlrrler~ ol ~udl ~d ~nd Wrlhs op0rdior~.

R~d op~ta~lon 1~ raq~ toGi by Iho CPU 11 dutln~i ~snaral ~ecor~str~ction, a Read pro~3s~ t~Eiln3 ~ Em Indepondent bln concurren~ ~sk (---IGUFIE " Step 40).Tnl~ mo~n~ ~hal ihe reconslructlon t~k contr~uoa ~o exeCUtO, but Ulat ihe ~ad .. 1~ak and recons~uctlon la~k conlond ~c~r ~ygt~m rcs::1uroes, In known ~ashlon, lhe controller 3 pre~r~bly locks tho rocon~tructlon ~tnpe ~o pro~JQnt Inadvonr~n~t~rRblon o~ the sttlp~ undor~olng r~construcUon (Step ~ e F(~d rcquost Is 7er ~ dst~ block not on U~e roplac0ment (~ "n~) storage unR S1' ~Stap 4Z), the controll~l 3 par~rmg a normal Re~d opcr2Uon ol~d trr~n~le~ the da~a block h ~e CPU 1 (St~p ~). Boc~u~o d~ta ~rom the othe~ s~orr~ge unlts ~S2~3 In tho 0x~rnple ot Fl(~URE 2) 18 corroct, no lo~ o~ p~rformance occur~. ~he oonbroaar 3 th~n unbck~ U~e ~cens~lon strlpe (St0p 4~), and Ula R3ad ta~k tormlnate~
~;tap 45), 1~ ~ho Ro~ oporatlon r0qu~st~d by the CPU 1 Is for a dat~ block on tho rapl~c~
~0 rn~nt ctota~s unlt S1' (Step 42), tho procodute is diHerent, Th~ cornroller 3 Ir~tead reCOnstrlJC[s tt e stripe c~nt~lnln~ the requsst~ data ~loc~ ~on~
Thq controll~r 3 reads all of the other data block6 and U~e correspondln~ pa~itybbck In th- ~trlpc (Step 47). The bbck~ are th~n XOR'd to genora~e a correc~ed dsta blocl~, whlall Is tt~an trnns~rred to the CPU 1 v~a the controllet 3 ~St~p 4~).
2g Thare~ner, the controllflr 3 unlocks tho recor~ttuoUon sttlpa (Stop 44), and ~ne Rsad t~4k t~rmlnates (~tep 4~).

FIC~URE 2D Illustr~tes both oasos of Rer~dln~. Strlpe D, shown hi~hllyht~d, 18 the r~wrl6tructlon st~lpe. Il a Read rcquest 18 ma~e ror data ~lock ~52, tl~e conlroll~r PC1-~O7~;~

3 wlll aca~s ritc~r~ nl~ S~ In ~ pe A, ~nd dl~cl~ r~ bln~ry O firom Ih0 ~tc)r~0 llnit iclr 2r~n~r le ~h~ ~PV 1.

1~ a Fi~a~ r~u~r~t 1~ m~ r cl~ blo~k ~1, whlah 1~ not y~t r~ tructæd, th0 eonlr~ r ~ wlll acc~sa ~a~or~ ~nl}~ 32~ h strip~ tJ ar~ r~çi ~1 ot 2hr~
71~ bl~ck6 ~e til~n XOF~'d, ~r~n~r~ln0 ~ blnary U, which ~a ~r~na~err~d to 1~1~ CPU
.

~Irn rly, 1~ ~ Read re~uesl Is masl0 i~r ~iat~ bloc~ c~nlralbt 3 ~tal ~c~ar3~ etorage unl~ 5~5 In ~ripe ~ anc3 rea~ all ol l~O bl~ck~. ~e bl~ck6 aro -~ then XOR'd, a~ner~ln~ a blnary ~, ~rhlc~ ~ tran~?0rr~ ~o Ihe CPU 1.

0 ~hu3, l~r ~li R~d op~raUon~ In ~he pre~erred efnbo~lm~n~ tho d~t~ win 4~ys be ~orroct~ by teoonstruo~lorl ~ven thou~h U13 C;'u i m~y bl~ ro~dln6i trom a 6trlp0 2h~ 18 ~l~iy rucon~ruc~d and the C;i~U - there~ore could hsve reaci tl~ ciat~
dlrectiy. ~a ~cldl~lor~l llme tr~ corr~ ~ooc oata will oniy be Incurrec d~dn~ ~ereeon~ n proc~, whlch ~hould occur InfrequenUy. Thls approtAch allowa . tt~ d~t~ ~tw~e 8ub8y5tem to always m~ln~ln ~nd present correo;t dat~ withou~
h~vlna to kcsp track o~ whe~er or r;ot a requc~tad clr~ts black b ~ow~ ~e r~onaSruotlon ~rlpo.

Alt~m~ ly, th~ Read ta~l< ~ kaep trac1< of wh~thot or not e reques~e~ data blwk b ~abov~ th~ recor~s~ructlon 8trlp~. ~e~errlng to FIGURE 4, r~ r the controlbr clotorrnlne~ that ~ re~ue~ d~ bhck Is on th~ replacom~nt dorQ~o unl;t Sl' ~Step 42~, the controlhr compu0s the posltlon o~ the roque~lod d~ta block In tho ~torago unlt S1 wllh ~a curr~m poonion ol tll~ rocorY~tructlon strlpo (Shp 46, Indtcated ~ opUonal by dD~hod ar~). Il the r~qu~stcd block 1~ not sbovo the reconslrucllon lln~, the wntroRer 3 reconstructs the clrlp0 conlalnln~the r~quwtad d2trl block bn-Ur~e~ by rwdln~ ~nd XOR~n~ ~11 o~ ~he o~hor drlte blocks ~nd th~ corr~pondh~ p~lty olock In Ihe ~trlP~ (Steps 47 ~nd ~8).
HO~NQV01', N U~e rr~qu~sted block k ~bovoU tho roconYtructlon Ihe, the block hn~atre~dy been r~constructed. th~reror0, She conlroller 3 per70nn3 n normal R3ad ot tho d~ta block ~nd trane~er~ Ihe dat~ blocls lo the CPU 1 (Step U). Whlle -1 ~ PA~ENT
PD~ 3D

ppro~ch r~qulree tr~ckln~ th~r w n~t a requ~ad d~t~ bloc~ br~
lh0 r~ons1ruction s~rlpo, ~Im4 is ~Q~ In Rea~ln$~ prevlou~y r~constructod date bl~ck~ on ~ roplacernen~ s~ora~4 unlL

a~d~
H ~ Write operation is requ~ted by th~ CPU ~ durlns~ n~rsl re~n~Pucllon, ~
Wrl~ proc~ b~ln~ n In~4p~n~ent ~ut concunen~ t~stc (FIGUR~ 5, Sl~p ~.
Thl~ me~n~ that lh~ r~cons~ ctlon ta6k contlrluo~ ~o ~x~cuto, but 1hnt tho Wrl~o~ ~nd recor~truotlon l~sk oonterui Sor systorn ro60ure~, In known 1e6hlon.

....... The oon~r~ r ~ locks~ the rocon6tn~10n g~rlpe to pr~vent ~dvertent ~Iter~tlon of ~ qtrlp~ under~oln~ reoons~suot~n (Stap 61). 1~ the Wrlte operatlon s~quwted IB tor ~ data bhck eind corr~pondlnç~ par~ bhc~ nos on ~ho replacemertt slora~e unK St' ~Step 52), the controller 3 pertorms a normal Read-MDdiiy-Wrlte s~u~.
That IB, 1he old pQri~y block and old d~ta b oc~t aro Re~d from the ~rr y (Slep 53~, ~d tho old p~rlty btock b modifi~d by Dsu~ actin~ our (XOF('ln~) ths dd d~t~
76 bl~ nd ~acl~in~ in" ~ ) ~e now data block (Step ~ The r~w p~rlty blook ~nd the now d~ta block 3re then Written to the ana~ (Step 5q. A~br ~e Wrlte opar~Uon b cornploted, Ulo r,,ontrollor S unlock3 th0 r~constructlon ~tl1pe ~tep 66), and th~ Wrlt~ t~rmlnatos (Step 57~.

tt does not mr~ner In thls ca~e wheUlur or no~ the correspondln~ dat~ block on ;i~ the re,olacem~nt stora~e unlt S 1 ' ha~ been correct~d. Because lhe re~ussted d~ta block ~nd Itl; parlty blo~k ue not on the replr~mont stotaSo unlt 51', tho R~sd pul of the R~a~Modi~y-Writ~ sequenc~ doe~ not need to reconstruct th- data.
mus, essenUally no loss o~ perlorm~nco ocaur~, Il the Wr~te op~r~Uon r~que~t~d by Uho CPU 1 1~ tot a data block whero olthar tho d~ta blr~k or 11~ eorre~pondlrlg parlty blod~ aru on tl~ replacement slora~e unll S1' (Step 52), tha prweduro Is dl~rent. The oontroll~r 3 Instoad recomputes a parlt,y block tor tho strlpo ~on-the-ny- ~y readlng all o~ the othcr d~t~ block8 In U~o ~trlpe ~Step 58) and XOR'in~ ~h0n~ (Sbp 59). The new data block le then XOR'd v 1th the recomput~d parity block to ~enerate a new psrlty block ~SIep Bû). ~h0 Pl:~[)7SD

naw t~t~ l~lock ov~r~r;l~ ul~ d~t3 bl~k and th4 n~w parlty ~lock e~/~8 t~ old parily blook ~S~p 5~ r th~ Wrl~ opcr~n Is ~ompl~ed, ~e controller 3 unlwks tl1~ r~or~tru~lon ~trlp~ (S~ep 6~ he W~ite ~k t~mln~ (St~p 57~.

An cll~rrl~livb pr~dure, Indl~tad ~y dash~d llncs In FIGU~E 5, Is 10 nrst r~or~ttu~ Un~ bl~ck on th3 r4plac~ nt ~tor~ unlt S1'. The ~r~llu 3 readc ~ll o~ ~ho o~or blook~ In tho ~ttlpa t~ap ~ hon XOR's Ulem t8top 62).
ma r~sult may b~ stor~d on the rsple44man~ sitora~o unlt Sl~ (Step ~) or ~n 4 ~mpc,r~ry momory. ~he reconslruc~ed block esn thsn ~e u88d In A norrnol Resd-1G .. Mo~ Wrlt~ s0qu~nc3 (Step~ 57).

1~ hould be no~ed IhQt the ~bov~ dlsw~ahn ol a Wrlte oper~llon pert~lr~ to Iho ~nar~ ca~0 o~ ~ss th~n ~l o~ in ~ suipe h~ln~ rno~inod. In ~o l:~p~3Ckll ca80 wh~ro all blocks In s 6~rlpe aro b~ln~ m~dln0d, then the ~o~d ModHy-Wrlts ~quano~ i8 ~nnecessary. ~t l~, the Initi~l Read operatEon b~ not qulr~d, ~Ince ~he parHy ~ or th0 strlpe can bo computs~ o~y on tho næw data. Th~ computed p~ty bbck and the naw data blochs ~ an ~Imply ~tored on ths strlpo. 8trlcUy ~poaldn~, no ~reconstruc~on~ ol the strlpe i8 per~ormed.

It the CPU 1 Wrltes to strlp~ that havo airoady b~n cor(~cted, tho new data is recorded correctly. It ~ Ci'U 1 Writes to sSrlpes Ulat have nol yet heen cor~cted, the new data 18 r~orded corroc~ly and wl~ corr~ed~ H~aln ~lthough N 18 no lon~er Incorrect) ~han the reconstructlon strips advances to the 8t11i~ oonts~'nin~ ~he new datn. Tho ~dditional Um~ to rc~ort~ct ~ooci data wiiibe Incurrad wi~h dlminishln~ ~requency durh~ tho r~con~truction process r~ Ih0 2s r~on~tructlon ~trlpe advancos. Tl~ ppro~ch a~owe the d~t~ ~toraaa sub~y~-tern to alw~y~ malntain an~ pres~nt ootteot data wllhoul ha~ln~ to keep tta~k o~~vhether or not a requ~stod dstJ biock 1- ~ab~ve' or ~beiow~ ~e recon~t~ on strlp3.

-1~ PAT~NT
P~0807S~

~itern~tiYoly, an ~dc~llor~l ~tep mey b0 ~dd~ to tho racor~6lr~c~Jon 1~sk (FIGURE
t ~hl3th0~ ~ ~trlpe h~s rllr~udy ~en rec~n~trucl~ ~y a wr~ta op~atlon (Sbp ~1 H r~constru~lon h~ ~ir~sdy occurrsd, ti~e outrenl r0~nutnl~tion sttlpe mey be unl~cked ~nd proc~a~lng conllnu~i ~t Y r~xt ~trlpe (Step~ & 35). Il r~G~n~ruc~lon h~ not occutrcd, tha ~n~r~l r~c~n3~ctlon l~k ~ntlnu~
~3tocutl~n (Step 33).

A nurnb~r of em~o~irtl~t~ ~ ~ pres~nt Inv0ntl0n ha~ ~n dsu:rliogd.
~bv~rthelea~ wlll be und~rs~ood ~hat varlou~ ml~alio~tk~ns mey ~ made withotlt ~i0p~rtinç~ Irom th~ 3plr~l end ~cope o~ tJle Irr~/enUon. For exampl~, tne pres0nt ~nvontlon csn b~ u~d wlth RAID 3, RAID 4, or RA D 5 ~ys~em~. Furthormoto, t,~n ~rror~ott~ctlon me~hod In addltlen lo or In 0eu o~ the XOR~ner~lod parlly rnay be u~ or Il1e neccssary radun~ancy IntorrnaUon. One such rr~thod usln~
Reed~Sobmon codw l~ dlucle~d In U.S. Patent Applic~tion S~rl~J No. 270,713, nl0~ 1111418~, ~nUtlod AArr~yod ûlsk ~ti~e ~y~m and Method~ ~d ~lgn~d lo Ule ~ssl5anee ot th~ ptssent inv~ntlon. W~th Ihe structure and mothod ~ught i~y th~ referenc0, ~h~ ptcsetlt Invontlon c~n eccornmoda~e ~he loss ot tw~ J~or-~o units H both XOil and Recd~olomon pr any other sys~em) redund~ncy 1~
Accordln~ly, It 1~ to bu und~r~tood that Ihe Invantlon le no~ lo be llmlted by ~e ~pecl~lc Illw~ra~ed em~odiment, but only by ~ho scope o! Ihe eppended dalms.

Claims

PD-0807SD

1. A method for on-line reconstruction of a failed data storage unit in a redundant array of storage units coupled to a controller, the storage units having a plurality of stripes each containing a plurality of data blocks and at least one associated error-correction block, comprising the steps of:
a. providing a replacement storage unit for the failed storage unit;
b. reconstructing the data block of the failed storage unit for each stripe in the arrays;
c. storing the reconstructed data block on the replacement storage unit in the corresponding stripe;
d. if the controller requests a read operation for a data block during reconstruction, then;
(1) If the requested data block is located on the replacement storage unit then:
(a) accessing all of the other data blocks and at least one corre-sponding error-correction block in the stripe containing the requested data block;
(b) reconstructing the requested data block from the accessed blocks;
(c) providing the reconstructed data block to the controller;
(2) If the requested data block is not located on the replacement storage unit, then:
(a) accessing the requested data block directly;
(b) providing the requested data block to the controller.

2. A method for on-line reconstruction of a failed data storage unit in a redundant array of storage units coupled to a controller, the storage units having a plurality of stripes each containing a plurality of data blocks and at least one associated error-correction block, comprising the steps of:
a. providing a replacement storage unit for the failed storage unit;
b. reconstructing the data block of the failed storage unit for each stripe in the array;
c. storing the reconstructed data block on the replacement storage unit in the corresponding stripe;
d. if the controller requests a write operation for a new data block during reconstruction, then:
(1) If the data block to be modified and its at least one associated error-correction block are not located on the replacement storage unit, then:
(a) updating the at least one associated error-correction block in the striped containing the data block to be modified;
(b) writing the updated at least one error-correction block and the new data block in the stripe containing the data block to be modified;
(2) if the data block to be modified is located on the replacement storage unit, then:
(a) accessing all of the other blocks in the stripe containing the data block to be modified;
(b) reconstructing the data block to be modified from the accessed blocks;
(c) updating the at least one associated error-correction block in the stripe containing the data block to be modified;
(d) writing the updated at least one error-correction block and the new data block in the stripe containing the data block to be modified.

3. The method of claim 2, wherein the write operation step includes the further steps of:
a. if the at least one error-correction block associated with the data block to be modified is located on the replacement storage unit, then:
(1) accessing all of the other data blocks in the stripe containing the data block to be modified;
(2) computing at least one error-correction block from the accessed blocks and the new data block;
(3) writing the computed at least one error-correction block and the new data block in the stripe containing the data block to be modified.

4. A method for on-line reconstruction of a failed data storage unit in a redundant array of storage units coupled to a controller, the storage units having a plurality of stripes each containing a plurality of data blocks and at least on associated error-correction block, comprising the steps of:
a. providing a replacement storage unit for the failed storage unit;
b. reconstructing the data block of the failed storage unit for each stripe in the array;
c. storing the reconstructed data block on the replacement storage unit in the corresponding stripe;
d. if the controller requests a write operation for a new data block during reconstruction, then:
(1) if the data block to be modified and its at least one associated error-correction block are not located on the replacement storage unit, then:
(a) updating the at least one associated error-correction block in the stripe containing the data block to be modified;
(b) writing the updated at least one error-correction block and the new data block in the stripe containing the data block to be modified:
(2) if the data block to be modified or its at least one associated error-correction block are located on the replacement storage unit, then:
(a) accessing all of the other data blocks in the striped containing the data block to be modified;
(b) computing at least one error-correction block from the accessed blocks and the new data block;
(c) writing the computed at least one error-correction block and the new data block in the stripe containing the data block to be modified.

5. The method of claim 1, further including the steps of:
a. if the controller requests a write operation for a new data block during reconstruction, then:
(1) if the data block to be modified and its at least one associated error-correction block are not located on the replacement storage unit, then:
(a) updating the at least one associated error-correction block in the stripe containing the data block to be modified;
(b) writing the updated at least one error-correction block and the new data block in the stripe containing the data block to be modified;
(2) if the data block to be modified is located on the replacement storage unit, then:
(a) accessing all of the other blocks in the stripe containing the data block to be modified;
(b) reconstructing the data block to be modified from the accessed blocks;
(c) updating the at least one associated error-correction block in the stripe containing the data block to be modified;
(d) writing the updated at least one error-correction block and the new data block in the stripe containing the data block to be modified.

6. The method of claim 5, wherein the write operation step include the further steps of:
a. if the at least one error-correction block associated with the data block to be modified is located on the replacement storage unit, then:
(1) accessing all of the data blocks in the stripe containing the data block to be modified;
(2) computing at least one error-correction block from the accessed blocks and the new data block;
(3) writing the computed at least one error-correction block and the new data block in the stripe containing the data block to be modified.

7. The method of claim 1, further including the steps of:
a. if the controller requests a write operation for a new data block during reconstruction, then:
(1) if the data block to be modified and its at least one associated error-correction block are not located on the replacement storage unit, then:
(a) updating the at least one associated error-correction block in the stripe containing the data block to be modified;
(b) writing the updated at least one error-correction block and the new data block in the stripe containing the data block to be modified;
(2) if the data block to be modified or its at least one associated error-correction block are located on the replacement storage unit, then:
(a) accessing all of the other data blocks in the stripe containing data block to be modified;
(b) computing at least one error-correction block from the accessed blocks and the new data block;
(c) writing the computed at least one error-correction block and the new data block in the stripe containing the data block to be modified.

8. The method of claims 1, 2, or 4, wherein at least one error-correction block contains parity information, and the step of reconstructing each data block of the failed storage unit comprises the steps of:
a. accessing all of the other data blocks and the corresponding parity error-correction block in the corresponding stripe;
b. exclusively-OR'ing all of the accessed blocks to generate a new data block.

9. The method of claims 2, 4, 5, or 7, wherein at least one error-correction block contains parity information, and the step of updating the at least one associated error-correction block during a write operation comprises the steps of:
a. reading the data block to be modified and the associated parity error-correction block;
b. exclusively-OR'ing the parity error-correction block with the data block to be modified and the new data block to generate an updated parity error-correction block.

10. The method of claims 3, 4, 6, or 7, wherein the step of computing at leastone error-correction block during a write operation comprises exclusively-OR'ing the accessed blocks and the new data block.

11. The method of claim 1, wherein the step of reconstructing the requested data block from the accessed blocks comprises exclusively-OR'ing the accessed blocks.

12. The method of claims 2 or 5, wherein the step of reconstructing the data block to be modified from the accessed blocks comprises exclusively-OR'ing the accessed block.

13. The method of claims 1, 5, or 7, wherein the requested data block is located on the replacement storage unit;
a. if the block has previously been reconstructed, then:
(1) accessing the requested data block directly;
(2) providing the requested data block to the controller;
b. otherwise:
(1) accessing all of the other data blocks and at least one corre-sponding error-correction block in the stripe containing the requested data block;
(2) reconstructing the requested data block from the accessed blocks;
(3) providing the reconstructed data block to the controller.

14. The method of claims 1, 2, or 4, wherein the step of reconstructing the data block of the failed storage unit for each stripe in the array is performed only for stripes that have not previously been reconstructed.

15. Control means for on-line reconstruction to a replacement storage unit of afailed data storage unit in a redundant array of storage units, the storage units having a plurality of stripes each containing a plurality of data blocks and an associated error-correction block, the control means being coupled to the array of storage units and including:
a. reconstruction means for reconstructing the data block of the failed storage unit for each stripe in the array and storing the reconstructed data block on the replacement storage unit in the corresponding stripe;
b. read operation means, for reading a requested data block during operation of the reconstruction means by:
(1) if the requested data block is located on the replacement storage unit then:
(a) accessing all of the other data blocks and the correspond-ing error-correction block in the stripe containing the requested data block;
(b) reconstructing the requested data block from the accessed blocks;
(c) providing the reconstructed data block to the control means:
(2) if the requested data block is not located on the replacement storage unit, then:
(a) accessing the requested data block directly;
(b) providing the requested data block to the control means.

10. Control means for on-line reconstruction to a replacement storage unit of afailed data storage unit in a redundant array of storage units, the storage units having a plurality of stripes each containing a plurality of data blocks and an associated error-correction block, the control means being coupled to the array of storage units and including:
a. reconstruction means for reconstructing the data block of the failed storage unit for each stripe in the array and storing the reconstructed data block on the replacement storage unit in the corresponding stripe;
b. write operation means, for writing a new data block during operation of the reconstruction means by:
(1) if the data block to be modified and the associated error-correction block are not located on the replacement storage unit, then:
(a) updating the associated error-correction block in the stripe containing the data block to be modified;
(b) writing the updated error-correction block and the new data block in the stripe containing the data block to be modified;
(2) if the data block to be modified is located on the replacement storage unit, then:
(a) accessing all the other blocks in the strips containing the data block to modified;
(b) reconstructing the data block to be modified from the accessed blocks;
(c) updating the associated error-correction block in the stripe containing the data block to be modified;
(d) writing the updated error-correction block and the new da block in the stripe containing the dance block to be modified.

17. Control means for on-line reconstruction to a replacement storage unit of afailed data storage unit in a redundant array of storage units, the storage units having a plurality of stripes each containing a plurality of data blocks and an associated error-correction block, the control means being coupled to the array of storage units and including:
a. reconstruction means for reconstructing the data block of the failed storage unit for each stripe in the array and storing the reconstructed data block on the replacement storage unit in the corresponding stripe;
b. write operation means, for writing a new data block during operation of the reconstruction means, by:
(1) if the data block to be modified and its associated error-correction block are not located on the replacement storage unit, then:
(a) updating the associated error-correction block in the stripe containing the data block to be modified:
(b) writing the updated error-correction block and the new data block in the stripe containing the data block to be modified;
(2) if the data block to be modified or its associated error-correction block are located on the replacement storage until, then:
(a) accessing all of the other data blocks in the stripe containing the data block to be modified;
(b) computing a error-correction block and the new data block in the stripe containing the data block to be modified.