WO2003009286A3 - Block-appended checksums - Google Patents

Block-appended checksums Download PDF

Info

Publication number
WO2003009286A3
WO2003009286A3 PCT/US2001/051321 US0151321W WO03009286A3 WO 2003009286 A3 WO2003009286 A3 WO 2003009286A3 US 0151321 W US0151321 W US 0151321W WO 03009286 A3 WO03009286 A3 WO 03009286A3
Authority
WO
WIPO (PCT)
Prior art keywords
block
appended
data
checksum
file
Prior art date
Application number
PCT/US2001/051321
Other languages
French (fr)
Other versions
WO2003009286A2 (en
Inventor
Andy Kahn
Rajesh Sundaram
Srinivasan Viswanathan
Steven R Kleiman
Original Assignee
Network Appliance Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Network Appliance Inc filed Critical Network Appliance Inc
Priority to EP01274308A priority Critical patent/EP1364368A2/en
Publication of WO2003009286A2 publication Critical patent/WO2003009286A2/en
Publication of WO2003009286A3 publication Critical patent/WO2003009286A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing
    • G11B20/18—Error detection or correction; Testing, e.g. of drop-outs
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
    • H03M13/09—Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
    • H03M13/095—Error detection codes other than CRC and single parity bit codes
    • H03M13/096—Checksums

Abstract

A method and apparatus for a reliable data storage system using block level checksums appended to data blocks. Files are stored on hard disks in storage blocks, including data blocks and block-appended checksums. The block-appended checksum includes a checksum of the data block, a VBN, a DBN, and an embedded checksum for checking the integrity of the block-appended checksum itself. A file system includes file blocks with associated block-appended checksum to the data blocks. The file blocks with block-appended checksums are written to storage blocks. In a preferred embodiment a collection of disk drives are formatted with 520 bytes of data per sector. For each 4,096-byte file block, a corresponding 64-byte block-appended checksum is appended to the file block with the first 7 sectors including most of the file block data while the 8th sector includes the remaining file block data and the 64-byte block-appended checksum.
PCT/US2001/051321 2000-10-25 2001-10-25 Block-appended checksums WO2003009286A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP01274308A EP1364368A2 (en) 2000-10-25 2001-10-25 Block-appended checksums

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/696,666 US6952797B1 (en) 2000-10-25 2000-10-25 Block-appended checksums
US09/696,666 2000-10-25

Publications (2)

Publication Number Publication Date
WO2003009286A2 WO2003009286A2 (en) 2003-01-30
WO2003009286A3 true WO2003009286A3 (en) 2003-09-12

Family

ID=24798055

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/051321 WO2003009286A2 (en) 2000-10-25 2001-10-25 Block-appended checksums

Country Status (3)

Country Link
US (1) US6952797B1 (en)
EP (1) EP1364368A2 (en)
WO (1) WO2003009286A2 (en)

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6637007B1 (en) 2000-04-28 2003-10-21 Network Appliance, Inc. System to limit memory access when calculating network data checksums
US6751635B1 (en) * 2000-08-18 2004-06-15 Network Appliance, Inc. File deletion and truncation using a zombie file space
US6728922B1 (en) 2000-08-18 2004-04-27 Network Appliance, Inc. Dynamic data space
US6952797B1 (en) 2000-10-25 2005-10-04 Andy Kahn Block-appended checksums
US7007220B2 (en) * 2002-03-01 2006-02-28 Broadlogic Network Technologies, Inc. Error correction coding across multiple channels in content distribution systems
US7634614B2 (en) * 2003-01-13 2009-12-15 Sierra Logic Integrated-circuit implementation of a storage-shelf router and a path controller card for combined use in high-availability mass-storage-device shelves and that support virtual disk formatting
US7526622B1 (en) 2004-05-26 2009-04-28 Sun Microsystems, Inc. Method and system for detecting and correcting data errors using checksums and replication
US20060075281A1 (en) * 2004-09-27 2006-04-06 Kimmel Jeffrey S Use of application-level context information to detect corrupted data in a storage system
US7549089B1 (en) 2004-09-27 2009-06-16 Network Appliance, Inc. Lost write detection in a storage redundancy layer of a storage server
US7770076B2 (en) * 2004-11-02 2010-08-03 Nvidia Corporation Multi-platter disk drive controller and methods for synchronous redundant data operations
JP4634157B2 (en) 2005-01-17 2011-02-16 株式会社日立製作所 Storage system
US7689877B2 (en) * 2005-11-04 2010-03-30 Sun Microsystems, Inc. Method and system using checksums to repair data
US7716519B2 (en) * 2005-11-04 2010-05-11 Oracle America, Inc. Method and system for repairing partially damaged blocks
JP5089901B2 (en) * 2006-03-28 2012-12-05 株式会社日立製作所 Storage control device and control method of storage control device
US7991865B2 (en) * 2006-05-23 2011-08-02 Cisco Technology, Inc. Method and system for detecting changes in a network using simple network management protocol polling
US7730239B2 (en) * 2006-06-23 2010-06-01 Intel Corporation Data buffer management in a resource limited environment
US8087017B1 (en) * 2007-04-09 2011-12-27 Moka5, Inc. Trace-assisted prefetching of virtual machines in a distributed system
US7873878B2 (en) * 2007-09-24 2011-01-18 International Business Machines Corporation Data integrity validation in storage systems
JP5298393B2 (en) * 2007-11-21 2013-09-25 マーベル ワールド トレード リミテッド Parallel Reed-Solomon RAID (RS-RAID) architecture, device, and method
US8613074B2 (en) 2010-09-30 2013-12-17 Micron Technology, Inc. Security protection for memory content of processor main memory
US9037788B2 (en) * 2010-09-30 2015-05-19 Micron Technology, Inc. Validating persistent memory content for processor main memory
US9009569B2 (en) * 2010-10-18 2015-04-14 Xyratex Technology Limited Detection and correction of silent data corruption
EP2515499B1 (en) * 2011-04-21 2015-10-28 Wibu-Systems AG Method for generating a cryptographic key for a secure digital data object on the basis of the current components of a computer
US9424128B2 (en) 2011-08-12 2016-08-23 Futurewei Technologies, Inc. Method and apparatus for flexible RAID in SSD
US9477682B1 (en) * 2013-03-13 2016-10-25 Emc Corporation Parallel compression of data chunks of a shared data object using a log-structured file system
US10353771B1 (en) * 2014-03-31 2019-07-16 EMC IP Holding Company LLC Managing data storage
US9990298B2 (en) * 2014-05-12 2018-06-05 Western Digital Technologies, Inc System and method for caching solid state device read request results
US10452642B1 (en) * 2015-03-20 2019-10-22 Tintri By Ddn, Inc. Detecting and pinpointing data corruption
US10180875B2 (en) * 2016-07-08 2019-01-15 Toshiba Memory Corporation Pool-level solid state drive error correction
US10243583B2 (en) 2017-06-16 2019-03-26 Western Digital Technologies, Inc. CPU error remediation during erasure code encoding

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0466296A2 (en) * 1990-07-09 1992-01-15 Seagate Technology International A data recovery channel in a fault tolerant disk drive array and a method of correcting errors therein
US5517484A (en) * 1993-02-09 1996-05-14 Matsushita Electric Industrial Co., Ltd. Optical disk and optical disk recording and reproducing apparatus
US5696775A (en) * 1994-09-23 1997-12-09 Cirrus Logic, Inc. Method and apparatus for detecting the transfer of a wrong sector
US5903532A (en) * 1994-09-07 1999-05-11 Kabushiki Kaisha Toshiba Disk, recording device, reproducing device, recording method and reproducing method
EP0997900A2 (en) * 1998-10-02 2000-05-03 Matsushita Electric Industrial Co., Ltd. Method and apparatus for recording and reproducing information on and from disk

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4761785B1 (en) 1986-06-12 1996-03-12 Ibm Parity spreading to enhance storage access
US5283791A (en) * 1988-08-02 1994-02-01 Cray Research Systems, Inc. Error recovery method and apparatus for high performance disk drives
US5195100A (en) 1990-03-02 1993-03-16 Micro Technology, Inc. Non-volatile memory storage of write operation identifier in data sotrage device
US5088081A (en) 1990-03-28 1992-02-11 Prime Computer, Inc. Method and apparatus for improved disk access
JPH0731582B2 (en) 1990-06-21 1995-04-10 インターナショナル・ビジネス・マシーンズ・コーポレイション Method and apparatus for recovering parity protected data
US5208813A (en) 1990-10-23 1993-05-04 Array Technology Corporation On-line reconstruction of a failed redundant array system
US5235601A (en) 1990-12-21 1993-08-10 Array Technology Corporation On-line restoration of redundancy information in a redundant array system
US5239640A (en) 1991-02-01 1993-08-24 International Business Machines Corporation Data storage system and method including data and checksum write staging storage
US5278838A (en) 1991-06-18 1994-01-11 Ibm Corp. Recovery from errors in a redundant array of disk drives
US5313626A (en) 1991-12-17 1994-05-17 Jones Craig S Disk drive array with efficient background rebuilding
US5341381A (en) 1992-01-21 1994-08-23 Tandem Computers, Incorporated Redundant array parity caching system
DE69320388T2 (en) 1992-03-06 1999-05-12 Data General Corp Data handling in a system with a processor to control access to a plurality of data storage disks
US5469566A (en) 1992-03-12 1995-11-21 Emc Corporation Flexible parity generation circuit for intermittently generating a parity for a plurality of data channels in a redundant array of storage units
US5392290A (en) * 1992-07-30 1995-02-21 International Business Machines Corporation System and method for preventing direct access data storage system data loss from mechanical shock during write operation
KR0155702B1 (en) * 1992-08-13 1998-12-15 강진구 2-division error correction method and decoder
US5666511A (en) * 1992-10-08 1997-09-09 Fujitsu Limited Deadlock suppressing schemes in a raid system
DE69435146D1 (en) 1993-06-03 2008-11-13 Network Appliance Inc Method and apparatus for describing arbitrary areas of a file system
US5963962A (en) 1995-05-31 1999-10-05 Network Appliance, Inc. Write anywhere file-system layout
EP0701715A4 (en) 1993-06-04 1999-11-17 Network Appliance Corp A method for providing parity in a raid sub-system using a non-volatile memory
US5537567A (en) 1994-03-14 1996-07-16 International Business Machines Corporation Parity block configuration in an array of storage devices
JP3661205B2 (en) 1994-09-09 2005-06-15 株式会社日立製作所 Disk array system and method for generating parity data of disk array system
US5864440A (en) * 1994-12-27 1999-01-26 International Business Machines Corporation Data processing method and data storage system
US5758057A (en) * 1995-06-21 1998-05-26 Mitsubishi Denki Kabushiki Kaisha Multi-media storage system
US5864655A (en) * 1996-09-09 1999-01-26 International Business Machines Corporation Managing removable media in raid and rail environments
US5950225A (en) 1997-02-28 1999-09-07 Network Appliance, Inc. Fly-by XOR for generating parity for data gleaned from a bus
JP3674227B2 (en) * 1997-03-14 2005-07-20 株式会社日立製作所 Storage device for storing portable media
US6317844B1 (en) 1998-03-10 2001-11-13 Network Appliance, Inc. File server storage arrangement
US6637007B1 (en) 2000-04-28 2003-10-21 Network Appliance, Inc. System to limit memory access when calculating network data checksums
US6952797B1 (en) 2000-10-25 2005-10-04 Andy Kahn Block-appended checksums

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0466296A2 (en) * 1990-07-09 1992-01-15 Seagate Technology International A data recovery channel in a fault tolerant disk drive array and a method of correcting errors therein
US5517484A (en) * 1993-02-09 1996-05-14 Matsushita Electric Industrial Co., Ltd. Optical disk and optical disk recording and reproducing apparatus
US5903532A (en) * 1994-09-07 1999-05-11 Kabushiki Kaisha Toshiba Disk, recording device, reproducing device, recording method and reproducing method
US5696775A (en) * 1994-09-23 1997-12-09 Cirrus Logic, Inc. Method and apparatus for detecting the transfer of a wrong sector
EP0997900A2 (en) * 1998-10-02 2000-05-03 Matsushita Electric Industrial Co., Ltd. Method and apparatus for recording and reproducing information on and from disk

Also Published As

Publication number Publication date
EP1364368A2 (en) 2003-11-26
WO2003009286A2 (en) 2003-01-30
US6952797B1 (en) 2005-10-04

Similar Documents

Publication Publication Date Title
WO2003009286A3 (en) Block-appended checksums
US8731897B2 (en) Tape emulating disk based storage system and method with automatically resized emulated tape capacity
KR101910840B1 (en) Dynamic granule-based intermediate storage
WO2001033569A1 (en) Portable audio playback device and removable disk drive
EP0676761A4 (en) Information management method, data recording medium, data recording method, information retrieving method and information retrieving apparatus.
TW200512728A (en) System and method for writing data to HDD in bands
EP1978520A3 (en) Recording apparatus, recording method, and disc shaped recording medium
EP0364176A3 (en) System and method for formatting high density storage media
EP1791122A3 (en) Information recording medium, recording apparatus and method for an information recording medium, reproducing apparatus and method for an information recording medium computer program for controlling record or reproduction, and data structure including control signal
AU2003203274A1 (en) Information recording medium, information recording apparatus and method, information reproducing apparatus and method, information recording/reproducing apparatus and method, recording or reproduction control computer program, and data structure containing control signal
WO2005050528A3 (en) Method of creating playback control file for recording medium and method and apparatus for reproducing data using the playback control file
US20040148476A1 (en) Performance-predicated automatic defragmentation of hard disks, particularly for non-pc mobile digital devices including music recorder/players
EP0798713A3 (en) Optical disk device and replacement processing method
AU2003268714A1 (en) Information recording medium, information recording device and method, information reproduction device and method, information recording/reproduction device and method, recording or reproduction control computer program, and data structure containing control signal
AU2001265206A1 (en) Method and apparatus for emulating read/write file system on a write-once data storage disk
WO2001073537A3 (en) System for and method of accessing blocks on a storage medium
EP1677299A3 (en) Recording medium with copy protection indicating information and apparatus and methods for forming, recording, reproducing and restricting reproduction of the recording medium
EP1130588A3 (en) Data transfer with abnormality control
TW200603118A (en) Optical disc medium, optical disc apparatus using the same, and data recording method in the same
AU2003268715A1 (en) Information recording medium, information recording device and method, information reproduction device and method, information recording/reproduction device and method, recording or reproduction control computer program, and data structure containing control signal
EP1137004A3 (en) Methods for increasing file storage capacity of optical discs
WO2001031444A3 (en) Method for preventing repeating non-recoverable read errors at same physical location on data storage media
AU2003252294A1 (en) Information recoding medium, information recording device and method, information reproduction device and method, information recording/reproduction device and method, computer program, and data structure
AU2003271076A1 (en) Information recording medium, information recording device and method, information reproduction device and method, information recording/reproduction device and method, recording or reproduction control computer program, and data structure containing control signal
SG143276A1 (en) Information recording medium, recording/reproducing method and apparatus, and host apparatus

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): JP

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
REEP Request for entry into the european phase

Ref document number: 2001274308

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2001274308

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2001274308

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: JP