US20080140905A1 - Magnetic disk unit, file management system, and file management method - Google Patents
Magnetic disk unit, file management system, and file management method Download PDFInfo
- Publication number
- US20080140905A1 US20080140905A1 US11/971,368 US97136808A US2008140905A1 US 20080140905 A1 US20080140905 A1 US 20080140905A1 US 97136808 A US97136808 A US 97136808A US 2008140905 A1 US2008140905 A1 US 2008140905A1
- Authority
- US
- United States
- Prior art keywords
- data
- recording medium
- read
- cache memory
- full
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0605—Improving or facilitating administration, e.g. storage management by facilitating the interaction with a user or administrator
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0643—Management of files
Definitions
- the present invention relates to a control to read/write data from and into a magnetic disk unit, and specifically, relates to a logical operation for input-output data.
- a hard disk unit As a storage device for computer devices, a hard disk unit is widely used, which is randomly accessible and the read/write process is quick. Physically, the hard disk unit magnetically records and reads data into and from a data recording area of a magnetic disk, which is partitioned into tracks and sectors. However, data is inputted/outputted under management of a file system in terms of logical units called blocks independently of the physical partitions of the data recording area. The blocks constitute a file and is used in computer processing.
- the file system manages files on an external storage device and is provided as a function of an operating system (OS) or the like.
- OS operating system
- FIG. 7 is a diagram showing relationship between a general hard disk unit and a file system.
- a hard disk unit 710 includes a magnetic disk 711 as a recording medium, a controller 712 which controls reading and writing data from and into the magnetic disk 711 , and a nonvolatile memory 713 used as a cache memory.
- the file system 720 issues requests (input and output requests) for the hard disk unit 710 to read and write data in block units. Also, the file system 720 uses an address called a logical block address (LBA) (see “Transistor Gijutsu”, p. 180-181, December 2001, CQ publishing, Co., Ltd., for example) to identify a block stored in the hard disk unit 710 .
- LBA logical block address
- the file system 720 when a file I/O (input/output) request arrives at the file system 720 from an application or the like, the file system 720 creates and issues to the hard disk unit 710 input/output requests necessary to handle the request. Next, when the file system 720 issues a write request to the hard disk unit 710 , such as a case where there is a request to write data into a file, the file system 720 determines a layout of data on the hard disk unit 710 .
- the hard disk unit makes a magnetic head to seek to a desired track on a rotating magnetic disk to read and write data. For this structural reason, when the hard disk unit performs operations for blocks having consecutive LBAs in both reading and writing data (sequential reading and writing), the seek operation of the magnetic head is minimized, and the performance thereof is maximized.
- the hard disk unit 710 translates the layout of blocks which has been translated into LBAs in the input/output requests outputted from the file system 720 into a layout enabling a good input/output performance. This is because it is generally impossible to estimate the layout structure on the hard disk unit 710 from a write designation having been translated into LBAs.
- the file system 720 has hitherto determined a proper layout on the hard disk unit 710 to improve the performance of executing input/output requests.
- the conventional file system 720 often takes a following strategy. If possible, the file system 720 does not immediately perform writing into the hard disk unit 710 in response to the write request and stores to some extent data to be written in the cache memory. The file system 720 then determines a proper layout and performs writing into the magnetic disk. By storing the data in the cache memory, input/output transactions concerning writing into the same area are reduced, and unnecessary movement of the magnetic head of the hard disk unit 710 is reduced, thus increasing writing throughput, and as well as a layout can be employed which allows less fragmentation in order not to bring disadvantages in reading.
- the layout on the magnetic disk advantageous in writing and the layout on the magnetic disk advantageous in reading are generally different from each other as described above. Therefore, if data is written into the magnetic disk in the layout advantageous in writing and rearranged to a layout advantageous in reading before reading the data, the performance of the file system can be significantly improved.
- the conventional file system and hard disk unit use LBAs to specify a target where data is written, and information concerning the continuity of blocks as a file is owned by the file system and not known by the hard disk unit.
- the hard disk unit cannot identify whether data is written into a predetermined LBA has an optimum arrangement, or the arrangement of the predetermined LBA is inappropriate but the data is written into the predetermined LBA because commit write must be executed. Therefore, the hard disk unit cannot perform defragmentation (rearrangement of data on a magnetic disk to eliminate fragmentation) as a function thereof.
- the hard disk unit In order for the hard disk unit to freely change the layout of data on the magnetic disk, a construction can be conceived in which the hard disk unit incorporates the file system.
- the hard disk unit includes all information necessary for optimization of the data arrangement. Accordingly, the reduction in the read/write performance due to the layout of data on the magnetic disk can be avoided, or at least can be dissolved by the hard disk unit alone.
- the file size which can be handled by the storage capacity of the hard disk unit may be prescribed.
- the storage capacity of the hard disk unit becomes an upper limit of the file size, or the OS is required to have a function to make a plurality of files appear as a single file. Both are not practical from the viewpoint of the input-output throughput.
- the function required for the file system varies depending on the OS.
- the file system of the hard disk unit is inevitably specialized in a predetermined OS. Therefore, the hard disk unit depends on the OS and cannot be used for general purposes.
- An object of the present invention which achieves the aforementioned objects is to provide a file management system which enables data on a magnetic disk to be rearranged by control of a hard disk unit itself, and the management method thereof.
- Another object of the present invention is to improve the read/write performance of data from and into the magnetic disk.
- this magnetic disk unit includes: a magnetic disk as a recording medium; a controller which controls reading/writing data from and into the magnetic disk; and an address translator section.
- the address translator section translates between first logical addresses used to specify predetermined data when reading/writing the data from and into the magnetic disk and second logical addresses used to exchange the data with an external device.
- the controller rearranges data (defragmentation) recorded on the magnetic disk based on the second logical addresses.
- the second logical addresses used here are set in address space for each file such that data within a single file is continuous.
- the controller releases the second logical addresses associated with the disused data.
- the aforementioned magnetic disk unit according to the present invention can further include a cache memory which holds data to be written into the magnetic disk and data read from the magnetic disk.
- the controller classifies and manages data held in the cache memory based on whether or not the data is saved in the magnetic disk and assignment of the second logical addresses to the data, and rearranges data recorded on the magnetic disk in accordance with the classification.
- Another magnetic disk unit including: a magnetic disk as a recording medium; a controller which controls reading and writing data from and into the magnetic disk; and a cache memory.
- the controller holds target data for reading/writing process from and into the magnetic disk in the cache memory after classifying the data based on whether or not the data is stored in the magnetic disk and assignment of predetermined logical addresses to the data, and stores the data held in the cache memory into the magnetic disk in accordance with the classification to optimize a layout of the data stored in the magnetic disk.
- the controller selects from data held in the cache memory a data which is at least not contained in the magnetic disk and assigned with the logical addresses consecutive over a length of not less than a single stride, and stores the data in a continuous recording area of the magnetic disk.
- the stride is a management unit of cache data, and indicates a size allowing reading and writing into the magnetic disk to be performed with enough efficiency. In other words, even if reading and writing is performed in terms of a unit larger than the stride, the efficiency is little improved.
- the stride is a size uniquely determined for the entire system.
- Another aspect of the present invention to achieve the aforementioned object is implemented as a file management system including an external storage device of a computer; and a file management section which manages files stored in the external storage device.
- the file management section manages data to be read and written from and into the external storage device such that data in a file is continuous in a predetermined logical address space, and specifies data requested to be read and written from and into the external storage device with logical addresses in the logical address space.
- the external storage device translates the logical addresses specifying the data requested to be read and written into logical block addresses (LBAs), and performs processing of reading and writing data from and into a recording medium.
- LBAs logical block addresses
- the file management section issues a command to the external storage device to remove disused data.
- the external storage device removes information concerning the data specified by the command from correspondence information for translation between the logical addresses and the LBAs in accordance with the command.
- the present invention is implemented as a method of managing files in an external storage device of a computer as follows.
- the method of managing files includes the steps of: receiving a read/write request on data specified by a logical address, which is set such that data in a file is continuous in a predetermined logical address space; and translating the logical addresses of target data in the read/write request to a logical block address (LBA) and performing read/write process from and into a recording medium based on the LBA.
- LBA logical block address
- Another method of managing files according to the present invention includes the steps of: performing read/write process from and into a recording medium based on a read/write request on data received from the computer; holding target data of the read/write process into a cache memory after classifying the data based on whether or not the data is stored in the recording medium and assignment of predetermined logical addresses to the data; and optimizing a layout of the data stored in the recording medium by storing data held in the cache memory into the recording medium in accordance with the classification at a predetermined time.
- FIG. 1 is a view showing a concept of a frame address used in an embodiment.
- FIG. 2 is a diagram showing a construction of a file management system in the embodiment.
- FIG. 3 is a diagram showing an operation in the case where a write request is issued from a file management section to a hard disk unit in the embodiment.
- FIG. 4 is a diagram showing an operation in the case where a read request is issued from the file management section to the hard disk unit in the embodiment.
- FIG. 5 is a diagram showing an operation in the case where a remove request is issued from the file management section to the hard disk unit in the embodiment.
- FIG. 6 is a diagram showing an operation of defragmentation that the hard disk unit independently performs in the embodiment.
- FIG. 7 is a diagram showing a relationship between a general hard disk unit and a file system.
- logical address space called frame address is newly defined, and data stored in the hard disk unit is managed using this address space.
- FIG. 1 is a diagram showing a concept of the frame address.
- the frame address is a bit string expressing huge address space.
- the bit length of the frame address is assumed to be 1024 bits, but actually may be any length as long as it is long enough compared to the storage capacity of the hard disk unit.
- the file system arranges individual files on frame address space such that data in each file is continuous and free space large enough is arranged between each files, namely, the presence of another file does not cause insufficiency of an area for a desired file to be assigned.
- a hard disk unit with a storage capacity of 10 gigabytes when a space of 10 gigabytes is given for a file, enough free space is secured between files, and data in a file can certainly be made continuous on the frame address space.
- the high-order 512 bits of 1024 bits are assigned to a file handle, and the low-order 512 bits are assigned to an offset (byte) from the top of the file.
- the hard disk unit writes data into the magnetic disk while creating a correspondence table between frame addresses and an actual layout on the magnetic disk.
- the layout on the magnetic disk in writing is properly determined by a controller of the hard disk unit.
- the hard disk unit reads data of a specified frame addresses based on the layout correspondence table created at writing.
- the controller performs defragmentation process based on the read data to optimize the layout of data on the magnetic disk if the controller determines it to be appropriate.
- the controller may independently perform the defragmentation process at a time determined appropriate. Since there is no change in the frame addresses by the defragmentation processing, the defragmentation processing can be performed transparently from the file system.
- the frame address space does not directly correspond to the physical blocks on the magnetic disk unlike the LBAs, it is required to explicitly recover disused frame addresses and the corresponding physical blocks on the magnetic disk.
- the LBAs directly corresponded to the physical blocks on the magnetic disk. Therefore, when a predetermined LBA was no longer used, it was possible to reuse the disused LBA and the physical block on the magnetic disk by using the relevant LBA for another purpose.
- the LBAs corresponding to the disused frame addresses and the physical blocks on the magnetic disk are not reused for another purpose, so that it is required to explicitly release the disused address. Accordingly, when a predetermined file on the file system is deleted, the frame addresses corresponding to the file is explicitly removed, namely, eliminated from the layout correspondence table.
- FIG. 2 is a diagram showing a construction of the file management system according to the present embodiment.
- the file management system of the present embodiment includes a hard disk unit 10 and a file management section 20 which controls the hard disk unit 10 to read and write files.
- the hard disk unit 10 is an external storage device connected to a computer device.
- the file management section 20 is a software block implemented by a program-controlled CPU of the computer device using the hard disk unit 10 as the external storage device.
- the file management section 20 is implemented by a file system provided as a function of the OS. Blocks (data) exchanged between the hard disk unit 10 and the file management section 20 are specified by the frame addresses (FAs) in accordance with a read/write request from the file management section 20 .
- FAs frame addresses
- the hard disk unit 10 includes a magnetic disk 11 as a recording medium, a controller 12 which controls reading and writing data from and into the magnetic disk 11 , a cache memory 13 implemented by a nonvolatile memory or the like, and an FA-LBA translator section 14 which translates between frame addresses and LBAs.
- the controller 12 and the FA-LBA translator section 14 are software blocks each implemented by a processor or the like controlled by a program stored in a not-shown ROM or the like.
- the magnetic disk 11 magnetically records data under the control of the controller 12 .
- the controller 12 receives requests to write and read data from the file management section 20 and controls various units (not shown) to perform mechanical operations such as seek of the magnetic head and read and write data from and into the magnetic disk 11 .
- Blocks exchanged between the controller 12 and the magnetic disk 11 are specified by LBAs.
- the cache memory 13 temporarily holds data read from the magnetic disk 11 , the read/write requests sent from the file management section 20 , and data contained in these requests.
- the FA-LBA translator section 14 translates between the frame addresses used to specify blocks between the file management section 20 and the controller 12 , and the LBAs used to specify blocks between the controller 12 and the magnetic disk 11 .
- the translation of the address space uses the layout correspondence table. This layout correspondence table is created during writing data into the magnetic disk 11 and held in the cache memory 13 (indicated by a dashed line in FIG. 2 ).
- a cache area of the cache memory 13 is classified into a total of four areas with two categories.
- the first category is classified into “clean” and “dirty,” which indicate whether cache data has already been saved on the magnetic disk.
- the “clean” indicates that an image on the magnetic disk agrees with the cache data (the data is saved).
- the “dirty” indicates that the image on the magnetic disk does not agree with data in the cache (the data is not saved) and therefore the cache data needs to be saved on the magnetic disk.
- the second category is classified into “full” and “fragmented”, which indicate assignment of the frame addresses to data.
- the cache manages cache data in terms of a unit called stride. In the case of reading, the stride is a size large enough so that sequential reading of data on the magnetic disk is made with efficiency.
- the “full” indicates that all the contents within a stride is composed of blocks constituted by consecutive frame addresses.
- the “fragmented” indicates that the contents of a stride is composed of blocks which are pointed out by nonconsecutive frame addresses.
- the cache area of the cache memory 13 is classified into four types of a clean/full cache area, a clean/fragmented cache area, a dirty/full cache area, and a dirty/fragmented cache area.
- FIG. 3 is a diagram showing an operation in the case where a write request is issued from the file management section 20 to the hard disk unit 10 .
- the controller 12 receives the write request and stores data to be written into the cache memory 13 (arrow 1 in FIG. 3 ). At this point, the data is managed with frame addresses.
- the amount of data to be written fulfills a stride
- the data is registered in the dirty/full cache area.
- frame addresses of the data and other cache data are compared.
- a combination of data which fulfills a stride is made, such data are collected and registered in the dirty/full cache area. In other cases, the data to be written is registered in the dirty/fragmented cache area.
- the controller 12 writes the cache data held in the dirty/full cache area into the magnetic disk (arrow 2 in FIG. 3 ).
- an area where the data is to be written is determined on the free space of the magnetic disk such that a cache image of a single stride has consecutive LBAs.
- the cache data which has been written is moved to the clean/full cache area.
- the FA-LBA translator section 14 registers corresponding frame addresses and LBAs in the layout correspondence table (arrow 3 in FIG. 3 ). When the frame addresses of the cache data held in the dirty/fragmented cache area are registered in the layout correspondence table, these frame addresses are released.
- FIG. 4 is a diagram showing an action in the case where a read request is issued from the file management section 20 to the hard disk unit 10 .
- the controller 12 accepts the read request (arrow 1 in FIG. 4 ).
- the controller 12 inquires the FA-LBA translator section 14 to acquire the LBAs corresponding to the frame addresses in a range requested to be read (arrow 2 in FIG. 4 ), and checks whether data in the range to be read is in the cache of the cache memory 13 (arrow 3 in FIG. 4 ).
- the data in the cache memory can be part of data in the range to be read.
- the controller 12 reads part of data to be read which was not in the cache memory 13 from the magnetic disk 11 to the cache memory 13 (arrow 4 in FIG. 4 ). At this time, data outside the required range is read such that the size of the data to be read is the same as the full stride size.
- the controller 12 stores the data read with the full stride size into the dirty/full cache area in the cache memory 13 . This allows the data of the full stride size to be continuously written into the magnetic disk 11 later. In other cases, the read data is stored in the clean/full data cache area.
- the controller 12 arranges the cached data blocks sequentially as possible and resisters the data as the dirty/fragmented data.
- the controller 12 returns the data in the range requested to be read to the file management section 20 (arrow 5 in FIG. 4 ). Reading data requested from the file management section 20 is thus completed.
- the controller 12 writes the cache data held in the dirty/full cache area into the magnetic disk 11 (arrow 6 in FIG. 4 ).
- free areas are utilized such that the cache data can be written into the magnetic disk 11 in full strides and that the individual strides are arranged in frame address order.
- the cache images are sequentially moved to the clean/full cache area on finishing being written into the magnetic disk 11 .
- the FA-LBA translator section 14 sequentially registers into the layout correspondence table the cache data which is write complete. (arrow 7 in FIG. 4 ) In the aforementioned operations, the old correspondence concerning the data read from the magnetic disk 11 is deleted from the layout correspondence table.
- a delete command called a remove request is defined and issued from the file management section 20 .
- the remove request specifies the top address (frame address) and the data length of the data which is no longer to be used.
- FIG. 5 is a diagram showing an operation when a remove request is issued from the file management section 20 to the hard disk unit 10 .
- the controller 12 accepts the remove request (arrow 1 in FIG. 5 ), and then identifies all frame addresses to be deleted based on the top address and the range (data length) of the data specified in the remove request.
- the FA-LBA translator section 14 deletes the frame addresses to be deleted which are specified by the controller 12 from the layout correspondence table (arrow 2 in FIG. 5 ).
- the hard disk unit 10 performs defragmentation of data written in the magnetic disk 11 while the hard disk unit 10 has no request received from the file management section 20 , namely, while the hard disk unit 10 is under low-load.
- the correspondence between frame addresses and LBAs are managed by the hard disk unit 10 , so that defragmentation can be executed as an independent operation of the hard disk unit 10 .
- FIG. 6 is a diagram showing a reaction of defragmentation that the hard disk unit independently performs.
- the controller 12 inquires the FA-LBA translator section 14 to examine a file with a range of frame addresses not less than a single stride size and being discontinuously arranged on the magnetic disk (arrow 1 in FIG. 6 ). Moreover, it is examined whether data stored in the dirty and fragment cache area includes a data which can constitute a full stride with a data registered in the layout correspondence table (arrow 2 in FIG. 6 ).
- the controller 12 selects a single stride of a frame address from these examination results, and then reads a block which does not exist in the cache from the magnetic disk 11 .
- the controller 12 further recompiles the contents of the cache into stride units and holds the recompiled data in the clean/full cache area (arrow 3 in FIG. 6 ).
- the controller 12 writes the cache data held in the dirty/full cache area into a continuous area on the magnetic disk 11 (arrow 4 in FIG. 6 ).
- the cache data which has been written into the magnetic disk 11 is moved to the clean/full cache area.
- the FA-LBA translator section 14 registers data in an order from data that is write complete into the magnetic disk 11 , into the layout correspondence table the correspondence between the frame addresses and the LBAs (arrow 5 in FIG. 6 ). Concerning the data which was read from the magnetic disk 11 , the old correspondence between the frame address and the LBA is deleted.
- the cache data in the clean/full and the clean/fragmented cache area is first removed in accordance with a least recently used system (LRU) to secure a cache area.
- LRU least recently used system
- the data written into the magnetic disk 11 is moved to the clean/fragmented cache area and then removed from the cache area to secure free space.
- the hard disk unit independently rearranges data on the magnetic disk appropriately. Hence, data can be read and written in layouts on the magnetic disk which are advantageous in reading and writing data, respectively. Therefore, the read/write performance of data can be improved.
Abstract
A file management system including a hard disk unit and a file management unit. The file management unit manages data to be read and written from and into the hard disk unit such that data in a file is continuous in a predetermined logical address space and specifies a target data requested to be read and written from and into the hard disk unit by a predetermined logical address within the logical address space. The hard disk unit performs processing of reading and writing data after translating the predetermined logical addresses specifying the data requested to be read and written into LBAs. Then the hard disk unit performs defragmentation of data recorded on a magnetic disk based on the predetermined logical addresses.
Description
- This application is a continuation of U.S. patent application Ser. No. 10/923,991 filed on Aug. 23, 2004, which claims the benefit of priority Japanese Patent Application JP 2003-127866 filed May 6, 2003. The contents of the above Applications are incorporated herein by reference.
- 1. Field of the Invention
- The present invention relates to a control to read/write data from and into a magnetic disk unit, and specifically, relates to a logical operation for input-output data.
- 2. Background Art
- As a storage device for computer devices, a hard disk unit is widely used, which is randomly accessible and the read/write process is quick. Physically, the hard disk unit magnetically records and reads data into and from a data recording area of a magnetic disk, which is partitioned into tracks and sectors. However, data is inputted/outputted under management of a file system in terms of logical units called blocks independently of the physical partitions of the data recording area. The blocks constitute a file and is used in computer processing. The file system manages files on an external storage device and is provided as a function of an operating system (OS) or the like.
-
FIG. 7 is a diagram showing relationship between a general hard disk unit and a file system. - Referring to
FIG. 7 , ahard disk unit 710 includes amagnetic disk 711 as a recording medium, acontroller 712 which controls reading and writing data from and into themagnetic disk 711, and anonvolatile memory 713 used as a cache memory. Thefile system 720 issues requests (input and output requests) for thehard disk unit 710 to read and write data in block units. Also, thefile system 720 uses an address called a logical block address (LBA) (see “Transistor Gijutsu”, p. 180-181, December 2001, CQ publishing, Co., Ltd., for example) to identify a block stored in thehard disk unit 710. - In a system like this, when a file I/O (input/output) request arrives at the
file system 720 from an application or the like, thefile system 720 creates and issues to thehard disk unit 710 input/output requests necessary to handle the request. Next, when thefile system 720 issues a write request to thehard disk unit 710, such as a case where there is a request to write data into a file, thefile system 720 determines a layout of data on thehard disk unit 710. - The hard disk unit makes a magnetic head to seek to a desired track on a rotating magnetic disk to read and write data. For this structural reason, when the hard disk unit performs operations for blocks having consecutive LBAs in both reading and writing data (sequential reading and writing), the seek operation of the magnetic head is minimized, and the performance thereof is maximized. However, it is very difficult that the
hard disk unit 710 translates the layout of blocks which has been translated into LBAs in the input/output requests outputted from thefile system 720 into a layout enabling a good input/output performance. This is because it is generally impossible to estimate the layout structure on thehard disk unit 710 from a write designation having been translated into LBAs. - Therefore, the
file system 720 has hitherto determined a proper layout on thehard disk unit 710 to improve the performance of executing input/output requests. - In writing data, in the case where write requests for a plurality of files arrive at the
hard disk unit 710 in parallel, if sequential writing is performed in the arrival order of the write requests, Rblocks storing data of the respective files divide each other. This state is called fragmentation. - On the other hand, in reading data, even when read requests for a plurality of files are simultaneously generated, since the contents of the files can be read ahead and cached in a file system level, it is desirable that the files are sequentially arranged, namely, it is desirable that there is no fragmentation.
- Accordingly, the
conventional file system 720 often takes a following strategy. If possible, thefile system 720 does not immediately perform writing into thehard disk unit 710 in response to the write request and stores to some extent data to be written in the cache memory. Thefile system 720 then determines a proper layout and performs writing into the magnetic disk. By storing the data in the cache memory, input/output transactions concerning writing into the same area are reduced, and unnecessary movement of the magnetic head of thehard disk unit 710 is reduced, thus increasing writing throughput, and as well as a layout can be employed which allows less fragmentation in order not to bring disadvantages in reading. - As described above, in reading/writing data in a hard disk unit, compatible implementation of sequential reading and writing contributes to an increase in performance. Accordingly, a block layout has hitherto been controlled by using the cache memory.
- However, when a commit write request is generated in the file system, the file system must immediately write into an external storage device. In this case, since data to be written cannot be stored in the cache memory, the file system is obliged to change an original seek schedule of the magnetic head, and the layout on the magnetic disk inevitably becomes one where fragmentation tends to occur.
- Even if the commit write request is not generated, it remains that the contents of the cache memory must be written into the magnetic disk before the cache memory fully stores data to a storage capacity thereof. As described above, in order to obtain the layout on the magnetic disk advantageous in reading data, the data are written into discrete areas, and it is difficult to perform fast writing which is possible at sequential writing.
- As a result, even if optimization is performed using the cache memory, it remains that a possible write speed of the hard disk unit limits the writing performance of the file system.
- The layout on the magnetic disk advantageous in writing and the layout on the magnetic disk advantageous in reading are generally different from each other as described above. Therefore, if data is written into the magnetic disk in the layout advantageous in writing and rearranged to a layout advantageous in reading before reading the data, the performance of the file system can be significantly improved.
- However, the conventional file system and hard disk unit use LBAs to specify a target where data is written, and information concerning the continuity of blocks as a file is owned by the file system and not known by the hard disk unit. As a result, the hard disk unit cannot identify whether data is written into a predetermined LBA has an optimum arrangement, or the arrangement of the predetermined LBA is inappropriate but the data is written into the predetermined LBA because commit write must be executed. Therefore, the hard disk unit cannot perform defragmentation (rearrangement of data on a magnetic disk to eliminate fragmentation) as a function thereof.
- In the light of such situations, defragmentation of data (block) stored in the hard disk has hitherto been performed by control of the file system, but this has put an extra load to a computer having the file system. Especially, even in the case where the file system operates a plurality of external storage devices, there is a disadvantage that the rearrangement processing in one of the storage devices burdens the entire file system.
- In order for the hard disk unit to freely change the layout of data on the magnetic disk, a construction can be conceived in which the hard disk unit incorporates the file system.
- With this construction, the hard disk unit includes all information necessary for optimization of the data arrangement. Accordingly, the reduction in the read/write performance due to the layout of data on the magnetic disk can be avoided, or at least can be dissolved by the hard disk unit alone.
- However, there is a problem in mounting the file system on the hard disk unit.
- For example, the file size which can be handled by the storage capacity of the hard disk unit may be prescribed. As an example, it is assumed that four hard disk units each having a storage capacity of 60 gigabytes are connected to be used. In this case, if the file system is located on the OS side and the file system (and file system format thereof) supports a Logical Volume Manager function, a file of 240 (=60×4) gigabytes can be created. However, if the hard disk unit incorporates the entire file system, the storage capacity of the hard disk unit becomes an upper limit of the file size, or the OS is required to have a function to make a plurality of files appear as a single file. Both are not practical from the viewpoint of the input-output throughput.
- The function required for the file system varies depending on the OS. When the file system is incorporated in the hard disk unit, the file system of the hard disk unit is inevitably specialized in a predetermined OS. Therefore, the hard disk unit depends on the OS and cannot be used for general purposes.
- An object of the present invention which achieves the aforementioned objects is to provide a file management system which enables data on a magnetic disk to be rearranged by control of a hard disk unit itself, and the management method thereof.
- Another object of the present invention is to improve the read/write performance of data from and into the magnetic disk.
- In order to achieve the above described objects, the present invention is implemented as a magnetic disk unit constructed as follows. Specifically, this magnetic disk unit includes: a magnetic disk as a recording medium; a controller which controls reading/writing data from and into the magnetic disk; and an address translator section. The address translator section translates between first logical addresses used to specify predetermined data when reading/writing the data from and into the magnetic disk and second logical addresses used to exchange the data with an external device.
- More specifically, the controller rearranges data (defragmentation) recorded on the magnetic disk based on the second logical addresses. Additionally, the second logical addresses used here are set in address space for each file such that data within a single file is continuous.
- Moreover, when predetermined data is no longer used, the controller releases the second logical addresses associated with the disused data.
- In addition, the aforementioned magnetic disk unit according to the present invention can further include a cache memory which holds data to be written into the magnetic disk and data read from the magnetic disk. In this case, the controller classifies and manages data held in the cache memory based on whether or not the data is saved in the magnetic disk and assignment of the second logical addresses to the data, and rearranges data recorded on the magnetic disk in accordance with the classification.
- Another magnetic disk unit according to the present invention including: a magnetic disk as a recording medium; a controller which controls reading and writing data from and into the magnetic disk; and a cache memory. The controller holds target data for reading/writing process from and into the magnetic disk in the cache memory after classifying the data based on whether or not the data is stored in the magnetic disk and assignment of predetermined logical addresses to the data, and stores the data held in the cache memory into the magnetic disk in accordance with the classification to optimize a layout of the data stored in the magnetic disk.
- Herein, the controller selects from data held in the cache memory a data which is at least not contained in the magnetic disk and assigned with the logical addresses consecutive over a length of not less than a single stride, and stores the data in a continuous recording area of the magnetic disk. The stride is a management unit of cache data, and indicates a size allowing reading and writing into the magnetic disk to be performed with enough efficiency. In other words, even if reading and writing is performed in terms of a unit larger than the stride, the efficiency is little improved. The stride is a size uniquely determined for the entire system.
- Another aspect of the present invention to achieve the aforementioned object is implemented as a file management system including an external storage device of a computer; and a file management section which manages files stored in the external storage device. In this file management system, the file management section manages data to be read and written from and into the external storage device such that data in a file is continuous in a predetermined logical address space, and specifies data requested to be read and written from and into the external storage device with logical addresses in the logical address space. The external storage device translates the logical addresses specifying the data requested to be read and written into logical block addresses (LBAs), and performs processing of reading and writing data from and into a recording medium.
- More preferably, the file management section issues a command to the external storage device to remove disused data. The external storage device removes information concerning the data specified by the command from correspondence information for translation between the logical addresses and the LBAs in accordance with the command.
- Furthermore, the present invention is implemented as a method of managing files in an external storage device of a computer as follows. The method of managing files includes the steps of: receiving a read/write request on data specified by a logical address, which is set such that data in a file is continuous in a predetermined logical address space; and translating the logical addresses of target data in the read/write request to a logical block address (LBA) and performing read/write process from and into a recording medium based on the LBA.
- Another method of managing files according to the present invention includes the steps of: performing read/write process from and into a recording medium based on a read/write request on data received from the computer; holding target data of the read/write process into a cache memory after classifying the data based on whether or not the data is stored in the recording medium and assignment of predetermined logical addresses to the data; and optimizing a layout of the data stored in the recording medium by storing data held in the cache memory into the recording medium in accordance with the classification at a predetermined time.
- For a more complete understanding of the present invention and the advantages thereof, reference is now made to the following description taken in conjunction with the accompanying drawings.
-
FIG. 1 is a view showing a concept of a frame address used in an embodiment. -
FIG. 2 is a diagram showing a construction of a file management system in the embodiment. -
FIG. 3 is a diagram showing an operation in the case where a write request is issued from a file management section to a hard disk unit in the embodiment. -
FIG. 4 is a diagram showing an operation in the case where a read request is issued from the file management section to the hard disk unit in the embodiment. -
FIG. 5 is a diagram showing an operation in the case where a remove request is issued from the file management section to the hard disk unit in the embodiment. -
FIG. 6 is a diagram showing an operation of defragmentation that the hard disk unit independently performs in the embodiment. -
FIG. 7 is a diagram showing a relationship between a general hard disk unit and a file system. - Although the preferred embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and alternations can be made therein without departing from spirit and scope of the invention as defined by the appended claims.
- Hereinafter, the present invention will be described in detail based on an embodiment shown in the accompanying drawings.
- In the present embodiment, logical address space called frame address is newly defined, and data stored in the hard disk unit is managed using this address space.
-
FIG. 1 is a diagram showing a concept of the frame address. - As shown in
FIG. 1 , the frame address is a bit string expressing huge address space. In an example shown in the drawing, the bit length of the frame address is assumed to be 1024 bits, but actually may be any length as long as it is long enough compared to the storage capacity of the hard disk unit. - The file system arranges individual files on frame address space such that data in each file is continuous and free space large enough is arranged between each files, namely, the presence of another file does not cause insufficiency of an area for a desired file to be assigned. For example, in a hard disk unit with a storage capacity of 10 gigabytes, when a space of 10 gigabytes is given for a file, enough free space is secured between files, and data in a file can certainly be made continuous on the frame address space. In the frame address shown in
FIG. 1 , the high-order 512 bits of 1024 bits are assigned to a file handle, and the low-order 512 bits are assigned to an offset (byte) from the top of the file. - In writing, the hard disk unit writes data into the magnetic disk while creating a correspondence table between frame addresses and an actual layout on the magnetic disk. The layout on the magnetic disk in writing is properly determined by a controller of the hard disk unit. On the other hand, in reading, the hard disk unit reads data of a specified frame addresses based on the layout correspondence table created at writing. In the case where a block holding frame addresses adjacent thereto is located at a position apart therefrom on the magnetic disk, the controller performs defragmentation process based on the read data to optimize the layout of data on the magnetic disk if the controller determines it to be appropriate. Also, the controller may independently perform the defragmentation process at a time determined appropriate. Since there is no change in the frame addresses by the defragmentation processing, the defragmentation processing can be performed transparently from the file system.
- Since the frame address space does not directly correspond to the physical blocks on the magnetic disk unlike the LBAs, it is required to explicitly recover disused frame addresses and the corresponding physical blocks on the magnetic disk. Specifically, in the case of LBAs, the LBAs directly corresponded to the physical blocks on the magnetic disk. Therefore, when a predetermined LBA was no longer used, it was possible to reuse the disused LBA and the physical block on the magnetic disk by using the relevant LBA for another purpose. On the contrary, in the case of frame addresses, the LBAs corresponding to the disused frame addresses and the physical blocks on the magnetic disk are not reused for another purpose, so that it is required to explicitly release the disused address. Accordingly, when a predetermined file on the file system is deleted, the frame addresses corresponding to the file is explicitly removed, namely, eliminated from the layout correspondence table.
-
FIG. 2 is a diagram showing a construction of the file management system according to the present embodiment. - Referring to
FIG. 2 , the file management system of the present embodiment includes ahard disk unit 10 and afile management section 20 which controls thehard disk unit 10 to read and write files. Thehard disk unit 10 is an external storage device connected to a computer device. Thefile management section 20 is a software block implemented by a program-controlled CPU of the computer device using thehard disk unit 10 as the external storage device. Specifically, thefile management section 20 is implemented by a file system provided as a function of the OS. Blocks (data) exchanged between thehard disk unit 10 and thefile management section 20 are specified by the frame addresses (FAs) in accordance with a read/write request from thefile management section 20. - The
hard disk unit 10 includes amagnetic disk 11 as a recording medium, acontroller 12 which controls reading and writing data from and into themagnetic disk 11, acache memory 13 implemented by a nonvolatile memory or the like, and an FA-LBA translator section 14 which translates between frame addresses and LBAs. Thecontroller 12 and the FA-LBA translator section 14 are software blocks each implemented by a processor or the like controlled by a program stored in a not-shown ROM or the like. - The
magnetic disk 11 magnetically records data under the control of thecontroller 12. Thecontroller 12 receives requests to write and read data from thefile management section 20 and controls various units (not shown) to perform mechanical operations such as seek of the magnetic head and read and write data from and into themagnetic disk 11. Blocks exchanged between thecontroller 12 and themagnetic disk 11 are specified by LBAs. Thecache memory 13 temporarily holds data read from themagnetic disk 11, the read/write requests sent from thefile management section 20, and data contained in these requests. The FA-LBA translator section 14 translates between the frame addresses used to specify blocks between thefile management section 20 and thecontroller 12, and the LBAs used to specify blocks between thecontroller 12 and themagnetic disk 11. The translation of the address space uses the layout correspondence table. This layout correspondence table is created during writing data into themagnetic disk 11 and held in the cache memory 13 (indicated by a dashed line inFIG. 2 ). - A cache area of the
cache memory 13 is classified into a total of four areas with two categories. - The first category is classified into “clean” and “dirty,” which indicate whether cache data has already been saved on the magnetic disk. The “clean” indicates that an image on the magnetic disk agrees with the cache data (the data is saved). The “dirty” indicates that the image on the magnetic disk does not agree with data in the cache (the data is not saved) and therefore the cache data needs to be saved on the magnetic disk.
- The second category is classified into “full” and “fragmented”, which indicate assignment of the frame addresses to data. The cache manages cache data in terms of a unit called stride. In the case of reading, the stride is a size large enough so that sequential reading of data on the magnetic disk is made with efficiency. The “full” indicates that all the contents within a stride is composed of blocks constituted by consecutive frame addresses. The “fragmented” indicates that the contents of a stride is composed of blocks which are pointed out by nonconsecutive frame addresses.
- Therefore, the cache area of the
cache memory 13 is classified into four types of a clean/full cache area, a clean/fragmented cache area, a dirty/full cache area, and a dirty/fragmented cache area. - Next, a description will be given of an operation in the embodiment for each request issued from the
file management section 20 to thehard disk unit 10. - 1. Operation for a Write Request
-
FIG. 3 is a diagram showing an operation in the case where a write request is issued from thefile management section 20 to thehard disk unit 10. - First, the
controller 12 receives the write request and stores data to be written into the cache memory 13 (arrow 1 inFIG. 3 ). At this point, the data is managed with frame addresses. When the amount of data to be written fulfills a stride, the data is registered in the dirty/full cache area. On the contrary, when the data to be written does not fulfill a stride, frame addresses of the data and other cache data are compared. When a combination of data which fulfills a stride is made, such data are collected and registered in the dirty/full cache area. In other cases, the data to be written is registered in the dirty/fragmented cache area. - Next, the
controller 12 writes the cache data held in the dirty/full cache area into the magnetic disk (arrow 2 inFIG. 3 ). At this time, an area where the data is to be written is determined on the free space of the magnetic disk such that a cache image of a single stride has consecutive LBAs. The cache data which has been written is moved to the clean/full cache area. - Next, concerning the data written into the magnetic disk 11 (cache data moved to the clean/full cache area), the FA-
LBA translator section 14 registers corresponding frame addresses and LBAs in the layout correspondence table (arrow 3 inFIG. 3 ). When the frame addresses of the cache data held in the dirty/fragmented cache area are registered in the layout correspondence table, these frame addresses are released. - 2. Operation for a Read Request
-
FIG. 4 is a diagram showing an action in the case where a read request is issued from thefile management section 20 to thehard disk unit 10. - First, the
controller 12 accepts the read request (arrow 1 inFIG. 4 ). Thecontroller 12 inquires the FA-LBA translator section 14 to acquire the LBAs corresponding to the frame addresses in a range requested to be read (arrow 2 inFIG. 4 ), and checks whether data in the range to be read is in the cache of the cache memory 13 (arrow 3 inFIG. 4 ). The data in the cache memory can be part of data in the range to be read. - Next, the
controller 12 reads part of data to be read which was not in thecache memory 13 from themagnetic disk 11 to the cache memory 13 (arrow 4 inFIG. 4 ). At this time, data outside the required range is read such that the size of the data to be read is the same as the full stride size. In the case where the layout of the data to be read is discontinuous on themagnetic disk 11 at a stride size level, thecontroller 12 stores the data read with the full stride size into the dirty/full cache area in thecache memory 13. This allows the data of the full stride size to be continuously written into themagnetic disk 11 later. In other cases, the read data is stored in the clean/full data cache area. - When the range of the frame addresses corresponding to the read data does not reach the full stride size, the
controller 12 arranges the cached data blocks sequentially as possible and resisters the data as the dirty/fragmented data. - Next, the
controller 12 returns the data in the range requested to be read to the file management section 20 (arrow 5 inFIG. 4 ). Reading data requested from thefile management section 20 is thus completed. - Thereafter, the
controller 12 writes the cache data held in the dirty/full cache area into the magnetic disk 11 (arrow 6 inFIG. 4 ). At this time, free areas are utilized such that the cache data can be written into themagnetic disk 11 in full strides and that the individual strides are arranged in frame address order. The cache images are sequentially moved to the clean/full cache area on finishing being written into themagnetic disk 11. In the end, the FA-LBA translator section 14 sequentially registers into the layout correspondence table the cache data which is write complete. (arrow 7 inFIG. 4 ) In the aforementioned operations, the old correspondence concerning the data read from themagnetic disk 11 is deleted from the layout correspondence table. - 3. Operation for a Remove Request
- As previously described, it is required to explicitly recover frame address space corresponding to a file removed on the
file management section 20. In the embodiment, a delete command called a remove request is defined and issued from thefile management section 20. Thus, frame addresses corresponding to the disused data are released. The remove request specifies the top address (frame address) and the data length of the data which is no longer to be used. -
FIG. 5 is a diagram showing an operation when a remove request is issued from thefile management section 20 to thehard disk unit 10. - First, the
controller 12 accepts the remove request (arrow 1 inFIG. 5 ), and then identifies all frame addresses to be deleted based on the top address and the range (data length) of the data specified in the remove request. Next, the FA-LBA translator section 14 deletes the frame addresses to be deleted which are specified by thecontroller 12 from the layout correspondence table (arrow 2 inFIG. 5 ). - 4. Operation with No Request Accepted
- The
hard disk unit 10 according to the embodiment performs defragmentation of data written in themagnetic disk 11 while thehard disk unit 10 has no request received from thefile management section 20, namely, while thehard disk unit 10 is under low-load. In the embodiment, the correspondence between frame addresses and LBAs are managed by thehard disk unit 10, so that defragmentation can be executed as an independent operation of thehard disk unit 10. -
FIG. 6 is a diagram showing a reaction of defragmentation that the hard disk unit independently performs. - The
controller 12 inquires the FA-LBA translator section 14 to examine a file with a range of frame addresses not less than a single stride size and being discontinuously arranged on the magnetic disk (arrow 1 inFIG. 6 ). Moreover, it is examined whether data stored in the dirty and fragment cache area includes a data which can constitute a full stride with a data registered in the layout correspondence table (arrow 2 inFIG. 6 ). - Next, the
controller 12 selects a single stride of a frame address from these examination results, and then reads a block which does not exist in the cache from themagnetic disk 11. Thecontroller 12 further recompiles the contents of the cache into stride units and holds the recompiled data in the clean/full cache area (arrow 3 inFIG. 6 ). - Next, the
controller 12 writes the cache data held in the dirty/full cache area into a continuous area on the magnetic disk 11 (arrow 4 inFIG. 6 ). The cache data which has been written into themagnetic disk 11 is moved to the clean/full cache area. - In the end, the FA-
LBA translator section 14 registers data in an order from data that is write complete into themagnetic disk 11, into the layout correspondence table the correspondence between the frame addresses and the LBAs (arrow 5 inFIG. 6 ). Concerning the data which was read from themagnetic disk 11, the old correspondence between the frame address and the LBA is deleted. - Next, reuse of the cache will be described.
- When data contained in the write request from the
file management section 20 or the data read from themagnetic disk 11 is for cache use, in the case where there is no free space in the cache area of thecache memory 13, the cache data in the clean/full and the clean/fragmented cache area is first removed in accordance with a least recently used system (LRU) to secure a cache area. There is no problem if the data belonging to the clean category is deleted because the data in the clean category has already been recorded on themagnetic disk 11. - Since data in the dirty/full cache area is written into the
magnetic disk 11 capable of being written into as appropriate, it is impossible that data in use remains in this cache area. - When area could not be acquired from both the clean/full and the clean/fragmented cache areas, free space is secured from the dirty/fragmented cache area. In this case, since data held in this cache area is waiting to be written into the
magnetic disk 11, the data in the cache area cannot be just discarded. Therefore, cache images therein are written into themagnetic disk 11 in accordance with the LRU. In this case, the strides are continuously written into themagnetic disk 11. The layout correspondence table corresponding to the strides written into themagnetic disk 11 is updated. - The data written into the
magnetic disk 11 is moved to the clean/fragmented cache area and then removed from the cache area to secure free space. - Incidentally, when the utilization rate of the
magnetic disk 11 of thehard disk unit 10 is near 100%, effort required for the defragmentation process generally increases. Therefore, to prevent the utilization rate of themagnetic disk 11 from increasing to near 100%, instead of allowing users to use the entire storage capacity of themagnetic disk 11, preferably, a capacity to be used by thecontroller 12 is previously secured, and a capacity remaining after subtracting this margin may be presented to thefile management section 20. - As described above, according to the present invention, it is possible to provide a file management system which can rearrange data (defragmentation) on the magnetic disk by control of the hard disk unit itself and to provide a management method thereof.
- Moreover, according to the present invention, the hard disk unit independently rearranges data on the magnetic disk appropriately. Hence, data can be read and written in layouts on the magnetic disk which are advantageous in reading and writing data, respectively. Therefore, the read/write performance of data can be improved.
Claims (5)
1. A method of managing files in an external storage device of a computer, comprising the steps of:
performing deletion of data on a recording medium based on a remove request received from the computer;
classifying the data based on whether or not the data is stored in the recording medium and assignment of a predetermined second logical address in the recording medium to the data,
holding in a cache memory target data of a read/write process between the external storage device and the computer,
classifying and managing data held in the cache memory as one of clean/full, clean/fragmented, dirty/full, and dirty/fragmented based on whether or not the data is saved in the recording medium, wherein clean indicates that data has been saved, dirty indicates that data has not been saved, full indicates that all of the contents within a stride is composed of blocks of consecutive frame addresses, and fragmented indicates that the contents of a stride is composed of blocks of nonconsecutive frame addresses;
optimizing a layout of the data stored in the recording medium by storing data held in the cache memory into the recording medium in accordance with the classification, based on the second logical address, at a predetermined time to eliminate fragmentation; and
defragmenting data recorded on the recording medium in accordance with the classification of the data responsive to whether or not the data is stored in the recording medium and whether or not the data has a second logical address.
2. The method described in claim 1 , further comprising the steps of
registering data held in the cache memory in a layout correspondence table; and
updating the layout correspondence table responsive to the deletion of data.
3. A method of performing defragmentation of data written on a recording medium from a computer comprising the steps of:
performing a read/write process from and into the recording medium based on a read/write request on the data received from the computer;
holding in a cache memory target data of the read/write process; said cache memory holding data to be written to the recording medium or data read from the recording medium;
classifying and managing data held in the cache memory as one of clean/full, clean/fragmented type, dirty/full type, and dirty/fragmented type based on whether or not the data is saved in the recording medium, wherein clean indicates that data has been saved, dirty indicates that data has not been saved, full indicates that all of the contents within a stride is composed of blocks of consecutive frame addresses, and fragmented indicates that the contents of a stride is composed of blocks of nonconsecutive frame addresses;
registering data held in the cache memory in a layout correspondence table;
examining a file with a range of the frame addresses; the file having not less than a single stride size and being discontinuously arranged on the magnetic disk;
selecting at least one stride of a frame address by examining whether data of the dirty/fragment type includes data which can constitute a full stride and comparing with data registered in the layout correspondence table;
recompiling data in the cache memory into stride units to produce recompiled data and holding the recompiled data as clean/full type;
writing data held in the dirty/full cache area into a continuous area on the recording medium; and
updating the layout correspondence table responsive to the read/write process.
4. A method of reusing cache memory in an external storage device of a computer comprising the steps of:
performing a read/write process from and into a recording medium based on a read/write request on data received from the computer;
holding in the cache memory target data of the read/write process; said cache memory holding data to be written to the recording medium or data read from the recording medium;
classifying and managing data held in the cache memory as one of clean/full, clean/fragmented type, dirty/full type, and dirty/fragmented type based on whether or not the data is saved in the recording medium, wherein clean indicates that data has been saved, dirty indicates that data has not been saved, full indicates that all of the contents within a stride is composed of blocks of consecutive frame addresses, and fragmented indicates that the contents of a stride is composed of blocks of nonconsecutive frame addresses;
removing data from clean/full and the clean/fragmented cache areas; and
if additional cache space is needed, removing of data from the dirty/fragmented cache area.
5. The method in claim 4 further comprising the steps of
registering data held in the cache memory in a layout correspondence table; and
updating the layout correspondence table in response to writing data in the cache memory to the recording medium.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/971,368 US20080140905A1 (en) | 2003-05-06 | 2008-01-09 | Magnetic disk unit, file management system, and file management method |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2003127866A JP4257834B2 (en) | 2003-05-06 | 2003-05-06 | Magnetic disk device, file management system and method thereof |
JP2003-127866 | 2003-05-06 | ||
US10/923,991 US7350017B2 (en) | 2003-05-06 | 2004-08-23 | Magnetic disk unit, file management system, and file management method |
US11/971,368 US20080140905A1 (en) | 2003-05-06 | 2008-01-09 | Magnetic disk unit, file management system, and file management method |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/923,991 Continuation US7350017B2 (en) | 2003-05-06 | 2004-08-23 | Magnetic disk unit, file management system, and file management method |
Publications (1)
Publication Number | Publication Date |
---|---|
US20080140905A1 true US20080140905A1 (en) | 2008-06-12 |
Family
ID=33504220
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/923,991 Expired - Fee Related US7350017B2 (en) | 2003-05-06 | 2004-08-23 | Magnetic disk unit, file management system, and file management method |
US11/971,368 Abandoned US20080140905A1 (en) | 2003-05-06 | 2008-01-09 | Magnetic disk unit, file management system, and file management method |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/923,991 Expired - Fee Related US7350017B2 (en) | 2003-05-06 | 2004-08-23 | Magnetic disk unit, file management system, and file management method |
Country Status (2)
Country | Link |
---|---|
US (2) | US7350017B2 (en) |
JP (1) | JP4257834B2 (en) |
Cited By (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080104193A1 (en) * | 2005-02-09 | 2008-05-01 | Hitachi, Ltd. | Clustered Storage System with External Storage Systems |
US8688660B1 (en) * | 2010-09-28 | 2014-04-01 | Amazon Technologies, Inc. | System and method for providing enhancements of block-level storage |
US20140359211A1 (en) * | 2013-06-03 | 2014-12-04 | Samsung Electronics Co., Ltd. | Method for disk defrag handling in solid state drive caching environment |
US9110600B1 (en) | 2012-03-19 | 2015-08-18 | Amazon Technologies, Inc. | Triggered data shelving to a different storage system and storage deallocation |
US9246996B1 (en) | 2012-05-07 | 2016-01-26 | Amazon Technologies, Inc. | Data volume placement techniques |
US9304867B2 (en) | 2010-09-28 | 2016-04-05 | Amazon Technologies, Inc. | System and method for providing flexible storage and retrieval of snapshot archives |
US9503517B1 (en) | 2012-05-07 | 2016-11-22 | Amazon Technologies, Inc. | Data volume placement techniques |
US9563385B1 (en) | 2013-09-16 | 2017-02-07 | Amazon Technologies, Inc. | Profile-guided data preloading for virtualized resources |
US9804993B1 (en) | 2012-05-07 | 2017-10-31 | Amazon Technologies, Inc. | Data volume placement techniques |
US9823840B1 (en) | 2012-05-07 | 2017-11-21 | Amazon Technologies, Inc. | Data volume placement techniques |
US10250673B1 (en) | 2014-03-14 | 2019-04-02 | Amazon Technologies, Inc. | Storage workload management using redirected messages |
US10452296B1 (en) | 2018-03-23 | 2019-10-22 | Amazon Technologies, Inc. | Accelerated volumes |
CN111090396A (en) * | 2019-12-10 | 2020-05-01 | 联想(北京)有限公司 | File processing method and device and electronic equipment |
US10715460B2 (en) | 2015-03-09 | 2020-07-14 | Amazon Technologies, Inc. | Opportunistic resource migration to optimize resource placement |
US10768835B1 (en) | 2018-06-27 | 2020-09-08 | Amazon Technologies, Inc. | Opportunistic storage service |
US10776173B1 (en) | 2018-04-30 | 2020-09-15 | Amazon Technologies, Inc. | Local placement of resource instances in a distributed system |
US10931750B1 (en) | 2018-07-30 | 2021-02-23 | Amazon Technologies, Inc. | Selection from dedicated source volume pool for accelerated creation of block data volumes |
US10956442B1 (en) | 2018-07-30 | 2021-03-23 | Amazon Technologies, Inc. | Dedicated source volume pool for accelerated creation of block data volumes from object data snapshots |
US10983719B1 (en) | 2019-03-28 | 2021-04-20 | Amazon Technologies, Inc. | Replica pools to support volume replication in distributed storage systems |
US11023157B2 (en) | 2018-04-30 | 2021-06-01 | Amazon Technologies, Inc. | Intermediary duplication to facilitate copy requests in distributed storage systems |
US11068192B1 (en) | 2019-03-26 | 2021-07-20 | Amazon Technologies, Inc. | Utilizing mutiple snapshot sources for creating new copy of volume in a networked environment wherein additional snapshot sources are reserved with lower performance levels than a primary snapshot source |
US11121981B1 (en) | 2018-06-29 | 2021-09-14 | Amazon Technologies, Inc. | Optimistically granting permission to host computing resources |
US11182095B2 (en) | 2018-04-30 | 2021-11-23 | Amazon Technologies, Inc. | Rapid volume backup generation from distributed replica |
US11262918B1 (en) | 2020-09-30 | 2022-03-01 | Amazon Technologies, Inc. | Data storage system with uneven drive wear reduction |
US11343314B1 (en) | 2018-04-30 | 2022-05-24 | Amazon Technologies, Inc. | Stream-based logging for distributed storage systems |
US11379354B1 (en) | 2012-05-07 | 2022-07-05 | Amazon Technologies, Inc. | Data volume placement techniques |
US11526286B1 (en) | 2021-06-29 | 2022-12-13 | Amazon Technologies, Inc. | Adaptive snapshot chunk sizing for snapshots of block storage volumes |
Families Citing this family (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7692310B2 (en) * | 2006-03-27 | 2010-04-06 | Intel Corporation | Forming a hybrid device |
JP2007328856A (en) * | 2006-06-07 | 2007-12-20 | Toshiba Corp | Magnetic disk drive and data recording method |
JP4984677B2 (en) * | 2006-06-27 | 2012-07-25 | ソニー株式会社 | Information processing device |
US7562203B2 (en) * | 2006-09-27 | 2009-07-14 | Network Appliance, Inc. | Storage defragmentation based on modified physical address and unmodified logical address |
US20080162821A1 (en) * | 2006-12-27 | 2008-07-03 | Duran Louis A | Hard disk caching with automated discovery of cacheable files |
KR100891333B1 (en) | 2007-04-11 | 2009-03-31 | 삼성전자주식회사 | Hard disk drive capable of data writing/reading regardless of file system and method thereof |
US8359430B1 (en) * | 2007-08-30 | 2013-01-22 | Network Appliance, Inc. | Techniques for efficient mass storage layout optimization |
KR20090035938A (en) * | 2007-10-08 | 2009-04-13 | 삼성전자주식회사 | Apparatus and method for disk defragment in hybrid hard disk |
US8325724B2 (en) | 2009-03-31 | 2012-12-04 | Emc Corporation | Data redistribution in data replication systems |
EP2256615A1 (en) * | 2009-05-29 | 2010-12-01 | Thomson Licensing | Self-defragmenting storage device |
US8521972B1 (en) | 2010-06-30 | 2013-08-27 | Western Digital Technologies, Inc. | System and method for optimizing garbage collection in data storage |
US9189392B1 (en) * | 2011-06-30 | 2015-11-17 | Western Digital Technologies, Inc. | Opportunistic defragmentation during garbage collection |
US8819375B1 (en) | 2011-11-30 | 2014-08-26 | Western Digital Technologies, Inc. | Method for selective defragmentation in a data storage device |
US8788778B1 (en) | 2012-06-04 | 2014-07-22 | Western Digital Technologies, Inc. | Garbage collection based on the inactivity level of stored data |
US20140281194A1 (en) * | 2013-03-15 | 2014-09-18 | Seagate Technology Llc | Dynamically-sizeable granule storage |
US9588886B2 (en) | 2013-03-15 | 2017-03-07 | Seagate Technology Llc | Staging sorted data in intermediate storage |
US9436606B2 (en) * | 2014-01-02 | 2016-09-06 | Qualcomm Incorporated | System and method to defragment a memory |
JP2016018473A (en) * | 2014-07-10 | 2016-02-01 | 株式会社東芝 | Semiconductor storage device, memory controller, and memory controller control method |
US10503445B2 (en) * | 2015-03-23 | 2019-12-10 | Netapp, Inc. | Data structure store and data management |
JP6691669B2 (en) * | 2015-07-06 | 2020-05-13 | 日本電気株式会社 | Information processing system, storage control device, storage control method, and storage control program |
US10877701B2 (en) | 2017-02-23 | 2020-12-29 | Hitachi, Ltd. | Scale-out type storage system |
JP7069811B2 (en) * | 2018-02-22 | 2022-05-18 | 富士通株式会社 | Information processing equipment and information processing method |
CN113360095B (en) * | 2021-06-04 | 2023-02-17 | 重庆紫光华山智安科技有限公司 | Hard disk data management method, device, equipment and medium |
CN114461134A (en) * | 2021-11-19 | 2022-05-10 | 中航航空电子有限公司 | Hard disk zero fragment read-write device, method, computer equipment and storage medium |
CN114579055B (en) * | 2022-03-07 | 2023-01-31 | 重庆紫光华山智安科技有限公司 | Disk storage method, device, equipment and medium |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5206939A (en) * | 1990-09-24 | 1993-04-27 | Emc Corporation | System and method for disk mapping and data retrieval |
US5623701A (en) * | 1995-06-06 | 1997-04-22 | International Business Machines Corporation | Data compression method and structure for a direct access storage device |
US5652857A (en) * | 1995-03-09 | 1997-07-29 | Fujitsu Limited | Disk control apparatus for recording and reproducing compression data to physical device of direct access type |
US20010007120A1 (en) * | 1998-09-11 | 2001-07-05 | Fujitsu Limited | Storage device |
US20010014929A1 (en) * | 1998-05-22 | 2001-08-16 | Yuichi Taroda | Disk control device and method processing variable-block and fixed-block accesses from host devices |
US6389503B1 (en) * | 1997-08-04 | 2002-05-14 | Exabyte Corporation | Tape drive emulation by removable disk drive and media formatted therefor |
US6606651B1 (en) * | 2000-05-03 | 2003-08-12 | Datacore Software Corporation | Apparatus and method for providing direct local access to file level data in client disk images within storage area networks |
US6975752B2 (en) * | 2001-01-31 | 2005-12-13 | General Electric Company | Imaging system including detector framing node |
-
2003
- 2003-05-06 JP JP2003127866A patent/JP4257834B2/en not_active Expired - Fee Related
-
2004
- 2004-08-23 US US10/923,991 patent/US7350017B2/en not_active Expired - Fee Related
-
2008
- 2008-01-09 US US11/971,368 patent/US20080140905A1/en not_active Abandoned
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5206939A (en) * | 1990-09-24 | 1993-04-27 | Emc Corporation | System and method for disk mapping and data retrieval |
US5652857A (en) * | 1995-03-09 | 1997-07-29 | Fujitsu Limited | Disk control apparatus for recording and reproducing compression data to physical device of direct access type |
US5623701A (en) * | 1995-06-06 | 1997-04-22 | International Business Machines Corporation | Data compression method and structure for a direct access storage device |
US6389503B1 (en) * | 1997-08-04 | 2002-05-14 | Exabyte Corporation | Tape drive emulation by removable disk drive and media formatted therefor |
US20010014929A1 (en) * | 1998-05-22 | 2001-08-16 | Yuichi Taroda | Disk control device and method processing variable-block and fixed-block accesses from host devices |
US20010007120A1 (en) * | 1998-09-11 | 2001-07-05 | Fujitsu Limited | Storage device |
US6606651B1 (en) * | 2000-05-03 | 2003-08-12 | Datacore Software Corporation | Apparatus and method for providing direct local access to file level data in client disk images within storage area networks |
US6975752B2 (en) * | 2001-01-31 | 2005-12-13 | General Electric Company | Imaging system including detector framing node |
Cited By (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7895287B2 (en) * | 2005-02-09 | 2011-02-22 | Hitachi, Ltd. | Clustered storage system with external storage systems |
US20110113192A1 (en) * | 2005-02-09 | 2011-05-12 | Hitachi, Ltd. | Clustered storage system with external storage systems |
US8862812B2 (en) | 2005-02-09 | 2014-10-14 | Hitachi, Ltd. | Clustered storage system with external storage systems |
US20080104193A1 (en) * | 2005-02-09 | 2008-05-01 | Hitachi, Ltd. | Clustered Storage System with External Storage Systems |
US9304867B2 (en) | 2010-09-28 | 2016-04-05 | Amazon Technologies, Inc. | System and method for providing flexible storage and retrieval of snapshot archives |
US8688660B1 (en) * | 2010-09-28 | 2014-04-01 | Amazon Technologies, Inc. | System and method for providing enhancements of block-level storage |
US11543983B2 (en) | 2010-09-28 | 2023-01-03 | Amazon Technologies, Inc. | System and method for providing enhancements of block-level storage |
US9886213B2 (en) | 2010-09-28 | 2018-02-06 | Amazon Technologies, Inc. | System and method for providing flexible storage and retrieval of snapshot archives |
US9110600B1 (en) | 2012-03-19 | 2015-08-18 | Amazon Technologies, Inc. | Triggered data shelving to a different storage system and storage deallocation |
US11379354B1 (en) | 2012-05-07 | 2022-07-05 | Amazon Technologies, Inc. | Data volume placement techniques |
US9804993B1 (en) | 2012-05-07 | 2017-10-31 | Amazon Technologies, Inc. | Data volume placement techniques |
US9823840B1 (en) | 2012-05-07 | 2017-11-21 | Amazon Technologies, Inc. | Data volume placement techniques |
US9246996B1 (en) | 2012-05-07 | 2016-01-26 | Amazon Technologies, Inc. | Data volume placement techniques |
US9503517B1 (en) | 2012-05-07 | 2016-11-22 | Amazon Technologies, Inc. | Data volume placement techniques |
US20140359211A1 (en) * | 2013-06-03 | 2014-12-04 | Samsung Electronics Co., Ltd. | Method for disk defrag handling in solid state drive caching environment |
US10037281B2 (en) | 2013-06-03 | 2018-07-31 | Samsung Electronics Co., Ltd. | Method for disk defrag handling in solid state drive caching environment |
US9201799B2 (en) * | 2013-06-03 | 2015-12-01 | Samsung Electronics Co., Ltd. | Method for disk defrag handling in solid state drive caching environment |
US9563385B1 (en) | 2013-09-16 | 2017-02-07 | Amazon Technologies, Inc. | Profile-guided data preloading for virtualized resources |
US11061584B2 (en) | 2013-09-16 | 2021-07-13 | Amazon Technologies, Inc. | Profile-guided data preloading for virtualized resources |
US10250673B1 (en) | 2014-03-14 | 2019-04-02 | Amazon Technologies, Inc. | Storage workload management using redirected messages |
US10715460B2 (en) | 2015-03-09 | 2020-07-14 | Amazon Technologies, Inc. | Opportunistic resource migration to optimize resource placement |
US11093148B1 (en) | 2018-03-23 | 2021-08-17 | Amazon Technologies, Inc. | Accelerated volumes |
US10452296B1 (en) | 2018-03-23 | 2019-10-22 | Amazon Technologies, Inc. | Accelerated volumes |
US11023157B2 (en) | 2018-04-30 | 2021-06-01 | Amazon Technologies, Inc. | Intermediary duplication to facilitate copy requests in distributed storage systems |
US11343314B1 (en) | 2018-04-30 | 2022-05-24 | Amazon Technologies, Inc. | Stream-based logging for distributed storage systems |
US10776173B1 (en) | 2018-04-30 | 2020-09-15 | Amazon Technologies, Inc. | Local placement of resource instances in a distributed system |
US11182095B2 (en) | 2018-04-30 | 2021-11-23 | Amazon Technologies, Inc. | Rapid volume backup generation from distributed replica |
US10768835B1 (en) | 2018-06-27 | 2020-09-08 | Amazon Technologies, Inc. | Opportunistic storage service |
US11347413B2 (en) | 2018-06-27 | 2022-05-31 | Amazon Technologies, Inc. | Opportunistic storage service |
US11121981B1 (en) | 2018-06-29 | 2021-09-14 | Amazon Technologies, Inc. | Optimistically granting permission to host computing resources |
US10956442B1 (en) | 2018-07-30 | 2021-03-23 | Amazon Technologies, Inc. | Dedicated source volume pool for accelerated creation of block data volumes from object data snapshots |
US10931750B1 (en) | 2018-07-30 | 2021-02-23 | Amazon Technologies, Inc. | Selection from dedicated source volume pool for accelerated creation of block data volumes |
US11068192B1 (en) | 2019-03-26 | 2021-07-20 | Amazon Technologies, Inc. | Utilizing mutiple snapshot sources for creating new copy of volume in a networked environment wherein additional snapshot sources are reserved with lower performance levels than a primary snapshot source |
US10983719B1 (en) | 2019-03-28 | 2021-04-20 | Amazon Technologies, Inc. | Replica pools to support volume replication in distributed storage systems |
CN111090396A (en) * | 2019-12-10 | 2020-05-01 | 联想(北京)有限公司 | File processing method and device and electronic equipment |
US11262918B1 (en) | 2020-09-30 | 2022-03-01 | Amazon Technologies, Inc. | Data storage system with uneven drive wear reduction |
US11526286B1 (en) | 2021-06-29 | 2022-12-13 | Amazon Technologies, Inc. | Adaptive snapshot chunk sizing for snapshots of block storage volumes |
Also Published As
Publication number | Publication date |
---|---|
US20050021900A1 (en) | 2005-01-27 |
US7350017B2 (en) | 2008-03-25 |
JP4257834B2 (en) | 2009-04-22 |
JP2004334419A (en) | 2004-11-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7350017B2 (en) | Magnetic disk unit, file management system, and file management method | |
US6216199B1 (en) | Hardware mechanism for managing cache structures in a data storage system | |
US7401174B2 (en) | File system defragmentation and data processing method and apparatus for an information recording medium | |
US8065473B2 (en) | Method for controlling memory card and method for controlling nonvolatile semiconductor memory | |
US7769952B2 (en) | Storage system for controlling disk cache | |
US6021408A (en) | Methods for operating a log device | |
KR101354152B1 (en) | Method for command scheduling of virtual file system embodied in non-volatile data storage and apparatus thereof | |
US20050216665A1 (en) | Storage system and method for controlling block rearrangement | |
JPH096540A (en) | Built-in directory method for data compression of direct-access storage device and directory record | |
JP6608468B2 (en) | Storage apparatus and control method thereof | |
KR20100089229A (en) | Method and apparatus for data management in flash memory by address mapping | |
CN108628542B (en) | File merging method and controller | |
KR20000022716A (en) | Efficient volume copy using pre-configuration of log structured target storage | |
KR102264119B1 (en) | Casedb: low-cost put-intensive key-value store for edge computing | |
CN110968269A (en) | SCM and SSD-based key value storage system and read-write request processing method | |
JP2018181202A (en) | Device, method, and program for storage control | |
JP7131053B2 (en) | Storage device, information processing program and information processing system | |
JP4189342B2 (en) | Storage apparatus, storage controller, and write-back cache control method | |
JPH1063551A (en) | Information processor | |
Zhang et al. | Building GC-free key-value store on HM-SMR drives with ZoneFS | |
Chen et al. | Co-optimizing storage space utilization and performance for key-value solid state drives | |
JPS59220853A (en) | Disc cache system | |
US6209057B1 (en) | Storage device having data buffer | |
JP3111912B2 (en) | Disk cache control method | |
JP2006127377A (en) | Fat file system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:OKUYAMA, KENICHI;TAGO, KAZUYA;NEGISHI, YASUSHI;REEL/FRAME:020341/0419;SIGNING DATES FROM 20040804 TO 20040818 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE |