CA1322242C - Method for eliminating file fragmentation and reducing average seek times in a magnetic disk media environment - Google Patents
Method for eliminating file fragmentation and reducing average seek times in a magnetic disk media environmentInfo
- Publication number
- CA1322242C CA1322242C CA000615272A CA615272A CA1322242C CA 1322242 C CA1322242 C CA 1322242C CA 000615272 A CA000615272 A CA 000615272A CA 615272 A CA615272 A CA 615272A CA 1322242 C CA1322242 C CA 1322242C
- Authority
- CA
- Canada
- Prior art keywords
- files
- file
- disk
- free space
- storage medium
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/061—Improving I/O performance
- G06F3/0611—Improving I/O performance in relation to response time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/0644—Management of space entities, e.g. partitions, extents, pools
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0674—Disk device
- G06F3/0676—Magnetic disk device
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/12—Formatting, e.g. arrangement of data block or words on the record carriers
- G11B20/1217—Formatting, e.g. arrangement of data block or words on the record carriers on discs
- G11B20/1252—Formatting, e.g. arrangement of data block or words on the record carriers on discs for discontinuous data, e.g. digital information signals, computer programme data
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
- G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
- G11B27/32—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier
- G11B27/327—Table of contents
- G11B27/329—Table of contents on a disc [VTOC]
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B2020/10916—Seeking data on the record carrier for preparing an access to a specific address
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99952—Coherency, e.g. same view to multiple users
- Y10S707/99953—Recoverability
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99956—File allocation
- Y10S707/99957—Garbage collection
Abstract
METHOD FOR ELIMINATING FILE FRAGMENTATION
AND REDUCING AVERAGE SEEK TIMES IN A
MAGNETIC DISK MEDIA ENVIRONMENT
ABSTRACT OF THE DISCLOSURE
In accordance with the present invention, a method of intelli-gently organizing a magnetic storage medium is provided. The mag-netic storage medium has free space and a plurality of files distributed thereon. The plurality of files includes volatile files created within a first predetermined period of time. First, the free space is substan-tially consolidated on the magnetic storage medium. Then, the volatile files are substantially positioned contiguous to the consolidated free space.
AND REDUCING AVERAGE SEEK TIMES IN A
MAGNETIC DISK MEDIA ENVIRONMENT
ABSTRACT OF THE DISCLOSURE
In accordance with the present invention, a method of intelli-gently organizing a magnetic storage medium is provided. The mag-netic storage medium has free space and a plurality of files distributed thereon. The plurality of files includes volatile files created within a first predetermined period of time. First, the free space is substan-tially consolidated on the magnetic storage medium. Then, the volatile files are substantially positioned contiguous to the consolidated free space.
Description
~3222~2 METHOD FOR ELIMINATING FILE FRAGMENTATION
AND REDUCING AVERAGE SEEK TIMES IN A
;UAGNETIC DISK MEDIA ENVIRONMENT
BACK~;ROUND 3F THE INVEP~TION
The present invention generally relates to the field of elec-tron~c computer systems. and more particularly, to a method of main-taining the contiguity of files stored on ma~netic disl~s and positioning those files in a manner that reduces the average seek time for read and write accesses to the disk. Although this method was developed for mass storage peripherals connected to Digital E~uipment Corpora-tion VAX~ computers running the VMS~ operating system, the method is not limited in this respect and is broadly applicable to mass storage devices on other hardware platforms. VAX and VMS are registered trademarks of Digital Equlpment Corporation.
In the course of normal data processing operations, data, in the form of files, are created and deleted on magnetic disk media. Gene~
ally, the operating system of the computer attemp~s to create new files contiguously on the disk. However, as the disk fills and less free space ~s available, it becomes highly lLtsely that the contiguity of files cannot be maintained. When this happens, the operating system will search for the largest pieces of available free space and will allocate space in this sequence until the entire file is created. The net result is that files become fragmente~, i.e., in several "pieces~ on the disk.
Conversely, as files are deleted, it creates more ~holesl~ of free space.
As contiguous free space deteriorates, the problem of contiguous file creatlon is exacerbated.
File fragmentation is undesirable. Under the VMS opera~ing system, each disk contains an index file designated INDEXF.SYS that : - ' ,, ~ '''~.
~ .
~i.. . . ... . . . .
.. . .
, . ~ , ~ . .
. ., . , :
,. . i , .
.
, ~22242 - 2 -~
specifies the location of all files on the disk~ When a file i~s to be accessed, an Initial access is made to the appropriate blocl~ (512 cha~
acter aggregate of disk space) in the index file. This block tel~s the operating system the address of the target file. If the target file is fragmented, the heads on the disk will have tO make multiple accesses (I/O~s) to read the entire file. In cases of extrem~ fragmentation, another access must be made to the index file to get the addresses for additional pieces of the file. This is called a "windsw turn~ and requires significant CPU resource~s as well as multiple VO~s. As I/O~s increase and window turns utilize CPU cycles, overall performance on the system is degraded, resulting in I/O bottlenee~cs, reduced pro~uc-tivity, and a deteriora~ion of terminal response time for in~eractive users.
VMS system~s offer one solution to the file fragmentation pro~
lem. This solution entails doing an indexed backup to tape of the entire disk, re-initializing the disk, and r~storing the disk from th~
tape. This will be referred to below as the backup model. As Figure 6 illu~strates, all of the free space lands at the back (high logical block number (LBN)3 end of the disk. As files are created and deleted, the disk quickly degenerates to its former state. The defragmentation of the disk using tape backups is labo~intensive and time ccnsuming, talcing up to four hours for a 456Mb disk.
Niany commercial products attempt to address and overcome the problem of fil~ fragmentation. One such product is PC Too~s Deluxe ~ by Central Point Sof tware, Inc. The sof tware package includes a feature which arranges the files on a hard disk or diskette such that eaeh f~e ~ contained in one contiguous area. Another fea-ture defragm~nt~ f~es and moves free space to the back of the dlsk.
The software also permit~ files to be arranged in a predetermined manner on the hard dislc. For example~ all fil~ for a given subK~rectory may be kept together to keep data and program overlay files adjacent one another. This reduces the amoun~ of dlsk head movemen~ needed. A dirçctory sort feature permits the files within directories to ~ sorted by file name, fil~ time, file extension, or fil~
siæO InIormation regardilag these features may be fo~d in PC Too~
.
~3~22~2 Deluxe, Version 5, December 1988. However, this and similar prod-ucts arrange free space at the back of the disk. As des~ribed abo~re, as files are created and deleted, contiguous free space deteriorates and the îragmentation problems return.
The above ~echniques may be illustrated as in Figure 1.
Figure 1 illustrates a disk having a plurality of files (designated H in ~he Figure) and free space (dasignated F) arranged as shown. If a new file is created. it may be necessary to write the file to two or morP
non-contiguous blocks of fre~ us~r space, re~ulting in file fragmenta tion. File fragmen~s FRl, FR2, and FR3 represent a file written to three non-cantiguous blocl~s of free space. The heads on the disk w~ll ha~re to access each of the fragments FRl~ FR2, and FR3, resulting in degraded system performance. The above-described systems for defragmenting files will result in disk arranged as illustrated in Figure 2. However. as files are deleted and written to the free space, fragmentation again occurs and sys~em perfor~ance is degraded.
SUMMAR~ OF THB INVEN~ION
Accordingly, it iu an object of an a~pect of the pre3ent invention to provide an online method and apparatu~ for eliminating f~agm2ntation without nsed for performing backup and re~tore via tape.
It i~ an object of an a~pect of the preaent invention to recognize different file types on the di~k and group them intelligently in a manner that will minimi~e the average ~e~k time to compl~te an I/O.
These objects are achieved in accordance with the present invention by providing a file movement algorithm that can recognize a file by its usage characteristics and position it on the disk in a con-tiguous state without ths need for operator intervention.
The maJor benefits of the inven~ion are (1) operators need no longer be present to mount backup and restnre tapes related to disk d~fragmentation: (2) average seek time is reduced due to intelligent file pla¢ement: (3) productivity increases due to elimination of I/O
bottlenecks resulting from multiple see~.
In accordance with the pres2nt invention, a method of inteLlli-gently organizin~ a magnetic storage medium is provided. The mag-netic storage medium has free space and a plurality of files .
' , 2~2 q distributed thereon. The plurality of ~iles includes volatile files cr~
ated within a first predetermined period of time. First, the ~ree space is su~stantially consolidaced on the magnetiC S~Grage medium.
Then, the volatile files are su~stantially positioned contiguous tO the con~olidated free space.
Another a~pect of this invention i9 as follow~:
A method of organizing a magnetic storage medium so ~c to reduce se~k time for reducing read and write acceSSes to said sto~
age medium. sa~d magnetlc storage medium having free space and a plurality of files distr~buted ~hereon. said plurality of files including volatile files created Withill a flrsr predetermined period of tlme and warehou~se files last ac~essed within a second predetermined period of time, th~ method includin~ the steps of:
su~stantial~y moving the warehouse files ~o one end of the magnetic storage medium;
substan~ially consolidating the ~re~ space distr~buted on sai~ magnetic storage medium su~h that the free space is substar~tially centered in the space not occupied by Ihe warQhouse files; and substantially pcsicioning the volatil~ files contiguous to the corwlidated (ree space~
BRE~ DESC~IPlION OF THE DRAWINGS
A more complet~ appreciation of the present invention and many of the attendant adYantages thereof will be readily obeained as the invention becomes better understood through the f ollowing detailed descr~ption with reference to the accompany~ng drawings.
Figure 1 illustrates fil~ and free space distr~buted on a disk.
Flgure 2 illustrares ~he effec~ of prior art techniques on the disk of Figure 1.
Figure 3 illustrate~ the dist~ibution of the files of Table II on a d s~.
Figure 4 is a flow chart ill~straling the method of the present invention.
Figure 5 illustrates the redu~ed head movement in accordance with Ch~ file distributiorl of the presen~ invention.
, ;
3~242 - 4a -Flgur~ 6 illustral:es head movement in accordance w~th prior art file distr1bution.
Figure ? illustra~es a file type map genera~ecl during a disk scan in accordance with the present invention.
Figure 8 illustrates the flles of ~igure 3 arranged in a-~cordance with the pl~esent invention.
DIESCRIPll[QN OF THE PREFE~aRED E~BOD~qENT
A simple example will serYe to demons~rate the operation of the present invention. It should be recognized that the example below is illllstrative only and the invention is not limited in this resp~t. The important asp~ct of the present invention is the differ-entiation of file types on the disk.
Assume that a diSk includes 200 disk bloeks and that during a initia~zatlon scan descr~ in greater de~ail below, the disk is fou to c~ntain files as follows:
~ ~ ~r :~ ,s i ~3~22~2 - s -TABLE I
Warehouse (W) - 50 disk blocks Index (I) - 10 disk blocks Free (F) - 50 disk blocks Volatile (V) - 4û disk blocks Mrectory(D) - 10 disk blocks Ordinary(O) - 40 disk blocks In accordance with the present invention, warehouse files, i.e., those files which have not been accessed for a first predetermined number of days, are migrated or moved to the back portion (LBN high) of the disk. Thus, in this example, disk blocks 151-200 would hold warehouse files. Typically, warehouse files are the least 3ikely files to b~ accessed. and thus disk blocks 1-150 constitute what will be referred to below and is illustrated in the figures as the "actively accessed region". The present invention attempts to substantially center the ~ree space within this actively accessed re~on. Thus, to the extent possible, the free space will be positioned at disk blocks 51-lO0 in the above example~
While in a preferred embodiment, warehouse files are migrated to the back of the disk, the invention is not limited in this respe~t.
Warehouse files may also be migrated to the front portion (LBN low) of the disk and the placemen~ of the files would be inverted. Thus, utiliz-ing th~ abo~te example, if warehouse files were migrated to the front portion of the disk, blocks l-S0 would hold warehouse files and disk blocks 51-200 would constitute the actively accessed region.
An important feature of the present inven~ion is that ~he vola-tile files are positioned contiguous to the free space. Volatile files are statistically the most likely files to be accessed and deletedl and since new file creations will occur in free spa-~e, the read/write head mov~
ment may be further localized or focused by concentrating its activity in one area of or portion the disk. This region is denoted ~'focusedll in Figure 5.
The present invention will be explained in de~ail with refer~n~e to the sample file directory of Table II~
' .
, ~ 3~2~
Ta~le II
Name Size Date Cre~ted FLle l 15 Feb. 23 - 12:1~
~lle 2 23 Dec . 11 - 16: 42 File 3 7 Apr. 05 - 1:10 File 4 9 Dec. 13 - 20:32 Plle 5 11 Mar. 14 - 22:17 File 6 17 Dec. 08 - 10:56 File 7 17 May 27 - 23 :01 8 8 Dec. 13 00:39 Fll~ 9 8 D~c . 09 - 14: lg ~ile 10 15 Dec . 09 - L5 :10 Index 10 Direclory 10 The files of Table Il, some fragmented, some defragmented, are shown as they may be distributed on a disk in Figure 3. It can be seen that File 2 and File 7 are fragmented and that the free space is not contiguous.
The index file is created by the VMS opera~ing system and is typically lo~ated in the approximate center of the disk. The location of the index file is maintained in memory. Generally, all operating sys~ems include index file~ which may usually be positioned on the disk as desired by the system manager. The present invention does not move the index file from its location on the disk. Thus, in attempting to center the free space in the actively accessed region, the present invention must "work around" the index file. Cer~ain other files are no~ moYed from their disk locations by the present invention. Files which are open when the invention ic active are not moved. Other examples of immovabie file~ are so-called hidden files associated ~nth copy-protected files and system files. The bit map is also immovable.
In general, files whi~h would be adv~rsely affe~ted by movement or files whose movement would afIect system operation are no~ mov~.
Mrectory files contain the directory and subdirectory tree structure for ownership of all fil~i. As noted above, warehouse files ar~ those which have not been ac~essed for a first predetermined num~
ber OI days~ The last time a file was accessed may be determined in ac~rdance with a Vh1S facility known as volume retention dates. 3y ~.
1322~2 enabling this volume retention date function on a disk device, it is pos-sible to determine how much time has elapsed since the last time a given file was accessed. Other operating systems have similar func-tions generally known as date of last access functions. Although this f irst predetermined number is generally chosen in ac~ordance with system requirements related to disk activity, in a preferred embodi-ment, the predetermined number of days is chosen tO be about eight days, although the invention is not limited in this respect. Thus, if the present invention were implemented on December 13 at 23:00 (11:00 p.m.), File~ 1, 3, 5 and 7 would be determined to be warehouse files.
Volatile files are files created within a second predetermined time period prior to the pre5ent time. Volatile files can be recognized by examining the create date in the file header block. Again, like ~he first predetermined time period. this second time period will be de~e~
mined in accordance with system requirements. In a preferred embodi-ment, the second predetermined timë period is chosen to be about 72 hours, although the invention is not limited in this respect. Thus9 if the present invention were implemented on December 13 at 23:00, Files 2, 4, and 8 would be determined to be volatile files. Statistically, it is these volatile files which are most likely to be accessed at any given time. Finally, ordinary files are files which are accessed on a regular basis and do not meet any of the above criteria. Free space is disk blocks which are unused.
When the present invention is initiated, it scans the disk device to be defragmented and collects data on file size, type, number of frag-ments, and free space distribution. In the preferred embediment, the disk scan moves from LBN 0 to LBN end. I~ the warehouse files are to be stored at the front end of the disk, the dlslc scan would preferably move from LBN end to LBN 0. During the scan, the data collected is used to determine how much disk space will be necessary for each above~escribed file catego~. By determining how much disk space is requ~red for each file categ~ry, a manner of intelligently dlstributing the files such that the free space will be located in the approximate center of the actively accessed por~ion is developed.
..
' -..~
~ 13222~2 The method of the present invention will be explained with ref-erence to Figure 4. In the preferred embodiment, the meth~ for mak-ing files conligUous LS consistent with normal file movement under VMS. In general, the method of making files contiguous should be con-sistent with ~he normal file movement operation under the operating system being utilized. As indicated at step 10, the invention begins by scanning the disk and identifying the file types pres~nt thereon. The available free space ~s also mapped. By determining the file types and sizes, the file placement of the subsequent steps may be planned. In step 20, disk analysis reports are printed so that the system manager, for example, may obtain a picture of the disk file distribution. Typical information which may be printed includes the total disk space~ ths total d~l~ space used, the total free space, the number of each of the various file types: and the number of fragmented files. This printed analysis is not a critical feature of the present invention, but serves to give an overall picture o~ disk file Structure. The printed informat~on may be in numeric form or d~splayed graphieally.
At step 309 the invention begins file movement. Starting at the beginning of the disk (LBN O), the first file is copied to a scratch area.
The scratch area is free space on the target disk or on another device.
Preferably, the scratch area is free space on another disk. After the file has been ~opied, a pointer flag is set at step 40. This pointer flag serves to point to the copied file and p~events an interruption such as a power failure from resulting in a file which is later unable to ~e located. At step 50, the space the copied file occupied is truncated and control passes to step 60 in which the invention searches for a file that will both fi~ into this space contiguously and in accordance with the file type map generated during the disk scan. Truncate as used herein refers to the deallocation of space such that the operating system re~
ognizes that the space is unallocated. Contigl~ity as used herein refers to the positioning of a file physically on a disk in sequential LBN order.
Both the scratch space and the disk are searched for files to it in the space. In the preferred embodiment, the largest file in accordance with the file type map is moved into the space. This is g0nerally to prevent . . .
~ ...
~3222~2 the possibllity of running out of disk space as the back of the disk is reached~
The file type map utiliæs the information gathered during the disk scan regarding file si2e and type to determine how the files sho~d be distributed on the disk. Again, a~ described above, the invention attempts to center free space in the actively acce~ssed reg~on and p~si-tion volatile files contiguous therewith. Ordinary file~s, directory files, and the index file are positioned on either side of the volatile files.
This will bs illustrated in more detail below.
If a file is found that fits into the truncated space contiguously, it is copied thereto at step 70. Then, at step 80, the pointer flag is reset and the index file is appropriately updated with the new îile loca-tion. If no fit is found, the next file is copied out and the space is again truncated. This truncation creates a larger hole and the invention again searches for a file to fit within this new larger space.
This procedure continues until the last file has been mo~ed, at which time control passes to step 90. At step 90, disk analysis reports are printed. These r~ports are similar to those printed at step 20 and provide an indication to the system manager o~ ~he current disk status.
An important feature of the file movement is that it attempts to consolidate all the free space on the disk. (Free space may not, how-ever, be totally consolidated due ~o randomly distributed immovable files.) If, for example, free space is 2n% of the disk, the invention will attempt to center the free space in the active 80~6 area. Volatile files are generally placed on either side of the fres space. The distribution of immovable files may, for example, limit the placement of vola~ile files to only one side of the free space. If volatile files are statistically most likely to be accessed or deleted soon and new file creations will occur in free space, the read/write head movemerlt can be localized by concentrating its activity in one area of the disk.
The present invention will now be explained with reference to the files of Tables I and II shown distributed on a disk in Figure 3.
Based on the results of an initialization on December 13 at 23:009 the disk scan would determine that warehouse f~ occupy 50 d~sk blocks and that there are 50 disk bloclcs of free space. This means that there ~ ~3~2~2 are 150 disk blocks in the actively aecessed region. Since there are 50 blocks of free space, the file type map l,vill attempt to substantially .
center the free space within this actively accessed region or at blocks 51-100. In addition, there are 10 disk blocks of index files. While these block positions may not be adhered to faithfully in a given initializa-tion, they serve as guidelines for positioning the files.
A disk scan initiated at 23:00 on December 13 would determine that files 1, 3, 5, and 7 sa~isfy the criteria of warehouse files and that the to~al number nf disk blocks occupied by these files is 50 or ~ (25%) of the disk. Since these files are to be moved to the back of the disk the îile type map will position warehouse files, to the extent possible, at disk blocks 151-200 or from 75 to 100% of the disk. Thus, the actively accessed re~ion would constitute disk blocks 1-150 or from 0-75% of the dis~. ;
Th~ disk scan would further determine that there are 50 disk blocks of free space, or ~ (25%) of the disk. The file type map wiLI su~
stantially position the free space, to the extent possible, in the cen~er of the actively acce~sed region or at disk bloccs 51-100 (25 to 50% of the ~sk), However, as described below, the scan will a~so find that the immovable index file is positioned at blocl~s 96-105 and the file type map will note that no free space or other file types may be allocated to these disk blocks. ~Substantially~' is used hereirl in recognition tha~ due to immova~le files, for example, it may not be po~sible to strictly adhere to the file type map as described below.
The disk scan would also determine that files 2, 4, and 8 are vol-a~ile files which must be positioned contiguous with the free space.
The total number of disk blocks occupied by volatile files is 40 or 20%
of the dislc. Since, as suggested by the overlap noted above, the free space will be contiguous with the index file at one end, the file type map will attempt to position the volatile files at the other end of the free space or at disk blocks 11-50.
Th~ disk scan would also determine that files 6, 9 and 10 are ordiQary files arld that the total number of disk blocks occupied by these files is 40 or 1/5 (20%) of the disk. The directory file would be determined to occupy 10 disk blocks and in the preferred embodliment, ~: " ' ' ' ,.
. ~
'' ' ' ~322~2 ~ 11 would be placed contiguous with the index file or at disk blocks 106-115. The file Iype map would position the ordinary files in the remaining disk blocks. The file type map generated by the abov~
described disk scan is illustrated in Figure 7. The index file ~ is cross-hatched to indicate ~hat it is immovable. ln positioning the files utiliz-ing the algorithm described in Fi~ure 4, the present inven~ion attempts tO adhere to this file ~ype map tO the extent possible.
With reference tO Figure 3, file 4 is the first file moved to scratch space and the space file 4 occupied is truncated, thereby open-ing 9 disk blocks at the front of the dislc. In accordance with the alg~
rithm, the invention will search the disk and scratch space for a file which fits in the open space and i5 consistent to the extent pos~ible with the file type map. File 9 5 an ordinary file consisting of 8 disk blocks which fitc into the truncated space. In accordance wi~h the invention, f ile 9 would then be moved to this space as shown in Figure 8.
Next, the space which previo~!~ly held file 9 is truncated le vmg an open space of 21 blocks (8 blocks of newly truncated, 1 block remaining from previouc truncated space and 12 free ~locks). The pre sent invention will now sear~h for a volatile file or a very small ordi-nary file to occupy this space. Since the smallest remaining ordinary file ic 15 disk block(s. placement of an ordinary file in the open space would be inconsLctent with the file type map. File 4 is a vola~ile file o~
9 disk blocks and file 8 3s a volatile file of 8 disk blocks. Both satis~y th~ file type map ar~d koth would fit in the open space. However, as noted above, in the preferred embodimen~, file 4 ~ould be moved since it is the larger file. The pa6itioning of file 4 3s shown in Figure 8.
Nex~, file ~ is de~ragmented and written contiguously to scra~ch space. The first fragment ~left in Figure 3) of f~le 7 plus the remaining open space is truncated to create an opening of 22 b~cks. File 8 is a volatile file which is consisterlt w~th the file type map and would fit in ~he open space. Thus~ file 8 would be positioned æ shown in Figure 8.
Next, file 1 is wri~ten to scratch space. The open space is trlm-cated and an opening of 29 disk blo~ks is ~reated. (Slze of file 1 and 14 blocks from previou~ truncated spa~e). File 2 is a volatile file .
:~32~2~2 consistent wi~h the file type map and which fits in the opening. Thus, file 2 would be made contiguous and would be positioned as shown in Figure 8.
Next, the the invention will attempt to coosolidate free space since no more volatile files remain. The invention would attempt to write sufficient files tO scratch space to corlsolidate the free spaeP9 in this example. 50 disk blocks. Thus, in the present example~ since the last block of volatile files is at block 48, the invention would attempt to free blocks 49-98. However, the immovable index file occupie~
blocks 96-105. Thus, blocks 96-98 cannot be freed and non-contiguous free space will be generated. However, it is apparent that9 for the most part, free space has been made contiguous and substantially posi-tioned in accordance with the file type map. This is shown in Figure 8.
Next, file 3 is written to scratch space. This space is truncated and creates an open space of 13 blocl~ blocks of file 3 and 6 blocks of free space). As noted, the directory file is su~stantially positioned contiguous with the index file in a preferred embodiment and thus file D is moved as shown in Figure ~.
The remaining files are moved and positioned in accordanee with the procedures outlined and will not be discussed in detail here. The end result of the procedure yields a dislc with the files as distributed in Figure 8. The non-contiguous free space may be positioned contiguous to the warehouse files as shown but this is not critical.
In summary, a file type map is generated during an initial disk scan. Files are distributed on the disl~ to the extent pa~sible, in acco~
dance with this file type map. Implementation o~ the above~escribed routine ~s straightiorward and will be apparent to those skille~ in the art.
Figure 5 illustrates how the present invention focuses head movement. In Figure 5B volatile files have been deleted and then recreated, the net rEsult being that the number of free space holes increases from 1 to 3. When alternate volatile files are again deleted and recreated as shown in Figure SC, it shows that ~ree spaee hole~
have actually reduced from 3 to 2. The3e figures, taken from actual runs demonstrate that (1) by focusing head movemen~ average head . ~, .
P ~3~2~2 see~ time can be reduced; and t2) despite user activity ~he actively accessed portion of the disk remains constant.
Conversely, the backup model (Figure 6) shows that failure to differentiate between files types and failure to intelligently positior files leads to (1) more rapid de~radations of the disk structure, and (2) rapid fragmentation of free space; and (33 the actively accessed portion of ~he disk increases over time.
In Figure 6B every other volatile f ile has been deleted and recreated. The result is the number of f ree space "holes~l have iwreased and the actively accessed portion of the disk has increased, In Figure 6C another pass of deletions and recreations aga~n causes more fragmented free space and more dislc area to be covered by the read/write heads. Eventually, the large piece of contiguous free space is consumed and the next file to be created will be fragmented since it must use the smaller pieces of free space.
The invention has been described in detail in connection rrith the preferred embodiments. These embodiments, however, are merely for example only, and the invention is not limited thereto. It will be easily understood by those skilled in the art that other variations and modifications may easily be made within the scope of this invention as defined by the ap~ended claims.
AND REDUCING AVERAGE SEEK TIMES IN A
;UAGNETIC DISK MEDIA ENVIRONMENT
BACK~;ROUND 3F THE INVEP~TION
The present invention generally relates to the field of elec-tron~c computer systems. and more particularly, to a method of main-taining the contiguity of files stored on ma~netic disl~s and positioning those files in a manner that reduces the average seek time for read and write accesses to the disk. Although this method was developed for mass storage peripherals connected to Digital E~uipment Corpora-tion VAX~ computers running the VMS~ operating system, the method is not limited in this respect and is broadly applicable to mass storage devices on other hardware platforms. VAX and VMS are registered trademarks of Digital Equlpment Corporation.
In the course of normal data processing operations, data, in the form of files, are created and deleted on magnetic disk media. Gene~
ally, the operating system of the computer attemp~s to create new files contiguously on the disk. However, as the disk fills and less free space ~s available, it becomes highly lLtsely that the contiguity of files cannot be maintained. When this happens, the operating system will search for the largest pieces of available free space and will allocate space in this sequence until the entire file is created. The net result is that files become fragmente~, i.e., in several "pieces~ on the disk.
Conversely, as files are deleted, it creates more ~holesl~ of free space.
As contiguous free space deteriorates, the problem of contiguous file creatlon is exacerbated.
File fragmentation is undesirable. Under the VMS opera~ing system, each disk contains an index file designated INDEXF.SYS that : - ' ,, ~ '''~.
~ .
~i.. . . ... . . . .
.. . .
, . ~ , ~ . .
. ., . , :
,. . i , .
.
, ~22242 - 2 -~
specifies the location of all files on the disk~ When a file i~s to be accessed, an Initial access is made to the appropriate blocl~ (512 cha~
acter aggregate of disk space) in the index file. This block tel~s the operating system the address of the target file. If the target file is fragmented, the heads on the disk will have tO make multiple accesses (I/O~s) to read the entire file. In cases of extrem~ fragmentation, another access must be made to the index file to get the addresses for additional pieces of the file. This is called a "windsw turn~ and requires significant CPU resource~s as well as multiple VO~s. As I/O~s increase and window turns utilize CPU cycles, overall performance on the system is degraded, resulting in I/O bottlenee~cs, reduced pro~uc-tivity, and a deteriora~ion of terminal response time for in~eractive users.
VMS system~s offer one solution to the file fragmentation pro~
lem. This solution entails doing an indexed backup to tape of the entire disk, re-initializing the disk, and r~storing the disk from th~
tape. This will be referred to below as the backup model. As Figure 6 illu~strates, all of the free space lands at the back (high logical block number (LBN)3 end of the disk. As files are created and deleted, the disk quickly degenerates to its former state. The defragmentation of the disk using tape backups is labo~intensive and time ccnsuming, talcing up to four hours for a 456Mb disk.
Niany commercial products attempt to address and overcome the problem of fil~ fragmentation. One such product is PC Too~s Deluxe ~ by Central Point Sof tware, Inc. The sof tware package includes a feature which arranges the files on a hard disk or diskette such that eaeh f~e ~ contained in one contiguous area. Another fea-ture defragm~nt~ f~es and moves free space to the back of the dlsk.
The software also permit~ files to be arranged in a predetermined manner on the hard dislc. For example~ all fil~ for a given subK~rectory may be kept together to keep data and program overlay files adjacent one another. This reduces the amoun~ of dlsk head movemen~ needed. A dirçctory sort feature permits the files within directories to ~ sorted by file name, fil~ time, file extension, or fil~
siæO InIormation regardilag these features may be fo~d in PC Too~
.
~3~22~2 Deluxe, Version 5, December 1988. However, this and similar prod-ucts arrange free space at the back of the disk. As des~ribed abo~re, as files are created and deleted, contiguous free space deteriorates and the îragmentation problems return.
The above ~echniques may be illustrated as in Figure 1.
Figure 1 illustrates a disk having a plurality of files (designated H in ~he Figure) and free space (dasignated F) arranged as shown. If a new file is created. it may be necessary to write the file to two or morP
non-contiguous blocks of fre~ us~r space, re~ulting in file fragmenta tion. File fragmen~s FRl, FR2, and FR3 represent a file written to three non-cantiguous blocl~s of free space. The heads on the disk w~ll ha~re to access each of the fragments FRl~ FR2, and FR3, resulting in degraded system performance. The above-described systems for defragmenting files will result in disk arranged as illustrated in Figure 2. However. as files are deleted and written to the free space, fragmentation again occurs and sys~em perfor~ance is degraded.
SUMMAR~ OF THB INVEN~ION
Accordingly, it iu an object of an a~pect of the pre3ent invention to provide an online method and apparatu~ for eliminating f~agm2ntation without nsed for performing backup and re~tore via tape.
It i~ an object of an a~pect of the preaent invention to recognize different file types on the di~k and group them intelligently in a manner that will minimi~e the average ~e~k time to compl~te an I/O.
These objects are achieved in accordance with the present invention by providing a file movement algorithm that can recognize a file by its usage characteristics and position it on the disk in a con-tiguous state without ths need for operator intervention.
The maJor benefits of the inven~ion are (1) operators need no longer be present to mount backup and restnre tapes related to disk d~fragmentation: (2) average seek time is reduced due to intelligent file pla¢ement: (3) productivity increases due to elimination of I/O
bottlenecks resulting from multiple see~.
In accordance with the pres2nt invention, a method of inteLlli-gently organizin~ a magnetic storage medium is provided. The mag-netic storage medium has free space and a plurality of files .
' , 2~2 q distributed thereon. The plurality of ~iles includes volatile files cr~
ated within a first predetermined period of time. First, the ~ree space is su~stantially consolidaced on the magnetiC S~Grage medium.
Then, the volatile files are su~stantially positioned contiguous tO the con~olidated free space.
Another a~pect of this invention i9 as follow~:
A method of organizing a magnetic storage medium so ~c to reduce se~k time for reducing read and write acceSSes to said sto~
age medium. sa~d magnetlc storage medium having free space and a plurality of files distr~buted ~hereon. said plurality of files including volatile files created Withill a flrsr predetermined period of tlme and warehou~se files last ac~essed within a second predetermined period of time, th~ method includin~ the steps of:
su~stantial~y moving the warehouse files ~o one end of the magnetic storage medium;
substan~ially consolidating the ~re~ space distr~buted on sai~ magnetic storage medium su~h that the free space is substar~tially centered in the space not occupied by Ihe warQhouse files; and substantially pcsicioning the volatil~ files contiguous to the corwlidated (ree space~
BRE~ DESC~IPlION OF THE DRAWINGS
A more complet~ appreciation of the present invention and many of the attendant adYantages thereof will be readily obeained as the invention becomes better understood through the f ollowing detailed descr~ption with reference to the accompany~ng drawings.
Figure 1 illustrates fil~ and free space distr~buted on a disk.
Flgure 2 illustrares ~he effec~ of prior art techniques on the disk of Figure 1.
Figure 3 illustrate~ the dist~ibution of the files of Table II on a d s~.
Figure 4 is a flow chart ill~straling the method of the present invention.
Figure 5 illustrates the redu~ed head movement in accordance with Ch~ file distributiorl of the presen~ invention.
, ;
3~242 - 4a -Flgur~ 6 illustral:es head movement in accordance w~th prior art file distr1bution.
Figure ? illustra~es a file type map genera~ecl during a disk scan in accordance with the present invention.
Figure 8 illustrates the flles of ~igure 3 arranged in a-~cordance with the pl~esent invention.
DIESCRIPll[QN OF THE PREFE~aRED E~BOD~qENT
A simple example will serYe to demons~rate the operation of the present invention. It should be recognized that the example below is illllstrative only and the invention is not limited in this resp~t. The important asp~ct of the present invention is the differ-entiation of file types on the disk.
Assume that a diSk includes 200 disk bloeks and that during a initia~zatlon scan descr~ in greater de~ail below, the disk is fou to c~ntain files as follows:
~ ~ ~r :~ ,s i ~3~22~2 - s -TABLE I
Warehouse (W) - 50 disk blocks Index (I) - 10 disk blocks Free (F) - 50 disk blocks Volatile (V) - 4û disk blocks Mrectory(D) - 10 disk blocks Ordinary(O) - 40 disk blocks In accordance with the present invention, warehouse files, i.e., those files which have not been accessed for a first predetermined number of days, are migrated or moved to the back portion (LBN high) of the disk. Thus, in this example, disk blocks 151-200 would hold warehouse files. Typically, warehouse files are the least 3ikely files to b~ accessed. and thus disk blocks 1-150 constitute what will be referred to below and is illustrated in the figures as the "actively accessed region". The present invention attempts to substantially center the ~ree space within this actively accessed re~on. Thus, to the extent possible, the free space will be positioned at disk blocks 51-lO0 in the above example~
While in a preferred embodiment, warehouse files are migrated to the back of the disk, the invention is not limited in this respe~t.
Warehouse files may also be migrated to the front portion (LBN low) of the disk and the placemen~ of the files would be inverted. Thus, utiliz-ing th~ abo~te example, if warehouse files were migrated to the front portion of the disk, blocks l-S0 would hold warehouse files and disk blocks 51-200 would constitute the actively accessed region.
An important feature of the present inven~ion is that ~he vola-tile files are positioned contiguous to the free space. Volatile files are statistically the most likely files to be accessed and deletedl and since new file creations will occur in free spa-~e, the read/write head mov~
ment may be further localized or focused by concentrating its activity in one area of or portion the disk. This region is denoted ~'focusedll in Figure 5.
The present invention will be explained in de~ail with refer~n~e to the sample file directory of Table II~
' .
, ~ 3~2~
Ta~le II
Name Size Date Cre~ted FLle l 15 Feb. 23 - 12:1~
~lle 2 23 Dec . 11 - 16: 42 File 3 7 Apr. 05 - 1:10 File 4 9 Dec. 13 - 20:32 Plle 5 11 Mar. 14 - 22:17 File 6 17 Dec. 08 - 10:56 File 7 17 May 27 - 23 :01 8 8 Dec. 13 00:39 Fll~ 9 8 D~c . 09 - 14: lg ~ile 10 15 Dec . 09 - L5 :10 Index 10 Direclory 10 The files of Table Il, some fragmented, some defragmented, are shown as they may be distributed on a disk in Figure 3. It can be seen that File 2 and File 7 are fragmented and that the free space is not contiguous.
The index file is created by the VMS opera~ing system and is typically lo~ated in the approximate center of the disk. The location of the index file is maintained in memory. Generally, all operating sys~ems include index file~ which may usually be positioned on the disk as desired by the system manager. The present invention does not move the index file from its location on the disk. Thus, in attempting to center the free space in the actively accessed region, the present invention must "work around" the index file. Cer~ain other files are no~ moYed from their disk locations by the present invention. Files which are open when the invention ic active are not moved. Other examples of immovabie file~ are so-called hidden files associated ~nth copy-protected files and system files. The bit map is also immovable.
In general, files whi~h would be adv~rsely affe~ted by movement or files whose movement would afIect system operation are no~ mov~.
Mrectory files contain the directory and subdirectory tree structure for ownership of all fil~i. As noted above, warehouse files ar~ those which have not been ac~essed for a first predetermined num~
ber OI days~ The last time a file was accessed may be determined in ac~rdance with a Vh1S facility known as volume retention dates. 3y ~.
1322~2 enabling this volume retention date function on a disk device, it is pos-sible to determine how much time has elapsed since the last time a given file was accessed. Other operating systems have similar func-tions generally known as date of last access functions. Although this f irst predetermined number is generally chosen in ac~ordance with system requirements related to disk activity, in a preferred embodi-ment, the predetermined number of days is chosen tO be about eight days, although the invention is not limited in this respect. Thus, if the present invention were implemented on December 13 at 23:00 (11:00 p.m.), File~ 1, 3, 5 and 7 would be determined to be warehouse files.
Volatile files are files created within a second predetermined time period prior to the pre5ent time. Volatile files can be recognized by examining the create date in the file header block. Again, like ~he first predetermined time period. this second time period will be de~e~
mined in accordance with system requirements. In a preferred embodi-ment, the second predetermined timë period is chosen to be about 72 hours, although the invention is not limited in this respect. Thus9 if the present invention were implemented on December 13 at 23:00, Files 2, 4, and 8 would be determined to be volatile files. Statistically, it is these volatile files which are most likely to be accessed at any given time. Finally, ordinary files are files which are accessed on a regular basis and do not meet any of the above criteria. Free space is disk blocks which are unused.
When the present invention is initiated, it scans the disk device to be defragmented and collects data on file size, type, number of frag-ments, and free space distribution. In the preferred embediment, the disk scan moves from LBN 0 to LBN end. I~ the warehouse files are to be stored at the front end of the disk, the dlslc scan would preferably move from LBN end to LBN 0. During the scan, the data collected is used to determine how much disk space will be necessary for each above~escribed file catego~. By determining how much disk space is requ~red for each file categ~ry, a manner of intelligently dlstributing the files such that the free space will be located in the approximate center of the actively accessed por~ion is developed.
..
' -..~
~ 13222~2 The method of the present invention will be explained with ref-erence to Figure 4. In the preferred embodiment, the meth~ for mak-ing files conligUous LS consistent with normal file movement under VMS. In general, the method of making files contiguous should be con-sistent with ~he normal file movement operation under the operating system being utilized. As indicated at step 10, the invention begins by scanning the disk and identifying the file types pres~nt thereon. The available free space ~s also mapped. By determining the file types and sizes, the file placement of the subsequent steps may be planned. In step 20, disk analysis reports are printed so that the system manager, for example, may obtain a picture of the disk file distribution. Typical information which may be printed includes the total disk space~ ths total d~l~ space used, the total free space, the number of each of the various file types: and the number of fragmented files. This printed analysis is not a critical feature of the present invention, but serves to give an overall picture o~ disk file Structure. The printed informat~on may be in numeric form or d~splayed graphieally.
At step 309 the invention begins file movement. Starting at the beginning of the disk (LBN O), the first file is copied to a scratch area.
The scratch area is free space on the target disk or on another device.
Preferably, the scratch area is free space on another disk. After the file has been ~opied, a pointer flag is set at step 40. This pointer flag serves to point to the copied file and p~events an interruption such as a power failure from resulting in a file which is later unable to ~e located. At step 50, the space the copied file occupied is truncated and control passes to step 60 in which the invention searches for a file that will both fi~ into this space contiguously and in accordance with the file type map generated during the disk scan. Truncate as used herein refers to the deallocation of space such that the operating system re~
ognizes that the space is unallocated. Contigl~ity as used herein refers to the positioning of a file physically on a disk in sequential LBN order.
Both the scratch space and the disk are searched for files to it in the space. In the preferred embodiment, the largest file in accordance with the file type map is moved into the space. This is g0nerally to prevent . . .
~ ...
~3222~2 the possibllity of running out of disk space as the back of the disk is reached~
The file type map utiliæs the information gathered during the disk scan regarding file si2e and type to determine how the files sho~d be distributed on the disk. Again, a~ described above, the invention attempts to center free space in the actively acce~ssed reg~on and p~si-tion volatile files contiguous therewith. Ordinary file~s, directory files, and the index file are positioned on either side of the volatile files.
This will bs illustrated in more detail below.
If a file is found that fits into the truncated space contiguously, it is copied thereto at step 70. Then, at step 80, the pointer flag is reset and the index file is appropriately updated with the new îile loca-tion. If no fit is found, the next file is copied out and the space is again truncated. This truncation creates a larger hole and the invention again searches for a file to fit within this new larger space.
This procedure continues until the last file has been mo~ed, at which time control passes to step 90. At step 90, disk analysis reports are printed. These r~ports are similar to those printed at step 20 and provide an indication to the system manager o~ ~he current disk status.
An important feature of the file movement is that it attempts to consolidate all the free space on the disk. (Free space may not, how-ever, be totally consolidated due ~o randomly distributed immovable files.) If, for example, free space is 2n% of the disk, the invention will attempt to center the free space in the active 80~6 area. Volatile files are generally placed on either side of the fres space. The distribution of immovable files may, for example, limit the placement of vola~ile files to only one side of the free space. If volatile files are statistically most likely to be accessed or deleted soon and new file creations will occur in free space, the read/write head movemerlt can be localized by concentrating its activity in one area of the disk.
The present invention will now be explained with reference to the files of Tables I and II shown distributed on a disk in Figure 3.
Based on the results of an initialization on December 13 at 23:009 the disk scan would determine that warehouse f~ occupy 50 d~sk blocks and that there are 50 disk bloclcs of free space. This means that there ~ ~3~2~2 are 150 disk blocks in the actively aecessed region. Since there are 50 blocks of free space, the file type map l,vill attempt to substantially .
center the free space within this actively accessed region or at blocks 51-100. In addition, there are 10 disk blocks of index files. While these block positions may not be adhered to faithfully in a given initializa-tion, they serve as guidelines for positioning the files.
A disk scan initiated at 23:00 on December 13 would determine that files 1, 3, 5, and 7 sa~isfy the criteria of warehouse files and that the to~al number nf disk blocks occupied by these files is 50 or ~ (25%) of the disk. Since these files are to be moved to the back of the disk the îile type map will position warehouse files, to the extent possible, at disk blocks 151-200 or from 75 to 100% of the disk. Thus, the actively accessed re~ion would constitute disk blocks 1-150 or from 0-75% of the dis~. ;
Th~ disk scan would further determine that there are 50 disk blocks of free space, or ~ (25%) of the disk. The file type map wiLI su~
stantially position the free space, to the extent possible, in the cen~er of the actively acce~sed region or at disk bloccs 51-100 (25 to 50% of the ~sk), However, as described below, the scan will a~so find that the immovable index file is positioned at blocl~s 96-105 and the file type map will note that no free space or other file types may be allocated to these disk blocks. ~Substantially~' is used hereirl in recognition tha~ due to immova~le files, for example, it may not be po~sible to strictly adhere to the file type map as described below.
The disk scan would also determine that files 2, 4, and 8 are vol-a~ile files which must be positioned contiguous with the free space.
The total number of disk blocks occupied by volatile files is 40 or 20%
of the dislc. Since, as suggested by the overlap noted above, the free space will be contiguous with the index file at one end, the file type map will attempt to position the volatile files at the other end of the free space or at disk blocks 11-50.
Th~ disk scan would also determine that files 6, 9 and 10 are ordiQary files arld that the total number of disk blocks occupied by these files is 40 or 1/5 (20%) of the disk. The directory file would be determined to occupy 10 disk blocks and in the preferred embodliment, ~: " ' ' ' ,.
. ~
'' ' ' ~322~2 ~ 11 would be placed contiguous with the index file or at disk blocks 106-115. The file Iype map would position the ordinary files in the remaining disk blocks. The file type map generated by the abov~
described disk scan is illustrated in Figure 7. The index file ~ is cross-hatched to indicate ~hat it is immovable. ln positioning the files utiliz-ing the algorithm described in Fi~ure 4, the present inven~ion attempts tO adhere to this file ~ype map tO the extent possible.
With reference tO Figure 3, file 4 is the first file moved to scratch space and the space file 4 occupied is truncated, thereby open-ing 9 disk blocks at the front of the dislc. In accordance with the alg~
rithm, the invention will search the disk and scratch space for a file which fits in the open space and i5 consistent to the extent pos~ible with the file type map. File 9 5 an ordinary file consisting of 8 disk blocks which fitc into the truncated space. In accordance wi~h the invention, f ile 9 would then be moved to this space as shown in Figure 8.
Next, the space which previo~!~ly held file 9 is truncated le vmg an open space of 21 blocks (8 blocks of newly truncated, 1 block remaining from previouc truncated space and 12 free ~locks). The pre sent invention will now sear~h for a volatile file or a very small ordi-nary file to occupy this space. Since the smallest remaining ordinary file ic 15 disk block(s. placement of an ordinary file in the open space would be inconsLctent with the file type map. File 4 is a vola~ile file o~
9 disk blocks and file 8 3s a volatile file of 8 disk blocks. Both satis~y th~ file type map ar~d koth would fit in the open space. However, as noted above, in the preferred embodimen~, file 4 ~ould be moved since it is the larger file. The pa6itioning of file 4 3s shown in Figure 8.
Nex~, file ~ is de~ragmented and written contiguously to scra~ch space. The first fragment ~left in Figure 3) of f~le 7 plus the remaining open space is truncated to create an opening of 22 b~cks. File 8 is a volatile file which is consisterlt w~th the file type map and would fit in ~he open space. Thus~ file 8 would be positioned æ shown in Figure 8.
Next, file 1 is wri~ten to scratch space. The open space is trlm-cated and an opening of 29 disk blo~ks is ~reated. (Slze of file 1 and 14 blocks from previou~ truncated spa~e). File 2 is a volatile file .
:~32~2~2 consistent wi~h the file type map and which fits in the opening. Thus, file 2 would be made contiguous and would be positioned as shown in Figure 8.
Next, the the invention will attempt to coosolidate free space since no more volatile files remain. The invention would attempt to write sufficient files tO scratch space to corlsolidate the free spaeP9 in this example. 50 disk blocks. Thus, in the present example~ since the last block of volatile files is at block 48, the invention would attempt to free blocks 49-98. However, the immovable index file occupie~
blocks 96-105. Thus, blocks 96-98 cannot be freed and non-contiguous free space will be generated. However, it is apparent that9 for the most part, free space has been made contiguous and substantially posi-tioned in accordance with the file type map. This is shown in Figure 8.
Next, file 3 is written to scratch space. This space is truncated and creates an open space of 13 blocl~ blocks of file 3 and 6 blocks of free space). As noted, the directory file is su~stantially positioned contiguous with the index file in a preferred embodiment and thus file D is moved as shown in Figure ~.
The remaining files are moved and positioned in accordanee with the procedures outlined and will not be discussed in detail here. The end result of the procedure yields a dislc with the files as distributed in Figure 8. The non-contiguous free space may be positioned contiguous to the warehouse files as shown but this is not critical.
In summary, a file type map is generated during an initial disk scan. Files are distributed on the disl~ to the extent pa~sible, in acco~
dance with this file type map. Implementation o~ the above~escribed routine ~s straightiorward and will be apparent to those skille~ in the art.
Figure 5 illustrates how the present invention focuses head movement. In Figure 5B volatile files have been deleted and then recreated, the net rEsult being that the number of free space holes increases from 1 to 3. When alternate volatile files are again deleted and recreated as shown in Figure SC, it shows that ~ree spaee hole~
have actually reduced from 3 to 2. The3e figures, taken from actual runs demonstrate that (1) by focusing head movemen~ average head . ~, .
P ~3~2~2 see~ time can be reduced; and t2) despite user activity ~he actively accessed portion of the disk remains constant.
Conversely, the backup model (Figure 6) shows that failure to differentiate between files types and failure to intelligently positior files leads to (1) more rapid de~radations of the disk structure, and (2) rapid fragmentation of free space; and (33 the actively accessed portion of ~he disk increases over time.
In Figure 6B every other volatile f ile has been deleted and recreated. The result is the number of f ree space "holes~l have iwreased and the actively accessed portion of the disk has increased, In Figure 6C another pass of deletions and recreations aga~n causes more fragmented free space and more dislc area to be covered by the read/write heads. Eventually, the large piece of contiguous free space is consumed and the next file to be created will be fragmented since it must use the smaller pieces of free space.
The invention has been described in detail in connection rrith the preferred embodiments. These embodiments, however, are merely for example only, and the invention is not limited thereto. It will be easily understood by those skilled in the art that other variations and modifications may easily be made within the scope of this invention as defined by the ap~ended claims.
Claims (9)
1. A method of organizing a magnetic storage medium, said magnetic storage medium having free space and a plurality of files distributed thereon. said plurality of files including volatile files cre-ated within a first predetermined period of time, the method including the steps of:
substantially consolidating the free space distributed on said magnetic storage medium; and substantially positioning the volatile files contiguous to the consolidated free space.
substantially consolidating the free space distributed on said magnetic storage medium; and substantially positioning the volatile files contiguous to the consolidated free space.
2. A method of organizing a magnetic storage medium so as to reduce seek time for reducing read and write accesses to said stor-age medium, said magnetic storage medium having free space and a plurality of files distributed thereon, said plurality of files including volatile files created within a first predetermined period of time and warehouse files last accessed within a second predetermined period of time, the method including the steps of:
substantially moving the warehouse files to one end of the magnetic storage medium;
substantially consolidating the free space distributed on said magnetic storage medium such that the free space is substantially centered in the space not occupied by the warehouse files; and substantially positioning the volatile files contiguous to the consolidated free space.
substantially moving the warehouse files to one end of the magnetic storage medium;
substantially consolidating the free space distributed on said magnetic storage medium such that the free space is substantially centered in the space not occupied by the warehouse files; and substantially positioning the volatile files contiguous to the consolidated free space.
3. The method according to claim 2 further comprising the step of:
defragmenting a volatile file prior to its positioning con-tiguous to the consolidated free space.
defragmenting a volatile file prior to its positioning con-tiguous to the consolidated free space.
4. The method according to claim 2 further comprising the step of:
defragmenting a warehouse file prior to its moving to the one end of the disk.
defragmenting a warehouse file prior to its moving to the one end of the disk.
5. The method according to claim 2 wherein the warehouse files are moved to the end of the magnetic storage medium having high logical block numbers.
6. The method according to Claim 2 wherein the warehouse files are moved to the end of the magnetic storage medium having low logical block numbers.
7. The method according to Claim 1 wherein said plurality of files further comprise warehouse files last accessed within a second predetermined period of time, the method further comprising the step of:
defragmenting a warehouse file prior to moving it to the one end of the disk.
defragmenting a warehouse file prior to moving it to the one end of the disk.
8. The method according to Claim 1 wherein said plurality of files further comprise warehouse files last accessed within a second predetermined period of time, the method wherein the warehouse files are moved to the end of the magnetic storage medium having high logical block numbers.
9. The method according to Claim 1 wherein said plurality of files further comprise warehouse files last accessed within a second predetermined period of time, the method wherein the warehouse files are moved to the end of the magnetic storage medium having low logical block numbers.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US359,316 | 1989-05-31 | ||
| US07359316 US5398142B1 (en) | 1989-05-31 | 1989-05-31 | Method for eliminating file fragmentation and reducing average seek times in a magnetic disk media environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA1322242C true CA1322242C (en) | 1993-09-14 |
Family
ID=23413300
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA000615272A Expired - Fee Related CA1322242C (en) | 1989-05-31 | 1989-09-29 | Method for eliminating file fragmentation and reducing average seek times in a magnetic disk media environment |
Country Status (3)
| Country | Link |
|---|---|
| US (2) | US5398142B1 (en) |
| AU (1) | AU627029B2 (en) |
| CA (1) | CA1322242C (en) |
Families Citing this family (52)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5398142B1 (en) * | 1989-05-31 | 1997-09-16 | Raxco Inc | Method for eliminating file fragmentation and reducing average seek times in a magnetic disk media environment |
| US5371885A (en) * | 1989-08-29 | 1994-12-06 | Microsoft Corporation | High performance file system |
| JPH04211844A (en) * | 1990-01-19 | 1992-08-03 | Texas Instr Inc <Ti> | Apparatus and method for deflagment of filing system |
| GB2251324B (en) * | 1990-12-31 | 1995-05-10 | Intel Corp | File structure for a non-volatile semiconductor memory |
| GB2251323B (en) * | 1990-12-31 | 1994-10-12 | Intel Corp | Disk emulation for a non-volatile semiconductor memory |
| US5390315A (en) * | 1992-06-15 | 1995-02-14 | International Business Machines Corporation | Allocation of uniform contiguous blocks of DASD storage by maintaining both a bit and a bit map record of available storage |
| JP3378594B2 (en) * | 1992-09-25 | 2003-02-17 | 富士通株式会社 | Processing unit that performs database relocation |
| US5623669A (en) * | 1993-07-21 | 1997-04-22 | International Business Machines Corporation | High speed online copy of partitioned data |
| KR0167587B1 (en) * | 1993-07-26 | 1999-03-20 | 모리시타 요이찌 | Apparatus for recording and reproducing digital data |
| JP3745398B2 (en) * | 1994-06-17 | 2006-02-15 | 富士通株式会社 | File disk block control method |
| JPH086843A (en) * | 1994-06-23 | 1996-01-12 | Fujitsu Ltd | Semiconductor storage device |
| US5765175A (en) * | 1994-08-26 | 1998-06-09 | Intel Corporation | System and method for removing deleted entries in file systems based on write-once or erase-slowly media |
| US5574907A (en) * | 1994-11-30 | 1996-11-12 | Microsoft Corporation | Two-pass defragmentation of compressed hard disk data with a single data rewrite |
| US5604902A (en) * | 1995-02-16 | 1997-02-18 | Hewlett-Packard Company | Hole plugging garbage collection for a data storage system |
| US5819290A (en) * | 1995-04-10 | 1998-10-06 | Sony Corporation | Data recording and management system and method for detecting data file division based on quantitative number of blocks |
| US5832526A (en) * | 1996-01-24 | 1998-11-03 | Symantec Corporation | Method and apparatus using slack area of file storage structures for file reconstruction |
| US5799324A (en) * | 1996-05-10 | 1998-08-25 | International Business Machines Corporation | System and method for management of persistent data in a log-structured disk array |
| US6185575B1 (en) | 1996-09-19 | 2001-02-06 | Powerquest Corporation | In-place disk partition canonization and storage optimization |
| JPH10162560A (en) * | 1996-12-04 | 1998-06-19 | Matsushita Electric Ind Co Ltd | Video editing method and non-linear video editing apparatus |
| US5930828A (en) * | 1997-03-26 | 1999-07-27 | Executive Software International | Real-time apparatus and method for minimizing disk fragmentation in a computer system |
| US5956745A (en) * | 1997-04-23 | 1999-09-21 | Novell, Inc. | System and method for automatically resizing a disk drive volume |
| US5963982A (en) * | 1997-07-02 | 1999-10-05 | Webtv Networks, Inc. | Defragmentation of stored data without pointer indirection |
| US6182200B1 (en) * | 1997-09-24 | 2001-01-30 | Sony Corporation | Dense edit re-recording to reduce file fragmentation |
| US6047360A (en) * | 1997-09-24 | 2000-04-04 | Sony Corporation | System and method of organizing and defragmenting audio events recorded on a storage medium |
| US6205529B1 (en) | 1997-09-25 | 2001-03-20 | Emc Corporation | Method and apparatus for defragmenting a storage device using a copy function in the device control logic |
| US6070174A (en) * | 1997-09-30 | 2000-05-30 | Infraworks Corporation | Method and apparatus for real-time secure file deletion |
| US6070170A (en) * | 1997-10-01 | 2000-05-30 | International Business Machines Corporation | Non-blocking drain method and apparatus used to reorganize data in a database |
| US6304880B1 (en) * | 1997-12-12 | 2001-10-16 | International Business Machines Corporation | Automated reclamation scheduling override in a virtual tape server |
| CN1150541C (en) * | 1998-05-15 | 2004-05-19 | 松下电器产业株式会社 | Data copying method for disc recorded medium, data recording method, data editing method and disc recorder |
| US6324497B1 (en) * | 1998-07-15 | 2001-11-27 | Sutmyn Storage Corporation | Tape drive emulation system including tape library interface |
| US6438642B1 (en) * | 1999-05-18 | 2002-08-20 | Kom Networks Inc. | File-based virtual storage file system, method and computer program product for automated file management on multiple file system storage devices |
| JP2000048549A (en) * | 1998-08-03 | 2000-02-18 | Fujitsu Ltd | Tape device |
| US6321239B1 (en) | 1998-09-28 | 2001-11-20 | International Business Machines Corporation | Efficient volume copy using pre-configuration of log structured target storage |
| JP2000137632A (en) * | 1998-10-30 | 2000-05-16 | Square Co Ltd | Record medium which computer can read, optical disk and information processor |
| US6571261B1 (en) | 2000-07-13 | 2003-05-27 | International Business Machines Corporation | Defragmentation utility for a shared disk parallel file system across a storage area network |
| US6757801B1 (en) * | 2000-10-26 | 2004-06-29 | International Business Machines Corporation | Method to modify that an operation needs to be done on a file system |
| US20020063985A1 (en) * | 2000-11-29 | 2002-05-30 | Jack Chen | Disc processing and treating system |
| KR20050088451A (en) * | 2002-12-20 | 2005-09-06 | 코닌클리케 필립스 일렉트로닉스 엔.브이. | Class allocation for volatile files |
| TW200634756A (en) * | 2005-03-16 | 2006-10-01 | Lite On It Corp | Method for storage space allocation and data recording on a DVD re-writable disc |
| US7711876B2 (en) * | 2005-03-23 | 2010-05-04 | International Business Machines Corporation | Dynamic category compression in a data storage library |
| US7657579B2 (en) * | 2005-04-14 | 2010-02-02 | Emc Corporation | Traversing data in a repeatable manner |
| US20080065663A1 (en) * | 2005-04-14 | 2008-03-13 | Emc Corporation | Reestablishing process context |
| GB0517305D0 (en) * | 2005-08-24 | 2005-10-05 | Ibm | Method and apparatus for the defragmentation of a file system |
| US7870128B2 (en) * | 2006-07-28 | 2011-01-11 | Diskeeper Corporation | Assigning data for storage based on speed with which data may be retrieved |
| US20090132621A1 (en) * | 2006-07-28 | 2009-05-21 | Craig Jensen | Selecting storage location for file storage based on storage longevity and speed |
| US9052826B2 (en) * | 2006-07-28 | 2015-06-09 | Condusiv Technologies Corporation | Selecting storage locations for storing data based on storage location attributes and data usage statistics |
| US20080172430A1 (en) * | 2007-01-11 | 2008-07-17 | Andrew Thomas Thorstensen | Fragmentation Compression Management |
| US7747578B2 (en) * | 2007-03-09 | 2010-06-29 | Microsoft Corporation | Reducing database fragmentation |
| US9043373B2 (en) | 2010-08-17 | 2015-05-26 | Iolo Technologies, Llc | System and method for efficient data storage |
| CN102693325B (en) * | 2012-06-12 | 2014-06-11 | 腾讯科技（深圳）有限公司 | File storing method and device |
| US8782005B2 (en) * | 2012-12-12 | 2014-07-15 | Storagecraft Technology Corporation | Pruning previously-allocated free blocks from a synthetic backup |
| CN110716884B (en) * | 2019-09-20 | 2023-11-17 | 深圳市网心科技有限公司 | Fragment data storage method, electronic equipment and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4827462A (en) * | 1987-03-26 | 1989-05-02 | International Business Machines Corporation | Modular data storage directories for large-capacity data storage units |
| US5398142B1 (en) * | 1989-05-31 | 1997-09-16 | Raxco Inc | Method for eliminating file fragmentation and reducing average seek times in a magnetic disk media environment |
-
1989
- 1989-05-31 US US07359316 patent/US5398142B1/en not_active Expired - Lifetime
- 1989-09-29 CA CA000615272A patent/CA1322242C/en not_active Expired - Fee Related
-
1990
- 1990-05-29 AU AU56088/90A patent/AU627029B2/en not_active Ceased
-
1995
- 1995-03-10 US US08/402,451 patent/US5808821A/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| AU627029B2 (en) | 1992-08-13 |
| US5398142A (en) | 1995-03-14 |
| AU5608890A (en) | 1990-12-06 |
| US5808821A (en) | 1998-09-15 |
| US5398142B1 (en) | 1997-09-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA1322242C (en) | Method for eliminating file fragmentation and reducing average seek times in a magnetic disk media environment | |
| US7350017B2 (en) | Magnetic disk unit, file management system, and file management method | |
| US9002795B2 (en) | Object-based data storage device | |
| US5734861A (en) | Log-structured disk array with garbage collection regrouping of tracks to preserve seek affinity | |
| CN1118035C (en) | Storage of computer data | |
| CA2400870C (en) | File management method | |
| US6901497B2 (en) | Partition creating method and deleting method | |
| US8051115B2 (en) | Measuring fragmentation on direct access storage devices and defragmentation thereof | |
| KR20030011100A (en) | Methods and devices for recording or reading files on/from a sequential medium and sequential medium | |
| CN108255408A (en) | Date storage method and system | |
| CN104065906B (en) | Video recording method and device of digital video recording equipment | |
| Jung et al. | Lifetime-leveling LSM-tree compaction for ZNS SSD | |
| JP4502375B2 (en) | File system and control method thereof | |
| US20020133683A1 (en) | Method and system for reducing fragmentation | |
| US20010042167A1 (en) | Method of storing data and data storage device using the same | |
| JP6905183B2 (en) | Information processing equipment, programs and information processing methods | |
| KR20030061948A (en) | : Apparatus storing information and method for controlling the File using its | |
| JP3061385B2 (en) | Data management device and data management method | |
| CN112433673B (en) | Method and device for storing data in solid state disk | |
| Hafeez | Role of File System in Operating System | |
| KR100816820B1 (en) | Apparatus and method for managing buffer linked with flash memory | |
| JPH0922331A (en) | Information recording and reproducing method | |
| JPS62177642A (en) | File management system for postscript filing device | |
| JP2689116B2 (en) | Optical disk storage management method | |
| JPH04287140A (en) | File managing system for dividing into plural physical files |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MKLA | Lapsed |