Documentation/filesystems/ext4.txt

   1
   2 Ext4 Filesystem
   3 ===============
   4
   5 Ext4 is an an advanced level of the ext3 filesystem which incorporates
   6 scalability and reliability enhancements for supporting large filesystems
   7 (64 bit) in keeping with increasing disk capacities and state-of-the-art
   8 feature requirements.
   9
  10 Mailing list:   linux-ext4@vger.kernel.org
  11 Web site:       http://ext4.wiki.kernel.org
  12
  13
  14 1. Quick usage instructions:
  15 ===========================
  16
  17 Note: More extensive information for getting started with ext4 can be
  18       found at the ext4 wiki site at the URL:
  19       http://ext4.wiki.kernel.org/index.php/Ext4_Howto
  20
  21   - Compile and install the latest version of e2fsprogs (as of this
  22     writing version 1.41.3) from:
  23
  24     http://sourceforge.net/project/showfiles.php?group_id=2406
  25
  26         or
  27
  28     ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/
  29
  30         or grab the latest git repository from:
  31
  32     git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git
  33
  34   - Note that it is highly important to install the mke2fs.conf file
  35     that comes with the e2fsprogs 1.41.x sources in /etc/mke2fs.conf. If
  36     you have edited the /etc/mke2fs.conf file installed on your system,
  37     you will need to merge your changes with the version from e2fsprogs
  38     1.41.x.
  39
  40   - Create a new filesystem using the ext4 filesystem type:
  41
  42         # mke2fs -t ext4 /dev/hda1
  43
  44     Or to configure an existing ext3 filesystem to support extents:
  45
  46         # tune2fs -O extents /dev/hda1
  47
  48     If the filesystem was created with 128 byte inodes, it can be
  49     converted to use 256 byte for greater efficiency via:
  50
  51         # tune2fs -I 256 /dev/hda1
  52
  53     (Note: we currently do not have tools to convert an ext4
  54     filesystem back to ext3; so please do not do try this on production
  55     filesystems.)
  56
  57   - Mounting:
  58
  59         # mount -t ext4 /dev/hda1 /wherever
  60
  61   - When comparing performance with other filesystems, it's always
  62     important to try multiple workloads; very often a subtle change in a
  63     workload parameter can completely change the ranking of which
  64     filesystems do well compared to others.  When comparing versus ext3,
  65     note that ext4 enables write barriers by default, while ext3 does
  66     not enable write barriers by default.  So it is useful to use
  67     explicitly specify whether barriers are enabled or not when via the
  68     '-o barriers=[0|1]' mount option for both ext3 and ext4 filesystems
  69     for a fair comparison.  When tuning ext3 for best benchmark numbers,
  70     it is often worthwhile to try changing the data journaling mode; '-o
  71     data=writeback,nobh' can be faster for some workloads.  (Note
  72     however that running mounted with data=writeback can potentially
  73     leave stale data exposed in recently written files in case of an
  74     unclean shutdown, which could be a security exposure in some
  75     situations.)  Configuring the filesystem with a large journal can
  76     also be helpful for metadata-intensive workloads.
  77
  78 2. Features
  79 ===========
  80
  81 2.1 Currently available
  82
  83 * ability to use filesystems > 16TB (e2fsprogs support not available yet)
  84 * extent format reduces metadata overhead (RAM, IO for access, transactions)
  85 * extent format more robust in face of on-disk corruption due to magics,
  86 * internal redundancy in tree
  87 * improved file allocation (multi-block alloc)
  88 * lift 32000 subdirectory limit imposed by i_links_count[1]
  89 * nsec timestamps for mtime, atime, ctime, create time
  90 * inode version field on disk (NFSv4, Lustre)
  91 * reduced e2fsck time via uninit_bg feature
  92 * journal checksumming for robustness, performance
  93 * persistent file preallocation (e.g for streaming media, databases)
  94 * ability to pack bitmaps and inode tables into larger virtual groups via the
  95   flex_bg feature
  96 * large file support
  97 * Inode allocation using large virtual block groups via flex_bg
  98 * delayed allocation
  99 * large block (up to pagesize) support
 100 * efficent new ordered mode in JBD2 and ext4(avoid using buffer head to force
 101   the ordering)
 102
 103 [1] Filesystems with a block size of 1k may see a limit imposed by the
 104 directory hash tree having a maximum depth of two.
 105
 106 2.2 Candidate features for future inclusion
 107
 108 * Online defrag (patches available but not well tested)
 109 * reduced mke2fs time via lazy itable initialization in conjuction with
 110   the uninit_bg feature (capability to do this is available in e2fsprogs
 111   but a kernel thread to do lazy zeroing of unused inode table blocks
 112   after filesystem is first mounted is required for safety)
 113
 114 There are several others under discussion, whether they all make it in is
 115 partly a function of how much time everyone has to work on them. Features like
 116 metadata checksumming have been discussed and planned for a bit but no patches
 117 exist yet so I'm not sure they're in the near-term roadmap.
 118
 119 The big performance win will come with mballoc, delalloc and flex_bg
 120 grouping of bitmaps and inode tables.  Some test results available here:
 121
 122  - http://www.bullopensource.org/ext4/20080818-ffsb/ffsb-write-2.6.27-rc1.html
 123  - http://www.bullopensource.org/ext4/20080818-ffsb/ffsb-readwrite-2.6.27-rc1.html
 124
 125 3. Options
 126 ==========
 127
 128 When mounting an ext4 filesystem, the following option are accepted:
 129 (*) == default
 130
 131 ro                      Mount filesystem read only. Note that ext4 will
 132                         replay the journal (and thus write to the
 133                         partition) even when mounted "read only". The
 134                         mount options "ro,noload" can be used to prevent
 135                         writes to the filesystem.
 136
 137 journal_async_commit    Commit block can be written to disk without waiting
 138                         for descriptor blocks. If enabled older kernels cannot
 139                         mount the device.
 140
 141 journal=update          Update the ext4 file system's journal to the current
 142                         format.
 143
 144 journal_dev=devnum      When the external journal device's major/minor numbers
 145                         have changed, this option allows the user to specify
 146                         the new journal location.  The journal device is
 147                         identified through its new major/minor numbers encoded
 148                         in devnum.
 149
 150 noload                  Don't load the journal on mounting.  Note that
 151                         if the filesystem was not unmounted cleanly,
 152                         skipping the journal replay will lead to the
 153                         filesystem containing inconsistencies that can
 154                         lead to any number of problems.
 155
 156 data=journal            All data are committed into the journal prior to being
 157                         written into the main file system.
 158
 159 data=ordered    (*)     All data are forced directly out to the main file
 160                         system prior to its metadata being committed to the
 161                         journal.
 162
 163 data=writeback          Data ordering is not preserved, data may be written
 164                         into the main file system after its metadata has been
 165                         committed to the journal.
 166
 167 commit=nrsec    (*)     Ext4 can be told to sync all its data and metadata
 168                         every 'nrsec' seconds. The default value is 5 seconds.
 169                         This means that if you lose your power, you will lose
 170                         as much as the latest 5 seconds of work (your
 171                         filesystem will not be damaged though, thanks to the
 172                         journaling).  This default value (or any low value)
 173                         will hurt performance, but it's good for data-safety.
 174                         Setting it to 0 will have the same effect as leaving
 175                         it at the default (5 seconds).
 176                         Setting it to very large values will improve
 177                         performance.
 178
 179 barrier=<0|1(*)>        This enables/disables the use of write barriers in
 180 barrier(*)              the jbd code.  barrier=0 disables, barrier=1 enables.
 181 nobarrier               This also requires an IO stack which can support
 182                         barriers, and if jbd gets an error on a barrier
 183                         write, it will disable again with a warning.
 184                         Write barriers enforce proper on-disk ordering
 185                         of journal commits, making volatile disk write caches
 186                         safe to use, at some performance penalty.  If
 187                         your disks are battery-backed in one way or another,
 188                         disabling barriers may safely improve performance.
 189                         The mount options "barrier" and "nobarrier" can
 190                         also be used to enable or disable barriers, for
 191                         consistency with other ext4 mount options.
 192
 193 inode_readahead=n       This tuning parameter controls the maximum
 194                         number of inode table blocks that ext4's inode
 195                         table readahead algorithm will pre-read into
 196                         the buffer cache.  The default value is 32 blocks.
 197
 198 orlov           (*)     This enables the new Orlov block allocator. It is
 199                         enabled by default.
 200
 201 oldalloc                This disables the Orlov block allocator and enables
 202                         the old block allocator.  Orlov should have better
 203                         performance - we'd like to get some feedback if it's
 204                         the contrary for you.
 205
 206 user_xattr              Enables Extended User Attributes.  Additionally, you
 207                         need to have extended attribute support enabled in the
 208                         kernel configuration (CONFIG_EXT4_FS_XATTR).  See the
 209                         attr(5) manual page and http://acl.bestbits.at/ to
 210                         learn more about extended attributes.
 211
 212 nouser_xattr            Disables Extended User Attributes.
 213
 214 acl                     Enables POSIX Access Control Lists support.
 215                         Additionally, you need to have ACL support enabled in
 216                         the kernel configuration (CONFIG_EXT4_FS_POSIX_ACL).
 217                         See the acl(5) manual page and http://acl.bestbits.at/
 218                         for more information.
 219
 220 noacl                   This option disables POSIX Access Control List
 221                         support.
 222
 223 reservation
 224
 225 noreservation
 226
 227 bsddf           (*)     Make 'df' act like BSD.
 228 minixdf                 Make 'df' act like Minix.
 229
 230 debug                   Extra debugging information is sent to syslog.
 231
 232 abort                   Simulate the effects of calling ext4_abort() for
 233                         debugging purposes.  This is normally used while
 234                         remounting a filesystem which is already mounted.
 235
 236 errors=remount-ro       Remount the filesystem read-only on an error.
 237 errors=continue         Keep going on a filesystem error.
 238 errors=panic            Panic and halt the machine if an error occurs.
 239                         (These mount options override the errors behavior
 240                         specified in the superblock, which can be configured
 241                         using tune2fs)
 242
 243 data_err=ignore(*)      Just print an error message if an error occurs
 244                         in a file data buffer in ordered mode.
 245 data_err=abort          Abort the journal if an error occurs in a file
 246                         data buffer in ordered mode.
 247
 248 grpid                   Give objects the same group ID as their creator.
 249 bsdgroups
 250
 251 nogrpid         (*)     New objects have the group ID of their creator.
 252 sysvgroups
 253
 254 resgid=n                The group ID which may use the reserved blocks.
 255
 256 resuid=n                The user ID which may use the reserved blocks.
 257
 258 sb=n                    Use alternate superblock at this location.
 259
 260 quota                   These options are ignored by the filesystem. They
 261 noquota                 are used only by quota tools to recognize volumes
 262 grpquota                where quota should be turned on. See documentation
 263 usrquota                in the quota-tools package for more details
 264                         (http://sourceforge.net/projects/linuxquota).
 265
 266 jqfmt=<quota type>      These options tell filesystem details about quota
 267 usrjquota=<file>        so that quota information can be properly updated
 268 grpjquota=<file>        during journal replay. They replace the above
 269                         quota options. See documentation in the quota-tools
 270                         package for more details
 271                         (http://sourceforge.net/projects/linuxquota).
 272
 273 bh              (*)     ext4 associates buffer heads to data pages to
 274 nobh                    (a) cache disk block mapping information
 275                         (b) link pages into transaction to provide
 276                             ordering guarantees.
 277                         "bh" option forces use of buffer heads.
 278                         "nobh" option tries to avoid associating buffer
 279                         heads (supported only for "writeback" mode).
 280
 281 stripe=n                Number of filesystem blocks that mballoc will try
 282                         to use for allocation size and alignment. For RAID5/6
 283                         systems this should be the number of data
 284                         disks *  RAID chunk size in file system blocks.
 285
 286 delalloc        (*)     Defer block allocation until just before ext4
 287                         writes out the block(s) in question.  This
 288                         allows ext4 to better allocation decisions
 289                         more efficiently.
 290 nodelalloc              Disable delayed allocation.  Blocks are allocated
 291                         when the data is copied from userspace to the
 292                         page cache, either via the write(2) system call
 293                         or when an mmap'ed page which was previously
 294                         unallocated is written for the first time.
 295
 296 max_batch_time=usec     Maximum amount of time ext4 should wait for
 297                         additional filesystem operations to be batch
 298                         together with a synchronous write operation.
 299                         Since a synchronous write operation is going to
 300                         force a commit and then a wait for the I/O
 301                         complete, it doesn't cost much, and can be a
 302                         huge throughput win, we wait for a small amount
 303                         of time to see if any other transactions can
 304                         piggyback on the synchronous write.   The
 305                         algorithm used is designed to automatically tune
 306                         for the speed of the disk, by measuring the
 307                         amount of time (on average) that it takes to
 308                         finish committing a transaction.  Call this time
 309                         the "commit time".  If the time that the
 310                         transaction has been running is less than the
 311                         commit time, ext4 will try sleeping for the
 312                         commit time to see if other operations will join
 313                         the transaction.   The commit time is capped by
 314                         the max_batch_time, which defaults to 15000us
 315                         (15ms).   This optimization can be turned off
 316                         entirely by setting max_batch_time to 0.
 317
 318 min_batch_time=usec     This parameter sets the commit time (as
 319                         described above) to be at least min_batch_time.
 320                         It defaults to zero microseconds.  Increasing
 321                         this parameter may improve the throughput of
 322                         multi-threaded, synchronous workloads on very
 323                         fast disks, at the cost of increasing latency.
 324
 325 journal_ioprio=prio     The I/O priority (from 0 to 7, where 0 is the
 326                         highest priorty) which should be used for I/O
 327                         operations submitted by kjournald2 during a
 328                         commit operation.  This defaults to 3, which is
 329                         a slightly higher priority than the default I/O
 330                         priority.
 331
 332 auto_da_alloc(*)        Many broken applications don't use fsync() when
 333 noauto_da_alloc         replacing existing files via patterns such as
 334                         fd = open("foo.new")/write(fd,..)/close(fd)/
 335                         rename("foo.new", "foo"), or worse yet,
 336                         fd = open("foo", O_TRUNC)/write(fd,..)/close(fd).
 337                         If auto_da_alloc is enabled, ext4 will detect
 338                         the replace-via-rename and replace-via-truncate
 339                         patterns and force that any delayed allocation
 340                         blocks are allocated such that at the next
 341                         journal commit, in the default data=ordered
 342                         mode, the data blocks of the new file are forced
 343                         to disk before the rename() operation is
 344                         committed.  This provides roughly the same level
 345                         of guarantees as ext3, and avoids the
 346                         "zero-length" problem that can happen when a
 347                         system crashes before the delayed allocation
 348                         blocks are forced to disk.
 349
 350 Data Mode
 351 =========
 352 There are 3 different data modes:
 353
 354 * writeback mode
 355 In data=writeback mode, ext4 does not journal data at all.  This mode provides
 356 a similar level of journaling as that of XFS, JFS, and ReiserFS in its default
 357 mode - metadata journaling.  A crash+recovery can cause incorrect data to
 358 appear in files which were written shortly before the crash.  This mode will
 359 typically provide the best ext4 performance.
 360
 361 * ordered mode
 362 In data=ordered mode, ext4 only officially journals metadata, but it logically
 363 groups metadata information related to data changes with the data blocks into a
 364 single unit called a transaction.  When it's time to write the new metadata
 365 out to disk, the associated data blocks are written first.  In general,
 366 this mode performs slightly slower than writeback but significantly faster than journal mode.
 367
 368 * journal mode
 369 data=journal mode provides full data and metadata journaling.  All new data is
 370 written to the journal first, and then to its final location.
 371 In the event of a crash, the journal can be replayed, bringing both data and
 372 metadata into a consistent state.  This mode is the slowest except when data
 373 needs to be read from and written to disk at the same time where it
 374 outperforms all others modes.  Currently ext4 does not have delayed
 375 allocation support if this data journalling mode is selected.
 376
 377 References
 378 ==========
 379
 380 kernel source:  <file:fs/ext4/>
 381                 <file:fs/jbd2/>
 382
 383 programs:       http://e2fsprogs.sourceforge.net/
 384
 385 useful links:   http://fedoraproject.org/wiki/ext3-devel
 386                 http://www.bullopensource.org/ext4/
 387                 http://ext4.wiki.kernel.org/index.php/Main_Page
 388                 http://fedoraproject.org/wiki/Features/Ext4