← Back to list
Computing & Embedded
#파일 시스템#저널링#inode#VFS#Copy-on-Write

File System Structure and Journaling

1. Overview

A. Definition

A file system is a core operating-system component that organizes, names, accesses and protects the block set of secondary storage (disk, SSD, etc.) as logical units called files and directories, managing storage space so that consistency survives even a power failure. Here, journaling is a crash-consistency technique that records the intent of a change in a journal (log) first, before applying the data to its home location, so the file system can be recovered to a contradiction-free state even if a failure strikes mid-write.

The fundamental role of a file system is to turn an "ocean of structureless byte blocks" into named objects that people and programs can handle. Storage devices offer only a flat address space accessible by sector/block number; the metadata scheme that layers meaning on top — "which block belongs to which file, what directory tree the files form, who may access them" — is precisely the file system. Without this abstraction every application would have to manage the disk's physical layout directly, and sharing and protection in a multi-user, multi-process environment would be impossible.

B. Background and Necessity

In early computing, applications dealt with storage directly in blocks and had to track the location, size and ownership of data themselves. This bred three defects: lack of portability (change the device and the code changes), impossibility of sharing (two programs could not safely divide the same data), and lack of naming (access only by physical address). The file system resolved all three at once by providing name-based access, hierarchical directories, access permissions and a device-independent interface.

But a more urgent need arose from failure recovery. Even the simple operation of adding one file is internally completed only by writing several blocks in sequence — the data block, an inode update, a directory entry, a bitmap update. If power is cut mid-operation, a contradictory state remains: "data written but not in the directory," or "in the directory but the block unallocated." Early Unix used fsck at boot to scan the whole disk and fix such contradictions, but on multi-TB disks the check alone took tens of minutes to hours, severely harming availability. Journaling solved this by making "recovery time proportional not to disk size but to the number of incomplete transactions," and it has become the standard of nearly every general-purpose file system today, including ext4, XFS and NTFS.

C. Characteristics

A file system has several essential characteristics. First, durability: data must be preserved even when power is off, a decisive property distinguishing it from volatile memory. Second, consistency: metadata and data must always remain mutually contradiction-free. Third, a performance–safety trade-off: writing immediately to disk is safe but slow, while gathering writes in a buffer cache for deferred write-back is fast but raises the risk of loss on failure — it is no exaggeration to say the history of file-system design is the process of solving this trade-off ever more cleverly.

These characteristics stem from the file system's position as the manager of the storage hierarchy. Just as the CPU cache caches memory and virtual memory caches disk, the file system caches disk blocks in memory's page cache (buffer cache) while deciding the synchronization policy between the two. When to flush dirty pages to disk, in what order to write, and what to guarantee on failure are all the file system's responsibility, and the quality of this policy governs the data safety and throughput of the whole system.

A fourth characteristic is protection and sharing in a multi-user, concurrent environment. The file system enforces who may access what through owner/group/permission bits and ACLs, and must provide locks and a consistent view when multiple processes read and write the same file simultaneously. This means that, beyond mere storage, the file system is the front line of OS security and isolation. For instance, containers share image layers through namespaces and an overlay file system (union mount) while giving each container an independent root file system — all of which rests on the file-system abstraction.

2. The Layered Structure and Components of a File System

A file system consists of several layers that translate an application's abstract file operations into physical block I/O. Below is an overall structure diagram based on Linux.

flowchart TD
    APP["Application: open/read/write/close"] --> SC["System call interface"]
    SC --> VFS["VFS(Virtual File System): common abstract layer"]
    VFS --> FS1["ext4 / XFS"]
    VFS --> FS2["Btrfs / ZFS(CoW)"]
    VFS --> FS3["NFS / network FS"]
    FS1 --> PC["Page cache(buffer cache)"]
    FS2 --> PC
    PC --> BIO["Block I/O layer, I/O scheduler"]
    BIO --> DRV["Device driver"]
    DRV --> DEV[("HDD / SSD / NVMe")]

The heart of this diagram is the common abstract layer called the VFS (Virtual File System). Applications use the same open/read/write interface without knowing the file-system type, and the VFS connects it to each file system's implementation (ext4, XFS, NFS, etc.). This lets different file systems be mounted into a single directory tree and mixed transparently. Below it, the page cache keeps recently accessed blocks in memory to accelerate repeat access, and the block I/O layer merges and reorders many requests (I/O scheduling) to raise efficiency according to device characteristics.

A. The Virtual File System (VFS) Abstraction

The VFS defines a common object model every file system must implement. The representative objects are the superblock (whole-file-system metadata), the inode (metadata of an individual file/directory), the dentry (a directory entry, a name↔inode mapping cache), and the file (the state of an open file). Each file system registers an operation table (a set of function pointers) for these objects, and the VFS implements polymorphism by calling those pointers. This is a classic case of realizing the object-oriented separation of interface and implementation through function pointers in kernel C code.

The practical significance of this abstraction lies in extensibility and interoperability. Adding a new file system (e.g., flash-only F2FS) to the kernel changes not a single line of application code, and local, network and pseudo file systems (/proc, /sys, etc.) can be handled in the same tree. For example, /proc/cpuinfo is not a disk file but a virtual file the kernel generates on the fly, yet to an application it looks like an ordinary file.

Another key the VFS provides is path name resolution. Opening /home/user/a.txt, the VFS descends from the root one component at a time, following directory inodes while querying and building the dentry cache. The more directory entries there are, the more this traversal becomes a bottleneck, so modern file systems structure directories not as linear lists but as a hash tree (ext4's HTree) or a B+ tree (XFS, Btrfs), guaranteeing logarithmic-time lookup even among tens of thousands to millions of entries. Jumping transparently from a mount point to another file system is also handled by the VFS during path resolution.

B. Metadata Structures — Superblock, inode, Directory

A file system on a storage device is usually partitioned into boot block → superblock → inode region (and bitmaps) → data block region. The superblock holds global information such as block size, total/free block count and inode count; it is read first at mount and is replicated in several copies against corruption. An inode (index node) holds all of a file's metadata — owner, permissions, size, timestamps, and crucially the location of the data blocks — but does not hold the file name. The name↔inode mapping is kept by the directory, and thanks to this separation hard links — attaching several names to one file (inode) — are naturally possible.

Another effect of separating name from metadata is reference-count-based deletion. The inode records the number of hard links pointing to it (link count), and rm deletes only the directory entry and decrements the link count by one. Only when the link count reaches 0 and no process has the file open are the data blocks reclaimed. The Unix-specific behavior whereby "deleting an open file keeps its contents alive until it is closed," and the common operational mishap where the space of a deleted log file is not returned until the process restarts, both arise from this structure.

The inode's way of pointing to data blocks traditionally uses multi-level indexing. By combining a few direct block pointers (handling small files quickly) with single/double/triple indirect blocks (deepening one level at a time for larger files), small files are accessed instantly and large files via a pointer tree. For example, with 4KB blocks and 4-byte pointers, a single indirect block can point to 1024 blocks (4MB) and a double indirect block to about 4GB, expanding file size practically without limit. The drawback of increased pointer chasing on large files is offset by modern file systems with the extent scheme below.

C. Block Allocation Methods — Contiguous, Linked, Indexed, Extent

The allocation method that assigns disk blocks to a file governs performance and fragmentation. Contiguous allocation places a file in consecutive blocks, making both sequential and random access fast, but external fragmentation is severe and file growth is hard. Linked allocation has each block point to the next, so growth is easy but random access is slow and it is vulnerable to pointer corruption. Indexed allocation, like the inode above, gathers block addresses into an index block, supporting both random access and growth. The modern extent scheme expresses a large contiguous region as a single entry of "start block + run length," dramatically reducing the metadata of large files and raising sequential throughput — the reason ext4, XFS and NTFS all adopt extents.

In concrete numbers the effect of extents is clear. With 4KB blocks a 1GB file consists of about 260,000 blocks; a block-pointer index scheme points to all of them with 4-byte pointers, consuming about 1MB of indirect-block metadata. If the file is laid out contiguously on disk, however, the extent scheme finishes with a single "{start, 260,000}" entry, shrinking metadata to a few bytes and eliminating pointer-chasing I/O on sequential reads. For exactly this reason, extent file systems maximize large-file throughput by combining with delayed allocation — holding writes and assigning the largest possible contiguous region at once at the moment of actual commit — to suppress fragmentation.

The table below compares the four methods, but the point lies in how one compromises along the trade-off axes of "random accessibility, ease of growth, fragmentation, metadata size." General-purpose file systems converged on indexed/extent because it is the most balanced point across the four axes.

Allocation method Random access File growth External fragmentation Representative use
Contiguous Very fast Hard Severe ISO9660(CD), some RT FS
Linked Slow Easy None FAT(variant)
Indexed Fast Easy Moderate Traditional Unix(ext2)
Extent Fast Easy Low ext4, XFS, NTFS

3. Crash Consistency and Journaling

The hardest problem of a file system is "applying an update that spans multiple blocks atomically." Below is the write flow by which journaling guarantees this.

flowchart LR
    TX["File operation starts(transaction)"] --> JW["① Write changed blocks to the journal(write)"]
    JW --> CMT["② Write the commit record(commit)"]
    CMT --> CKP["③ Checkpoint: apply to home location"]
    CKP --> FREE["④ Reclaim journal space"]
    CMT -.->|"Failure before ③"| RCV1["Reboot: redo committed transaction(redo)"]
    JW -.->|"Failure before ②"| RCV2["Reboot: discard incomplete transaction"]

The core principle is "write first, apply later (Write-Ahead Logging)." Before writing changes to their home location, they are written to the journal first, and only after the commit record has been fully recorded in the journal is it applied to the home location (checkpoint). On reboot, replaying the journal redoes any transaction that reached commit, applying it without fail, and discards any transaction cut off before commit. Thus a "half-applied" contradiction never remains in principle, and recovery cost is proportional only to the unprocessed transactions left in the journal, not to the whole disk.

A. The Crash-Consistency Problem

Let us see concretely why a single logical operation like adding a file is dangerous. To allocate a block to a new file, three things must all be written: ① writing the data block, ② updating the inode's size/pointers, ③ marking the block as in-use in the block bitmap. If only ③ is applied and power is cut, that block becomes a permanently leaked block that is "in use but pointed to by no file." Conversely, if only ② is applied, the inode points to a block unallocated in the bitmap, and the instant that block is reallocated to another file, cross-contamination of data occurs. Partial updates like these can destroy data integrity, not merely waste space.

The essence of the problem is that the unit of atomicity the storage device guarantees stops at a single-sector (or single-block) write. There is no hardware mechanism to bind a logical operation spanning several blocks into an atomic unit that is "either fully applied or not at all," so software must create one. Journaling is exactly the technique that synthesizes this "atomicity of multi-block writes" out of a separate region called the journal and a commit record, applying to the file system essentially the same idea as the database's WAL (write-ahead logging) and ARIES recovery technique.

B. The Principle and Modes of Journaling

Journaling splits on performance and safety by "what is written to the journal." Metadata journaling writes only metadata such as inodes and bitmaps to the journal and writes the actual data directly to the home location. It guarantees metadata integrity, but if the data block is applied later than the metadata, a window can open in which a new inode points to data not yet written (garbage values). ext4's three modes compromise this trade-off differently.

Mode Journal target Data ordering guarantee Performance Safety
journal Metadata + data Full(data also via journal) Slowest Highest
ordered(default) Metadata only Data written before metadata Medium High
writeback Metadata only No ordering guarantee Fastest Low(garbage exposure possible)

In practice most use ordered mode by default. It avoids the duplication of writing data once to the journal and once to the home location (the 2× write cost of journal mode) while preventing garbage exposure through the ordering rule of "flush the data to disk first, then commit the metadata." Only in domains that absolutely forbid data loss, such as finance and healthcare, is the full journaling of journal mode chosen.

Journaling also splits along the physical/logical axis. Physical journaling records the entire image of changed blocks to the journal, making implementation simple and recovery certain but the journal capacity/write volume large. Logical journaling records only operation-level changes such as "set inode N's size to M," keeping the journal small, but the replay logic is complex and care is needed in its interaction with partial writes. XFS reduced journal overhead in large-scale, highly parallel environments with logical journaling, whereas ext4's jbd2 chose a block-based physical journal, prioritizing simplicity and stability — a representative design fork.

C. Alternative Techniques — Soft Updates, CoW, Log-structured

Journaling is not the only solution. Soft Updates (BSD FFS) constrain the write order cleverly without a journal, so that even on failure the possible contradictions are limited to harmless forms such as "leaked blocks" (no separate journal space, but a complex implementation instead). Copy-on-Write (CoW) (ZFS, Btrfs) writes to a new block without overwriting the existing one, then atomically swaps the root pointer, so that whenever a failure strikes it always remains in a consistent prior state — this property naturally enables snapshots and per-block checksums. A log-structured file system (LFS) drives all writes into one huge sequential log, turning random writes into sequential writes, which shines especially on media where sequential writes are favorable, like flash/SSD, becoming the foundation of F2FS described later.

Yet no technique is free. CoW and log-structuring gain consistency and snapshots thanks to "always writing to a new location," but at the cost of accumulating fragmentation and incurring the cost of garbage collection (segment cleaning) that gathers old valid blocks to reclaim space. In particular, under workloads that repeatedly overwrite the same block in small pieces, like databases, CoW can amplify write amplification and cause performance to plunge. Therefore the engineer must clearly recognize that "the choice of consistency technique" is not a simple superiority of journaling versus CoW, but a design compromise considering together the workload's write pattern, the medium, and required features (snapshots, checksums).

4. Comparison of Major File Systems

The choice of file system varies with "workload characteristics (large sequential vs. small random), medium (HDD vs. flash), and required features (snapshot, checksum, scalability)." From an engineer's viewpoint, understanding the difference in design philosophy matters more than a simple comparison of performance numbers.

File system Consistency technique Key features Main use
ext4 Metadata journaling Extent, stability, compatibility Linux general default
XFS Metadata journaling Parallel I/O, large-scale scalability RHEL default, large servers
NTFS Journaling($LogFile) ACL, compression, USN journal Windows
APFS Copy-on-Write Snapshot, clone, encryption macOS, iOS(2017~)
ZFS / Btrfs Copy-on-Write Snapshot, checksum, pool management NAS, storage servers
F2FS Log-structured Flash optimization, sequential write Mobile, SSD

The differences in this table are no accident but the result of the medium and demands at the time of emergence dictating the design. ext4/XFS achieved the stability/scalability of the HDD era through journaling; ZFS/Btrfs/APFS solved the new demand of "data-integrity verification (checksum) and snapshots" with CoW; and F2FS accepted the medium characteristic of "flash where overwriting is expensive" with a log structure. For example, Samsung built F2FS in 2012 because small random writes on flash induce garbage collection and write amplification that erode lifespan and performance, and the FAST '15 paper indeed reports F2FS showing up to 2.5× the throughput of ext4 on a SATA SSD.

Real-world cases back this philosophical difference. In an environment where hundreds of millions of iPhones and Macs all use flash, Apple replaced HFS+ with the CoW-based APFS in 2017, gaining both safety and space efficiency through atomic snapshots and instant clones during app installs and system updates. Conversely, large Linux servers and cloud instances still make the decades-proven ext4/XFS their default, reflecting an enterprise's conservative choice that prioritizes "predictable stability and ecosystem maturity over new features."

5. Advanced — Latest Trends and Next-Generation File Systems

File systems are still evolving to match changes in media and workloads. First, flash nativization. F2FS has settled in as the data-partition default on many Android devices (Pixel, etc.), reducing flash write amplification with its log-structured design. Yet ext4 is still widely used, and a conservative choice on reliability/maturity grounds coexists. Second, the mainstreaming of CoW file systems: following ZFS/Btrfs, the next-generation CoW bcachefs was merged experimentally into Linux 6.7 (2024), but amid development-governance conflicts it was removed from the mainline over the 6.17–6.18 kernels in 2025 and is maintained as an external DKMS module — a case showing how long a new file system takes to stabilize.

Third, Persistent Memory and DAX. As byte-addressable non-volatile memory emerged, a path was added to ext4/XFS for applications to map storage directly like memory (DAX, Direct Access), bypassing the block layer and page cache. Fourth, asynchronous I/O innovation: Linux io_uring reduces system-call counts and context switches with shared ring buffers, letting applications actually draw out the hundreds of thousands to millions of IOPS an NVMe SSD delivers. Alongside this, a "host–device cooperation" model such as ZNS (Zoned Namespace) SSD — where the device exposes write regions in zone units so the file system writes sequentially aligned to flash erase-block boundaries — is also rising, meshing naturally with log-structured design to lower write amplification further. Fifth, distributed file systems: beyond a single node, systems that distribute and replicate data across thousands of nodes — Google's Colossus (GFS's successor), open-source Ceph/Lustre/HDFS — function as the de facto file systems of large data centers. They bear design challenges different from local file systems, such as metadata-server separation, replication/erasure coding, and relaxed consistency models. Sixth, computational storage, where attempts to perform filtering, compression and encryption directly inside the device holding the data — reducing data movement to the host — are underway along with standardization (NVMe Computational Storage), a trend that re-asks "the boundary between file system and storage device."

6. Considerations and Implications

First, the data-safety/performance trade-off must be designed to the required level. The choice of journal mode (journal/ordered/writeback), the frequency of fsync calls, and the write-barrier setting all reduce to the business requirement of "how much data loss can be tolerated." Choosing the safest setting unconditionally can halve throughput, so it is advisable to weigh RPO (Recovery Point Objective) against throughput needs and apply different policies by tier.

Second, medium characteristics dictate file-system choice. On an HDD, sequential access is tens of times faster than random, so defragmentation and block locality matter, whereas on SSD/flash the sequential/random gap is small but write amplification and lifespan (TBW) are the crux. Flash-based systems should therefore choose F2FS/log-structuring or settings that mesh with TRIM/over-provisioning to reduce the garbage-collection burden.

Third, integrity verification and recovery schemes must extend beyond the file system. Journaling only guarantees "crash consistency"; it cannot prevent bit rot or silent corruption. ZFS/Btrfs's per-block checksum + scrubbing detects and repairs these, and only by combining regular snapshots, off-site backup and RAID is end-to-end data protection completed. One must beware the misconception that "because there is journaling, backups are unnecessary."

Fourth, scalability and operational automation must be considered proactively. A single file system's limits on inode count and volume size, directory-lookup performance under hundreds of millions of small files, and whether online resize/migration is possible all affect the service throughout its lifetime. If there is a chance of growing to multi-PB scale and multiple nodes, reflecting a migration path to a distributed file system or object storage in the architecture from the start is the way to reduce redesign cost.

Fifth, alignment with security and regulatory requirements must be secured. In-transit/at-rest encryption (file-system-level encryption, fscrypt), access control (POSIX ACL, extended attributes), audit logging, and whether physical complete erasure is guaranteed when destroying personal data are the crux of regulatory compliance. In CoW/flash environments, overwriting may not guarantee actual physical erasure, so an alternative technique such as encryption-key destruction (crypto-shredding) must be designed in together.

References


In one line: A file system organizes flat block space into named files through VFS, inodes, directories and extents, while journaling (write-ahead logging) guarantees multi-block updates atomically — making recovery time proportional to incomplete transactions rather than disk size — and it keeps evolving for the flash and large-scale era through CoW, log-structuring and distribution.