UNIX: ZFS

scottalanmiller

Originally created for the Sun Microsystems AMD64 and Solaris based open storage Sun Fire X4500 "Thumper" appliance, ZFS or the zettabyte file system was a ground breaking attempt to build a modern, large scale file system that could address many of the perceived needs of the new world being introduced through large chassis, large hard drives of multiple terabytes, and new low cost SATA connections. ZFS was designed primarily around data protection and scalability and at the time was able to go much larger than other filesystems and incorporated many cutting edge features. ZFS was not designed for performance and often under-performs other filesystem types.

The biggest change from other filesystems was that ZFS was not only a filesystem, but a filesystem, logical volume manager, and software RAID system all wrapped up in a single package. This can make ZFS very powerful, but also potentially overwhelming and confusing. ZFS heralded in a new era for filesystems and would be followed by other "integrated file systems" like BtrFS in the Linux world, and ReFS on Windows, both of which primarily focused on copying ZFS' features and approach. So ZFS is really a "storage stack" rather than an individual component. And while the components can be used individually, it is generally assumed that they will be used together. Some of ZFS' features depend on integration between components.

ZFS is very mature and broadly available. While it originated on Solaris it has been ported to Solaris derivatives (OpenSolaris / Illumos, OpenIndiana, Nexenta, etc.), most BSD family members including FreeBSD, Linux, and macOS. It is one of the most broadly available filesystems today. Originally ZFS was a single implementation by Sun and later Oracle, and at one point had been open source. ZFS proper is now closed source, and a fork of its open source version known as OpenZFS is the version used by all platforms other than Solaris proper.

Linux Note: At this time, Ubuntu is the only major Linux distribution including ZFS natively

BSD Note: ZFS has become a key component in FreeBSD's storage strategy and this the primary ZFS use case today.

Key features of ZFS include:

Advanced checksumming stored away from the data
Broad Software RAID features with RAID 0, 1, 5, 6, & 7 all supported (ZFS is the sole implementation of RAID 7.)
Software RAID with variable width stripes to protect against the "write hole" common in software RAID solutions.
Native storage tiering and caching
Copy-on-Write Snapshots (CoW)
Compression
Deduplication
Advanced RAID reconstruction using knowledge of overlying file structure.
Send / Receive functionality to stream an entire filesystem to another system.
Incredible 128bit scale allowing for zettabytes of storage in a single file system.

Using ZFS can be complex as it includes so many features and so much functionality, and because it is generally so different from traditional POSIX filesystems. Using ZFS effectively generally requires specialty knowledge separate from what is already expected for the operating system. It is currently used rarely, but does have a solid place and is often a consideration.

Part of a series on Linux Systems Administration by Scott Alan Miller

scottalanmiller

Placeholder