New NVMe spec brings new support for hard drives

The new NVM Express 2.0 has been released and with it a surprise: The non-volatile memory express protocol—best known for handling SSD speeds—is now offering full-blown support for traditional hard-disk drives.

This is quite unexpected because SSDs are orders of magnitude faster than traditional HDDs.

The first flash-based SSDs used SATA/SAS physical interfaces borrowed from existing hard drive-based enterprise server/ storage systems. However, none of these interfaces and protocols were designed for high-speed storage media and the SATA/SAS bus became a bottleneck for the much faster SSD.

The initial fix was SSDs on a PCI Express (PCIe) card. While much faster, PCIe was also proprietary and meant as a point-to-point transfer. A PCIe SSD in one server could not be directly accessed by any other server, it had to go through the system bus, adding all kinds of latency.

NVMe was soon developed, offering a highly scalable storage protocol that connects the host to the memory subsystem. This made SSDs accessible to computers/servers beyond the one where they were physically located. NVMe wasn’t meant as a replacement for PCIe—it ran on top of it—using PCIe’s much greater speed than SATA.

So it’s a big surprise that NVMe 2.0 added support for “rotating media,” as the spec puts it. A current 7200-r.p.m. hard drive cannot fully saturate current SATA 3.0 connections, let alone PCIe Gen 3, which is twice as fast as SATA 3. Now PCIe Gen 4 is coming to market, with double the throughput of Gen 3.

HDD won’t die anytime soon

Even as we move to a SSD world, hard drives have their place, namely, large capacity.  Sure there are 8TB SSD. They are also insanely expensive. HDDs are reaching the 20TB barrier for a lot less than an equivalent SSD, so it’s unlikely BackBlaze or other cloud-storage firms will move quickly to SSDs.

And while HDDs are slow, don’t count that part out, either. For example, Seagate’s Mach.2 HDD uses two separate drive-head mechanics do double read/write activity. Seagate recently announced that its Mach.2 hard drives can reach up to 524MB/s of I/O, which is SATA SSD speed. Mach.2 is sampling to key customers, so general availability has to be approaching.

The SATA spec has not been updated in 12 years, which in the tech industry qualifies it as abandoned. By moving HDDs to PCIe/NVMe, the SATA and SAS buses can be effectively removed from the motherboard and free up space. That won’t happen overnight, of course, but it could and should happen.

Storage vendors argue back and forth on whether hybrid-array or all-flash will dominate the data center. The NVMe 2.0 spec appears to be preparing for either eventuality.