Storage considerations for Hybrid IT

More data is generated daily than ever, and it has to go someplace. Here are some things to consider when evaluating storage needs.

The technology world is both ever-changing and constant. Some technologies have endured for decades, while others die off rapidly. The x86 processor architecture is nearly 40 years old, as is Ethernet networking and the Sendmail protocol.

What keeps any technology alive is evolution. After the revolution (the CPU’s invention) comes an evolution (its growth over generations). The Xeon x86 of today is clearly not the same as the 8086 chip Intel released in 1980, for example.

Storage is in much the same boat, particularly in enterprise computing. Spinning media is on its way out in favor of NAND flash-based storage. But don’t count the hard disk out just yet. It has its own tricks.

Flash will replace disk—but perhaps not immediately

With flash storage coming down in price, it’s becoming more commonplace as a replacement for hard disks. Everybody is putting data requiring fast access on flash, says Jim Handy, principal analyst with Objective Analysis, which follows the storage market. Businesses are using hard disks mostly for backup and cold storage.

But adoption is uneven right now. “It depends on who you talk to,” Handy notes, citing one large online vendor in China doing everything on flash, both fast access and cold storage. “I heard a speaker from a large financial institution say a year ago that every time they decommission a hard drive, they replace it with [a solid-state drive]. It’s because the total cost of ownership is lower than with hard drives.”

But that online vendor is an exception, not the rule, Handy adds. Over the long term, he says, flash will become a layer of storage in the system, but other hard drive-based layers will stick around because they are still cheaper than the next faster thing.

Gavin Cohen, vice president of product and solutions marketing at Hewlett Packard Enterprise (and formerly with flash storage vendor Nimble Storage before HPE acquired it) sees flash arrays being adopted for the majority of primary workloads, particularly those requiring performance. He also sees hybrid storage (a combination of flash and disk) taking on non-performance-sensitive primary and secondary workloads.

Flash can match hard disk for capacity, Handy says, but the price keeps it from taking over, since it costs about 10 times more per gigabyte.

Slower spinning drives—the 5,400 and 7,200 rpm drives that make up the bulk of the market—are still selling well. It’s the fast enterprise drives, which spin at 10,000 or 15,000 rpm, that are threatened by SSDs.

Traditionally, 10,000 and 15,000 rpm drives have been used for hot storage, which is data storage that is most frequently accessed by applications. These drives spin faster for faster data access but also generate much more heat, use more power, cost much more per gigabyte, and break more often than 7,200 rpm drives.

Yet, 10,000 and 15,000 rpm drives have not gone away, Handy says, and Western Digital and Seagate, the two main hard drive vendors on the market, are defending that market strenuously.

But the trend is clearly away from the hard drives. According to Cohen, 10,000 rpm and 15,000 rpm drives are in decline. “It gets to the point where flash becomes a no-brainer. Even if disk is cheaper for your primary apps, flash is now cheap enough for this purpose. Last year’s disk budget might buy this year’s flash, so why put your primary apps on disk?” he says.

Paul Sabin, senior network and infrastructure manager at law firm Baker Botts, puts it quite succinctly: “I don’t think I’m going to buy another storage array with 10K/15K drives. It will be SSDs for us all the way.”

The next stage

Flash memory can fluctuate, but going off the average price, flash costs around 40 cents per gigabyte, says Craig Stice, senior principal analyst for technology, media, and telecom at IHS Markit, which follows the tech supply chain. Hard disk storage costs around 5 to 10 cents per gigabyte. But by 2020, SSD prices per gigabyte are likely to be in the 10-cent range, he says. “It’s all about price and cost of ownership. The TCO is still an advantage of hard disks in price per gig.”

However, when looking at the overall storage system cost, and factoring in data reduction (such as compression and deduplication), something standard on most flash arrays, the disparity between flash and disk arrays is much smaller.

One major influence on the flash memory market is that all of the vendors are making a process conversion, which means they are changing the way they make memory. Instead of 2D printing, where the cells are written across the top of the silicon, the chips are stacked vertically. Of course, this is done at a microscopic level, so it’s not like the chips will suddenly become thicker.

The move to 3D happened out of necessity, not just because it allowed for greater density. NAND hit a scaling limit; the manufacturing process could not be shrunk below 15 nanometers. So because manufacturers couldn’t compact the cells in 2D space, they had to stack them up.

As Handy explains, 3D stacking is one of the next big waves in flash because of the gains in density. The fact that every maker of NAND flash is doing it is an indicator of how important the manufacturers view the 3D manufacturing process.

All of the NAND flash manufacturers except Samsung have been transitioning from 2D to 3D stacking. As an innovator in the 3D process, Samsung got started early and is done with the transition. Pretty much all Samsung flash products, consumer and enterprise, now use 3D memory.

Any time you go through a technology shift, old capacity has to come offline to be converted, and sometimes the new technology yields aren’t as good, says Stice. So the market suffers a drop in available memory, causing prices to rise. “That should end by the end of the year, and we’ll start to an ease in prices,” he says. “It will bring higher capacities at lower costs. That will all go back into pricing declines everyone has been waiting for.”

You can also look for performance improvements. One recent effort by SSD makers is to get the data on and off the SSD much faster. SSDs first shipped using the SATA interface used in hard disks, which hardly took advantage of the speed and throughput of flash memory. The PCI Express (PCIe) bus, which is many times faster than SATA, has been the interface of choice.

Why does Hybrid IT need Flash Storage? Get ESG's perspective.

But one of the less obvious SSD benefits is that it can cover “the sins of doing terrible coding,” as Baker Bott’s Paul Sabin put it. SSD makes applications that run poorly on a hard disk-based system run better, just because of the speed. “Instead of making [developers] fix their code, I just use flash and it makes the sins of poor coding go away,” he says.

For example, Sabin knew of one corporate application that wasn’t written to support multi-terabyte databases. As a result, it performed poorly. But when running on a flash system, the array never sweats and response time is instantaneous. “It almost becomes irrelevant that the software is not written well. It just screams,” says Sabin.

Flash can speed up many databases for more than the obvious reason, he notes. If a database is running slow, you can split the database and run it on two servers to double the database performance. But at some point, you need to reunite all the data with a SQL join. SSDs help out here by making the process of running the database fast enough that you don’t need to split it.

Future developments

Although SSDs are fast, for applications needing ultra-low latency, connectivity can be a bottleneck. The fix is something called NVMe over fabric. NVMe, which stands for Non-Volatile Memory Express, is the pipeline between flash memory chips and the PCIe slot. It’s designed to take full advantage of the speed of flash memory and the PCI Express bus and flash chips’ ability to transfer much more data simultaneously than a standard hard disk.

NVMe over fabric decreases the latency and also allows servers to share their PCI-based SSD storage. It allows server admins to hook up a whole bunch of storage devices to a bunch of servers and run those servers at NVMe speeds, which is considerably faster even than InfiniBand.

Handy of Objective Analysis predicts the next big technological step is persistent memory. It will be similar to a flash drive in function, but it sits in a memory socket instead of a PCI Express slot. Like flash, the contents of persistent memory storage are still good if power is lost. The memory interface is much faster than PCIe and will allow for near-DRAM-like performance.

Some storage trends aren’t about hardware per se, but about learning where the data is best stored—locally or in the cloud. Sabin of Baker and Botts says many enterprises are moving to cloud-based storage because, as with computational tasks, cloud storage permits elasticity. “When we have a case come in with a huge amount of data, I find myself having to buy an additional shelf of storage to accommodate a new request, especially if it’s a one-time use,” Sabin says. “The cloud is solving the problem of having to buy one-off storage. Now I can be more accurate with where growth will be. Thanks to being able to use the cloud for one-offs, it looks more like a clear, linear growth pattern.”

For Cohen, the next major change in storage is in autonomous storage. This involves pairing predictive analytics (to predict, prevent, and automatically resolve issues) with fast flash storage. This builds on Infosight Predictive Analytics, developed five years ago by Nimble Storage (now part of HPE), which analyzes millions of sensor data points every second. Today, Infosight automatically resolves 86 percent of issues before a customer is even away of a problem. “You buy flash storage so that your applications can run fast and reliably, but even with flash many issues still exist,” says Cohen.

Cohen expects this technology to evolve, and permit the infrastructure to run completely autonomously, with “touchless” management. “The goal is to eliminate the headaches and pain that are normally involved with managing infrastructure,” Cohen says. “We are already working on the foundation for this.” Work has already commenced to extend InfoSight to work with other products, starting with 3PAR.

Lessons for leaders

  • Flash is coming down in price, but that isn’t the only reason making it more appealing. Flash’s durability is comparable to hard disks, and uses far less power.
  • Flash is becoming increasingly faster thanks to advances in memory and interfaces. That makes a difference in enabling real-time or instantaneous business decisions.
  • Once the memory industry completes its transition to 3D stacking, expect a considerable drop in price as capacity ramps up and supply increases. That transition should be done by next year.
  • Pairing flash with predictive analytics can fundamentally improve both the sheer performance of the storage, as well as the entire experience so that problems are automatically avoided or resolved.
  • Poor-performing applications can be greatly improved just by moving them to a flash system, rather than fighting with a vendor or overloaded software development department to improve performance.

Related link:

Protect Your Applications with 3PAR Flash-Integrated and Cloud-Ready Data Protection

This article/content was written by the individual writer identified and does not necessarily reflect the view of Hewlett Packard Enterprise Company.