More Cattle, Fewer Pets Reaching the Tipping Point in High-cost Storage Disruption
Brent Compton, Director, Storage Solutions, Red Hat
Brent Compton, Director, Storage Solutions, Red Hat
The Open Stack industry coined the metaphor 'pets and cattle.' Legacy datacenters have pets – large, expensive proprietary appliances which are given friendly names, coddled by high-cost administrators and meant not to fail. Cloud datacenters have cattle – smaller, inexpensive commodity servers which inherit their purpose through software, are managed as a herd and are raised to be replaced.
First to fall were high-cost proprietary RISC server pets running proprietary Unix. These big iron systems were deemed ‘too big to fail,’ and were named, coddled and fed large sums of maintenance fees to keep them from going down. In the last decade, we witnessed the systematic commercialization of this market by x86 servers running mission-critical Linux and hypervisor variants. Competition among the x86 server vendors brought a vast array of compute servers driving down costs and increas-ing choice. If there is any question that this battle has already been fought and won, look-up Unix vs. Linux server shipments by volume and revenue from your favorite analyst. Cattle have replaced pets for compute servers. Now falling are many of the high-cost proprietary SAN and NAS storage appliances. This disruption can be seen materializing in the public financial statements of storage vendors. Bellwether SAN vendor, EMC, reported its high-end storage business suffered a decline of (7%) for the quarter when compared to the same quarter last year, as reported in their Nov 2014 10-Q filing. NetApp, a mainstay NAS vendor, also reported lower revenues (1%) in their Nov 2014 10-Q filing for the first half of its fiscal year compared to the same period last year. Is this because the storage market as a whole is shrinking? Not so. We all know that the social-mobile-IoT juggernaut is filling our industry datacenters with mounds of unstructured data… petabytes from chatty machines, teenage selfies, toddler video clips, and endless on-line entertainment. No, the storage market is not shrinking. But it is indeed shifting from pets to cattle. In 2013, analyst firm IDC projected a 3.2% decline in scale-up file storage servers between 2012-2017, while scale-out file storage was forecasted to grow by 16% and scale-out object storage by 27%.Scale-up storage appliances are pets. Most estimates indicate that their numbers are declining.
Scale-out storage servers are cattle. Most estimates show that their numbers are growing.
What causes the tipping point in this transition from pets to cattle? Because it’s fundamentally the disaggregation of proprietary appliances, it requires evolution of the individual constituent parts -- de-coupling and maturing the software and then commoditizing the servers. With compute servers, it was the maturation of Linux software, led by Red Hat and the community, which elevated this working-class heir to the throne vacated by UNIX. But alone, this was not enough. It took comparable maturation of the x86 server from a cushy workgroup server role to meeting the most stringent operating requirements of telecom carriers and stock exchange trading houses. When both the operating system software and commodity servers were ready for one another, a market shift occurred.
The same pattern now appears to be occurring in the storage industry. We have front-row seats to the same play, different act. Commoditization opening new doors of innovation. And innovation opening new doors of commoditization.
The storage innovation that solutions such as Hadoop’s HDFS, Cassandra, Splunk, Red Hat’s Ceph and Gluster, and Scality have in common is to consume data on lots of commodity servers. Cattle expected to fail and be replaced. The following quip from Cassandra’s on-line documentation could describe the storage design of any one of these software systems. “Cassandra is designed to handle big data workloads across multiple nodes with no single point of failure. Its architecture is based on the understanding that system and hardware failures can and do occur.” This maturation of storage software, weaning itself from decades of reliance on big, expensive SAN/NAS storage servers, began the disruption.
And now, seeing this storage software breaking free, more server vendors are clamoring to meet the new greenfield opportunity for x86 servers.
First came the high-capacity 2U servers, offering 12-16x 3.5” drives within a twin-socket server. Servers such as the Dell R730xd, HP DL380, Cisco C24oM4 and Lenovo System x3650M5. These systems balanced performance, cost, and capacity, illustrated by 2-socket, 16-core servers with 128GB RAM and dual 10GbE adaptors. That’s 1.6 petabytes in a rack, with room left for a 2U switch.
Now, breaking the field wide open for high-capacity object and file storage are the 4U chassis offering 50-72x 3.5” drives driven by 1-2x twin-socket servers including servers such as the HP ProLiant SL4540, Cisco C3160, and Supermicro 6047R-E1R72L2K. These systems are optimized for cost, capacity and throughput, illustrated by 2x 2-socket, 16-core servers with 128GB RAM and dual 10GbE adaptors for 300TB in 4 rack-units. Or optimized for cold-storage archive, illustrated by 1x 2-socket 16-core servers with 128GB RAM and dual 10GbE adaptors for a whopping 360TB in 4 rack-units. That’s more than 3 petabytes in rack.
The constituent parts of storage appliances, namely software and x86 servers, are readying themselves to reach the tipping point in the disruption of high-priced, storage appliance pets.
If we were prone to buzzwords, we’d call this Act 2 of the software-defined datacenter play. Act 3? Converting high-cost networking appliance pets into cattle. But let’s enjoy the storage disaggregation show first.
