We didn’t find in 2018, as well as in 2017, revolutionizing storage technologies even if there will be a lot of enhancements in the current $100 billion+ storage industry. Look to several start-ups, especially in software.
There is a strong demand. The price will finally going down this year with new fabs, especially in China, and consequently they will continue to faster replace HDDs from PCs and notebooks, and up to high-end storage systems. The massive move from 2D to 3D NAND will allow bigger capacities and lower prices from all flash chip makers (Intel/Micron, Powerchip, Samsung, SK Hynix, Toshiba, Western Digital/SanDisk). USB keys and portable phones will also benefit. As well the fast PCIe interface with NVMe, with higher bandwidth and lower latency, will flood the market to progressively taking the place of 6Gb SATA and 6Gb/12Gb SAS connections. High-priced but extremely fast3D XPoint/Optane from Intel/Micron will begin to be adopted.
Their areal density is extremely difficult to increase. Now the only way is to add more disks into drives filled with helium inside the units to reach now 14TB in 3.5-inch form factor and with different technologies: Toshiba Conventional Magnetic Recording, WD SMR. Next step will be reached with Seagate HAMR supposed to arrive this year, and MAMR from WD the next one. Hybrid HDDs are going to disappear. But globally the market is going to declining in units shipped as it’s the case since 2014. Prices per gigabyte of HDDs and SSDs continue to be in favor of the mechanical devices and all of them were decreasing these past years, more rapidly for SSDs because of the end of supply tightness of NAND chips.
The market will not rebound with the arrival of LTO-8, at 12TB native, that became a proprietary technology with only one drive maker (IBM) and very few Japanese cartridge manufacturers (maybe also only one, FujiFilm, and no one confirming commitment officially).
With limited capacity, 300GB on one disc on Blu-ray and shortly 500GB, it’s nothing compared to LTO-8, and vendors are in limited number, essentially Panasonic and Sony..
The industry will jump on the bandwagon of PCIe/NVMe/NVMe-oF and USB Type C, this later inaugurated by Apple will become the main interface for all notebooks, PCs and even smartphones for its fast speed. But Thunderbolt hook on with 40Gb/s bandwidth. SAS and SATA will decline like FC even with new 32Gb/s speed but with less and less offerings.
Promising markets and applications:
- The storage industry is hoping to increase sales with the adoption of General Data Protection Regulation, a compliance from EU with 28 countries to go into effect next May 25 and necessary also for companies doing business with European organizations.
- 25/50/100GbE will gain adoption over 10GbE and Mellanox IB and Intel OmniPath connection will be adopted for HPC only.
- Convergence and more than that hyper-convergence, even for secondary storage, continue to be two major trends. It’s obvious when you see the huge growth of public company like Nutanix.
- All-flash arrays continue to be a growth market in competition with flash as a software-defined tier. Hybrid configurations in competition with a mix of SSDs and HDDs will suffer with decreasing prices of flash.
- That’s the end of monolithic storage subsystems replaced by lower-priced scale-out architecture comprising several nodes (servers with HDDs or SSDs that can be add on demand) on a fast GbE network with, on top of that, a software managing the entire system and excluding expansive disk array controllers as one or more copies of data are split on different nodes.
- Traditional storage administrator is going to be replaced by DevOps.
- Growing usage of data management via metadata is expected.
- Among the most promising market for storage continues to be the video industry including surveillance and post-production with the increase of image definitions needing huge storage capacities and archiving capabilities
- Ransomware protection will be more and more added to backup software.
- IoT, market evaluated at $772 billion in 2018 by IDC, is exploding and it needs storage.
- Continue to look at what happens in storage following the Dell EMC merger as, up to now, it was not a hit with big competitor NetApp in better shape.
Object Storage and S3
Perfectly illustrated in 2017 with several vendors offering an S3 API, object storage will suffer from this paradox. Object storage got fewer impacts in the industry but S3 will continue to raise and attract more and more users. Kinetic and similar approaches, besides their technology attractiveness, got stuck to finally almost disappear. The paradox comes from the fact than S3 APIs are offered by none object storage vendors. And it’s easier to build a S3 API on top on a file system. And to confirm that object storage vendors will continue to morph to S3 storage technology providers reducing their differentiators once again. Vendors to follow in 2018: Cloudian and Minio.
2017 was a key year for this storage technology and NAS and parallel file storage got a lot of traction. We anticipate a more important presence by such vendors leveraging flash drives and NVMe connectivity to offer high performance and high scalable file storage solutions. Some of them delivers super fast NFS and SMB operations and for a smaller group even more with a parallel file system extension. We also predict a reduced adoption for Spectrum Scale-based solutions. Vendors to follow in 2018: Avere Systems, Elastifile, Panasas, Qumulo, Rozo Systems and WekaIO.
Both object storage and file storage vendors confirms the SDS adoption wave and 2018 will continue in that direction supported by users’ wish to be hardware independent and able to run bare metal, within a virtualized or containerized environment or in the cloud as a pure software stack. But 2017 had a few dead end stories such Coho Data or Formation Data Systems, and we expect to see more sad news as the number of companies is too large.
We detect different interesting directions with cloud-to-cloud backup and distributed applications protection. The large adoption to SaaS-based applications such Salesforce, GSuite or Office365 will confirm the need for a dedicated data protection strategy. Vendors to follow in 2018: Datto Backupify, Odaseva, OwnBackup, Spanning and SpinBackup. NoSQL databases such MongoDB, Cassandra and other distributed applications built on Hadoop and other big data platforms have proven their role in the business especially coupled with the cloud. New backup vendors have emerged and will continue to accelerate in 2018. Vendors to follow in 2018: Datos IO and Imanis Data. Both segments confirmed the interest for continuous data protection aka CDP, a key fundamental aspect of data protection at scale.
In the past, a not so attractive part of the storage industry, secondary storage, got more and more attention in 2017. The main reason is the data volume continues to explode, we don’t see reduction on the horizon, the need to keep data for business reason and compliance and the gold mine it represents. We wish the market to finally deliver an efficient storage approach for the best ROI for such platforms, it means data protection with erasure coding, optimized capacity with data reduction and some energy rationalization to reduce power consumption. In other words, do users will consider storage platforms without these 3 core features to store data for 10 years? Do they let the platform powered up for 10 years? We recommend users to ask that question about these 3 core features to vendors when they’re looking for secondary storage platforms. Vendors to follow in 2018: Cohesity, Rubrik and Spectra Logic.
AWS, Azure and GCP are clearly the top 3 players with IBM, Oracle and Alibaba following them aggressively. VMware and HPE have made real bad bets on cloud and even VMware recently sold its vCloud Air to OVH recognizing the domination of the top 3 players. HPE has demonstrated its inability to build, develop and sustain a public cloud business with a pretty complex story with OpenStack and Helion. S3 storage will continue to lead the market with growing offering from Azure and GCP and of course their respective in-cloud storage for computing instances. We anticipate also a continuing wave towards a multi-cloud approach with workload migration, copy, backup or DR services, but also tiering and archiving to make on-prem and cloud a true storage continuum. Vendors to follow in 2018: Hedvig, Komprise, CloudEndure, CloudVelox and VeloStrata.
To make the IT infrastructure even more efficient both in term of ROI and resources utilization, we started to see some technology initiatives around composable infrastructure. The idea is to dynamically in real-time connect IT resources together to build a on-prem comprehensive virtual system. Imagine a rack with blade servers, some switches and JBOF plus this intelligence and you’re able to build a gigantic – scale-up and scale-out – system. Vendors to follow in 2018: DriveScale and Liqid.
Artificial Intelligence (AI) and Machine Learning (ML)
It’s not a scoop, AI, ML and other related topics are more and more present in our life. This is true also in the storage industry as many vendors have started to add logic within their systems to learn the environment, recommend some actions, operate tasks to finally improve the service and simplify users’ life, management, control and operations. Of course, we’ll see even more of these in 2018 for our benefits, selection will be tough but experiment will be key.
Happy New Year to our numerous loyal readers!
Jean-Jacques Maleval and Philippe Nicolas