The technology backbone of enterprise storage, RAID, is about to die. This is going to create a tectonic shift like the storage market has not seen in 30 years. It will be a shift as significant as virtualization was to computing or even smartphones in mobile.
The indicators that this will happen are there for all to see. As with most disruptions, failure to compete will be from denial and not lack of awareness. I have blogged about changes coming in enterprise storage for almost 2 years and we are just now starting to see the cracks in the foundation.
To summarize: Here are the four reasons why I believe RAID will become extinct:
- It is too expensive due to specialized hardware (like mirrored cache and redundant controllers) and software requirements. Any capacity cost savings from RAID is minimal compared with the additional costs.
- Repair/rebuild times take too much time and are too complicated.
- Transactional complexities (like read/modify/write) make it very inefficient when using relatively slow media (like disks)
- RAID Array systems are “tightly coupled” which make them difficult to engineer and not hyper-scalable. I talked about that specifically in a previous post.
I believe that this problem is best solved with a combination of new technologies:
- Simple data replication and erasure coding across independent storage nodes. Loosely coupled, hyper scale systems. Use erasure coding if it has to be super-cheap.
- Combinatorial data management that lets you use the same data copies for production, test, DR, backup. This lets the same copies serve multiple needs.
- Asymmetric data protection (e.g. one data copy on flash for access speed and backup copy/copies on disk)
- Inline data dedupe/single-instancing
- Erasure coding for archive (with no prescribed rebuilds)
- Data in place management - no separate backups - DR integrated as a part of disk fault protection.
Using this new approach, enterprise storage will become better in almost every respect. Systems will be more reliable, performant and scalable while at the same time be much lower cost.
In part 2, I will look some specific TCO numbers.