Software defined storage, SDS, seems to be the marketing phrase these days. Wikipedia calls it a marketing theme for promoting storage technologies that is lacking consensus. Wikipedia does go on to list some characteristics of SDS:
- Automation with policy-driven storage provisioning – SLAs replace technology details
- Virtual volumes – allowing a more transparent mapping between large volumes and the VM disk images within them, to allow better performance and data management optimizations
- Commodity hardware with storage logic abstracted into a software layer
- Programmability – management interfaces that span traditional storage array products, as a particular definition of separating “control plane” from “data plane”
- Abstraction of the logical storage services and capabilities from the underlying physical storage systems, including techniques such as in-band storage virtualization
- Scale-out architecture
I would say that this provides a fairly good description of what I would envision as SDS, except for the point on commodity hardware with storage logic abstracted into a software layer. This has led some analysts to predict that storage functions and intelligence will shift to the server and purchasing will shift to commoditized hardware. On the contrary, storage hardware must become more intelligent to include the storage logic, which can be initiated or automated by events or polices that are defined by application or server software. This has already happened with VMware, as evidenced by the company’s motivation to off load server functions to storage systems through APIs like VAAI and VASA in order for VMware to be more efficient in supporting virtual machines. This requires more intelligence in the storage to support these APIs and provide visibility through vCenter.
In addition to scale-out architecture, I would add the requirement to scale up as a characteristic of SDS. Virtual servers that host many virtual machines are today’s mainframes. Even though they may run on x86 blades and use the SCSI protocol, their I/O workload and demand for availability scales up with the addition of each virtual machine. Scaling out by adding another storage node in a cluster does not help the storage ports that are supporting the virtual server. You need the ability to scale up the power and capacity behind those ports by adding processors, cache, and storage dynamically as the workload increases. This takes intelligence in the storage system. Scaling out is a commodity play that does not meet the demands of virtual servers.
The goal of SDS should not be to commoditize storage in order to reduce the cost of storage, the goal should be to automate storage services to increase the return on total IT assets. This automation should include:
- Retention services
- Provisioning services
- Performance optimization services
- Availability and reliability services
- Capacity optimization services
The unique characteristic of storage, which sets it apart from servers and networks, is its need to preserve the data. Retention services include the requirements for security, privacy, immutability, audit trail, encryption and shredding. Provisioning services include dynamic provisioning, formatting, non-disruptive migration, and connectivity. Performance optimization services, includes the ability to not only record, search and retrieve data to meet application requirements, but also to provision and refresh the storage infrastructure to meet business requirements. Availability and reliability requires data protection, snapshots, DR, pre-emptive maintenance and quality of service. Capacity optimization or efficiency requires thin provisioning, dynamic provisioning, a pool of shared resources, and the elimination of storage silos for different applications.
These are just a few examples of services that need to be automated to reduce OPEX and CAPEX and increase return on IT assets, which includes, storage, server, network, people and environmental assets. In order to achieve this automation the storage system must be designed to do this from the inside out. You cannot do this with software that is running outside of the storage system. Software can request these services or set policies to initiate them, but the services need to be executed by the storage system so that the server and host software can do what they do best, serve the application and its clients.
My colleague Greg Knieriemen sums it up best.
“The value for the consumer of enterprise storage is in automation. As the role of the IT admin evolves, they want to spend less time managing their IT resources. As I see it, there are two significant elements to software defined storage as the industry perceives it today:
- The abstraction of physical storage resources
- SLA/policy driven management of those abstracted storage resources to deliver a defined QoS automatically
The capabilities within Hitachi Virtual Storage Platform (VSP) (and Hitachi Unified Storage VM) give us a significant advantage over most of the other vendors in this regard.
I also think some in the industry have confused the abstraction and commoditization of storage resources to mean we’ll be using magical storage white boxes as storage components. While the storage resource logic may be abstracted and managed by software, the storage hardware itself still retains unique value in delivering QoS.”
Storage has always been about software and that software is growing as more functionality is pushed down into the storage through APIs like VMware VAAI, adapters and providers like SQL Server Remote Blob Store Provider, and Oracle plugins for RMAN integration. The intelligence in storage systems is in millions of lines of code — called microcode. The requisites for SDS called out in the Wikipedia post: virtual volumes, abstraction of storage logic, programmability – all the functionality prerequisite to automation of data and storage services are present in VSP, HUS VM, Hitachi Content Platform (HCP), and Hitachi NAS Platform (HNAS). Hitachi Unified Compute Platform (UCP) extends this automation to a software defined data center with the orchestrated management of server, storage, and network through the vCenter interface for VMware.