In this post we will see how there are a number of necessary and some would say mandatory features that you should be making sure your flash- or SSD-based array vendor offers. This post will also highlight the use of quality-of-service features by one cloud company. In today’s post we will not focus too much on whether we are using VMware, KVM or some other virtualization technology (that can wait for another day and another post).
If you only pay attention to IOPS you will end up with, well, IOPS. Now you may not be able to allocate these IOPS in a granular way, or upgrade the hardware that gives you these IOPS without a major disruption or allow a reduction of duplicate copies of files and images or be able to non-disruptively grow these IOPS in one or more big clusters. But, you will have IOPS.
IOPS and low latency is only a fraction of the problem. Whether it is SSD-based systems or flash module-based systems you will get excellent performance. That’s no longer the issue. Next-generation flash storage companies have emerged aimed squarely at virtualization and at the cloud. Let’s look at a few mandatory features that you should make sure your flash-storage hardware provider supports :
- non-disruptive upgrade (NDU) – you want to be able to upgrade your array without taking it down and with minimal performance impact.
- quality-of-service. Anyone that has supported virtualized environments should make sure that all those IOPS that are available can be managed as a resource. Really, you want resource management in the same way you want resource management of CPU and memory. You may even want to monetize those IOPS with guaranteed IOPS allocation. Imagine specifying the IOPS you need on a per LUN/volume basis and turn around and connect it to an account.
- native data deduplication and compression – is a way of saving expensive flash storage. Why duplicate the same VM image, over and over again. Let the flash array do this under-the-covers and reduce the duplicates and storage required.
- thin-provisioning. You could thick-provision your virtualized guest but that would mean that you need as much storage as you allocate. Or you could allocate storage to your guests on as-needed basis.
- Non-Disruptive Scale-out Clusters. Why in the world would you buy expensive flash storage arrays that simply creates loosely connected islands of data and that you don’t have a way to natively cluster them ? Why would you buy a solution that you couldn’t grow into one large namespace. Imagine five array nodes that cluster and can be treated as one. Now imagine being able to grow that cluster – if that is what you want. You are building a large, highly multi-tenant, heavily virtualized cloud why would you want small data islands for that cloud ? Yes. There are times you do – but most of the time : Not.
- Snapshots & Clones.
Given the choice between building a cloud with or without these- it’s pretty clear that these features are not really optional. Let’s start with one vendor first – SolidFire. They target virtualization and the cloud and do a very good job of it. For example, one hosting/cloud provider saw “during maximum IOPS load” 2ms latency, during normal load they were seeing sub-1ms latency. Consider this performance coupled to the above features. Consider that some flash vendor “ferraris” don’t have most of those features.
There are a number of other interesting vendors out there – Nutanix, Nimble and Tintri they are targeting virtualization and the cloud in a serious way. We will cover some of these in a future post. Check them out as well, they are doing some great work.
A visual view of what SolidFire is providing with regards to quality-of-service :
You can watch that companies are using it to manage and monetize guaranteed IOPS in virtualized environment – here is a discussion about what one company, Crucial, is building :
Here is a bit more on how they do it – which shows how Crucial uses the OnApp and SolidFire integration to create and manage the virtual machines. It shows you how a virtual machine can be created and customized including the option to guarantee IOPS :
In our next post we will continue with other considerations in building out a highly virtualized, flash-based cloud infrastructure.
Go to more posts on storage and flash storage blogs at http://digitalcld.com/cld/category/storage.