Why Lego Makes Sense in Toys, Software, Servers and Storage

Update : For my most recent post on the new class of flash arrays that fully support deduplication and a full range of storage features, Recommendations for All-Flash Storage Arrays; Aiming Beyond Simply IOPS and Low Latency.

Legos are remarkable. You can almost build anything with them, however, that is really not the only remarkable aspect of legos. The underlying lesson is that from very small inter-legooperable parts you can efficiently build very big things. The lego approach has found itself into software practices and as well to servers. Good software design emphasizes small generic functions that can be used extensively and flexibly by other functions. On the server front – while very large servers still exist and serve many useful functions – the majority of data centers are building clouds from a large number of  small servers. These servers are 1RU or 2 RU at most.  And yes we see the larger 4RU and greater sized servers, but they are infrequent and very specific in function. For example, If you check out Hadoop in data centers, you will discover that they run on these smaller servers in clustered groups. If you check out database-as-a-service – increasingly they run on an architecture of small servers and small storage units. Software such as Mesos and Chef make it easier to provision, manage and create clouds populated by hundreds and thousands of small servers. The sweet spot is the two processor Intel server.  Oracle’s Exadata architecture is a database poster-child for this approach.  A popular version of Exadata is comprised of 14 storage servers and 8 database servers. Basically going from an eighth rack to a full rack approaches it in a lego-like manner. The reality is that the combination of operating systems software, application software and small servers have been combined to offer architectures that can do very big things. Increasingly, software plays a ever bigger role in everything. In storage, the lego approach is a big win for enterprise and cloud providers because it offers extreme scale-out and flexibility when combined with software, storage features and fast underlying hardware. Yes, you could get a 3RU or 18RU array architecture – but the win is in building architectures with smaller 1RU arrays that perform elastically, resiliently and flexibly like legos and construct large storage architectures and scale-out storage spaces that offer much more than what is possible from a single large storage array.  When I say more – I don’t necessarily mean IOPS – all these flash storage arrays provide more IOPS than are needed and arguably hybrid arrays also provide excellent performance.  More important – the architects can grow their architecture in an as-needed basis. They don’t have to buy huge blocks of expensive flash storage if they don’t need them. Let’s look at an example of the lego model.  I see SolidFire as employing a lego-like model. A starter configuration include five 1RU storage nodes with complete data protection across the storage nodes. Key – data protection doesn’t just happen at the node or array level, it takes place at the cluster level.  A good example of a lego-like storage model is SolidFire’s all-flash storage nodes, because of the operating system software each node offers features that become more powerful as they are aggregated.  First, you can add new nodes and aggregate the flash storage into a single view and single pool. Five, Six, Seven… ten … one hundred nodes can be pulled together to create large storage pools if desired. This is key you can add new nodes as demand dictates resulting in immediate capacity and increased performance. You add these nodes (or remove them) with no downtime and with minimal performance effects. This scale-out behavior allows you to go from five to one hundred nodes. SolidFire’s petabyte scale-out goes well beyond 280 TB of some competing systems, it goes to 3.4 PB. Second, you can guarantee performance levels because SolidFire offers quality-of-service features.  Third, each node can be upgraded dedupsfnon-disruptively. Fourth, all the usual data reduction suspects are available – thin provisioning, deduplication and compression are offered. Fifth, as in most clouds, automation is a key – REST APIs and an advanced user interface allows automation of the storage cloud. Sixth, failure happens and these node provide for redundancy so data is not lost. Seventh, real time replication is offered to cope with potential disaster recovery scenarios.  Eighth, these nodes come with snapshot and recovery software. Ninth, complete high availability that provides high availability in a distributed manner.   Tenth, these lego nodes offer encryption.  Finally, and importantedly, you can mix the older and newer storage nodes in the same pool.  This lego like model fits what has happened with servers and how enterprise and cloud designs like new engineered systems are moving to.  SolidFire is only one example of a flash storage vendor that is doing approaching this right. There are others that are adopting this lego model.

In the end, most clouds and many enterprises care about the up-front costs, floor and rack space consumed, non-disruptive upgrades feature, power & cooling costs, performance, latency, capacity, data resiliency and high availability – and importantedly – providing enterprise features like extreme scale-out storage pools, quality of service tunability, deduplication and an ability to add arrays or nodes incrementally without incurring huge costs.  The lego approach to storage delivers on this.

gotostorageGo to more posts on storage and flash storage at http://digitalcld.com/cld/category/storage.