Recommended Viewing : The BlueKai Playbook for Scaling to 10 Trillion Transaction a Month

Good talk on delivering a highly scalable solution. Ted Wallace, VP of Data Delivery at BlueKai discusses how BlueKai scales to 10 trillion data transactions per month.  BlueKai provides data-driven marketing and as a result needs highly scalable solutions. Ted Wallace discussed how they do this. He provides some good details – they use Aerospike to get the high database performance – average read/write response times are between 1 – 2 ms. Six Aerospike clusters with 6 to 10 server in each of three geographically located data centers. They use standard Linux hardware with four Intel 800G SSDs in each and 128 GB to 256 GB of RAM. Lots more details in the talk.  Select the image to go to the talk.

Aerospike_video

Recommended Use-Case : Snapdeal, India’s Largest Online Marketplace, Using Aerospike

Nice use-case write-up on Snapdeal, India’s largest online marketplace, using Aerospike (NoSQL) databases to power its services.  Snapdeal looked at MongoDB, Couchbase, Redis, Terracota Big Memory Max, Amazon’s DynoDB and Aerospike.  The chose Aerospike for a number of reasons. The rationale and results can be found in a nice use-case write-up.  Among them the in-memory Aerospike database maintained sub-millisecond latency on Amazon’s EC2 while managing 100 million objects, predictable low latency with 95-99% of transactions completing within 10 milliseconds, low maintenance, full replication across EC2 servers and more.

snapdealInteresting read.  If you want to learn more about the architecture and choices made by Snapdeal they are presenting on this topic on Wednesday, March 12th.

aerospikesnt02

 

 

Web Talk : Getting The Most Out Of Your Flash/SSDs For NoSQL DB

Aerospike, the highest performing NoSQL database that I know of, is offering a talk on how to get the best performance by using Flash/SSDs. Aerospike is specifically tuned to delivering high performance from flash/SSDs and they have best practices around this. One of the most important things you can do to improve the performance of your flash/SSDs with Aerospike is to properly prepare them. This Webinar will go through how to select, test, and prepare the drives so that you will get the best performance and lifetime out of them. This web talk is scheduled for February 11th at 10 am.

aerospikebt01

 


gotostorageGo to more posts on storage and flash storage at http://digitalcld.com/cld/category/storage.


 

Architecture : SSD-Based Solutions Show Advantages In the NoSQL DB Tier (Video)

Today we look at the NoSQL database tier.  Some of this is taken from notes from a work-in-progressAn Introduction to Using High-Performance Flash-Based Storage in Constructing High Volume Transaction Architectures – A Manager’s Guide to Selecting Flash Storage.  This is not a complete look at Big Data, rather a partial look at some of the things Aerospike, one of the more interesting NoSQL databases, is doing. 

Aerospike and the NoSQL Database Tier.  An alternative or in addition to the relational database tier, there is a NoSQL database tier. With the arrival in recent years of Big Data architectures, new elements of a new architecture for dealing with both structured and unstructured data has arrived and with it some databases, like Aerospike, offer an extreme high performance solution in transaction-oriented environments.  Quite a bit different from typical Hadoop implementations as one of Aerospike’s real differentiators is that Aerospike was built as an in-memory database. Traditionally, in the past, this tier we have seen a number of spinning disks.  However, in the past few years, especially with the need for real-time information there has been a move to SSDs and PCIe-based flash cards.  Using Aerospike’s NoSQL database provides a means to get those high performance results. It is built to be run in-memory or in-flash. A partial glimpse into an architecture.  It is built to run on relatively low cost clustered hardware with either lots of memory and/or flash storage.  It supports ACID properties and as a NoSQL database also leverages a key-value store. If we look at an example in this tier – you can see the an example architecture where various transactions are occurring within applications and Aerospike interacts with these. It should be noted that with App tier, Aerospike uses a Smart Client to communicate to the Aerospike cluster.

nosqlarch

Of course, the producing/consuming sources may vary dramatically – from applications, web services, hadoop clusters, mobile devices, weblogs, marketing data repositories and many more.   Aerospike  is a best-of-breed of the NoSQL databases. You can see an example of a typical deployment is (from the Aerospike presentation below) :

aerospike100

And some of the Aerospike server deployments :

aerospike101

Aerospike offers support for the ACID standard and support for a high performance, clustered architecture.

aero_ssd

Of course, there are other databases such as MongoDB, Cassandra and HBase to name a few. You may choose  to use NoSQL database over relational databases. It depends wholly on what you are doing. The NoSQL database tier’s storage on these servers can use SSDs, flash PCIe cards and flash arrays.  Traditionally this tier has adopted a “share-nothing” philosophy using traditional spinning disks, SSDs or flash PCIe cards. Up to recently, flash arrays seemed like not only over-kill but also seemingly moving against the grain of the “share-nothing” philosophy.  SSDs and cards, like Micron’s P320h offer excellent performance and offer a price/performance advantage over arrays.  As prices of flash drops flash arrays are becoming a consideration in this tier and there are a number of recent deployments leveraging flash arrays for the NoSQL DB tier.  Recently, Aerospike tested Micron’s P320h (SLC SSD) PCIe card.  It “blew away the competition” according to people doing the testing.  You can read more here:

aero_ssd

More information on the P320h :

tomshwp320h

It should be noted that there are two versions of this from Micron. Micron offers a 2.5″ Flash PCIe form-factor which is hot-swappable.  You can read more here :

digimicron

It should be noted that competitors are not standing still and Virident, Fusion-IO and others have and are coming out with new cards that are worth looking at.

To understand what Aerospike is doing it is worth watching this video :

Aerospike_video

If you want to learn more, it is worth visiting Aerospike’s BrightTalk site

brighttalk_aerospike

 


gotostorageGo to more posts on storage and flash storage at http://digitalcld.com/cld/category/storage.