Datomic : Moving Beyond the Current Generation of Relational and NoSQL Databases

It’s really seldom that you get a chance to see a different thought pattern emerge in database design that literally covers the key advantages of both relational and NoSQL databases and offers a new set of features. Datomic is an aggressive attempt to create a next-generation database that moves us beyond the usual and is built around running in current and future cloud architectures. Let’s start with the some of the features in Datomic. It provides support for ACID transactions, Joins, a sound data model and a logical query language called Datalog. Most of all it breaks with the rigidity of document-based models and relational database models. It also breaks with a model of mutability and a lack of audit data trails. To understand it is worth taking a tour of both it and it’s query language. One surprising aspect that it provides this by using engines like Couchbase, Cassandra,Riak, etc as nothing more than storage engines or services.  Let’s start with the tour  of Datomic :

By deconstructing the current generation of databases and implementing a modern database focused on breaking with designs from decades ago or some of the issues with the newer NoSQL databases (lack of Joins, lack of ACID, etc).   To understand the power of the database you have to explore the power of the datalog query language:

Large, medium and small companies using Datomic.  So you may be interested in the “why”.  In this presentation – developers of the Brazilian NuBank expose four hidden super-powers of Datomic.  In this case, the Clojure programming language is used.

and here is another use-case from Room Key ( joint project of Hilton, Choice, Hyatt, IHT, Marriot, Windom) :

 

More reading :

flying-studios


examples

 

 

Recommended Reading : An Archaeology-Inspired Database

Yoav Rubin shows how a change in a common perspective affects the design and archdb2implementation of a well-studied type of software: a database.  In this excellent examination of how a shift in perspective changes everything, Yoav provides us with a rich examination of what we thought we knew but it turns out that perhaps there are better ways to approach databases. “Database systems are designed to store and query data. This is something that all information workers do; however, the systems themselves were designed by computer scientists. As a result, modern database systems are highly influenced by computer scientists’ definition of what data is, and what can be done with it.

For example, most modern databases implement updates by overwriting old data in-place instead of appending the new data and keeping the old. This mechanism, nicknamed “place-oriented programming” by Rich Hickey, saves storage space but makes it impossible to retrieve the entire history of a particular record. This design decision reflects the computer scientist’s perspective that “history” is less important than the price of its storage.”

As Yoav puts it succintly -If you were to instead ask an archaeologist where the old data can be found, the answer would be “hopefully, it’s just buried underneath”.

Leveraging the richness of the Clojure language and 360 lines of code – we have an “archeology-inspired” database. To read the article, select:

archdb

Presentation/Demo : Couchbase in Containers with Bare-Metal Performance and Triton’s Remarkable Elastic Provisioning

Virtualization is a key aspect of modern computing architectures. Often the choice to go to hardware-level virtualization induces a damage to the performance characteristics of our virtual machines. As I have mentioned before – SmartOS zones and Docker offer a better way to go.  In this presentation, Bryan Cantrill of Joyent provides a rapid-fire and humorous presentation highlighting the history of virtualization and the advantages of running Couchbase containers leveraging Triton, SmartOS and Docker. Also demonstrated is a remarkable display of Triton elasticity – easily creating a number of Couchbase servers on-the-fly all within lightweight virtualized containers running across a datacenter.  What is  offered is a sophisticated,highly scalable, highly performant, elastic solution for a datacenter.

Also can be found here.

It gets better.  If you are interested in deploying the Couchbase containers yourself it is fairly straightforward and you can get the “recipe” from the following blog :

screenshot_605

 

Couchbase Use-Case : LinkedIn

Following on the previous post, today’s post discusses an interesting Couchbase use-case.

Often the questions about a particular technology or product are –

  • who is using it successfully ?
  • how is it being used ?
  • how scalable is it ?
  • does it have good performance (usually within a context) ?

In Couchbase’s case they have a large volume examples of customer use-cases.  One example is LinkedIn.  In the first presentation there is a discussion of how LinkedIn uses Couchbase :

Within this context, an obvious second presentation is a presentation of Couchbase server scalability and performance at LinkedIn:

 

Recommended Reading : Reference Architecture for Oracle on Cisco UCS Server and Tegile

There is a nice reference architecture white paper which provides a lot of details into setting up an architecture that supports Oracle Database deployment and operational details on infrastructure consisting of Cisco UCS servers and Tegile Hybrid Arrays. Select the item below to see it.

screenshot_414In addition, there are a number of new flash arrays – the following brief discusses Oracle and these new arrays :

screenshot_415

 

Recommended Reading : Two Reads on Building Apps in Clojure

In case you missed this comment on Clojure from Adrian Cockroft :

The really cool things I’m seeing being built, some of them are in Go, and the other one is Clojure. A lot of the best programmers and the most productive programmers I know are writing everything in Clojure and swearing by it, and then just producing ridiculously sophisticated things in a very short time.”  From thenewstack.io

 

I bumped into two articles, the first is a nice write-up by a Rails-developer trying out Clojure (and loving it).  He writes 5 recommendations when developing Clojure Web apps.

natescottwest

The second article provides a how-to build single page app with CouchDB and Clojure.

cljrcouchdb

 

 

Recommended: Flash Accelerating Oracle RAC in 16RU & VMware Power Savings with Local Flash

Fusion-io, Dell, and Mellanox have built a very small footprint (16RU), extremely fast, and price-sensitive reference architecture for Oracle RAC 12c. It is very impressive on a number of counts.  It further highlights the move to using local flash storage over remote flash storage arrays.  This solution aims at reducing hardware sprawl and power consumption by achieving higher performance in a smaller footprint.  From the description – “This reference architecture features four Fusion ION Accelerators and four Oracle RAC database nodes all connected through redundant Mellanox SX6036 switches. Each ION Accelerator in this reference architecture uses a Dell PowerEdge R720 with three 56 Gbps Mellanox ConnectX-3 InfiniBand cards and four industry-leading Fusion ioDrive2 2.4TB flash storage devices for a total of 9.6TB of all flash storage. Each of these ION Accelerator units fits in 2 rack units (2RU) and delivers well over 645,000 8K database IOPS and up to 12 GB/s sustained throughput. Data redundancy is maintained across pairs of ION Accelerators using synchronous writes, thus providing system and data high availability.  The Oracle RAC nodes consist of Dell PowerEdge R620 servers. Each two-socket server consumes a mere 1U of rack space and yet is capable of pulling 1.4 million 8K IOPS from the ION Accelerator storage layer as measured by the Flexible IO Tester utility.  The Oracle RAC nodes are connected to the ION Accelerators through redundant Mellanox switches. Each Oracle RAC node sees the ION Accelerator storage as simple multipath block storage. The multipath devices are aggregated by Oracle ASM to create a large and powerful diskgroup.  Expanding the size and performance of the database is as easy as adding more ION Accelerator devices to the ASM disk group.”  The links to the article and the Reference Architecture follow.

FIO_OracleRAC_1

FIO_OracleRAC_2

Increasingly, software vendors  see the virtue of putting their flash storage locally on the server.  For example, VMware engineers wrote recently that by replacing array shared storage with PCIe flash card storage on the servers they could substantially reduce power consumption and maintain high levels of performance.

VMware_Power_01

 

 

Clojure Example : Building an Online Community with Clojure, Datomic, Angular and Ansibile

I ran into this very nice article the other day that goes into the how of creating an online community. Besides using Clojure, it also uses Datomic as the database, Angular for building the front-end and Ansible and Vagrant for provisioning and deployment.  I just started using Vagrant on my desktop to provision a number of VirtualBox instances for some search engine work I am doing.  This article provides a nice level of depth to the  discussion of a number of technologies and shows more than a basic outline of the architecture.

flyingmach-grateful