The first talk hits the mark on providing a rapid-fire quick history and update on machine learning and also provides an introduction to Amazon’s 16 GPU Amazon EC2 instance to train or act as the hardware foundation for the new world of machine language frameworks.
From the Deep Water website – “Using complex multi-layer artificial neural networks, Deep Learning helps us derive insights from large unstructured data such as images, videos, sound and text or structured data from transactional databases such as financial data or time series. It enables innovations in domains as varied as medicine, social media, customer service, targeted marketing, automotive safety, security or fraud detection.”
In the next video, Amazon offers up a 8 and 16 GPU EC2 instance. A P3 instance can offer 1 PetaFLOP in a single instance.
A side note: We can differ on the optimism expressed but we are looking at the building of a new machine learning technology which will have a deeply profound ability to change society in radical ways. This is no longer speculation. An equivalent of a “Cambrian explosion” is on its way and will hit human societies in relatively short order. For example, what happens when you don’t need drivers, cashiers, analysts, and a wide range of other jobs because machine-learning agents do the work? When technology will not generate new jobs for humans but rather for AI? What effect will this have on human societies in general to see such wide-spread job losses. Will governments act on behalf of their citizens or on behalf of other interests. It is an interesting question worth pondering now (as opposed to when we are faced with the problem). Two authors Robert McChesney (professor at the University of Illinois at Urbana–Champaign) and John Nichols (journalist) offered a view of this future in 2016 in the book “People Get Ready – The fight against a jobless economy and a citizenless democracy “.
Joyent has announced a very interesting benchmark and an accompanying whitepaper. Joyent has optimized Postgres to run fast in their cloud. In head-to-head benchmark tests using standard Master/Slave Postgres configurations. According to Joyent – “Joyent’s virtual databases completed tasks up to 15X faster than a comparable Postgres multi-node virtualized database configurations running on Amazon Web Services.” Very interesting read, their performance was very good, even beating AWS SSD configurations.
I’ve seen the adoption of immutable servers at very large companies – it is spreading. In the next article and video you get a behind the seen view of one such deployment. They are using AWS. The video discusses the types of servers and infrastructure that goes into such a deployment. Also discussed are issues and problems encountered. Chad Fowler recently wrote a nice post on immutable deployments.
Chad Fowler recently discussed immutable infrastructure on “Foodfight” :
To get at the notes/outline information – more at the site. If you still are asking yourself, what is an “immutable deployment” ? There is an excellent explanation from Martin Fowler available :
You can find more here and here.
Netflix is doing some amazing things. If you have the service, you know they are dependent on Amazon Web Services but their cloud practices transcend that dependency. Adrian Cockroft has delivered some really excellent talks explaining how they do what they do.
and also a nice talk on how they moved to Cassandra to do a lot of the heavy lifting.
and he provided another very nice presentation at the Cassandra conference C*2012 about running Cassandra on AWS.
Interesting in the two Cassandra talks he discusses use of SSDs to improve Cassandra performance. He talks about moving from 2 drives (1.7 TB) to 2 SSD volumes (2 TB). He shows results from a hard disk versus SSD comparison. Netflix is offering a number of Cassandra-related software as open source, such as Priam (for Cassandra automation), Astyanax (client, front-end into Cassandra) and more (like Aegisthus, Zeno, Chaos Monkey, Zuul, Pythias, etc). Note that AppDynamics is used throughout these presentations. One other project I’m aware of is a non-JVM way of getting to the recipes in Astyanax is STAASH. You can follow all of this on the Netflix technical blog.
Also a post that may be of interest : Some Thoughts on Why We Want To Run Databases on Flash
If you are curious about the performance aspects of various web frameworks then it is worth looking at pretty impressive suite of benchmarks at techempower :
In the Round 7 benchmarks they benchmarked 84 web frameworks. You can see the results here for JSON Serialization, Single Query, Multiple Queries, Fortunes, Data Updates and Plaintext benchmarks. Done both on an i7 server and Amazon Web Services.
There is a new Kinvey AWS re:Invent Hub which has a lot of AWS content. Lots of video presentations, interviews and tutorials at the YouTube AWS re:Invent site. The set also points to a number of blog content. In addition, there is a number of tips & tricks, AWS Usage & Deployment and cloud computing news.
It’s possible to make direct calls to Amazon’s :
- S3 to store and retrieve objects at any scale,
- SQS to read from and write to message queues,
- SNS to generate and process notifications to mobile and
- DynamoDB to store and retrieve any amount of data at any access rate.
More at Amazon’s Web Services blog.
This is pretty interesting. I was reading about a relatively small $1.5 million investment from VCs to MadeiraCloud. This made me interested in MadeiraCloud. Aiming at enabling AWS and making it more accessible and simpler to use, they have produced a very nice GUI reminiscent of scientific visualization GUI from the early 1990s – AVS (Ardent Visualization System). It might be compared to it as it takes the same idea and allows you to design and deploy applications on AWS. Since the video was created – they have provided even more functionality.
More on this startup here.