Improving Performance on Solaris, Linux and SmartOS

For many commercial companies the center of the Solaris world is at Oracle. However, for many of us that have lived and breathed Solaris, that center has shifted quite a while back to Joyent.  Oracle’s high licensing costs around a Solaris x86 license makes it uninteresting to many. Meanwhile, Joyent offers an open-source version of Solaris that actually offers a number of significant improvements. Joyent’s SmartOS has become one the most innovative operating systems available on the scene today.  A derivative of OpenSolaris – it has vastly improved on it, creating an excellent cloud-centric operating system with a wealth of features.  Improved are all the compelling Solaris technologies – zones, ZFS, dtrace, SMF, etc – but more important is how they have added new technologies (KVM) and offered a wealth of new features. One of the big wins with SmartOS is improved performance.  If you are interested you can find more information on SmartOS here :


One of the areas where Solaris and SmartOS excel is dtrace – a framework that allows  observation of the running operating system and it’s applications and processes at a very deep level. On more than one occasion I was able to observe the details of applications running in production by using dtrace.  It is typical to see huge performance wins by using dtrace to diagnose performance issues or simply examine running applications.  Twenty examples of performance wins – with the bulk coming from use of dtrace :



Brendan Gregg is a performance guru and he is really well worth listening to. One of the more interesting articles he has written compared Solaris Zones, KVM and Xen and looked at in detail at them.


He recently presented SCaLE 12x and his presentation is, as usual, informative. Discussing what Linux can learn from Solaris-based systems and vice-versa when it comes to performance.
Brendan Gregg has written a number of books on the topic of performance.  He co-authored a book on DTrace :

bgreggdtraceand also authored a comprehensive book on performance :



Extremely Recommended Viewing : Micron’s P320h 2.5″ Flash PCIe form-factor Offers Hot-Swap

Updated : This is very exciting upgrade to the P320h flash card.  Here is a differentiated solution and potential disruption to many of the PCIe cards available. In case you missed it, Aerospike, the NoSQL database company that tunes for high performance on flash, provided a great review on the P320h (to quote – “it blew away the competition”).  I have updated this post with a link to StorageReview’s review of the P320h and two videos – one from StorageReviews and one from Micron (overview). 

Very interesting.  There are two form factors to the P320h from Micron. The first form factor is the card.  The second is a new 2.5″ form-factor of the P320h.  I have since found a couple of reviews on this card.  The Storage Review did a review of the P320h PCIe flash card and compared to Fusion IO and LSI flash cards


and from another review we look at a the new Micron flash PCIe 2.5″ form factor . This form factor allows you to hot-swap it :


You can see that Dell offers this form factor in their R720 with a specialized backplane :


Here is the demo of 2.5″ P320h being hot-swapped :

and a discussion between Dell and Micron on the Micron 2.5″ P320h PCIe hot-swap device :

Updated : If you are interested – StorageReviews has a nice review and video showing the  testbed on a SuperMicro and Infiniband.  They are running Microsoft Windows Server 2012  and Microsoft Storage Spaces :

and a review of the two products by Micron.

The P320h (non-2.5″) version was tested in this video (running iometer/4k workload) and gets 650,000 IOPS on Windows and 750,000 on Linux :


gotostorageGo to more posts on storage and flash storage at


Comparing Linux and Solaris

Some Linux advocates suggest that Oracle Solaris 11 is struggling against Linux distributions and they point to benchmarks in Phoronix that show Linux doing reasonably well against Solaris. One key issue is that the benchmarks use the slower GCC 4.5.2 compiler on Solaris, instead of the Solaris Studio C compiler which is a high performance compiler. Another aspect is that Solaris-based distributions offer a number of features that are lacking in Linux  or in early releases on Linux (ZFS, Zones, DTrace, Fault Management Framework, Management Framework, etc) and it can easily be argued that not everything is about performance. Linux emerged as an alternative to commercial versions of Unix by creating open-source copies of Unix technologies. Now, Oracle itself is trying to add Solaris features to their Linux distribution – but it is a difficult process.  Oracle itself makes a good case for Solaris in a comparison between Solaris and Red Hat Enterprise Linux.

Comparing SolarisLinux

Attempts at Moving Solaris Technologies to Linux.  Oracle is trying to move Solaris technologies to Linux – for example,  Zones on Linux and DTrace on Linux.  Work is also underway on porting ZFS to Linux.  Interestingly, Parallels, IBM and Google have all aimed at delivering something akin to Solaris Zones to Linux. Despite the release of Solaris 11, a good case can be made that Solaris has suffered at the hands of Oracle’s stewardship of the platform.  Oracle’s non-support of OpenSolaris created a vacuum that has made possible a wholly independent version of OpenSolaris called illumos.  The illumos distribution is offering a plethora of updates to OpenSolaris technologies and these have been contributed by key software and cloud companies.  Companies such as Joyent, Delphix and Nexenta have contributed to this distribution.  Joyent, for example, has released a highly innovative cloud-oriented version based on illumos called SmartOS which has added Linux’s KVM, ZFS IO throttling and a number of other features.  Joyent’s moving of KVM to Solaris is an interesting move of a major Linux technology to Solaris. An interesting conversation about SmartOS   :

In another video – you can see that the Linux community is discovering lightweight zone virtualization :

Joyent, meanwhile, has improved their SmartMachine experience in their cloud (effectively zones) by being able to deeply introspect each zone.  You can see more about that on their Cloud Analytics page. While Linux has become the more popular flavor of Unix-oriented operating systems, both Solaris-based distributions and Linux offer strong Unix-based variants.  The emergence of illumos-based distributions offers an independent, open source alternative to Linux that offers a wealth of Solaris features. One thing is clear – Linux and illumos-based Solaris distributions are benefiting from each other’s existence.


gotostorageGo to more posts on storage and flash storage at