It certainly is worth looking at the latest white paper from Symantec and Intel. They have dropped a small bomb on the flash array party. In a white paper, Software-defined Storage at the Speed of Flash, the duo provide a look at a nice Oracle database architecture where they show both price/performance advantages and comparable performance to flash arrays from Violin Memory, EMC and a CISCO solution. Two Intel R1 208WTTGS 1RU servers were outfitted with four Intel P3700 Series SSDs, 128 GB DDR4 Memory, Symantec Storage Foundation Cluster File System 6.2, Oracle 11gR2 and Red Hat Enterprise 6.5 OS. The two servers are interconnected with high speed dual-port Intel Ethernet Converged Network Adapter. The white paper goes into quite a bit of detail and offers a nice chart comparing the converged solution with the flash arrays solutions.
There is a nice reference architecture white paper which provides a lot of details into setting up an architecture that supports Oracle Database deployment and operational details on infrastructure consisting of Cisco UCS servers and Tegile Hybrid Arrays. Select the item below to see it.
In addition, there are a number of new flash arrays – the following brief discusses Oracle and these new arrays :
Fusion-io, Dell, and Mellanox have built a very small footprint (16RU), extremely fast, and price-sensitive reference architecture for Oracle RAC 12c. It is very impressive on a number of counts. It further highlights the move to using local flash storage over remote flash storage arrays. This solution aims at reducing hardware sprawl and power consumption by achieving higher performance in a smaller footprint. From the description – “This reference architecture features four Fusion ION Accelerators and four Oracle RAC database nodes all connected through redundant Mellanox SX6036 switches. Each ION Accelerator in this reference architecture uses a Dell PowerEdge R720 with three 56 Gbps Mellanox ConnectX-3 InfiniBand cards and four industry-leading Fusion ioDrive2 2.4TB flash storage devices for a total of 9.6TB of all flash storage. Each of these ION Accelerator units fits in 2 rack units (2RU) and delivers well over 645,000 8K database IOPS and up to 12 GB/s sustained throughput. Data redundancy is maintained across pairs of ION Accelerators using synchronous writes, thus providing system and data high availability. The Oracle RAC nodes consist of Dell PowerEdge R620 servers. Each two-socket server consumes a mere 1U of rack space and yet is capable of pulling 1.4 million 8K IOPS from the ION Accelerator storage layer as measured by the Flexible IO Tester utility. The Oracle RAC nodes are connected to the ION Accelerators through redundant Mellanox switches. Each Oracle RAC node sees the ION Accelerator storage as simple multipath block storage. The multipath devices are aggregated by Oracle ASM to create a large and powerful diskgroup. Expanding the size and performance of the database is as easy as adding more ION Accelerator devices to the ASM disk group.” The links to the article and the Reference Architecture follow.
Increasingly, software vendors see the virtue of putting their flash storage locally on the server. For example, VMware engineers wrote recently that by replacing array shared storage with PCIe flash card storage on the servers they could substantially reduce power consumption and maintain high levels of performance.
Ran across this on twitter. If you are interested in getting datapoints on Oracle Exadata there are some nice new data points from Trivadis that are certainly worth reading. Keeping in mind that this is Exadata X3 (1/8 rack = 2 servers) and that Exadata X4 is now available. The report is in PDF format.
Second, new hardware. Think of a server that holds 32 SPARC 3.6Ghz processors – each with 12 cores (and 96 hardware threads per processor), it holds 32 TB memory. This is the killer server Oracle announced. The M6-32. The interconnect architecture provides a 384 port silicon switching network with monster bandwidth. Three numbers Larry Ellison showed in his announcement of the new systems, the Oracle M6 offers 1 TB per second IO bandwidth, 1.4 TB per second memory bandwidth and 3 TB per second system bandwidth.
Database cloning with snapshots is an important feature. Many flash/SSD-based storage arrays offer this feature. Pure Storage has done a very nice job of showing off this feature. In the first example they show how to clone a live Oracle database very easily. Snapshots offer a quick way to copy a production database for reporting or testing or getting a copy of a database at point-in-time.
You can learn more about Snapshots with another Pure Storage video – in this video 500 snapshots are created while a performance monitor is running :
One basic question that is often asked is – should I use PCIe-based Flash Card or Flash/SSD array? Flash/SSD arrays are being used within enterprise and cloud settings but a considerable number of companies are also using PCIe-based flash cards inside their servers to run databases and applications. The dominant PCIe vendor is clearly Fusion IO. Hands down. You begin to see why when you see how much time and energy they have put into the software side of their product as well as hardware. They have targeted some use-cases which make perfect sense for using PCIe flash cards within a cloud – for example Facebook is running MySQL on servers with PCIe flash cards. Apple also is using PCIE cards in conjunction with database in iCloud. Specifically around the MySQL ecosystem – Fusion IO is has been working with SkySQL to get atomic writes in MariaDB (a MySQL fork) and into the Percona Server. This optimization on MariaDB has yielded over 50% improvement in write performance. It is this type of value that differentiates vendors and is missing from some of the other PCIe vendors. Another excellent use-case is around NoSQL databases like Aerospike. It is such an excellent use case that they have created a benchmark and they recommend SSDs or other flash devices.
One very interesting read is the Fusion IO whitepaper describing Reference Architecture for Oracle RAC with ION Data Accelerator and Dell r720 Servers. It shows a resilient architecture for running Oracle RAC. The aim of this architecture is to reduce hardware costs and per-core Oracle licensing by 50% compared to an Exadata X3-2 Half Rack. At the same time, there is a reduction of power consumption and delivering high IOPS for this architecture. When compared to an Oracle Exadata X3-2 the Fusion IO/Dell architecture consumes only a quarter rack (compared to half), 32 instead of 64 cores, requires only two database servers (compared to 4), requires only two storage servers compared to seven and provides 800,000 Read Only IOPS (compared to Exadata’s 750,000 IOPS) and provides a 50% cost reduction in licenses. Pretty worthwhile reading is Fusion IO’s whitepaper on how they do this :