Oracle Storage Guy

Upcoming Webcast for Oracle Database Virtualization using VMware

Please join me in my upcoming EMC Live! event. I will be presenting on Efficient and High Performance Virtualization of Oracle Database Environments using vSphere, similar to my presentation at VMworld in September. However, I have added significant material, so those who saw the VMworld presentation should still join this webcast. Anyone who is interested in virtualization of Oracle database servers should find this presentation interesting as well.

This event will be presented live on October 29 at 11:00 a.m. EDT. Details can be found here.

Charles Phillips' response to customer input

For information on the response to Charles Phillips request for customer input on VMware, see Chad Sakac's blog. I heartily agree with Chad's comments. In addition, I will add the following point, which I think is key to the discussion.

Charles Phillips makes effectively the same statement that is contained in the metalink statement on support, i.e.:

What we've said on other VMs is that we will respond to support issues as they arise but so far we've elected not to formally certify any third party VMs....

The metalink support statement is very similar. It says:

Oracle has not certified any of its products on VMware virtualized environments.

In my opinion, both statements are slightly disingenuous. Certainly, they are misleading.

Other than RAC, Oracle simply does not certify things like this. There is no Oracle certification, so far as I can tell, for any hardware platform, other than RAC certified configurations. And we are not talking about RAC here.

As Chad points out, Oracle does not maintain a Hardware Compatibility Program for its software. For this reason, telling customers not to run VMware because it is not "certified" to run with Oracle software is simply meaningless. The same can be said of Dell servers, Intel CPUs, Emulex HBAs, Broadcom NICs and almost all of the rest of the stack on which Oracle software commonly runs.

Ask yourself this simple question: What is it that VMware provides? Folks think of VMware as a software company. In many respects this is wrong headed thinking. VMware is a platform vendor. What VMware provides is virtualized hardware. Thus, VMware provides a virtualized CPU, network connection, SAN connectivity, etc.

In that respect, VMware is far more similar to, say, Dell than it is to, say, Red Hat. Certainly, VMware does not provide an OS environment upon which the Oracle software runs directly. Rather, VMware provides a hardware (albeit virtualized hardware) stack upon which an OS environment like Red Hat Enterprise Linux (RHEL) runs.

Oracle does provide certification of things like RHEL. But, as I have pointed out before, RHEL runs just fine on VMware, and is actually certified to do so by Red Hat.

Perhaps it would help if VMware behaved more like a platform vendor and less like a software developer. That is another conversation. Personally, I would welcome that. Such a change would require VMware to be more involved in solutions development for applications vendors like Oracle, which I think would be a very welcome development.

In the meantime, I respectfully disagree with the assertion by Charles Phillips that Oracle software should not be run on VMware because it is not "certified" by Oracle. That assertion is simply inaccurate and misleading in my view.

Your comments are welcome as usual.

Charles Phillips Asks for Customer Input on VMware

I just received the following email:

Re: Need a favor - Oracle

You are not going to believe this.  Some VMware folks met with Charles Phillips, the president or CEO of Oracle and he said no customers had ever mentioned to him that they wanted Oracle to support their products on VMware.  Or modify the licensing scheme.  He offered if anyone knew of customers who did want better or more support for Oracle on VMware, or virtualization friendly licensing, to email him directly.  His email is Charles.phillips@oracle.com and he really needs to hear that customers run Oracle on VMware, and better support / licensing would be nice!

For any customers who are interested in Oracle on VMware, please send email to Mr. Phillips directly.

VMworld Aftermath and Oracle OpenWorld Premath

Realizing that it has been a long time since I blogged (the past few months have been crazy), much of interest has happened. EMC continues to make significant inroads into the NAS / Oracle database storage market, in competition with NetApp, which delights me. I was a speaker at VMworld 2009 earlier this month, speaking on VMware virtualization of Oracle database servers. And, of course, OOW looms, for which I am a demo captain and responsible for a major EMC high-end NAS demo as well as a speaker in the VMware booth.

In this regard, I have a couple of questions for you:

1. How many of you are doing actual virtualization of Oracle database servers with VMware? I really need to hear from you. EMC is doing quite a bit of work in the area of VMware virtualization, and some of that will be present at OOW.
2. How many of you are using EMC NAS with Oracle? I know about some of you, but not all. If you are using this combination, please comment to this blog. Let us know of your experiences, please.

I promise to be more vocal, and less remote, in the next few weeks as we approach OOW. I will be at the show, obviously, and very much look forward to talking to many of you live. Please come by the booth and say hello.

Paul Maritz Keynote at EMCWorld Mentions Oracle Performance on VMWare

As you all know, EMCWorld was this week, and I attended. One of the most interesting aspects of the event was the Paul Maritz keynote. Paul Maritz is the new President and CEO of VMware as most of you undoubtedly know as well.

During the keynote, Mr. Maritz said the "o" word, i.e. Oracle. In fact, Oracle was a very prominent feature of his discussion. He presented the results of an OLTP performance benchmark that VMware has done recently. The basic results are as follows:

Metric	Native	VM
Throughput in business transactions per minute (1,000s)	293	250
Disk IOPS (1,000s)	71	60
Disk bandwidth (MB/s)	305	258
Network packet rate receive (1,000s)	12	10
Network packet rate send  (1,000s)	19	17
Network bandwidth receive (MB/s)	25	21
Network bandwidth send (MB/s)	66	56

Overall, this indicates that VMware virtualization has no more than a 15% performance penalty over physically booted, which I consider to be actually very good. Certainly, given the other benefits of virtualization with VMware, this would be acceptable for many customers for many of their Oracle database environments.

Now, to deal with the obvious question which I expect to be raised: Why did VMware not see the performance advantage of virtualization that I did in my previous post on comparing physically booted performance to virtualized performance?

The answer will becomes apparent when you review the VMware performance study, linked above. My peformance comparison was between the following:

1. Oracle RAC 10g (i.e. clustered)
2. VMware ESX 3.5 with HA Cluster (i.e. clustered)

Their study is between:

1. Oracle Database 11g (i.e. not clustered)
2. VMware ESX 4.0 (also not clustered)

The most important consideration is the issue of clustering. Their study shows a single 8-way Intel white box being used as a server for both physically booted and virtualized. (Similar to our study, they used identical hardware for both configurations.) Our study shows a cluster of 4 8-way Dell PE2900 servers being used for both configurations.

Closely reading the VMware study, they used a single database server with identical memory and CPU settings for both physically booted and virtualized database servers. Thus, in the VMware study, everything was done within a single instance database server with a single database image.

Another issue is that we federated the databases on the virtualized side. Thus we had 4 clustered instances and 1 database on the physically booted side, and 8 independent database servers (consisting of a non-clustered instance and a database image) on the virtualized side. As I said in my previous post (linked above), creating a larger number of database images had a performance advantage, especially compared to RAC, where cache fusion and lock management add significant overhead.

I have received many comments that this makes the results from the EMC study non-comparable. I somewhat agree, and I will deal with that issue at length in a future post. The VMware performance result is certainly more apples-to-apples than mine.

However, in my defense, I will say (as I will explain at length in my up-coming post) that the configuration I used is identical to that used by Microsoft SQL Server in their clustered product. No one seriously contends that Microsoft SQL Server with MSCS is not competitive to Oracle RAC. Certainly, Oracle considers it to be competitive, as the abundant technical literature on their own website demonstrates.

Thus, the study performed by the Oracle CSV group at EMC is a natural and obvious step: We need to understand the performance differences (as well as cost and manageability differences) between Oracle RAC and Oracle Database with VMware HA Cluster. And when we do that study, we need to configure the databases in the most natural and logical way, taking full advantage of the features of each environment. This necessarily leads you down the path of a federated environment on the virtualized side, and a single image on the RAC / physically booted side.

The VMware study is undoubtedly very valuable and important: It demonstrates that VMware can provide world-class performance for Oracle production OLTP database environments. It shows that the performance penalty of VMware in this context is modest and manageable.

More on this later.

Oracle Wants To Win The Virtualization Wars – But Will Customers Lose?

In my previous post on Oracle's support of VMware virtualization, I pointed out that Oracle is behaving like a monopolist: Attempting to establish control over the entire IT stack, including application, OS, storage and virtualization. Oracle's recent moves in the area of virtualization further demonstrate this trend, potentially benefiting Oracle to the detriment of their customers and the industry.

Oracle recently revised one of its metalink support statements regarding virtualization of Oracle software products. This is the statement concerning support for virtualization with Oracle E-Business Suite. The revised version can be found here. (Note: This URL requires a metalink account.)

The changes in this statement are very interesting, in that they speak volumes concerning Oracle's intention regarding the use of VMware virtualization with their products, especially Oracle Database. Prior to May 8, the title of this support statement was:

Platform Vendor Virtualization Technologies and Oracle E-Business Suite

With the revision on May 8, the title was changed to:

Hardware Vendor Virtualization Technologies on non x86/x86-64 Architectures and Oracle E-Business Suite

Note the use of "Hardware Vendor" rather than "Platform Vendor" and the addition of "on non x86/x86-64 Architectures". Further, prior to May 8, the text of the support statement included the following:

The use of platform (emphasis mine) vendors' virtualization technologies (both software and hardware based) (emphasis mine) to host Oracle E-Business Suite 11i and R12 is covered by Oracle's policy with regards to 3rd-party products - that is, they are 'not explicitly certified, but supported' (emphasis theirs).

On May 8, this was revised to read:

The use of hardware (emphasis mine) vendors' virtualization technologies to host Oracle E-Business Suite 11i and R12 follows the same policy as Oracle's policy with regards to customizations - that is, they are 'not explicitly certified, but supported' (emphasis theirs).

Note the replacement of the term "platform vendor" with "hardware vendor", and the deletion of the phrase "both software and hardware based". Clearly, Oracle intends to carefully limit the support statement to only hardware vendors' virtualization on non-x86/x86-64 platforms at this point. Or to state it in the reverse, to exclude software vendors' virtualization products on the x86/x86-64 platform. And who is the dominant market leader for software virtualization on that platform? VMware, of course.

Further, the earlier version of the support statement included VMware explicitly, and this was deleted from the revised version.

I am speculating, but my guess is that this move on Oracle's part was prompted by the post to Chris Wolf's blog (which he issued on May 6), stating:

The bottom line – Oracle now offers best effort support for all of its E-Business Suite applications on any x86 hypervisor.

This post was based upon the earlier version of the support statement, obviously, which granted best efforts support to all virtualization products, both software and hardware, on all platforms, and included VMware explicitly. Others posted as well, basically claiming victory on VMware virtualization of Oracle software products. The timing is extremely suspicious, and it seems pretty clear that Oracle closed the loophole which might be perceived to grant official support to virtualization of Oracle Database using VMware.

The other move is, of course, Oracle's acquisition of Virtual Iron, a startup which has been nipping at VMware's heels for some time. The intent of that acquisition is obvious: To shore up the Oracle VM offering which has big gaps when compared to VMware.

Combined together, these moves present a fairly stark and obvious message: It appears that Oracle is carefully and intentionally excluding VMware from its support statement. It also seems very obvious that this is without any technical justification whatsoever, given that Oracle has granted explicit support for virtualization products from other vendors which are completely homologous and comparable to VMware. The apparent reason is to benefit OVM, Oracle's competitive offering against VMware, by forcing Oracle Database customers to run that virtualization product instead.

If VMware were a niche player in the virtualization market, that would be one thing. Excluding such a player might be seen as mean spirited, but the impact would be minimal. In the case of VMware, though, Oracle is excluding the vast, vast market leader in the virtualization market.

This is exactly like Oracle attempting to exclude Microsoft Windows from its list of supported OS platforms and promoting Linux as the exclusive OS platform for running the Oracle Database software product. Permitting Linux as an alternative to Windows as your OS platform for running the Oracle Database is fine. Even promoting Linux is the preferred alternative is imminently justifiable. But attempting to exclude an OS platform with the market dominance of Windows would be met with howls of protest by numerous customers. I submit that this is exactly what Oracle is attempting to do by excluding VMware from its best efforts support policy.

I have several questions for Oracle:

1. Do you have any conceivable technical justification for excluding VMware from best-efforts support, given your support for less mature, less popular software virtualization products sold by other vendors, yourself included?
2. How are your customers served by forcing them onto a virtualization platform with less maturity and less market penetration than VMware?
3. As Chuck Hollis pointed out in his blog, competing on your own merits to own the datacenter is one thing. Using your dominance in one area (databases) to force your customers to accept dominance in another area (virtualization) smacks of anti-competitiveness. Can't you tolerate a free and fair competitive environment? What are you afraid of?
4. Given that there is abundant evidence from extensive testing (including at EMC) that VMware virtualization of Oracle databases is both viable and compelling, what is your real issue here?
5. If you have no reasonable response to the above questions, then when will you extend official best-efforts support to VMware which you have already done to other equivalent virtualization products?

Response to comments from Alessandro Perilli blog

I have noticed that a large number of comments have been submitted to the recent post on Alessandro Perilli’s blog which links to my recent post on Oracle’s support policy for VMware virtualization. Many, if not most, of these comments are actually comments directed at my post, not Alessandro’s post. Therefore, I will respond to them here. I am also attempting to post this as a comment on Alessandro’s blog, but that has not appeared on his site as of yet, and I wanted my response to be available.

To respond to several of the points raised by these comments:

Why would anyone want to run Oracle on a single vCPU under VMware?

Not sure how to parse this one. VMware ESX 3.5 allows 4 vCPUs per VM. Our performance testing was done with 4 vCPU VMs (2 VMs per server on a 8 core box, consisting of 2 Quad-core processors). 4 Servers were in the HA cluster, so a total of 32 physical CPUs were in the configuration, all of which were allocated to vCPUs on 8 VMs. This is laid out pretty thoroughly in the reference architecture published on Powerlink and EMC.com. You can find the EMC.com version here.

In terms of why you would want to run Oracle in a virtualized environment using VMware, that is pretty well laid out in my blog. But to summarize:

1. We found better performance in a virtualized environment using VMware HA cluster vs. Oracle RAC. I will deal with the fairness of that comparison on my next point.
2. The cost of VMware HA cluster is less than Oracle RAC, and VMware is appropriate for many customers. Not all, but many. For those customers, RAC would also work, but is, again, vastly more expensive. I lay out the usage cases where VMware HA cluster works as an alternative to RAC in another item below.
3. Manageability is higher with VMware HA cluster than RAC. Believe me, I should know. I run both routinely in our testing environment.

Why should I believe that performance of VMware HA cluster is higher than RAC when VMware will not allow third parties to benchmark their product?

Well, I am a third party, and the program I manage publishes performance benchmark results of both virtualized and physically booted Oracle production environments. These are available on Powerlink. The actual performance results are pretty confidential and proprietary, but I tend to open the kimono on this blog, as you have seen. We are using a TPC-C-like workload run under Quest Benchmark Factory for Databases. We do not claim that this is a published and audited TPC-C result, obviously. However, we have lots of experience running this.

I can tell you no one was more surprised by the VMware performance result than I was. We are still continuing to actively profile performance of both physically booted and virtualized Oracle environments. (In fact, we are beginning to profile OVM as well.) I had no particular axe to grind on this either way. The results were simply what they were.

RAC and VMware HA cluster are not comparable, are they?

Of course they are. Both Oracle Cluster Ready Services (the underlying technology behind Oracle RAC) and VMware HA cluster are cluster software products. They do basically the same sort of thing: Provide high availability for applications. They simply do so in a different sort of way. RAC provides one single database image across multiple physically booted servers. CRS provides transparent failover of VMs using VMware VMotion technology. Both are intended to protect application uptime and client access by providing a high availability solution.

Granted, they have different levels of HA. I would consider RAC to be a fault tolerant technology, in that the physical loss of a node will not result in database downtime (but may result in loss of client access). VMware HA cluster is a high availability product. A brief downtime is inevitable when a VM is being rebooted on a surviving node of the cluster after node failure. In our experience, VMware HA cluster works pretty well at getting the VMs back up and running, though.

In the end, it depends on what the customer needs, the level of HA being one of the issues. I cover that more in a later section.

Isn't RAC free with SE?

This is true, but very misleading. RAC is free with SE when the total CPU core count in the entire cluster is 2. This is a trivial cluster. Beyond that, EE is required.

And that is where the cost savings come in. EE is required for basically two products in most customer configurations:

1. RAC
2. Data Guard

RAC also carries the RAC upcharge above the cost of EE, making it the most expensive Oracle database software product, by far.

Assuming Data Guard is not required (and storage vendors have been competing with Data Guard using products like MirrorView and RecoverPoint for many years), then it comes down to RAC.

Assuming you can provide HA in another manner, then RAC is not required, and therefore SE can be used instead of EE, at 25% of the cost, plus savings on the RAC upcharge. I think you see the point.

The bottom line is that VMware HA cluster provides costs savings assuming the customer scenario allows this product to be used. Which is what I cover next.

Not all customers can use VMware for an HA solution, and need RAC instead. Right?

Not actually in the comments, but important. Again, EMC is a strong supporter of RAC. We run it in our lab, and we will continue to do so. No one would say that RAC does not work as well as VMware HA cluster or that it is not a great product. It is. But it is expensive and complex.

What RAC provides is two things:

1. A single database image
2. True fault tolerance

These are both great advantages, but not all customers need this. For example, I personally visited a very large Fortune 100 company. For confidentiality reasons, I will not use the name here, but believe me, they are a household name. This customer had many single-instance database servers running on 1U and 2U unclustered machines throughout their datacenter. The cost to manage these servers was immense, in the 9 figures per year. Yeah, that's a lot of jack.

The customer wished to consolidate all of these servers into a database cloud. I postulate that the way to do that could consist either of RAC or VMware HA cluster. But consider:

• What is the level of HA provided by the current configuration? Very low. Each physical server is presently a single point of failure. VMware HA cluster would be a big improvement.
• Does the customer need a single database image? Not at all. These servers are already islands of data.

The simplest and quickest way for this customer to consolidate the servers in this scenario, given the choice between RAC and VMware HA cluster, is VMware HA cluster. It is also the least expensive of those two alternatives.

I suspect that many, many customers are in this situation. For many applications VMware HA cluster is a viable high availability and consolidation solution for production Oracle databases. I would certainly not put a large ERP system on VMware. Nor the billing application for a large telco. But many, many applications could be put up on VMware just fine.

This is in many ways comparable to NAS vs. SAN. I am employed by EMC, so it may surprise you to hear me say this, but remember I came to EMC from NetApp. In my experience (not a scientific sample, but still) approximately 90% of the Oracle databases running in the world could be stored on an NFS server with absolutely no change in the client experience or uptime of the database. The same it probably true with VMware. Perhaps the percentage is higher or lower. I certainly have less experience with VMware at this point than NAS. But time will tell.

BTW, it would help me if folks could post comments to content written by me on my own blog, rather than having to run around and find it elsewhere. Just a thought...

To RAC or not to RAC (reprise part 2)

In my last post I contrasted the use of Oracle RAC with VMware HA Cluster for creating a database cloud. In that post, I promised to describe further the suitability of the VMware HA Cluster solution as opposed to Oracle RAC. That is the purpose of this post.

There are two areas of concern when evaluating the use of VMware HA Cluster for creating an Oracle database cloud. These are:

1. Scalability
2. Availability

Analysts like to use the so-called "magic diagram" to describe the market space for a given technology or solution. This is very interesting in this case. The following diagram is useful:

 
Database Cluster Technology Magic Diagram

Think of this graphic as conceptual, but it makes an important point. At the bottom of Availability scale are databases that you don't even back up. Everyone has a few of these. One example is a staging database for temporary storage of summary data which is then inserted into a data mart or data warehouse. If this database fails, you will simply recreate it and rerun the job.

At the top of the same scale is your company's ERP, web catalog or online trading application. If this type of database is down for any length of time (even a few seconds), money is lost, or even worse hard legal requirements are not met and liability results. These databases absolutely have to be up and running at all times in other words.

In the middle part of the scale is everything else. These databases have various levels of HA requirements. The SLA may require no more than 2 minutes of downtime in a single month. Or perhaps several hours of downtime can be handled over a weekend. And up to half an hour of unplanned downtime may be OK in a six month period. Again, the requirements vary greatly.

The Scalability scale is similar. At the bottom of the scale (to the left in this case) are databases that are very small and have small I/O requirements. A small database stored on a personal laptop or PDA would be an example. At the top of this scale are databases which are many terabytes in size, have thousands of online users at all times, and where every user has to have the ability to see every row in every table in the entire database. The latter is referred to as a "scale-up". These databases exist in virtually every enterprise of any significant size, and frequently they are also the databases which have high HA requirements. Thus, there is a lot of commonality between these two scales. Again, the company ERP, web catalog, or online trading application may be multiple terabytes in size, have a large number of online users, high I/O demands, and the requirement that all users have access to all of the data in the database, making federating this database impractical or impossible. Very large databases with high I/O requirements probably are pretty important, or we wouldn't be spending all the money, time and effort required to maintain them, after all.

In the middle of this scale are databases that have various levels of scalability requirements. Many of these fall into the "scale-out" category, where the scalability demands can be met by creating a large number of separate databases. In my experience, this may be true for a couple of reasons:

1. The database workload is naturally partitionable because a given class of users only needs to see a subset of the total database data. This is a federated database solution. The classic example of this is the software as a service (SaaS) customer. For example, I presented to a customer that publishes a software package for managing hair and nail salons. They have over 5.000 salons in North America that use their software. Obviously, no salon needs to be able to see the database data of any other salon, making the workload very easily partitionable. New users can either be added to an existing database, or a brand new database can be created.
2. The customer has created a mess. I encounter this fairly frequently, even in very large enterprise accounts. Instead of creating a large database cloud, they have created a large number of small projects. Now they are experiencing poor utilization, poor manageability, high cost and data center sprawl. This is the classic "server hog" scenario, in which each separate group developing a software project wants their own small data center, consisting of servers, network switches, storage, power, cooling and the like. The result is that the overall costs are very high, including the Oracle license cost. Oddly, this is the customer that Oracle most often uses as an example for the suitability of RAC, although I find it far less compelling than the scale-up scenario, for reasons which will shortly become clear.

Mapping this set of concepts onto our magic diagram, we can see the following:

Database Cluster Technology Magic Diagram with Products

Bear in mind this is my opinion about the way I see the market. Others will probably disagree, which I welcome. But here is my take: Oracle RAC owns the high end where availability and scalability are both very high. And it should. Oracle RAC provides unparalleled uptime with great scalability. This comes at a cost however.

At the bottom end, you have MySQL, Microsoft Access, and other user-oriented databases. (While both Oracle and Microsoft have personal versions of their high-end databases, I have always considered them developmental tools. For a simple user to create an address book with Oracle would be like swatting a fly with a nuclear bomb.)

In the middle you have a pitched battle among a variety of products, including Oracle SE, Microsoft SQL Server and IBM DB2. This is actually the bulk of the database market, as I show in the diagram. And this is where VMware HA Cluster actually helps Oracle substantially (a fact which I think many at Oracle have yet to fully appreciate).

Having configured both Oracle RAC and VMware HA Cluster many times, I can tell you that the manageability advantages of VMware HA Cluster are very strong. With VMware HA Cluster, you do not need to establish ssh equivalence with passwordless authentication as you do with RAC, for example. Granted if you do this often (as I do), this is no big deal. For the occasional user, it is mind-numbingly technical, with intimate knowledge of the various ssh files (known_hosts, authorized_keys, id_dsa.pub and the like) being required. And this is only one of a number of similarly technical issues that must be mastered in order to install and configure Oracle RAC.

To configure VMware HA Cluster, you simply establish a port on a vSwitch on each node in the cluster, create the cluster, and then drag and drop the nodes into the cluster. Believe me, those three steps (all accomplished using the VIC GUI) are far easier than creating a cluster with Oracle RAC.

Which gets to my point. Given the alternatives, and judging things like cost, manageability, and the like, unless you are either a very high-end customer (which Oracle owns as I have said), or simply slavishly committed to Oracle (these undoubtedly exist as well), you would be crazy not to try to create a moderately HA database solution with something other than Oracle RAC. Thus, you would tend to go down either the Microsoft SQL Server (with MSCS) or IBM DB2 (with the HA option) path, both of which are available at significantly less cost than Oracle RAC. And this is where VMware HA Cluster can actually help Oracle out. By giving Oracle another HA solution with simpler management and lower cost, Oracle can actually compete successfully with Microsoft SQL Server and DB2 in this space. Which is, again, the bulk of the database market. Thus, VMware HA Cluster helps Oracle to move down market, which in my experience is where alot of the interesting money is, as well as the next generation of new customers for Oracle.

Where VMware HA Cluster falls down is, not surprisingly, in these two areas: Availability and scalability. First, in the area of availability, Oracle RAC provides absolutely no downtime on the database as long as at least one node in the cluster survives. With VMware HA Cluster, the loss of a node in the cluster will result in the VMs on that node being rebooted onto one or more surviving nodes in the cluster. This is downtime. Granted, only a couple of minutes of downtime, but downtime nonetheless. For many users with many databases, this is acceptable. If your database requires absolutely no downtime, then Oracle RAC is your solution. Otherwise, VMware HA Cluster may be a viable option.

The other area is scalability. VMware HA Cluster cannot scale any given database very high. Since each database must run in single instance on a separate virtualized OS image, the size of the database in terms of I/O is limited to the maximum that you can achieve in a VMware VM. Presently, this is fairly modest.  The following chart compares our physically booted RAC solution to our VMware HA Cluster solution in terms of scalability. As you can see, the RAC solution scales far higher in terms of a single database image.

Solution	Instances	CPUs	Memory	Database Size
Physically booted Oracle RAC	4	8 per node	24 GB per node	2000 warehouses in one TPC-C database
Physically booted Oracle RAC	8	4 per instance	12 GB per instance	250 warehouses in per TPC-C database (8 total)

Again, in the scale-out scenario this is no big deal. As the scalability requirements of any given database customer are fairly modest, and each database customer can be placed on a separate server, this works fine. It does not provide for a scale-up scenario though. If you have a database that is many terabytes in size, has high I/O requirements, and where each user of the database must be able to see every row in the database, then you have a scale-up and you need Oracle RAC.

Where VMware HA Cluster shines is in several areas:

1. In an SaaS scale-out, VMware HA Cluster (and VMware in general) provide very convenient tools for creating new database servers. You can provision a new server almost instantly. I have tried to accomplish this with RAC, and it is quite a bit more difficult.
2. In a mixed environment, such as the server hog scale-out (where many versions of Oracle and different operating systems have been used), you can easily virtualize a given database project in place, without an expensive, time-consuming and risky migration. You have the Windows 2000 version of Oracle 8i running in production? No problem. You can p2v that puppy into a VMware environment, in the process getting all the advantages of mobility, manageability and improved utilization that VMware provides. Doing the same thing in RAC would require you to migrate all of your existing projects to a single target set of OS and Oracle versions, as RAC does not allow for any heterogeneity. This is a daunting prospect for many customers in this situation. VMware HA Cluster provides a quicker, simpler and cheaper path to a database cloud in this case.
3. In both scale-out scenarios, VMware actually provides a more natural management of a given database image than RAC does. Assuming SaaS, you would like to be able to backup and restore a given database customer's data without any disruption of the other customer's data. If the customer's data is stored in a single database, this is very easy and natural. If it is consolidated withiin a larger RAC database, this becomes more difficult. Especially if you want to do a point-in-time recovery. Ever tried to do a tablespace point-in-time recovery? I have. Great fun, believe me. The same thing is true with things like Data Guard. Data Guard is a log shipping / log apply product. You do not get to pick and choose which log you want to apply to the target database. If each customer's data is stored in a separate physical database, this works great. If a large number of database customers are consolidated, this may create issues. The same sorts of issues exist with the server hog scale-out as well.

To conclude: Both Oracle RAC and VMware HA Cluster have their place as HA solutions within the Oracle production database space. Oracle RAC owns the high end in terms of scalability and availability. VMware HA Cluster can significantly help Oracle to address the mid-market where the costs and manageability issues of Oracle RAC make that product overkill. Both products have advantages and disadvantages, and each have their place. Oracle would do well to evaluate the benefits of working with VMware and enabling their product to move down market and compete more effectively with the likes of Microsoft SQL Server.

To RAC or not to RAC (reprise)

In my post To RAC or not to RAC, That is the Question, I raise the issue of whether Oracle RAC is a cost effective product for many customers. We have now done quite a bit of work in this area, and I am proud to announce that a solution featuring VMware HA cluster as an alternative to Oracle RAC will be shipping in our program this November. I presented on this solution at VMWorld last week, and will also be presenting in VMware's booth in OOW this week on the same subject.

To give you an idea of the magnitude of the solution we are presenting, I will show you some highlights here in this post. First, this is the overall architecture of the solution from a hardware standpoint:

Physically booted hardware

This shows the Oracle RAC physically booted configuration. We compared this solution to a virtualized solution using VMware HA cluster on the same hardware. Here is the same configuration running the VMware solution:

 

Virtualized hardware

The high level network diagrams for these two solutions are as follows:

Physically Booted Oracle RAC Network Diagram

 
 

Virtualized Oracle Database Network Diagram

As you can see, in all respects the hardware configuration of these two solutions was identical; only the software was changed. The storage and networking configurations were identical as well.

First comparing the cost of the two solutions, see the following charts:

 

Oracle RAC Solution Component Costs

 

VMware Solution Component Costs

Overall Price Comparison Between the Two Solutions

So, obviously the price for the VMware solution is far less than the RAC solution. No surpirse there, since Oracle Database 11g Standard Edition was used instead of Oracle RAC 11g Enterprise Edition, a huge savings in terms of license costs.

The surprise came in the area of performance. We ran a TPC-C style performance test (using the Quest Benchmark Factory tool) against both solutions, and produced the following results:

Transactions per Second

 

Users

 

Software License Costs

As you can see, the performance of the virtualized solution actually exceed the RAC solution significantly.

In future posts, I will be discussing more of how we accomplished this feat, and what the areas of suitability of virtualization are with respect to Oracle. If you would like to discuss this with me live, please come by the EMC booth at OOW this week. I will be there, as well as presenting this solution in in VMware booth.

See you at OOW.

Why Oracle is the most interesting technology space for EMC

I read Jason Kotsaftis's recent blog post with interest, since I too am very biased. Rather than commenting on Jason's post (everyone knows comments very seldom get read), I thought I would wade in with my own take on this issue.

When I came to EMC, I was shocked at how Microsoft-centric we are. Not that Microsoft is an uninteresting technology space. Far from it. I simply maintain that Oracle is more interesting, and that Oracle has been woefully neglected by EMC in the past (although that is turning around rapidly due to the great work of folks in Jason's organization).

Looking at the two spaces: Microsoft and Oracle, which is more interesting for us? I maintain that Oracle is more interesting because of several factors:

• First is the halo effect. I talk about this a lot, but for those who don't know me, here is the idea. Oracle is the most expensive piece of software ever written for general purpose use. (It is also the most complex.) To give you an idea of the cost of Oracle, I recently visited with a major Fortune 500 company and met with their DBA group. They shared the cost of their Oracle license with me. That cost for them is $22 per GB per month. They have petabytes of this stuff. Think about that for a minute. That is an astounding number. This company is launching an entire project to reduce the cost of the Oracle license, because it is such a huge component of their IT budget. The effect of the fact of the cost of Oracle is simply this: The customer cares, and cares deeply, about the Oracle infrastructure. It is their crown jewel. They have paid dearly for it. In order to play in that space, you need to be the most enterprise-ready, robust, reliable, resilent technology in there entire environment. What does this mean for you? If you can store and manage the Oracle database data, you are by definition the best vendor in their datacenter. You are handling and helping manage the customer's most important, expensive, precious environment. Are you then good enough to store and manage the Microsoft data? I would say, virtually by definition, yes. So, if you qualify yourself to store Microsoft data, you do not automatically qualify yourself to store Oracle data. On the other hand, if you have qualified yourself to store the Oracle database data, you are almost all of the way there in building credibility in the Microsoft space, especially with the higher level managers in the customer's organization who have visibility into both those technology spaces. This is the halo effect. It is very real. I have seen this work in exactly this manner many, many times in my career.
• Of the two companies, Microsoft and Oracle, who is more aligned with EMC? Microsoft still makes the vast majority of their revenues off of the sale of consumer-oriented desktop software. Oracle is truly the enterprise software company. They dwarf Microsoft in that space. This is exactly the business we are in. If you look at the Symmetrix and compare it to Oracle, it amazing how many similarities there are in terms of the market they address. We own this space and so do they.

 

Is Microsoft easier? Absolutely. Oracle is a tough company to work with. They have a killer technology and they know it. They want to own the market. They do not believe that they need us. They know we need them. Their technology is far more complex and difficult than Microsoft.

All of that is true. But people make money doing things that are hard. Doing a great job of addressing the Oracle market will yield huge rewards, far beyond the costs of doing so. I strongly support the effort of Jason, Jeff, and Vince in building the Oracle relationship, and I am in for the long haul in helping them to do so.