dNFS Performance Reprise

In a recent post, I stated that I would circle back around and give you an update on Oracle 11g performance once we had tested the 64 bit version. I have done so at this point, and I must admit that I am rather disappointed.

To remind you of our approach to performance, we do not do any exotic tunings. We simply tune in a manner which a typical customer would find reasonable. No hidden leading underbar parameters, or that sort of thing. This is very, very different from the typical TPC-C you see posted on the TPC.org website.

On 10g, what we found was that tuning memory is easy. We set the parameters SGA_TARGET and SGA_MAX_TARGET and we are away to the races. Of course, we tune Huge Pages appropriately as well. For more information on that, see the excellent article on the Puschitz website.

We got a lot of improvement by tuning Huge Pages. Even as a storage company, who make money by selling disks, we recognize the scalability improvements from adding memory. In general, memory is our number one scaling bottleneck. We never recommend that customers add disk before we have investigated memory first.

So, needless to say the issue of memory is very central to our methodology on tuning Oracle.

When the 32 bit version of Oracle 11g came out, as I stated previously, the memory model was the main reason that we were not able to scale as well as we did on the 64 bit version of 10g. Hence, I had high hopes that the 64 bit version of 11g would scale well. In fact, I hypothesized that we could transition our entire program to 11g.

Wrong. The memory model on the 64 bit version of 11g is no better than the 32 bit version. I find this very strange, and honestly consider it to be a bug. The bottom line is that we have not been able to scale the 64 bit version of 11g any better than we did the 32 bit version.

For this reason, again, we cannot really assess the performance impact of dNFS in a real-world performance workload. (Very contrived testing could be conducted to prove that dNFS saves CPU cost, but what would that prove? Others have done that testing and we have no interest in doing so.) Our focus is on a real-world workload running on a real-world configuration. In this situation, 11g over dNFS does not perform better than 10g over kernel NFS.

We have gone back to 10g Release 2 for now. I will come back to 11g later, perhaps when Release 2 is available (although I have been told I may wait a while for that). In the meantime, I will continue to update you on what we find out in our testing. Our current focus is on the use of VMware in the Oracle context. More on that later.

Comments

[Chen Shapira]

I just discovered your blog and I can already see I'll spend the rest of the evening reading it.

Thats good information you have here.

According to Oracle, dNFS will scale better than kernel NFS if we use multiple NICs. Are you planning to test this?

----------------------

Response by TOSG:

Already did. The testing we did used multiple NICs and we tested both kernel NFS with Linux bonding as well as dNFS. As I state in my post, the performance differences between kNFS and dNFS are washed out by the inferior memory model of 11g versus 10g. This is a case of a negative upgrade, i.e. a new feature which works worse the old one did. This makes 11g significantly less scalable than 10g, and thus dNFS buys you nothing. At some point I may spin up another 11g testbed and report this as a bug to Oracle. Right now, I am more focused on 10g and integrating that with our technology.

[Kevin Closson]

"Wrong. The memory model on the 64 bit version of 11g is no better than the 32 bit version. I find this very strange, and honestly consider it to be a bug."

Would you please do us a favor and tell us what you mean by "memory model?"

[Krishna Manoharan]

Hi Jeff,

Can you please post any stats you have collected (if possible)?

Thanks
Krishna Manoharan

[Dan Dressel]

Hi Jeff,

Some very interesting information on your blog.

I'm in the process of testing 11g on the Linux x86-64 platform. What do you mean by an inferior memory model in 11g? Can you provide some specifics?

Are you referring to ASMM using the memory_max_target and the memory_target parameters? What problems are you seeing? Please share.

Thanks,
Dan

[Val]

Hi,

Could you please explain how 11g memory model is different from 10g and why, due to the difference, performance is worse ? Are database memory cache algorithms inferior, or is it something entirely different ?

thanks.