This post is an update on the 11g new feature called Direct
NFS (dNFS). A lot of marketing buzz has been issued on this feature recently. I
hope to add a bit of reality to the discussion.
Here is the theory behind dNFS. I/O on a database server
occurs in a combination of user space and kernel space. Context swaps between the
two spaces are expensive in terms of CPU cost. If you can move a part of that activity from kernel
space into user space, you can save CPU cost due to reduced context swapping.
One example is the difference between Linux kernel IP port bonding and the
ability of dNFS to make use of multiple paths. Linux kernel bonding is expensive in
terms of CPU cost and less efficient than dNFS, according to the dNFS white
paper referenced below.
Further, dNFS avoids double-caching by skipping file system
NFS client-side caching. The data is only cached in the database buffer cache, thus avoiding
wasted memory and CPU cost in caching the data twice.
These benefits are thoroughly covered in the dNFS white
paper on Oracle’s web site. This paper was co-authored by Kevin Clossen, and it
does a pretty good job of stating the potential performance benefits of dNFS.
Fundamentally, I agree with these potential benefits.
Problem is, that is all they are right now: potential. This is true because in
our testing Oracle RAC 10g Release 2 provides far better performance than
Oracle RAC 11g Release 1 does, even considering that 10g is running over
plain-vanilla kernel NFS (kNFS) and 11g is running over dNFS. The reason for
this is simple: 11g is only available in 32 bit. 10g is running in full-blown
64 bit.
You can say that comparing 10g 64 bit to 11g 32 bit is
apples-to-oranges, and I will not necessarily disagree with you. However, bear
in mind that in all other respects, the
configuration was identical.
Here are the numbers. On 10g using Release 2 for x86-64, we
are getting 10,200 users with a bit over 500 TPS. (Again, bear in mind that
these are TPC-C style transactions. Do not compare these to normal Oracle
transactions.) On 11g using Release 1 for x86-32 on identical hardware, after
an excruciating level of tuning, we ultimately maxed out at 7,300 users at
around 380 TPS.
Now could we get more on 11g R1 using dNFS than 11g R1 on
kNFS? Maybe. I didn’t test it that thoroughly. Honestly, I don’t consider it
particularly interesting. 10g is readily available and provides far more performance than 11g on dNFS.
So when does dNFS become interesting? Or stated in a larger
context, when does 11g become interesting? From a performance standpoint (which
is something near and dear to my heart), 11g becomes interesting when it ships in 64 bit. Until then,
this is all simply theoretical. That will happen, from what I am told, sometime
in Q4. Stay tuned on that.
Will we jump on 11g performance testing as soon as we have a
64 bit version with reasonable performance? Most definitely. Do I expect to see
a performance benefit for dNFS at that point? Yes, I certainly do.
The other misconception which has arisen with respect to
dNFS has to do with the press release issued by NetApp stating that the dNFS
feature of 11g was developed collaboratively by NetApp and Oracle together. This press release
is true but misleading.
It is certainly true that NetApp was directly involved in the
development of dNFS. While I was not directly involved in this development,
much of it occurred while I was at NetApp. I think I can even claim some minor amount credit
for the initial idea behind this feature. This occurred way back in the late
90s. There were a series of discussions among several of members of the NetApp
staff, myself included (most of whom are no longer there) around how you would
optimize Oracle I/O to a NAS device. The outcome of those discussions was the idea of taking the NFS client and putting that inside the Oracle kernel. This is what dNFS effectively does.
So, yes, NetApp was certainly integrally involved in the
development of the dNFS protocol. I do not question that at all.
The misleading aspect of the press release is the unspoken implication that dNFS
will somehow work differently or better on NetApp NAS gear than the equipment
manufactured by other vendors, including EMC. That implication is completely
false.
Remember what we are talking about here: CPU cost on the
database server. File system caching on the database server. All of this is
about optimizing the use of resources on the database server. dNFS is an NFS
version 3 implementation, as the Oracle white paper clearly states. As such
dNFS will work in exactly the same manner, with identical performance benefits, on any
NAS device from any vendor. Including Celerra by EMC.
