Dual CPU Database Server Comparison
by Johan De Gelas on December 2, 2004 12:11 AM EST- Posted in
- IT Computing
The benchmark
Our client application that fires off SQL requests is called DBconn, written in VB.NET. As I am no .NET expert, DBconn was written by one of my best students, Jo Neve.DBconn is similar to AnandTech's SQL Loader, with a few differences. DBconn increases the workload progressively by augmenting the concurrency level. In other words, the client simulates that more and more users request data from the database system.
You can see the simplified interface of DBconn above. DBconn can also connect to different databases, and at the moment, DB2, MySQL and MS SQL server are supported. The precision of the tests can be increased in two ways. We can specify the number of repetitions of one test, and the total number of requests. A request can be described as a "package" of SQL statements sent to the Database. Every request is done by one thread.
The databases are not accessed via ODBC, but by the following drivers:
- Microsoft SQL server driver (for MS SQL testing, to be discussed later)
- IBM beta DB2 driver, .NET data provider (IBM.Data.DB2.dll 15/06/2004)
- MYSQL beta4 byteFX driver (ByteFX.MySqlClient.dll 15/06/2004)
First, we used this code for making threads. The "join" method in this VB.Net code should normally wait until the last thread is finished. However, we found out by experiment that sometimes previous threads are still running. To avoid this, a special method in DBconn checks if all threads are indeed done before firing off a new load of threads.
The application also performs randomly in two ways: variables in SQL statements are randomized to avoid getting the same results over and over again. Queries are also called in a randomized order.
DBconn proved to be pretty accurate and results were repeatable. After weeks of tuning the application, we measured an error margin of +/- 2% to 3% between runs in DB2 8.1.3 and MS SQL server 2000.
The database was about 1 GB and has been extracted out of a database of about 7 GB of data. We cannot go into much detail, but the database is the backend of an e-commerce site, which provides a vast number of products from many suppliers. Extensive descriptions of products exist in the form of articles and newsflashes about the latest developments, so the database is smaller, but similar to Anandtech with its news and articles.
The following is a typical and very common SQL statement in DBconn:
SELECT table.x ..... table.z, function(table.y) FROM table3 LEFT JOIN othertable ON matching keys LEFT JOIN yetanothertable ON matching keys WHERE othertable.y IN condition AND condition1 AND condition2 GROUP BY table.xyz ORDER BY othertable.zyx;
There are queries to request the newspage, to request an overview of certain category of products ordered in different ways, and so on. The mix of queries is based on the real world use of the database.
Facebook
Twitter
RSS
46 Comments
View All Comments
JohanAnandtech - Thursday, December 2, 2004 - link
About SLES9 and NUMA: NUMA is also supported by Linux kernel 2.4.21 and it boosts performance only a tiny bit. The reason are the very speedy HT links which keep latency at a minimum.It is still possible that kernel 2.6 NUMA support is far better of course, but I doubt it makes a difference for quad or dual systems as there is only hop in quad systems. With two hops (8 CPUs) from CPU 1 to 8 for example, this will become important.
AtaStrumf - Thursday, December 2, 2004 - link
A TYPO:So for now, the Opteron has an advantage still, but it ***can*** /can't/ knock out the Xeon, as it could have a few months ago, before the Xeon Nocona arrived.
HardwareD00d - Thursday, December 2, 2004 - link
There have been enough benchmarks on the web for a long time which show that Opteron generally wipes Xeon's a$$ hands down, and scales far better in multi processor configurations. The latest Xeon is nothing special compared to prior versions and will no doubt preform better mostly due to its increased clock speed. Xeon will never be better than Opteron no matter how much cache and tweaks Intel adds.Maybe Intel's next server architecture will be something to woo, but that's a ways off.
jshaped - Thursday, December 2, 2004 - link
as a long-time reader of aceshardware, i'll be the first to welcome Johan here, great first article. keep them coming!!!!HardwareD00d - Thursday, December 2, 2004 - link
I don't think there are enough variations of the way requests are handled to make a realistic conclusion for either chip. I'm sure you could create a situation where Intel bests AMD in My Sql, and vice versa. This article really needs more benchmarks and more in-depth analysis. Still, it provides enough information to conclude that both Xeon and Opteron have their strengths and weaknesses.mczak - Thursday, December 2, 2004 - link
Nice read. I really think though you should have used SLES 9. Not only does it use kernel 2.6, but it's also NUMA-aware (and DB2 should specifically support it, though it might not have been released yet). SLES 9 also ought to be faster especially on x86_64 due to newer compiler (not that it would matter much with precompiled databases, but every bit counts...). Though for 2-cpu boxes, NUMA might not be that important - but it's safe to predict a landslide victory for a 4-cpu opteron with NUMA support vs. a 4-cpu xeon box. Xeons simply don't scale to 4 cpus, intel might sell them but they are useless (especially since the Xeon MPs are still limited to 400 (or was that 533?) Mhz FSB.A pity though the quad opterons don't support ddr-400. I guess manufacturers decided it's more important to have a boatload of ram slots than fewer slots (with shorter traces) with higher speeds...
And btw, where are the 90nm Opterons? AMD's latest roadmap shows them as available in 2004, which doesn't leave too much time...
bthomas - Thursday, December 2, 2004 - link
Bogus conclusions about IBM tests IMO. From the
article:
> If we had published a similar report back in
> August, the Opteron would enjoyed a landslide
> victory. Luckily for Intel, Nocona is very
> competitive and is about 5% faster than the Opteron 250.
and later in the "conclusion":
> Nevertheless, AMD cannot sit on its laurels.
> Intel made a very good comeback with Nocona, as > this 3.6 GHz CPU is just a tiny bit faster in >
DB2.
It has not.
You fail to specify that this is comparing the _32 bit_ mode for the Opteron. IF you compare the Nocoma performance to the Opteron 64 bit capability...it sweeps the the Nocona in all tests.
The true conclusion is that based on the results in the article, for neither of the databases tested do *any* of the Intel processors compete with the Opteron.
fitten - Thursday, December 2, 2004 - link
Randomized benchmarks are hard to verify as well. You could get a "good" distribution that really takes advantage of cache locality while another randomization may be very cache unfriendly. I agree with #5 to a degree. A database that fits entirely inside of RAM isn't very interesting, ultimately.Still, I am happy that AnandTech is going down these paths of benchmarking instead of just being about Doom3, HL2, and FarCry like most other sites. I eagerly await further database benchmark articles.
PrinceXizor - Thursday, December 2, 2004 - link
#5 - Since when do top tier e-commerce cites compare to mid-level company database users as the beginning of the article mentioned?My company is an engineering firm that does custom electronics. Our database server handles all the transactions for our Inventory/MRP system which is mostly reads. These benchmarks are very appropriate. I wish I could have convinced my boss to go Opteron. Its funny, they had Athlon MP's before and then switched to Xeons when Opteron was out. Go figure.
Anyway, great article. I'm not IT guy by any stretch, but I enjoyed the article.
P-X
Jason Clark - Thursday, December 2, 2004 - link
#6, done ages ago..http://www.anandtech.com/IT/showdoc.aspx?i=2205
http://www.anandtech.com/IT/showdoc.aspx?i=1982