Dual CPU Database Server Comparison
by Johan De Gelas on December 2, 2004 12:11 AM EST- Posted in
- IT Computing
The benchmark
Our client application that fires off SQL requests is called DBconn, written in VB.NET. As I am no .NET expert, DBconn was written by one of my best students, Jo Neve.DBconn is similar to AnandTech's SQL Loader, with a few differences. DBconn increases the workload progressively by augmenting the concurrency level. In other words, the client simulates that more and more users request data from the database system.
You can see the simplified interface of DBconn above. DBconn can also connect to different databases, and at the moment, DB2, MySQL and MS SQL server are supported. The precision of the tests can be increased in two ways. We can specify the number of repetitions of one test, and the total number of requests. A request can be described as a "package" of SQL statements sent to the Database. Every request is done by one thread.
The databases are not accessed via ODBC, but by the following drivers:
- Microsoft SQL server driver (for MS SQL testing, to be discussed later)
- IBM beta DB2 driver, .NET data provider (IBM.Data.DB2.dll 15/06/2004)
- MYSQL beta4 byteFX driver (ByteFX.MySqlClient.dll 15/06/2004)
First, we used this code for making threads. The "join" method in this VB.Net code should normally wait until the last thread is finished. However, we found out by experiment that sometimes previous threads are still running. To avoid this, a special method in DBconn checks if all threads are indeed done before firing off a new load of threads.
The application also performs randomly in two ways: variables in SQL statements are randomized to avoid getting the same results over and over again. Queries are also called in a randomized order.
DBconn proved to be pretty accurate and results were repeatable. After weeks of tuning the application, we measured an error margin of +/- 2% to 3% between runs in DB2 8.1.3 and MS SQL server 2000.
The database was about 1 GB and has been extracted out of a database of about 7 GB of data. We cannot go into much detail, but the database is the backend of an e-commerce site, which provides a vast number of products from many suppliers. Extensive descriptions of products exist in the form of articles and newsflashes about the latest developments, so the database is smaller, but similar to Anandtech with its news and articles.
The following is a typical and very common SQL statement in DBconn:
SELECT table.x ..... table.z, function(table.y) FROM table3 LEFT JOIN othertable ON matching keys LEFT JOIN yetanothertable ON matching keys WHERE othertable.y IN condition AND condition1 AND condition2 GROUP BY table.xyz ORDER BY othertable.zyx;
There are queries to request the newspage, to request an overview of certain category of products ordered in different ways, and so on. The mix of queries is based on the real world use of the database.
Facebook
Twitter
RSS
46 Comments
View All Comments
markwrob - Friday, February 4, 2005 - link
Great article. One thing I'm looking for onthis and similar comparisons is how the rigs
or their components do on power consumption.
Are there any places to find that out easily?
markwrob - Friday, February 4, 2005 - link
vaxinator - Monday, December 13, 2004 - link
Very interesting review. I think you are wise to narrow your scope to basically in-memory queries, since this pretty much ensures you are testing just the CPU and memory. Disk I/O is another topic.I would be interested to know a few more details about the database (the data model) and the actual queries used.
But the key element you need to watch for is ensuring your client box is not overloaded. I have seen lots of "performance reviews" where the client box was smaller than the device under test...wrong! It should be under-utilized and the cpu consumption/memory use etc should be reported as well. If necessary use lots of client boxes, but watch the network load.
rogerjlarsson - Monday, December 6, 2004 - link
MySQL v. IBM DB2 Single v. Dual"MySQL doesn't seem to scale as well as DB2."
Yea, right... It all depends on what level you start from!
MySQL _Single_ Xeon Concurrency 50: 157
DB2 _Dual_ Xeon Concurrency 50: 102
MySQL scales a little to _Dual_ Conc. 50: 222
DB2 is not even near!!! MySQL might be network
limited rather than Memory or CPU...
davesbeer - Sunday, December 5, 2004 - link
DecoderOracle's ETL tool is $5k also comes with
JDeveloper
Forms Designer
Reports Designer
etc.
etc
I've worked with some of the biggest financial institutions in the world including ABN Amro... etc...
Oracle's DB also is less costly then MS.. Not thier EE version but then again MS has nothing to compete with that one..
Also you mentioned MS tools are the best.. not likely.. they are the easiest to use but far from the best.. The worlds biggest datawarehouses, datamarts etc.. are not on MS and they don't use MS's tools... except for maybe MS's themselves...
jensend - Sunday, December 5, 2004 - link
Comparing these systems in 32-bit mode with a 2.4-based distro makes very little sense. Among a lot of other reasons, the 2.4 scheduler is AFAIK rather poorly suited for use with combined SMT & SMP.marcuri - Saturday, December 4, 2004 - link
as #5 methods are far from being anywhere remotely 'mature' as the article states. Actually, after talking to people who run databases, its close to useless. The benchmarks used test cpu speed, not memory or memory management. Opterons are a better solution because they are 64 bit and thus handel the management of 4+gigs of memory much better (it starts in excess of 1gig that 64 helps out i believe). By using a data set of 1 gig the benchmarks are completely missing whats important, its niave at best.Decoder - Saturday, December 4, 2004 - link
davesbeer:I've worked with Oracle 8i, 9, IBM UDB v7, SQL Server 6.5,7, 2K and 2005. I also do J2EE, ETL (Informatica, Datastage, DTS), .NET etc. $ for $, MS SQL Server 2005 is a much better value.
Have you ever built a data mart using MS SQL Server DTS? It's an ETL tool that comes with MS SQL server for free. Try building a data mart on Oracle and you will find yourself spending 100K US for an ETL tool. Also from a developers perspective, SQL Server tools are the best (Read up on SQL Profiler, Query Analyzer etc).
.NET not scalable? Scalable compared to what? J2EE EJB's?? Please. Sure there's a limit to scalability on 32-bit boxes, but with x86-64 and 64bit OS's, that limit will go away.
davesbeer: "SQL Server is for people who can't program real databases"
Program databases? Please elaborate. You code to a database, design a database, write procs for a database but what does program databases mean?
Also, tell that to the Finanical Services companies in New York.
davesbeer: "SQL Server will be delayed again... (you can remember I said that) and the technology is already outdated."
SQL Server 2005 will be extremely popular. Also SQL Server 2005 DTS (a complete rewrite) will penetrate the ETL market.
davesbeer - Saturday, December 4, 2004 - link
Decoder.I work with companies that have over 6000 processors actively running multiple databases in thier environments. MS just does'nt compete.. .Net is interesting but again you are stuck with M$ and scalabilty is just not there for real apps. Most financial industries run Sybase... its thier last bastion of full fledged support, I worked with 15 of the largest American Financial institutes, I know what I am taking about. I currently work with some of the largest IT departments in the world. Thousands of developers in a single company alone.. I know IT.
Oracle, DB2 etc's 64 bit knowledge is far superior to M$ and will remain that way for a long time.. they'll been writing 64bit code for years. SQL Server will be delayed again... (you can remember I said that) and the technology is already outdated. SQL Server is for people who can't program real databases (can feel the flames now) sorry Anandtech but your databases are lightweight... once they grow up you will switch to Oracle or DB2 etc..
at80eighty - Saturday, December 4, 2004 - link
First off: Welcome Johan : )next...im a frickin n00b, so pardon my ignorance : )
my questions may not be entirely on topic, but i hope someone...ANYONE can help...
im lookin to *build* an entry level server, so i've got the following doubts (based on the following factors) :
Company: startup...less than a year old
Expected # of initial users: 8 (2 people will be responsible for writing data, all employees will read db) requirements should go up to 15-20 people (2-4 write: all read) by year end (should business expand as planned)
Server: Mail + DB + File server in one
Cost: would be nice to keep things economical , but im willing to stretch for a middle ground : )
1) What CPU is best bang for buck?
2) OS will be Windows SBS Premium...was going for this since i gathered MS SQL is the best way to go (more so coz Anandtech uses SQL too :p )...after reading some of the comments im a little confused viz. DB2 v/s Oracle v/s MS SQL...plus Exchange looks kinda sweet too : )
3) Should i wait for SQL 2005? What benefits if any (with reference to costs) ?
4) Which is the best front-end in your opinion? i was looking to use Infopath as i found the GUI VERY easy to use and frankly the look is quite easy on the eyes too : p. Also, its important to note that I am *supposedly* the most computer literate in my company...so im looking to create a DB with an easy interface for my employees : p
5) Is Crystal Reports easy to integrate into MS SQL (or any of the above for that matter)? OR im lookin for a back-end that can export data to Excel, so that i can analyse & crunch numbers
6) Will i *need* an in-house IT guy to manage the DB? ...or even the server as a whole? (looking to keep costs to a minimum)
7)
i've got TONS of further queries (and i dont want to attention-whore on the thread :), so, should anyone have any spare time (and any inclination in the first place : ) to help a fellow AT'er, my mail add is: ucanmailme AT gmail DOT com
Thanks in advance
Cheers
'80'