Welcome to the IT Computing Section at AnandTech

Welcome to the IT Computing Section at AnandTech

Benchmarking is an art form. The subtle nuances of methodology design and test bed engineering can take months, even years, to master. With the nearly endless variety of PC usage scenarios, it's easy to make a mistake and draw the wrong conclusion. One false assumption and your results are worthless.

Case in point: The Corporate Enterprise. Mainstream enterprise computing is different from hobbyist/enthusiast computing. The expectations are different. The usage model is different. You need to understand both before you can formulate an effective testing methodology.

Sadly, many fail to grasp this simple truth. These "expert wannabes" approach virtually every testing project the same way: Load-up their favorite linear productivity and gaming benchmarks and fire-away. They give little or no thought to how a system will be used in the real-world and so often miss the point of why a platform is designed a certain way.

In an effort to educate novice readers on the realities of enterprise testing, I've compiled the following list of guidelines to help bring them up to speed:

1. Enterprise customers do not overclock. Period. Overclocking is something hobbyists do in their spare time with their own hardware. End users who over-clock their corporate desktops tend to find themselves unemployed shortly thereafter.

2. Enterprise customers do not run games. Again, gaming is something enthusiasts do at home in their spare time. As far as enterprise computing is concerned, gaming benchmarks are (or at least should be) irrelevant.

3. Enterprise customers do not operate in a vacuum. Virtually all enterprises are connected via some form of network infrastructure. These typically include various network-level services, like encryption, firewall protection, proxy redirection or network caching. Test scenarios that fail to incorporate connectivity (either online or offline, via network caching) to a realistic network environment are irrelevant.

4. Enterprise customers do not work in a "bare iron" environment. The typical enterprise client computing "stack" includes multiple layers of systems management, security and connectivity software. Many corporate login scripts are mini-applications in themselves, with synchronized single-sign-on to multiple, back-end resources. Simply loading up the OS and a benchmark script doesn't cut it. The PC must be tested within the environment for which it is being deployed, using features and functions relevant to its day-to-day usage model.

5. Enterprise customers do not single-task. Outside of the odd data entry kiosk or Point of Sale (POS) system, most enterprise users run more than one application at a time. Often these applications are active in the background, retrieving data, monitoring processes and integrating the user into the flow of corporate information. The old model of serial task switching between individual productivity applications is simply invalid.

Mastering the art of enterprise testing can be a challenge. You need to spend some quality time supporting an enterprise-caliber IT operation before you can fully grasp the unique needs of the modern knowledge worker. At CSA, we've spent years working with such organizations in an effort to refine our testing methodologies and create new tools that simplify the benchmark design process.

The first fruit of our labor, Benchmark Studio, represents our vision for a sophisticated load simulation and performance-testing platform, one that encompasses the full range of enterprise computing usage models. Our follow-on solution, OfficeBench 2001, is an attempt to distill our methodology even further by providing an easy to use complement for other currently available benchmarks.

Together, they give us a powerful toolkit with which to tackle the tough IT questions of PC platform scalability and client/server application performance. And thanks to Anand, we now have the opportunity to share our experiences with one of the strongest and most vocal communities on the web. We look forward to working with Anand and his talented staff and to making our own small contribution to the AnandTech community.

The IT Computing section on AnandTech will house articles that deal with the trials and tribulations of computing in the IT/Enterprise worlds. These articles and editorials will serve as a complement to AnandTech's current content, hopefully to provide you with a better overall picture of the computing industry for all aspects of your life both work and play.

Randall C. Kennedy
CSA Research