The Opteron 6276: a closer look
by Johan De Gelas on February 9, 2012 6:00 AM EST- Posted in
- IT Computing
- CPUs
- Bulldozer
- AMD
- Opteron
- Cloud Computing
- Interlagos
Making Sense of the New Interlagos Opteron
This second look at the current Xeon and Opteron platforms added OLAP, ERP, and OLTP power and performance data. Combine this with our first review and the other publicly available benchmark and power data and we should be able to evaluate the new Opteron 6200 more accurately. So in which situations does the Opteron 6200 make sense? We'll start with the perspective of the server buyer.
Positioning the Opteron 6276
First let's look at the pricing. The Opteron 6276 is priced similar to an E5649, which is clocked 5% lower than the X5650 we tested. If you calculate the price of a Dell R710 with the Xeon E5649 and compare it with a Dell R715 with the Opteron 6276 with similar specs, you end up more or less the same acquisition cost. However, the E5649 is an 80W TDP and should thus consume a bit less power. That is why we argued that the Opteron 6276 should at least offer a price/performance bonus and perform like an X5650. The X5650 is roughly $220 more expensive, so you end up with the dual socket Xeon system costing about $440 more. On a fully speced server, that is about a 10% price difference.
The Opteron 6276 offered similar performance to the Xeon in our MySQL OLTP benchmarks. If we take into account the hard to quantify TPC-C benchmarks, the Opteron 6276 offers equal to slightly better OLTP performance. So for midrange OLTP systems, the Opteron 6276 makes sense if the higher core count does not increase your software license. The same is true for low end ERP systems.
When we look at the higher end OLTP and the non low end ERP market, the cost of buying server hardware is lost in the noise. The Westmere-EX with its higher thread count and performance will be the top choice in that case: higher thread count, better RAS, and a higher number of DIMM slots.
AMD also lost the low end OLAP market: the Xeon offers a (far) superior performance/watt ratio on mySQL. In the midrange and high end OLAP market, the software costs of for example SQL Server increase the importance of performance and performance/watt and make server hardware costs a minor issue. Especially the "performance first" OLAP market will be dominated by the Xeon, which can offer up to 3.06GHz SKUs without increasing the TDP.
The strong HPC performance and the low price continue to make the Opteron a very attractive platform for HPC applications. While we haven't tested this ourself, even Intel admits that they are "challenged in that area".
The Xeon E5, aka Sandy Bridge EP
There is little doubt that the Xeon E5 will be a serious threat for the new Opteron. The Xeon E5 offers for example twice the peak AVX throughput. Add to this the fact that the Xeon will get a quad channel DDR3-1600 memory interface and you know that the Opteron's leadership in HPC applications is going to be challenged. Luckily for AMD, the 8-core top models of the Xeon E5 will not be cheap according to leaked price tables. Much will depend on how the 6-core midrange models fare against the Opteron.
The Hardware Enthusiast Point of View
The disappointing results in the non-server applications is easy to explain as the architecture is clearly more targeted at server workloads. However, the server workloads show a very blurry picture as well. Looking at the server performance results of the new Opteron is nothing less than very confusing. It can be very capable in some applications (OLTP, ERP, HPC) but disappointing in others (OLAP, Rendering). The same is true for the performance/watt results. And of course, if you name a new architecture Bulldozer and you target it at the server space, you expect something better than "similar to a midrange Xeon".
It is clear to us that quite a few things are suboptimal in the first implementation of this new AMD architecture. For example, the second integer cluster (CMT) is doing an excellent job. If you make sure the front end is working at full speed, we measured a solid 70 to 90% increase in performance enabling CMT (we will give more detail in our next article). CMT works superbly and always gives better results than SMT... until you end up with heavy locking contention issues. That indicates that something goes wrong in the front end. The software applications that do not scale well could be served well with low core count "Valencia" Opteron 4200s, but when we write this, the best AMD could offer was a 3.3GHz 6-core. The architecture is clearly capable of reaching very high clockspeeds, but we saw very little performance increase from Turbo Core.
What we end up with then is more questions. That means it's time for us to do some deep profiling and see if we can get some more answers. Until then, we hope you've enjoyed our second round of Interlagos benchmarking, and as always, comments and feedback on our testing methods are welcome.
Facebook
Twitter
RSS
46 Comments
View All Comments
tech6 - Thursday, February 9, 2012 - link
We have a number of these in our data center and they have been a disappointment. Single threaded performance is low and the memory sharing performance under VMWare is poor. That leaves it only competitive for DB and web servers work which is OK but it doesn't make a compelling case for the architecture as similar Intel offerings perform well in all tasks. AMD still has a small price advantage, but once you add the VMWare licensing and data center costs the percentage difference is negligible.duploxxx - Thursday, February 9, 2012 - link
Typical answers, I debate such results all the time with many IT departments all over the world so called "standardized on and runs bette on" statements. So you have a large number of these new Opteron 6200 series already in datacenter and already got this info out of it, yeah right. Our virtual datacenters with approx 1000 servers exist out of AMD based systems, the only time i thought about swapping to intel after performance/price/power review was when intel released the Nehalem (oh and perhaps the Socket R but can't disclose that yet, neither do I want to swap that fast knowing that abu dhabi is about to go in Beta samples already). In the low price 2s-4s (not talking about the over expensive EX versions) the AMD are still owning the virtualization with there multicore and more memory channels with lower price.Poor single thread has indeed been an issue for magny cours to a certain extend but not that it is noticable on a normal level of applications (you sound like a superpi user that only looks at theoretical results), poor memory sharing, care that to explain :). I suggest you have a look at general best practices on power settings for virtualized environments before complaining about response and throughput.
These response time result measured here are not noticable in any general application and once adding some kernel overhead from NFS/iscsi or whatever in medium - high load servers forcing these tasks to HT cores will show quite a bit different result in platform performance. Anandtech Vapp results from are nice but are still not a full reflection of datacenter performance and the results are heavy influenced by the webapp which clearly seems to favor the Intel architecture. Neither does have the Vapp testing any iscsi/nfs kernel related tasks which many sites do have (to reduce infrastructure cost)
Vmware licensing cost more for AMD? only for the enterprise edition, the abandoned vmware license version which was introduced again for v4 because of OEM pressure... time to investigate more before buying anything... advanced - enterprise plus has no difference for any core you would select and neither does v5 have.
JohanAnandtech - Thursday, February 9, 2012 - link
NFS/iSCSI: you seem to ignore the fact that besides OLTP, many apps (especially OLAP and web) run mostly out of memory. The whole idea of good data management is to make sure that your requests come out of memory. We have webcaches, database caches, file system caches, RAID caches and on disk caches...all these caches are made to make sure that the response time is NOT dominated by NFS/iSCSI.We have 5 years of experience building our vAPUS stresstesting client (not vApp) so don't discard our results so quickly.
duploxxx - Thursday, February 9, 2012 - link
I won't discard them as I said they are a of great value just like these testing posted today, but that doesn't change the fact that in current vApus stresstesting there is no use of NAS/ISCSI datastores which is very common these days now that 10G is affordable and creates additional overhead on systems, mostly not accounted for when selecting a platform. Sure for the review real performance conclusion are needed from cpu architectures and then this setup remains totally applicable. But you get people which i tried to answer before who take this final score result for granted and leverage it over a total platform as if that is the best choice.Second remains the fact that the final vApus calculation is based on all scores and the web based VM score is unbalancing the final score. I mentioned that years ago when the vApus1 and 2 were introduced, back in the old days.
haplo602 - Friday, February 10, 2012 - link
nothing that features heavy modifications will run out of memory .... do you cache your database redologs ? bad idea.similar for web servers that get heavy modifications.
all the caches exist to only make the transition between respons time jumps less painfull, but they never remove those limits.
if you have a static system (reads 99% of the time) then you can use caches a lot. untill the data set grows too large to cache and is sufficiently random.
Iketh - Thursday, February 9, 2012 - link
This was a fun read. Unfortunately, the thought still in the back of my mind while reading was why in the hell did AMD reinvent the P4??? I just don't get it.I hope the coorporate players who resigned fired some engineers prior...
duploxxx - Thursday, February 9, 2012 - link
very interesting info as usual, thx for the indepth review.In general I think it would be added value to mention what exact BIOS and power mngmnt settings have been set. Since the option exist of using PCC controlled power (BIOS - OEM) or through the OS, also settings like CE6 etc does influence the final results a lot on turbo etc, mainly towards your preliminary review and this one it's not always clear what exactly you have been using.
result wise it is very strange that the 16core does not scale further then the 12core in SQL, for the reference testing that would give clear results when you could have tested the 6234-6238 which are also 12core versions. It is hard to believe that these 16core do not scale further, that they lack about 10-15% IPC @ same ghz sure but not raw performance in core count.
Debatable is the fact that while using Dual rank 4MC on AMD and using 2 rows Dual rank 3MC on the intel, dual row dual rank will give more bandwidth, its not AMD fault they have 4MC vs intel only 3MC.... even Intel next gen will have a serie of 3MC and 4MC..... but this should not result in major differences.
Last point, which IS a fact --> the price compare. While listed price might be comparable, reallity is quite a bit different in retail sales price. For large volume handling the discount between vendor is huge, the discount that Intel allows on it's CPU is way less then AMD, this changes the final cost price a lot.
JohanAnandtech - Thursday, February 9, 2012 - link
"In general I think it would be added value to mention what exact BIOS and power mngmnt settings have been set"Most of them can be seen at:
http://www.anandtech.com/show/5279/the-opteron-627...
C6 was enabled
"dual row dual rank will give more bandwidth"
You can easily reverse that argument: if I use 4 GB DIMMs on the AMD, the clockspeed of the DIMMs will throttle back to 1333. The AMD IMC can only use 1600 MHz with 1 DIMM per channel. So this is really the best case for AMD.
"For large volume handling the discount between vendor is huge"
ok. 1. those people are probably 0.001% of our readership. 2. Those prices are a moving and unknown target.
duploxxx - Thursday, February 9, 2012 - link
yes I have seen those settings, but during the review with so many back and forth testing showing issues on the power and bios settings from a reader perspective it is no longer clear what exact bios settings have been used (ms os - hypervisor) and what has been used and if results were updated with the right BIOS settings.I am not yet convinced that the 1600 is really added value for the 6200 series, perhaps it will be more added value on piledriver with enhancements. While 8GB ram prices did drop it is still not a default selection certainly not on 1600 speed. I did mention debatable memory :) did you test the difference in the Database benchmarks?
Its not only large volume where the discount is even greater, just look at the HP website and order 3 "similar" designs from the same vendor:
dl380 dual E5649 - 8gb ram : 5953$ (add 8gb ram + 150$ approx)
dl380 dual X5660 - 24gb ram: 7178$ (remove 8 gb ram -150$, identical PSU -100$)
dl385 dual 6272 - 16GB ram: 5202$
so you can see quite a bit different price range.
http://h71016.www7.hp.com/dstore/MiddleFrame.asp?p...
http://h71016.www7.hp.com/dstore/MiddleFrame.asp?p...
http://h71016.www7.hp.com/dstore/MiddleFrame.asp?p...
alpha754293 - Thursday, February 9, 2012 - link
Johan:I know that you and I have spoke a little bit offline about possibly doing more HPC and HPC-related testing.
I, for one, would still like to see more of that because I think that's one area that a) is underserved by hardware review sites (sometimes with good reason), b) I think that it stresses the CPUs more/harder, and c) you can create/do/use a consistent benchmark test cases or suite of applications (like you mentioned about the SPEC OpenMP (although they have an MPI one as well, which I think is probably going to be even better).
I think that the biggest downside is that the HPC applications DO take a fairly significant time to run. (Some of them runs for days on end - just to do one pass).
And you can always throw more Hypervisors onto these systems, but I don't think that they're nearly as taxing as when you're running a computationally heavy/demanding application like simulating a car crashing into a wall at 35 mph. :oD
And it's quite possible that you might be able to script the entire benchmarking process...