Facebook's "Open Compute" Server tested
by Johan De Gelas on November 3, 2011 12:00 AM ESTThe Facebook Server
In the basement of the Palo Alto, California headquarters, three Facebook engineers built Facebook's custom-designed servers, power supplies, server racks, and battery backup systems. The Facebook server had to be much cheaper than the average server, as well as more power efficient.
The first change they made was the chassis height, going for a 1.5U high design as a compromise between density and making the server easier to cool. 1.5U allows them to use taller heatsinks, larger (60mm) lower-RPM fans than the screaming 40mm energy hoggers used in a 1U chassis. The result is that the fans consume only 2% to 4% of the total power, which is pretty amazing as we have seen 1U fans that can consume up to one third of the total system power. It seems that air-cooling in the Open Compute 1.5U server is as efficient as the best 3U servers.
At the same time, Facebook Engineering kept the chassis very simple, without any plastic. It makes the airflow through the server smoother and reduces weight. The bottom plate of one server serves as the top plate for the server beneath it.
Facebook has designed an AMD and an Intel motherboard, both manufactured by Quanta. Much attention was paid to the efficiency of the voltage regulators (94% efficiency). The other trick was again to remove anything that was not absolutely necessary. These motherboards have no BMC, very few USB (2) and NIC ports (2), one expansion slot, and are headless (no videochip).
The only thing that an administrator can do remotely is "reboot over LAN". The idea is that if that does not help, the problem is in 99% of cases severe enough that you have to send an administrator to the server anyway.
The AMD servers are mostly used as Memcached servers, as the four channels of AMD Magny-cours Opterons 6100 are capable of using 12 DIMMs per CPU, or 24 DIMMs in total. That works out to 384GB of caching memory.
In contrast the Facebook Open Compute Xeon servers only have six DIMM slots as they are used for processing intensive tasks such as the PHP "assembling" data servers.
Facebook
Twitter
RSS
67 Comments
View All Comments
fpsvash - Thursday, November 3, 2011 - link
In the middle of the paragraph below the image caption, the sentence reads "...and offers better slightly better performance..."Other than that, nice post!
ahmetmy330 - Thursday, December 29, 2016 - link
this is my university work about open servers. Thanks for posthttp://www.h2kjamaica.com/maddenmobilehack/
InternetGeek - Thursday, November 3, 2011 - link
It's interesting that no many players have taken a look at Open Compute.alent1234 - Thursday, November 3, 2011 - link
it's a solution for a specific workload. there are still a lot of workloads that require the traditional model of big database serversunlike your bank, facebook's noSQL is not ACID
FunBunny2 - Saturday, November 5, 2011 - link
Well, yes a voice of reason. OTOH, the Facebook et al folks are convinced that their back to the COBOL era is the future. As if a toy application, albeit pervasive, is "innovation".Sivar - Saturday, November 5, 2011 - link
It's a little difficult to look at a comment about Facebook being a toy application and take it seriously. Yes, Facebook is not directly processing bank transactions on a Tandem, but their site is used to conduct business -- and is even the basis for many businesses, all over the world.Zynga, the company that makes a few annoying games for Facebook, is worth $15 -- more than Electronic Arts.
Nearly every major online publisher, including Anandtech, uses their API for content distribution and often as the entire forum system for discussion of publications.
The founder is the youngest billionaire in history.
Calling theirs a toy application sounds like a Blockbuster customer calling Redbox a toy. It's denial of an obviously successful, large, powerful, innovative company because they don't do things "the old way."
I suspect what matters more is that the business is executing flawlessly, the actual problems with data loss or other non-ACID compliant traditional issues are minimal, and that they are making enough money that Google and Microsoft are feel seriously threatened.
One last thing -- if you really look into what ACID compliance means (and I know you did not specifically mention the acronymn, but replied to someone that did) none of the current major DBMS's are truly ACID compliant. It's too slow. Not Oracle. Not MSSQL. Not Greenplum. Not Teradata. None of them. They may be closer than NoSQL or the like, but then it's all about the right tool for the job, right?
Ceencee - Wednesday, November 9, 2011 - link
This is true but ACID can be over-rated for many workloads. How many pieces of data HAVE to be consistent across the entire cluster to be valid? What about NoSQL with configurable consistency like Cassandra?NoSQL databases provide the holy grail of system growth which is horizontal scaling and this is no small thing for anyone who has worked with a very large RDBMS like ORACLE and implemented RAC to find it doesn't scale all that linearly for most workloads.
ahmetmy330 - Sunday, January 1, 2017 - link
ctually i assume you might have actually stumble upon these dubious <a href="http://www.h2kjamaica.com/bowlingkinghack/"&g... King cheats </a>programs and software packages downloads but are convinced usahmetmy330 - Thursday, January 12, 2017 - link
http://www.onslow.us/8ballpoolhack/I have told you
ac2 - Thursday, November 3, 2011 - link
Wouldn't the presence of the graphics on the HP server account for the 32W idle load savings?