Retesting the ASUS Rampage IV Black Edition Audio and the Pitfalls of Audio Testing
by Ian Cutress on January 31, 2014 4:45 AM EST- Posted in
- Motherboards
- Asus
- Audio
- Audio Precision
In the motherboard section of AnandTech, we rarely touched the audio section of the motherboard until fairly recently. Audio is a delicate issue, and knowledge on the subject falls into a wide range from the tone deaf all the way to the hyper enthusiast. Professional and prosumer audio equipment can be honed to perfection and cost an appropriate amount, where discussion on quality over $10k per meter cables can happen. It is very hard to quantify an audio experience into a single set of rules, but as the motherboard arena can feature rather elaborate audio setups designed to enhance the quality or provide sufficient electrical filtering, and that these can be some of the major selling points, several of our readers asked us to start testing the audio.
To put this into perspective, a motherboard manufacturer can create an audio portion of the motherboard that costs $2 to buy and implement. Another manufacturer will spend $20 to get a better audio codec and DAC, better audio filters, headphone amplifiers, and specifically separate the digital and audio traces on the motherboard as to reduce potential sources of interference, as well as focusing on a single output to achieve all this. Part of testing these scenarios is producing such a test that a reader can relate to, as well as a series of numbers for comparison to put in the review – both manufacturers and readers want to see that their hard earned engineering money and effort went into something that can be compared.
In light of this request to test audio, looking into the best way to test it is a minefield. The correct equipment in order to get an accurate test of dynamic range, noise and distortion costs over $10k – for example, the audio equipment used in Chris Heinonen’s recent foray into smartphone audio testing was an Audio Precision APx582, which starts at eleven thousand. Audio Precision is often cited as the company that all audio manufacturers go to in order to test their hardware. Unfortunately unlike Chris I am not able to source Audio Precision testing equipment due to my location and contacts, but having the hardware on hand takes everything out of the equation except the audio generator and filtering pathway that a manufacturer decides to use.
One ‘quick and easy’ test that has been in the mix and in several press pack guides is Rightmark: Audio Analyzer (RM:AA). This is a software solution to what is ultimately a hardware problem, and relies on the same hardware doing both the signal production and the signal receiving. This causes a number of issues, such as channel leakage, harmonic resonance in adjacent signals and that the limiting factor of the hardware being testing might affect other parts of the test. More often than not it is also difficult to compare results between products, due to input levels and driver settings. In order to provide accurate and comparable results, the software has to be explicitly set up for consistency and comparison, something a lot of reviewers that use RM:AA do not do.
I must confess that for a while I was in this crowd. My RM:AA test was ultimately out of the box and produced wildly conflicting results. When audio solutions were quoted as 108 dB for signal-to-noise ratio were getting 92 dB, the conclusions were along the lines of differences in signaling and manufacture of the motherboard. In reality, the test was very wrong – the OS by default limits input/output to 16-bit + 44.1 kHz, producing an artificial ceiling, and the volume ratio of output/input in the testing loop was not standardized. While I kept the input/output the same, in fact each output of the audio jacks can have different responses to the audio in question.
After delving further into the topic, we standardized the RM:AA test at AnandTech. The OS is now set to 24-bit, 192 kHz input and output (or the highest possible based on the audio codec options), the output volume was set to 100% and the input volume adjusted to provide accurate results without over-distorting the input. Ultimately our testing is still limited to the level of the input on the codec, often cited around 104 dB SNR compared to a 108 dB or 112 dB peak output, depending on what audio codec + DAC are used. Even when the same 3.5mm shielded cable is used or different tests, cable losses in a uncontrolled environment can be excessive (1-2 dB SNR), and my location next to a busy main road might also introduce error.
This test is in no way perfect – we typically quote the SNR and THD+N (Total Harmonic Distortion + Noise) results as these were ultimately relevant to those who wanted cleaner and louder output, rather than a quality control test.
Recently we tested the ASUS Rampage IV Black Edition. In this review, we noted that for ASUS’ audio solution, they used their SupremeFX technology – a shielded audio codec with separated tracing, audio filters, headphone amplifiers and gold plated connectors. In this, ASUS use a Realtek audio codec to power a Cirrus Logic DAC, rated at 120 dB SNR and -107 dB THD+N. Our results were well below this in terms of SNR, ~95 dB, which is substantially lower. At the time I was unsure of the reason, and needless to say ASUS got back to me to confirm my testing methodology.
After retests on my end, and a more professional test from ASUS, we exchanged new numbers for comparison. My initial numbers were indeed erroneous. After retesting, the ASUS Rampage IV Black Edition falls in the middle of the pack:
ASUS and I compared our SNR graphs to see the differences:
ASUS achieve a -108 dB SNR compared to my -101 dB SNR result due to ASUS using Audio Precision testing tools and my test using the Line In + RMAA methodology. The RMAA test also relies on a 1 kHz pulse at -60 dB for its test rather than a range of frequencies, which as a result causes the peak seen at 1 kHz.
In light of the discrepancy, we are coming to a fork in the road for audio testing. I fall wholeheartedly into the ‘tone deaf’ part of the crowd when it comes to analyzing audio and prefer an easy-to-read number as a comparison, although I understand this is not the be-all and end-all when it comes to audio. Ideally I would like to use Chris’ setup for future motherboard tests. I have asked Chris to perform testing on a few select models if he has time in the future, which might be in the works. But I do not have access to an AP testing setup, and my current solution sits with RM:AA. ASUS has offered to provide their results in future press packs, although independent verification is what we want to report. In light of motherboard development we may see other manufacturers doing it also. In order to improve our reporting in this area, I will in future publish the SNR graphs we receive from RM:AA for comparison.
Facebook
Twitter
RSS
59 Comments
View All Comments
tim851 - Friday, January 31, 2014 - link
> I fall wholeheartedly into the ‘tone deaf’ part of the crowd when it comes to analyzing audio and prefer an easy-to-read number as a comparison, although I understand this is not the be-all and end-all when it comes to audio.<The problem is once you go beyond easy-to-read numbers, audiophiles go crazy and start forming sentences that contain more fluff than an Apple press release. I acknowledge that there is difference in sound quality of across devices, but the willingness of audiophiles to hear unicorns sing is amazing. The snake oil salesmen of old have all switched to audio gear, for sure.
So yeah, stick to numbers.
YukaKun - Friday, January 31, 2014 - link
Well, the fire proof test I have is to use Studio Monitors (Active ones) to test if the sound card is good or not (among other things, of course). They usually detect noise and other artifacts the sound cards produce right away, so you can either cry or give the sigh of relief.That being said. I've managed to nail down that a MoBo can be said to have a decent sound card if it posses at least an ALC892 in it. This chip/codec contains very good sound processing and options to actually enjoy anything (specially from the HTPC perspective). I also have a Creative Sound Blaster X-Fi HD USB and it very worth its price tag, even not being 5.1 or 7.1. If the sound card sucks, I attach this baby and fix everything.
Cheers!
Mr Perfect - Friday, January 31, 2014 - link
I would agree, please stick to objective testing and quantifiable numbers like SNR. Subjective audio testing really can get bizarre. As soon as reviews start commenting on how "warm" or "brown" a sound card is, I'm going to start skipping the audio sections.bim27142 - Monday, February 3, 2014 - link
Honestly, I would prefer subjective audio listening for as long as comparison and description is clear and understandable... Let's say, subjectively the reviewer will compare the audio output of this motherboard's audio to a, say, Asus Xonar STX or even just the DX... of course, quality difference will be tremendous but at least from there, I can have an idea how it really sounds... numbers are just numbers...Sabresiberian - Friday, January 31, 2014 - link
Your analysis of the situation is born out of ignorance and your own bias. Numbers are still an inaccurate substitute for ears. We have a situation here something like the reporting of frames per second as a measure of video quality - it is a factor, but hardly tells the whole picture, as has (finally) been proven recently with the focus on measuring consistency in frames started by The Tech Report. SNR and THD are good numbers to know, but they are just a starting point for evaluating sound quality and a rather small part of the big picture.It is no fault of Anandtech or any hardware site that better analysis isn't done, as it is expensive, time-consuming, and until the last couple of years not all that important in mainboard testing. However, since Asus and others are now claiming they have some kind of better sound solutions on their boards, it is good to have as much information as we can. We can't yet get a total package of measurements that will indicate clearly what we will hear, and a site like this getting involved in subjective impressions (meaning using their ears to measure information they can pout in a review) is difficult, so reporting the numbers that are possible is a good thing, but hardly paints a complete picture.
I'm sure there is plenty of "snake oil" in the audio business, as there is in all businesses, but it isn't to the degree you indicate. People should buy with their ears and their budget, and if an inexpensive on-board sound solution works for you, good news, you can save a lot of money. Others have better ears and want to get more out of their listening experiences, sneering at them because you can't hear what they do is childish at best.
argosreality - Friday, January 31, 2014 - link
There's no if's, and's or buts about snake oil in the audio business - it flat out exists. Numbers don't like (but can be misreported or misrecorded as this article indicates) but ears...ears *DO* lie. They lie for a number of reasons. Genetics, age but even more...what you want to hear. Spend 10k on those speakers? You're going to damn well believe they sound amazing. that 1k wooden nob that "isolates" the sound of your receivers volume? Yea, you want it sound good.It also boils down to personal preference. Some people want the most accurate sound reproduction possible. Others want a warm, natural sound. Some just want booming bass. To each of those, their "ears" will find issues. The numbers, the science? They'll show the truth. So, don't let your own ignorance and bias color YOUR responses
JlHADJOE - Friday, January 31, 2014 - link
IMO what really needs to be done is to find a way to quantify these aspects of audio that are currently hidden behind fancy words. Aside from SNR and THD+N, things like "attack" can actually be seen by plotting the square wave response of a DAC or amplifier.So that's one snake-oil buzzword down, now what we need to do is try and get the rest of them.
We can render very detailed plots from frequency/levels using raw digital files. Now run the same file through the audio equipment, capture the output and see how the new plot compares to the pure digital original and we have an objective way of seeing exactly how the equipment changes the original waveform, and slowly, but almost certainly we can begin to find what quantifiable differences there are (or the lack thereof) when an audiophile reviewer says something like "airy", or "forward".
ThreeDee912 - Saturday, February 1, 2014 - link
There's a good intro to DACs and digital signals from a guy at Xiph, which are the same guys that write the FLAC audio codec. It addresses some common myths about digital audio, like "stairstep" samples, bit-depth, and dithering. Even just from this ~24 mins of video, you can see a lot of silly audio rumors still floating around online today are absolutely false.http://xiph.org/video/vid2.shtml (needs Chrome or Firefox)
TruePath - Saturday, February 1, 2014 - link
It's a little more complicated than that. After all the original was no doubt recorded (then post-processed) in a studio. You are playing that audio back in a very different environment with a different speaker setup and most likely without the kind of professional quality microphone used in the studio.So, sure, it's that simple for the studio to evaluate and tweak their audio setup. Simply use a high-quality mic and make sure the original sound matches the playback as closely as possible at that microphone (repeat with the mic at other points you care about playback quality).
But since you don't have the original room nor the original mic this begs the question of WHAT sound you should be reproducing. Different studios are likely to take the exact same original and capture it in a slightly different manner depending on the type and number of microphones (where it's sensitive) they use and other factors. So how do you even define a standard against which your sound can be compared?
I tend to think the best way is to insist on binaural recordings (recordings made by mics placed on each ear of a dummy human head) and standardize on a particular kind of mic that should be used to compare against the original. Of course, how you weight different distortions is going to make a bunch of difference but at least you have a clearly defined notion of perfect reproduction to start from.
CalaverasGrande - Thursday, February 6, 2014 - link
Attack can be quantified by slew rate and rise time.I'm happy you brought that up. I have a nice little home studio based around a relatively ancient Motu 828MKII firewire soundcard. These boxes have reasonable sound quality, and work really well with Macs.
Mine is not stock, it has had the audio opamps and digital clocking upgraded to tighter specs.
Ironically it actually has slightly worse SNR now. However the THD+N is much lower, and the bandwidth gained an octave on either side.
There is nothing subjective about things rattling off your desk, or being able to record frequencies higher than I can hear.
This brings up my next point. A lot of mobos have optical outs. Those are neat, but they are next to useless if the signal is so jittery that it sounds worse than the analog signal, or worse, is so jittery it can't communicate the audio to your AV device.