Updated , Sam Vafaei

Our Sound Quality Score and Tests
Headphones

What it is: How accurately the audio is reproduced. The tests are performed with the headphones' most commonly used features enabled (noise-cancelling, wireless, etc.)
Score distribution

Sound quality is how accurately audio is reproduced as intended by the producer/engineer. Our reference for headphones sound quality is the loudspeaker. The ideal headphone sounds like the ideal stereo speaker setup in the ideal room.

From music and movies to podcasts and spoken word, audio is arranged/mixed in a studio to have a distinct sound, and accurate headphones are able to reproduce that sound as intended. Though some listeners might prefer a more hyped bass or mid-range, in our tests, we consider an accurate and neutral reproduction to be more desirable.

For our sound quality score, we evaluate the bass, mid-range, and treble frequency response, as well as frequency response consistency, total harmonic distortion, soundstage, and imaging of the headphones we test.

Test results

Our tests

Bass

What it is: Frequency Response from 20Hz-250Hz
When it matters: When the material is heavy on bass frequencies, such as those of kick drums and bass guitar.
Score components:
Score distribution

Bass frequency response describes how accurately headphones reproduce the low-frequency region of the audible frequency spectrum. Bass ranges from 20Hz up to 250Hz and represents the low thump/rumble, punch/kick and melodious basslines you can hear in tracks.

Good bass does not overpower the presence of instruments and vocals and adds excitement to tracks that would otherwise sound weak or thinned out. In our frequency response score, bass is assigned the same weight as treble and the mid-range.

Learn more about this test

Mid

What it is: Frequency Response from 250Hz-2KHz
When it matters: When the material is heavy on mid-range frequencies. This is the case for the majority of audio content.
Score components:
Score distribution

Mid-range frequency response describes how accurately headphones reproduce the mid region of the audible spectrum. The mid-range spans 250Hz to 2KHz and represents the lower and higher harmonics of instruments and vocals, as well as their comprehensibility and clarity.

The mid-range is where the bulk of the audible audio frequencies reside, and when unbalanced, causes the instruments and vocals in music to sound thin, distant or muddy/cluttered. In our frequency response score, the mid-range is assigned the same weight as treble and bass.

Learn more about this test

Treble

What it is: Frequency Response from 2KHz-20KHz
When it matters: When the material is heavy on high-range frequencies, such as voice, cymbals, and any other material where brightness, brilliance and airiness is desired.
Score components:
Score distribution

Treble frequency response describes how accurately headphones reproduce the high frequencies of the audible frequency spectrum. Treble ranges from 2KHz to 20KHz and represents the higher harmonics of lead instruments and vocals, cymbals, the sibilant tones (S and T sounds) and the airiness you can hear in tracks.

When treble is lacking, the higher harmonics of instruments and vocals lose detail and brilliance. This is significant, as the absence of good treble may make audio sound dark and lacking detail and presence. In our frequency response score, treble is assigned the same weight as the mid-range and bass, even though very high frequencies are less audible to older listeners.

Learn more about this test

Frequency Response Consistency

What it is: The amount of deviation of each frequency response pass, from the average frequency response.
When it matters: Shows how consistently the headphones perform after re-positioning them.
Score components:
Score distribution

Frequency Response Consistency describes the variations in a headphones' frequency response due to their fit on your head. Headphones will sound slightly different depending on the size and shape of your head, how they interact with your ears and whether you wear glasses.

Frequency response is the most important part of a good audio reproduction. Although most listeners naturally compensate for variations in frequency response caused by their unique features, such as the shape and size of their heads, other factors like their preferred headphone position, and how the ear cups' acoustics interact with their ears, may cause additional variations in the response. An inconsistent headphone may have a drastically different sound from listener to listener. This means a headphone that will sound bass-heavy for most could lack a lot of bass for listeners who wear glasses. The ideal headphone should consistently reproduce the same response regardless of positioning or head/ear shape.

Learn more about this test

Total Harmonic distortion

What it is: The subtle, unwanted frequencies (harmonics) produced alongside the intended frequencies.
When it matters: When clean and pure sound reproduction is desired, though its effect is not as noticable as frequency response.
Score components:
Score distribution

Total harmonic distortion is a flaw in audio reproduction that generates unintended frequencies that reduces the clarity of the intended sound. For examples, if a 100Hz signal is fed to a headphone with second order harmonic distortion, it will produce a 200Hz tone in addition to the intended 100Hz tone. The higher the amount of harmonic distortion, the higher the level of the 200Hz tone will be, relative to the 100Hz signal. At high levels of harmonic distortion, audio quality deteriorates because the unwanted frequencies make tracks sound unclear and fuzzy.

Luckily, harmonic distortion is hard to perceive by ear alone and although noticeable at higher amounts or by very sensitive ears, for most people it’s rarely audible. This is why we have not assigned much weight to this test.

Learn more about this test

Soundstage

What it is: Soundstage qualities are not inherent to the audio content, the headphones have to 'create' them rather than 'reproduce' them. They determine whether the sound is perceived to be coming from inside or in front of the head, how open and spacious the soundstage is, how much the headphones acoustically interact with the environment, and how strong the phantom center is.
When it matters: When an accurately produced, large and spacious soundstage, similar to that of a stereo loudspeaker setup is desired.
Score distribution

Soundstage describes the perceived location, size, and environment where the music/sound is happening. Headphones that can reproduce this effect give the impression that sound is coming from outside in a room, rather than inside a vacuum in your head. A headphone with a good soundstage will sound more like speakers in a room compared to headphones on your head. 

Soundstage gives spaciousness to an audio reproduction, which sounds more natural and open. This makes soundstage an important component of sound quality.

Learn more about this test

Imaging

What it is: Imaging qualities are inherent to the audio content, the headphones have to 'reproduce' them rather than 'create' them. They determine how accurately the objects are positioned in the stereo image, and how transparent the imaging is.
When it matters: When accurate positioning of the objects in the stereo image, and clear and transparent imaging is desired.
Score distribution

Imaging describes the accurate reproduction of location, stereo width/balance, and transparency of instruments/objects, in the soundstage as intended by the audio source. Headphones with good imaging reproduce the slight time and amplitude differences between the L/R channels of the audio, which are responsible for generating a stereo image.

However some aspects of imaging are very hard to notice for the average listener and therefore, imaging may not be as important for everyone.

Learn more about this test

What is not included

A few elements that you could care about are not included in the score:

  • Format of audio (Compression, bitrate, digital, analog, etc...)
  • Source of audio output
  • Impedance and Driveability (Coming soon)
  • CSD (Cumulative Spectral Decay)
  • Dynamic Range Compression

If you feel there is an item missing that should be included, please let us know in the Q&A section.

Conclusion

Sound quality is how accurately audio is reproduced compared to the original audio arrangement. An ideal headphone's audio reproduction sounds like a pair of loudspeakers in a room and is balanced and accurate to the source material as it was intended to be heard. For sound quality, we evaluate the bass mid and treble frequency response, as well as frequency response consistency, total harmonic distortion, soundstage, and imaging. Headphones that score highly for sound quality will sound great no matter what you are listening to.

 

Questions Found an error?

Let us know what is wrong in this question or in the answer.

Email:

Questions & Answers

4 ANSWERED QUESTIONS
5

Will you be reviewing more open-back headphones? What about some planars? I am curious about the following: HiFiMan HE400s, HE400i, HE560, Oppo PM-2, Audeze EL-8, and Audeze LC-2.In regards to dynamic open-back, a few headphones of great interest are: Sennheiser HD600, HD650, AKG 7xx (previous massdrop), and AKG 702 Anniversary edition (basis of current massdrop). And of course the Audeze Sine (closed planar).

It would be great to create some reference tests for soundstage. IMO, soundstage and imaging are what make music immersive, and it's quite difficult to engineer great sound with immersiveness. Drivability/matchability would be another good metric, with a subscore for phone/tablet pairing.

Thanks for your suggestions.

We are currently in the process of refining our soundstage and bass response tests. That's why we are holding off on more high-end open-back headphones until we can better measure the qualties that sets them apart from lower-end models. Planar headphones are especially interesting for our new sounstage and bass response tests, so it's likely that we will review some in the coming months.

Adding impedance and drivability-related tests are also on the agenda, as well as publishing articles on our testing methodology and philosophy. However, these are of a lower priority and will be added after sound-quality-related tests.

1
Is it just me, or do your score components, equal 99%? ( I know I got a D- in beginning algebra, but c'mon..lol )
This is a rounding error in our algorithm. We are aware of the issue and working on a fix.
0
I suspect that the way you compute scores from the various parts of the frequency response is a bit overly simplistic with regard to the way the human ear perceives variations in frequency response. Specifically, it is well-known that when it comes to resonances (i.e. frequency response peaks), low-Q (broad) variations are way more audible than high-Q (narrow) variations. Probably the most well-known study on the subject is the one by Toole, Floyd E., "The Modification of Timbre by Resonances: Perception and Measurement", JAES 36, 1988. What this means in practice is that, for example, a headphone that is wiggling around in the treble (as most headphones do) is way less objectionable than a headphone that has a somewhat smooth, but tilted, response. At the most extreme, an overall bass/treble balance tilt is the most audible of them all (the study above showed that it's possible to notice a wide spectral tilt of only 0.1 dB!), and it's going to be way more audible than narrow peaks and valleys in the response. The problem with your calculated score is that it is based on a standard deviation calculation, which is blind to these subtleties. Indeed, when calculating standard deviation, a series like [-10, 10, -5, 5] (to take a small example) will yield exactly the same value as [-10, -5, 5, 10], but if these were numbers from a frequency response a human subject would not agree that these sound as good as each other: the first series is simply hovering around a central value, while the second series is showing a clear overall tilt. They will be perceived very differently, and the deviations in the second series will be significantly more audible than in the first series. Another phenomenon that your scoring function does not take into account is the well-known fact that peaks in the frequency response are more audible than valleys or dips. See Bücklein, Roland, "The Audibility of Frequency Response Irregularities", JAES 29, 1981. In light of these findings I hope you'll agree that your scoring function has room for improvement, and perhaps it would be possible to come up with more appropriate math that would better account for the psychoacoustic properties of the human ear.
That's correct. At the moment our scoring is a weighted standard deviation, with the weight curve being quite similar to the equal loudness curve (less weight to low-bass/high-treble). However, adding more perceptual components to our scoring algorithm, like the ones you pointed out, is something we have been meaning to do for a while. Chances are that'll be one of the high priority tasks on the list after we publish our microphone test results.
0
Most of the headphones that have been reviewed, if not all, have the ability of being a little tuned by using an equalizer. Since I couldn't find any information about it, I just wanted to know if you tune the headphones that you review or you just get them out of the box and test them.
We measure headphones with their factory settings and in their mostly used configuration. So a wireless noise cancelling headphone that comes with an EQ will be measured wirelessly with noise cancelling enabled, and EQ set to default (flat).
Questions Have a question?

Before asking a question, make sure you use the search function of our website. The majority of the answers are already here.

Current estimated response time, based on # of pending questions: 3.5 business days.

:
:
A valid email is required. We answer most questions directly by email to prevent cluttering the site.