Measurement

FFTs and wild goose chases#

FFT analysers are fantastic. But many people don't understand them enough to be able to use them correctly. I don't understand the math itself at any meaninful depth. I am not a mathematician and do not pretend to be. But I understand what is happening conceptually - the moving parts from the perspective of an audio engineer - in enough detail to work with existing FFT math libraries in my own codebase and produce fully functioning spectrum analysers. Most plugin developers use existing FFT libraries and aren't sitting there hand-rolling twiddle factor calculations.

For audio engineers, I think it is an easy case to make that it is more important to understand how the result of the FFT is used, rather than understanding the FFT math itself. Specifically, if your analyser is just a sequence of FFT snapshots, like a movie strip, with no overlap, that's more or less unusable for audio work. You have to overlap multiple FFTs to get a decent, smooth result.

The way you overlap is incredibly important: a plain sequential rectangular window creates massive spectral leakage. In other words, the analyser is extremely "noisey". The result of this "noise" (spectral leakage) has a name: sidelobing. To reduce the sidelobing you need to do a kind of "crossfading" / "fading in and out" of each window of audio individually before performing each FFT snapshot in time - called a windowing function, and these different windowing techniques have different names, like Hann, Blackman, Blackman–Harris etc, named after the mathematicians who arrived at them.

And the reason why there are many is because they have different trade-offs. You can have higher precision in one place but that gives you lower precision in another place. The most classic tradeoff is trading sidelobing for mainlobe precision. Put simply, the less "background noise" you want on your spectrum analyser, the wider the peak looks for a single sine wave. You would think a sine wave would be displayed on an analyser as a single very narrow line at the frequency it's playing at, with no other frequencies being displayed - because no other frequencies are playing. But that's not really possible without using significant post processing trickery. Either the sine way looks like a fat bell shape without noise, or a skinny line with lots of noise, or something in between the two, but never a skinny line with no noise.

So someone who doesn't know this will be lead on a wild goose chase, in persuit of sounds which don't actually exist and are just artifacts of the visual representation method.

But that's not the only possible wild goose chase. The second most classic trade off is time VS frequency accuracy. You can have all the accuracy as you want. But only in one domain at a time, and at the cost of the other domain. And the lower the frequency, the tougher it becomes, as FFT bins are space linearly. Meaning, you get only a handful of bins in the bass but half of the bins are between 10k - 20k. So the resolution is excessive in the highs and insufficient in the bass - by nomological necessity.

In short, if you have great frequency precision, the analyser is very slow to respond, you smear all the transients away and you just see a slowly morphing squiggle which seemingly bares no resemblance to the music. If you have go the other way and optimise for speed, so you see the analyser dancing perfectly with the music, you will have horrendous accuracy in the bass frequencies - everything below 200hz will look like one undefined fast moving blob. You can slide between the extremes but ultimately you can't have your cake and eat it.

This leads people astray who set their FFT windows too short: they see DC offset issues and "mud" in the lows which doesn't actually exist, and miss problematic information obscured by the low resolution.

Inversly, people who set their FFTs too long can miss important clues about transient issues, lack a visual connection to the music they are working on and can start seeing "problematic resonances" everywhere, none of which are actually problematic.

Measurement takes the adjectives out#

An analyser doesn't care what the interface looks like or what the plugin costs. It shows what the thing does. So when I evaluate gear or plugins, I prefer measurement to subjective description, and I still pair it with level-matched listening examples.

When I'm not measuring, I say so. When I rank headphones, that is my trained judgement as a mastering engineer, labelled as such and limited to models I've actually heard. Objective data like frequency response, THD and waterfall plots exists and matters. You're free to disagree with a subjective ranking. The point is being clear about which kind of claim you're making.

A visible difference is not necessarily an audible one#

Delta Expose is sensitive enough to show my own compressor going fuzzy when I switch on oversampling, which you don't normally hear. So before a measured difference is called meaningful, it has to be checked against hearing.

The just noticeable difference depends on frequency and level. With sine tones at around 80 dB SPL it averages about 0.5 dB, and anything under a quarter of a dB can safely be disregarded. Low bass at low listening levels has a very large JND. Single sine-tone data isn't fully representative of music, so I treat it as a guide. And many adults can't hear above about 17 or 18 kHz anyway.

Whether a difference is better is a separate question#

When a processor does something unidentified, I measure it and say that it does something. That doesn't tell me whether that something is good. In my EQ tests one emulation was clearly doing some unspecified thing to the sound. It might have been an improvement. It might have been worse. Existence and value are two different judgements.

A frequency response chart is not the whole picture#

"The chart doesn't lie, I did a sine sweep at home and it looked flat." A flat frequency response chart doesn't tell you a speaker is accurate in the time domain or low in distortion, and that is where small ported speakers give themselves away. With ports and passive radiators, the response changes when you turn the volume up or down. Most people don't know that.

Firm conclusions need an anechoic chamber or a better setup than a room.

Simple test signals#

Compressors can't be nulled the way EQs can, because their behaviour depends on many factors and you cannot reliably isolate enough of a compressor to perform any meaningful comparisons, You can compare their gain-reduction curvature instead. My old CompView plugin did that with a single high-frequency test tone, and I was well aware of its shortcomings: if a compressor's character came from how it reacts to frequency, a 12k tone wouldn't show it. Some people pushed back that a single sine wave isn't harmonically complex enough. But in Delta Expose the setting I recommend is a raw, aliasing square wave modulating at different levels that you can sweep across the whole range. That tests frequency and program dependence quite well, at a resolution of one sample per pixel. See null testing.

Ears need training for the time domain#

Many people have come to be able to identify frequency based problems. With the help of a spectrum analyser you can even point at the problems on your screen. A hi-hat that is way too loud can even be obvious to someone who's never touched music. But hearing looseness in bass, ringing or smeared transients takes a bit more training. And that difficulty is further compounded by the fact that poor room acoustics destroys time domain accuracy. I use the kick and sub pattern on Radiohead's "Feral" to demonstrate this on speakers.