Hardware Profiling - The math behind the idea

Screenshots on this post are taken from my complete write-up.

The measurement system is implement as a JUCE standalone desktop application targeting macOS. It provide a real-time audio enginer, a structured capture workflow, and an offline analysis engine that fits the L-N-L model. This model can then be interpreted for accurate plugin recreation of the device.

The amount of distortion added by an analog unit is different from setting to setting, so we must design an adaptive process to measure what those spectral differences are to recreate them. The adaptive filter model is a gray-box approach, it fixes the model topology from physical knowledge of analog circuits while estimating parameters from data.

For this project, I use a second-order Volterra kernel to capture the nonlinear memory effects that the memoryless table alone cannot model.

The device is modeled as a a Linear-Nonlinear-Linear cascade where h1 and h2 are the FIR impulse responses and N is a static point-wise nonlinearity.

Frequency-Response Estimation via Wiener Filter

For each SineSweep capture pair (x,y), both signals are zero-padded to the next power-of-two length M and their DFTs are computed. The complex cross-spectrum and reference power spectrum are accumulated over P sweeps:

The Wiener magnitude estimate is

Waveshaper Table Fitting

For each sine-tone capture, a 65536-point Hann-windowed FFT is taken of the central region of the recording (avoiding onset and release transients). The fundamental bin k0 is located by peak-finding within ±10 bins of the nominal frequency [?]. THD is computed across harmonics 2–9:

Where A_h is the peak magnitude near harmonic h. THD is reported per gain label to characterize the onset and degree of saturation. This process essentially maps our different harmonic distortion levels to the gain stage they were at on the original device, ideally mapping the overall tonal color to match the original device when set to the original settings.

Scatter Accumulation

The waveshaper table has T = 1024 bins, uniformly spanning x E [-1, 1]:

For every sample pair (xref[n], yrec[n]) in the analysis window:

Where y is the DC offset of the recording segment, removed to prevent interface or device offsets from biasing the table.

The populated bins are averaged, and gaps are filled by linear interpolation between the nearest populated neighbours. Extreme unpopulated bins are extrapolated using the local slope with an exponential taper to prevent unbounded growth:

L2 Output Filter Identification

Any residual linear coloration after L1 and the waveshaper is identified by running the Wiener estimator on the synthesised L1 ◦ N output versus the actual recording. The L1 convolution is computed via the overlap-add algorithm (block size B = 4096) for O(N log N) complexity.