Audio Descriptors Extraction

Programming Enviroment used fot this purpose : *SuperCollider*

*Spectrograph Data:*A spectrogram, is a visual representation of the spectrum of frequencies in a sound or other signal as they vary with time or some other variable. Spectrograms are sometimes called spectral waterfalls, voiceprints, or voicegrams. Spectrograms can be used to identify spoken words phonetically, and to analyse the various calls of animals. They are used extensively in the development of the fields of music, sonar, radar, and speech processing, mology, etc. The instrument that generates a spectrogram is called a spectrograph. The sample outputs on the right show a select block of frequencies going up the vertical axis, and time on the horizontal axis.

from: http://en.wikipedia.org/wiki/Spectrogram

Loudness: A perceptual loudness function which outputs loudness in sones; this is a variant of an MP3 perceptual model, summing excitation in ERB bands. It models simple spectral and temporal masking, with equal loudness contour correction in ERB bands to obtain phons (relative dB), then a phon to sone transform. The final output is typically in the range of 0 to 64 sones, though higher values can occur with specific synthesised stimuli.

Onsets detector: An onset detector for musical audio signals - detects the beginning of notes/drumbeats/etc. Outputs a control-rate trigger signal which is 1 when an onset is detected, and 0 otherwise. More at D. Stowell and M. D. Plumbley. Adaptive whitening for improved real-time audio onset detection. Proceedings of the International Computer Music Conference (ICMC'07), Copenhagen, Denmark, August 2007. http://www.elec.qmul.ac.uk/digitalmusic/papers/2007/StowellPlumbley07-icmc.pdf

Spectral centroid: Given an FFT(The fast fourier transform analyzes the frequency content of a signal, which can be useful for audio analysis or for frequency-domain sound processing (phase vocoder)) chain, this measures the spectral centroid, which is the weighted mean frequency, or the "centre of mass" of the spectrum. (DC is ignored.) This can be a useful indicator of the perceptual brightness of a signal.

Spectral Flatness measure: Given an FFT chain this calculates the Spectral Flatness measure, defined as a power spectrum's geometric mean divided by its arithmetic mean. This gives a measure which ranges from approx 0 for a pure sinusoid, to approx 1 for white noise. The measure is calculated linearly.

SuperCollider Code

-----------------
MACHINE LISTENING
-----------------

RESPONDERS
----------
r = MakeResponders.new;
r.all;
r.removeResponders;

AMP-FREQ
--------
a = SendAmpFreq.new;
a.start;

ONSETS
------
b = SendOnsets.new;
b.start;
b.stop
b.threshold(1);

SPECTROGRAPH
------------
e = SendFFTdata.new
OF.playSpectro("mirrorMode", 9); // 3,4,7,9
e.changeRate(0.04)
e.start
e.changeIntensity(1)
e.stop

LOUDNESS
--------
e = SendLoudness.new
e.start
e.impulseRate(1)
e.stop

SPECTRAL CENTROID
-----------------
c = SendSpecCentroid.new
c.dump
c.start
c.impulseRate(1)
c.stop;

SPECTRAL FLATNESS
-----------------
d = SendSpecFlatness.new
d.start
d.impulseRate(1)
d.stop

links