This page was generated from notebooks/mesonic-audification.ipynb.

Audification using mesonic

For Audification we can simply use a Buffer and the realtime mode of the Context

• The Audification will be mainly happening in the backend as the Synth defines what can be controlled

• This means that knownledge of sc3nb / SuperCollider is a requirement to create custom Audifications.

• Nevertheless the Timeline will store the interactions with the Synth. This could be used

  • to document what was done

  • to create a Record or non-realtime rendering of an interactive Audification session

  • to make use of a Synth using Granular Synthesis that allows controling the location of the Playback

Basic Audification

Let’s start with creating a mesonic Context.

import mesonic

context = mesonic.create_context()

Starting sclang process... Done.
Registering OSC /return callback in sclang... Done.
Loading default sc3nb SynthDefs... Done.
Booting SuperCollider Server... Done.

context.enable_realtime();

In this example we will use the EEG data from the Supplementary material for “sc3nb: a Python-SuperCollider Interface for Auditory Data Science”

import numpy as np

data = np.loadtxt("./files/epileptic-eeg.csv", delimiter=",")

We can simply create a stereo Buffer using this data

buf = context.buffers.from_data(data[:,[0,1]], sr=256)

And create a default Synth to play it back

buf_synth = context.synths.from_buffer(buf)

buf_synth.start(rate=20)

However the default Synth might not offer all the features required by the user.

buf_synth

Synth(sc3nb_playbuf_128, {'out': 0.0, 'bufnum': 128.0, 'rate': 20, 'loop': 0.0, 'pan': 0.0, 'amp': 0.3})

Audification using custom Synths

Let’s see what Synths names are known by the backend and what the corresponding Ugen graph is.

For more details on this refer to the sc3nb and SuperCollider documentation.

for name, code in context.synths.buffer_synthdefs.items():
    print(name, code)

playbuf
{ |out=0, bufnum={{BUFNUM}}, rate=1, loop=0, pan=0, amp=0.3 |
    var sig = PlayBuf.ar({{NUM_CHANNELS}}, bufnum,
        rate*BufRateScale.kr(bufnum),
        loop: loop,
        doneAction: Done.freeSelf);
    Out.ar(out, Pan2.ar(sig, pan, amp))
}

Note that there are slots: {{BUFNUM}} and {{NUM_CHANNELS}}

These will be filled by the backend using the functions stored in

context.synths.buffer_synthdefs_slots

{'NUM_CHANNELS': <function mesonic.backend.backend_sc3nb.SynthManagerSC3NB.<lambda>(scbuffer)>,
 'BUFNUM': <function mesonic.backend.backend_sc3nb.SynthManagerSC3NB.<lambda>(scbuffer)>}

Each of the functions will receive a sc3nb Buffer and then use it to get the bufnum and the number of channels.

These are required for SuperCollider to create a suitable SynthDef and thus a usable mesonic Synth for us.

Lets extend the selection of Synths with a custom Synth

• The timbralson Synth is from the Supplementary material for “sc3nb: a Python-SuperCollider Interface for Auditory Data Science”

• It uses all the channels of the data to modulate the amplitude of a harmonic of the fundamental frequency f0 for each channel.

context.synths.buffer_synthdefs["timbralson"]= r"""
{ |bufnum={{BUFNUM}}, f0=90, amp=0.1, rate=1 |
    var nch = {{NUM_CHANNELS}};
    var sines = SinOsc.ar(nch.collect{|i| f0*rate*(i+1)});
    var playbufs = PlayBuf.ar(nch, bufnum, BufRateScale.kr(bufnum)*rate, doneAction: 2 ) ;
    Out.ar(0, (sines * playbufs).sum * amp!2 )
}"""

Note that we also used the slots from above and that the slots can be extended as well.

And create a new Buffer with all the EEG data channels.

buf = context.buffers.from_data(data[14*256:24*256], sr=256)

buf

Buffer(19 x 48640 @ 256Hz = 10.000s)

timbralson_synth = context.synths.from_buffer(buf, synth_name="timbralson")

timbralson_synth.start({"f0": 90, "rate": 0.5})

The created Synth will offer the Parameters defined above and we can adapt them while the Synth plays.

timbralson_synth

Synth(sc3nb_timbralson_129, {'bufnum': 129.0, 'f0': 90, 'amp': 0.1, 'rate': 0.5})

timbralson_synth.f0 = 70

timbralson_synth.f0 = 100

timbralson_synth.rate = 1

context.close()

Quitting SCServer... Done.
Exiting sclang... Done.