This page was generated from notebooks/mesonic-sc3nb-comparison.ipynb.
Comparison of mesonic and sc3nb code snippets#
This notebook compares sc3nb and mesonic performance wise.
[1]:
import time
sc3nb preparation#
[2]:
import sc3nb as scn
[3]:
sc = scn.startup()
Starting sclang process... Done.
Registering OSC /return callback in sclang... Done.
Loading default sc3nb SynthDefs... Done.
Booting SuperCollider Server... Done.
mesonic preparation#
[4]:
import mesonic
[5]:
context = mesonic.create_context()
SC already started
sclang already started
scsynth already started
[6]:
s1i = context.synths.create("s1", mutable=False)
pb = context.create_playback()
mesonic does use the same sc3nb SC instance as sc3nb therefore the warnings are created.
This means that both implementation will use the same SuperCollider (SC) server.
[7]:
context.backend.sc is sc
[7]:
True
data preparation#
We use the random data generated by numpy and use the linlin function to generate valid parameters.
[8]:
import numpy as np
from sc3nb import linlin
We spawn 10.000 Synths over 5 secounds time.
A further reduction of the duration will lead to errors from SuperCollider: too many nodes.
This means we operate at the maximum of SC
[9]:
N = 10000
duration = 5
[10]:
onsets = [linlin(val, 0, 1, 0, duration) for val in np.random.rand(N)]
[11]:
freqs = [linlin(val, 0, 1, 300, 1000) for val in np.random.rand(N)]
mesonic vs sc3nb direct sending#
mesonic naive version#
Typical code (naive) snippet
[12]:
context.reset()
for idx, onset in enumerate(onsets):
with context.at(onset):
s1i.start(freq = freqs[idx], amp=0.005)
[14]:
pb.start()
[16]:
pb.stop()
Stop the time of the code snippet
[17]:
%%timeit
context.reset()
for idx, onset in enumerate(onsets):
with context.at(onset):
s1i.start(freq = freqs[idx], amp=0.005)
2.49 s ± 23.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
Profile the code snippet to see what is taking so long.
[18]:
%%prun
context.reset()
for idx, onset in enumerate(onsets):
with context.at(onset):
s1i.start(freq = freqs[idx], amp=0.005)
470009 function calls in 2.631 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
10000 2.383 0.000 2.394 0.000 timeline.py:49(insert)
10000 0.026 0.000 0.048 0.000 <attrs generated init mesonic.events.SynthEvent>:1(__init__)
10000 0.026 0.000 2.434 0.000 contextlib.py:116(__exit__)
10000 0.025 0.000 0.148 0.000 synth.py:252(start)
1 0.020 0.020 2.631 2.631 <string>:1(<module>)
10000 0.015 0.000 0.067 0.000 synth.py:337(_send_event)
20000 0.014 0.000 0.022 0.000 synth.py:127(_verify_and_adjust)
10000 0.014 0.000 0.022 0.000 synth.py:335(<dictcomp>)
20000 0.012 0.000 2.407 0.000 context.py:154(at)
10000 0.011 0.000 0.013 0.000 contextlib.py:81(__init__)
70000 0.009 0.000 0.009 0.000 synth.py:95(value)
10000 0.007 0.000 0.012 0.000 contextlib.py:107(__enter__)
10000 0.007 0.000 0.010 0.000 enum.py:319(__contains__)
10000 0.007 0.000 0.030 0.000 synth.py:326(_param_values)
20000 0.005 0.000 2.413 0.000 {built-in method builtins.next}
10000 0.005 0.000 0.014 0.000 validators.py:222(__call__)
10000 0.004 0.000 0.017 0.000 contextlib.py:237(helper)
10000 0.004 0.000 0.004 0.000 timeline.py:37(is_empty)
30000 0.004 0.000 0.004 0.000 {built-in method builtins.isinstance}
10000 0.004 0.000 0.004 0.000 <attrs generated init mesonic.timeline.TimeBundle>:1(__init__)
10000 0.003 0.000 0.004 0.000 validators.py:30(__call__)
10000 0.003 0.000 0.004 0.000 context.py:199(receive_event)
10000 0.003 0.000 0.003 0.000 events.py:69(__attrs_post_init__)
20000 0.003 0.000 0.003 0.000 synth.py:59(max)
20000 0.002 0.000 0.002 0.000 synth.py:69(min)
20000 0.002 0.000 0.002 0.000 synth.py:240(mutable)
10000 0.002 0.000 0.002 0.000 {method 'update' of 'dict' objects}
10000 0.002 0.000 0.002 0.000 {built-in method math.isfinite}
20000 0.002 0.000 0.002 0.000 {method 'items' of 'dict' objects}
10000 0.002 0.000 0.002 0.000 {built-in method builtins.getattr}
10000 0.002 0.000 0.002 0.000 {built-in method time.time}
10000 0.001 0.000 0.001 0.000 {method 'append' of 'list' objects}
10000 0.001 0.000 0.001 0.000 {built-in method builtins.len}
1 0.000 0.000 2.631 2.631 {built-in method builtins.exec}
1 0.000 0.000 0.000 0.000 timeline.py:95(reset)
1 0.000 0.000 0.000 0.000 context.py:281(reset)
1 0.000 0.000 0.000 0.000 playback.py:320(running)
1 0.000 0.000 0.000 0.000 context.py:226(is_realtime)
1 0.000 0.000 0.000 0.000 threading.py:1092(is_alive)
1 0.000 0.000 0.000 0.000 threading.py:507(is_set)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
Timeline insert is slow but we can optimize the code
mesonic optimized version#
The Timeline is a double linked list. This means by sorting the onsets we can use the insert best case of appending the data.
[19]:
%%timeit
context.reset()
for idx, onset in enumerate(sorted(onsets)):
with context.at(onset):
s1i.start(freq = freqs[idx], amp=0.005)
150 ms ± 1.4 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
The time Timeline.insert needs is now drastically reduced as expected.
What is now taking the most time?
[20]:
%%prun
context.reset()
for idx, onset in enumerate(sorted(onsets)):
with context.at(onset):
s1i.start(freq = freqs[idx], amp=0.005)
470010 function calls in 0.308 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
10000 0.099 0.000 0.119 0.000 <attrs generated init mesonic.events.SynthEvent>:1(__init__)
10000 0.023 0.000 0.213 0.000 synth.py:252(start)
10000 0.020 0.000 0.030 0.000 timeline.py:49(insert)
1 0.018 0.018 0.308 0.308 <string>:1(<module>)
20000 0.014 0.000 0.021 0.000 synth.py:127(_verify_and_adjust)
10000 0.014 0.000 0.022 0.000 synth.py:335(<dictcomp>)
10000 0.014 0.000 0.137 0.000 synth.py:337(_send_event)
20000 0.011 0.000 0.042 0.000 context.py:154(at)
10000 0.010 0.000 0.011 0.000 contextlib.py:81(__init__)
70000 0.008 0.000 0.008 0.000 synth.py:95(value)
10000 0.007 0.000 0.011 0.000 contextlib.py:107(__enter__)
10000 0.006 0.000 0.048 0.000 contextlib.py:116(__exit__)
10000 0.006 0.000 0.029 0.000 synth.py:326(_param_values)
10000 0.006 0.000 0.009 0.000 enum.py:319(__contains__)
20000 0.005 0.000 0.047 0.000 {built-in method builtins.next}
10000 0.004 0.000 0.013 0.000 validators.py:222(__call__)
10000 0.004 0.000 0.015 0.000 contextlib.py:237(helper)
10000 0.004 0.000 0.004 0.000 timeline.py:37(is_empty)
30000 0.004 0.000 0.004 0.000 {built-in method builtins.isinstance}
10000 0.003 0.000 0.004 0.000 context.py:199(receive_event)
10000 0.003 0.000 0.003 0.000 <attrs generated init mesonic.timeline.TimeBundle>:1(__init__)
10000 0.003 0.000 0.004 0.000 validators.py:30(__call__)
1 0.003 0.003 0.003 0.003 {built-in method builtins.sorted}
10000 0.003 0.000 0.003 0.000 events.py:69(__attrs_post_init__)
20000 0.003 0.000 0.003 0.000 synth.py:59(max)
20000 0.002 0.000 0.002 0.000 synth.py:69(min)
20000 0.002 0.000 0.002 0.000 synth.py:240(mutable)
20000 0.002 0.000 0.002 0.000 {method 'items' of 'dict' objects}
10000 0.002 0.000 0.002 0.000 {method 'update' of 'dict' objects}
10000 0.002 0.000 0.002 0.000 {built-in method math.isfinite}
10000 0.001 0.000 0.001 0.000 {built-in method builtins.getattr}
10000 0.001 0.000 0.001 0.000 {built-in method time.time}
10000 0.001 0.000 0.001 0.000 {method 'append' of 'list' objects}
10000 0.001 0.000 0.001 0.000 {built-in method builtins.len}
1 0.000 0.000 0.308 0.308 {built-in method builtins.exec}
1 0.000 0.000 0.000 0.000 context.py:281(reset)
1 0.000 0.000 0.000 0.000 timeline.py:95(reset)
1 0.000 0.000 0.000 0.000 context.py:226(is_realtime)
1 0.000 0.000 0.000 0.000 playback.py:320(running)
1 0.000 0.000 0.000 0.000 threading.py:1092(is_alive)
1 0.000 0.000 0.000 0.000 threading.py:507(is_set)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
Now most of the time is spend creating the SynthEvents.
Let’s compare this to sc3nb.
sc3nb direct sending#
sc3nb code snippet - direct sending
[21]:
# we need to add the latency in sc3nb by ourselves
delay = 0.2
[22]:
t0 = time.time() + delay
for idx, onset in enumerate(onsets):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
bundler = scn.Bundler(t0+onset, "/s_new", msg_params)
bundler.send()
We should sort the onsets here aswell to avoid late messages from the Server
[23]:
t0 = time.time() + delay
for idx, onset in enumerate(sorted(onsets)):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
bundler = scn.Bundler(t0+onset, "/s_new", msg_params)
bundler.send()
How long does the code snippet take?
[24]:
%%timeit
t0 = time.time() + delay
for idx, onset in enumerate(sorted(onsets)):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
bundler = scn.Bundler(t0+onset, "/s_new", msg_params)
bundler.send()
1.07 s ± 14.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
This is shorter than the naive mesonic version, but longer than the optimized version.
Let’s see what is taking the time.
[25]:
%%prun
t0 = time.time() + delay
for idx, onset in enumerate(sorted(onsets)):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
bundler = scn.Bundler(t0+onset, "/s_new", msg_params)
bundler.send()
2670005 function calls (2660005 primitive calls) in 1.568 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
10000 0.206 0.000 0.206 0.000 {method 'sendto' of '_socket.socket' objects}
100000 0.203 0.000 0.269 0.000 osc_types.py:49(get_string)
20000 0.146 0.000 0.540 0.000 osc_message.py:25(_parse_datagram)
10000 0.067 0.000 0.434 0.000 osc_message_builder.py:121(build)
10000 0.059 0.000 0.674 0.000 osc_communication.py:100(_build_message)
80000 0.055 0.000 0.122 0.000 osc_message_builder.py:67(add_arg)
70000 0.052 0.000 0.074 0.000 osc_types.py:105(get_int)
10000 0.051 0.000 0.815 0.000 osc_communication.py:881(send)
470000/460000 0.044 0.000 0.069 0.000 {built-in method builtins.isinstance}
80000 0.042 0.000 0.056 0.000 osc_message_builder.py:91(_get_arg_type)
50000 0.037 0.000 0.051 0.000 osc_types.py:34(write_string)
100000 0.033 0.000 0.033 0.000 {method 'replace' of 'bytes' objects}
120000 0.032 0.000 0.032 0.000 {built-in method _struct.unpack}
40000 0.031 0.000 0.045 0.000 osc_types.py:197(get_float)
10000 0.028 0.000 0.471 0.000 osc_communication.py:347(to_pythonosc)
420000 0.026 0.000 0.026 0.000 {built-in method builtins.len}
1 0.026 0.026 1.568 1.568 <string>:1(<module>)
10000 0.026 0.000 0.430 0.000 osc_bundle_builder.py:36(build)
10000 0.024 0.000 0.714 0.000 osc_communication.py:143(__init__)
10000 0.024 0.000 0.037 0.000 osc_communication.py:265(messages)
10000 0.024 0.000 0.315 0.000 osc_bundle.py:42(_parse_contents)
100000 0.021 0.000 0.021 0.000 {method 'decode' of 'bytes' objects}
10000 0.021 0.000 0.025 0.000 ntp.py:51(system_time_to_ntp)
280000 0.021 0.000 0.021 0.000 {method 'append' of 'list' objects}
10000 0.017 0.000 0.033 0.000 osc_types.py:302(get_date)
70000 0.017 0.000 0.017 0.000 {built-in method _struct.pack}
10000 0.013 0.000 0.361 0.000 osc_bundle.py:21(__init__)
10000 0.012 0.000 0.012 0.000 {method 'item' of 'numpy.generic' objects}
10000 0.012 0.000 0.484 0.000 osc_communication.py:190(dgram)
20000 0.011 0.000 0.551 0.000 osc_message.py:20(__init__)
10000 0.011 0.000 0.024 0.000 osc_communication.py:963(_handle_outgoing_message)
40000 0.011 0.000 0.020 0.000 osc_types.py:93(write_int)
50000 0.010 0.000 0.010 0.000 {method 'encode' of 'str' objects}
10000 0.010 0.000 0.825 0.000 osc_communication.py:295(send)
20000 0.009 0.000 0.009 0.000 osc_communication.py:379(_calc_timetag)
10000 0.008 0.000 0.683 0.000 osc_communication.py:67(__init__)
10000 0.008 0.000 0.032 0.000 osc_types.py:331(write_date)
10000 0.008 0.000 0.012 0.000 osc_types.py:129(get_uint64)
30000 0.008 0.000 0.008 0.000 {method 'startswith' of 'str' objects}
10000 0.006 0.000 0.006 0.000 osc_message_builder.py:142(<listcomp>)
10000 0.006 0.000 0.006 0.000 {built-in method _abc._abc_subclasscheck}
10000 0.006 0.000 0.021 0.000 typing.py:718(__subclasscheck__)
10000 0.006 0.000 0.027 0.000 typing.py:715(__instancecheck__)
20000 0.006 0.000 0.006 0.000 {method 'startswith' of 'bytes' objects}
20000 0.005 0.000 0.010 0.000 osc_types.py:185(write_float)
20000 0.005 0.000 0.005 0.000 __init__.py:1614(isEnabledFor)
10000 0.005 0.000 0.014 0.000 {built-in method builtins.issubclass}
10000 0.004 0.000 0.004 0.000 {method 'setdefault' of 'dict' objects}
10000 0.004 0.000 0.006 0.000 osc_communication.py:91(address)
10000 0.004 0.000 0.006 0.000 osc_communication.py:421(convert_to_sc3nb_osc)
10000 0.004 0.000 0.004 0.000 osc_message_builder.py:33(__init__)
10000 0.004 0.000 0.006 0.000 osc_communication.py:1003(get_reply_address)
10000 0.003 0.000 0.007 0.000 osc_bundle.py:70(dgram_is_bundle)
10000 0.003 0.000 0.003 0.000 ntp.py:61(ntp_time_to_system_epoch)
10000 0.003 0.000 0.004 0.000 osc_message.py:91(size)
10000 0.003 0.000 0.009 0.000 abc.py:141(__subclasscheck__)
1 0.003 0.003 0.003 0.003 {built-in method builtins.sorted}
10000 0.003 0.000 0.003 0.000 osc_bundle_builder.py:18(__init__)
10000 0.003 0.000 0.003 0.000 sc.py:128(get_default)
10000 0.003 0.000 0.004 0.000 osc_message.py:86(dgram_is_message)
10000 0.003 0.000 0.004 0.000 osc_bundle_builder.py:28(add_content)
10000 0.002 0.000 0.002 0.000 {method 'join' of 'str' objects}
10000 0.002 0.000 0.002 0.000 sc.py:286(server)
10000 0.002 0.000 0.002 0.000 {method 'get' of 'dict' objects}
10000 0.002 0.000 0.002 0.000 osc_bundle.py:90(dgram)
10000 0.002 0.000 0.002 0.000 osc_message.py:81(address)
10000 0.001 0.000 0.001 0.000 osc_message.py:96(dgram)
10000 0.001 0.000 0.001 0.000 {method 'items' of 'dict' objects}
10000 0.001 0.000 0.001 0.000 osc_communication.py:96(to_pythonosc)
1 0.000 0.000 1.568 1.568 {built-in method builtins.exec}
1 0.000 0.000 0.000 0.000 {built-in method time.time}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
Most time is spend by sending the OSC to SuperCollider.
The sc3nb code sends the OSC directly and this means we need to make the I/O operations for sending the packets.
This is different to the mesonic version, where the OSC packets for SuperCollider will be created by the Playback object in a separate thread.
This is also the reason why we hear the Synths playing when we use the sc3nb snippet. The mesonic version allows to execute them separately.
It would be a more fair comparison if we would also insert the Bundler into the TimedQueue that will perform the execution later. We will do this down below.
sc3nb without sending#
However let’s see what is happening when we simply create the Bundler without sending it.
Time the code snippet without sending the Bundler.
[26]:
%%timeit
t0 = time.time() + delay
for idx, onset in enumerate(sorted(onsets)):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
bundler = scn.Bundler(t0+onset, "/s_new", msg_params)
# bundler.send()
413 ms ± 2.45 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
It still does take longer.
Let’s run the profiler to see what is taking the time.
[27]:
%%prun
t0 = time.time() + delay
for idx, onset in enumerate(sorted(onsets)):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
bundler = scn.Bundler(t0+onset, "/s_new", msg_params)
# bundler.send()
1580005 function calls (1570005 primitive calls) in 0.672 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
50000 0.100 0.000 0.131 0.000 osc_types.py:49(get_string)
10000 0.070 0.000 0.264 0.000 osc_message.py:25(_parse_datagram)
10000 0.060 0.000 0.404 0.000 osc_message_builder.py:121(build)
80000 0.053 0.000 0.118 0.000 osc_message_builder.py:67(add_arg)
10000 0.053 0.000 0.619 0.000 osc_communication.py:100(_build_message)
80000 0.040 0.000 0.054 0.000 osc_message_builder.py:91(_get_arg_type)
400000/390000 0.034 0.000 0.053 0.000 {built-in method builtins.isinstance}
50000 0.033 0.000 0.046 0.000 osc_types.py:34(write_string)
30000 0.022 0.000 0.032 0.000 osc_types.py:105(get_int)
10000 0.020 0.000 0.652 0.000 osc_communication.py:143(__init__)
1 0.016 0.016 0.672 0.672 <string>:1(<module>)
50000 0.015 0.000 0.015 0.000 {method 'replace' of 'bytes' objects}
20000 0.015 0.000 0.022 0.000 osc_types.py:197(get_float)
210000 0.013 0.000 0.013 0.000 {built-in method builtins.len}
50000 0.013 0.000 0.013 0.000 {built-in method _struct.unpack}
170000 0.012 0.000 0.012 0.000 {method 'append' of 'list' objects}
50000 0.010 0.000 0.010 0.000 {method 'decode' of 'bytes' objects}
50000 0.009 0.000 0.009 0.000 {built-in method _struct.pack}
50000 0.009 0.000 0.009 0.000 {method 'encode' of 'str' objects}
10000 0.007 0.000 0.627 0.000 osc_communication.py:67(__init__)
10000 0.007 0.000 0.007 0.000 {method 'item' of 'numpy.generic' objects}
30000 0.007 0.000 0.013 0.000 osc_types.py:93(write_int)
10000 0.006 0.000 0.270 0.000 osc_message.py:20(__init__)
10000 0.005 0.000 0.005 0.000 osc_message_builder.py:142(<listcomp>)
10000 0.005 0.000 0.016 0.000 typing.py:718(__subclasscheck__)
20000 0.005 0.000 0.009 0.000 osc_types.py:185(write_float)
10000 0.004 0.000 0.021 0.000 typing.py:715(__instancecheck__)
20000 0.004 0.000 0.004 0.000 {method 'startswith' of 'str' objects}
10000 0.004 0.000 0.010 0.000 {built-in method builtins.issubclass}
10000 0.004 0.000 0.004 0.000 {built-in method _abc._abc_subclasscheck}
1 0.003 0.003 0.003 0.003 {built-in method builtins.sorted}
10000 0.003 0.000 0.003 0.000 osc_message_builder.py:33(__init__)
10000 0.003 0.000 0.006 0.000 abc.py:141(__subclasscheck__)
10000 0.002 0.000 0.002 0.000 {method 'join' of 'str' objects}
10000 0.002 0.000 0.002 0.000 sc.py:128(get_default)
10000 0.002 0.000 0.002 0.000 sc.py:286(server)
1 0.000 0.000 0.672 0.672 {built-in method builtins.exec}
1 0.000 0.000 0.000 0.000 {built-in method time.time}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
Most of the time is now spend by creating the OSC classes.
The functions taking the longest time are from the OSC backend of sc3nb, which is python-osc. As the name suggests it is written in Python and there might be room for improvements by exchanging the OSC backend of sc3nb.
mesonic vs sc3nb direct sending - conclusion#
Now we have both tools at the same spot, as mesonic does spend the most time creating the SynthEvents aswell, but these are created faster than the more complex OSC classes.
Nevertheless the Playback will need to create OSC packets from the SynthEvents aswell. * But as shown above this currently can be done in the defined latency of the BundleProcessor as there are no late messages from the SC server. * This means that the latency will ensure that the OSC packets arrive at the server before the execution time. * The latency used is also the default latency of SuperCollider itself.
Note that sc3nb would also allow inserting the Bundler into the TimedQueue. * This also offers the separation of creating and adding the data and then sending it with another thread. * Nevertheless the creation time of a SynthEvent will stay shorter and the insertion of the Bundler into the TimedQueue will only add up the time.
Alltogether this demonstrates that the abstraction of the OSC messages into SynthEvents could help to reduce the latency needed.
mesonic vs sc3nb TimedQueue#
For completeness we will look at the TimedQueue as well.
[28]:
queue_sc = scn.TimedQueueSC()
[29]:
# we need a bigger delay or we run into problems (lates / too many nodes)
delay = 0.5
t0 = time.time()
for idx, onset in enumerate(sorted(onsets)):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
queue_sc.put_bundler(t0+onset+delay, scn.Bundler(t0+onset+delay*2, "/s_new", msg_params))
[31]:
queue_sc.close()
Let’s look at the performance
We close the queue so the task will not be spawned multiple times as this would be too many nodes for SC.
[32]:
%%timeit
queue_sc = scn.TimedQueueSC()
queue_sc.close()
delay = 0.5 # we need a bigger delay or run into problems
t0 = time.time()
for idx, onset in enumerate(sorted(onsets)):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
queue_sc.put_bundler(t0+onset+delay, scn.Bundler(t0+onset+delay*2, "/s_new", msg_params))
1 s ± 11.2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
[33]:
%%prun
queue_sc = scn.TimedQueueSC()
queue_sc.close()
delay = 0.5 # we need a bigger delay or run into problems
t0 = time.time()
for idx, onset in enumerate(sorted(onsets)):
msg_params = ["s1", -1, 1, 0, "freq", freqs[idx], "amp", 0.005]
queue_sc.put_bundler(t0+onset+delay, scn.Bundler(t0+onset+delay*2, "/s_new", msg_params))
2090060 function calls (2070060 primitive calls) in 1.390 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
10000 0.136 0.000 0.323 0.000 function_base.py:4495(insert)
10000 0.130 0.000 0.130 0.000 {method 'reduce' of 'numpy.ufunc' objects}
50000 0.101 0.000 0.136 0.000 osc_types.py:49(get_string)
10000 0.100 0.000 0.654 0.000 timed_queue.py:113(put)
10000 0.073 0.000 0.274 0.000 osc_message.py:25(_parse_datagram)
10000 0.063 0.000 0.426 0.000 osc_message_builder.py:121(build)
10000 0.056 0.000 0.658 0.000 osc_communication.py:100(_build_message)
80000 0.054 0.000 0.120 0.000 osc_message_builder.py:67(add_arg)
20000 0.041 0.000 0.063 0.000 numeric.py:1341(normalize_axis_tuple)
80000 0.041 0.000 0.055 0.000 osc_message_builder.py:91(_get_arg_type)
420000/410000 0.039 0.000 0.064 0.000 {built-in method builtins.isinstance}
29999/19999 0.039 0.000 0.391 0.000 {built-in method numpy.core._multiarray_umath.implement_array_function}
50000 0.036 0.000 0.050 0.000 osc_types.py:34(write_string)
10000 0.035 0.000 0.118 0.000 numeric.py:1404(moveaxis)
30000 0.034 0.000 0.034 0.000 {built-in method numpy.array}
1 0.029 0.029 1.390 1.390 <string>:1(<module>)
10000 0.024 0.000 0.697 0.000 osc_communication.py:143(__init__)
30000 0.023 0.000 0.033 0.000 osc_types.py:105(get_int)
280001 0.021 0.000 0.021 0.000 {built-in method builtins.len}
9999 0.020 0.000 0.020 0.000 {method 'searchsorted' of 'numpy.ndarray' objects}
50000 0.017 0.000 0.017 0.000 {method 'replace' of 'bytes' objects}
20000 0.016 0.000 0.023 0.000 osc_types.py:197(get_float)
50000 0.014 0.000 0.014 0.000 {built-in method _struct.unpack}
180000 0.013 0.000 0.013 0.000 {method 'append' of 'list' objects}
20000 0.012 0.000 0.016 0.000 numeric.py:1391(<listcomp>)
50000 0.012 0.000 0.012 0.000 {method 'decode' of 'bytes' objects}
50000 0.012 0.000 0.012 0.000 {built-in method _struct.pack}
10001 0.011 0.000 0.011 0.000 {built-in method numpy.empty}
50000 0.010 0.000 0.010 0.000 {method 'encode' of 'str' objects}
10001 0.009 0.000 0.009 0.000 {built-in method builtins.sorted}
10000 0.009 0.000 0.009 0.000 {method 'item' of 'numpy.generic' objects}
10000 0.008 0.000 0.666 0.000 osc_communication.py:67(__init__)
30000 0.008 0.000 0.015 0.000 osc_types.py:93(write_int)
9999 0.007 0.000 0.035 0.000 fromnumeric.py:1281(searchsorted)
30000 0.007 0.000 0.007 0.000 {built-in method numpy.core._multiarray_umath.normalize_axis_index}
10000 0.007 0.000 0.347 0.000 <__array_function__ internals>:2(insert)
9999 0.007 0.000 0.060 0.000 <__array_function__ internals>:2(searchsorted)
10000 0.006 0.000 0.132 0.000 <__array_function__ internals>:2(moveaxis)
10000 0.006 0.000 0.006 0.000 {method 'transpose' of 'numpy.ndarray' objects}
10000 0.006 0.000 0.660 0.000 timed_queue.py:249(put_bundler)
9999 0.006 0.000 0.028 0.000 fromnumeric.py:52(_wrapfunc)
10000 0.006 0.000 0.026 0.000 typing.py:715(__instancecheck__)
10000 0.006 0.000 0.280 0.000 osc_message.py:20(__init__)
10000 0.006 0.000 0.020 0.000 typing.py:718(__subclasscheck__)
10000 0.006 0.000 0.006 0.000 osc_message_builder.py:142(<listcomp>)
20000 0.006 0.000 0.006 0.000 {method 'startswith' of 'str' objects}
10000 0.006 0.000 0.140 0.000 {method 'any' of 'numpy.ndarray' objects}
10000 0.005 0.000 0.005 0.000 {built-in method _abc._abc_subclasscheck}
20000 0.005 0.000 0.009 0.000 osc_types.py:185(write_float)
10000 0.004 0.000 0.013 0.000 {built-in method builtins.issubclass}
10000 0.004 0.000 0.135 0.000 _methods.py:53(_any)
10000 0.004 0.000 0.004 0.000 osc_message_builder.py:33(__init__)
10000 0.003 0.000 0.008 0.000 _asarray.py:23(asarray)
10000 0.003 0.000 0.003 0.000 timed_queue.py:29(__init__)
10000 0.003 0.000 0.003 0.000 numeric.py:1467(<listcomp>)
10000 0.003 0.000 0.003 0.000 {method 'insert' of 'list' objects}
10000 0.003 0.000 0.008 0.000 abc.py:141(__subclasscheck__)
10000 0.003 0.000 0.003 0.000 {method 'join' of 'str' objects}
10001 0.002 0.000 0.002 0.000 sc.py:128(get_default)
20000 0.002 0.000 0.002 0.000 {built-in method _operator.index}
10001 0.002 0.000 0.002 0.000 sc.py:286(server)
9999 0.002 0.000 0.002 0.000 {built-in method builtins.getattr}
10000 0.002 0.000 0.002 0.000 {method 'item' of 'numpy.ndarray' objects}
10000 0.001 0.000 0.001 0.000 numeric.py:1400(_moveaxis_dispatcher)
9999 0.001 0.000 0.001 0.000 fromnumeric.py:1277(_searchsorted_dispatcher)
10000 0.001 0.000 0.001 0.000 function_base.py:4491(_insert_dispatcher)
10000 0.001 0.000 0.001 0.000 {built-in method builtins.callable}
6 0.000 0.000 0.000 0.000 {method 'acquire' of '_thread.lock' objects}
1 0.000 0.000 1.390 1.390 {built-in method builtins.exec}
1 0.000 0.000 0.000 0.000 {built-in method _thread.start_new_thread}
1 0.000 0.000 0.000 0.000 timed_queue.py:77(__init__)
4 0.000 0.000 0.000 0.000 {built-in method _thread.allocate_lock}
2 0.000 0.000 0.000 0.000 threading.py:216(__init__)
1 0.000 0.000 0.000 0.000 threading.py:763(__init__)
1 0.000 0.000 0.000 0.000 threading.py:264(wait)
1 0.000 0.000 0.000 0.000 timed_queue.py:240(__init__)
1 0.000 0.000 0.000 0.000 threading.py:358(notify_all)
1 0.000 0.000 0.000 0.000 threading.py:534(wait)
1 0.000 0.000 0.000 0.000 threading.py:834(start)
1 0.000 0.000 0.000 0.000 threading.py:977(_stop)
2 0.000 0.000 0.000 0.000 threading.py:499(__init__)
1 0.000 0.000 0.000 0.000 threading.py:513(set)
1 0.000 0.000 0.000 0.000 _weakrefset.py:81(add)
1 0.000 0.000 0.000 0.000 threading.py:1012(join)
2 0.000 0.000 0.000 0.000 threading.py:243(__exit__)
1 0.000 0.000 0.000 0.000 threading.py:1050(_wait_for_tstate_lock)
2 0.000 0.000 0.000 0.000 threading.py:1225(current_thread)
1 0.000 0.000 0.000 0.000 _weakrefset.py:38(_remove)
2 0.000 0.000 0.000 0.000 threading.py:240(__enter__)
1 0.000 0.000 0.000 0.000 threading.py:728(_newname)
1 0.000 0.000 0.000 0.000 timed_queue.py:97(close)
1 0.000 0.000 0.000 0.000 threading.py:335(notify)
2 0.000 0.000 0.000 0.000 threading.py:255(_is_owned)
1 0.000 0.000 0.000 0.000 threading.py:249(_release_save)
1 0.000 0.000 0.000 0.000 threading.py:252(_acquire_restore)
2 0.000 0.000 0.000 0.000 threading.py:1116(daemon)
2 0.000 0.000 0.000 0.000 {method 'discard' of 'set' objects}
2 0.000 0.000 0.000 0.000 {method '__enter__' of '_thread.lock' objects}
2 0.000 0.000 0.000 0.000 {built-in method _thread.get_ident}
2 0.000 0.000 0.000 0.000 threading.py:507(is_set)
2 0.000 0.000 0.000 0.000 {method '__exit__' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 {method 'add' of 'set' objects}
1 0.000 0.000 0.000 0.000 {built-in method time.time}
1 0.000 0.000 0.000 0.000 {method 'append' of 'collections.deque' objects}
2 0.000 0.000 0.000 0.000 {method 'release' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 {method 'locked' of '_thread.lock' objects}
Let’s simplify the code to cut the time spend in python-osc
[34]:
%%timeit
queue_sc = scn.TimedQueueSC()
queue_sc.close()
fun = lambda x: x
for onset in sorted(onsets):
queue_sc.put(onset, fun)
446 ms ± 6.79 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
[35]:
%%prun
queue_sc = scn.TimedQueueSC()
queue_sc.close()
fun = lambda x: x
for onset in sorted(onsets):
queue_sc.put(onset, fun)
500059 function calls (490059 primitive calls) in 0.556 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
10000 0.121 0.000 0.121 0.000 {method 'reduce' of 'numpy.ufunc' objects}
10000 0.117 0.000 0.279 0.000 function_base.py:4495(insert)
10000 0.049 0.000 0.539 0.000 timed_queue.py:113(put)
20000 0.038 0.000 0.053 0.000 numeric.py:1341(normalize_axis_tuple)
29999/19999 0.034 0.000 0.338 0.000 {built-in method numpy.core._multiarray_umath.implement_array_function}
10000 0.031 0.000 0.101 0.000 numeric.py:1404(moveaxis)
30000 0.029 0.000 0.029 0.000 {built-in method numpy.array}
9999 0.017 0.000 0.017 0.000 {method 'searchsorted' of 'numpy.ndarray' objects}
1 0.013 0.013 0.556 0.556 <string>:1(<module>)
10001 0.010 0.000 0.010 0.000 {built-in method numpy.empty}
10001 0.008 0.000 0.008 0.000 {built-in method builtins.sorted}
9999 0.007 0.000 0.031 0.000 fromnumeric.py:1281(searchsorted)
30000 0.007 0.000 0.007 0.000 {built-in method numpy.core._multiarray_umath.normalize_axis_index}
10000 0.006 0.000 0.301 0.000 <__array_function__ internals>:2(insert)
9999 0.006 0.000 0.052 0.000 <__array_function__ internals>:2(searchsorted)
70001 0.006 0.000 0.006 0.000 {built-in method builtins.len}
10000 0.006 0.000 0.114 0.000 <__array_function__ internals>:2(moveaxis)
20000 0.006 0.000 0.010 0.000 numeric.py:1391(<listcomp>)
10000 0.005 0.000 0.005 0.000 {method 'transpose' of 'numpy.ndarray' objects}
9999 0.005 0.000 0.024 0.000 fromnumeric.py:52(_wrapfunc)
10000 0.005 0.000 0.129 0.000 {method 'any' of 'numpy.ndarray' objects}
10000 0.003 0.000 0.124 0.000 _methods.py:53(_any)
10000 0.003 0.000 0.003 0.000 timed_queue.py:29(__init__)
20000 0.003 0.000 0.003 0.000 {built-in method builtins.isinstance}
10000 0.003 0.000 0.006 0.000 _asarray.py:23(asarray)
10000 0.003 0.000 0.003 0.000 numeric.py:1467(<listcomp>)
10000 0.002 0.000 0.002 0.000 {method 'insert' of 'list' objects}
9999 0.002 0.000 0.002 0.000 {built-in method builtins.getattr}
20000 0.002 0.000 0.002 0.000 {built-in method _operator.index}
10000 0.002 0.000 0.002 0.000 {method 'item' of 'numpy.ndarray' objects}
9999 0.001 0.000 0.001 0.000 fromnumeric.py:1277(_searchsorted_dispatcher)
10000 0.001 0.000 0.001 0.000 numeric.py:1400(_moveaxis_dispatcher)
10000 0.001 0.000 0.001 0.000 function_base.py:4491(_insert_dispatcher)
10000 0.001 0.000 0.001 0.000 {method 'append' of 'list' objects}
10000 0.001 0.000 0.001 0.000 {built-in method builtins.callable}
6 0.001 0.000 0.001 0.000 {method 'acquire' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 {built-in method _thread.start_new_thread}
1 0.000 0.000 0.556 0.556 {built-in method builtins.exec}
1 0.000 0.000 0.000 0.000 _weakrefset.py:38(_remove)
1 0.000 0.000 0.000 0.000 timed_queue.py:77(__init__)
4 0.000 0.000 0.000 0.000 {built-in method _thread.allocate_lock}
1 0.000 0.000 0.000 0.000 threading.py:977(_stop)
2 0.000 0.000 0.000 0.000 threading.py:216(__init__)
1 0.000 0.000 0.000 0.000 threading.py:763(__init__)
1 0.000 0.000 0.000 0.000 timed_queue.py:240(__init__)
1 0.000 0.000 0.000 0.000 threading.py:264(wait)
1 0.000 0.000 0.000 0.000 threading.py:513(set)
1 0.000 0.000 0.000 0.000 threading.py:534(wait)
1 0.000 0.000 0.000 0.000 threading.py:1012(join)
1 0.000 0.000 0.000 0.000 threading.py:834(start)
1 0.000 0.000 0.000 0.000 threading.py:335(notify)
1 0.000 0.000 0.000 0.000 threading.py:358(notify_all)
2 0.000 0.000 0.000 0.000 threading.py:243(__exit__)
2 0.000 0.000 0.000 0.000 threading.py:499(__init__)
2 0.000 0.000 0.000 0.000 threading.py:1225(current_thread)
1 0.000 0.000 0.000 0.000 _weakrefset.py:81(add)
1 0.000 0.000 0.000 0.000 threading.py:1050(_wait_for_tstate_lock)
1 0.000 0.000 0.000 0.000 timed_queue.py:97(close)
2 0.000 0.000 0.000 0.000 threading.py:240(__enter__)
2 0.000 0.000 0.000 0.000 {method 'discard' of 'set' objects}
2 0.000 0.000 0.000 0.000 threading.py:255(_is_owned)
1 0.000 0.000 0.000 0.000 threading.py:728(_newname)
2 0.000 0.000 0.000 0.000 threading.py:1116(daemon)
1 0.000 0.000 0.000 0.000 threading.py:249(_release_save)
2 0.000 0.000 0.000 0.000 {method '__enter__' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 threading.py:252(_acquire_restore)
1 0.000 0.000 0.000 0.000 sc.py:286(server)
2 0.000 0.000 0.000 0.000 {method '__exit__' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 sc.py:128(get_default)
2 0.000 0.000 0.000 0.000 threading.py:507(is_set)
2 0.000 0.000 0.000 0.000 {built-in method _thread.get_ident}
1 0.000 0.000 0.000 0.000 {method 'add' of 'set' objects}
1 0.000 0.000 0.000 0.000 {method 'locked' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 {method 'append' of 'collections.deque' objects}
2 0.000 0.000 0.000 0.000 {method 'release' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
The insertion of the TimedQueue Events is now what is taking the most time.
And mesonic still performs better.
Note that timeit will restart the Playback several times.
[36]:
%%timeit
context.reset()
for idx, onset in enumerate(sorted(onsets)):
with context.at(onset):
s1i.start(freq = freqs[idx], amp=0.005)
pb.start()
158 ms ± 1.78 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
Also note that the playback is just started when the insertion is finished.
[39]:
%%prun
context.reset()
for idx, onset in enumerate(sorted(onsets)):
with context.at(onset):
s1i.start(freq = freqs[idx], amp=0.005)
pb.start()
470034 function calls in 0.259 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
10000 0.049 0.000 0.069 0.000 <attrs generated init mesonic.events.SynthEvent>:1(__init__)
10000 0.023 0.000 0.163 0.000 synth.py:252(start)
10000 0.020 0.000 0.030 0.000 timeline.py:49(insert)
1 0.018 0.018 0.259 0.259 <string>:1(<module>)
20000 0.014 0.000 0.021 0.000 synth.py:127(_verify_and_adjust)
10000 0.014 0.000 0.087 0.000 synth.py:337(_send_event)
10000 0.014 0.000 0.022 0.000 synth.py:335(<dictcomp>)
20000 0.011 0.000 0.043 0.000 context.py:154(at)
10000 0.010 0.000 0.011 0.000 contextlib.py:81(__init__)
70000 0.008 0.000 0.008 0.000 synth.py:95(value)
10000 0.007 0.000 0.011 0.000 contextlib.py:107(__enter__)
10000 0.006 0.000 0.049 0.000 contextlib.py:116(__exit__)
10000 0.006 0.000 0.009 0.000 enum.py:319(__contains__)
10000 0.006 0.000 0.029 0.000 synth.py:326(_param_values)
20000 0.004 0.000 0.047 0.000 {built-in method builtins.next}
10000 0.004 0.000 0.013 0.000 validators.py:222(__call__)
10000 0.004 0.000 0.015 0.000 contextlib.py:237(helper)
10000 0.004 0.000 0.004 0.000 timeline.py:37(is_empty)
30000 0.004 0.000 0.004 0.000 {built-in method builtins.isinstance}
10000 0.004 0.000 0.004 0.000 <attrs generated init mesonic.timeline.TimeBundle>:1(__init__)
10000 0.003 0.000 0.004 0.000 validators.py:30(__call__)
10000 0.003 0.000 0.004 0.000 context.py:199(receive_event)
1 0.003 0.003 0.003 0.003 {built-in method builtins.sorted}
10000 0.003 0.000 0.003 0.000 events.py:69(__attrs_post_init__)
20000 0.002 0.000 0.002 0.000 synth.py:59(max)
20000 0.002 0.000 0.002 0.000 synth.py:240(mutable)
20000 0.002 0.000 0.002 0.000 synth.py:69(min)
20000 0.002 0.000 0.002 0.000 {method 'items' of 'dict' objects}
10000 0.002 0.000 0.002 0.000 {method 'update' of 'dict' objects}
10000 0.002 0.000 0.002 0.000 {built-in method math.isfinite}
10000 0.001 0.000 0.001 0.000 {built-in method builtins.getattr}
10000 0.001 0.000 0.001 0.000 {built-in method time.time}
10000 0.001 0.000 0.001 0.000 {method 'append' of 'list' objects}
10000 0.001 0.000 0.001 0.000 {built-in method builtins.len}
1 0.000 0.000 0.259 0.259 {built-in method builtins.exec}
3 0.000 0.000 0.000 0.000 threading.py:1092(is_alive)
2 0.000 0.000 0.000 0.000 threading.py:1050(_wait_for_tstate_lock)
1 0.000 0.000 0.000 0.000 playback.py:325(start)
1 0.000 0.000 0.000 0.000 timeline.py:95(reset)
1 0.000 0.000 0.000 0.000 context.py:281(reset)
1 0.000 0.000 0.000 0.000 playback.py:283(time)
2 0.000 0.000 0.000 0.000 playback.py:320(running)
1 0.000 0.000 0.000 0.000 threading.py:534(wait)
1 0.000 0.000 0.000 0.000 context.py:226(is_realtime)
2 0.000 0.000 0.000 0.000 threading.py:240(__enter__)
2 0.000 0.000 0.000 0.000 threading.py:243(__exit__)
2 0.000 0.000 0.000 0.000 {method 'acquire' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 playback.py:185(_wait_for_event)
1 0.000 0.000 0.000 0.000 threading.py:524(clear)
4 0.000 0.000 0.000 0.000 threading.py:507(is_set)
1 0.000 0.000 0.000 0.000 playback.py:297(rate)
2 0.000 0.000 0.000 0.000 {method '__enter__' of '_thread.lock' objects}
2 0.000 0.000 0.000 0.000 {method '__exit__' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
mesonic vs sc3nb TimedQueue - conclusion#
The direct comparison revealed that the Timeline of mesonic does have a better performance over the TimedQueue of sc3nb on already sorted onsets.
However, the onsets must be sorted, but this is often achievable.
To improve the performance on unsorted data a skip list could be used for the Timeline instead of the currently used double linked list.
This should also achieve O(log n) performance like the TimedQueue which uses
np.searchsortedwhich is based on binary search.
[41]:
context.close()
Quitting SCServer... Done.
Exiting sclang... Done.
[ ]: