[Arch Linux] Very high DIN MIDI latency (USB works fine)

tegenfase · June 18, 2019, 7:40pm

Hey!

I have a very weird issue with my pisound. I’m using with a Raspberry Pi 3 B running Arch and Supercollider (with audio over jack). Long story short I trigger a sampler within a MIDI response function, but there is a good few 100ms of latency (I’m guessing by ear) which is sometimes accompanied by high pitched audio artifacts. This only happens when using the built-in DIN MIDI of the pisound, MIDI over USB works fine as does OSC…

This leads me to think that it might be a hardware or firmware issue. Could you help me figure this one out? I would like to be able to use the built in MIDI as well.

Giedrius · June 19, 2019, 11:52am

Hi, could you attach the dmesg log when the issue happens?

Is there any way we could try to reproduce the issue on our own? Could you share your patch (we can keep it in private) or could you produce an example patch demonstrating the issue?

Could you try running your patch using Patchbox OS? There’s been a couple of fixes to the Pisound driver a while ago, I don’t know what is the current driver that Arch Linux for Raspbian currently has…

tegenfase · June 19, 2019, 9:19pm

The patch itself has gotten quite out of hand, but the MIDI functionality implemented is very basic. I uploaded it to github for you. The MIDI bits are all the way at the top and bottom. I will try and slim it down to a simpler patch so that you can try to run it yourself (this one does weird file system parsing to find samples to load).

github.com

tegenfase/supercollider/blob/master/sampler.scd


(
s.waitForBoot({
	var c = Condition.new;
	var numOfChannels = 16;


MIDIClient.init;
MIDIIn.connectAll;

	s.sync(c);
	"[!] Writing synthdefs to: ".post; SynthDef.synthDefDir.postln;
	SynthDef.new(\reverb, {
		arg in = 0, mix = 1, room = 1, damp = 0.5, gain = 1, out = 0;
		var sig;
		sig = In.ar(in, 2);
		sig = FreeVerb2.ar(sig[0],sig[1], mix, room, damp, gain);

		// sig = SVF.ar(sig, 500, 0.1, 0, 1, 1, 0,);
		Out.ar(out,sig);

This file has been truncated. show original

The dmesg I get is linked below. Does this mean the pisound spi transfer is trying to interrupt the kernel? Arch uses the snd_soc_core module as a driver, I could try patchbox os when I have time.

github.com

tegenfase/supercollider/blob/master/dmesg

[195697.932518] BUG: scheduling while atomic: kworker/u8:3/3299/0x00000002
[195697.940012] Modules linked in: snd_usb_audio snd_hwdep snd_usbmidi_lib snd_seq_dummy snd_hrtimer snd_seq_midi snd_seq_midi_event snd_seq 8021q garp stp mrp llc brcmfmac brcmutil cfg80211 spidev bcm2835_v4l2(C) bcm2835_codec(C) v4l2_mem2mem bcm2835_mmal_vchiq(C) v4l2_
common videobuf2_vmalloc videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common raspberrypi_hwmon videodev snd_soc_pisound hwmon snd_rawmidi media snd_seq_device spi_bcm2835 snd_soc_bcm2835_i2s i2c_bcm2835 snd_soc_pcm5102a vc_sm_cma(C) bcm2835_rng rng_core
 snd_soc_core snd_compress snd_pcm_dmaengine fixed uio_pdrv_genirq uio sch_fq_codel i2c_dev i2c_bcm2708 snd_bcm2835(C) snd_pcm snd_timer snd ip_tables x_tables ipv6
[195697.968737] Preemption disabled at:
[195697.968748] [<00000000>]   (null)
[195697.983303] CPU: 3 PID: 3299 Comm: kworker/u8:3 Tainted: G        WC        4.19.50-1-ARCH #1
[195697.990559] Hardware name: BCM2835
[195697.997689] Workqueue: pisnd_workqueue pisnd_work_handler [snd_soc_pisound]
[195698.004977] [<8011011c>] (unwind_backtrace) from [<8010c700>] (show_stack+0x10/0x14)
[195698.012221] [<8010c700>] (show_stack) from [<80b0c61c>] (dump_stack+0xb8/0xe0)
[195698.019510] [<80b0c61c>] (dump_stack) from [<801548e8>] (__schedule_bug+0x98/0xd4)
[195698.026806] [<801548e8>] (__schedule_bug) from [<80b21e58>] (__schedule+0x800/0xa58)
[195698.034009] [<80b21e58>] (__schedule) from [<80b220f4>] (schedule+0x44/0xb0)
[195698.041336] [<80b220f4>] (schedule) from [<80b25b64>] (schedule_timeout+0x1c0/0x474)
[195698.048448] [<80b25b64>] (schedule_timeout) from [<80b22df8>] (wait_for_common+0x130/0x168)
[195698.055741] [<80b22df8>] (wait_for_common) from [<8086d33c>] (spi_transfer_one_message+0x134/0x534)
[195698.062866] [<8086d33c>] (spi_transfer_one_message) from [<8086daac>] (__spi_pump_messages+0x370/0x744)
[195698.070315] [<8086daac>] (__spi_pump_messages) from [<8086e0cc>] (__spi_sync+0x240/0x25c)
[195698.077671] [<8086e0cc>] (__spi_sync) from [<8086e10c>] (spi_sync+0x24/0x3c)

This file has been truncated. show original

Giedrius · June 19, 2019, 9:34pm

This error in dmesg seems to be the cause of the latency. This may be specific to the way the kernel gets built for Arch Linux, I don’t think I’ve seen this exact stack trace and error getting reported in the regular RPi and RPi Real Time kernel configurations. Patchbox OS has the latter integrated.

Please try to reproduce the issue running Patchbox OS when you have time and let us know how it goes!

tegenfase · June 20, 2019, 3:34pm

Yeah using Patchbox solved the issue. No more latency and no dmesg (at least with some test code that only prints out MIDI).

Would it be possible to solve this issue on your side or would that be up to the Arch Linux ARM team? I would like to continue using Arch if it’s possible. Otherwise I guess I’ll have to switch to Patchbox (which seems quite nice actually, but I don’t want/need all the GUI stuff).

Giedrius · June 20, 2019, 3:57pm

You can choose whether to boot to GUI or shell, in the latter case, only the space taken in SD card is impacted, but not the system performance.

Of course it is possible to get it working fine on Arch Linux one way or another, but we do have limited time to support multiple platforms while continuing development of future products, so Raspbian and Patchbox OS get the most support and stability. I’ll add fixing this on Arch to my todo, I should be able to get it fixed some time next week. However, at some point in the future, it may break again in some way.

Anyone volunteering to actively support Pisound on Arch Linux would be welcomed!

tegenfase · June 29, 2019, 2:13pm

Yeah I know, it’s not a deal breaker for me, I just spent quite some time on getting to know Arch. Because a while back supercollider support on Raspbian was a bit lackluster and on Arch it worked out of the box with the latest version.

I wouldn’t mind switching to Patchbox though, the added support (and hopefully stability) kind of outweighs having the latest stuff. Especially since I want this to be part of a live set-up at some point

Have you gotten around to fixing it yet? I wouldn’t mind cross-checking my project on Arch for compatibility and helping out with support. I am, however, not that familiar with Linux low level stuff. I would like to get into that as I have plans for designing my own interface that’s eurorack compatible.

Giedrius · July 2, 2019, 5:30pm

Patchbox OS has 3.9.3 version of Supercollider which I think is newer than what was available in “stretch” Raspbian at the time.

Unfortunately no, but I should be able to attempt that towards the end of this week or next week definitely. I’ll let you know how it goes.

Giedrius · July 12, 2019, 3:28pm

Hey! Ok, I spent some time on this issue today, and submitted a fix for the Pisound kernel module on Raspberry Pi kernel: https://github.com/raspberrypi/linux/pull/3069

Some time next week, a new kernel build should get published, versions exclusively greater than 4.19.57-v7+ should have the fix in them. I think pacman -Syu should be able to install the latest kernel package, once it becomes available for Arch Linux.

Giedrius · July 17, 2019, 10:01am

Ok, the fix was just merged today, as 4.19.58 is already released, we should expect the fix in versions >= 4.19.59

tegenfase · July 21, 2019, 6:28pm

Nice, I will check it out when I get the chance, was on vacation for a while

tegenfase · July 28, 2019, 12:01pm

I’ve just updated my Arch setup to linux-raspberrypi-4.19.59-1 and the issue has been resolved. Thanks a bunch!

I’m just curious as to why this was only an issue on Arch, does it have to do with how their build of the kernel handles spinlock?

Giedrius · July 29, 2019, 8:27am

Yes, Arch Linux seems to have their own kernel builds and the configs to build them, different from the ones used by Raspbian. I don’t know what exact flag causes this difference of behavior, but it seems Arch has some debug check for spinlocks enabled.