Hmmm… the dualboot sounds like a solution - though it seems to be needing a keyboard + monitor which I don‘t have and to keep the pi headless.
Would still prefer another solution - and maybe realtime capabilities of jackd would also be an advantage for having a slightly better a/v sync when using the pi to stream audio from the AppleTV to the pi with airplay. But I suppose this is too advanced for me and switching SD card is the solution for me not understanding enough of linux audio.