Another update,
I flashed a fresh Raspberry buster and used the AstroPI3 script to install everything. All drivers I'm using (incl sv305) worked. What was missing was the ekos part of kstars. That was solved by installing it from the ppa.
So in short I have a setup that works, but I don't know if I'll ever dare to update it...