Core Shell does not authenticate via public key but plain "ssh" does?

No, Core Shell does not use system ssh binary, it uses OpenSSH XPC component via Apple’s XPC technology.

OpenSSH 8.8 disables RSA signatures using the SHA-1 hash algorithm by default, please find the solution in this topic:

Kindly Regards,

Yang