Connect to server using ssh with -L flag?

It depends on operating system implementation if you do not force ssh chooses one.

I didn't do any test on it, but according a cite in this article, macOS would prefer IPv6 address:

Mac OS X prefers IPv6 if a DNS query results in IPv4 and IPv6 addresses being sent back (A and AAAA RRs).

You can enable debug log to confirm this, Core Shell would print actual IP address on connecting: