I can log into the proxy server and then from there log in to the production server just fine when done manually with ssh command.
I guess I never really explained what I am trying to do here. I need to login in to the proxy server using my local ssh key and then when logged into the proxy server I need to log in to the production server using the ssh key stored in the proxy server. From the production server I can finally reach out to our aws DB service.
Using Core Tunnel I can login to the proxy server but then fail authentication when attempting to login to production server (from reading the debug log). I assume that the proxy server is not using the ssh key stored there when trying to connect to production server. Again, this was working for me before until I restarted my machine.