Weird problem with Core Tunnel and Catalina?

Core Tunnel Ask for Support
1

I've been starting to use Core Tunnel for punching back emergency reverse tunnels over LTE connections for remote servers. I'm seeing a weird problem on a Mac Mini running 10.15.6 that's trying to connect a reverse ssh tunnel to an Ubuntu 20 host.

09:30:40 Connecting…

09:30:40 Using Core Helper 4.8 (r48)

09:30:40 load pubkey "/Users/smackie/Library/Group Containers/E78WKS7W4U.io.coressh.ssh/.ssh/privatekey/id_rsa": invalid format

09:30:40 Authenticating…

09:30:41 Connected

The tunnel works correctly when I connect manually but if I leave it up for any point in time, it eventually fails with a permission error and a prompt for the password (which I don't want to use - I only want to use the ssh identity for the tunnel).

The private key appears to load correctly in Preferences.

Any pointers?

Cheers

Scott...

2

Hi Scott, could you please set debug level to DEBUG3, and paste the failed log?

It would help me to identify the problem.

Thank you,

Yang

3

Yang,

Here you go -

Equivalent Command: ssh -i "/Users/smackie/Library/Group Containers/E78WKS7W4U.io.coressh.ssh/.ssh/privatekey/id_rsa" -vvv -R 0.0.0.0:10083:localhost:22 -o PasswordAuthentication=no -o BindInterface=en7 -o ExitOnForwardFailure=yes -o ServerAliveCountMax=3 -o ServerAliveInterval=15 smackie@skyport.cedaroffice.org
13:42:01 Connecting…
13:42:01 Using Core Helper 4.8 (r48)
13:42:01 OpenSSH_8.3p1, OpenSSL 1.1.1g  21 Apr 2020
13:42:01 debug1: Reading configuration data /etc/ssh/ssh_config
13:42:01 debug1: /etc/ssh/ssh_config line 47: Applying options for *
13:42:01 debug2: resolving "skyport.cedaroffice.org" port 22
13:42:01 debug2: ssh_connect_direct
13:42:01 debug1: Connecting to skyport.cedaroffice.org [63.249.66.100] port 22.
13:42:01 debug1: ssh_create_socket: bound to 100.232.107.38
13:42:01 debug1: Connection established.
13:42:01 load pubkey "/Users/smackie/Library/Group Containers/E78WKS7W4U.io.coressh.ssh/.ssh/privatekey/id_rsa": invalid format
13:42:01 debug1: identity file /Users/smackie/Library/Group Containers/E78WKS7W4U.io.coressh.ssh/.ssh/privatekey/id_rsa type -1
13:42:01 debug1: identity file /Users/smackie/Library/Group Containers/E78WKS7W4U.io.coressh.ssh/.ssh/privatekey/id_rsa-cert type -1
13:42:01 debug1: Local version string SSH-2.0-OpenSSH_8.3
13:42:01 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.1
13:42:01 debug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.1 pat OpenSSH* compat 0x04000000
13:42:01 debug2: fd 5 setting O_NONBLOCK
13:42:01 debug1: Authenticating to skyport.cedaroffice.org:22 as 'smackie'
13:42:01 debug3: hostkeys_foreach: reading file "/Users/smackie/.ssh/known_hosts"
13:42:01 Authenticating…
13:42:01 debug3: record_hostkey: found key type ECDSA in file /Users/smackie/.ssh/known_hosts:9
13:42:01 debug3: load_hostkeys: loaded 1 keys from skyport.cedaroffice.org
13:42:01 debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
13:42:01 debug3: send packet: type 20
13:42:01 debug1: SSH2_MSG_KEXINIT sent
13:42:01 debug3: receive packet: type 20
13:42:01 debug1: SSH2_MSG_KEXINIT received
13:42:01 debug2: local client KEXINIT proposal
13:42:01 debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
13:42:01 debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,sk-ssh-ed25519@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
13:42:01 debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
13:42:01 debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
13:42:01 debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
13:42:01 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
13:42:01 debug2: compression ctos: none,zlib@openssh.com,zlib
13:42:01 debug2: compression stoc: none,zlib@openssh.com,zlib
13:42:01 debug2: languages ctos:
13:42:01 debug2: languages stoc:
13:42:01 debug2: first_kex_follows 0
13:42:01 debug2: reserved 0
13:42:01 debug2: peer server KEXINIT proposal
13:42:01 debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
13:42:01 debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
13:42:01 debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
13:42:01 debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
13:42:01 debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
13:42:01 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
13:42:01 debug2: compression ctos: none,zlib@openssh.com
13:42:01 debug2: compression stoc: none,zlib@openssh.com
13:42:01 debug2: languages ctos:
13:42:01 debug2: languages stoc:
13:42:01 debug2: first_kex_follows 0
13:42:01 debug2: reserved 0
13:42:01 debug1: kex: algorithm: curve25519-sha256
13:42:01 debug1: kex: host key algorithm: ecdsa-sha2-nistp256
13:42:01 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
13:42:01 debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
13:42:01 debug3: send packet: type 30
13:42:01 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
13:42:01 debug3: receive packet: type 31
13:42:01 debug1: Server host key: ecdsa-sha2-nistp256 SHA256:a66/9nh3gWtgef4gnXlwW3GwcmoIQQyDMFp1YcTExFo
13:42:01 debug3: hostkeys_foreach: reading file "/Users/smackie/.ssh/known_hosts"
13:42:01 debug3: record_hostkey: found key type ECDSA in file /Users/smackie/.ssh/known_hosts:9
13:42:01 debug3: load_hostkeys: loaded 1 keys from skyport.cedaroffice.org
13:42:01 debug3: hostkeys_foreach: reading file "/Users/smackie/.ssh/known_hosts"
13:42:01 debug3: record_hostkey: found key type ECDSA in file /Users/smackie/.ssh/known_hosts:9
13:42:01 debug3: load_hostkeys: loaded 1 keys from 63.249.66.100
13:42:01 debug1: Host 'skyport.cedaroffice.org' is known and matches the ECDSA host key.
13:42:01 debug1: Found key in /Users/smackie/.ssh/known_hosts:9
13:42:01 debug3: send packet: type 21
13:42:01 debug2: set_newkeys: mode 1
13:42:01 debug1: rekey out after 134217728 blocks
13:42:01 debug1: SSH2_MSG_NEWKEYS sent
13:42:01 debug1: expecting SSH2_MSG_NEWKEYS
13:42:01 debug3: receive packet: type 21
13:42:01 debug1: SSH2_MSG_NEWKEYS received
13:42:01 debug2: set_newkeys: mode 0
13:42:01 debug1: rekey in after 134217728 blocks
13:42:01 debug1: Will attempt key: /Users/smackie/Library/Group Containers/E78WKS7W4U.io.coressh.ssh/.ssh/privatekey/id_rsa  explicit
13:42:01 debug2: pubkey_prepare: done
13:42:01 debug3: send packet: type 5
13:42:01 debug3: receive packet: type 7
13:42:01 debug1: SSH2_MSG_EXT_INFO received
13:42:01 debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com>
13:42:01 debug3: receive packet: type 6
13:42:01 debug2: service_accept: ssh-userauth
13:42:01 debug1: SSH2_MSG_SERVICE_ACCEPT received
13:42:01 debug3: send packet: type 50
13:42:01 debug3: receive packet: type 51
13:42:01 debug1: Authentications that can continue: publickey,password
13:42:01 debug3: start over, passed a different list publickey,password
13:42:01 debug3: preferred publickey,keyboard-interactive
13:42:01 debug3: authmethod_lookup publickey
13:42:01 debug3: remaining preferred: keyboard-interactive
13:42:01 debug3: authmethod_is_enabled publickey
13:42:01 debug1: Next authentication method: publickey
13:42:01 debug1: Trying private key: /Users/smackie/Library/Group Containers/E78WKS7W4U.io.coressh.ssh/.ssh/privatekey/id_rsa
13:42:01 debug1: read_passphrase: can't open /dev/tty: Device not configured
13:42:03 debug3: sign_and_send_pubkey: RSA SHA256:9wep346Y8hSjfR/LJANd7L5QhqDJtYMua8SCjQyMR4A
13:42:03 debug3: sign_and_send_pubkey: signing using rsa-sha2-512 SHA256:9wep346Y8hSjfR/LJANd7L5QhqDJtYMua8SCjQyMR4A
13:42:03 debug3: send packet: type 50
13:42:03 debug2: we sent a publickey packet, wait for reply
13:42:03 debug3: receive packet: type 52
13:42:03 debug1: Authentication succeeded (publickey).
13:42:03 Authenticated to skyport.cedaroffice.org ([63.249.66.100]:22).
13:42:03 debug1: Remote connections from 0.0.0.0:10083 forwarded to local address localhost:22
13:42:03 debug3: send packet: type 80
13:42:03 debug1: ssh_init_forwarding: expecting replies for 1 forwards
13:42:03 debug2: fd 5 setting TCP_NODELAY
13:42:03 debug3: ssh_packet_set_tos: set IP_TOS 0x48
13:42:03 debug1: Requesting no-more-sessions@openssh.com
13:42:03 debug3: send packet: type 80
13:42:03 debug1: Entering interactive session.
13:42:03 Connected
13:42:03 debug1: pledge: network
13:42:03 debug2: fd 6 setting O_NONBLOCK
13:42:03 debug2: fd 7 setting O_NONBLOCK
13:42:04 debug3: receive packet: type 80
13:42:04 debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
13:42:04 debug3: receive packet: type 4
13:42:04 debug1: Remote: /home/smackie/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
13:42:04 debug3: receive packet: type 81
13:42:04 debug1: remote forward success for: listen 0.0.0.0:10083, connect localhost:22
13:42:04 debug1: forwarding_success: all expected forwarding replies received
13:42:19 debug3: send packet: type 80
13:42:19 debug3: receive packet: type 82
13:42:34 debug3: send packet: type 80
13:42:34 debug3: receive packet: type 82
13:42:49 debug3: send packet: type 80
13:42:49 debug3: receive packet: type 82
13:43:04 debug3: send packet: type 80
13:43:04 debug3: receive packet: type 82
13:43:19 debug3: send packet: type 80
13:43:19 debug3: receive packet: type 82
13:43:34 debug3: send packet: type 80
13:43:34 debug3: receive packet: type 82
13:43:49 debug3: send packet: type 80
13:43:49 debug3: receive packet: type 82
13:44:04 debug3: send packet: type 80

The id_rsa key is there in the location listed. It's header is

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,B9AE1E9EDA1DD4FB85385FD594E37DD2
4

I'm using ~/.ssh/config for all my settings, see below. You can do the same in Core Tunnel so maybe worth checking.

# $HOME/.ssh/config
# 
# Permissions: User R/W, NOT Writable by others
# chmod 700 $HOME/.ssh
# chmod 600 $HOME/.ssh/config
Host *
	IdentityFile ~/.ssh/id_rsa
	ServerAliveInterval 240
	ServerAliveCountMax 2
	TCPKeepAlive yes
5

09:30:40 load pubkey "/Users/smackie/Library/Group Containers/E78WKS7W4U.io.coressh.ssh/.ssh/privatekey/id_rsa": invalid format

This error does not affect the authentication, id_rsa is a private key and not a public key. Although the embedded OpenSSH printed this error message, it will try to extract public key from private key afterwards.

I guess the actual problem is your private key authentication was timed out, then embedded OpenSSH attempted to next auth method, aka. password authentication.

@mikael.fransson 's solution should work, tune the values of ServerAliveInterval and ServerAliveCountMax depend on your environment.

Another solution is only allow publickey authentication:

image
image1468×454 74.2 KB

Yang