Opensc-pkcs11.so

uthyr · May 23, 2024, 5:29pm

I am running macOS 14.5 on an M2 Studio.

I have installed opensc-pkcs11 using Homebrew. I specifically need this to access a private key on a hardware device (Nitrokey). I am able to successfully access the private key on the Nitrokey using openssh by specifiying "PKCS11Provider /Library/OpenSC/lib/opensc-pkcs11.so" in my ~/.ssh/config. I currently use iTerm2 for my default terminal.

However, when I specify the PKCS11Provider to this same value in Core Tunnel or Core Shell (or just let it be read from my ~/.ssh/config), I receive an error.

16:45:26 dlopen /Library/OpenSC/lib/opensc-pkcs11.so failed: dlopen(/Library/OpenSC/lib/opensc-pkcs11.so, 0x0002): tried: '/Library/OpenSC/lib/opensc-pkcs11.so' (code signature in <41F8E9B6-0716-3993-83CC-A985568638A4> '/Library/OpenSC/lib/opensc-pkcs11.so' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs), '/System/Volumes/Preboot/Cryptexes/OS/Library/OpenSC/lib/opensc-pkcs11.so' (no such file), '/Library/OpenSC/lib/opensc-pkcs11.so' (code signature in <41F8E9B6-0716-3993-83CC-A985568638A4> '/Library/OpenSC/lib/opensc-pkcs11.so' not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs)

This reads to me like a security feature disallowing two signed (?) files from two different applications to communicate. This strikes me as odd as I am using opensc-pkcs11.so and openssh, both installed from Homebrew.

Am I missing something? Am I doing this incorrectly or is there a workaround?

Thanks!

yang · May 24, 2024, 1:52am

Thanks a lot for reporting issue, version 3.8.8 is released for fix the problem, please do update let me know if this version not function.

Kindly Regards,

Yang

uthyr · May 24, 2024, 11:50am

Thanks for the update.

It appears that pkcs11.so is now accessed and communicates successfully! However, it finds no keys on the hardware device.

11:43:23 debug1: Connection established.
11:43:24 debug1: provider /Library/OpenSC/lib/opensc-pkcs11.so: manufacturerID <OpenSC Project> cryptokiVersion 2.20 libraryDescription <OpenSC smartcard framework> libraryVersion 0.25
11:43:25 debug1: pkcs11_register_provider: provider /Library/OpenSC/lib/opensc-pkcs11.so returned no slots

I receive a dialog for Password Authentication in Core Tunnel/Shell, but do not receive a dialog to interact with the hardware device (PIN entry or Multiplex Autoask Yes/No). I suspect this is preventing the hardware key from returning slot (key) information.

Thanks!

yang · May 24, 2024, 12:24pm

Could you please try again with this revised version:

Core Shell 3.8.8-PKCS11.zip

Kindly Regards,

Yang

yang · May 24, 2024, 12:26pm

BTW, does this work if run the equivalent command in Core Shell?

uthyr · May 24, 2024, 1:00pm

I tested in both Tunnel and Shell with the same result.

uthyr · May 24, 2024, 1:02pm

I tried the updated Core Shell binary in the zip file. Same issue, no dialog to enter PIN or Multiplex Autoask yes/no.

13:00:39 debug1: Connection established.
13:00:40 debug1: provider /Library/OpenSC/lib/opensc-pkcs11.so: manufacturerID <OpenSC Project> cryptokiVersion 2.20 libraryDescription <OpenSC smartcard framework> libraryVersion 0.25
13:00:41 debug1: pkcs11_register_provider: provider /Library/OpenSC/lib/opensc-pkcs11.so returned no slots

yang · May 24, 2024, 1:58pm

Do you try paste the ssh command to a Local shell of Core Shell? Does it work?

Yang

yang · May 24, 2024, 2:30pm

Please give the second revise version a try:
Core Shell 3.8.8-PKCS11-2.zip

Thank you,

Yang

uthyr · May 24, 2024, 2:32pm

Yes, if I launch 'Local Default' terminal in Core Shell and then run the command, it is successful.

ssh -v -o ServerAliveCountMax=3 -o ServerAliveInterval=15 -o ExitOnForwardFailure=yes -p XX example
...
debug1: Offering public key: cardno:XXXX ED25519 SHA256:XXXX agent
debug1: Server accepts key: cardno:XXXX ED25519 SHA256:XXXX agent
...
debug1: auto-mux: Trying existing master at '/Users/uthyr/.ssh/jump.sock'
debug1: mux_client_request_session: master session id: 4

uthyr · May 24, 2024, 2:47pm

The second revision is still unsuccessful. Now I am wondering if my dot files are being sourced to create a login environment which includes communication with my running ssh-agent?

yang · May 24, 2024, 2:59pm

Could you please send me desensitized verbose connection logs? Both success and failure logs, these logs could help me identified problem.

Thank you,

Yang

yang · May 24, 2024, 3:28pm

Well, I just got a YubiKey 4, will update this thread after the issue is reproduced.

Yang

yang · May 25, 2024, 3:26am

I did try YubiKey 4 with Core Tunnel version 3.8.8 (53E56), and seemed every thing works. Here are the steps how I use YubiKey for testing:

Reset then initialize the YubiKey, follow steps in Authenticating SSH with PIV and PKCS#11 (client) to create keys.
Install OpenSC and create a tunnel with PKCS11Provider directive set to /Library/OpenSC/lib/opensc-pkcs11.so
Connect the tunnel, and it shows using the YubiKey for authentication.

The PIN dialog:

Full connection log:

## 127.0.0.11 – PKCS11Provider ##

Equivalent Command: ssh -N -vvv -L 33333:codinn.com:22 -L 33334:localhost:22 -R localhost:3435:localhost:22 -o CheckHostIP=yes -o ServerAliveCountMax=3 -o ServerAliveInterval=15 -o PKCS11Provider=/Library/OpenSC/lib/opensc-pkcs11.so -o ConnectTimeout=15 -o TCPKeepAlive=no -o ExitOnForwardFailure=yes -p 22 yang@127.0.0.1
03:22:51 Connecting...
03:22:51 OpenSSH_8.8p1, OpenSSL 1.1.1l  24 Aug 2021
03:22:51 debug1: Reading configuration data /Users/yang/.ssh/config
03:22:51 debug1: Reading configuration data /etc/ssh/ssh_config
03:22:51 debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
03:22:51 debug1: /etc/ssh/ssh_config line 54: Applying options for *
03:22:51 debug2: resolve_canonicalize: hostname 127.0.0.1 is address
03:22:51 debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/Users/yang/.ssh/known_hosts'
03:22:51 debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/Users/yang/.ssh/known_hosts2'
03:22:51 debug3: ssh_connect_direct: entering
03:22:51 debug1: Connecting to 127.0.0.1 [127.0.0.1] port 22.
03:22:51 debug3: set_sock_tos: set socket 3 IP_TOS 0x48
03:22:51 debug2: fd 3 setting O_NONBLOCK
03:22:51 debug1: fd 3 clearing O_NONBLOCK
03:22:51 debug1: Connection established.
03:22:51 debug3: timeout: 15000 ms remain after connect
03:22:51 debug1: provider /Library/OpenSC/lib/opensc-pkcs11.so: manufacturerID <OpenSC Project> cryptokiVersion 2.20 libraryDescription <OpenSC smartcard framework> libraryVersion 0.25
03:22:51 debug1: provider /Library/OpenSC/lib/opensc-pkcs11.so slot 0: label <ssh> manufacturerID <piv_II> model <PKCS#15 emulate> serial <00000000> flags 0x40d
03:22:51 debug2: pkcs11_fetch_keys: provider /Library/OpenSC/lib/opensc-pkcs11.so slot 0: RSA SHA256:PdQXEDk9CVLzDOvOq1RuLFQ4FRCqe+CcpyKMCXLE4KE
03:22:51 debug1: have 1 keys
03:22:51 debug2: pkcs11_fetch_certs: provider /Library/OpenSC/lib/opensc-pkcs11.so slot 0: RSA SHA256:PdQXEDk9CVLzDOvOq1RuLFQ4FRCqe+CcpyKMCXLE4KE
03:22:51 debug2: pkcs11_fetch_certs: key already included
03:22:51 debug1: pkcs11_k11_free: parent 0x600003858160 ptr 0x600000f19960 idx 1
03:22:51 debug1: pkcs11_provider_unref: provider "/Library/OpenSC/lib/opensc-pkcs11.so" refcount 2
03:22:51 debug1: identity file /Users/yang/.ssh/id_rsa type 0
03:22:51 debug1: identity file /Users/yang/.ssh/id_rsa-cert type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_dsa type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_dsa-cert type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_ecdsa type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_ecdsa-cert type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_ecdsa_sk type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_ecdsa_sk-cert type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_ed25519 type 3
03:22:51 debug1: identity file /Users/yang/.ssh/id_ed25519-cert type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_ed25519_sk type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_ed25519_sk-cert type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_xmss type -1
03:22:51 debug1: identity file /Users/yang/.ssh/id_xmss-cert type -1
03:22:51 debug1: Local version string SSH-2.0-OpenSSH_8.8
03:22:51 debug1: Remote protocol version 2.0, remote software version OpenSSH_9.6
03:22:51 debug1: compat_banner: match: OpenSSH_9.6 pat OpenSSH* compat 0x04000000
03:22:51 debug2: fd 3 setting O_NONBLOCK
03:22:51 debug1: Authenticating to 127.0.0.1:22 as 'yang'
03:22:51 Authenticating...
03:22:51 debug3: record_hostkey: found key type ED25519 in file /Users/yang/.ssh/known_hosts:24
03:22:51 debug3: record_hostkey: found key type RSA in file /Users/yang/.ssh/known_hosts:25
03:22:51 debug3: record_hostkey: found key type ECDSA in file /Users/yang/.ssh/known_hosts:26
03:22:51 debug3: load_hostkeys_file: loaded 3 keys from 127.0.0.1
03:22:51 debug1: load_hostkeys: fopen /Users/yang/.ssh/known_hosts2: No such file or directory
03:22:51 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
03:22:51 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
03:22:51 debug3: order_hostkeyalgs: have matching best-preference key type ssh-ed25519-cert-v01@openssh.com, using HostkeyAlgorithms verbatim
03:22:51 debug1: pkcs11_k11_free: parent 0x60000385c210 ptr 0x0 idx 1
03:22:51 debug3: send packet: type 20
03:22:51 debug1: SSH2_MSG_KEXINIT sent
03:22:51 debug3: receive packet: type 20
03:22:51 debug1: SSH2_MSG_KEXINIT received
03:22:51 debug2: local client KEXINIT proposal
03:22:51 debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
03:22:51 debug2: host key algorithms: ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
03:22:51 debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
03:22:51 debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
03:22:51 debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
03:22:51 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
03:22:51 debug2: compression ctos: none,zlib@openssh.com,zlib
03:22:51 debug2: compression stoc: none,zlib@openssh.com,zlib
03:22:51 debug2: languages ctos: 
03:22:51 debug2: languages stoc: 
03:22:51 debug2: first_kex_follows 0 
03:22:51 debug2: reserved 0 
03:22:51 debug2: peer server KEXINIT proposal
03:22:51 debug2: KEX algorithms: sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-s,kex-strict-s-v00@openssh.com
03:22:51 debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
03:22:51 debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
03:22:51 debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
03:22:51 debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
03:22:51 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
03:22:51 debug2: compression ctos: none,zlib@openssh.com
03:22:51 debug2: compression stoc: none,zlib@openssh.com
03:22:51 debug2: languages ctos: 
03:22:51 debug2: languages stoc: 
03:22:51 debug2: first_kex_follows 0 
03:22:51 debug2: reserved 0 
03:22:51 debug1: kex: algorithm: curve25519-sha256
03:22:51 debug1: kex: host key algorithm: ssh-ed25519
03:22:51 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
03:22:51 debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
03:22:51 debug3: send packet: type 30
03:22:51 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
03:22:51 debug3: receive packet: type 31
03:22:51 debug1: SSH2_MSG_KEX_ECDH_REPLY received
03:22:51 debug1: Server host key: ssh-ed25519 SHA256:wNVGzlFtM7LlDCIm1akQ/8r6bG7BJjWYq0DAcYSSjLs
03:22:51 debug3: record_hostkey: found key type ED25519 in file /Users/yang/.ssh/known_hosts:24
03:22:51 debug3: record_hostkey: found key type RSA in file /Users/yang/.ssh/known_hosts:25
03:22:51 debug3: record_hostkey: found key type ECDSA in file /Users/yang/.ssh/known_hosts:26
03:22:51 debug3: load_hostkeys_file: loaded 3 keys from 127.0.0.1
03:22:51 debug1: load_hostkeys: fopen /Users/yang/.ssh/known_hosts2: No such file or directory
03:22:51 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
03:22:51 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
03:22:51 debug1: Host '127.0.0.1' is known and matches the ED25519 host key.
03:22:51 debug1: Found key in /Users/yang/.ssh/known_hosts:24
03:22:51 debug1: pkcs11_k11_free: parent 0x600003850580 ptr 0x0 idx 1
03:22:51 debug3: send packet: type 21
03:22:51 debug2: set_newkeys: mode 1
03:22:51 debug1: rekey out after 134217728 blocks
03:22:51 debug1: SSH2_MSG_NEWKEYS sent
03:22:51 debug1: expecting SSH2_MSG_NEWKEYS
03:22:51 debug3: receive packet: type 21
03:22:51 debug1: SSH2_MSG_NEWKEYS received
03:22:51 debug2: set_newkeys: mode 0
03:22:51 debug1: rekey in after 134217728 blocks
03:22:51 debug1: Will attempt key: PIV AUTH pubkey RSA SHA256:PdQXEDk9CVLzDOvOq1RuLFQ4FRCqe+CcpyKMCXLE4KE token
03:22:51 debug1: Will attempt key: /Users/yang/.ssh/id_rsa RSA SHA256:jT8FTKIyuj8CNzGCD6w2Bw+e172mxdTznx19SRQsNSg
03:22:51 debug1: Will attempt key: /Users/yang/.ssh/id_dsa 
03:22:51 debug1: Will attempt key: /Users/yang/.ssh/id_ecdsa 
03:22:51 debug1: Will attempt key: /Users/yang/.ssh/id_ecdsa_sk 
03:22:51 debug1: Will attempt key: /Users/yang/.ssh/id_ed25519 ED25519 SHA256:brLSA5srpw6fDYS3YFG52aYQuZTLh4EW4FcfCAw8Npk
03:22:51 debug1: Will attempt key: /Users/yang/.ssh/id_ed25519_sk 
03:22:51 debug1: Will attempt key: /Users/yang/.ssh/id_xmss 
03:22:51 debug2: pubkey_prepare: done
03:22:51 debug3: send packet: type 5
03:22:51 debug3: receive packet: type 7
03:22:51 debug1: SSH2_MSG_EXT_INFO received
03:22:51 debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256>
03:22:51 debug1: kex_input_ext_info: publickey-hostbound@openssh.com (unrecognised)
03:22:51 debug1: kex_input_ext_info: ping@openssh.com (unrecognised)
03:22:51 debug3: receive packet: type 6
03:22:51 debug2: service_accept: ssh-userauth
03:22:51 debug1: SSH2_MSG_SERVICE_ACCEPT received
03:22:51 debug3: send packet: type 50
03:22:51 debug3: receive packet: type 51
03:22:51 debug1: Authentications that can continue: publickey,password,keyboard-interactive
03:22:51 debug3: start over, passed a different list publickey,password,keyboard-interactive
03:22:51 debug3: preferred publickey,keyboard-interactive,password
03:22:51 debug3: authmethod_lookup publickey
03:22:51 debug3: remaining preferred: keyboard-interactive,password
03:22:51 debug3: authmethod_is_enabled publickey
03:22:51 debug1: Next authentication method: publickey
03:22:51 debug1: Offering public key: PIV AUTH pubkey RSA SHA256:PdQXEDk9CVLzDOvOq1RuLFQ4FRCqe+CcpyKMCXLE4KE token
03:22:51 debug3: send packet: type 50
03:22:51 debug2: we sent a publickey packet, wait for reply
03:22:51 debug3: receive packet: type 60
03:22:51 debug1: Server accepts key: PIV AUTH pubkey RSA SHA256:PdQXEDk9CVLzDOvOq1RuLFQ4FRCqe+CcpyKMCXLE4KE token
03:22:51 debug3: sign_and_send_pubkey: RSA SHA256:PdQXEDk9CVLzDOvOq1RuLFQ4FRCqe+CcpyKMCXLE4KE
03:22:51 debug3: sign_and_send_pubkey: signing using rsa-sha2-512 SHA256:PdQXEDk9CVLzDOvOq1RuLFQ4FRCqe+CcpyKMCXLE4KE
03:22:51 debug1: read_passphrase: requested to askpass
03:23:06 debug1: pkcs11_k11_free: parent 0x600003850790 ptr 0x0 idx 1
03:23:06 debug1: pkcs11_k11_free: parent 0x600003840000 ptr 0x0 idx 1
03:23:06 debug1: pkcs11_check_obj_bool_attrib: provider "/Library/OpenSC/lib/opensc-pkcs11.so" slot 0 object 105553161716192: attrib 514 = 0
03:23:10 debug3: send packet: type 50
03:23:10 debug1: pkcs11_k11_free: parent 0x6000038540b0 ptr 0x0 idx 1
03:23:10 debug3: receive packet: type 52
03:23:10 debug1: pkcs11_k11_free: parent 0x6000038582c0 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003858000 ptr 0x600000f4b180 idx 1
03:23:10 debug1: pkcs11_provider_unref: provider "/Library/OpenSC/lib/opensc-pkcs11.so" refcount 2
03:23:10 Authenticated to 127.0.0.1 ([127.0.0.1]:22) using "publickey".
03:23:10 debug1: check provider "/Library/OpenSC/lib/opensc-pkcs11.so"
03:23:10 debug1: pkcs11_provider_finalize: provider "/Library/OpenSC/lib/opensc-pkcs11.so" refcount 1 valid 1
03:23:10 debug1: pkcs11_provider_unref: provider "/Library/OpenSC/lib/opensc-pkcs11.so" refcount 1
03:23:10 debug1: Local connections to LOCALHOST:33333 forwarded to remote address codinn.com:22
03:23:10 debug3: channel_setup_fwd_listener_tcpip: type 2 wildcard 0 addr NULL
03:23:10 debug3: sock_set_v6only: set socket 5 IPV6_V6ONLY
03:23:10 debug1: Local forwarding listening on ::1 port 33333.
03:23:10 debug2: fd 5 setting O_NONBLOCK
03:23:10 debug3: fd 5 is O_NONBLOCK
03:23:10 debug1: channel 0: new [port listener]
03:23:10 debug1: Local forwarding listening on 127.0.0.1 port 33333.
03:23:10 debug2: fd 6 setting O_NONBLOCK
03:23:10 debug3: fd 6 is O_NONBLOCK
03:23:10 debug1: channel 1: new [port listener]
03:23:10 debug1: Local connections to LOCALHOST:33334 forwarded to remote address localhost:22
03:23:10 debug3: channel_setup_fwd_listener_tcpip: type 2 wildcard 0 addr NULL
03:23:10 debug3: sock_set_v6only: set socket 7 IPV6_V6ONLY
03:23:10 debug1: Local forwarding listening on ::1 port 33334.
03:23:10 debug2: fd 7 setting O_NONBLOCK
03:23:10 debug3: fd 7 is O_NONBLOCK
03:23:10 debug1: channel 2: new [port listener]
03:23:10 debug1: Local forwarding listening on 127.0.0.1 port 33334.
03:23:10 debug2: fd 8 setting O_NONBLOCK
03:23:10 debug3: fd 8 is O_NONBLOCK
03:23:10 debug1: channel 3: new [port listener]
03:23:10 debug1: Remote connections from localhost:3435 forwarded to local address localhost:22
03:23:10 debug3: send packet: type 80
03:23:10 debug1: ssh_init_forwarding: expecting replies for 1 forwards
03:23:10 debug2: fd 3 setting TCP_NODELAY
03:23:10 debug3: set_sock_tos: set socket 3 IP_TOS 0x48
03:23:10 debug1: Requesting no-more-sessions@openssh.com
03:23:10 debug3: send packet: type 80
03:23:10 debug1: Entering interactive session.
03:23:10 debug1: pledge: filesystem full
03:23:10 debug2: fd 9 setting O_NONBLOCK
03:23:10 debug2: fd 10 setting O_NONBLOCK
03:23:10 Connected
03:23:10 debug3: receive packet: type 80
03:23:10 debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
03:23:10 debug3: client_input_hostkeys: received RSA key SHA256:ua5jIGNjqD3rt8sXZlPn0T0pSB7BOwuoSwlgCqQl7R8
03:23:10 debug3: client_input_hostkeys: received ECDSA key SHA256:dGairz3UvCI2Dl9G/i4Um+wUwaHQQxRV0xCgqpasaeM
03:23:10 debug3: client_input_hostkeys: received ED25519 key SHA256:wNVGzlFtM7LlDCIm1akQ/8r6bG7BJjWYq0DAcYSSjLs
03:23:10 debug1: client_input_hostkeys: searching /Users/yang/.ssh/known_hosts for 127.0.0.1 / (none)
03:23:10 debug3: hostkeys_foreach: reading file "/Users/yang/.ssh/known_hosts"
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850370 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850630 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850370 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850630 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850370 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850630 ptr 0x0 idx 1
03:23:10 debug3: hostkeys_find: found ssh-ed25519 key at /Users/yang/.ssh/known_hosts:24
03:23:10 debug3: hostkeys_find: found ssh-rsa key at /Users/yang/.ssh/known_hosts:25
03:23:10 debug1: pkcs11_k11_free: parent 0x6000038504d0 ptr 0x0 idx 1
03:23:10 debug3: hostkeys_find: found ecdsa-sha2-nistp256 key at /Users/yang/.ssh/known_hosts:26
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850370 ptr 0x0 idx 1
03:23:10 debug3: hostkeys_find: found ssh-ed25519 key under different name/addr at /Users/yang/.ssh/known_hosts:32
03:23:10 debug1: pkcs11_k11_free: parent 0x6000038504d0 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850370 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850370 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850790 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850630 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850790 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850630 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850790 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850630 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850790 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850790 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850630 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x6000038504d0 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850790 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x6000038504d0 ptr 0x0 idx 1
03:23:10 debug1: pkcs11_k11_free: parent 0x600003850790 ptr 0x0 idx 1
03:23:10 debug1: client_input_hostkeys: searching /Users/yang/.ssh/known_hosts2 for 127.0.0.1 / (none)
03:23:10 debug1: client_input_hostkeys: hostkeys file /Users/yang/.ssh/known_hosts2 does not exist
03:23:10 debug3: client_input_hostkeys: 3 server keys: 0 new, 3 retained, 0 incomplete match. 0 to remove
03:23:10 debug1: client_input_hostkeys: no new or deprecated keys from server
03:23:10 debug1: pkcs11_k11_free: parent 0x6000038440b0 ptr 0x0 idx 1
03:23:10 debug3: receive packet: type 4
03:23:10 debug1: Remote: /Users/yang/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
03:23:10 debug3: receive packet: type 4
03:23:10 debug1: Remote: /Users/yang/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
03:23:10 debug3: receive packet: type 81
03:23:10 debug1: remote forward success for: listen localhost:3435, connect localhost:22
03:23:10 debug1: forwarding_success: all expected forwarding replies received
03:23:25 debug3: send packet: type 80
03:23:25 debug3: receive packet: type 82
03:23:40 debug3: send packet: type 80
03:23:40 debug3: receive packet: type 82

Could you please send me desensitized verbose connection logs? Both success and failure logs, these logs could help me identified problem.

Kindly Regards,

Yang

uthyr · May 25, 2024, 12:32pm

Thank you for testing.

Yes, I will work on getting logs today. Please stand by.

uthyr · May 25, 2024, 12:49pm

No port forwarding or multiplexing. A standard SSH connection for CLI.

Equivalent Command: ssh -v -o PKCS11Provider=/Library/OpenSC/lib/opensc-pkcs11.so -o ServerAliveCountMax=3 -o PreferredAuthentications=pubkey -o ServerAliveInterval=15 -o ExitOnForwardFailure=yes -p 38590 uthyr@REMOTEHOST
12:36:00 Connecting...
12:36:00 OpenSSH_8.8p1, OpenSSL 1.1.1l  24 Aug 2021
12:36:00 debug1: Reading configuration data /Users/uthyr/.ssh/config
12:36:00 debug1: Reading configuration data /Users/uthyr/.ssh/config.d/10.conf
12:36:00 debug1: Reading configuration data /Users/uthyr/.ssh/config.d/20.conf
12:36:00 debug1: Reading configuration data /Users/uthyr/.ssh/config.d/30.conf
12:36:00 debug1: Reading configuration data /Users/uthyr/.ssh/config.d/40.conf
12:36:00 debug1: /Users/uthyr/.ssh/config line 3: Applying options for *
12:36:00 debug1: Reading configuration data /etc/ssh/ssh_config
12:36:00 debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
12:36:00 debug1: /etc/ssh/ssh_config line 54: Applying options for *
12:36:01 debug1: Connecting to REMOTEHOST [192.168.1.230] port 38590.
12:36:01 debug1: Connection established.
12:36:02 debug1: provider /Library/OpenSC/lib/opensc-pkcs11.so: manufacturerID <OpenSC Project> cryptokiVersion 2.20 libraryDescription <OpenSC smartcard framework> libraryVersion 0.25
12:36:03 debug1: pkcs11_register_provider: provider /Library/OpenSC/lib/opensc-pkcs11.so returned no slots
12:36:03 debug1: identity file /Users/uthyr/.ssh/WorkStation.lan_ed25519 type 3
12:36:03 debug1: identity file /Users/uthyr/.ssh/WorkStation.lan_ed25519-cert type -1
12:36:03 debug1: Local version string SSH-2.0-OpenSSH_8.8
12:36:03 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
12:36:03 debug1: compat_banner: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
12:36:03 debug1: Authenticating to REMOTEHOST:38590 as 'uthyr'
12:36:03 debug1: SSH2_MSG_KEXINIT sent
12:36:03 Authenticating...
12:36:03 debug1: SSH2_MSG_KEXINIT received
12:36:03 debug1: kex: algorithm: curve25519-sha256
12:36:03 debug1: kex: host key algorithm: ssh-ed25519
12:36:03 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
12:36:03 debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
12:36:03 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
12:36:03 debug1: SSH2_MSG_KEX_ECDH_REPLY received
12:36:03 debug1: Server host key: ssh-ed25519 SHA256:HASH
12:36:03 debug1: load_hostkeys: fopen /Users/uthyr/.ssh/known_hosts2: No such file or directory
12:36:03 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
12:36:03 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
12:36:03 debug1: load_hostkeys: fopen /Users/uthyr/.ssh/known_hosts2: No such file or directory
12:36:03 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
12:36:03 debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
12:36:03 debug1: Host '[REMOTEHOST]:38590' is known and matches the ED25519 host key.
12:36:03 debug1: Found key in /Users/uthyr/.ssh/known_hosts:9
12:36:03 Host key fingerprint is SHA256:HASH
+--[ED25519 256]--+
|                 |
|                 |
|                 |
|                 |
|                 |
|                 |
|                 |
|                 |
|                 |
+----[SHA256]-----+
12:36:03 debug1: rekey out after 134217728 blocks
12:36:03 debug1: SSH2_MSG_NEWKEYS sent
12:36:03 debug1: expecting SSH2_MSG_NEWKEYS
12:36:03 debug1: SSH2_MSG_NEWKEYS received
12:36:03 debug1: rekey in after 134217728 blocks
12:36:03 debug1: Will attempt key: /Users/uthyr/.ssh/WorkStation.lan_ed25519 ED25519 SHA256:HASH explicit
12:36:03 debug1: SSH2_MSG_EXT_INFO received
12:36:03 debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
12:36:03 debug1: SSH2_MSG_SERVICE_ACCEPT received
12:36:03 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
12:36:03 debug1: No more authentication methods to try.
12:36:03 uthyr@REMOTEHOST: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
12:36:03 Abnormal Disconnect

yang · May 25, 2024, 2:18pm

Did you ever customized OpenSC config files? If you did, could you please send me a copy of your configuration?

Kindly Regards,

Yang

uthyr · May 25, 2024, 2:33pm

I have not modified any OpenSC config files. I have set up options for the hardware key to communicate with gpg-agent.

Workstation [~/.gnupg]$ cat gpg.conf                                                                         
no-emit-version
default-key <KEYID>
Workstation [~/.gnupg]$ cat gpg-agent.conf                                                                    
pinentry-program /opt/homebrew/bin/pinentry-mac
enable-ssh-support
default-cache-ttl 600
max-cache-ttl 7200
default-cache-ttl-ssh 600
max-cache-ttl-ssh 7200

yang · May 25, 2024, 2:45pm

OpenSC warns no slots found, so I guess there are permission issues, gpg-agent should not the cause. Could you please run following commands, and see what print:

cat /Library/OpenSC/etc/opensc.conf
pkcs11-tool -L
opensc-tool -l
pkcs15-tool -k
opensc-explorer

Output on my MacBook (M1 Pro, Sonoma 14.5):

❯ cat /Library/OpenSC/etc/opensc.conf
app default {
        # debug = 3;
        # debug_file = opensc-debug.txt;
}
❯ pkcs11-tool -L
Available slots:
Slot 0 (0x0): Yubico Yubikey 4 OTP+U2F+CCID
  token label        : ssh
  token manufacturer : piv_II
  token model        : PKCS#15 emulated
  token flags        : login required, rng, token initialized, PIN initialized
  hardware version   : 0.0
  firmware version   : 0.0
  serial num         : 00000000
  pin min/max        : 4/8
❯ opensc-tool -l
# Detected readers (pcsc)
Nr.  Card  Features  Name
0    Yes             Yubico Yubikey 4 OTP+U2F+CCID
❯ pkcs15-tool -k

Using reader with a card: Yubico Yubikey 4 OTP+U2F+CCID
Private RSA Key [PIV AUTH key]
        Object Flags   : [0x01], private
        Usage          : [0x2E], decrypt, sign, signRecover, unwrap
        Access Flags   : [0x1D], sensitive, alwaysSensitive, neverExtract, local
        Algo_refs      : 0
        ModLength      : 2048
        Key ref        : 154 (0x9A)
        Native         : yes
        Auth ID        : 01
        ID             : 01
❯ opensc-explorer
OpenSC Explorer version 0.25.1
Using reader with a card: Yubico Yubikey 4 OTP+U2F+CCID
unable to select MF: File not found

uthyr · May 28, 2024, 4:31pm

I realized that my defined gpg-agent.sock had been removed from the preferences when I upgraded (and was not re-populated by iCloud sync). When I added it back in back into the Core Tunnel/Shell Auth Agent preference, the hardware key correctly returned slots.

Thank you for all of your help