Core Tunnel does not attempt to reconnect with requested credentials

[Dave_Anderson]

Yes, this sounds strange. Let me explain. I see this with two of my configs, one that Yang.Y spent considerable time helping me with about a month ago and another that is a fairly typical tunnel to multiple hosts. I can provide exports if needed.

Basically all SSH passwords that I will be discussing here are one-time passwords. Once they have been used they are invalidated. This is important to note as you will see shortly.

Basically, on any given work day I go about my business and Core Tunnel happily does its job without calling attention to itself. Thanks for making such a great product! At the end of the day I shut my MacBook and go home. The next day, I open up my MacBook to begin work. After a moment Core Tunnel figures out that two of my tunnels are down and attempts to reconnect. It displays the first of two password prompts. So far this is all entirely reasonable and as expected.

The problem arises when I enter my new password. Core Tunnel essentially ignores it. I can tell because the dialog disappears, then another password dialog appears(second tunnel), then disappears when I enter my password. At this point the main screen of Core Tunnel shows trhe two tunnels disconnected. If I then click on "Connect" on one of these, I am prompted for my password and this time it connects. I know that the password was not used on the prior two attempts or it would be invalid at this stage.

If there is more I can do to help characterize this problem I am happy to.

[yang (Yang.Y)]

Thanks a lot for elaborating this problem, could you please set Log Level to Debug3 and try to reproduce it?

Enable logging

The default logging level is set to Info, Debug1, Debug2 and Debug3 each specify higher levels of verbose output. [image] Core Shell Open or go to Hosts tab, right click on the host and select Edit Settings… from context menu, navigate to Connection tab, then scroll down to Log Level option. Press ⌘ I (or select Shell -> Show Inspector from main menu) reveals shell session information, as well as log content. [image] Core Tunnel Navigate to tunnel's Settings…, Connection tab, then scrol…

I'd really appreciate if you can send me the log text.

Thank you!

[Dave_Anderson]

Will do, thanks -- will post logs probably in ~24 hrs.

[Dave_Anderson]

OK, here goes. I did not connect the second tunnel because the first one demonstrated the problem, but I am including both to be comprehensive. The point of shutdown/restart is marked in both cases with '^^^'. Since I cannot post the full logs due to post size limits I am uploading/attaching.20190719 - LogsForReconnectIssue.txt (62.4 KB)

[yang (Yang.Y)]

Thank you Dave, I will try to fix this problem in next update.

[Dave_Anderson]

Thank you!

[yang (Yang.Y)]

The latest version 1.7.3 should have fixed this problem, could you please do the upgrade and give a try?

Thanks

[Dave_Anderson]

Thanks for the change and for the mention, I have lived with it for ~24 hrs and it seemed to work fine initially but this morning it failed to authenticate. I entered my new passwords in the dialogs that popped up this morning as I logged in and the dialogs did not reappear but the authentication did not work either. I am attaching logs for both the jump config that you helped me with and the other config that failed which connects to three databases. Look for '==' to see the beginning of the second log in the attached.

20190806_TunnelConsole.txt (311.8 KB)

Not shown in these logs -- I hit the connect buttons for each config again, entering the same credentials for one connection and new creds for the other and they both connected successfully.

[yang (Yang.Y)]

Thank you Dave, I admit that I didn't test with host jumping. Will investigate and try to fix it in next update.

[Dave_Anderson]

I'm not sure that this relates to host jumping. One part of that logfile is with host jumping, the other part is a log from a config without host jumping. I see someone else reported disconnects with 1.7.3 -- maybe related?

[yang (Yang.Y)]

I didn't receive any feedback from that issue report:

Connection drops and reconnects

Hello, After recent update (1.7.3), connection to a remote Ubuntu server drops and reconnects after short moment( 1 to 5 minutes). This connection opens a PostgreSQL tunnel so clients are getting disconnected in the middle of a query. (Tunnel is stable if opened from shell, in command line: ssh -v -L -g -N user@host) Thanks Nicolas

Also, no other disconnect issue report in 3 weeks, so I guess it is unrelated with this one.

I've found a potential threat from preventing reconnect, will release an update this week, I hope this time will fix it eventually.

Thank you,

[Dave_Anderson]

Thanks, we have been getting disconnects on an ongoing basis but they also happen in terminal window. Initially this did not appear to be the case but over time we are pretty sure. Sorry I neglected to update with that info here.

[yang (Yang.Y)]

@Dave_Anderson

but they also happen in terminal window.

Do you mean host in Core Shell, or just vanilla ssh command? Does the log you have uploaded reveals this problem?

Thanks

[Dave_Anderson]

I was trying to convey that what I thought was a problem unique to Core Tunnel(Connection drops) turned out to be happening in the terminal as well(system command line SSH).

At first it seemed that the terminal connection was stable but after a couple weeks of observation the problem has proven to be affecting both Core Tunnel and the terminal, and only on the office network. It's our problem.

[yang (Yang.Y)]

I see, thank you for the explanation

[Dave_Anderson]

The problem that I started this thread with is still present in the latest release(yes I installed the latest helper as well).

[Dave_Anderson]

This problem still exists. Is there anything I can do to help troubleshoot further?

[Dave_Anderson]

Any intent to fix this defect?

[yang (Yang.Y)]

Hi Dave,

Sorry for the late reply. I finally have got time to review the entire thread and investigate it again.

The problem is caused by time gap and oneshot passwords.

The next day, I open up my MacBook to begin work …… password was not used on the prior two attempts or it would be invalid at this stage.

The first two password attempts were actually prompted by the retried connection yesterday

You might ask why Core Tunnel could not detect underlying connection initiatively, and order out the password dialog?

The answer is Core Tunnel use OpenSSH as the core SSH component. Due to the internal implementation of OpenSSH, for various reasons (security or architecture), it won't detect the underlying connection status while prompting user for passwords.

You can reproduced the issue in Terminal with OpenSSH, not matter what values you set for ServerAliveInterval, ServerAliveCountMax, ConnectTimeout directives, ssh simply ignore the connection state while waiting for user input.

Could this issue be fixed in Core Tunnel? Theoretically yes, the Core Tunnel could set a timer for authentication dialog, and end the dialog initiatively on timeout.

I would like to label this issue an "improvement", and try to work it out in future release.

Thank you again for reporting!

Yang