ssh-agent seems broken

[jgonggrijp]

I have trouble using agent forwarding with iPadOS 16.6 and Blink 16.1.12.800 connecting over SSH to a Mac mini with macOS 15.6 (remote home).

Steps after a fresh startup of Blink, three attempts in total. I suspect the final output might hold a clue to what is wrong.

blink> ssh-add -l
# no output
blink> ssh -A home
Connected to ...
Last login: ...
julian@home ~ % ssh-add -l
4096 SHA256:XUildWaQ6N9tk/x6n3cguZEtPTxVtAhI6YNeOZcFsUI id_rsa (RSA)
4096 SHA256:XUildWaQ6N9tk/x6n3cguZEtPTxVtAhI6YNeOZcFsUI id_rsa (RSA)
julian@home ~ % cd a/dir/with/a/git/repo
julian@home ~ % git fetch
Enter passphrase for key '/Users/julian/.ssh/id_rsa': 

julian@home ~ % exit
blink> ssh-add
Key id_rsa - added to agent.
blink> ssh-add -l
4096 SHA256:XUildWaQ6N9tk/x6n3cguZEtPTxVtAhI6YNeOZcFsUI id_rsa (RSA)
blink> ssh -A home
Connected to ...
Last login: ...
julian@home ~ % ssh-add -l
4096 SHA256:XUildWaQ6N9tk/x6n3cguZEtPTxVtAhI6YNeOZcFsUI id_rsa (RSA)
4096 SHA256:XUildWaQ6N9tk/x6n3cguZEtPTxVtAhI6YNeOZcFsUI id_rsa (RSA)
4096 SHA256:XUildWaQ6N9tk/x6n3cguZEtPTxVtAhI6YNeOZcFsUI id_rsa (RSA)
julian@home ~ % cd a/dir/with/a/git/repo
julian@home ~ % git fetch
Enter passphrase for key '/Users/julian/.ssh/id_rsa': 

julian@home ~ % exit
blink> ssh-agent
Failed loading ssh_agent_main from MAIN, cause = dlsym(RTLD_MAIN_ONLY, ssh_agent_main): symbol not found
ssh-agent: command not found

[carloscabanero]

Hey @jgonggrijp, sorry for not jumping on this sooner. We recently changed the way that an existing key in the agent will get handled if it is the same key as used to sign in. Would you mind to update and let us know if this is now fixed from your side?

[jgonggrijp]

Blink version 16.2.1.826, identical results.

[carloscabanero]

Ok, so let me start with how to do what you are want to, and then I will explain why your suggestion does not work (although it maybe should).

Blink runs one Host Agent and one "global" or Default Agent (the one you control with ssh-add). The way to do what you want is to add the key to the Host Agent instead of the Default Agent. To do so, go to Hosts -> and in the SSH Agent section, select Agent Forwarding and the key you want to pass. You can select a "Confirm" constraint for the key, so Blink will ask you every time it gets requested by the remote (more secure), or just do "Always" for unconstrained access. This has multiple advantages, the key will only be forwarded to that host, you don't have to do ssh-add all the time and ssh -A is done on your behalf too.

Now, on why. Blink uses multiple agents as it is generally considered more secure. Multiple agents allows us to identify which target is asking for a key, and that is how we are able to "Constraint" keys and even show a prompt: "The host X is trying to use the key Y, do you allow it?". This is similar to what ssh-ident does. Each Host Agent is linked to the Default Agent (the one you used with ssh-add). This allows to forward specific keys for a host, but then still have a "global agent", for example with a Git key that can be forwarded to multiple places. The ssh-agent protocol is really barebones, and all these "tricks" are improvements that usually people build on-top but we built them as part of our agent.

The problem you found is that you are adding to the Default agent the same key that you used to sign in (the per Host Agent). So the key is in the Host Agent (constrained only for logging in) and in the Default Agent, and that is why you see it multiple times. When you are trying to use it, the Agent will deny using the key, as the key in the Host Agent is used first and it can only be used for logging in. When you add it as I described, then that constraint is removed. But you do not need to add it to the default agent.

So we broke the convention (or the expectation) of how things are supposed to work in this scenario vs the barebones and global ssh-agent. I will create an Issue as fixing this should not be hard. But truth is, from a product perspective, Blink does not need the "default" agent anymore for most cases. We should consider to hide it or to make this behavior more obvious somehow.

Anyway, hope this was not a lot, let me know if you have any other questions about it and how it goes.

[carloscabanero]

Fixed the regular behavior here: #1849

Should be out on next TF v17.0.0

[jgonggrijp]

The way to do what you want is to add the key to the Host Agent instead of the Default Agent. To do so, go to Hosts -> and in the SSH Agent section, select Agent Forwarding and the key you want to pass. You can select a "Confirm" constraint for the key, so Blink will ask you every time it gets requested by the remote (more secure), or just do "Always" for unconstrained access. This has multiple advantages, the key will only be forwarded to that host, you don't have to do ssh-add all the time and ssh -A is done on your behalf too.

While I didn't mention it, I tried this, too, and it didn't work. But I will double-check later today whether this is still the case.

[carloscabanero]

I tried this yesterday while doing #1849 and it was working without issue. Try it again when you get the chance and let me know.

[jgonggrijp]

I tried, and I hereby confirm that the problem is still there. My steps:

1. Go to settings, then hosts, then the home host.
2. "Agent Forwarding" was already set to "Confirm". I changed it to "Always" just to rule out interactions between different bugs.
3. "Forward keys" was already set to the key I wanted to forward.
4. Click "Save" in the top right corner of the host config page.
5. Click the back button in the top left until I can go back no further (in case this is needed to really truly actually store the settings).
6. Click outside of the settings pane to close it.
7. Purge Blink from memory and reopen.
8. Check that "Agent Forwarding" is still set to "Always" and "Forward Keys" is still set to the key I want to forward.
9. ssh home
10. ssh-add -l lists my key twice (I don't mind it it does, by the way, but I mention it for completeness).
11. I am prompted for the password of my SSH key on the remote host when I try to use SSH, just like in the example output I posted before.