Rsa-sha2-512 Incorrect signature type at connection

teleport
#1

Since Teleport doesn’t work with openssh v6.6 (look Teleport compatibility with OpenSSH v6) I built and installed a package with openssh v7.9.

This configuration works, but when I connect, I see the message:

agent key RSA-CERT SHA256:BfFp6VxoxN305UzfXfDpji84Y0gmCZe2zvknkquSJOY returned incorrect signature type

part of debug with -vvv flag:

debug1: Offering public key: teleport:vagrant RSA-CERT SHA256:BfFp6VxoxN305UzfXfDpji84Y0gmCZe2zvknkquSJOY agent 
debug3: send packet: type 50 
debug2: we sent a publickey packet, wait for reply 
debug3: receive packet: type 60 
debug1: Server accepts key: teleport:vagrant RSA-CERT SHA256:BfFp6VxoxN305UzfXfDpji84Y0gmCZe2zvknkquSJOY agent 
debug3: sign_and_send_pubkey: RSA-CERT SHA256:BfFp6VxoxN305UzfXfDpji84Y0gmCZe2zvknkquSJOY 
debug2: sign_and_send_pubkey: using private key "teleport:vagrant" from agent for certificate 
debug3: sign_and_send_pubkey: signing using rsa-sha2-512-cert-v01@openssh.com 
agent key RSA-CERT SHA256:BfFp6VxoxN305UzfXfDpji84Y0gmCZe2zvknkquSJOY returned incorrect signature type 
debug3: sign_and_send_pubkey: signing using ssh-rsa-cert-v01@openssh.com

is it possible to use rsa-sha2-512-cert-v01@openssh.com instead of ssh-rsa-cert-v01@openssh.com?
Teleport version:

vagrant@u100:~$ tctl version
Teleport v4.3.5 git:v4.3.5-0-g122349e78 go1.13.2

I specified ca_signature_algo in the config file:

ca_signature_algo: "rsa-sha2-512"

but it didn’t help

#2

After you specify the CA signature algo in the config file, you would need to perform a CA rotation with tctl auth rotate for this to take effect. Note: this will invalidate any existing public keys that you have exported with tctl auth export - you would need to export these again.

#3

Hello, @gus
I deleted the contents of /var/lib/teleport directory after changing the configuration.
I think that I don’t need to execute the tctl auth rotate command in this case.
But that didn’t solve the problem.

#4

I also performed rotate and this is what I get:

vagrant@u100:~$ sudo head -2 /etc/teleport.yaml   
teleport:
 ca_signature_algo: "rsa-sha2-512"

vagrant@u100:~$ sudo tctl auth rotate
Initiated certificate authority rotation. To check status use 'tctl status'
vagrant@u100:~$ sudo tctl status  
Cluster  main                                                                                 
Version  4.3.5                                                                                
User CA  initialized (mode: auto, started: Nov  3 09:30:05 UTC, ending: Nov  4 15:30:05 UTC)  
Host CA  initialized (mode: auto, started: Nov  3 09:30:04 UTC, ending: Nov  4 15:30:04 UTC)  
CA pin   sha256:cb7353b9c56f1aa9c47f30403c3782c2287fd3cf56c3ea7a5ff054fa376ba6ae
vagrant@u100:~$ sudo tctl auth export
@cert-authority *.main ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbfFpP7Ooc0AsTgnZkQa+w0MVFQTgIb3v18M1s5aBFqiuWfl3x9SBXdYvlOOWaqYyNvLKg0KL+ss9rc67ORfKECf2FqG0hQSXvTLS7hVA/CK3Dj
99gqdIGEcv1HM5CTl+qwgl7cGiJiO9lghuGBPyzwDIgQiL0ZsYLZM070X6ozcpokmGGcLC3tvK1bkKHFL1XV4Oc+o0LpPO5XrYKFrN7PkVdyMiHapSgH0++IwVIo1s2vL2yuD5+gopsVS6yLoXzJ3TG085IvBXHbZlCZglVkP67o
R7ZY8uAW4KNwvvhDOXi2Bu5llaOM0km5BB5IzqfbuonsEOKNGlbxsXuDtZD type=host
@cert-authority *.main ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwkgZHEWl1juGS3zXz6DUFTFakp6LZoNYOehypgsLCHAh2RrsM1fFaelJuB2Gny+4eS5Qr3lop7wETo1II/MWwl6zi7520ZLNXI8LCzpFOuqRI
RpAJFcs7jiCVzV1mP3WEQCotEOnVxVSaI+OdBmSU/Ze8MHW9BwwieGkPNKBdIDCeXDGZxD7b/3JAFKKOM8pu1F/TKcLUWxoYGa6CkZNVlhkR4j1UXLfcSrRF1hQt5BuzwB7/5bo0Adr0C0JY92E8gNIKRQlRLXgr0KfhpvzKLvSf
8h0EXaPxrgNEKrf5uGtsNxwLMvTdo+u5/S57R/9H2Z764rPW8rVnGllb08/ type=host
cert-authority ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYHQSRubUC8LEhONPZTRZRPJyqao4IxZ3UZa94vlN26fkFBxSFDrGQi/xqizVAD/y+qMbYFePxrscAyk4IHyUGxRGIlky9gDNgtBvrXqtyQS5pRI906ctpe
dkXYCT7Mt/9HY9Hjt9TPT2f/qkBXZCEwP6YcUV7V2FmjZmrnSXW9+2P7zkFQAcFm/FHssbeIV3hGHSID6nCTDiwwlpSefLZSDui+V31CzMqghYhOXIH0tNTb9Gw+UJA6EgXggsH5rzMeSsPVIMjFSYXqAgVAqcVEcORFkK6AIo3Q
ffILD7Cl40KdJZ8GOo4CXFU+nOezxhsEkyQhU5O0ed9N4t3vtXr clustername=main&type=user
cert-authority ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD01Tju98D1NACunCBqlms1170TLiVGpHCbpF4tEQloMxiDpK49gkjnjFHHfK2U2Cw+eu43uv5EmAiMTLA/iHuGlDobUCzeY/HAptv5l7hzDTlcqHZ8KNGcr
RDlVlVRGZk2fOIOhqMhhgG9pm3Q/t0Je9u86GOZE1N1+ULLrFOoDvI71l3A9wXQRu69guSDjs9IUthIi4wI9ezkP693e9gBASUTbixj6kt0N7i58+5w37c+iPFlpOHwwW3TW7SsYi3Dz58szED4DsMjJvBIIsbbCfPpaR8ZGX1SO
vmwlqXD51nyqghMCvnJ1D9g120CtjDhtW/s5F6qmRHa7KV7EMxd clustername=main&type=user

Why do I have two host and two user records when exporting? How can I check that sha2 is used?
When I try to execute rotate again I get an error:

ERROR REPORT:
Original Error: *trace.BadParameterError can not initate rotation while another is in progress
Stack Trace:
       /go/src/github.com/gravitational/teleport/lib/auth/clt.go:495 github.com/gravitational/teleport/lib/auth.(*Client).RotateCertAuthority
       /go/src/github.com/gravitational/teleport/tool/tctl/common/auth_command.go:277 github.com/gravitational/teleport/tool/tctl/common.(*AuthCommand).RotateCertAuthority
       /go/src/github.com/gravitational/teleport/tool/tctl/common/auth_command.go:104 github.com/gravitational/teleport/tool/tctl/common.(*AuthCommand).TryRun
       /go/src/github.com/gravitational/teleport/tool/tctl/common/tctl.go:137 github.com/gravitational/teleport/tool/tctl/common.Run
       /go/src/github.com/gravitational/teleport/tool/tctl/main.go:35 main.main
       /opt/go/src/runtime/proc.go:212 runtime.main
       /opt/go/src/runtime/asm_amd64.s:1358 runtime.goexit
User Message:
#5

When you rotate, there is a default 48 hour grace period where both old and new CA certs are valid (hence why you have 2 - one old and one new). This is to allow users and hosts which are temporarily disconnected from the cluster to come online with their old certs and automatically be issued new certs.

You can change the default grace period with the --grace-period flag - e.g. tctl auth rotate --grace-period 1h

You can check the signing algorithm like this:

rsa-sha2-512 (new default signing algorithm)

$ ssh-keygen -Lf ~/.tsh/keys/example.gravitational.io/admin-cert.pub | grep "Signing CA"
        Signing CA: RSA SHA256:WyjQwuwSLoZIaG9RZvxyT67PXyzTHra3tHfXW6Ssi3k (using rsa-sha2-512)

ssh-rsa (old default signing mechanism)

~ » ssh-keygen -Lf ~/.tsh/keys/example2.gravitaitonal.io/admin-cert.pub | grep "Signing CA"
        Signing CA: RSA SHA256:jshyEv8jcvqd3ySpnqETnupSOTNbkWZL9pD7PrhwVVE (using ssh-rsa)

This is because you’re still within the 48 hour grace period. Once that’s expired, you will be able to perform another rotation if desired.

1 Like
#6

Thank you so much for the information!
ssh-keygen doesn’t display the signing algorithm type in openssh v7.6. This works with openssh version higher than v8.* :grinning:

Another interesting fact:
We are having trouble connecting on fedora 33 (openssh v8.4).
When connecting, it displayed an error:

debug1: Local version string SSH-2.0-OpenSSH_8.4
vagrant@u100.mysite.com: Permission denied (publickey).
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535

The problem was related to the ssh_config configuration:

/etc/ssh/ssh_config.d/50-redhat.conf

specifically, in the policies used

Include /etc/crypto-policies/back-ends/openssh.config

they don’t have a policy ssh-rsa-cert-v01@openssh.com. I added ssh-rsa-cert-v01@openssh.com in PubkeyAcceptedKeyTypes and this solved the problem.

It seems to me that when using rsa-sha2-512, it would be logical to use rsa-sha2-512-cert-v01@openssh.com. Or am I wrong?
I think this would also solve the problem with the notice

agent key RSA-CERT SHA256:BfFp6VxoxN305UzfXfDpji84Y0gmCZe2zvknkquSJOY returned incorrect signature type
#7

@russjones @awly folks can you take a look at this question?

#8

Hi @jimdell!
Sorry for all the issues you’re hitting.
It looks like you have workarounds for now (except for the warnings from OpenSSH), so I’ll just provide some context.

The root issue is with the SSH library we use: https://github.com/golang/crypto/tree/master/ssh
It has a few outstanding issues for SHA2 support, since OpenSSH is deprecating SHA1, https://github.com/golang/go/issues/37278 is the main one.

There are open patches for both of those, but they all seem to be stuck in review.

There are several distinct parts of the SSH protocol where the signature algorithm appears (certificate signatures, certificate type names, SSH handshake signatures) and OpenSSH is phasing out support for SHA1 in them one at a time.

We found a workaround for generating host certiicates (that’s the ca_signature_algo), which keeps Teleport compatible with the latest OpenSSH versions, at least for now.

If the Go standard library maintainers keep neglecting SSH, we will eventually fork it. It is a non-trivial maintenance burden for us, so we’re still hopeful that we can wait this out (i.e. Go standard library accepts the patches).