Puppet Code Manager setup issue with Bitbucket

Question

I have just installed puppet server enterprise and successfully added a few nodes and got some custom modules running also. I am now wanting to move to Code Manager before we get too deep in it.

I have followed the instructions for creating an empty Bitbucket repo here and initializing it with one single file environment.conf on a production branch as described in that link.

I have then followed the steps here to configure Code Manager but when I get to Test the control repository section to test the connection with puppet-code deploy --dry-run I get the following error:

--dry-run implies --all.
--dry-run implies --wait.
Dry-run deploying all environments.
2021/12/21 20:21:12 ERROR - [POST /deploys][500] Errors while collecting a list of environments to deploy (exit code: 1).
"/opt/puppetlabs/puppet/lib/ruby/gems/2.7.0/gems/rugged-0.27.7/lib/rugged/repository.rb:258: warning: Using the last argument as keyword parameters is deprecated\nERROR\t -\u003e Unable to determine current branches for Git source 'puppet' (/etc/puppetlabs/code-staging/environments)\nOriginal exception:\nFailed to authenticate SSH session: Unable to send userauth-publickey request at /opt/puppetlabs/server/data/code-manager/git/[email protected]\n"

I have added the puppet server's SSH pub key to the bitbucket repo's access tokens.

There are a few things in that error message im not fully understanding.

1. Unable to determine current branches for Git source 'puppet' - What is meant by source 'puppet' - my repo is called puppet-control-repo...?
2. Failed to authenticate SSH session: Unable to send userauth-publickey request - My puppet master's SSH keys are in the token list for that repo so confused here also.

Any guidance would be appreciated.

---

UPDATE (13-01-2022):

I can successfully clone on puppet server using command

git clone ssh://[email protected]:1234/project/puppet-control-repo.git --config core.sshCommand="ssh -i /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa"

Note sure why puppet is still returning:

Failed to authenticate SSH session: Unable to send userauth-publickey request

Answer 1

I don't know if you saw the instructions here https://puppet.com/docs/pe/2021.4/control_repo.html#managing_environments_with_a_control_repository but you can run

puppet infrastructure configure 

which makes sure the files have right permissions.

I would also test attempting a clone with keys works outside of code deploy

git clone -i /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa your_gir_url 

If this works it may be worth being aware of an issue we experienced on github https://puppet.com/blog/how-githubs-protocol-changes-impact-your-puppet-code-deployments/ which depending on bitbuckets approach to protocal may be having a similar affect.

We are updating docs to recommend the usage of more secure keys ed25519 creating as per the article.

if a manual clone doesnt work it suggests bitbucket doesn't have your public key correctly

Also a more complete debugging command is

runuser -u pe-puppet -- /opt/puppetlabs/puppet/bin/r10k -c /opt/puppetlabs/server/data/code-manager/r10k.yaml deploy environment production --puppetfile --verbose debug2

FOLLOWUP

On investigation we found https://support.puppet.com/hc/en-us/articles/227829007 which showed ssh:// was required at the start of r10k_remote making an example command of ssh://[email protected]:davidsandilands/control-repo.git

I have requested updates to https://support.puppet.com/hc/en-us/articles/227829007 to highlight this is not a version confined issue and asked for the puppet code manager configuration docs to be updated to reflect this may be required.

Answer 2

I see that you have a .pub file in the ssh directory. I believe it's expecting a private key there.

Also do you have the master class set up to point to your repo inside of Puppet Enterprise web ui?

You'll want to set the following parameters on that class.

code_manager_auto_configure =   true
r10k_private_key = $PRIVATE_KEY_IN_SSH_FOLDER_ABSOLUTE_PATH
r10k_remote = Your git URL

The PE Master can be found in Node Groups on the PE Web UI Node Groups -> PE Infrastructure -> PE Master

Answer 3

Thanks to @david-sandilands for helping me resolve this and guiding me to this article via the puppet community slack. Top guy!

EDIT 1:

The solution was documented here: https://support.puppet.com/hc/en-us/articles/227829007-Fix-your-Bitbucket-Stash-Code-Manager-configuration-in-Puppet-Enterprise-2015-3-to-2017-2

However the documentation was out of date as it affected version 2021.4 also.

In short:

r10k_remote = "ssh://[email protected]:1234/project/control-repo.git"

Not

r10k_remote = "[email protected]:1234/project/control-repo.git"

When working with Bitbucket Server.

EDIT 2:

Puppet have since updated their documentation: https://puppet.com/docs/pe/2021.5/code_mgr_config.html#code_mgr_enable