How do I store secrets with Hiera and EYaml for Puppet 7?

Question

I am having some issue with the configuration of Cloudflare within Puppet and Hiera. I am trying to work out how Hiera will pick up encrypted data such as an API_token and know what it is and how to use it. I am working with the Cloudflare plugin for puppet. This is what my scripts are looking like at the moment:

hiera.yaml

    ---
    version: 5
    defaults:
      # The default value for "datadir" is "data" under the same directory as the hiera.yaml
      # file (this file)
      # When specifying a datadir, make sure the directory exists.
      # See https://puppet.com/docs/puppet/latest/environments_about.html for further details on environments.
      # datadir: data
      # data_hash: yaml_data
    hierarchy:
      - name: "Secret data: per-node, per-datacenter, common"
        lookup_key: eyaml_lookup_key # eyaml backend
        paths:
          - "secrets/nodes/%{trusted.certname}.eyaml"  # Include explicit file extension
          - "secrets/location/%{facts.whereami}.eyaml"
          - "common.eyaml"
        options:
          pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
          pkcs7_public_key:  /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
      - name: "Per-node data (yaml version)"
        path: "nodes/%{::trusted.certname}.yaml"
      - name: "application"
        paths:
          - "application/%{facts.eb_facts.application}-%{facts.eb_facts.stage}.yaml"
          - "application/%{facts.eb_facts.application}.yaml"
      - name: "Other YAML hierarchy levels"
        paths:
          - "common.yaml"

web.my.company.yaml

---
classes:
   - roles::www

nginx::nginx_servers:
  'www.my.company.com':
    # ssl: true
    www_root: '/var/www/www.my.company.com'

# letsencrypt::email: '[email protected]'
letsencrypt::certonly:
  email: '[email protected]'
  letsencrypt::domains: ['web.my.company.com']

www.pp

class roles::www {
  include profiles::nginxcd
  include profiles::letsencrypt
  include profiles::letsencrypt::plugin::dns_cloudflare
}

cloudflareDns.pp

class { 'letsencrypt::plugin::dns_cloudflare':
  api_token => 'cloudflare-dns-my.company.com',
}

web.my.company.eyaml

letsencrypt::plugin::dns_cloudflare:

  private: >
  ENC[PKCS7, "this is where the encrypted API-token sits" ] 

I am struggling to work out how to store encrypted data within eyaml and puppet using hiera lookup to know where to look for the data. I have a feeling that I am doing something wrong or just not understanding it completely. Any help or pointers on this would be greatly appreciated.

Answer 1

I am struggling to work out how to store encrypted data within eyaml and puppet using hiera lookup to know where to look for the data. I have a feeling that I am doing something wrong or just not understanding it completely.

Using Encrypted Data

From the Puppet manifest side, using data stored (encrypted or not) in an Hiera EYAML data source is no different from using data stored in any other Hiera data source (supposing that the hiera-eyaml extension is installed, everything is configured correctly, etc.). Hiera handles the various data sources and the data hierarchy itself transparently to Puppet.

Specifically, Hiera is a key / value store. Puppet asks it for the data associated with a given key, and Hiera provides that data (or undef). The two primary ways in which Puppet can be instructed to ask Hiera for data is via the built-in lookup() function (or one of the several deprecated hiera_* functions), or via automatic data binding to class parameters.

The latter (class parameter data binding) is by far the more common. Taking parameter $api_token of class letsencrypt::plugin::dns_cloudflare as an example, one would

• in Hiera, associate the wanted value for that parameter with key letsencrypt::plugin::dns_cloudflare::api_token (which is the fully-qualified name of the parameter)

• in the Puppet manifest(s) where that class is declared, avoid specifying a value for the parameter. Supposing that all parameter customization will be done via Hiera (recommended), that could take any of these forms:

  # include-like forms
  
  include letsencrypt::plugin::dns_cloudflare
  # or
  require letsencrypt::plugin::dns_cloudflare
  # or 
  contain letsencrypt::plugin::dns_cloudflare
  
  # resource-like form
  class { 'letsencrypt::plugin::dns_cloudflare': }
  

  Do use one of the include-like forms if at all possible. If you're not sure which to choose, then try include first. The resource-like form is similar to include, and it has only disadvantages relative to include where include is in fact viable (cases where no class parameters are explicitly specified in the manifest).

Storing Encrypted Data

It is data storage where encryption comes into play with EYAML. The hiera-eyaml documentation addresses this. The hiera-eyaml extension must be installed on the Puppet server (where the data files must also be), and a keypair must have been generated and stored where the configuration expects. With that in place, the included eyaml command-line tool is the preferred way to encrypt data for use with EYAML. You can either encrypt individual string values via eyaml encrypt and then insert the result into the eyaml file, or you can use eyaml edit to edit the file more directly.

Since you are just getting started with hiera-eyaml, I suggest beginning with eyaml encrypt. For example, if the text to be encrypted is my_cloudflare_token then you might execute

eyaml encrypt -s my_cloudflare_token

The result will be something like

string: ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAKNBCkXENUf6C0diKcV1VPvB4r8q+AFzu9E4VsR9Ch50q0UJ5sO977VXWLkX1oYbEvqPZZrmH122gvrYp1xux+W+UuFZbCzMQ7AMNe8eiJ7FvYYs79/leJIYouylfPod9G/M1SC/Lw64fhzcC7dSOru+vJan3zT1Jp/7nmsen263VBihOshbtkHKLSoJ7n96MlFqF0CrzOzxoz/p3y2591FoSXqjljCGG0PmV9FGONe1n5vUwWuy/+YQlciZEtyjyUBCZyJgaWfFh6//6vJT4G+5i0Ui1xzAtvYaDKW968Yx3ldQYy7btiRYct4Xvh6giFWDLXIE5Mnfe4fH6NwwXHDA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBAXOTJRuXWXBSfxIlA9HqWfgBBhi06bLLsVsjQ2leNYg2N5]

That (starting with ENC[) is what you enter as a value in your eyaml file. If that were the only entry in the file then the whole content might be this:

# secrets/nodes/some_certname.eyaml
---
letsencrypt::plugin::dns_cloudflare::api_token: ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAKNBCkXENUf6C0diKcV1VPvB4r8q+AFzu9E4VsR9Ch50q0UJ5sO977VXWLkX1oYbEvqPZZrmH122gvrYp1xux+W+UuFZbCzMQ7AMNe8eiJ7FvYYs79/leJIYouylfPod9G/M1SC/Lw64fhzcC7dSOru+vJan3zT1Jp/7nmsen263VBihOshbtkHKLSoJ7n96MlFqF0CrzOzxoz/p3y2591FoSXqjljCGG0PmV9FGONe1n5vUwWuy/+YQlciZEtyjyUBCZyJgaWfFh6//6vJT4G+5i0Ui1xzAtvYaDKW968Yx3ldQYy7btiRYct4Xvh6giFWDLXIE5Mnfe4fH6NwwXHDA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBAXOTJRuXWXBSfxIlA9HqWfgBBhi06bLLsVsjQ2leNYg2N5]

If you want later to change the value of the token, then you may find eyaml edit to be convenient. Using that to edit the .eyaml file will enable you to edit the decrypted data directly. When you save the result, the plaintext data you entered will be encrypted automatically (but do read the docs and examples for specifics and caveats).

A side note on hiera-eyaml generally

It is important to understand that Hiera-eyaml protects your data only as it sits statically on the Puppet server, in the original eyaml file. It will be decrypted for Puppet's benefit during catalog runs. The decrypted results will be recorded in generated catalogs, cached in that form on the server, and conveyed in that form to the client. Moreover, the private key with which the data can be decrypted needs to be stored on the server too.

As a result, EYAML doesn't offer very much protection against an adversary who obtains privileged access to the server. Of course, conventional access controls suffice against adversaries who obtain only unprivileged access, mandatory access controls (e.g. SELinux) can defend against some adversaries with privilege, and, ideally, one hopes to prevent an adversary gaining any access at all. I suppose there's a defense-in-depth argument for engaging hiera-eyaml, but do not be oversold on the benefit.

I urge you to read the Puppet documentation on securing sensitive data, and to employ all the mechanisms that make sense for your use case.