Quorum Key Manager - Error ST100: failed to create Hashicorp key

Question

I am testing Quorum Key Manager with Hashicorp Vault. I am using Quorum Hashicorp Plugin to be able to manage ethereum keys, sign transactions, etc., using Quorum image "consensys/quorum-hashicorp-vault-plugin". I've been able to have both images connected running, and i'm able to get, create, delete secrets from QKM, but when I try to create a key, or an ethereum account issuing an API call like this

curl -X POST --data '{"keyId":"my-key-account"}' -H "Content-Type:application/json" 'http://localhost:8080/stores/my-ethereum-store/ethereum'

I get the following error:

{"message":"ST100: failed to create Hashicorp key","code":"ST100"}

Looking into QKM log, the following error ir shown:

key-manager_1  | 2022-08-04T12:42:38.593Z   ERROR   stores  failed to create Hashicorp key  {"name": "hashicorp-keys", "vault": "hashicorp-vault", "secret_store": "", "id": "my-key-account", "error": "ST100: Error making API request.\n\nURL: PUT http://vault:8200/v1/secret/keys\nCode: 404. Errors:\n\n"}
key-manager_1  | 2022-08-04T12:42:38.593Z   INFO    auth.accesslog  172.20.0.1 - - [04/Aug/2022:12:42:38 +0000] "POST /stores/my-ethereum-store/ethereum HTTP/1.1" 404 66

My QKM manifest file is this:

- kind: Vault
  type: hashicorp
  name: hashicorp-vault
  specs:
    mount_point: secret
    address: http://vault:8200
    token: xxxxxxxxxxxxxxxxxxx

- kind: Store
  type: secret
  name: hashicorp-secrets
  specs:
    vault: hashicorp-vault

- kind: Store
  type: key
  name: hashicorp-keys
  specs:
    vault: hashicorp-vault

- kind: Store
  type: ethereum
  name: my-ethereum-store
  specs:
    key_store: hashicorp-keys

- kind: Node
  name: besu-node
  specs:
    rpc:
      addr: http://localhost:8545
    tessera:
      addr: http://localhost:9080

When my vault is started, looks like the plugin is correctly started and binded to the vault:

vault_1           | 2022-08-04T10:37:04.725Z [DEBUG] secrets.quorum-hashicorp-vault-plugin.quorum-hashicorp-vault-plugin_9da34603.quorum-hashicorp-vault-plugin: starting plugin: path=/vault/plugins/quorum-hashicorp-vault-plugin args=["/vault/plugins/quorum-hashicorp-vault-plugin"]
vault_1           | 2022-08-04T10:37:04.725Z [DEBUG] secrets.quorum-hashicorp-vault-plugin.quorum-hashicorp-vault-plugin_9da34603.quorum-hashicorp-vault-plugin: plugin started: path=/vault/plugins/quorum-hashicorp-vault-plugin pid=103
vault_1           | 2022-08-04T10:37:04.725Z [DEBUG] secrets.quorum-hashicorp-vault-plugin.quorum-hashicorp-vault-plugin_9da34603.quorum-hashicorp-vault-plugin: waiting for RPC address: path=/vault/plugins/quorum-hashicorp-vault-plugin
vault_1           | 2022-08-04T10:37:04.854Z [INFO]  expiration: revoked lease: lease_id=sys/wrapping/wrap/hf12037a61bf51568f888f165bd13fdd59d6aacdd6d418421718bd22e87e2365b
vault_1           | 2022-08-04T10:37:04.858Z [DEBUG] secrets.quorum-hashicorp-vault-plugin.quorum-hashicorp-vault-plugin_9da34603.quorum-hashicorp-vault-plugin.quorum-hashicorp-vault-plugin: 2022-08-04T10:37:04.858Z [DEBUG] plugin address: network=unix address=/tmp/plugin644251873
vault_1           | 2022-08-04T10:37:04.858Z [DEBUG] secrets.quorum-hashicorp-vault-plugin.quorum-hashicorp-vault-plugin_9da34603.quorum-hashicorp-vault-plugin: using plugin: version=4

Any ideas on what can I be doing wrong? Thanks!

Answer 1

Just for the records. Finally we were able to make it work. The problem is the use of the mount point secret which is the default and reserved for secrets. In case of QKM, another mount point must be specified (like quorum) to be able to save ethereum keys. Also, if we wanted to have both, secrets and keys in the same instance, we need to have two vaults defined, one for keys, and one for secrets. Our final configuration file looks like this:

# Hashicorp secret vault manifest
- kind: Vault
  type: hashicorp
  name: hashicorp-vault-secrets
  specs:
    mount_point: secret
    address: http://vault:8200
    token: xxxxxxxx

# Secret store manifest
- kind: Store
  type: secret
  name: hashicorp-secrets
  specs:
    vault: hashicorp-vault-secrets

# ---------------------------------

# Hashicorp key vault manifest
- kind: Vault
  type: hashicorp
  name: hashicorp-vault-keys
  specs:
    mount_point: quorum # different mount point
    address: http://vault:8200
    # token: xxxxxxxx # same! 

# Key store manifest
- kind: Store
  type: key
  name: hashicorp-keys
  specs:
    # secret-store: hashicorp-secrets
    vault: hashicorp-vault-keys

# ETH store manifest
- kind: Store
  type: ethereum
  name: my-ethereum-store
  specs:
    key_store: hashicorp-keys

# ---------------------------------

# GoQuorum node manifest
- kind: Node
  name: besu-node
  specs:
    rpc:
      addr: http://localhost:8545