Protecting Secrets / Variables Using HashiCorp Vault Secret Manager

Posted on by Admin
Protecting Secrets / Variables Using HashiCorp Vault Secret Manager

Related Post


Recently I was asked to create a new integration between Cisco ASD (Automated Software Distribution) and Sonatype Nexus Repository. As both expose a decent API, it wasn’t a complicated task (just a matter of ~800 lines of Python code 😳).

However, one thing troubled me… Both Cisco ASD and Nexus Repo required the following secrets/variables (see below). Hmmm… what is the best way to protect the values of these secrets/variables?

  • ASD client_id
  • ASD client_secret
  • Nexus repo_token
  • More…

One way is to use Environment Variables. That would be good, but not good enough as they are configured locally, and they are visible (delete-able, set-able) if you are familiar with a few OS commands.

Another way is to use User Inputs. For example:

But since we will have about eight sensitive inputs, that is too much “copy & paste.” Another way is to use a Secret Manager such as AWS Secret Manager, Azure Key Vault​​​​​​​, Hashicorp Vault, and few others.

In this article, I would like to elaborate more about the Hasicorp Vault as it was a great fit for the customer air-gap environment. Going forward, I’m going to call it Vault for simplicity (don’t confuse it with Ansible Vault or any other vault).

So.. what is a Secret Manager? Secret Manager is a secure storage system for API keys, passwords, certificates, and other sensitive data. Secret Manager provides a central place and single source of truth to manage access and is capable of reporting/audit secrets.

Ok.. and what is so special about Vault?

  1. Vault is a Go application with a Rest/CLI interface.
  2. Vault helps organizations manage access to secrets and transmit them safely.
  3. Secrets are defined as any form of sensitive credentials that need to be controlled and monitored (for example, API keys, SSH keys, RSA tokens, or OTP)
  4. Vault makes it easy to create detailed audit logs (who accessed what)
  5. Vault provides “encryption as a service” encrypting data in transit (with TLS) and at rest (using AES 256-bit CBC encryption). This protects sensitive data from unauthorized access in two major ways: as it travels across the network as well as in storage in your cloud and datacenters
  6. Vault uses a token to allow you to see info inside it, tokens can be created and revoked on demand.
  7. Vault tokens are TTL and Use-Limit configurable (for example, the following command create a token that is valid ONLY for 1 hour and can be used ONLY twice: vault token create -ttl=1h -use-limit=2)

Let’s review the basic installation and configuration of the Vault Dev Server (❗️ Note: Development mode SHOULD NOT be used in production as Dev Vault runs entirely in-memory and starts unsealed with a single unseal key. For production, please use the Production server)

1. Navigate to the HashiCorp Vault Download page
2. Choose your OS. For this article, we will choose Linux and Centos/RHEL

3. Install the Vault Dev Server:

  • sudo yum install -y yum-utils
  • sudo yum-config-manager –add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
  • sudo yum -y install vault
$ sudo yum install -y yum-utils
Last metadata expiration check: 0:01:06 ago on Sun 11 Jul 2021 03:55:26 PM UTC.
Package yum-utils-4.0.18-4.el8.noarch is already installed.
Dependencies resolved.
Nothing to do.
Complete!
​
​
$ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
Updating Subscription Management repositories.
Adding repo from: https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
​
​
$ sudo yum -y install vault
Updating Subscription Management repositories.
Unable to read consumer identity
​
Hashicorp Stable - x86_64 7.9 MB/s | 550 kB 00:00 
Dependencies resolved.
===================================================================================================================
Package Architecture Version Repository Size
===================================================================================================================
Installing:
vault x86_64 1.7.3-1 hashicorp 54 M
​
Transaction Summary
===================================================================================================================
Install 1 Package
​
Total download size: 54 M
Installed size: 191 M
Downloading Packages:
vault-1.7.3-1.x86_64.rpm 33 MB/s | 54 MB 00:01 
-------------------------------------------------------------------------------------------------------------------
Total 33 MB/s | 54 MB 00:01 
warning: /var/cache/dnf/hashicorp-164999f2fbadbd87/packages/vault-1.7.3-1.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID a3219f7b: NOKEY
Hashicorp Stable - x86_64 167 kB/s | 3.1 kB 00:00 
Importing GPG key 0xA3219F7B:
 Userid : "HashiCorp Security (HashiCorp Package Signing) <security+packaging@hashicorp.com>"
 Fingerprint: E8A0 32E0 94D8 EB4E A189 D270 DA41 8C88 A321 9F7B
 From : https://rpm.releases.hashicorp.com/gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1 
Running scriptlet: vault-1.7.3-1.x86_64 1/1 
Installing : vault-1.7.3-1.x86_64 1/1 
Running scriptlet: vault-1.7.3-1.x86_64 1/1 
Generating Vault TLS key and self-signed certificate...
Generating a RSA private key
.........................................................++++
....................................++++
writing new private key to 'tls.key'
-----
Vault TLS key and self-signed certificate have been generated in '/opt/vault/tls'.
​
Verifying : vault-1.7.3-1.x86_64 1/1 
Installed products updated.
​
Installed:
vault-1.7.3-1.x86_64 
​
Complete!

4. Start the Vault Dev Server: vault server -dev (make sure to capture the following two values: Unseal Key and Root Token)​​​​​​​

$ vault server -dev
==> Vault server configuration:
​
            Api Address: http://127.0.0.1:8200
                    Cgo: disabled
        Cluster Address: https://127.0.0.1:8201
             Go Version: go1.15.13
             Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
              Log Level: info
                  Mlock: supported: true, enabled: false
          Recovery Mode: false
                Storage: inmem
                Version: Vault v1.7.3
            Version Sha: 5d517c864c8f10385bf65627891bcxxxxxxxxxxxxx
​
==> Vault server started! Log data will stream in below:
​
2021-07-11T00:21:20.779Z [INFO] proxy environment: http_proxy="" https_proxy="" no_proxy=""
2021-07-11T00:21:20.780Z [WARN] no `api_addr` value specified in config or in VAULT_API_ADDR; falling back to detection if possible, but this value should be manually set
2021-07-11T00:21:20.781Z [INFO] core: security barrier not initialized
2021-07-11T00:21:20.782Z [INFO] core: security barrier initialized: stored=1 shares=1 threshold=1
2021-07-11T00:21:20.782Z [INFO] core: post-unseal setup starting
2021-07-11T00:21:20.793Z [INFO] core: loaded wrapping token key
2021-07-11T00:21:20.793Z [INFO] core: successfully setup plugin catalog: plugin-directory=""
2021-07-11T00:21:20.793Z [INFO] core: no mounts; adding default mount table
2021-07-11T00:21:20.796Z [INFO] core: successfully mounted backend: type=cubbyhole path=cubbyhole/
2021-07-11T00:21:20.796Z [INFO] core: successfully mounted backend: type=system path=sys/
2021-07-11T00:21:20.796Z [INFO] core: successfully mounted backend: type=identity path=identity/
2021-07-11T00:21:20.800Z [INFO] core: successfully enabled credential backend: type=token path=token/
2021-07-11T00:21:20.800Z [INFO] core: restoring leases
2021-07-11T00:21:20.800Z [INFO] rollback: starting rollback manager
2021-07-11T00:21:20.801Z [INFO] identity: entities restored
2021-07-11T00:21:20.801Z [INFO] identity: groups restored
2021-07-11T00:21:20.801Z [INFO] expiration: lease restore complete
2021-07-11T00:21:20.801Z [INFO] core: post-unseal setup complete
2021-07-11T00:21:20.802Z [INFO] core: root token generated
2021-07-11T00:21:20.802Z [INFO] core: pre-seal teardown starting
2021-07-11T00:21:20.802Z [INFO] rollback: stopping rollback manager
2021-07-11T00:21:20.802Z [INFO] core: pre-seal teardown complete
2021-07-11T00:21:20.802Z [INFO] core.cluster-listener.tcp: starting listener: listener_address=127.0.0.1:8201
2021-07-11T00:21:20.803Z [INFO] core.cluster-listener: serving cluster requests: cluster_listen_address=127.0.0.1:8201
2021-07-11T00:21:20.803Z [INFO] core: post-unseal setup starting
2021-07-11T00:21:20.803Z [INFO] core: loaded wrapping token key
2021-07-11T00:21:20.803Z [INFO] core: successfully setup plugin catalog: plugin-directory=""
2021-07-11T00:21:20.803Z [INFO] core: successfully mounted backend: type=system path=sys/
2021-07-11T00:21:20.803Z [INFO] core: successfully mounted backend: type=identity path=identity/
2021-07-11T00:21:20.803Z [INFO] core: successfully mounted backend: type=cubbyhole path=cubbyhole/
2021-07-11T00:21:20.804Z [INFO] core: successfully enabled credential backend: type=token path=token/
2021-07-11T00:21:20.805Z [INFO] core: restoring leases
2021-07-11T00:21:20.805Z [INFO] rollback: starting rollback manager
2021-07-11T00:21:20.805Z [INFO] identity: entities restored
2021-07-11T00:21:20.805Z [INFO] identity: groups restored
2021-07-11T00:21:20.805Z [INFO] expiration: lease restore complete
2021-07-11T00:21:20.805Z [INFO] core: post-unseal setup complete
2021-07-11T00:21:20.805Z [INFO] core: vault is unsealed
2021-07-11T00:21:20.808Z [INFO] core: successful mount: namespace="" path=secret/ type=kv
2021-07-11T00:21:20.818Z [INFO] secrets.kv.kv_405db499: collecting keys to upgrade
2021-07-11T00:21:20.818Z [INFO] secrets.kv.kv_405db499: done collecting keys: num_keys=1
2021-07-11T00:21:20.818Z [INFO] secrets.kv.kv_405db499: upgrading keys finished
WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.
​
You may need to set the following environment variable:
​
    $ export VAULT_ADDR='http://127.0.0.1:8200'
​
The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.
​
Unseal Key: HTxvqXgm+Y9+DwevmOZmNUUbeuZ9Yxxxxxxxxxxxxxxxx
Root Token: s.4Gl4TLJb1D82OWxxxxxxxxxx
​
Development mode should NOT be used in production installations!

5. Launch a new terminal session, copy and run the export VAULT_ADDR=’http://127.0.0.1:8200′ command from the terminal output. This will configure the Vault client to talk to the dev server

6. Verify that the Vault Dev Server is running: vault status

$ vault status
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 1.7.3
Storage Type inmem
Cluster Name vault-cluster-xxxxxxxx
Cluster ID 391d2141-a878-aab8-dc1e-xxxxxxxxxxx
HA Enabled false

7. Write a Secret(s): vault kv put secret/ap client_id=123456789 client_secret=987654321 repo_token=a1b2c3d4e5

Note: This command creates three different secrets (client_id, client_secret, and repo_token)

$ vault kv put secret/ap client_id=123456789 client_secret=987654321 repo_token=a1b2c3d4e5
Key Value
--- -----
created_time 2021-07-11T00:34:36.029268163Z
deletion_time n/a
destroyed false
version 1

8. Check the Vault secret: vault kv get secret/ap

$ vault kv get secret/ap
====== Metadata ======
Key Value
--- -----
created_time 2021-07-11T00:38:41.218990549Z
deletion_time n/a
destroyed false
version 1
​
======== Data ========
Key Value
--- -----
client_id 123456789
client_secret 987654321
repo_token a1b2c3d4e5

You just configured your first Vault Secret!

Now, let’s see how we can call the Vault secrets values using a simple Python3 code.

  1. Make sure to install and import both getpass and hvac (e.g., sudo pip3 install hvac)
  2. Using your preferred editor, create a new file named hello.py
  3. Copy and paste the following code
import getpass
import hvac
​
VAULT_ADDR = 'http://127.0.0.1:8200'
VAULT_TOKEN = getpass.getpass('Hashicorp Vault Token ID: ')
​
client = hvac.Client()
client = hvac.Client(
url = VAULT_ADDR,
token = VAULT_TOKEN
)
​
response = client.secrets.kv.read_secret_version(path='ap')
​
client_id = response['data']['data']['client_id']
client_secret = response['data']['data']['client_secret']
repo_token = response['data']['data']['repo_token']
​
print("Client ID: " + client_id)
print("Client Secret: " + client_secret)
print("Repo Token: " + repo_token)

Let’s review the code:

  • Lines #1 – #2: Import of the required modules (Hasicorp Vault and getpass)
  • Line #4: Define the HashiCorp Vault server IP/Hostname
  • Line #5: Define a masked user input to enter the Vault token ID
  • Lines #7 – #11: Vault function (note how we call the Vault address in line #9, and Vault token in line #10)
  • Line #13: Generate the secrets read request
  • Lines #15 – #17: Parse the JSON response for each variable
  • Lines #19 – #21: Print the parsed variables

4. Run the python code: python3 hello.py (when asked, enter the “Root Token” from step #4, above)

$ python3 hello.py
Hashicorp Vault Token ID: [ --> Root Token: s.4Gl4TLJb1D82OWxxxxxxxxxx]
Client ID: 123456789
Client Secret: 987654321
Repo Token: a1b2c3d4e5

You successfully read the secrets/variables from the Vault server using Python script!

This article aims to cover the very basic functionality of Hasicorp Vault. Vault seems like a promising option to consider. Simple installation, highly customizable, tokenization concept, supports IaC approach. Build-In integration for cloud makes vault a perfect match to manage cloud as well as on prems.

  1. This article aims to cover the very basic functionality of Hasicorp Vault.
  2. The above procedure is a “POC Grade” that uses the root token to retrieve the stored data. That is not a recommended use case for production.
  3. Want to learn more? Want to learn about Vault production installation and configuration? Please review the official documentation or contact the Cross-Domain TAB team.

Related resources

 

We’d love to hear what you think. Ask a question or leave a comment below.
And stay connected with Cisco DevNet on social!

LinkedIn | Twitter @CiscoDevNet | Facebook |  Developer Video Channel

Share:





Source link