In this post, we’ll explore how to configure Hashicorp Vault and manage secrets in a Spring Boot application.

· Prerequisites
· Overview
∘ What is Vault?
∘ Why Vault?
· Set up HashiCorp Vault with PostgreSQL
∘ PostgreSQL configuration for Vault backend storage
∘ Vault HCL
∘ Initializing and Unsealing the HashiCorp Vault
· Continue reading
· References

This story is only a proof of concept (PoC) to illustrate how Hashicorp Vault works (NOT for a production environment). For best practices, see the Hashicorp Vault documentation.

Prerequisites

This is the list of all the prerequisites for Part 1:

Docker / Docker-compose installed

Overview

What is Vault?

HashiCorp Vault is an identity-based secrets and encryption management system. It provides encryption services that are gated by authentication and authorization methods to ensure secure, auditable and restricted access to secrets.

A secret is anything that you want to tightly control access to, such as tokens, API keys, passwords, encryption keys or certificates. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log.

https://developer.hashicorp.com/vault/docs/what-is-vault

Why Vault?

Vault was designed with these challenges in mind. It centralizes all of these credentials so that they are defined in one location, which reduces unwanted exposure to credentials. But Vault takes it a few steps further by making sure users, apps, and systems are authenticated and explicitly authorized to access resources while also providing an audit trail that captures and preserves a history of clients’ actions.

The key features of Vault are:

Secure Secret Storage: Arbitrary key/value secrets can be stored in Vault. Vault encrypts these secrets prior to writing them to persistent storage, so gaining access to the raw storage isn’t enough to access your secrets. Vault can write to disk, Consul, and more.
Dynamic Secrets: Vault can generate secrets on demand for some systems, such as AWS or SQL databases. For example, when an application needs to access an S3 bucket, it asks Vault for credentials, and Vault will generate an AWS keypair with valid permissions on demand. After creating these dynamic secrets, Vault will also automatically revoke them after the lease is up.
Data Encryption: Vault can encrypt and decrypt data without storing it. This allows security teams to define encryption parameters and developers to store encrypted data in a location such as a SQL database without having to design their own encryption methods.
Leasing and Renewal: All secrets in the Vault have a lease associated with them. At the end of the lease, Vault will automatically revoke that secret. Clients are able to renew leases via built-in renew APIs.
Revocation: Vault has built-in support for secret revocation. Vault can revoke not only single secrets, but a tree of secrets, for example, all secrets read by a specific user, or all secrets of a particular type. Revocation assists in key rolling as well as locking down systems in the case of an intrusion.

Set up HashiCorp Vault with PostgreSQL

The official Vault packages are available with supported package managers for macOS, Ubuntu/Debian, CentOS/RHEL, Amazon Linux, and Homebrew. This story uses the Docker approach.

PostgreSQL configuration for Vault backend storage

The PostgreSQL storage backend is used to persist Vault’s data in a PostgreSQL server or cluster. It does not automatically create the table. We need to add an SQL script to create the table and indexes.

Create a config directory and add a init.sql file with:

CREATE TABLE vault_kv_store (
  parent_path TEXT COLLATE "C" NOT NULL,
  path        TEXT COLLATE "C",
  key         TEXT COLLATE "C",
  value       BYTEA,
  CONSTRAINT pkey PRIMARY KEY (path, key)
);

CREATE INDEX parent_path_idx ON vault_kv_store (parent_path);

Vault HCL

HashiCorp Vault uses HCL (HashiCorp Configuration Language) for its configuration files (typically named vault.hcl). HCL is a declarative language designed by HashiCorp to configure their tools (like Vault, Terraform, Consul, and Nomad). Outside of development mode, Vault servers are configured using a file. The format of this file is HCL or JSON.

Create a vault.hcl file in theconfig directory:

storage "postgresql" {
  connection_url = "postgres://vaultuser:vaultpass@postgres:5432/vault?sslmode=disable"
  table = "vault_kv_store"          # Custom table name for secrets
  #max_parallel   = 4                 # Specifies the maximum number of concurrent requests to PostgreSQL.
  #ha_enabled     = true              # Enable High Availability mode
}

# TCP Listener Configuration
listener "tcp" {
  address = "0.0.0.0:8200"  # Bind to all network interfaces
  tls_disable = 1    # Disable TLS (not recommended for production - 🚨 Only for development!) 
  # tls_cert_file = "/vault/certs/cert.pem"  # Production TLS cert
  # tls_key_file  = "/vault/certs/key.pem"   # Production TLS key
}

# Cluster Configuration
api_addr = "http://vault:8200"         # External API addres
#cluster_addr  = "http://vault:8201"  # Internal cluster communication

# Administrative Options
ui = true        # Enable the web UI
default_lease_ttl = "768h"           # Default secret lifetime (32 days)
max_lease_ttl    = "8760h"           # Maximum secret lifetime (1 year maximum lease)
log_level     = "info"               # Logging level (trace, debug, info, warn, error)

Here’s a Docker Compose file to set up HashiCorp Vault with a PostgreSQL backend for persistent storage:

version: '3.8'

services:
  # PostgreSQL container for Vault storage backend
  postgres:
    image: postgres:15
    environment:
      POSTGRES_USER: vaultuser
      POSTGRES_PASSWORD: vaultpass
      POSTGRES_DB: vault
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./config/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    ports:
      - "5435:5432"
    networks:
      - vault-network
    restart: unless-stopped

  # Vault server container
  vault:
    image: hashicorp/vault:1.19
    container_name: vault
    environment:
      VAULT_ADDR: "http://0.0.0.0:8200"
      VAULT_API_ADDR: "http://0.0.0.0:8201"
      #VAULT_CACERT: "/vault/certs/cert.pem"  # For CLI trust
    volumes:
      - ./config:/vault/config # Stores Vault's configuration files.
      - ./vault-data:/vault/data # Stores Vault's data.
      #- ./certs:/vault/certs  # Mount certs directory
    ports:
      - "8200:8200"
    cap_add:
      - IPC_LOCK #ensures that the Vault container can lock memory, preventing sensitive in-memory data from being swapped out to disk.
    command:
      - "server"      
    healthcheck:
      test: ["CMD", "curl", "-f", "http://127.0.0.1:8200/v1/sys/health"]
      interval: 10s
      timeout: 5s
      retries: 5  
    depends_on:
      - postgres
    networks:
      - vault-network
    restart: unless-stopped


volumes:
  postgres_data:
  vault-data:

networks:
  vault-network:
    driver: bridge

Start the services:

docker-compose up -d

Vault service is ready:

Initializing and Unsealing the HashiCorp Vault

When you start a Vault server, it starts in a sealed state. In this state, Vault can access the physical storage, but it cannot decrypt any of the data on it.

Unsealing is the process of obtaining the plaintext root key that is necessary to read the decryption key. Prior to unsealing, the only possible Vault operations are to unseal the Vault and check the status of the server.

Open the browser to http://127.0.0.1:8200 and provide the key shares and the key threshold

By default, Vault splits the root key into 5 key shares and requires a quorum of 3 key shares to reconstruct the root key for use in rekey or unseal operations.

Vault returns the initial root token value and key shares obscured by squares. You can click an eye icon to view the information, a clipboard icon to copy the information to your system clipboard, or download the keys.

This is acceptable in certain circumstances, such as development or evaluation clusters, but you could require more protection of the key shares and initial root token when operating in production.

Click on the Continue to Unseal button, enter the Key(s) portion, and click on the Unseal button.

Based on the number of key shares and key threshold You will have to repeat the above step with a different key. (In our case, 3 keys)

The same initialization process can also be performed with the init operator command.

The login form should now appear. Enter “<root_token from downloaded file>” into the sign-in.

Continue reading

The second part of the post, which explains how to implement a Spring Boot 3 application, is available here.

Support me through GitHub Sponsors.

References

👉 Link to Medium blog

Spring Boot 3 — Manage Secrets using HashiCorp Vault and PostgreSQL as Backend: Part 1