HashiCorp Vault troubleshooting

Vault is sealed

vault status shows Sealed: true; API returns 503. Normal after restart until unsealed. Run vault operator unseal with enough key shards (threshold from init). Auto-unseal via KMS should work automatically — if not, check cloud KMS permissions and seal config in vault.hcl. Apps fail with connection errors until unsealed.

Permission denied

Token lacks policy for the path/operation. Verify: vault token lookup and vault token capabilities secret/data/myapp/foo. Common mistakes: KV v1 path vs v2 (secret/data/ prefix), missing list on parent path, wrong mount name, or expired token. Fix policy and re-login.

Token expired or invalid

permission denied or invalid token. Check TTL with vault token lookup. Renew if renewable: vault token renew. Use AppRole or K8s auth for apps with automatic re-auth instead of long-lived static tokens. Root token should not be used by applications.

Connection refused / TLS errors

Wrong VAULT_ADDR (http vs https, port 8200), firewall, or Vault not listening. Check systemctl status vault and ss -tlnp | grep 8200. TLS verify failures: use proper CA or fix cert (avoid VAULT_SKIP_VERIFY in prod). Test: curl -s $VAULT_ADDR/v1/sys/health.

Secret not found (404)

Wrong path, deleted secret, or KV version destroyed. List: vault kv list secret/. KV v2: check metadata vault kv metadata get secret/myapp for versions. Soft-deleted data may be recoverable with vault kv undelete. Typo in mount (secret vs secrets) is frequent.

Standby node / read fails on follower

HA Raft cluster: only the active node serves writes; standbys return 429 on /sys/health or forward requests. Ensure load balancer or client targets the leader, or enable performance standby/replication per your architecture. vault status shows HA mode and if this node is active.

Kubernetes injection / sidecar fails

Pod stuck initializing — Vault Agent Injector or CSI driver cannot authenticate. Check pod annotations, service account, K8s auth role binding in Vault, and injector logs. Verify VAULT_ADDR reachable from cluster. See Kubernetes lab for pod events and init container logs.

Database engine credentials not issued

Dynamic DB secrets need the database secrets engine configured with connection URL, plugin, and role. Test connection from Vault: vault read database/config/mydb, vault read database/creds/my-role. DB must allow Vault's management user to create short-lived users. Check lease TTL and max TTL on the role.

Audit log or disk full

Audit devices log every request — can fill disk. Rotate audit log files; monitor Raft storage path. Vault refuses operations if storage backend is full. See disk volumes lab.

Debugging workflow

1. Cluster health

2. Auth and policy

3. Secret path and audit