Better database password management

[droguljic]

To align with current security best practices, database password management needs to go beyond one-time password generation and storing a static password in AWS Secrets Manager.

There are three primary approaches to handle database access and rotation, each with its own set of strengths and weaknesses,

1. RDS Managed - RDS integrates with Secrets Manager to generate, store, and rotate password automatically.
2. IAM Authentication - passwordless approach where short-lived tokens are used instead of password for connection to database.
3. Secrets Manager with Lambda - pre-built Lambda function template is integrated with Secrets Manager to rotate password.

Common to all three approaches is that none eliminates the master password on the database. Instead, their goal is to minimize exposure, automate rotation, and remove it from application configuration/environment.

Comparisons

To compare available approaches, 6 dimensions important for our common use cases will be explored:

	RDS Managed	IAM Authentication	Secrets Manager \w Lambda
Replica support	No	Yes	Yes
Multi AZ	Yes	Yes	Yes
Throughput	High	Low	High
Target User	Master	Any mapped DB user	Any DB user
Infrastructure complexity	Low	Medium	High
Application requirements	Low*	High	Medium

*"RDS Managed" approach has "Low" application requirements due to missing replica support.

Deep Dive

RDS Managed
Hassle-free approach with the easiest setup and minimal changes to application code, but lacks replica support and targets only master credentials.

IAM Authentication
Most secure approach without password rotation needs, but requires biggest application changes and has limit on new connections per second (~200) due to computational overhead of IAM authentication for the database.

Secrets Manager with Lambda
The configurable option with support for custom use cases and modest application changes, but with drastically increased infrastructure complexity due to many moving parts and transfer of password rotation responsibility to a consumer.

Implementation Note

All three approaches require changes to application code to drop static credentials handling for dynamic one. Meaning, no more use of environment variables to pass the password to the application. Instead, application code must dynamically fetch a password from Secrets Manager or generate auth token and implement logic for handling authentication errors due to stale passwords/tokens.

Recommendation

In a perfect scenario we would add support for all three approaches, but as that is not possible we should opt for one that best fits our ideal use case. Due to reliance on read replicas, "RDS Managed" option is for now out of scope, meaning our focus should be narrowed to choose between "IAM Authentication" and "Secrets Manager with Lambda" approaches.