Pythonic

Learn more

Pydantic Series: Complex Settings Example

Managing application settings effectively is crucial for maintainability, security, and flexibility. In complex applications, configuration settings often span multiple layers, involve environment variables, and require dynamic validation. In this post, we’ll explore how Pydantic can be used to handle intricate application configurations, including hierarchical settings, conditional logic, and validation rules.

Why Pydantic for Application Settings?

Traditional configuration management involves parsing environment variables, .env files, or JSON/YAML configurations manually. This approach can quickly become unmanageable in large applications. Pydantic’s BaseSettings provides:

  • Structured validation to prevent misconfigurations.
  • Automatic parsing of environment variables.
  • Hierarchical configuration models for multi-layered applications.
  • Custom validation and transformation of settings before application boot.

Complex Application Configuration Example

Let’s build a real-world example: a microservices-based e-commerce platform with multiple layers of configuration, including:

  1. General application settings
  2. Database configurations
  3. Third-party API integrations
  4. Feature toggles (Feature Flags)
  5. Environment-specific overrides

Step 1: Define Base Settings

We’ll start by creating a base settings class:

from pydantic import BaseSettings, Field, PostgresDsn, RedisDsn, field_validator
from typing import Optional

class AppSettings(BaseSettings):
    app_name: str = Field("E-Commerce Platform", env="APP_NAME")
    environment: str = Field("development", env="ENVIRONMENT")
    debug: bool = Field(True, env="DEBUG")
    secret_key: str = Field(..., env="SECRET_KEY")

    @field_validator("environment")
    @classmethod
    def validate_environment(cls, value: str) -> str:
        allowed_envs = {"development", "staging", "production"}
        if value not in allowed_envs:
            raise ValueError(f"Invalid environment: {value}, must be one of {allowed_envs}")
        return value

    class Config:
        env_file = ".env"

This class ensures:

  • environment must be one of development, staging, or production.
  • Sensitive secret_key must be provided via environment variables.
  • Debug mode is enabled by default but can be overridden.

Step 2: Database Configuration

class DatabaseSettings(BaseSettings):
    database_url: PostgresDsn = Field(..., env="DATABASE_URL")
    redis_url: Optional[RedisDsn] = Field(None, env="REDIS_URL")
    pool_size: int = Field(10, env="DB_POOL_SIZE")
    timeout: int = Field(30, env="DB_TIMEOUT")

    @field_validator("pool_size", "timeout")
    @classmethod
    def validate_positive(cls, value: int) -> int:
        if value <= 0:
            raise ValueError("Database pool size and timeout must be positive integers")
        return value
  • The database_url and redis_url fields enforce valid connection strings.
  • Ensures pool_size and timeout are positive integers.

Step 3: Third-Party Integrations

class ThirdPartySettings(BaseSettings):
    stripe_api_key: Optional[str] = Field(None, env="STRIPE_API_KEY")
    sendgrid_api_key: Optional[str] = Field(None, env="SENDGRID_API_KEY")
    aws_s3_bucket: Optional[str] = Field(None, env="AWS_S3_BUCKET")

This allows for API keys and services to be loaded dynamically without hardcoding secrets.

Step 4: Feature Flags (Feature Toggles)

class FeatureFlags(BaseSettings):
    enable_experimental_checkout: bool = Field(False, env="ENABLE_EXPERIMENTAL_CHECKOUT")
    enable_advanced_analytics: bool = Field(False, env="ENABLE_ADVANCED_ANALYTICS")

Feature flags let us enable or disable features dynamically without redeploying the application.

Step 5: Aggregate All Settings

Finally, we combine everything into a unified settings manager:

class Settings(BaseSettings):
    app: AppSettings = AppSettings()
    database: DatabaseSettings = DatabaseSettings()
    third_party: ThirdPartySettings = ThirdPartySettings()
    features: FeatureFlags = FeatureFlags()

Now, we can access our settings like this:

settings = Settings()
print(settings.app.app_name)
print(settings.database.database_url)
print(settings.features.enable_experimental_checkout)

Environment-Specific Configuration

We can define different .env files for different environments:

.env (Development)

APP_NAME=E-Commerce Dev
ENVIRONMENT=development
DEBUG=True
DATABASE_URL=postgresql://dev_user:dev_pass@localhost/dev_db
REDIS_URL=redis://localhost:6379/0
SECRET_KEY=super-secret-key

.env (Production)

APP_NAME=E-Commerce Platform
ENVIRONMENT=production
DEBUG=False
DATABASE_URL=postgresql://prod_user:prod_pass@db.prod/prod_db
REDIS_URL=redis://cache.prod:6379/0
SECRET_KEY=super-secret-key-prod

By switching .env files, we can instantly configure our application for different environments.

Benefits of Using Pydantic for Settings

  1. Structured and Validated: Prevents configuration errors before application startup.
  2. Hierarchical Models: Organizes settings into logical sections.
  3. Environment-Specific: Easily switch configurations using .env files.
  4. Auto-Parsing: Automatically converts types (e.g., strings to integers, booleans).
  5. Security Best Practices: Keeps sensitive data out of code and allows for secure secrets management.

Conclusion

Managing application settings can be tedious and error-prone, but Pydantic simplifies the process with structured models, type validation, and environment integration. By leveraging BaseSettings, we can create highly scalable, secure, and maintainable configurations for any application.

Leave a comment