Environment Variables and Secrets

Environment variables are key-value pairs stored outside of the code itself, accessible to the application during runtime. This approach effectively separates configuration from the code, offering several benefits:

• Secure Storage of Sensitive Data: Sensitive information, such as database credentials or API keys, is not embedded directly into the code, reducing the risk of exposure.
• Ease of Configuration Management: Configuration can be easily managed and adapted based on the environment (development, staging, production, etc.), without modifying the code.

Example: Hardcoding Information vs. Using Environment Variables

For instance, consider accessing database from a Service in a traditional approach. You might hardcode the connection details directly into the code:

import psycopg2

# Connect to the database
conn = psycopg2.connect(
  database="your-database-name",
  user="your-username",
  password="your-password"
)

# Retrieve the secret value from the database
cursor = conn.cursor()
cursor.execute("SELECT some_value FROM table")
value = cursor.fetchone()[0]

However, if we hardcode the values, it will be difficult to manage across different environments since the database credentials will be different across dev, staging and prod. To solve this issue, you can utilize environment variables to store and access these values securely.

Configure environment variables like YOUR-DATABASE-URL, YOUR-USERNAME, and YOUR-PASSWORD during deployment configuration, and retrieve them within your code using os.environ(ENV_VAR_NAME)

import os

# Retrieve the secret value from the environment variable
database_url = os.environ['YOUR-DATABASE-URL']
database_url = os.environ['YOUR-USERNAME']
database_url = os.environ['YOUR-PASSWORD']

# Use the secret value to connect to the database
conn = psycopg2.connect(database_url)

# Retrieve the value from the database
cursor = conn.cursor()
cursor.execute("SELECT some_value FROM table")
value = cursor.fetchone()[0]

By separating configuration data from the code, you can seamlessly switch between different environments (production, development, etc.) without altering the code itself. Simply modify the environment variables during deployment configuration, allowing you to maintain a consistent codebase across different environments.

Secrets: Enhanced Protection for Highly Sensitive Data

For sensitive keys like database passwords, api-keys, its not advisable to provide them directly as environment variables in the deployment configuration since everyone can see the deployment spec and get access to these sensitive passwords. Hence, we usually store such sensitive values in secret managers (AWS SSM, AWS Secret Manager, GCP Secret Manager, Azure Vault or Hashicorp Vault) and then only provide the key to the secret in the environment configuration. Translating the key to the actual value is done at a different step.

Truefoundry provides an easy way to put the sensitive values in Secret Managers and then just provide the FQN of the secret (of the form tfy-secret://user:my-secret-group:my-secret) in the deployment spec. You can read about how to create the secrets here. Truefoundry will automatically fetch the value and inject it into the environment at runtime.

How to inject environment variables and secrets in TrueFoundry

Via the User Interface (UI)

Via the Python SDK

Both Service and Job classes have an argument env where you can pass a dictionary. The dictionary keys will be assumed as environment variable names and the values will be the environment variable values. In case of Secrets enter the Secret FQN in the value

from truefoundry.deploy import Build, Service, DockerFileBuild, Port

service = Service(
    name="my-service",
    image=Build(build_spec=DockerFileBuild()),
    ports=[
      Port(
        host="your_host",
        port=8501
      )
    ]
+    env={
+    	"MY_ENV_VAR_NAME": "MY_ENV_VAR_VALUE", 
+    	"MY_SECRET": "tfy-secret://user:my-secret-group:my-secret",
+    },
)
service.deploy(workspace_fqn="YOUR_WORKSPACE_FQN")