Settings and configuration management | by Itiel Maayan | AT&T Israel Tech Blog | April 2022

Each app has settings and configurations. They are used to store identifying information such as database connection string, defaults, URLs and home directories, system behavior, environment, tenant, locale settings by default, time zone and language, etc. It was common to store this data in a file or database configuration, but this approach had many drawbacks and challenges.

In this article, I will offer a different approach – a complete solution that meets all common challenges and requirements. The proposed solution is a specification, not a specific tool or technology, and can be implemented using existing tools or libraries, or built from scratch.

When implementing a settings and configuration management solution, we can either build it ourselves or use an existing solution such as Spring Cloud Config Server and HashiCorp Consul.

I I personally prefer the “build” option, because existing products are, in most cases, too expensive (licensing fees or modifications and improvements of an open source) and lack features that I defined as mandatory in the proposed solution. When you To make the decision, you have to compare “buy” (or “use” in the case of open source) to “build”, compiling the costs and meeting your requirements.

If you decide to choose “build”, you can create from scratch or build on an existing product.

In this article, I will use two terms: settings and configuration. Parameters are the attributes I want to manage, such as log level, database connection string, home directory, resource url, etc. Configuration is the value of specific settings in a specific environment.

Example settings and configurations

In this article, I will refer to two different types of applications: “application” denotes the line-of-business application you are creating that should use the parameter configuration values, and the “P&C appwhich is the central application or library for managing settings and configurations – the solution I am describing.

The following functions are required for our solution:

  • Ability to manage settings and configurations for multiple apps, app tenants, environments, and app versions.
  • Rules to validate parameter configurations, such as optional values, default values, single or multiple values, whether the application needs different values ​​for each environment, etc.
  • Hierarchical scope with inheritance: application level or microservice level, general or specific version, etc.
  • Dynamic and immediate running configuration updates
  • Storing values ​​in a persistent store and in a distributed cache
  • Ability to encrypt values ​​and store secrets in a vault
  • Change log to enable auditing, debugging and comparison of environment values
Logical architecture of the P&C application

The following functions are required for implementation:

  • The application cannot start if in the specific environment and version a required parameter configuration value is missing
  • All environment configuration values ​​must be replicated from the P&C application database to the P&C application’s distributed cache (such as Redis), where the application will retrieve these values ​​when running at from the distributed cache of the P&C application.
  • For a parameter set as encrypted, the P&C application must encrypt the value when setting it and decrypt the value when retrieving it
  • For a parameter set as secret, the P&C application should store the value in a vault when setting it and retrieve the value from the vault when the value is needed
  • When a parameter is defined as cacheable, the application must retrieve the configuration value on application load and store it in the application cache. When the parameter is set to not cacheable, the application must retrieve the value each time it is used
  • In the P&C application, everything can be done via the GUI (management console) or via an API
  • Typically, settings and configurations are handled manually through the GUI, by CI/CD pipelines through APIs, or both (e.g. development environment manually and all other environments through pipelines)
  • There must be an API to create all configurations with default values ​​for a specific environment. When the API is called from the GUI, a helper must request all configurations that do not have a default value
  • When a P&C application is used for multiple applications or for multiple application tenants, an application ID and an application tenant ID must be added to all data model entities

The data model can be implemented in SQL or in NoSQL. To be more clear, I used SQL tabular format.

Coding conventions:

PK — Primary key

M—Mandatory

AG — Generated automatically

FK — Foreign key (reference index to another table primary key)

ERD data model

Environments

The list of all application environments, such as development, test, integration, load and stress, time travel, training, pre-prod, prod, etc.

In recent applications, there are usually several development environments. It can be a microservice developer, a specific developer, a specific team, etc. Each environment will have a unique identifier (there will be no simple “DEV” environment).

Table attributes:

  • [Optional: AppID (PK, M): The app ID in multiple apps implementation]
  • [Optional: AppTenantID (PK, M): The app tenant ID in multiple apps tenant implementation]
  • ID (PK, M): Abbreviation of the environment name, in uppercase only. Examples of IDs: PROD, PPROD, EDU, DEV, etc.
  • Name (M): The full name of the environment
  • Description: The description of the environment
  • ProductionData (M): Boolean, whether or not the environment contains production data, and therefore should be protected and monitored, just like the production environment. Default: False

Settings

The list of all application settings. Parameters are always backward and forward compatible. For example, we can add parameters and add parameter business logic (configuration options), but we can never delete a parameter or delete/modify a parameter configuration option.

Table attributes:

  • [Optional: AppID (PK, M): The app ID in multiple apps implementation]
  • [Optional: AppTenantID (PK, M): The app tenant ID in multiple apps tenantimplementation]
  • ID (PK, M): abbreviation of the parameter name, must not contain spaces and must be in camel case or with an underscore between words. The app and PC app will always use the converted ID in uppercase
  • Name (M): The full name of the parameter
  • Description: the description of the parameter
  • Default: the default configuration value
  • OptionalValues: Optional values ​​for this parameter, enclosed in braces, comma delimited, case sensitive. Optional values ​​or validation are required. Example: {ALL}, {DEBUG}, {FATAL}, {ERROR}, {INFO}
  • Validation: a regular expression to validate the configuration value. Optional values ​​or validation are required.
  • Single value (M): Boolean. Does the parameter have a single configuration value or does it support multiple configuration values? Default: True
  • MultipleValuesSeparator: the separator when multiple configuration values ​​are allowed. Mandatory when SingleValue is False
  • Scopes (M): The list of all allowed scopes. The scope can either be the entire application or specific microservices. The field can include a list of scopes delimited by braces. The scope value can be “APP” for all application or microservice names. Default: {APP}. The value {APP} must be included in the allowed list. The specific configuration is hierarchical. For example, if a microservice has no configuration value, the value “APP” is used.
  • MustBeDifferent: Environments where the parameter value cannot be the same (for example, the DEV and PROD database connection string must be different). The field includes braces, comma-delimited pairs of environments that must have a different value. For example, for the database connection string field, the values ​​could be: {PROD,PPROD}, {PROD,TEST}, {PROD,DEV}, {PPROD,TEST}, {PPROD,DEV}
  • Cacheable (M): boolean, if the configuration value can be modified during execution or not. If we can modify it at runtime, it cannot be cached and the application must retrieve the value before each use. Default: True
  • Encrypted (M): Boolean, is the value considered sensitive and must therefore be encrypted. Default: False. The parameter cannot have both the Encrypted and Secret values ​​set to True.
  • Secret (M): boolean, is the value considered secret and must therefore be encrypted and stored in a safe. Default: False. The parameter cannot have both the Encrypted and Secret values ​​set to True.

Environment Setup

The configuration values ​​of a specific setting for a specific environment. Setting and changing a value should add a record to the changelog table.

Table attributes:

  • ID (PK, AG): GUID
  • [Optional: AppID (FK, M): The app ID in multiple apps implementation]
  • [Optional: AppTenantID (FK, M): The app tenant ID in multiple apps tenant implementation]
  • EnvironmentID (FK, M): ID of the environment, referenced in the environment table
  • ParameterID (FK, M): Parameter ID, referenced in the parameter table
  • Scope (M): Configuration scope (application level or specific microservice level). Checked against allowed scopes
  • Version: The version to which this configuration value applies. A null value means any version. For a specific version, the application will use the specific version value if it exists, otherwise the null version value will be used. The application must ensure that a record with scope {ALL} and version Null exists
  • Value (M): The configuration value. Checked against the allowed values ​​(OptionalValues, Validation, SingleValue and MultipleValuesSeparator) and against the values ​​assigned in the MustBeDifferent environments value.

Change log

The configuration value change log changes. This table includes the initial setup configuration and any changes made.

Table attributes:

  • ConfigurationID (PK, FK, M): ID of the environment configuration, referenced in the environment configuration table
  • Datime (PK, AG): GMT date and time of the change
  • Value: The new value of the configuration

Comments are closed.