The Conda Configuration Engine for Power Users
Released last fall, conda 4.2 brought with it configuration superpowers. The capabilities are extensive, and they’re designed with conda power users, devops engineers, and sysadmins in mind.
Configuration information comes from four basic sources:
- hard-coded defaults,
- configuration files,
- environment variables, and
- command-line arguments.
Each time a conda process initializes, an operating context is built that in a cascading fashion merges configuration sources. Command-line arguments hold the highest precedence, and hard-coded defaults the lowest.
The configuration file search path has been dramatically expanded. In order from lowest to highest priority, and directly from the conda code,
SEARCH_PATH = ( '/etc/conda/.condarc', '/etc/conda/condarc', '/etc/conda/condarc.d/', '/var/lib/conda/.condarc', '/var/lib/conda/condarc', '/var/lib/conda/condarc.d/', '$CONDA_ROOT/.condarc', '$CONDA_ROOT/condarc', '$CONDA_ROOT/condarc.d/', '~/.conda/.condarc', '~/.conda/condarc', '~/.conda/condarc.d/', '~/.condarc', '$CONDA_PREFIX/.condarc', '$CONDA_PREFIX/condarc', '$CONDA_PREFIX/condarc.d/', '$CONDARC', )
where environment variables and user home directory are expanded on first use.
$CONDA_ROOT is automatically set to the root environment prefix (and shouldn’t be set by users), and
$CONDA_PREFIX is automatically set for activated conda environments. Thus, conda environments can have their own individualized and customized configurations. For the “
.d” directories in the search path, conda will read in sorted order any (and only) files ending with .yml or .yaml extensions. The
$CONDARC environment variable can be any path to a file having a .yml or .yaml extension, or containing “condarc” in the file name; it can also be a directory.
Environment variables hold second-highest precedence, and all configuration parameters are able to be specified as environment variables. To convert from the condarc file-based configuration parameter name to the environment variable parameter name, make the name all uppercase and prepend
CONDA_. For example, conda’s
always_yes configuration parameter can be specified using a
CONDA_ALWAYS_YES environment variable.
Configuration parameters in some cases have aliases. For example, setting
always_yes: true or
yes: true in a configuration file is equivalent to the command-line flag
--yes. They’re all also equivalent to both
CONDA_YES=true environment variables. A validation error is thrown if multiple parameters aliased to each other are specified within a single configuration source.
There are three basic configuration parameter types: primitive, map, and sequence. Each follow a slightly different set of merge rules.
The primitive configuration parameter is the easiest to merge. Within the linearized chain of information sources, the last source that sets the parameter wins. There is one caveat: if the parameter is trailed by a
#!final flag, the merge cascade stops for that parameter. (Indeed, the markup concept is borrowed from the
!important rule in CSS.) While still giving end-users extreme flexibility in most cases, we also give sysadmins the ability to lock down as much configuration as needed by making files read-only.
Map configuration parameters have elements that are key-value pairs. Merges are at the per-key level. Given two files with the contents
# file: /etc/conda/condarc.d/proxies.yml proxy_servers: https: http://prod-proxy # file: ~/.conda/condarc.d/proxies.yml proxy_servers: http: http://dev-proxy:1080 https: http://dev-proxy:1081
proxy_servers configuration will be
proxy_servers: http: http://dev-proxy:1080 https: http://dev-proxy:1081
However, by modifying the contents of the first file to be
# file: /etc/conda/condarc.d/proxies.yml proxy_servers: https: http://prod-proxy #!final
the merged settings will be
proxy_servers: http: http://dev-proxy:1080 https: http://prod-proxy
Note the use of the
!final flag acts at the per-key level. A
!final flag can also be set for the parameter as a whole. With the first file again changed to
# file: /etc/conda/condarc.d/proxies.yml proxy_servers: #!final https: http://prod-proxy
the merged settings will be
proxy_servers: https: http://prod-proxy
http key defined.
The sequence parameter merges are the most involved. Consider contents of the three files
# file: /etc/conda/condarc channels: - one - two # file: ~/.condarc channels: - three - four # file: $CONDA_PREFIX/.condarc channels: - five - six the final merged configuration will be channels: - five - six - three - four - one - two
Sequence order within each individual configuration source is preserved, while still respecting sources’ overall precedence. Just like map parameters, a
!final flag can be used for a sequence parameter as a whole. However, the
!final flag does not apply to individual elements of sequence parameters, and instead
!bottom flags are available. Modifying the sequence example to the following
# file: /etc/conda/condarc channels: - one #!top - two # file: ~/.condarc channels: #!final - three - four #!bottom # file: $CONDA_PREFIX/.condarc channels: - five - six
will yield a final merged configuration
channels: - one - three - two - four
Managing all of these new sources of configuration could become difficult without some new tools. The most basic is
conda config --validate, which simply exits
0 if conda’s configured state passes all validation tests. The command
conda config --describe (recently added in 4.3.16) gives a detailed description of available configuration parameters.
We’ve also added the commands
conda config --show-sources and
conda config --show. The first displays all of the configuration information conda recognizes–in its non-merged form broken out per source. The second gives the final, merged values for all configuration parameters.
Conda’s configuration engine gives power users tools for ultimate control. If you’ve read to this point, that’s probably you. And as a conda power user, please consider participating in the conda canary program. Be on the cutting edge, and also help influence new conda features and behaviors before they’re solidified in general availability releases.