Announcing General Availability of conda 4.3
We’re excited to announce that conda 4.3 has been released for general availability. The 4.3 release series has several new features and substantial improvements. Below is a summary.
To get the latest, just run
Unlink and Link Packages in a Single Transaction: In the past, conda hasn’t always been safe and defensive with its disk-mutating actions. It has gleefully clobbered existing files; mid-operation failures left environments completely broken. In some of the most severe examples, conda can appear to “uninstall itself.” With this release, the unlinking and linking of packages for an executed command is done in a single transaction. If a failure occurs for any reason while conda is mutating files on disk, the environment will be returned its previous state. While we’ve implemented some pre-transaction checks (verifying package integrity for example), it’s impossible to anticipate every failure mechanism. In some circumstances, OS file permissions cannot be fully known until an operation is attempted and fails, and conda itself is not without bugs. Moving forward, unforeseeable failures won’t be catastrophic.
Progressive Fetch and Extract Transactions: Like package unlinking and linking, the download and extract phases of package handling have also been given transaction-like behavior. The distinction is that the rollback on error is limited to a single package. Rather than rolling back the download and extract operation for all packages, the single-package rollback prevents the need for having to re-download every package if an error is encountered.
Generic- and Python-Type Noarch/Universal Packages: Along with conda-build 2.1, a noarch/universal type for Python packages is officially supported. These are much like universal Python wheels. Files in a Python noarch package are linked into a prefix just like any other conda package, with the following additional features:
- conda maps the
directory to the correct location for the Python version in the environment,
- conda maps the
directory to either/bin
depending on platform,
- conda creates the Python entry points specified in the conda-build recipe, and
- conda compiles pyc files at install time when prefix write permissions are guaranteed.
Python noarch packages must be “fully universal.” They cannot have OS- or Python version-specific dependencies. They cannot have OS- or Python version-specific “scripts” files. If these features are needed, traditional conda packages must be used.
- conda maps the
Multi-User Package Caches: While the on-disk package cache structure has been preserved, the core logic implementing package cache handling has had a complete overhaul. Writable and read-only package caches are fully supported.
Python API Module: An oft requested feature is the ability to use conda as a Python library, obviating the need to “shell out” to another Python process. Conda 4.3 includes aconda.cli.python_api
module that facilitates this use case. While we maintain the user-facing command-line interface, conda commands can be executed in-process. There is also aconda.exports
module to facilitate longer-term usage of conda as a library across conda releases. However, conda’s Python code is considered internal and private, subject to change at any time across releases. At the moment, conda will not install itself into environments other than its original install environment.
Remove All Locks: Locking has never been fully effective in conda, and it often created a false sense of security. In this release, multi-user package cache support has been implemented for improved safety by hard-linking packages in read-only caches to the user’s primary user package cache. Still, users are cautioned that undefined behavior can result when conda is running in multiple process and operating on the same package caches and/or environments.
Conda now has the ability to refuse to clobber existing files that are not within the unlink instructions of the transaction. This behavior is configurable via thepath_conflict
configuration option, which has three possible values:clobber
. In 4.3, the default value isclobber
. This preserves existing behaviors, and it gives package maintainers time to correct current incompatibilities within their package ecosystem. In 4.4, the default will switch towarn
, which means these operations continue to clobber, but the warning messages are displayed. In 4.5, the default value will switch toprevent
. As we tighten up thepath_conflict
constraint, a new command line flag--clobber
will loosen it back up on an ad hoc basis. Using--clobber
overrides the setting forpath_conflict
to effectively beclobber
for that operation.
Conda signed packages have been removed in 4.3. Vulnerabilities existed, and an illusion of security is worse than not having the feature at all. We will be incorporating The Update Framework (TUF) into conda in a future feature release.
Conda 4.4 will drop support for older versions of conda-build.
Other Notable Improvements
A new “trace” log level is added, with output that is extremely verbose. To enable it, use-v -v -v
as a command-line flag, set averbose: 3
configuration parameter, or set aCONDA_VERBOSE=3
channel is now part of the default channels.
Package resolution/solver hints have been improved with better messaging.