Troubleshooting - Anaconda

This page details some common issues and their respective workarounds. For Anaconda installation or technical support options, visit our support offerings page.

403 error
Conda: Channel is unavailable/missing or package itself is missing
SSL issues
Using Redis

403 error

Problem

A 403 error is a generic Forbidden error issued by a web server in the event the client is forbidden from accessing a resource. The 403 error you are receiving may look like the following:

* Collecting package metadata (current_repodata.json): failed
* UnavailableInvalidChannel: The channel is not accessible or is invalid.
    * channel name: pkgs/main
    * channel url: https://repo.anaconda.com/pkgs/main
    * error code: 403

    * You will need to adjust your conda configuration to proceed.
    * Use `conda config --show channels` to view your configuration's current state, and use `conda config --show-sources` to view config file locations.

There are several reasons a 403 error could be received:

The user has misconfigured their channels in their configuration (for example, the secure location where the token is stored was accidentally deleted (most common)
A firewall or other security device or system is preventing user access (second most common)
We are blocking their access because of a potential terms of service violation (third most common)

Solution

First, run the following to undo your configuration:
```
conda config –remove-key default\_channels
```
Next, install or upgrade the conda-token tool:
```
conda install –freeze-installed conda-token
```
Lastly, re-apply the token and configuration settings:
```
# Replace <TOKEN> with your token
conda token set <TOKEN>
```

If this doesn’t resolve the issue, Anaconda recommends consulting our Terms of Service error page.

Conda: Channel is unavailable/missing or package itself is missing

Problem

After you have configured your .condarc, in some cases you may be unable to install packages. You may receive an error message that the channel or package is unavailable or missing.

Solution

One potential fix for all of these is to run the following command:

conda clean -i

This will clear the “index cache” and force conda to sync metadata from the repo server.

SSL issues

If you experience any trouble regarding SSL errors, confirm that you are adhering to the guidance in this section.

Moving CA certs to the docker-host

Anaconda recommends mounting the cacert.pem file to the docker-host and concatenating the required root CAs to that file on the docker-host, rather than copying the file to the containers.

First, place a cacert.pem file in ${BASE_INSTALL_DIR}/config/cacert.pem. You can obtain this file from any conda environment with certifi package installed ($<CONDA_PREFIX>/ssl/cacert.pem) or from one of the docker containers:
```
# Replace {BASE_INSTALL_DIR} with your base install directory.
docker cp $(docker ps | awk '/repo_api/ {print $1}'):/conda/ssl/cacert.pem  ${BASE_INSTALL_DIR}/config/cacert.pem
```

Concatenate the necessary root certs in pem format:

# Replace <YOUR_CERT> with your cert and {BASE_INSTALL_DIR} with your base install directory.
cat <YOUR_CERT> >> ${BASE_INSTALL_DIR}/config/cacert.pem

You now have two options for updating the application to use the docker-host cacert.pem:

Edit the docker-compose.yml file and add volume mounts to nginx_proxy, repo_api, repo_worker and repo_dispatcher, or

Use a separate .yml file (shown below) and let docker-compose merge it with the stock one when starting up:

cat << EOF > custom-cas.yml
services:
    nginx_proxy:
        volumes:
        - \${BASE_INSTALL_DIR}/config/cacert.pem:/conda/ssl/cacert.pem
    repo_api:
        volumes:
        - \${BASE_INSTALL_DIR}/config/cacert.pem:/conda/ssl/cacert.pem
    repo_worker:
        volumes:
        - \${BASE_INSTALL_DIR}/config/cacert.pem:/conda/ssl/cacert.pem
    repo_dispatcher:
        volumes:
        - \${BASE_INSTALL_DIR}/config/cacert.pem:/conda/ssl/cacert.pem
EOF
docker-compose -f docker-compose.yml -f custom-cas.yml up -d

Storing user credentials

If the proxy connection requires credentials, Anaconda recommends storing the credentials in the .env file (located in the same folder as the docker-compose.yml file) and referencing it in docker_compose.yml so that docker-compose.yml is readable for a broader set of users. During installation, an .env file is created alongside the docker-compose.yml file by default. You can edit this file and add variables to be referenced in the docker-compose.yml file as follows:

# example .env
BASE_INSTALL_DIR=/opt/anaconda/repo
DOCKER_REGISTRY=
DOMAIN=example.com
NGINX_PROXY_PORT=443
POSTGRES_HOST=postgres
POSTGRES_URI=postgresql://postgres:postgres@postgres/postgres
REDIS_ADDR=redis://redis:6379/0
VERSION=<YOUR_VERSION>
PROTOCOL=https
REPO_TOKEN_CLIENT_SECRET=<TOKEN>
REPO_KEYCLOAK_SYNC_CLIENT_SECRET=<SECRET>
DOCKER_UID=0
DOCKER_GID=0

# Added for example..
PROXY_USER=eden
PROXY_PW=eden-password

Then, merge the following with existing compose content in the docker-compose.yml file. Notice the environment variable from .env being referenced in the environment variables:

repo_worker:
  environment:
    - HTTP_PROXY=http://<PROXY_USER>:<PROXY_PW>@proxypy:8899
    - HTTPS_PROXY=http://<PROXY_USER>:<PROXY_PW>@proxypy:8899
repo_api:
  environment:
    - HTTP_PROXY=http://<PROXY_USER>:<PROXY_PW>@proxypy:8899
    - HTTPS_PROXY=http://<PROXY_USER>:<PROXY_PW>@proxypy:8899

Local mirrors

Another option is mirroring locally. You can do this by getting packages into an admin-managed channel—say, anaconda/main—and then mirroring filtered packages from that channel to other channels. This will make TE the source for the mirror. As such, TE needs to be able to validate its own certificate, which in most cases won’t work (in very much the same way PROXY with terminating SSL will not work). For this to work, then, you will most likely need to update the cacert.pem file of all containers. For this reason, Anaconda recommends hosting the cacert.pem file on the docker-host instead of the containers.

SSL verification error

Cause You may receive an SSL verification error if you have SSL enabled with a self-signed certificate. Solution Run the following command to disable SSL certificate check:

conda repo config --set ssl_verify False

Run the following to verify the above command worked:

conda repo config --show

OAuth2 with self-signed certificates

Cause Even if you have ssl_verify set to false while using self-signed certificates and SAML, you may still run into SSL verification errors. You may receive an error similar to the following:

ConnectionError: HTTPSConnectionPool(host='my_server_endpoint', port=443): Max retries exceeded with url: /endpoint (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fb73dc3b3d0>: Failed to establish a new connection: [Errno 110] Connection timed out',))

Solution Set an environment variable called a REQUESTS_CA_BUNDLE. For Windows, run the following:

SET REQUESTS_CA_BUNDLE=<PATH>
conda repo login

For Unix, run the following:

export REQUESTS_CA_BUNDLE=<PATH>
conda repo login

HTTP 000 CONNECTION FAILED

If you receive this error message, run the following command:

conda config --set ssl_verify false

Using Redis

Cause By default, Redis does not require a password. Not enabling a password requirement leaves your instance of Anaconda Server vulnerable. Solution Follow these steps to password protect your instance:

In the installation directory, update config/nginx/conf.d/repo.conf to include the add_header directive somewhere in the server block:
```
server {
    ...
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    ...
}
```
Create a directory called redis in the configs directory:
```
mkdir -p config/redis
```
Create a file called redis.conf inside config/redis with the following contents:
```
requirepass "mypassword"
```

Update the docker-compose.yml file to mount this custom redis config:

redis:
    image: ${DOCKER_REGISTRY}redis-ubi:${VERSION}
    restart: always
    volumes:
      - ${BASE_INSTALL_DIR}/config/redis/redis.conf:/usr/local/etc/redis/redis.conf
    command:
      - /usr/local/etc/redis/redis.conf

#Alternative:
#  redis:
#    image: ${DOCKER_REGISTRY}redis-ubi8:${VERSION}
#    restart: always
#    ports:
#    - 6379:6379
#    command:
#    - /usr/local/bin/redis-server
#    - --requirepass mypassword
# If you use this alternative configuration, there's no need to create a config/redis/redis.conf file to mount in.

Update REDIS_ADDR variable in .env file to include password:
```
...
REDIS_ADDR=redis://:mypassword@redis:6379/0
...
```
Restart docker-compose services so changes are picked up. You can do this using:
```
docker-compose up -d
```

Anaconda Server (6.3.0)

​403 error

​Problem

​Solution

​Conda: Channel is unavailable/missing or package itself is missing

​Problem

​Solution

​SSL issues

​Moving CA certs to the docker-host

​Storing user credentials

​Local mirrors

​SSL verification error

​OAuth2 with self-signed certificates

​HTTP 000 CONNECTION FAILED

​Using Redis

403 error

Problem

Solution

Conda: Channel is unavailable/missing or package itself is missing

Problem

Solution

SSL issues

Moving CA certs to the docker-host

Storing user credentials

Local mirrors

SSL verification error

OAuth2 with self-signed certificates

HTTP 000 CONNECTION FAILED

Using Redis