Example Docker configurations for running Sciveyor
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Charles Pence 2f987140d9
Add updated advice about secrets and env-vars.
1 year ago
.gitignore Add an initial docker-compose.yml. Still has startup order bugs. 1 year ago
COPYING Fix COPYING. 1 year ago
README.md Add updated advice about secrets and env-vars. 1 year ago
docker-compose.yml Add wait support to Docker Compose. 1 year ago
env-minimal Take out this header. 1 year ago

README.md

Sciveyor Logo

Sciveyor Docker Examples

This repository is designed to show examples of ways in which you can utilize the Docker images that come prepackaged with Sciveyor.

There are two included images: one to run Sciveyor's front-end web server, and one to run the backend analysis job worker.

Minimal Environment

The environment file env-minimal is designed to spin up a Sciveyor server that is connected (1) to an already running Solr instance (configured with the defaults from our Docker image) and (2) a locally running Postgres installation on the host, set up with access rights for your user account. (This is the basic setup that we recommend for development.) To that end, we connect to Solr at host.docker.internal (i.e., running on the host), and we directly mount /var/run/postgresql in the container so that these containers will have access to both.

To run the web server:

$ sudo docker run -d --name sciveyor-web \
    --env-file env-minimal \
    -p 3000:3000 \
    --add-host=host.docker.internal:host-gateway \
    -v /var/run/postgresql:/var/run/postgresql \
    pencechp/sciveyor-web

After a moment, the web front-end will appear at http://localhost:3000.

To run the analysis job worker:

$ sudo docker run -d --name sciveyor-worker \
    --env-file env-minimal \
    --add-host=host.docker.internal:host-gateway \
    -v /var/run/postgresql:/var/run/postgresq \
    pencechp/sciveyor-worker

The worker will not print anything to its logs until a job is created, but you'll be able to see the relevant ruby process running and waiting for tasks.

Note that you should not use a solution such as this in production, as this would require storing your secrets (database URL and password, Solr URL, etc.) as environment variables and/or in unencrypted environment files. This is insecure, and subject to a large number of potential cases of leakage (in logs, for instance). We recommend using something like Doppler, the AWS Secrets Manager, or a solution for connecting Docker secrets with environment variables.

Turnkey Solution: docker-compose

If you would like to run all the relevant servers for a Sciveyor installation, with effectively no configuration, you can do that via docker-compose. Simply execute docker-compose up. Postgres will be installed and the Sciveyor schema loaded, Minio will be installed to handle file storage (and a bucket will be created), and a web frontend and backend worker will be launched.

If you want to use this configuration as the basis for a production installation, the first thing to do is to mount the storage for Postgres, Solr, and file storage as Docker volumes; see the commented sections in the docker-compose.yml file to see how you might do that. More importantly, however, you will almost certainly want to host Solr, Postgres, the frontend, and the backend on different servers.

As above, this solution is not recommended in production: it requires the storage of secrets in your docker-compose.yml file. Investigate connecting docker-compose with Doppler, Vault, or Docker secrets.

Environment Variables

The available configuration parameters for Sciveyor are documented in our developer documentation. In particular, the following variables must be set:

  • SECRET_KEY_BASE and DEVISE_SECRET_KEY: Generate fresh random values for these variables by calling bin/rails secret in the Sciveyor source directory.
  • S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_BUCKET, optionally S3_REGION and S3_ENDPOINT: Credentials for an S3 bucket (or compatible storage) must be specified to give the web app and workers places to store files.
  • SOLR_URL: The URL to a running Solr installation loaded with the Sciveyor configuration. See our Docker image for easy setup.
  • DATABASE_URL: The URL to a Postgres database that has been loaded with our SQL schema.

Versioning

Version numbers for these Docker images follow the versioned releases of Sciveyor itself. You should, of course, always use the same version of the web and worker images.

License

As with Sciveyor itself, these Docker images are released under the MIT license.