Usage: Stand Alone¶
You will generally have three tabs (or windows) open in your terminal, which will be used for:
- Git operations.
Perform Git operations and general development in the repository,
git checkout master.
- Elasticsearch. Run an Elasticsearch (ES) instance. See instructions below.
- Django server. Start and stop the web server.
Server is started with
./runserver.sh, but see more details below.
What follows are the specific steps for each of these tabs.
1. Git operations¶
From this tab you can do Git operations, such as checking out our master branch:
git checkout master
Updating all dependencies¶
Each time you fetch from the upstream repository (this repo), run
This setup script will remove and reinstall the project dependencies
You may also run
if you only want to re-build the backend or front-end, respectively.
NODE_ENV environment variable can be set in your
.env file to either
production, which will affect how the build is made and what
gulp tasks are available. To install dependencies of one environment
or the other run
./frontend.sh (dependencies and devDependencies)
./frontend.sh production (dependencies but not devDependencies).
2. Run Elasticsearch (optional)¶
Elasticsearch is needed for certain pieces of this project but is not a requirement for basic functionality.
If Elasticsearch is installed via Homebrew, you can see instructions for running manually or as a background service using:
brew info elasticsearch
Typically to run Elasticsearch as a background service you can run:
brew services start elasticsearch
3. Launch Site¶
First, move into the
cfgov-refresh project directory
and ready your environment:
# Use the cfgov-refresh virtualenv. workon cfgov-refresh # cd into this directory (if you aren't already there) cd cfgov-refresh
From the project root, start the Django server:
If prompted to migrate database changes,
stop the server with
c and run these commands:
python cfgov/manage.py migrate ./initial-data.sh ./runserver.sh
To view the site browse to: http://localhost:8000
Using a different port
If you want to run the server at a port other than 8000 use
python cfgov/manage.py runserver <port number>
Specify an alternate port number, e.g.
To view the Wagtail admin login,
browse to http://localhost:8000/admin and login with username
admin (created in
initial-data.sh above; note that this
password will expire after 60 days).
Using HTTPS locally
To access a local server using HTTPS use
You'll need to ignore any browser certificate errors.
Available Gulp Tasks¶
There are a number of important gulp tasks,
which will build the project and test it, respectively.
Tasks are invoked via an
yarn run command so that the local gulp-cli can be used.
yarn run gulp -- --tasks command you can view all available tasks.
The important ones are listed below:
yarn run gulp build # Concatenate, optimize, and copy source files to the production /dist/ directory. yarn run gulp clean # Remove the contents of the production /dist/ directory. yarn run gulp lint # Lint the scripts and build files. yarn run gulp docs # Generate JSDocs from the scripts. yarn run gulp test # Run linting, unit and acceptance tests (see below). yarn run gulp test:unit # Run only unit tests on source code. yarn run gulp test:acceptance # Run only acceptance (in-browser) tests on production code. yarn run gulp audit # Run code quality audits.
Much of the guidance above for the "stand-alone" set-up still stands, and it is worth reviewing in full. Here are some things that might be different:
docker-composetakes care of running Elasticsearch for you, and all Elasticsearch, Postgres, and Python output will be shown in a single Terminal window or tab. (wherever you run
manage.pycommands can only be run after you've opened up a terminal in the Python container, which you can do with
- There is not yet a good way to use SSL/HTTPS, but that is in the works
- You won't ever need to use
How do I...¶
Use Docker Machine¶
If you used
mac-virtualbox-init.sh, then we used Docker Machine to create a
VirtualBox VM, running the Docker server. Here are some useful
- Start and stop the VM with
- get the current machine IP with
- if for some reason you want to start over,
docker-machine rm default, and
To enable Docker and Docker Compose commands, you'll always first need to run this command in any new shell:
eval $(docker-machine env)
It may be helpful to run
docker-machine env by itself, so you understand
what's happening. Those variables are what allows
docker-compose and the
docker command line tool, running natively on your Mac, to connect to the
Docker server running inside VirtualBox.
If you use autoenv (described in the stand-alone intructions) or something
similar, you might consider adding
eval $(docker-machine env) to your .env
file. You could also achieve the same results (and start the VM if it's not
running yet) with
Any further Docker documentation will assume you are either in a shell where
you have already run
eval $(docker-machine env), or you are in an environment
where that's not neccessary.
Run manage.py commands like migrate, shell, and dbshell, and shell scripts like refresh-data.sh¶
./shell.sh to open up a shell inside the Python container. From there,
cfgov/manage.py migrate should run as expected.
The same goes for scripts like
they will work as expected once you're inside the container.
In addition you can run single commands by passing them as arguments to
shell.sh, for example:
./shell.sh cfgov/manage.py migrate
./attach.sh to connect to the TTY session where
manage.py runserver is
running. If the app is paused at a PDB prompt, this is where you can access it.
Handle updates to Python requirements¶
If Compose is running, stop it with CTRL-C. Run:
docker-compose build python
This will update your Python image. The next time you run
the new requirements will be in place.
Set environment variables¶
Environment variables from your
.env file are sourced when the python container
starts and when you access a running container with
./shell.sh Your shell
environment variables, however, are not visible to applications running in Docker.
To add new environment variables, simply add them to the
.env file, stop compose
with ctrl-c, and start it again with
Get familiar with Docker Compose, and our configuration¶
docker-compose.yml contains a sort of "recipe" for running the site. Each entry in the Compose file describes a component of our application stack (Postgres, Elasticsearch, and Python), and either points to a public image on Dockerhub, or to a Dockerfile in cfgov-refresh. You can learn a lot more about Compose files in the docs
Similarly, a Dockerfile contains instructions for transforming some base image, to one that suits our needs. The Dockerfile sitting in the top level of cfgov-refresh is probably the most interesting. It starts with the public CentOS:7 image, and installs everything else neccessary to run our Python dependencies and the Django app itself. This file will only be executed:
- the first time you run
docker-compose up(or the first time after you re-create the Docker Machine VM)
- any time you run
That's why you need to run
docker-compose build after any changes to
There are other compose subcommands you might be interested in. Consider
top, and the
-d option for
Develop satellite apps¶
Check out any apps you are developing into the develop-apps directory. These will automatically be added to the PYTHONPATH, and apps contained within will be importable from Python running in the container.
For example, if your app is called 'foobar', in a repo called foobar-project, you could clone foobar-project in to develop apps:
git clone https://github.com/myorg/foobar-project
... which will create a directory at develop-apps/foobar-project. Assuming 'foobar' is at the top-level of 'foobar-project', you should be able to import it from your python code:
runserver has crashed! How do I start it again¶
In a separate terminal window or tab, running
docker-compose up python should
restart the server.