Developer’s guide¶

This section contains information on how to perform various task and an overview of how our development infrastructure is set up.

Local development¶

These tasks are usually performed on an individual developer’s machine.

Make database migrations¶

The makemigrations command cannot be run directly since the application directory is mounted read-only within the container. The work around is a little ugly at the moment:

In compose/development.yml, comment out the read_only: true value from the volume defined in development_app.
Run ./compose.sh development run --rm development_app ./manage.py makemigrations ... as usual.
Uncomment the read_only: true commented out in 1.
Change permissions on the migration: sudo chmod $USER path/to/migration.py

Run the test suite¶

The tox automation tool is used to run tests inside their own virtualenv. This way we can be sure that we know which packages are required to run the tests. By default tests are run in a Postgres database created by docker-compose. Other databases can be used by setting the DJANGO_DB_... environment variables. See Specifying the database.

$ ./tox.sh

By default, tox will run the test suite using the version of Python used when we deploy and will compile a local version of the documentation. The -e flag may be used to explicitly specify an environment to run. For example, to build only the documentation:

$ ./tox.sh -e doc

Tox runs tests within persistent virtualenvs. Tox attempts to determine when it should rebuild the virtualenv but its logic can sometimes be imperfect. This is especially true in the case of dependencies hidden deep in the various requirements/{...}.txt files. Should it appear that a module is not installed when it should be, the -r flag can be passed to tox:

$ ./tox.sh -r          # recreate all environments
$ ./tox.se -e py36 -r  # recreate only the py36 environment

tox environments¶

The following tox environments are available.

py3: Run by default. Launch the test suite under Python 3. Generate a code-coverage report and display a summary coverage report.
doc: Run by default. Build documentation and write it to the build/doc/ directory.
flake8: Run by default. Check for code-style violations using the flake8 linter.
manage: Run management commands. Positional arguments are passed to manage.py.

Run the development server¶

Django comes with a development web server which can be run via:

$ ./compose.sh development

The server should now be browsable at http://localhost:8080/.

Building the documentation¶

This documentation may be built using the “doc” tox environment.

Docker images¶

The application is deployed using Docker containers on the Google Container Engine. Similarly, docker-compose can be used to run the application locally in a Docker container.

Note

If the requirements.txt file is modified, you’ll need to re-build the container image via docker-compose build.

Occasionally, it is useful to get an interactive Python shell which is set up to be able to import the application code and to make database queries, etc. You can launch such a shell via:

$ ./compose.sh development up -d  # if the server is not yet running
$ ./compose.sh development run --rm development_app ./manage.py shell

Running the production docker image¶

The production Docker image is built with the top-level Dockerfile. To test it, you can build and run the production image via:

$ ./compose.sh development down  # if already started
$ ./compose.sh production build
$ ./compose.sh production up -d  # start server in background
$ ./compose.sh production exec production_app ./manage.py migrate

Note

The production docker-compose configuration does not mount the working directory inside the container so you have to make sure that you re-build the image.

Cloud infrastructure¶

This section provides a brief outline of cloud infrastructure for development.

Source control¶

The panel is hosted on GitHub at https://github.com/uisautomation/media-webapp. The repository has master set up to be writeable only via pull request. It is intended that local development happens in personal forks and is merged via pull request. The main rationale for this is a) it guards against accidentally git push-ing the wrong branch and b) it reduces the number of “dangling” branches in the main repository.

Unit tests¶

The project is set up on CircleCI to automatically run unit tests and build documentation on each commit to a branch and on each pull request.

In order to better match production, CircleCI is set up to run unit tests using the PostgreSQL database and not sqlite. If you only run unit tests locally with sqlite then it is possible that some tests may fail.

Code-coverage¶

Going to CodeCov, logging in with GitHub and adding the media-webapp repository will start code coverage reporting on pull-requests.

Documentation¶

CircleCI has been set up so that when the master branch is built, the documentation is deployed to https://uisautomation.github.io/media-webapp via GitHub pages. The UIS robot machine account’s personal ssh is set up in CircleCI.

Code-style¶

The tox test runner will automatically check the code with flake8 to ensure PEP8 compliance. Sometimes, however, rules are made to be broken and so you may find yourself needing to use the noqa in-line comment mechanism to silence individual errors.

To run the flake8 tests manually, specify the tox environment:

$ ./compose.sh tox run --rm tox -e flake8

Debugging¶

If you need to debug the application running in docker-compose, configuration is included to allow you debug along the lines described in this post on using PDB with docker-compose. Also PuDB, a full-screen, console-based visual debugger, is installed in development as a more sophisticated alternative to PDB. To use PuDB simply set breakpoints as follows:

import pudb; pudb.set_trace()

Documentation¶

This documentation is re-built on each commit to master by CircleCI and posted to GitHub pages at https://uisautomation.github.io/media-webapp/.