Patterns

Dependency management

Warehouse’s approach to dependency management can be summarized as follows:

• Separate requirements files for different environments (deploy, development, docs, linting, testing, etc.);

• All dependencies are pinned to precise versions, and include artifact hashes;

• Pinned requirements and subdependencies are compiled from .in files.

We install all dependencies with pip, and we use pip-tools to compile dependencies.

In practice, developers need to interact with our dependencies in three ways:

Upgrading existing dependencies

Dependencies are automatically upgraded via Dependabot pull requests, and occasionally merged by maintainers.

Adding new dependencies

Deciding to add a new dependency should be made carefully. Generally, we are not opposed to adding more dependencies, however some effort should be made to ensure that a given dependency:

• Is reasonably stable;

• Is currently maintained;

• Doesn’t introduce a large amount of sub-dependencies.

All top-level dependencies are included in one or more .in files, which are then compiled into .txt files with precise versions and artifact hashes.

When adding a new dependency, it’s important to add it to the correct .in file:

File	Purpose
deploy.in	Required only to run in production
docs.in	For our documentation
lint.in	For linting our docs and codebase
main.in	Every dependency of our web service
tests.in	Required to run our tests

Dependencies that are either private or aren’t deployed to production aren’t compiled:

File	Purpose
dev.txt	Various development dependencies
ipython.txt	Specific to using IPython as your shell

To add a new dependency:

1. Add the project name to the appropriate .in file

2. From the repositories root directory, recompile the dependencies for each modified .in file:

   $ make requirements/{file}.txt
   

3. Commit the changes

Removing existing dependencies

Only top-level dependencies should be removed. The process is similar to the process for adding new dependencies:

1. Remove the project name from the appropriate .in file

2. From the repositories root directory, recompile the dependencies for each modified .in file:

   $ make requirements/{file}.txt
   

3. Commit the changes

Returning vs Raising HTTP Exceptions

Pyramid allows the various HTTP Exceptions to be either returned or raised, and the difference between whether you return or raise them are subtle. The differences between returning and raising a response are:

• Returning a response commits the transaction associated with the request, while raising rolls it back.

• Returning a response does not invoke the exec_view handler, while raising does.

The follow table shows what the default method should be for each type of HTTP exception, this is only the default and judgement should be applied to each situation.

Class	Method
HTTPSuccessful (2xx)	Return
HTTPRedirection (3xx)	Return
HTTPClientError (4xx)	Raise, except for HTTPNotFound which should be return.
HTTPServerError (5xx)	Raise