When developing Python code we are constantly adding and committing changes. However, nothing stops us from committing low-quality code, e.g. code containing unused imports, unformatted code, or functions that do not work correctly. Therefore, to make sure the code we commit is of sufficient quality, we use formatting-, linting- and testing tools to check our code. These checks will result in increased code readability, maintainability, and quality and will improve collaboration amongst colleagues. We can run these checks manually, but this is time-consuming and prone to errors. Therefore, we recommend automating these checks to simplify your development workflow.
In this blog, we are looking into a tool stack that helps us run these checks. We will highlight such a setup with the tools pre-commit and Makefile. When set up right, they complement each other. We will describe why you would want to use them, how they work and what a minimal setup looks like for your new project.
To ensure our code quality we often make use of formatting-, linting- and testing tools. Each of the three has its own purpose and complements another:
In our setup, we use Black as formatting-, Pylint as linting- and Pytest as a testing tool. Black is a PEP8 compliant formatter without much to configure. Pylint comes with a code rating and is the most used linting tool for Python. Pytest makes it easy to write unit tests without boilerplate.
Pre-commit is a tool that is used to run scripts (hooks) to automatically identify issues in your code. Configured hooks are triggered when committing your code to Git.
To use pre-commit you need to:
pip install pre-commit
.pre-commit-config.yaml. This file will contain the hooks you want to run and their settings.
There are many hooks that can be used to check your code, like hooks from pre-commit itself, or third-party tools, like Black and Pylint. A minimal example of a
.pre-commit-config.yaml file is shown below:
repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v3.4.0 hooks: - id: check-docstring-first - id: check-merge-conflict - id: trailing-whitespace - id: end-of-file-fixer - id: check-yaml - id: check-ast - repo: https://github.com/ambv/black rev: 20.8b1 hooks: - id: black - repo: local hooks: - id: pylint name: pylint entry: pylint language: python types:
This setup ensures that for every commit, pre-commit will automatically run the set of specified hooks. First, a set of hooks from pre-commit reformats and checks your code. Secondly, Black reformats your Python code. Finally, Pylint scans your code and gives an overall rating. Only if your committed code passes all checks your commit will be completed. Otherwise, you (or the hooks) have to change the code accordingly and commit again.
Note that Pytest is not included in our pre-commit configuration as running tests is usually expensive. A list of useful pre-commit hooks can be found here.
Using pre-commit, checks are automatically run on each commit. However, we might also want to run checks before committing code. Although running checks can be done with separate commands, a more efficient setup can be achieved by using a Makefile. A Makefile makes it possible to create shortcuts that easily run a task (such as your pre-commit) or a set of tasks (such as pre-commit and testing). These shortcuts can then be triggered with the
make command in your terminal. An example of a
Makefile is shown below:
## Lint your code using pylint .PHONY: lint lint: python -m pylint --version python -m pylint src## Run tests using pytest .PHONY: test test: python -m pytest --version python -m pytest tests## Format your code using black .PHONY: black black: python -m black --version python -m black .## Run ci part .PHONY: ci ci: precommit lint test
With this setup, you can run multiple tasks with simple shortcuts (stated after
.PHONY and opening each task). The
make precommit command runs all your pre-commit hooks on all your Python files. The
make black command runs Black on all Python files in your current working directory. The
make lint command runs Pylint on your
src directory. The
make test command runs Pytest using the
tests directory. Finally, the
make ci command runs previously described tasks in sequence.
Pre-commit and Makefile are great tools to deliver quality Python code in an early stage. They will speed up your development workflow which results in more time for code logic. Both have their own purpose:
As it is easy to integrate pre-commit and Makefile, make sure you try them in your new project! If you need help, please respond in the comments below.