Flask Testing – Unittest and Coverage


I have been experimenting with Flask recently. I am interested in learning about test driven development. This looked like a good opportunity to merge the two together.

Test driven development is based around writing tests before you write the code. That way you write the code to pass the test, not the test based on the code. There are a few modules I use to test my flask apps. These being unittest, coverage and nose. Unittest is used to write the tests and coverage checks to make sure all the code is tested. Nose acts as the test runner. The actual testing is done by the in-built client in the Flask library.

The structure my Flask apps look something like the following:

.
-- app/
---- __init__.py
---- module1/
---- module2/
---- ...
-- test/
---- __init__.py
---- test_module1.py
---- test_module2.py
---- ...
-- ...

This structure lends itself to the app factory method of instantiating Flask apps. See here for more.

The base class

When writing all my tests they inherit from one base class. This base class sets up the app and defines the teardown. This prevents lots of duplicated code and saves lots of wasted lines in the long run.

class BaseTest(TestCase): 
    def setUp(self) -> None:
        # Create the application
        self.app = create_app(Testing) 
        self.app_context = self.app.app_context() 
        self.app_context.push() 
        self.client = self.app.test_client() 
        # Setup the database
        db.create_all() 
 
        # Create a admin user for each test (used for logins)
        self.admin_basic_auth = base64 \ 
            .b64encode(b"admin:admin") \ 
            .decode("utf-8") 
        self.admin_user = User( 
            "admin@admin.admin", 
            "admin", 
            "admin", 
            admin=True 
        ) 
        self.admin_user.verified = True 
        db.session.add(self.admin_user) 
        db.session.commit() 
        # Save the admin api token for logins
        self.admin_token = self.admin_user.generate_token().decode("utf-8") 
        db.session.commit() 
 
        # Define a new test user but don't create them
        self.test_email = "test@test.test" 
        self.test_username = "test_account" 
        self.test_password = "test_password" 
 
    def tearDown(self) -> None: 
        db.session.remove() 
        db.drop_all() 
        self.app_context.pop()

Every test I write, extends from this class. This class overrides some methods from the TestCase class from unittest. The setUp function is run each time a new test is run on the app. This means that each test receives a fresh, unaltered copy of the application to run on. The tearDown function is designed to undo all the changes made by a test. Within the setup function, an admin user is created. This is because much of my app functionality requires a login. By creating the admin here, I save 7 lines, per test.

A sample test

class AuthTest(BaseTest): 
    def test_http_basic_auth_correct(self) -> None:
        """
        Tests a successful login using a username and password with
        HTTP Basic authentication
        :return: None
        """ 
        response = self.client.post( 
            "/auth/login", 
            headers={"Authorization": f"Basic {self.admin_basic_auth}"} 
        ) 
        self.assertEqual(response.status_code, STATUS_CODES["OK"]) 
        json_data = response.get_json() 
        self.assertIn("token", json_data)

#    [ 7 more tests in this file ]

This is a sample test that tests my authentication module. By inherting the TestCase class we gain access to some assert methods. These compare two objects together based on the method called e.g. assertEqual tests whether 2 objects are the same.

As we can see in this file there are 8 tests. Mentioned above is a saving of 7 lines per test so the savings in this file alone is 56 lines.

You can find more information about testing Flask applications here


Running the tests

Now that there is a test for us to run, we need to run it. By using Nose to run our tests we can automatically run the coverage tests as well. With Nose installed run the following command from your project root:

nosetests --with-coverage \
          --cover-package app \
          --cover-erase \
          --cover-html \
          --cover-html-dir htmlcov/

This will run all of our tests and generate a coverage report in the directory htmlcov/. Open up the index.html file of the coverage report you should see an overview of how your code did. In my case I get a report like this: Example Coverage Report