How to test error messages with pytest

In this short article, you will learn, how and when to test the error message of an exception with pytest. First we need some code that tests if an exception is raised.

Set up the test code

import pytest


def divide(x, y):
    return x / y


def test_raises():
    with pytest.raises(ZeroDivisionError):
        divide(3, 0)

How to test the error message?

Now we can add the handy match parameter to pytest.raises to check the error message of the exception. It takes a regular expression string and checks if the error message matches the string.

def test_raises():
    with pytest.raises(ZeroDivisionError, match=r"division by zero"):
        divide(3, 0)

When to test error messages?

In general, the text of an error message is rarely part of the public interface, so tests that rely on it tend to be flaky. So be careful when you do it and think about if your code can be written and/or tested differently.

That said, I found it quite handy if you use assert statements to test properties of data or something in a data processing or machine learning pipeline. Also, sometimes a piece of code could raise a specific type of exception at multiple places, so it might be easier to distinguish them by the error message.