Skip to content

NAM101: a test that pytest will never run

Junior

pytest runs the functions whose name starts with test, and only those. A function in a test file that contains an assert under another name is never run. The suite stays green. The code it was meant to cover goes untested.

# tests/test_users.py
def check_user_is_valid():  # NAM101: pytest never calls this
    assert build_user().is_valid()


def test_user_is_valid():  # runs
    assert build_user().is_valid()

To fix it, rename the function so its name starts with test. If it is a helper that other tests call, give it an argument or a leading underscore, and namelint will leave it alone.

Senior

The cost of this mistake is silent: a test that never runs looks exactly like a test that passes. It usually comes from a rename, a copy from another file, or a helper that lost its argument.

namelint treats a file as a test module when it is named test_*.py or *_test.py, or sits under a tests/ directory. Within it, a function is reported when it contains an assertion, takes no arguments and doesn't start with test.

It stays quiet on:

  • a decorated function, since that is how fixtures look.
  • a _-prefixed helper, whose name already says "not collected".
  • setup and teardown hooks, both unittest's (setUp, tearDownClass) and pytest's (setup_module, teardown_method).
  • a helper that takes the thing it checks, such as assert_ok(response).

A with pytest.raises(...) block counts as an assertion.

Expert

The arity test is the discriminator. A zero-argument function with an assertion can only check module state or build its own input, which is what a test does. A helper takes its subject as an argument. For methods, self and cls don't count. Any decorator silences the check, since telling a fixture from other decorators would mean resolving names.

Assertions are found in the function's own body, without descending into nested functions. An assert inside an inner helper says nothing about the outer function.

In the corpus, the first run reported 39 findings. A third were unittest and pytest setup and teardown hooks. Those contain assertions and take no arguments, yet they are not tests. They are now exempt by name. The other false-positive class, _-prefixed helpers, accounted for every finding in an earlier run and is exempt by prefix. The current rate is 0.33 per thousand functions in CPython and 0.98 in pydantic.