hypothesis - Man Page

This is a lightning introduction to the most important features of Hypothesis; enough to get you started writing tests. The tutorial <> introduces these features (and more) in greater detail.

pip install hypothesis

Create a new file called example.py, containing a simple test:

# contents of example.py
from hypothesis import given, strategies as st

@given(st.integers())
def test_integers(n):
    print(f"called with {n}")
    assert isinstance(n, int)

test_integers()

@given <#hypothesis.given> is the standard entrypoint to Hypothesis. It takes a strategy, which describes the type of inputs you want the decorated function to accept. When we call test_integers, Hypothesis will generate random integers (because we used the integers() <#hypothesis.strategies.integers> strategy) and pass them as n. Let's see that in action now by running python example.py:

called with 0
called with -18588
called with -672780074
called with 32616
...

We just called test_integers(), without passing a value for n, because Hypothesis generates random values of n for us.

Note:

A Hypothesis test is still a regular python function, which means pytest or unittest will pick it up and run it in all the normal ways.

# contents of example.py
from hypothesis import given, strategies as st

@given(st.integers(0, 200))
def test_integers(n):
    assert n < 50

This test will clearly fail, which can be confirmed by running pytest example.py:

$ pytest example.py

    ...

    @given(st.integers())
    def test_integers(n):
>       assert n < 50
E       assert 50 < 50
E       Falsifying example: test_integers(
E           n=50,
E       )

You can pass multiple arguments to @given <#hypothesis.given>:

@given(st.integers(), st.text())
def test_integers(n, s):
    assert isinstance(n, int)
    assert isinstance(s, str)

Or use keyword arguments:

@given(n=st.integers(), s=st.text())
def test_integers(n, s):
    assert isinstance(n, int)
    assert isinstance(s, str)

Note:

See @given <#hypothesis.given> for details about how @given <#hypothesis.given> handles different types of arguments.

Sometimes, you need to remove invalid cases from your test. The best way to do this is with .filter() <#hypothesis.strategies.SearchStrategy.filter>:

@given(st.integers().filter(lambda n: n % 2 == 0))
def test_integers(n):
    assert n % 2 == 0

For more complicated conditions, you can use assume() <#hypothesis.assume>, which tells Hypothesis to discard any test case with a false-y argument:

@given(st.integers(), st.integers())
def test_integers(n1, n2):
    assume(n1 != n2)
    # n1 and n2 are guaranteed to be different here

Note:

You can learn more about .filter() <#hypothesis.strategies.SearchStrategy.filter> and assume() <#hypothesis.assume> in the Adapting strategies <> tutorial page.

You may want an input to depend on the value of another input. For instance, you might want to generate two integers n1 and n2 where n1 <= n2.

You can do this using the @composite <#hypothesis.strategies.composite> strategy. @composite <#hypothesis.strategies.composite> lets you define a new strategy which is itself built by drawing values from other strategies, using the automatically-passed draw function.

@st.composite
def ordered_pairs(draw):
    n1 = draw(st.integers())
    n2 = draw(st.integers(min_value=n1))
    return (n1, n2)

@given(ordered_pairs())
def test_pairs_are_ordered(pair):
    n1, n2 = pair
    assert n1 <= n2

In more complex cases, you might need to interleave generation and test code. In this case, use data() <#hypothesis.strategies.data>.

@given(st.data(), st.text(min_size=1))
def test_string_characters_are_substrings(data, string):
    assert isinstance(string, str)
    index = data.draw(st.integers(0, len(string) - 1))
    assert string[index] in string

Hypothesis works with pytest features, like pytest.mark.parametrize:

import pytest

from hypothesis import given, strategies as st

@pytest.mark.parametrize("operation", [reversed, sorted])
@given(st.lists(st.integers()))
def test_list_operation_preserves_length(operation, lst):
    assert len(lst) == len(list(operation(lst)))

Hypothesis also works with pytest fixtures:

import pytest

@pytest.fixture(scope="session")
def shared_mapping():
    return {n: 0 for n in range(101)}

@given(st.integers(0, 100))
def test_shared_mapping_keys(shared_mapping, n):
    assert n in shared_mapping

The Hypothesis tutorial introduces the main features of Hypothesis. We suggest reading through in order until completing Custom strategies <>, at which point you can choose to read what seems interesting to you.

If you're in a hurry, the quickstart <> is a much faster bare-bones version.

This page introduces two fundamental parts of Hypothesis (@given <#hypothesis.given>, and strategies) and shows how to test a selection sort implementation using Hypothesis.

First, let's install Hypothesis:

pip install hypothesis

Hypothesis tests are defined using two things; @given <#hypothesis.given>, and a strategy, which is passed to @given <#hypothesis.given>. Here's a simple example:

from hypothesis import given, strategies as st

@given(st.integers())
def test_is_integer(n):
    assert isinstance(n, int)

Adding the @given <#hypothesis.given> decorator turns this function into a Hypothesis test. Passing integers() <#hypothesis.strategies.integers> to @given <#hypothesis.given> says that Hypothesis should generate random integers for the argument n when testing.

We can run this test by calling it:

from hypothesis import given, strategies as st

@given(st.integers())
def test_is_integer(n):
    print(f"called with {n}")
    assert isinstance(n, int)

test_is_integer()

Note that we don't pass anything for n; Hypothesis handles generating that value for us. The resulting output looks like this:

called with 0
called with -18588
called with -672780074
called with 32616
...

Suppose we have implemented a simple selection sort:

# contents of example.py
from hypothesis import given, strategies as st

def selection_sort(lst):
    result = []
    while lst:
        smallest = min(lst)
        result.append(smallest)
        lst.remove(smallest)
    return result

and want to make sure it's correct. We can write the following test by combining the integers() <#hypothesis.strategies.integers> and lists() <#hypothesis.strategies.lists> strategies:

...

@given(st.lists(st.integers()))
def test_sort_correct(lst):
    print(f"called with {lst}")
    assert selection_sort(lst.copy()) == sorted(lst)

test_sort_correct()

When running test_sort_correct, Hypothesis uses the lists(integers()) strategy to generate random lists of integers. Feel free to run python example.py to get an idea of the kinds of lists Hypothesis generates (and to convince yourself that this test passes).

This test is a good start. But selection_sort should be able to sort lists with floats, too. If we wanted to generate lists of either integers or floats, we can change our strategy:

# changes to example.py
@given(st.lists(st.integers() | st.floats()))
def test_sort_correct(lst):
    pass

The pipe operator | takes two strategies, and returns a new strategy which generates values from either of its strategies. So the strategy integers() | floats() can generate either an integer, or a float.

Note:

Even though test_sort_correct passed when we used lists of integers, it actually fails now that we've added floats! If you run python example.py, you'll likely (but not always; this is random testing, after all) find that Hypothesis reports a counterexample to test_sort_correct. For me, that counterexample is [1.0, nan, 0]. It might be different for you.

The issue is that sorting in the presence of nan is not well defined. As a result, we may decide that we don't want to generate them while testing. We can pass floats(allow_nan=False) to tell Hypothesis not to generate nan:

# changes to example.py
@given(st.lists(st.integers() | st.floats(allow_nan=False)))
def test_sort_correct(lst):
    pass

And now this test passes without issues.

Note:

>>> st.lists(st.integers() | st.floats(allow_nan=False)).example()
[-5.969063e-08, 15283673678, 18717, -inf]

If we wanted to pass multiple arguments to a test, we can do this by passing multiple strategies to @given <#hypothesis.given>:

from hypothesis import given, strategies as st

@given(st.integers(), st.lists(st.floats()))
def test_multiple_arguments(n, lst):
    assert isinstance(n, int)
    assert isinstance(lst, list)
    for f in lst:
        assert isinstance(f, float)

We can also pass strategies using keyword arguments:

@given(lst=st.lists(st.floats()), n=st.integers())  #  <-- changed
def test_multiple_arguments(n, lst):
    pass

Note that even though we changed the order the parameters to @given <#hypothesis.given> appear, we also explicitly told it which parameters to pass to by using keyword arguments, so the meaning of the test hasn't changed.

In general, you can think of positional and keyword arguments to @given <#hypothesis.given> as being forwarded to the test arguments.

Note:

There are a few ways to run a Hypothesis test.

• Explicitly call it, like test_is_integer(), as we've seen. Hypothesis tests are just normal functions, except @given <#hypothesis.given> handles generating and passing values for the function arguments.
• Let a test runner such as pytest <https://pypi.org/project/pytest/> pick up on it (as long as the function name starts with test_).

Concretely, when running a Hypothesis test, Hypothesis will:

• generate 100 random inputs,
• run the body of the function for each input, and
• report any exceptions that get raised.

Note:

The number of examples can be controlled with the max_examples <#hypothesis.settings.max_examples> setting. The default is 100.

Property-based testing is a powerful addition to unit testing. It is not always a replacement.

If you're having trouble coming up with a property in your code to test, we recommend trying the following:

• Look for round-trip properties: encode/decode, serialize/deserialize, etc. These property-based tests tend to be both powerful and easy to write.
• Look for @pytest.mark.parametrize in your existing tests. This is sometimes a good hint you can replace the parametrization with a strategy. For instance, @pytest.mark.parametrize("n", range(0, 100)) could be replaced by @given(st.integers(0, 100 - 1)).
• Simply call your code with random inputs (of the correct shape) from Hypothesis! You might be surprised how often this finds crashes. This can be especially valuable for projects with a single entrypoint interface to a lot of underlying code.

Other examples of properties include:

• An optimized implementation is equivalent to a slower, but clearly correct, implementation.
• A sequence of transactions in a financial system always "balances"; money never gets lost.
• The derivative of a polynomial of order n has order n - 1.
• A type-checker, linter, formatter, or compiler does not crash when called on syntactically valid code.
• And more <https://fsharpforfunandprofit.com/posts/property-based-testing-2/>.

This page shows some of the strategies that Hypothesis provides for you.

Here is a selection of strategies provided by Hypothesis that may be useful to know:

integers() <#hypothesis.strategies.integers>

Generates integers.

floats() <#hypothesis.strategies.floats>

Generates floats.

booleans() <#hypothesis.strategies.booleans>

Generates booleans.

text() <#hypothesis.strategies.text>

Generates unicode strings (i.e., instances of python:str). Can be constrained to ASCII with st.text(st.characters(codec="ascii")).

lists() <#hypothesis.strategies.lists>

Generates lists with elements from the strategy passed to it. st.lists(st.integers()) generates lists of integers.

tuples() <#hypothesis.strategies.tuples>

Generates tuples of a fixed length. st.tuples(st.integers(), st.floats()) generates tuples with two elements, where the first element is an integer and the second is a float.

one_of() <#hypothesis.strategies.one_of>

Generates from any of the strategies passed to it. st.one_of(st.integers(), st.floats()) generates either integers or floats. You can also use | to construct one_of() <#hypothesis.strategies.one_of>, like st.integers() | st.floats().

builds() <#hypothesis.strategies.builds>

Generates instances of a class (or other callable) by specifying a strategy for each argument, like st.builds(Person, name=st.text(), age=st.integers()).

just() <#hypothesis.strategies.just>

Generates the exact value passed to it. st.just("a") generates the exact string "a". This is useful when something expects to be passed a strategy. For instance, st.lists(st.integers() | st.just("a")) generates lists whose elements are either integers or the string "a".

sampled_from() <#hypothesis.strategies.sampled_from>

Generates a random value from a list. st.sampled_from(["a", 1]) is roughly equivalent to st.just("a") | st.just(1).

none() <#hypothesis.strategies.none>

Generates None. Useful for parameters that can be optional, like st.integers() | st.none().

See also:

This page discusses ways to adapt strategies to your needs, either by transforming them inline with .map() <#hypothesis.strategies.SearchStrategy.map>, or filtering out unwanted inputs with .filter() <#hypothesis.strategies.SearchStrategy.filter> and assume() <#hypothesis.assume>.

Sometimes you want to apply a simple transformation to a strategy. For instance, we know that we can generate lists of integers with lists(integers()). But maybe we wanted to instead generate sorted lists. We could use an inline .map() <#hypothesis.strategies.SearchStrategy.map> to achieve this:

>>> lists(integers()).map(sorted).example()
[-25527, -24245, -93, -70, -7, 0, 39, 65, 112, 6189, 19469, 32526, 1566924430]

In general, strategy.map(f) returns a new strategy which transforms all the examples generated by strategy by calling f on them.

Many strategies in Hypothesis offer some control over the kinds of values that get generated. For instance, integers(min_value=0) generates positive integers, and integers(100, 200) generates integers between 100 and 200.

Sometimes, you need more control than this. The inputs from a strategy may not match exactly what you need, and you just need to filter out a few bad cases.

For instance, suppose we have written a simple test involving the modulo operator %:

from hypothesis import given, strategies as st

@given(st.integers(), st.integers())
def test_remainder_magnitude(a, b):
    # the remainder after division is always less than
    # the divisor
    assert abs(a % b) < abs(b)

Hypothesis will quickly report a failure for this test: ZeroDivisionError: integer modulo by zero. Just like division, modulo isn't defined for 0. The case of b == 0 isn't interesting for the test, and we would like to get rid of it.

The best way to do this is with the .filter() <#hypothesis.strategies.SearchStrategy.filter> method:

from hypothesis import assume, given, strategies as st

@given(st.integers(), st.integers().filter(lambda n: n != 0))
def test_remainder_magnitude(a, b):
    # b is guaranteed to be nonzero here, thanks to the filter
    assert abs(a % b) < abs(b)

This test now passes cleanly.

Calling .filter() <#hypothesis.strategies.SearchStrategy.filter> on a strategy creates a new strategy with that filter applied at generation-time. For instance, integers().filter(lambda n: n != 0) is a strategy which generates nonzero integers.

.filter() <#hypothesis.strategies.SearchStrategy.filter> lets you filter test inputs from a single strategy. Hypothesis also provides an assume() <#hypothesis.assume> function for when you need to filter an entire test case, based on an arbitrary condition.

The assume() <#hypothesis.assume> function skips test cases where some condition evaluates to True. You can use it anywhere in your test. We could have expressed our .filter() <#hypothesis.strategies.SearchStrategy.filter> example above using assume() <#hypothesis.assume> as well:

from hypothesis import assume, given, strategies as st

@given(st.integers(), st.integers())
def test_remainder_magnitude(a, b):
    assume(b != 0)
    # b will be nonzero here
    assert abs(a % b) < abs(b)

Where possible, you should use .filter() <#hypothesis.strategies.SearchStrategy.filter>. Hypothesis can often rewrite simple filters into more efficient sampling methods than rejection sampling, and will retry filters several times instead of aborting the entire test case (as with assume() <#hypothesis.assume>).

For more complex relationships that can't be expressed with .filter() <#hypothesis.strategies.SearchStrategy.filter>, use assume() <#hypothesis.assume>.

Here's an example of a test where we want to filter out two different types of examples:

from hypothesis import assume, given, strategies as st

@given(st.integers(), st.integers())
def test_floor_division_lossless_when_b_divides_a(a, b):
    # we want to assume that:
    # * b is nonzero, and
    # * b divides a
    assert (a // b) * b == a

We could start by using assume() <#hypothesis.assume> for both:

from hypothesis import assume, given, strategies as st

@given(st.integers(), st.integers())
def test_floor_division_lossless_when_b_divides_a(a, b):
    assume(b != 0)
    assume(a % b == 0)
    assert (a // b) * b == a

And then notice that the b != 0 condition can be moved into the strategy definition as a .filter() <#hypothesis.strategies.SearchStrategy.filter> call:

from hypothesis import assume, given, strategies as st

@given(st.integers(), st.integers().filter(lambda n: n != 0))
def test_floor_division_lossless_when_b_divides_a(a, b):
    assume(a % b == 0)
    assert (a // b) * b == a

However, the a % b == 0 condition has to stay as an assume() <#hypothesis.assume>, because it expresses a more complicated relationship between a and b.

One other way we could have avoided the divide-by-zero error inside the test function is to early-return when b == 0:

from hypothesis import assume, given, strategies as st

@given(st.integers(), st.integers())
def test_remainder_magnitude(a, b):
    if b == 0:
        # bad plan - test "passes" without checking anything!
        return
    assert abs(a % b) < abs(b)

While this would have avoided the divide-by-zero, early-returning is not the same as using assume() <#hypothesis.assume>. With assume() <#hypothesis.assume>, Hypothesis knows that a test case has been filtered out, and will not count it towards the max_examples <#hypothesis.settings.max_examples> limit. In contrast, early-returns are counted as a passing test, even though the assertions didn't run! In more complicted cases, this could end up testing your code less than you expect, because many test cases get discarded without Hypothesis knowing about it.

In addition, assume() <#hypothesis.assume> lets you skip the test case at any point in the test, even inside arbitrarily deep nestings of functions.

This page describes how to write a custom strategy, for when the built-in strategies don't quite fit your needs.

Sometimes you might find it useful to write helper functions, to more concisely express a common pattern for your project. For example, it's much easier to write (and read!) response=json() than to have the whole implementation inline:

def json(*, finite_only=True):
    """Helper function to describe JSON objects, with optional inf and nan."""
    numbers = st.floats(allow_infinity=not finite_only, allow_nan=not finite_only)
    return st.recursive(
        st.none() | st.booleans() | st.integers() | numbers | st.text(),
        extend=lambda xs: st.lists(xs) | st.dictionaries(st.text(), xs),
    )

If a strategy in Hypothesis doesn't match what you need, you can write your own strategy.

For instance, suppose we want to generate a list of floats which sum to 1. We might start implementing this by generating lists of integers between 0 and 1 with lists(floats(0, 1)). But now we're a bit stuck, and can't go any further with the standard strategies.

One way to define a new strategy is using the @composite <#hypothesis.strategies.composite> decorator. @composite <#hypothesis.strategies.composite> lets you define a new strategy that uses arbitrary Python code. For instance, to implement the above:

from hypothesis import strategies as st

@st.composite
def sums_to_one(draw):
    l = draw(st.lists(st.floats(0, 1)))
    return [f / sum(l) for f in l]

@composite <#hypothesis.strategies.composite> passes a draw function to the decorated function as its first argument. draw is used to draw a random value from another strategy. We return from sums_to_one a value of the form we wanted to generate; in this case, a list that sums to one.

Let's see this new strategy in action:

import pytest

from hypothesis import given, strategies as st

@st.composite
def sums_to_one(draw):
    lst = draw(st.lists(st.floats(0.001, 1), min_size=1))
    return [f / sum(lst) for f in lst]

@given(sums_to_one())
def test(lst):
    # ignore floating point errors
    assert sum(lst) == pytest.approx(1)

Note:

Just like all other strategies, we called sums_to_one before passing it to @given <#hypothesis.given>. @composite <#hypothesis.strategies.composite> should be thought of as turning its decorated function into a function which returns a strategy when called. This is actually the same as existing strategies in Hypothesis; integers() <#hypothesis.strategies.integers> is really a function, which returns a strategy for integers when called.

You can add parameters to functions decorated with @composite <#hypothesis.strategies.composite>, including keyword-only arguments. These work as you would normally expect.

For instance, suppose we wanted to generalize our sums_to_one function to sums_to_n. We can add a parameter n:

import pytest

from hypothesis import assume, given, strategies as st

@st.composite
def sums_to_n(draw, n=1):  #  <-- changed
    lst = draw(st.lists(st.floats(0, 1), min_size=1))
    assume(sum(lst) > 0)
    return [f / sum(lst) * n for f in lst]  #  <-- changed

@given(sums_to_n(10))
def test(lst):
    assert sum(lst) == pytest.approx(10)

And we could just as easily have made n a keyword-only argument instead:

import pytest

from hypothesis import assume, given, strategies as st

@st.composite
def sums_to_n(draw, *, n=1):  #  <-- changed
    lst = draw(st.lists(st.floats(0, 1), min_size=1))
    assume(sum(lst) > 0)
    return [f / sum(lst) * n for f in lst]

@given(sums_to_n(n=10))  #  <-- changed
def test(lst):
    assert sum(lst) == pytest.approx(10)

Another scenario where @composite <#hypothesis.strategies.composite> is useful is when generating a value that depends on a value from another strategy. For instance, suppose we wanted to generate two integers n1 and n2 with n1 <= n2. We can do this using @composite <#hypothesis.strategies.composite>:

@st.composite
def ordered_pairs(draw):
    n1 = draw(st.integers())
    n2 = draw(st.integers(min_value=n1))
    return (n1, n2)

@given(ordered_pairs())
def test_pairs_are_ordered(pair):
    n1, n2 = pair
    assert n1 <= n2

Note:

We could also have written this particular strategy as st.tuples(st.integers(), st.integers()).map(sorted) (see Adapting strategies <>). Some prefer this inline approach, while others prefer defining well-named helper functions with @composite <#hypothesis.strategies.composite>. Our suggestion is simply that you prioritize ease of understanding when choosing which to use.

When using @composite <#hypothesis.strategies.composite>, you have to finish generating the entire input before running your test. But maybe you don't want to generate all of the input until you're sure some initial test assertions have passed. Or maybe you have some complicated control flow which makes it necessary to generate something in the middle of the test.

data() <#hypothesis.strategies.data> lets you to do this. It's similar to @composite <#hypothesis.strategies.composite>, except it lets you mix test code and generation code.

Note:

For instance, here's how we would write our earlier @composite <#hypothesis.strategies.composite> example using data() <#hypothesis.strategies.data>:

import pytest

from hypothesis import given, strategies as st

@given(st.data())
def test(data):
    lst = data.draw(st.lists(st.floats(0.001, 1), min_size=1))
    lst = [f / sum(lst) for f in lst]
    # ignore floating point errors
    assert sum(lst) == pytest.approx(1)

This page discusses how to configure the behavior of a single Hypothesis test, or of an entire test suite.

Hypothesis lets you configure the default behavior of a test using the @settings <#hypothesis.settings> decorator. You can use settings to configure how many examples Hypothesis generates, how Hypothesis replays failing examples, and the verbosity level of the test, among others.

Using @settings <#hypothesis.settings> on a single test looks like this:

from hypothesis import given, settings, strategies as st

@given(st.integers())
@settings(max_examples=200)
def runs_200_times_instead_of_100(n):
    pass

You can put @settings <#hypothesis.settings> either before or after @given <#hypothesis.given>. Both are equivalent.

If you have a test which is very expensive or very cheap to run, you can change the number of examples (inputs) Hypothesis generates with the max_examples <#hypothesis.settings.max_examples> setting:

from hypothesis import given, settings, strategies as st

@given(st.integers())
@settings(max_examples=5)
def test(n):
    print("prints five times")

The default is 100 examples.

Note:

Here are a few of the more commonly used setting values:

• settings.phases <#hypothesis.settings.phases> controls which phases of Hypothesis run, like replaying from the database or generating new inputs.
• settings.database <#hypothesis.settings.database> controls how and if Hypothesis replays failing examples.
• settings.verbosity <#hypothesis.settings.verbosity> can print debug information.
• settings.derandomize <#hypothesis.settings.derandomize> makes Hypothesis deterministic. (Two kinds of testing <https://blog.nelhage.com/post/two-kinds-of-testing/> discusses when and why you might want that).

Note:

See the settings <#hypothesis.settings> reference for a full list of possible settings.

In addition to configuring individual test functions with @settings <#hypothesis.settings>, you can configure test behavior across your test suite using a settings profile. This might be useful for creating a development settings profile which runs fewer examples, or a settings profile in CI which connects to a separate database.

To create a settings profile, use register_profile() <#hypothesis.settings.register_profile>:

from hypothesis import HealthCheck, settings

settings.register_profile("fast", max_examples=10)

You can place this code in any file which gets loaded before your tests get run. This includes an __init__.py file in the test directory or any of the test files themselves. If using pytest, the standard location to place this code is in a confest.py file (though an __init__.py or test file will also work).

Note that registering a new profile will not affect tests until it is loaded with load_profile() <#hypothesis.settings.load_profile>:

from hypothesis import HealthCheck, settings

settings.register_profile("fast", max_examples=10)

# any tests executed before loading this profile will still use the
# default active profile of 100 examples.

settings.load_profile("fast")

# any tests executed after this point will use the active fast
# profile of 10 examples.

There is no limit to the number of settings profiles you can create. Hypothesis creates a profile called "default", which is active by default. You can also explicitly make it active at any time using settings.load_profile("default"), if for instance you wanted to revert a custom profile you had previously loaded.

Using an environment variable to load a settings profile is a useful trick for choosing a settings profile depending on the environment:

>>> import os
>>> from hypothesis import settings, Verbosity
>>> settings.register_profile("long", max_examples=1000)
>>> settings.register_profile("fast", max_examples=10)
>>> settings.register_profile("debug", max_examples=10, verbosity=Verbosity.verbose)
>>> settings.load_profile(os.getenv("HYPOTHESIS_PROFILE", "default"))

If using pytest, you can also easily select the active profile with --hypothesis-profile:

$ pytest --hypothesis-profile fast

See the Hypothesis pytest plugin <#pytest-plugin>.

Replaying failures found by your Hypothesis tests is almost as important as finding failures in the first place. Hypothesis therefore contains several ways to replay failures: they are automatically saved to (and replayed from) a local ExampleDatabase <#hypothesis.database.ExampleDatabase>, and can be manually replayed via @example <#hypothesis.example> or @reproduce_failure <#hypothesis.reproduce_failure>.

When a test fails, Hypothesis automatically saves the failure so it can be replayed later. For instance, the first time you run the following code, it will take up to a few seconds to fail:

import time

from hypothesis import strategies as st

@given(st.integers())
def f(n):
    assert n < 50
    time.sleep(0.1)

f()

But the next time you run this code, it will fail instantly. When Hypothesis saw the failure the first time, it automatically saved that failing input. On future runs, Hypothesis retries any failing inputs (in Phase.explain <#hypothesis.Phase.explain>) before generating new inputs (in Phase.generate <#hypothesis.Phase.generate>)

Hypothesis saves failures to the settings.database <#hypothesis.settings.database> setting. By default, this is a DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase> in the local .hypothesis directory.

You can disable the database by passing database=None to @settings <#hypothesis.settings>:

@settings(database=None)
def f(n): ...

If you want Hypothesis to always run a specific input, you can use @example <#hypothesis.example>. @example <#hypothesis.example> adds an explicit input which Hypothesis will run every time, in addition to the randomly generated examples. You can think of @example <#hypothesis.example> as combining unit-testing with property-based testing.

For instance, suppose we write a test using integers() <#hypothesis.strategies.integers>, but want to make sure we try a few special prime numbers every time we run the test. We can add these inputs with @example <#hypothesis.example>:

# two mersenne primes
@example(2**17 - 1)
@example(2**19 - 1)
@given(st.integers())
def test_integers(n):
    pass

test_integers()

Hypothesis runs all explicit examples first, in the Phase.explicit <#hypothesis.Phase.explicit> phase, before generating additional random examples in the Phase.generate <#hypothesis.Phase.generate> phase.

Note that unlike examples generated by Hypothesis, examples provided using @example <#hypothesis.example> do not shrink. We can see this by adding a failing assertion:

@example(2**17 - 1)
@given(st.integers())
def test_something_with_integers(n):
    assert n < 100

Hypothesis will print Falsifying explicit example: test_something_with_integers(n=131071), instead of shrinking to n=100.

While the database is useful for quick local iteration, Hypothesis may invalidate it when upgrading (because e.g. the internal format may have changed). Changes to the source code of a test function may also change its database key, invalidating its stored entries. We therefore recommend against relying on the database for the correctness of your tests. If you want to ensure an input is run every time, use @example <#hypothesis.example>.

If settings.print_blob <#hypothesis.settings.print_blob> is set to True (the default in the ci settings profile), and a test fails, Hypothesis will print an @reproduce_failure <#hypothesis.reproduce_failure> decorator containing an opaque blob as part of the error message:

>>> from hypothesis import settings, given
>>> import hypothesis.strategies as st
>>> @given(st.floats())
... @settings(print_blob=True)
... def test(f):
...     assert f == f
...
>>> test()

...
Falsifying example: test(
    f=nan,
)
You can reproduce this example by temporarily adding @reproduce_failure('6.131.23', b'ACh/+AAAAAAAAA==') as a decorator on your test case

You can add this decorator to your test to reproduce the failure. This can be useful for locally replaying failures found by CI. Note that the binary blob is not stable across Hypothesis versions, so you should not leave this decorator on your tests permanently. Use @example <#hypothesis.example> with an explicit input instead.

If you work with multiple developers, or want to share failures across environments (such as locally replaying a failure found by CI), another option is to share the Hypothesis database.

For instance, by setting settings.database <#hypothesis.settings.database> to an instance of a networked database like RedisExampleDatabase <#hypothesis.extra.redis.RedisExampleDatabase>, any developer connecting to that networked database will automatically replay any failures found by other developers.

Similarly, setting settings.database <#hypothesis.settings.database> to GitHubArtifactDatabase <#hypothesis.database.GitHubArtifactDatabase> will automatically replay any failures found by the connected CI artifact.

When a test fails, Hypothesis will normally print output that looks like this:

Falsifying example: test_a_thing(x=1, y="foo")

Sometimes you want to add some additional information to a failure, such as the output of some intermediate step in your test. The note() <#hypothesis.note> function lets you do this:

>>> from hypothesis import given, note, strategies as st
>>> @given(st.lists(st.integers()), st.randoms())
... def test_shuffle_is_noop(ls, r):
...     ls2 = list(ls)
...     r.shuffle(ls2)
...     note(f"Shuffle: {ls2!r}")
...     assert ls == ls2
...
>>> try:
...     test_shuffle_is_noop()
... except AssertionError:
...     print("ls != ls2")
...
Falsifying example: test_shuffle_is_noop(ls=[0, 1], r=RandomWithSeed(1))
Shuffle: [1, 0]
ls != ls2

note() <#hypothesis.note> is like a print statement that gets attached to the falsifying example reported by Hypothesis. It's also reported by observability <#observability>, and shown for all examples (if settings.verbosity <#hypothesis.settings.verbosity> is set to Verbosity.verbose <#hypothesis.Verbosity.verbose> or higher).

Note:

Have you ever had a test fail, and then you re-run it, only for the test to magically pass? That is a flaky test. A flaky test is one which might behave differently when called again. You can think of it as a test which is not deterministic.

Any test can be flaky, but because Hypothesis runs your test many times, Hypothesis tests are particularly likely to uncover flaky behavior.

Note that Hypothesis does not require tests to be fully deterministic. Only the sequence of calls to Hypothesis APIs like draw from @composite <#hypothesis.strategies.composite> and the outcome of the test (pass or fail) need to be deterministic. This means you can use randomness, threads, or nondeterminism in your test, as long as it doesn't impact anything Hypothesis can see.

Hypothesis raises an exception when it detects flakiness. This might seem extreme, relative to a simple warning. But there are good reasons to consider flakiness a fatal error.

• Hypothesis relies on deterministic behavior for the database to work.
• Flakiness makes debugging failures substantially harder if the failing input reported by Hypothesis only flakily reproduces.
• Flakiness makes effectively exploration of the test's behavior space by Hypothesis difficult or impossible.

Here is a quick and non-exhaustive enumeration of some reasons you might encounter flakiness:

• Decisions based on global state.
• Explicit dependencies between inputs.
• Test depends on filesystem or database state which isn't reset between inputs.
• Un-managed sources of randomness. This includes standard PRNGs (see also register_random() <#hypothesis.register_random>), but also thread scheduling, network timing, etc.

Note:

If your tests depend on global state, consider replacing that state with shared() <#hypothesis.strategies.shared>. This is a common way to refactor your test to bring conceptually-global state under the control and visibility of Hypothesis.

When Hypothesis detects that a test is flaky, it will raise one of two Flaky <#hypothesis.errors.Flaky> exceptions.

The most common form of flakiness is that Hypothesis finds a failure, but then replaying that input does not reproduce the failure. For example, here is a contrived test which only fails the first time it is called:

called = False

@given(st.integers())
def test_fails_flakily(n):
    global called
    if not called:
        called = True
        assert False

The first time Hypothesis generates an input, this test will fail. But when Hypothesis tries replaying that failure—by generating the same input—the test will succeed. This test is flaky.

As a result, running test_fails_flakily() will raise FlakyFailure <#hypothesis.errors.FlakyFailure>. FlakyFailure <#hypothesis.errors.FlakyFailure> is an ExceptionGroup, which contains the origin failure as a sub-exception:

+ Exception Group Traceback (most recent call last):
| hypothesis.errors.FlakyFailure: Hypothesis test_fails_flakily(n=0) produces unreliable results: Falsified on the first call but did not on a subsequent one (1 sub-exception)
| Falsifying example: test_fails_flakily(
|     n=0,
| )
| Failed to reproduce exception. Expected:
| Traceback (most recent call last):
|   File "/Users/tybug/Desktop/sandbox2.py", line 13, in test_fails_flakily
|     assert False
|            ^^^^^
| AssertionError
+-+---------------- 1 ----------------
  | ...
  | Traceback (most recent call last):
  |   File "/Users/tybug/Desktop/sandbox2.py", line 13, in test_fails_flakily
  |     assert False
  |            ^^^^^
  | AssertionError
  +------------------------------------

The solution to seeing FlakyFailure <#hypothesis.errors.FlakyFailure> is to refactor the test to not depend on external state. In this case, the external state is the variable called.

Each strategy must 'do the same thing' (again, as seen by Hypothesis) if we replay a previously-seen input.  Failing to do so is a more subtle but equally serious form of flakiness, which leaves us unable to shrink to a minimal failing input, or even reliably report the failure in future runs.

One easy way for this to occur is if a strategy depends on external state. For example, this strategy filters out previously-generated integers, including those seen in any previous test case:

seen = set()

@st.composite
def unique_ints(draw):
    while (n := draw(st.integers())) in seen:
        pass
    seen.add(n)
    return n

@given(unique_ints(), unique_ints())
def test_ints(x, y): ...

By using seen, this test is relying on outside state! In the first test case where integers() <#hypothesis.strategies.integers> generates 0, unique_ints draws only one integer. But if in the next test case integers() <#hypothesis.strategies.integers> generates 0, unique_ints has to draw two integers because 0 is already in seen. This means data generation is not deterministic.

As a result, running test_ints() will raise FlakyStrategyDefinition <#hypothesis.errors.FlakyStrategyDefinition>. The solution is to refactor the strategy to not depend on external state. One way to do this is using shared() <#hypothesis.strategies.shared>:

@st.composite
def unique_ints(draw):
    seen_this_test = draw(st.shared(st.builds(set), key="seen_ints"))
    while (n := draw(st.integers())) in seen_this_test:
        pass
    seen_this_test.add(n)
    return n

These how-to pages are practical guides for applying Hypothesis in specific scenarios. Each page addresses a particular question about using Hypothesis.

Hypothesis sometimes raises a HealthCheck <#hypothesis.HealthCheck> to indicate that your test may be less effective than you expect, slower than you expect, unlikely to generate effective examples, or otherwise has silently degraded performance.

While HealthCheck <#hypothesis.HealthCheck> can be useful to proactively identify issues, you may not care about certain classes of them. If you want to disable a HealthCheck <#hypothesis.HealthCheck> everywhere, you can register and load a settings profile with register_profile() <#hypothesis.settings.register_profile> and load_profile() <#hypothesis.settings.load_profile>. Place the following code in any file which is loaded before running your tests (or in conftest.py, if using pytest):

from hypothesis import HealthCheck, settings

settings.register_profile(
    "my_profile", suppress_health_check=[HealthCheck.filter_too_much]
)
settings.load_profile("my_profile")

This profile in particular suppresses the HealthCheck.filter_too_much <#hypothesis.HealthCheck.filter_too_much> health check for all tests. The exception is if a test has a @settings <#hypothesis.settings> which explicitly sets a different value for suppress_health_check, in which case the profile value will be overridden by the local settings value.

Warning:

We strongly recommend that you suppress health checks as you encounter them, rather than using a blanket suppression. Several health checks check for subtle interactions that may save you hours of debugging, such as HealthCheck.function_scoped_fixture <#hypothesis.HealthCheck.function_scoped_fixture> and HealthCheck.differing_executors <#hypothesis.HealthCheck.differing_executors>.

If you really want to suppress all health checks, for instance to speed up interactive prototyping, you can:

from hypothesis import HealthCheck, settings

settings.register_profile("my_profile", suppress_health_check=list(HealthCheck))
settings.load_profile("my_profile")

Hypothesis provides type hints for all strategies and functions which return a strategy:

from hypothesis import strategies as st

reveal_type(st.integers())
# SearchStrategy[int]

reveal_type(st.lists(st.integers()))
# SearchStrategy[list[int]]

SearchStrategy <#hypothesis.strategies.SearchStrategy> is the type of a strategy. It is parametrized by the type of the example it generates. You can use it to write type hints for your functions which return a strategy:

from hypothesis import strategies as st
from hypothesis.strategies import SearchStrategy

# returns a strategy for "normal" numbers
def numbers() -> SearchStrategy[int | float]:
    return st.integers() | st.floats(allow_nan=False, allow_infinity=False)

It's worth pointing out the distinction between a strategy, and a function that returns a strategy. integers() <#hypothesis.strategies.integers> is a function which returns a strategy, and that strategy has type SearchStrategy[int]. The function st.integers therefore has type Callable[..., SearchStrategy[int]], while the value s = st.integers() has type SearchStrategy[int].

When writing type hints for strategies defined with @composite <#hypothesis.strategies.composite>, use the type of the returned value (not SearchStrategy):

@st.composite
def ordered_pairs(draw) -> tuple[int, int]:
    n1 = draw(st.integers())
    n2 = draw(st.integers(min_value=n1))
    return (n1, n2)

SearchStrategy <#hypothesis.strategies.SearchStrategy> is covariant, meaning that if B < A then SearchStrategy[B] < SearchStrategy[A]. In other words, the strategy st.from_type(Dog) is a subtype of the strategy st.from_type(Animal).

To define your own ExampleDatabase <#hypothesis.database.ExampleDatabase> class, implement the save() <#hypothesis.database.ExampleDatabase.save>, fetch() <#hypothesis.database.ExampleDatabase.fetch>, and delete() <#hypothesis.database.ExampleDatabase.delete> methods.

For example, here's a simple database class that uses sqlite as the backing data store:

import sqlite3
from collections.abc import Iterable

from hypothesis.database import ExampleDatabase

class SQLiteExampleDatabase(ExampleDatabase):
    def __init__(self, db_path: str):
        self.conn = sqlite3.connect(db_path)

        self.conn.execute(
            """
            CREATE TABLE examples (
                key BLOB,
                value BLOB,
                UNIQUE (key, value)
            )
        """
        )

    def save(self, key: bytes, value: bytes) -> None:
        self.conn.execute(
            "INSERT OR IGNORE INTO examples VALUES (?, ?)",
            (key, value),
        )

    def fetch(self, key: bytes) -> Iterable[bytes]:
        cursor = self.conn.execute("SELECT value FROM examples WHERE key = ?", (key,))
        yield from [value[0] for value in cursor.fetchall()]

    def delete(self, key: bytes, value: bytes) -> None:
        self.conn.execute(
            "DELETE FROM examples WHERE key = ? AND value = ?",
            (key, value),
        )

Database classes are not required to implement move() <#hypothesis.database.ExampleDatabase.move>. The default implementation of a move is a delete() <#hypothesis.database.ExampleDatabase.delete> of the value in the old key, followed by a save() <#hypothesis.database.ExampleDatabase.save> of the value in the new key. You can override move() <#hypothesis.database.ExampleDatabase.move> to override this behavior, if for instance the backing store offers a more efficient move implementation.

To support change listening in a database class, you should call _broadcast_change() <#hypothesis.database.ExampleDatabase._broadcast_change> whenever a value is saved, deleted, or moved in the backing database store. How you track this depends on the details of the database class. For instance, in DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase>, Hypothesis installs a filesystem monitor via watchdog <https://pypi.org/project/watchdog/> in order to broadcast change events.

Two useful related methods are _start_listening() <#hypothesis.database.ExampleDatabase._start_listening> and _stop_listening() <#hypothesis.database.ExampleDatabase._stop_listening>, which a database class can override to know when to start or stop expensive listening operations. See documentation for details.

How to dynamically determine whether a test function has been defined with Hypothesis.

The most straightforward way is to use is_hypothesis_test() <#hypothesis.is_hypothesis_test>:

from hypothesis import is_hypothesis_test

@given(st.integers())
def f(n): ...

assert is_hypothesis_test(f)

This works for stateful tests as well:

from hypothesis import is_hypothesis_test
from hypothesis.stateful import RuleBasedStateMachine

class MyStateMachine(RuleBasedStateMachine): ...

assert is_hypothesis_test(MyStateMachine.TestCase().runTest)

If you're working with pytest <https://pypi.org/project/pytest/>, the Hypothesis pytest plugin <#pytest-plugin> automatically adds the @pytest.mark.hypothesis mark to all Hypothesis tests. You can use node.get_closest_marker("hypothesis") or similar methods to detect the existence of this mark.

Sometimes you might want to point a traditional fuzzer like python-afl <https://github.com/jwilk/python-afl> or Google's atheris <https://pypi.org/project/atheris/> at your code, to get coverage-guided exploration of native C extensions.  The associated tooling is often much less mature than property-based testing libraries though, so you might want to use Hypothesis strategies to describe your input data, and our world-class shrinking and observability <#observability> tools to wrangle the results.  That's exactly what this how-to guide is about!

Note:

In order to support this workflow, Hypothesis exposes the fuzz_one_input() <#hypothesis.core.HypothesisHandle.fuzz_one_input> method. fuzz_one_input() <#hypothesis.core.HypothesisHandle.fuzz_one_input> takes a bytestring, parses it into a test case, and executes the corresponding test once. This means you can treat each of your Hypothesis tests as a traditional fuzz target, by pointing the fuzzer at fuzz_one_input() <#hypothesis.core.HypothesisHandle.fuzz_one_input>.

For example:

from hypothesis import given, strategies as st

@given(st.integers())
def test_ints(n):
    pass

# this parses the bytestring into a test case using st.integers(),
# and then executes `test_ints` once.
test_ints.hypothesis.fuzz_one_input(b"\x00" * 50)

Note that fuzz_one_input() <#hypothesis.core.HypothesisHandle.fuzz_one_input> bypasses the standard test lifecycle. In a standard test run, Hypothesis is responsible for managing the lifecycle of a test, for example by moving between each Phase <#hypothesis.Phase>. In contrast, fuzz_one_input() <#hypothesis.core.HypothesisHandle.fuzz_one_input> executes one test case, independent of this lifecycle.

See the documentation of fuzz_one_input() <#hypothesis.core.HypothesisHandle.fuzz_one_input> for details of how it interacts with other features of Hypothesis, such as @settings <#hypothesis.settings>.

Here is an example that uses fuzz_one_input() <#hypothesis.core.HypothesisHandle.fuzz_one_input> with the Atheris <https://github.com/google/atheris> coverage-guided fuzzer (which is built on top of libFuzzer <https://llvm.org/docs/LibFuzzer.html>):

import json
import sys

import atheris

from hypothesis import given, strategies as st

@given(
    st.recursive(
        st.none() | st.booleans() | st.integers() | st.floats() | st.text(),
        lambda j: st.lists(j) | st.dictionaries(st.text(), j),
    )
)
def test_json_dumps_valid_json(value):
    json.dumps(value)

atheris.Setup(sys.argv, test_json_dumps_valid_json.hypothesis.fuzz_one_input)
atheris.Fuzz()

Generating valid JSON objects based only on Atheris' FuzzDataProvider interface would be considerably more difficult.

You may also want to use atheris.instrument_all or atheris.instrument_imports in order to add coverage instrumentation to Atheris.  See the Atheris <https://github.com/google/atheris> documentation for full details.

These explanation pages are oriented towards deepening your understanding of Hypothesis, including its design philosophy.

Note:

This page is primarily for users who may be familiar with other property-based testing libraries, and who expect control over the distribution of inputs in Hypothesis, via e.g. a scale parameter for size or a frequency parameter for relative probabilities.

Hypothesis makes a distinction between the domain of a strategy, and the distribution of a strategy.

The domain is the set of inputs that should be possible to generate. For instance, in lists(integers()), the domain is lists of integers. For other strategies, particularly those that use @composite <#hypothesis.strategies.composite> or assume() <#hypothesis.assume> in their definition, the domain might be more complex.

The distribution is the probability with which different elements in the domain should be generated. For lists(integers()), should Hypothesis generate many small lists? Large lists? More positive or more negative numbers? etc.

Hypothesis takes a philosophical stance that while users may be responsible for selecting the domain, the property-based testing library—not the user—should be responsible for selecting the distribution. As an intentional design choice, Hypothesis therefore lets you control the domain of inputs to your test, but not the distribution.

We recommend using the most-general strategy for your test, so that it can in principle generate any edge case for which the test should pass.  Limiting the size of generated inputs, and especially limiting the variety of inputs, can all too easily exclude the bug-triggering values from consideration - and be the difference between a test which finds the bug, or fails to do so.

Sometimes size limits are necessary for performance reasons, but we recommend limiting your strategies only after you've seen substantial slowdowns without limits.  Far better to find bugs slowly, than not find them at all - and you can manage performance with the phases <#hypothesis.settings.phases> or max_examples <#hypothesis.settings.max_examples> settings rather than weakening the test.

There are a few reasons Hypothesis doesn't let users control the distribution.

• Humans are pretty bad at choosing bug-finding distributions! Some bugs are "known unknowns": you suspected that a part of the codebase was buggy in a particular way. Others are "unknown unknowns": you didn't know that a bug was possible until stumbling across it. Humans tend to overtune distributions for the former kind of bug, and not enough for the latter.
• The ideal strategy distribution depends not only on the codebase, but also on the property being tested. A strategy used in many places may have a good distribution for one property, but not another.
• The distribution of inputs is a deeply internal implementation detail. We sometimes change strategy distributions, either explicitly, or implicitly from other work on the Hypothesis engine. Exposing this would lock us into a public API that may make improvements to Hypothesis more difficult.

Finally, we think distribution control is better handled with alternative backends <#alternative-backends>. If existing backends like hypofuzz and crosshair don't suit your needs, you can also write your own. Backends can automatically generalize and adapt to the strategy and property being tested and avoid most of the problems above.

We're not saying that control over the distribution isn't useful! We occasionally receive requests to expose the distribution in Hypothesis (e.g. <https://github.com/HypothesisWorks/hypothesis/issues/4205>), and users wouldn't be asking for it if it wasn't. However, adding this to the public strategy API would make it easy for users to unknowingly weaken their tests, and would add maintenance overhead to Hypothesis, and so we haven't yet done so.

An exact answer depends on both the strategy or strategies for the tests, and the code being tested - but we can make some general remarks, starting with "it's actually really complicated".

Hypothesis' default configuration uses a distribution which is tuned to maximize the chance of finding bugs, in as few executions as possible.  We explicitly don't aim for a uniform distribution, nor for a 'realistic' distribution of inputs; Hypothesis' goal is to search the domain for a failing input as efficiently as possible.

The test case distribution remains an active area of research and development, and we change it whenever we think that would be a net improvement for users.  Today, Hypothesis' default distribution is shaped by a wide variety of techniques and heuristics:

• some are statically designed into strategies - for example, integers() <#hypothesis.strategies.integers> upweights range endpoints, and samples from a mixed distribution over integer bit-widths.
• some are dynamic features of the engine - like replaying prior examples with subsections of the input 'cloned' or otherwise altered, for bugs which trigger only when different fields have the same value (which is otherwise exponentially unlikely).
• some vary depending on the code under test - we collect interesting-looking constants from imported source files as seeds for test cases.

And as if that wasn't enough, alternative backends <#alternative-backends> can radically change the distribution again - for example hypofuzz <https://pypi.org/project/hypofuzz/> uses runtime feedback to modify the distribution of inputs as the test runs, to maximize the rate at which we trigger new behaviors in that particular test and code.  If Hypothesis' defaults aren't strong enough, we recommend trying Hypofuzz!

This is a trickier question than you might expect. The short answer is "exactly max_examples <#hypothesis.settings.max_examples> times", with the following exceptions:

• Less than max_examples <#hypothesis.settings.max_examples> times, if Hypothesis exhausts the search space early.

• More than max_examples <#hypothesis.settings.max_examples> times, if Hypothesis retries some examples because either:

  • They failed an assume() <#hypothesis.assume> or .filter() <#hypothesis.strategies.SearchStrategy.filter> condition, or
  • They were too large to continue generating.
• Either less or more than max_examples <#hypothesis.settings.max_examples> times, if Hypothesis finds a failing example.

Read on for details.

If Hypothesis detects that there are no more examples left to try, it may stop generating examples before it hits max_examples <#hypothesis.settings.max_examples>. For example:

from hypothesis import given, strategies as st

calls = 0

@given(st.integers(0, 19))
def test_function(n):
    global calls
    calls += 1

test_function()
assert calls == 20

This runs test_function 20 times, not 100, since there are only 20 unique integers to try.

The search space tracking in Hypothesis is good, but not perfect. We treat this more as a bonus than something to strive for.

If an example fails to satisfy an assume() <#hypothesis.assume> or .filter() <#hypothesis.strategies.SearchStrategy.filter> condition, Hypothesis will retry generating that example and will not count it towards the max_examples <#hypothesis.settings.max_examples> limit. For instance:

from hypothesis import assume, given, strategies as st

@given(st.integers())
def test_function(n):
    assume(n % 2 == 0)

will run roughly 200 times, since half of the examples are discarded from the assume() <#hypothesis.assume>.

Note that while failing an assume() <#hypothesis.assume> triggers an immediate retry of the entire example, Hypothesis will try several times in the same example to satisfy a .filter() <#hypothesis.strategies.SearchStrategy.filter> condition. This makes expressing the same condition using .filter() <#hypothesis.strategies.SearchStrategy.filter> more efficient than assume() <#hypothesis.assume>.

Also note that even if your code does not explicitly use assume() <#hypothesis.assume> or .filter() <#hypothesis.strategies.SearchStrategy.filter>, a builtin strategy may still use them and cause retries. We try to directly satisfy conditions where possible instead of relying on rejection sampling, so this should be relatively uncommon.

For performance reasons, Hypothesis places an internal limit on the size of a single example. If an example exceeds this size limit, we will retry generating it and will not count it towards the max_examples <#hypothesis.settings.max_examples> limit. (And if we see too many of these large examples, we will raise HealthCheck.data_too_large <#hypothesis.HealthCheck.data_too_large>, unless suppressed with settings.suppress_health_check <#hypothesis.settings.suppress_health_check>).

The specific value of this size limit is an undocumented implementation detail. The majority of Hypothesis tests do not come close to hitting it.

If Hypothesis finds a failing example, it stops generation early, and may call the test function additional times during the Phase.shrink <#hypothesis.Phase.shrink> and Phase.explain <#hypothesis.Phase.explain> phases. Sometimes, Hypothesis determines that the initial failing example was already as simple as possible, in which case Phase.shrink <#hypothesis.Phase.shrink> will not result in additional test executions (but Phase.explain <#hypothesis.Phase.explain> might).

Regardless of whether Hypothesis runs the test during the shrinking and explain phases, it will always run the minimal failing example one additional time to check for flakiness. For instance, the following trivial test runs with n=0 twice, even though it only uses the Phase.generate <#hypothesis.Phase.generate> phase:

from hypothesis import Phase, given, settings, strategies as st

@given(st.integers())
@settings(phases=[Phase.generate])
def test_function(n):
    print(f"called with {n}")
    assert n != 0

test_function()

The first execution finds the initial failure with n=0, and the second execution replays n=0 to ensure the failure is not flaky.

The technical API reference for Hypothesis is split into four pages:

• API Reference <>. Non-strategy Hypothesis objects, classes, and functions. @given <#hypothesis.given> and others live here.
• Strategies Reference <>. Hypothesis strategies, including for extras <>.
• Integrations Reference <>. Features with a defined interface, but no code API.
• Hypothesis internals <>. Internal APIs for developers building tools, libraries, or research on top of Hypothesis.

Reference for non-strategy objects that are part of the Hypothesis API. For documentation on strategies, see the strategies reference <>.

hypothesis.given(*_given_arguments, **_given_kwargs)

The @given decorator turns a function into a Hypothesis test. This is the main entry point to Hypothesis.

See also:

See also the Introduction to Hypothesis <> tutorial, which introduces defining Hypothesis tests with @given.

Arguments to @given may be either positional or keyword arguments:

@given(st.integers(), st.floats())
def test_one(x, y):
    pass

@given(x=st.integers(), y=st.floats())
def test_two(x, y):
    pass

If using keyword arguments, the arguments may appear in any order, as with standard Python functions:

# different order, but still equivalent to before
@given(y=st.floats(), x=st.integers())
def test(x, y):
    assert isinstance(x, int)
    assert isinstance(y, float)

If @given is provided fewer positional arguments than the decorated test, the test arguments are filled in on the right side, leaving the leftmost positional arguments unfilled:

@given(st.integers(), st.floats())
def test(manual_string, y, z):
    assert manual_string == "x"
    assert isinstance(y, int)
    assert isinstance(z, float)

# `test` is now a callable which takes one argument `manual_string`

test("x")
# or equivalently:
test(manual_string="x")

The reason for this "from the right" behavior is to support using @given with instance methods, by automatically passing through self:

class MyTest(TestCase):
    @given(st.integers())
    def test(self, x):
        assert isinstance(self, MyTest)
        assert isinstance(x, int)

If (and only if) using keyword arguments, @given may be combined with **kwargs or *args:

@given(x=integers(), y=integers())
def test(x, **kwargs):
    assert "y" in kwargs

@given(x=integers(), y=integers())
def test(x, *args, **kwargs):
    assert args == ()
    assert "x" not in kwargs
    assert "y" in kwargs

It is an error to:

• Mix positional and keyword arguments to @given.
• Use @given with a function that has a default value for an argument.
• Use @given with positional arguments with a function that uses *args, **kwargs, or keyword-only arguments.

The function returned by given has all the same arguments as the original test, minus those that are filled in by @given. See the notes on framework compatibility <#framework-compatibility> for how this interacts with features of other testing libraries, such as pytest <https://pypi.org/project/pytest/> fixtures.

hypothesis.infer

An alias for ... (python:Ellipsis). infer can be passed to @given or builds() <#hypothesis.strategies.builds> to indicate that a strategy for that parameter should be inferred from its type annotations.

In all cases, using infer is equivalent to using ....

In some cases, Hypothesis can work out what to do when you omit arguments. This is based on introspection, not magic, and therefore has well-defined limits.

builds() <#hypothesis.strategies.builds> will check the signature of the target (using python:inspect.signature()). If there are required arguments with type annotations and no strategy was passed to builds() <#hypothesis.strategies.builds>, from_type() <#hypothesis.strategies.from_type> is used to fill them in. You can also pass the value ... (Ellipsis) as a keyword argument, to force this inference for arguments with a default value.

>>> def func(a: int, b: str):
...     return [a, b]
...
>>> builds(func).example()
[-6993, '']

@given does not perform any implicit inference for required arguments, as this would break compatibility with pytest fixtures. ... (python:Ellipsis), can be used as a keyword argument to explicitly fill in an argument from its type annotation.  You can also use the infer alias if writing a literal ... seems too weird.

@given(a=...)  # or @given(a=infer)
def test(a: int):
    pass

# is equivalent to
@given(a=from_type(int))
def test(a):
    pass

@given(...) can also be specified to fill all arguments from their type annotations.

@given(...)
def test(a: int, b: str):
    pass

# is equivalent to
@given(a=..., b=...)
def test(a, b):
    pass

Hypothesis does not inspect PEP 484 <https://peps.python.org/pep-0484/> type comments at runtime. While from_type() <#hypothesis.strategies.from_type> will work as usual, inference in builds() <#hypothesis.strategies.builds> and @given will only work if you manually create the __annotations__ attribute (e.g. by using @annotations(...) and @returns(...) decorators).

The python:typing module changes between different Python releases, including at minor versions.  These are all supported on a best-effort basis, but you may encounter problems.  Please report them to us, and consider updating to a newer version of Python as a workaround.

See also:

class hypothesis.example(*args, **kwargs)

Add an explicit input to a Hypothesis test, which Hypothesis will always try before generating random inputs. This combines the randomized nature of Hypothesis generation with a traditional parametrized test.

For example:

@example("Hello world")
@example("some string with special significance")
@given(st.text())
def test_strings(s):
    pass

will call test_strings("Hello World") and test_strings("some string with special significance") before generating any random inputs. @example may be placed in any order relative to @given and @settings.

Explicit inputs from @example are run in the Phase.explicit phase. Explicit inputs do not count towards settings.max_examples. Note that explicit inputs added by @example do not shrink. If an explicit input fails, Hypothesis will stop and report the failure without generating any random inputs.

@example can also be used to easily reproduce a failure. For instance, if Hypothesis reports that f(n=[0, math.nan]) fails, you can add @example(n=[0, math.nan]) to your test to quickly reproduce that failure.

Arguments to @example have the same behavior and restrictions as arguments to @given. This means they may be either positional or keyword arguments (but not both in the same @example):

@example(1, 2)
@example(x=1, y=2)
@given(st.integers(), st.integers())
def test(x, y):
    pass

Noting that while arguments to @given are strategies (like integers() <#hypothesis.strategies.integers>), arguments to @example are values instead (like 1).

See the Arguments to @given section for full details.

example.xfail(condition=True, *, reason='', raises=<class 'BaseException'>)

Mark this example as an expected failure, similarly to pytest.mark.xfail(strict=True).

Expected-failing examples allow you to check that your test does fail on some examples, and therefore build confidence that passing tests are because your code is working, not because the test is missing something.

@example(...).xfail()
@example(...).xfail(reason="Prices must be non-negative")
@example(...).xfail(raises=(KeyError, ValueError))
@example(...).xfail(sys.version_info[:2] >= (3, 12), reason="needs py 3.12")
@example(...).xfail(condition=sys.platform != "linux", raises=OSError)
def test(x):
    pass

Note:

Expected-failing examples are handled separately from those generated by strategies, so you should usually ensure that there is no overlap.

@example(x=1, y=0).xfail(raises=ZeroDivisionError)
@given(x=st.just(1), y=st.integers())  # Missing `.filter(bool)`!
def test_fraction(x, y):
    # This test will try the explicit example and see it fail as
    # expected, then go on to generate more examples from the
    # strategy.  If we happen to generate y=0, the test will fail
    # because only the explicit example is treated as xfailing.
    x / y

example.via(whence, /)

Attach a machine-readable label noting what the origin of this example was. example.via is completely optional and does not change runtime behavior.

example.via is intended to support self-documenting behavior, as well as tooling which might add (or remove) @example decorators automatically. For example:

# Annotating examples is optional and does not change runtime behavior
@example(...)
@example(...).via("regression test for issue #42")
@example(...).via("discovered failure")
def test(x):
    pass

Note:

HypoFuzz <https://hypofuzz.com/> uses example.via to tag examples in the patch of its high-coverage set of explicit inputs, on the patches page <https://hypofuzz.com/example-dashboard/#/patches>.

See also:

hypothesis.reproduce_failure(version, blob)

Run the example corresponding to the binary blob in order to reproduce a failure. blob is a serialized version of the internal input representation of Hypothesis.

A test decorated with @reproduce_failure always runs exactly one example, which is expected to cause a failure. If the provided blob does not cause a failure, Hypothesis will raise DidNotReproduce.

Hypothesis will print an @reproduce_failure decorator if settings.print_blob is True (which is the default in CI).

@reproduce_failure is intended to be temporarily added to your test suite in order to reproduce a failure. It is not intended to be a permanent addition to your test suite. Because of this, no compatibility guarantees are made across Hypothesis versions, and @reproduce_failure will error if used on a different Hypothesis version than it was created for.

See also:

See also the Replaying failed tests <> tutorial.

hypothesis.seed(seed)

Seed the randomness for this test.

seed may be any hashable object. No exact meaning for seed is provided other than that for a fixed seed value Hypothesis will produce the same examples (assuming that there are no other sources of nondeterminisim, such as timing, hash randomization, or external state).

For example, the following test function and RuleBasedStateMachine will each generate the same series of examples each time they are executed:

@seed(1234)
@given(st.integers())
def test(n): ...

@seed(6789)
class MyMachine(RuleBasedStateMachine): ...

If using pytest, you can alternatively pass --hypothesis-seed on the command line.

Setting a seed overrides settings.derandomize, which is designed to enable deterministic CI tests rather than reproducing observed failures.

Hypothesis will only print the seed which would reproduce a failure if a test fails in an unexpected way, for instance inside Hypothesis internals.

Functions that can be called from anywhere inside a test, to either modify how Hypothesis treats the current test case, or to give Hypothesis more information about the current test case.

hypothesis.assume(condition)

Calling assume is like an assert that marks the example as bad, rather than failing the test.

This allows you to specify properties that you assume will be true, and let Hypothesis try to avoid similar examples in future.

hypothesis.note(value)

Report this value for the minimal failing example.

hypothesis.event(value, payload='')

Record an event that occurred during this test. Statistics on the number of test runs with each event will be reported at the end if you run Hypothesis in statistics reporting mode.

Event values should be strings or convertible to them.  If an optional payload is given, it will be included in the string for Test statistics <#statistics>.

You can mark custom events in a test using event():

from hypothesis import event, given, strategies as st

@given(st.integers().filter(lambda x: x % 2 == 0))
def test_even_integers(i):
    event(f"i mod 3 = {i%3}")

These events appear in observability <#observability> output, as well as the output of our pytest plugin <#pytest-plugin> when run with --hypothesis-show-statistics.

For instance, in the latter case, you would see output like:

test_even_integers:

  - during generate phase (0.09 seconds):
      - Typical runtimes: < 1ms, ~ 59% in data generation
      - 100 passing examples, 0 failing examples, 32 invalid examples
      - Events:
        * 54.55%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
        * 31.06%, i mod 3 = 2
        * 28.79%, i mod 3 = 0
        * 24.24%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
        * 15.91%, i mod 3 = 1
  - Stopped because settings.max_examples=100

Arguments to event can be any hashable type, but two events will be considered the same if they are the same when converted to a string with python:str.

Targeted property-based testing combines the advantages of both search-based and property-based testing.  Instead of being completely random, targeted PBT uses a search-based component to guide the input generation towards values that have a higher probability of falsifying a property.  This explores the input space more effectively and requires fewer tests to find a bug or achieve a high confidence in the system being tested than random PBT. (Löscher and Sagonas <http://proper.softlab.ntua.gr/Publications.html>)

This is not always a good idea - for example calculating the search metric might take time better spent running more uniformly-random test cases, or your target metric might accidentally lead Hypothesis away from bugs - but if there is a natural metric like "floating-point error", "load factor" or "queue length", we encourage you to experiment with targeted testing.

We recommend that users also skim the papers introducing targeted PBT; from ISSTA 2017 <http://proper.softlab.ntua.gr/papers/issta2017.pdf> and ICST 2018 <http://proper.softlab.ntua.gr/papers/icst2018.pdf>. For the curious, the initial implementation in Hypothesis uses hill-climbing search via a mutating fuzzer, with some tactics inspired by simulated annealing to avoid getting stuck and endlessly mutating a local maximum.

from hypothesis import given, strategies as st, target

@given(st.floats(0, 1e100), st.floats(0, 1e100), st.floats(0, 1e100))
def test_associativity_with_target(a, b, c):
    ab_c = (a + b) + c
    a_bc = a + (b + c)
    difference = abs(ab_c - a_bc)
    target(difference)  # Without this, the test almost always passes
    assert difference < 2.0

hypothesis.target(observation, *, label='')

Calling this function with an int or float observation gives it feedback with which to guide our search for inputs that will cause an error, in addition to all the usual heuristics.  Observations must always be finite.

Hypothesis will try to maximize the observed value over several examples; almost any metric will work so long as it makes sense to increase it. For example, -abs(error) is a metric that increases as error approaches zero.

Example metrics:

• Number of elements in a collection, or tasks in a queue
• Mean or maximum runtime of a task (or both, if you use label)
• Compression ratio for data (perhaps per-algorithm or per-level)
• Number of steps taken by a state machine

The optional label argument can be used to distinguish between and therefore separately optimise distinct observations, such as the mean and standard deviation of a dataset.  It is an error to call target() with any label more than once per test case.

Note:

The more examples you run, the better this technique works.

As a rule of thumb, the targeting effect is noticeable above max_examples=1000, and immediately obvious by around ten thousand examples per label used by your test.

Test statistics <#statistics> include the best score seen for each label, which can help avoid the threshold problem <https://hypothesis.works/articles/threshold-problem/> when the minimal example shrinks right down to the threshold of failure (issue #2180 <https://github.com/HypothesisWorks/hypothesis/issues/2180>).

See also:

class hypothesis.settings(parent=None, *, max_examples=not_set, derandomize=not_set, database=not_set, verbosity=not_set, phases=not_set, stateful_step_count=not_set, report_multiple_bugs=not_set, suppress_health_check=not_set, deadline=not_set, print_blob=not_set, backend=not_set)

A settings object controls the following aspects of test behavior: max_examples, derandomize, database, verbosity, phases, stateful_step_count, report_multiple_bugs, suppress_health_check, deadline, print_blob, and backend.

A settings object can be applied as a decorator to a test function, in which case that test function will use those settings. A test may only have one settings object applied to it. A settings object can also be passed to register_profile() or as a parent to another settings.

Settings objects are immutable once created. When a settings object is created, it uses the value specified for each attribute. Any attribute which is not specified will inherit from its value in the parent settings object. If parent is not passed, any attributes which are not specified will inherit from the current settings profile instead.

For instance, settings(max_examples=10) will have a max_examples of 10, and the value of all other attributes will be equal to its value in the current settings profile.

Changes made from activating a new settings profile with load_profile() will be reflected in settings objects created after the profile was loaded, but not in existing settings objects.

While you can register additional profiles with register_profile(), Hypothesis comes with two built-in profiles: default and ci.

By default, the default profile is active. If the CI environment variable is set to any value, the ci profile is active by default. Hypothesis also automatically detects various vendor-specific CI environment variables.

The attributes of the currently active settings profile can be retrieved with settings() (so settings().max_examples is the currently active default for settings.max_examples).

The settings attributes for the built-in profiles are as follows:

default = settings.register_profile(
    "default",
    max_examples=100,
    derandomize=False,
    database=not_set,  # see settings.database for default behavior
    verbosity=Verbosity.normal,
    phases=tuple(Phase),
    stateful_step_count=50,
    report_multiple_bugs=True,
    suppress_health_check=(),
    deadline=duration(milliseconds=200),
    print_blob=False,
    backend="hypothesis",
)

ci = settings.register_profile(
    "ci",
    parent=default,
    derandomize=True,
    deadline=None,
    database=None,
    print_blob=True,
    suppress_health_check=[HealthCheck.too_slow],
)

You can replace either of the built-in profiles with register_profile():

# run more examples in CI
settings.register_profile(
    "ci",
    settings.get_profile("ci"),
    max_examples=1000,
)

class hypothesis.Phase(*values)

Options for the settings.phases argument to @settings.

explicit = 'explicit'

Controls whether explicit examples are run.

reuse = 'reuse'

Controls whether previous examples will be reused.

generate = 'generate'

Controls whether new examples will be generated.

target = 'target'

Controls whether examples will be mutated for targeting.

shrink = 'shrink'

Controls whether examples will be shrunk.

explain = 'explain'

Controls whether Hypothesis attempts to explain test failures.

The explain phase has two parts, each of which is best-effort - if Hypothesis can't find a useful explanation, we'll just print the minimal failing example.

class hypothesis.Verbosity(*values)

Options for the settings.verbosity argument to @settings.

quiet = 'quiet'

Hypothesis will not print any output, not even the final falsifying example.

normal = 'normal'

Standard verbosity. Hypothesis will print the falsifying example, alongside any notes made with note() (only for the falsfying example).

verbose = 'verbose'

Increased verbosity. In addition to everything in Verbosity.normal, Hypothesis will:

• Print each test case as it tries it
• Print any notes made with note() for each test case
• Print each shrinking attempt
• Print all explicit failing examples when using @example, instead of only the simplest one

debug = 'debug'

Even more verbosity. Useful for debugging Hypothesis internals. You probably don't want this.

class hypothesis.HealthCheck(*values)

A HealthCheck is proactively raised by Hypothesis when Hypothesis detects that your test has performance problems, which may result in less rigorous testing than you expect. For example, if your test takes a long time to generate inputs, or filters away too many  inputs using assume() or .filter() <#hypothesis.strategies.SearchStrategy.filter>, Hypothesis will raise a corresponding health check.

A health check is a proactive warning, not an error. We encourage suppressing health checks where you have evaluated they will not pose a problem, or where you have evaluated that fixing the underlying issue is not worthwhile.

With the exception of HealthCheck.function_scoped_fixture and HealthCheck.differing_executors, all health checks warn about performance problems, not correctness errors.

Health checks can be disabled by settings.suppress_health_check. To suppress all health checks, you can pass suppress_health_check=list(HealthCheck).

See also:

Some health checks report potential correctness errors, rather than performance problems.

• HealthCheck.function_scoped_fixture indicates that a function-scoped pytest fixture is used by an @given test. Many Hypothesis users expect function-scoped fixtures to reset once per input, but they actually reset once per test. We proactively raise HealthCheck.function_scoped_fixture to ensure you have considered this case.
• HealthCheck.differing_executors indicates that the same @given test has been executed multiple times with multiple distinct executors.

We recommend treating these particular health checks with more care, as suppressing them may result in an unsound test.

class hypothesis.database.ExampleDatabase

A Hypothesis database, for use in settings.database.

Hypothesis automatically saves failures to the database set in settings.database. The next time the test is run, Hypothesis will replay any failures from the database in settings.database for that test (in Phase.reuse).

The database is best thought of as a cache that you never need to invalidate. Entries may be transparently dropped when upgrading your Hypothesis version or changing your test. Do not rely on the database for correctness; to ensure Hypothesis always tries an input, use @example.

A Hypothesis database is a simple mapping of bytes to sets of bytes. Hypothesis provides several concrete database subclasses. To write your own database class, see Write a custom Hypothesis database <>.

An optional extension to ExampleDatabase is change listening. On databases which support change listening, calling add_listener() adds a function as a change listener, which will be called whenever a value is added, deleted, or moved inside the database. See add_listener() for details.

All databases in Hypothesis support change listening. Custom database classes are not required to support change listening, though they will not be compatible with features that require change listening until they do so.

Note:

Required methods:

• save()
• fetch()
• delete()

Optional methods:

• move()

Change listening methods:

class hypothesis.database.InMemoryExampleDatabase

A non-persistent example database, implemented in terms of an in-memory dictionary.

This can be useful if you call a test function several times in a single session, or for testing other database implementations, but because it does not persist between runs we do not recommend it for general use.

class hypothesis.database.DirectoryBasedExampleDatabase(path)

Use a directory to store Hypothesis examples as files.

Each test corresponds to a directory, and each example to a file within that directory.  While the contents are fairly opaque, a DirectoryBasedExampleDatabase can be shared by checking the directory into version control, for example with the following .gitignore:

# Ignore files cached by Hypothesis...
.hypothesis/*
# except for the examples directory
!.hypothesis/examples/

Note however that this only makes sense if you also pin to an exact version of Hypothesis, and we would usually recommend implementing a shared database with a network datastore - see ExampleDatabase, and the MultiplexedDatabase helper.

class hypothesis.database.GitHubArtifactDatabase(owner, repo, artifact_name='hypothesis-example-db', cache_timeout=datetime.timedelta(days=1), path=None)

A file-based database loaded from a GitHub Actions <https://docs.github.com/en/actions> artifact.

You can use this for sharing example databases between CI runs and developers, allowing the latter to get read-only access to the former. This is particularly useful for continuous fuzzing (i.e. with HypoFuzz <https://hypofuzz.com/>), where the CI system can help find new failing examples through fuzzing, and developers can reproduce them locally without any manual effort.

Note:

You must provide GITHUB_TOKEN as an environment variable. In CI, Github Actions provides this automatically, but it needs to be set manually for local usage. In a developer machine, this would usually be a Personal Access Token <https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens>. If the repository is private, it's necessary for the token to have repo scope in the case of a classic token, or actions:read in the case of a fine-grained token.

In most cases, this will be used through the MultiplexedDatabase, by combining a local directory-based database with this one. For example:

local = DirectoryBasedExampleDatabase(".hypothesis/examples")
shared = ReadOnlyDatabase(GitHubArtifactDatabase("user", "repo"))

settings.register_profile("ci", database=local)
settings.register_profile("dev", database=MultiplexedDatabase(local, shared))
# We don't want to use the shared database in CI, only to populate its local one.
# which the workflow should then upload as an artifact.
settings.load_profile("ci" if os.environ.get("CI") else "dev")

Note:

Because this database is read-only, you always need to wrap it with the ReadOnlyDatabase.

A setup like this can be paired with a GitHub Actions workflow including something like the following:

- name: Download example database
  uses: dawidd6/action-download-artifact@v9
  with:
    name: hypothesis-example-db
    path: .hypothesis/examples
    if_no_artifact_found: warn
    workflow_conclusion: completed

- name: Run tests
  run: pytest

- name: Upload example database
  uses: actions/upload-artifact@v3
  if: always()
  with:
    name: hypothesis-example-db
    path: .hypothesis/examples

In this workflow, we use dawidd6/action-download-artifact <https://github.com/dawidd6/action-download-artifact> to download the latest artifact given that the official actions/download-artifact <https://github.com/actions/download-artifact> does not support downloading artifacts from previous workflow runs.

The database automatically implements a simple file-based cache with a default expiration period of 1 day. You can adjust this through the cache_timeout property.

For mono-repo support, you can provide a unique artifact_name (e.g. hypofuzz-example-db-frontend).

class hypothesis.database.ReadOnlyDatabase(db)

A wrapper to make the given database read-only.

The implementation passes through fetch, and turns save, delete, and move into silent no-ops.

Note that this disables Hypothesis' automatic discarding of stale examples. It is designed to allow local machines to access a shared database (e.g. from CI servers), without propagating changes back from a local or in-development branch.

class hypothesis.database.MultiplexedDatabase(*dbs)

A wrapper around multiple databases.

Each save, fetch, move, or delete operation will be run against all of the wrapped databases.  fetch does not yield duplicate values, even if the same value is present in two or more of the wrapped databases.

This combines well with a ReadOnlyDatabase, as follows:

local = DirectoryBasedExampleDatabase("/tmp/hypothesis/examples/")
shared = CustomNetworkDatabase()

settings.register_profile("ci", database=shared)
settings.register_profile(
    "dev", database=MultiplexedDatabase(local, ReadOnlyDatabase(shared))
)
settings.load_profile("ci" if os.environ.get("CI") else "dev")

So your CI system or fuzzing runs can populate a central shared database; while local runs on development machines can reproduce any failures from CI but will only cache their own failures locally and cannot remove examples from the shared database.

class hypothesis.database.BackgroundWriteDatabase(db)

A wrapper which defers writes on the given database to a background thread.

Calls to fetch() wait for any enqueued writes to finish before fetching from the database.

class hypothesis.extra.redis.RedisExampleDatabase(redis, *, expire_after=datetime.timedelta(days=8), key_prefix=b'hypothesis-example:', listener_channel='hypothesis-changes')

Store Hypothesis examples as sets in the given Redis datastore.

This is particularly useful for shared databases, as per the recipe for a MultiplexedDatabase.

Note:

If a test has not been run for expire_after, those examples will be allowed to expire.  The default time-to-live persists examples between weekly runs.

class hypothesis.stateful.RuleBasedStateMachine

A RuleBasedStateMachine gives you a structured way to define state machines.

The idea is that a state machine carries the system under test and some supporting data. This data can be stored in instance variables or divided into Bundles. The state machine has a set of rules which may read data from bundles (or just from normal strategies), push data onto bundles, change the state of the machine, or verify properties. At any given point a random applicable rule will be executed.

hypothesis.stateful.rule(*, targets=(), target=None, **kwargs)

Decorator for RuleBasedStateMachine. Any Bundle present in target or targets will define where the end result of this function should go. If both are empty then the end result will be discarded.

target must be a Bundle, or if the result should be replicated to multiple bundles you can pass a tuple of them as the targets argument. It is invalid to use both arguments for a single rule.  If the result should go to exactly one of several bundles, define a separate rule for each case.

kwargs then define the arguments that will be passed to the function invocation. If their value is a Bundle, or if it is consumes(b) where b is a Bundle, then values that have previously been produced for that bundle will be provided. If consumes is used, the value will also be removed from the bundle.

Any other kwargs should be strategies and values from them will be provided.

hypothesis.stateful.consumes(bundle)

When introducing a rule in a RuleBasedStateMachine, this function can be used to mark bundles from which each value used in a step with the given rule should be removed. This function returns a strategy object that can be manipulated and combined like any other.

For example, a rule declared with

@rule(value1=b1, value2=consumes(b2), value3=lists(consumes(b3)))

will consume a value from Bundle b2 and several values from Bundle b3 to populate value2 and value3 each time it is executed.

hypothesis.stateful.multiple(*args)

This function can be used to pass multiple results to the target(s) of a rule. Just use return multiple(result1, result2, ...) in your rule.

It is also possible to use return multiple() with no arguments in order to end a rule without passing any result.

class hypothesis.stateful.Bundle(name, *, consume=False, draw_references=True)

A collection of values for use in stateful testing.

Bundles are a kind of strategy where values can be added by rules, and (like any strategy) used as inputs to future rules.

The name argument they are passed is the they are referred to internally by the state machine; no two bundles may have the same name. It is idiomatic to use the attribute being assigned to as the name of the Bundle:

class MyStateMachine(RuleBasedStateMachine):
    keys = Bundle("keys")

Bundles can contain the same value more than once; this becomes relevant when using consumes() to remove values again.

If the consume argument is set to True, then all values that are drawn from this bundle will be consumed (as above) when requested.

hypothesis.stateful.initialize(*, targets=(), target=None, **kwargs)

Decorator for RuleBasedStateMachine.

An initialize decorator behaves like a rule, but all @initialize() decorated methods will be called before any @rule() decorated methods, in an arbitrary order.  Each @initialize() method will be called exactly once per run, unless one raises an exception - after which only the .teardown() method will be run. @initialize() methods may not have preconditions.

hypothesis.stateful.precondition(precond)

Decorator to apply a precondition for rules in a RuleBasedStateMachine. Specifies a precondition for a rule to be considered as a valid step in the state machine, which is more efficient than using assume() within the rule.  The precond function will be called with the instance of RuleBasedStateMachine and should return True or False. Usually it will need to look at attributes on that instance.

For example:

class MyTestMachine(RuleBasedStateMachine):
    state = 1

    @precondition(lambda self: self.state != 0)
    @rule(numerator=integers())
    def divide_with(self, numerator):
        self.state = numerator / self.state

If multiple preconditions are applied to a single rule, it is only considered a valid step when all of them return True.  Preconditions may be applied to invariants as well as rules.

hypothesis.stateful.invariant(*, check_during_init=False)

Decorator to apply an invariant for rules in a RuleBasedStateMachine. The decorated function will be run after every rule and can raise an exception to indicate failed invariants.

For example:

class MyTestMachine(RuleBasedStateMachine):
    state = 1

    @invariant()
    def is_nonzero(self):
        assert self.state != 0

By default, invariants are only checked after all @initialize() rules have been run. Pass check_during_init=True for invariants which can also be checked during initialization.

If you want to bypass the TestCase infrastructure you can invoke these manually. The stateful module exposes run_state_machine_as_test(), which takes an arbitrary function returning a RuleBasedStateMachine and an optional settings parameter and does the same as the class based runTest provided.

hypothesis.stateful.run_state_machine_as_test(state_machine_factory, *, settings=None, _min_steps=0)

Run a state machine definition as a test, either silently doing nothing or printing a minimal breaking program and raising an exception.

state_machine_factory is anything which returns an instance of RuleBasedStateMachine when called with no arguments - it can be a class or a function. settings will be used to control the execution of the test.

Custom exceptions raised by Hypothesis.

class hypothesis.errors.HypothesisException

Generic parent class for exceptions thrown by Hypothesis.

class hypothesis.errors.HypothesisDeprecationWarning

A deprecation warning issued by Hypothesis.

Actually inherits from FutureWarning, because DeprecationWarning is hidden by the default warnings filter.

You can configure the python:warnings module to handle these warnings differently to others, either turning them into errors or suppressing them entirely.  Obviously we would prefer the former!

class hypothesis.errors.NonInteractiveExampleWarning

Emitted when .example() <#hypothesis.strategies.SearchStrategy.example> is used outside of interactive use.

.example() <#hypothesis.strategies.SearchStrategy.example> is intended for exploratory and interactive work, not to be run as part of a test suite.

class hypothesis.errors.Flaky

Base class for indeterministic failures. Usually one of the more specific subclasses (FlakyFailure or FlakyStrategyDefinition) is raised.

See also:

See also the flaky failures tutorial <>.

class hypothesis.errors.FlakyStrategyDefinition

This function appears to cause inconsistent data generation.

Common causes for this problem are:

1.

The strategy depends on external state. e.g. it uses an external random number generator. Try to make a version that passes all the relevant state in from Hypothesis.

See also:

See also the flaky failures tutorial <>.

class hypothesis.errors.FlakyFailure(msg, group)

This function appears to fail non-deterministically: We have seen it fail when passed this example at least once, but a subsequent invocation did not fail, or caused a distinct error.

Common causes for this problem are:

1. The function depends on external state. e.g. it uses an external random number generator. Try to make a version that passes all the relevant state in from Hypothesis.
2. The function is suffering from too much recursion and its failure depends sensitively on where it's been called from.
3. The function is timing sensitive and can fail or pass depending on how long it takes. Try breaking it up into smaller functions which don't do that and testing those instead.

See also:

See also the flaky failures tutorial <>.

class hypothesis.errors.FlakyBackendFailure(msg, group)

A failure was reported by an alternative backend <#alternative-backends>, but this failure did not reproduce when replayed under the Hypothesis backend.

When an alternative backend reports a failure, Hypothesis first replays it under the standard Hypothesis backend to check for flakiness. If the failure does not reproduce, Hypothesis raises this exception.

class hypothesis.errors.InvalidArgument

Used to indicate that the arguments to a Hypothesis function were in some manner incorrect.

class hypothesis.errors.ResolutionFailed

Hypothesis had to resolve a type to a strategy, but this failed.

Type inference is best-effort, so this only happens when an annotation exists but could not be resolved for a required argument to the target of builds(), or where the user passed ....

class hypothesis.errors.Unsatisfiable

We ran out of time or examples before we could find enough examples which satisfy the assumptions of this hypothesis.

This could be because the function is too slow. If so, try upping the timeout. It could also be because the function is using assume in a way that is too hard to satisfy. If so, try writing a custom strategy or using a better starting point (e.g if you are requiring a list has unique values you could instead filter out all duplicate values from the list)

class hypothesis.errors.DidNotReproduce

class hypothesis.errors.DeadlineExceeded(runtime, deadline)

Raised when an input takes too long to run, relative to the settings.deadline setting.

See also:

Hypothesis offers a number of features specific for Django testing, available in the hypothesis[django] extra <>.  This is tested against each supported series with mainstream or extended support - if you're still getting security patches, you can test with Hypothesis.

class hypothesis.extra.django.TestCase(*args, **kwargs)

Using it is quite straightforward: All you need to do is subclass hypothesis.extra.django.TestCase or hypothesis.extra.django.SimpleTestCase or hypothesis.extra.django.TransactionTestCase or LiveServerTestCase or StaticLiveServerTestCase and you can use @given as normal, and the transactions will be per example rather than per test function as they would be if you used @given with a normal django test suite (this is important because your test function will be called multiple times and you don't want them to interfere with each other). Test cases on these classes that do not use @given will be run as normal for django:django.test.TestCase or django:django.test.TransactionTestCase.

class hypothesis.extra.django.SimpleTestCase(*args, **kwargs)

class hypothesis.extra.django.TransactionTestCase(*args, **kwargs)

class hypothesis.extra.django.LiveServerTestCase(*args, **kwargs)

class hypothesis.extra.django.StaticLiveServerTestCase(*args, **kwargs)

We recommend avoiding TransactionTestCase unless you really have to run each test case in a database transaction. Because Hypothesis runs this in a loop, the performance problems django:django.test.TransactionTestCase normally has are significantly exacerbated and your tests will be really slow. If you are using TransactionTestCase, you may need to use @settings(suppress_health_check=[HealthCheck.too_slow]) to avoid a HealthCheck error due to slow example generation.

Having set up a test class, you can now pass @given a strategy for Django models with from_model() <#hypothesis.extra.django.from_model>. For example, using the trivial django project we have for testing <https://github.com/HypothesisWorks/hypothesis/blob/master/hypothesis-python/tests/django/toystore/models.py>:

>>> from hypothesis.extra.django import from_model
>>> from toystore.models import Customer
>>> c = from_model(Customer).example()
>>> c
<Customer: Customer object>
>>> c.email
'jaime.urbina@gmail.com'
>>> c.name
'\U00109d3d\U000e07be\U000165f8\U0003fabf\U000c12cd\U000f1910\U00059f12\U000519b0\U0003fabf\U000f1910\U000423fb\U000423fb\U00059f12\U000e07be\U000c12cd\U000e07be\U000519b0\U000165f8\U0003fabf\U0007bc31'
>>> c.age
-873375803

Hypothesis has just created this with whatever the relevant type of data is.

Obviously the customer's age is implausible, which is only possible because we have not used (eg) MinValueValidator to set the valid range for this field (or used a PositiveSmallIntegerField, which would only need a maximum value validator).

If you do have validators attached, Hypothesis will only generate examples that pass validation.  Sometimes that will mean that we fail a HealthCheck because of the filtering, so let's explicitly pass a strategy to skip validation at the strategy level:

>>> from hypothesis.strategies import integers
>>> c = from_model(Customer, age=integers(min_value=0, max_value=120)).example()
>>> c
<Customer: Customer object>
>>> c.age
5

If you have a custom Django field type you can register it with Hypothesis's model deriving functionality by registering a default strategy for it:

>>> from toystore.models import CustomishField, Customish
>>> from_model(Customish).example()
hypothesis.errors.InvalidArgument: Missing arguments for mandatory field
    customish for model Customish
>>> from hypothesis.extra.django import register_field_strategy
>>> from hypothesis.strategies import just
>>> register_field_strategy(CustomishField, just("hi"))
>>> x = from_model(Customish).example()
>>> x.customish
'hi'

Note that this mapping is on exact type. Subtypes will not inherit it.

For the moment there's no explicit support in hypothesis-django for generating dependent models. i.e. a Company model will generate no Shops. However if you want to generate some dependent models as well, you can emulate this by using the .flatmap() <#hypothesis.strategies.SearchStrategy.flatmap> function as follows:

from hypothesis.strategies import just, lists

def generate_with_shops(company):
    return lists(from_model(Shop, company=just(company))).map(lambda _: company)

company_with_shops_strategy = from_model(Company).flatmap(generate_with_shops)

Let's unpack what this is doing:

The way flatmap works is that we draw a value from the original strategy, then apply a function to it which gives us a new strategy. We then draw a value from that strategy. So in this case we're first drawing a company, and then we're drawing a list of shops belonging to that company: The just() <#hypothesis.strategies.just> strategy is a strategy such that drawing it always produces the individual value, so from_model(Shop, company=just(company)) is a strategy that generates a Shop belonging to the original company.

So the following code would give us a list of shops all belonging to the same company:

from_model(Company).flatmap(lambda c: lists(from_model(Shop, company=just(c))))

The only difference from this and the above is that we want the company, not the shops. This is where the inner map comes in. We build the list of shops and then throw it away, instead returning the company we started for. This works because the models that Hypothesis generates are saved in the database, so we're essentially running the inner strategy purely for the side effect of creating those children in the database.

If your model includes a custom primary key that you want to generate using a strategy (rather than a default auto-increment primary key) then Hypothesis has to deal with the possibility of a duplicate primary key.

If a model strategy generates a value for the primary key field, Hypothesis will create the model instance with update_or_create(), overwriting any existing instance in the database for this test case with the same primary key.

Django forms feature the MultiValueField which allows for several fields to be combined under a single named field, the default example of this is the SplitDateTimeField.

class CustomerForm(forms.Form):
    name = forms.CharField()
    birth_date_time = forms.SplitDateTimeField()

from_form() <#hypothesis.extra.django.from_form> supports MultiValueField subclasses directly, however if you want to define your own strategy be forewarned that Django binds data for a MultiValueField in a peculiar way. Specifically each sub-field is expected to have its own entry in data addressed by the field name (e.g. birth_date_time) and the index of the sub-field within the MultiValueField, so form data for the example above might look like this:

{
    "name": "Samuel John",
    "birth_date_time_0": "2018-05-19",  # the date, as the first sub-field
    "birth_date_time_1": "15:18:00",  # the time, as the second sub-field
}

Thus, if you want to define your own strategies for such a field you must address your sub-fields appropriately:

from_form(CustomerForm, birth_date_time_0=just("2018-05-19"))

hypothesis.core.HypothesisHandle.fuzz_one_input = <property object>

Run the test as a fuzz target, driven with the buffer of bytes.

Depending on the passed buffer one of three things will happen:

• If the bytestring was invalid, for example because it was too short or was filtered out by assume() or .filter() <#hypothesis.strategies.SearchStrategy.filter>, fuzz_one_input() returns None.
• If the bytestring was valid and the test passed, fuzz_one_input() returns a canonicalised and pruned bytestring which will replay that test case. This is provided as an option to improve the performance of mutating fuzzers, but can safely be ignored.
• If the test failed, i.e. raised an exception, fuzz_one_input() will add the pruned buffer to the Hypothesis example database and then re-raise that exception.  All you need to do to reproduce, minimize, and de-duplicate all the failures found via fuzzing is run your test suite!

To reduce the performance impact of database writes, fuzz_one_input() only records failing inputs which would be valid shrinks for a known failure - meaning writes are somewhere between constant and log(N) rather than linear in runtime.  However, this tracking only works within a persistent fuzzing process; for forkserver fuzzers we recommend database=None for the main run, and then replaying with a database enabled if you need to analyse failures.

Note that the interpretation of both input and output bytestrings is specific to the exact version of Hypothesis you are using and the strategies given to the test, just like the database and @reproduce_failure.

fuzz_one_input() uses just enough of Hypothesis' internals to drive your test function with a bytestring, and most settings therefore have no effect in this mode.  We recommend running your tests the usual way before fuzzing to get the benefits of health checks, as well as afterwards to replay, shrink, deduplicate, and report whatever errors were discovered.

• settings.database is used by fuzz_one_input() - adding failures to the database to be replayed when you next run your tests is our preferred reporting mechanism and response to the 'fuzzer taming' problem <https://blog.regehr.org/archives/925>.
• settings.verbosity and settings.stateful_step_count work as usual.
• The deadline, derandomize, max_examples, phases, print_blob, report_multiple_bugs, and suppress_health_check settings do not affect fuzz_one_input().

@given(st.text())
def test_foo(s): ...

# This is a traditional fuzz target - call it with a bytestring,
# or a binary IO object, and it runs the test once.
fuzz_target = test_foo.hypothesis.fuzz_one_input

# For example:
fuzz_target(b"\x00\x00\x00\x00\x00\x00\x00\x00")
fuzz_target(io.BytesIO(b"\x01"))

Tip:

If you expect to discover many failures while using fuzz_one_input(), consider wrapping your database with BackgroundWriteDatabase, for low-overhead writes of failures.

Tip:

Want an integrated workflow for your team's local tests, CI, and continuous fuzzing?
Use HypoFuzz <https://hypofuzz.com/> to fuzz your whole test suite, and find more bugs with the same tests!

See also:

Hypothesis provides you with a hook that lets you control how it runs examples.

This lets you do things like set up and tear down around each example, run examples in a subprocess, transform coroutine tests into normal tests, etc. For example, TransactionTestCase in the Django extra runs each example in a separate database transaction.

The way this works is by introducing the concept of an executor. An executor is essentially a function that takes a block of code and run it. The default executor is:

def default_executor(function):
    return function()

You define executors by defining a method execute_example on a class. Any test methods on that class with @given used on them will use self.execute_example as an executor with which to run tests. For example, the following executor runs all its code twice:

from unittest import TestCase

class TestTryReallyHard(TestCase):
    @given(integers())
    def test_something(self, i):
        perform_some_unreliable_operation(i)

    def execute_example(self, f):
        f()
        return f()

Note: The functions you use in map, etc. will run inside the executor. i.e. they will not be called until you invoke the function passed to execute_example.

An executor must be able to handle being passed a function which returns None, otherwise it won't be able to run normal test cases. So for example the following executor is invalid:

from unittest import TestCase

class TestRunTwice(TestCase):
    def execute_example(self, f):
        return f()()

and should be rewritten as:

from unittest import TestCase

class TestRunTwice(TestCase):
    def execute_example(self, f):
        result = f()
        if callable(result):
            result = result()
        return result

An alternative hook is provided for use by test runner extensions such as pytest-trio <https://pypi.org/project/pytest-trio/>, which cannot use the execute_example method. This is not recommended for end-users - it is better to write a complete test function directly, perhaps by using a decorator to perform the same transformation before applying @given.

@given(x=integers())
@pytest.mark.trio
async def test(x): ...

# Illustrative code, inside the pytest-trio plugin
test.hypothesis.inner_test = lambda x: trio.run(test, x)

For authors of test runners however, assigning to the inner_test attribute of the hypothesis attribute of the test will replace the interior test.

Note:

If the end user has also specified a custom executor using the execute_example method, it - and all other execution-time logic - will be applied to the new inner test assigned by the test runner.

hypothesis.is_hypothesis_test(f)

Returns True if f represents a test function that has been defined with Hypothesis. This is true for:

• Functions decorated with @given
• The runTest method of stateful tests

For example:

@given(st.integers())
def f(n): ...

class MyStateMachine(RuleBasedStateMachine): ...

assert is_hypothesis_test(f)
assert is_hypothesis_test(MyStateMachine.TestCase().runTest)

See also:

See also the Detect Hypothesis tests <> how-to.

hypothesis.currently_in_test_context()

Return True if the calling code is currently running inside an @given or stateful test, and False otherwise.

This is useful for third-party integrations and assertion helpers which may be called from either traditional or property-based tests, and can only use e.g. assume() or target() in the latter case.

Strategies are the way Hypothesis describes the values for @given <#hypothesis.given> to generate.  For instance, passing the strategy st.lists(st.integers(), min_size=1) to @given <#hypothesis.given> tells Hypothesis to generate lists of integers with at least one element.

This reference page lists all of Hypothesis' first-party functions which return a strategy. There are also many provided by third-party libraries <>.  Note that we often say "strategy" when we mean "function returning a strategy"; it's usually clear from context which one we mean.

Strategies can be passed to other strategies as arguments, combined using combinator strategies, or modified using .filter(), .map(), or .flatmap().

hypothesis.strategies.none()

Return a strategy which only generates None.

Examples from this strategy do not shrink (because there is only one).

hypothesis.strategies.nothing()

This strategy never successfully draws a value and will always reject on an attempt to draw.

Examples from this strategy do not shrink (because there are none).

hypothesis.strategies.just(value)

Return a strategy which only generates value.

Note: value is not copied. Be wary of using mutable values.

If value is the result of a callable, you can use builds(callable) instead of just(callable()) to get a fresh value each time.

Examples from this strategy do not shrink (because there is only one).

hypothesis.strategies.booleans()

Returns a strategy which generates instances of python:bool.

Examples from this strategy will shrink towards False (i.e. shrinking will replace True with False where possible).

See also:

hypothesis.strategies.integers(min_value=None, max_value=None)

Returns a strategy which generates integers.

If min_value is not None then all values will be >= min_value. If max_value is not None then all values will be <= max_value

Examples from this strategy will shrink towards zero, and negative values will also shrink towards positive (i.e. -n may be replaced by +n).

hypothesis.strategies.floats(min_value=None, max_value=None, *, allow_nan=None, allow_infinity=None, allow_subnormal=None, width=64, exclude_min=False, exclude_max=False)

Returns a strategy which generates floats.

• If min_value is not None, all values will be >= min_value (or > min_value if exclude_min).
• If max_value is not None, all values will be <= max_value (or < max_value if exclude_max).
• If min_value or max_value is not None, it is an error to enable allow_nan.
• If both min_value and max_value are not None, it is an error to enable allow_infinity.
• If inferred values range does not include subnormal values, it is an error to enable allow_subnormal.

Where not explicitly ruled out by the bounds, subnormals <https://en.wikipedia.org/wiki/Subnormal_number>, infinities, and NaNs are possible values generated by this strategy.

The width argument specifies the maximum number of bits of precision required to represent the generated float. Valid values are 16, 32, or 64. Passing width=32 will still use the builtin 64-bit python:float class, but always for values which can be exactly represented as a 32-bit float.

The exclude_min and exclude_max argument can be used to generate numbers from open or half-open intervals, by excluding the respective endpoints. Excluding either signed zero will also exclude the other. Attempting to exclude an endpoint which is None will raise an error; use allow_infinity=False to generate finite floats.  You can however use e.g. min_value=-math.inf, exclude_min=True to exclude only one infinite endpoint.

Examples from this strategy have a complicated and hard to explain shrinking behaviour, but it tries to improve "human readability". Finite numbers will be preferred to infinity and infinity will be preferred to NaN.

hypothesis.strategies.complex_numbers(*, min_magnitude=0, max_magnitude=None, allow_infinity=None, allow_nan=None, allow_subnormal=True, width=128)

Returns a strategy that generates python:complex numbers.

This strategy draws complex numbers with constrained magnitudes. The min_magnitude and max_magnitude parameters should be non-negative Real numbers; a value of None corresponds an infinite upper bound.

If min_magnitude is nonzero or max_magnitude is finite, it is an error to enable allow_nan.  If max_magnitude is finite, it is an error to enable allow_infinity.

allow_infinity, allow_nan, and allow_subnormal are applied to each part of the complex number separately, as for floats().

The magnitude constraints are respected up to a relative error of (around) floating-point epsilon, due to implementation via the system sqrt function.

The width argument specifies the maximum number of bits of precision required to represent the entire generated complex number. Valid values are 32, 64 or 128, which correspond to the real and imaginary components each having width 16, 32 or 64, respectively. Passing width=64 will still use the builtin 128-bit python:complex class, but always for values which can be exactly represented as two 32-bit floats.

Examples from this strategy shrink by shrinking their real and imaginary parts, as floats().

If you need to generate complex numbers with particular real and imaginary parts or relationships between parts, consider using builds(complex, ...) or @composite respectively.

hypothesis.strategies.decimals(min_value=None, max_value=None, *, allow_nan=None, allow_infinity=None, places=None)

Generates instances of python:decimal.Decimal, which may be:

• A finite rational number, between min_value and max_value.
• Not a Number, if allow_nan is True.  None means "allow NaN, unless min_value and max_value are not None".
• Positive or negative infinity, if max_value and min_value respectively are None, and allow_infinity is not False.  None means "allow infinity, unless excluded by the min and max values".

Note that where floats have one NaN value, Decimals have four: signed, and either quiet or signalling.  See the decimal module docs <https://docs.python.org/3/library/decimal.html#special-values> for more information on special values.

If places is not None, all finite values drawn from the strategy will have that number of digits after the decimal place.

Examples from this strategy do not have a well defined shrink order but try to maximize human readability when shrinking.

hypothesis.strategies.fractions(min_value=None, max_value=None, *, max_denominator=None)

Returns a strategy which generates Fractions.

If min_value is not None then all generated values are no less than min_value.  If max_value is not None then all generated values are no greater than max_value.  min_value and max_value may be anything accepted by the Fraction constructor.

If max_denominator is not None then the denominator of any generated values is no greater than max_denominator. Note that max_denominator must be None or a positive integer.

Examples from this strategy shrink towards smaller denominators, then closer to zero.

Note:

The uuids() and ip_addresses() strategies generate instances of UUID and IPAddress respectively. You can generate corresponding string values by using .map(), such as st.uuids().map(str).

hypothesis.strategies.text(alphabet=characters(codec='utf-8'), *, min_size=0, max_size=None)

Generates strings with characters drawn from alphabet, which should be a collection of length one strings or a strategy generating such strings.

The default alphabet strategy can generate the full unicode range but excludes surrogate characters because they are invalid in the UTF-8 encoding.  You can use characters() without arguments to find surrogate-related bugs such as bpo-34454 <https://bugs.python.org/issue34454>.

min_size and max_size have the usual interpretations. Note that Python measures string length by counting codepoints: U+00C5 Å is a single character, while U+0041 U+030A Å is two - the A, and a combining ring above.

Examples from this strategy shrink towards shorter strings, and with the characters in the text shrinking as per the alphabet strategy. This strategy does not normalize() examples, so generated strings may be in any or none of the 'normal forms'.

hypothesis.strategies.characters(*, codec=None, min_codepoint=None, max_codepoint=None, categories=None, exclude_categories=None, exclude_characters=None, include_characters=None)

Generates characters, length-one python:strings, following specified filtering rules.

• When no filtering rules are specified, any character can be produced.
• If min_codepoint or max_codepoint is specified, then only characters having a codepoint in that range will be produced.
• If categories is specified, then only characters from those Unicode categories will be produced. This is a further restriction, characters must also satisfy min_codepoint and max_codepoint.
• If exclude_categories is specified, then any character from those categories will not be produced.  You must not pass both categories and exclude_categories; these arguments are alternative ways to specify exactly the same thing.
• If include_characters is specified, then any additional characters in that list will also be produced.
• If exclude_characters is specified, then any characters in that list will be not be produced. Any overlap between include_characters and exclude_characters will raise an exception.
• If codec is specified, only characters in the specified codec encodings <https://docs.python.org/3/library/codecs.html#encodings-and-unicode> will be produced.

The _codepoint arguments must be integers between zero and python:sys.maxunicode.  The _characters arguments must be collections of length-one unicode strings, such as a unicode string.

The _categories arguments must be used to specify either the one-letter Unicode major category or the two-letter Unicode general category <https://en.wikipedia.org/wiki/Unicode_character_property>.  For example, ('Nd', 'Lu') signifies "Number, decimal digit" and "Letter, uppercase".  A single letter ('major category') can be given to match all corresponding categories, for example 'P' for characters in any punctuation category.

We allow codecs from the codecs module and their aliases, platform specific and user-registered codecs if they are available, and python-specific text encodings <https://docs.python.org/3/library/codecs.html#python-specific-encodings> (but not text or binary transforms). include_characters which cannot be encoded using this codec will raise an exception.  If non-encodable codepoints or categories are explicitly allowed, the codec argument will exclude them without raising an exception.

Examples from this strategy shrink towards the codepoint for '0', or the first allowable codepoint after it if '0' is excluded.

hypothesis.strategies.from_regex(regex, *, fullmatch=False, alphabet=None)

Generates strings that contain a match for the given regex (i.e. ones for which python:re.search() will return a non-None result).

regex may be a pattern or compiled regex. Both byte-strings and unicode strings are supported, and will generate examples of the same type.

You can use regex flags such as python:re.IGNORECASE or python:re.DOTALL to control generation. Flags can be passed either in compiled regex or inside the pattern with a (?iLmsux) group.

Some regular expressions are only partly supported - the underlying strategy checks local matching and relies on filtering to resolve context-dependent expressions.  Using too many of these constructs may cause health-check errors as too many examples are filtered out. This mainly includes (positive or negative) lookahead and lookbehind groups.

If you want the generated string to match the whole regex you should use boundary markers. So e.g. r"\A.\Z" will return a single character string, while "." will return any string, and r"\A.$" will return a single character optionally followed by a "\n". Alternatively, passing fullmatch=True will ensure that the whole string is a match, as if you had used the \A and \Z markers.

The alphabet= argument constrains the characters in the generated string, as for text(), and is only supported for unicode strings.

Examples from this strategy shrink towards shorter strings and lower character values, with exact behaviour that may depend on the pattern.

hypothesis.strategies.binary(*, min_size=0, max_size=None)

Generates python:bytes.

The generated python:bytes will have a length of at least min_size and at most max_size.  If max_size is None there is no upper limit.

Examples from this strategy shrink towards smaller strings and lower byte values.

hypothesis.strategies.emails(*, domains=domains())

A strategy for generating email addresses as unicode strings. The address format is specified in RFC 5322 Section 3.4.1 <https://datatracker.ietf.org/doc/html/rfc5322.html#section-3.4.1>. Values shrink towards shorter local-parts and host domains.

If domains is given then it must be a strategy that generates domain names for the emails, defaulting to domains().

This strategy is useful for generating "user data" for tests, as mishandling of email addresses is a common source of bugs.

hypothesis.provisional.domains(*, max_length=255, max_element_length=63)

Generate RFC 1035 <https://datatracker.ietf.org/doc/html/rfc1035.html> compliant fully qualified domain names.

Warning:

The domains() strategy is provisional. Its interface may be changed in a minor release, without being subject to our deprecation policy <#deprecation-policy>. That said, we expect it to be relatively stable.

hypothesis.provisional.urls()

A strategy for RFC 3986 <https://datatracker.ietf.org/doc/html/rfc3986.html>, generating http/https URLs.

The generated URLs could, at least in theory, be passed to an HTTP client and fetched.

Warning:

The urls() strategy is provisional. Its interface may be changed in a minor release, without being subject to our deprecation policy <#deprecation-policy>. That said, we expect it to be relatively stable.

hypothesis.strategies.lists(elements, *, min_size=0, max_size=None, unique_by=None, unique=False)

Returns a list containing values drawn from elements with length in the interval [min_size, max_size] (no bounds in that direction if these are None). If max_size is 0, only the empty list will be drawn.

If unique is True (or something that evaluates to True), we compare direct object equality, as if unique_by was lambda x: x. This comparison only works for hashable types.

If unique_by is not None it must be a callable or tuple of callables returning a hashable type when given a value drawn from elements. The resulting list will satisfy the condition that for i != j, unique_by(result[i]) != unique_by(result[j]).

If unique_by is a tuple of callables the uniqueness will be respective to each callable.

For example, the following will produce two columns of integers with both columns being unique respectively.

>>> twoints = st.tuples(st.integers(), st.integers())
>>> st.lists(twoints, unique_by=(lambda x: x[0], lambda x: x[1]))

Examples from this strategy shrink by trying to remove elements from the list, and by shrinking each individual element of the list.

hypothesis.strategies.tuples(*args)

Return a strategy which generates a tuple of the same length as args by generating the value at index i from args[i].

e.g. tuples(integers(), integers()) would generate a tuple of length two with both values an integer.

Examples from this strategy shrink by shrinking their component parts.

hypothesis.strategies.sets(elements, *, min_size=0, max_size=None)

This has the same behaviour as lists, but returns sets instead.

Note that Hypothesis cannot tell if values are drawn from elements are hashable until running the test, so you can define a strategy for sets of an unhashable type but it will fail at test time.

Examples from this strategy shrink by trying to remove elements from the set, and by shrinking each individual element of the set.

hypothesis.strategies.frozensets(elements, *, min_size=0, max_size=None)

This is identical to the sets function but instead returns frozensets.

hypothesis.strategies.dictionaries(keys, values, *, dict_class=<class 'dict'>, min_size=0, max_size=None)

Generates dictionaries of type dict_class with keys drawn from the keys argument and values drawn from the values argument.

The size parameters have the same interpretation as for lists().

Examples from this strategy shrink by trying to remove keys from the generated dictionary, and by shrinking each generated key and value.

hypothesis.strategies.fixed_dictionaries(mapping, *, optional=None)

Generates a dictionary of the same type as mapping with a fixed set of keys mapping to strategies. mapping must be a dict subclass.

Generated values have all keys present in mapping, in iteration order, with the corresponding values drawn from mapping[key].

If optional is passed, the generated value may or may not contain each key from optional and a value drawn from the corresponding strategy. Generated values may contain optional keys in an arbitrary order.

Examples from this strategy shrink by shrinking each individual value in the generated dictionary, and omitting optional key-value pairs.

hypothesis.strategies.iterables(elements, *, min_size=0, max_size=None, unique_by=None, unique=False)

This has the same behaviour as lists, but returns iterables instead.

Some iterables cannot be indexed (e.g. sets) and some do not have a fixed length (e.g. generators). This strategy produces iterators, which cannot be indexed and do not have a fixed length. This ensures that you do not accidentally depend on sequence behaviour.

hypothesis.strategies.dates(min_value=datetime.date.min, max_value=datetime.date.max)

A strategy for dates between min_value and max_value.

Examples from this strategy shrink towards January 1st 2000.

hypothesis.strategies.times(min_value=datetime.time.min, max_value=datetime.time.max, *, timezones=none())

A strategy for times between min_value and max_value.

The timezones argument is handled as for datetimes().

Examples from this strategy shrink towards midnight, with the timezone component shrinking as for the strategy that provided it.

hypothesis.strategies.datetimes(min_value=datetime.datetime.min, max_value=datetime.datetime.max, *, timezones=none(), allow_imaginary=True)

A strategy for generating datetimes, which may be timezone-aware.

This strategy works by drawing a naive datetime between min_value and max_value, which must both be naive (have no timezone).

timezones must be a strategy that generates either None, for naive datetimes, or tzinfo objects for 'aware' datetimes. You can construct your own, though we recommend using one of these built-in strategies:

• with the standard library: hypothesis.strategies.timezones();
• with dateutil <https://pypi.org/project/python-dateutil/>: hypothesis.extra.dateutil.timezones(); or
• with pytz <https://pypi.org/project/pytz/>: hypothesis.extra.pytz.timezones().

You may pass allow_imaginary=False to filter out "imaginary" datetimes which did not (or will not) occur due to daylight savings, leap seconds, timezone and calendar adjustments, etc.  Imaginary datetimes are allowed by default, because malformed timestamps are a common source of bugs.

Examples from this strategy shrink towards midnight on January 1st 2000, local time.

hypothesis.strategies.timezones(*, no_cache=False)

A strategy for python:zoneinfo.ZoneInfo objects.

If no_cache=True, the generated instances are constructed using ZoneInfo.no_cache instead of the usual constructor.  This may change the semantics of your datetimes in surprising ways, so only use it if you know that you need to!

Note:

The tzdata package is required on Windows <https://docs.python.org/3/library/zoneinfo.html#data-sources>. pip install hypothesis[zoneinfo] installs it, if and only if needed.

hypothesis.strategies.timezone_keys(*, allow_prefix=True)

A strategy for IANA timezone names <https://en.wikipedia.org/wiki/List_of_tz_database_time_zones>.

As well as timezone names like "UTC", "Australia/Sydney", or "America/New_York", this strategy can generate:

• Aliases such as "Antarctica/McMurdo", which links to "Pacific/Auckland".
• Deprecated names such as "Antarctica/South_Pole", which also links to "Pacific/Auckland".  Note that most but not all deprecated timezone names are also aliases.
• Timezone names with the "posix/" or "right/" prefixes, unless allow_prefix=False.

These strings are provided separately from Tzinfo objects - such as ZoneInfo instances from the timezones() strategy - to facilitate testing of timezone logic without needing workarounds to access non-canonical names.

Note:

The tzdata package is required on Windows <https://docs.python.org/3/library/zoneinfo.html#data-sources>. pip install hypothesis[zoneinfo] installs it, if and only if needed.

On Windows, you may need to access IANA timezone data via the tzdata <https://pypi.org/project/tzdata/> package.  For non-IANA timezones, such as Windows-native names or GNU TZ strings, we recommend using sampled_from() with the dateutil <https://pypi.org/project/python-dateutil/> package, e.g. dateutil:dateutil.tz.tzwin.list().

hypothesis.strategies.timedeltas(min_value=datetime.timedelta.min, max_value=datetime.timedelta.max)

A strategy for timedeltas between min_value and max_value.

Examples from this strategy shrink towards zero.

hypothesis.strategies.recursive(base, extend, *, min_leaves=None, max_leaves=100)

base: A strategy to start from.

extend: A function which takes a strategy and returns a new strategy.

min_leaves: The minimum number of elements to be drawn from base on a given run.

max_leaves: The maximum number of elements to be drawn from base on a given run.

This returns a strategy S such that S = extend(base | S). That is, values may be drawn from base, or from any strategy reachable by mixing applications of | and extend.

An example may clarify: recursive(booleans(), lists) would return a strategy that may return arbitrarily nested and mixed lists of booleans. So e.g. False, [True], [False, []], and [[[[True]]]] are all valid values to be drawn from that strategy.

Examples from this strategy shrink by trying to reduce the amount of recursion and by shrinking according to the shrinking behaviour of base and the result of extend.

hypothesis.strategies.deferred(definition)

A deferred strategy allows you to write a strategy that references other strategies that have not yet been defined. This allows for the easy definition of recursive and mutually recursive strategies.

The definition argument should be a zero-argument function that returns a strategy. It will be evaluated the first time the strategy is used to produce an example.

Example usage:

>>> import hypothesis.strategies as st
>>> x = st.deferred(lambda: st.booleans() | st.tuples(x, x))
>>> x.example()
(((False, (True, True)), (False, True)), (True, True))
>>> x.example()
True

Mutual recursion also works fine:

>>> a = st.deferred(lambda: st.booleans() | b)
>>> b = st.deferred(lambda: st.tuples(a, a))
>>> a.example()
True
>>> b.example()
(False, (False, ((False, True), False)))

Examples from this strategy shrink as they normally would from the strategy returned by the definition.

hypothesis.strategies.randoms(*, note_method_calls=False, use_true_random=False)

Generates instances of random.Random. The generated Random instances are of a special HypothesisRandom subclass.

• If note_method_calls is set to True, Hypothesis will print the randomly drawn values in any falsifying test case. This can be helpful for debugging the behaviour of randomized algorithms.
• If use_true_random is set to True then values will be drawn from their usual distribution, otherwise they will actually be Hypothesis generated values (and will be shrunk accordingly for any failing test case). Setting use_true_random=False will tend to expose bugs that would occur with very low probability when it is set to True, and this flag should only be set to True when your code relies on the distribution of values for correctness.

For managing global state, see the random_module() strategy and register_random() function.

hypothesis.strategies.random_module()

Hypothesis always seeds global PRNGs before running a test, and restores the previous state afterwards.

If having a fixed seed would unacceptably weaken your tests, and you cannot use a random.Random instance provided by randoms(), this strategy calls python:random.seed() with an arbitrary integer and passes you an opaque object whose repr displays the seed value for debugging. If numpy.random is available, that state is also managed, as is anything managed by hypothesis.register_random().

Examples from these strategy shrink to seeds closer to zero.

hypothesis.register_random(r)

Register (a weakref to) the given Random-like instance for management by Hypothesis.

You can pass instances of structural subtypes of random.Random (i.e., objects with seed, getstate, and setstate methods) to register_random(r) to have their states seeded and restored in the same way as the global PRNGs from the random and numpy.random modules.

All global PRNGs, from e.g. simulation or scheduling frameworks, should be registered to prevent flaky tests. Hypothesis will ensure that the PRNG state is consistent for all test runs, always seeding them to zero and restoring the previous state after the test, or, reproducibly varied if you choose to use the random_module() strategy.

register_random only makes weakrefs <https://docs.python.org/3/library/weakref.html#module-weakref> to r, thus r will only be managed by Hypothesis as long as it has active references elsewhere at runtime. The pattern register_random(MyRandom()) will raise a ReferenceError to help protect users from this issue. This check does not occur for the PyPy interpreter. See the following example for an illustration of this issue

def my_BROKEN_hook():
    r = MyRandomLike()

    # `r` will be garbage collected after the hook resolved
    # and Hypothesis will 'forget' that it was registered
    register_random(r)  # Hypothesis will emit a warning


rng = MyRandomLike()


def my_WORKING_hook():
    register_random(rng)

hypothesis.strategies.one_of(*args)

Return a strategy which generates values from any of the argument strategies.

This may be called with one iterable argument instead of multiple strategy arguments, in which case one_of(x) and one_of(*x) are equivalent.

Examples from this strategy will generally shrink to ones that come from strategies earlier in the list, then shrink according to behaviour of the strategy that produced them. In order to get good shrinking behaviour, try to put simpler strategies first. e.g. one_of(none(), text()) is better than one_of(text(), none()).

This is especially important when using recursive strategies. e.g. x = st.deferred(lambda: st.none() | st.tuples(x, x)) will shrink well, but x = st.deferred(lambda: st.tuples(x, x) | st.none()) will shrink very badly indeed.

hypothesis.strategies.builds(target, /, *args, **kwargs)

Generates values by drawing from args and kwargs and passing them to the callable (provided as the first positional argument) in the appropriate argument position.

e.g. builds(target, integers(), flag=booleans()) would draw an integer i and a boolean b and call target(i, flag=b).

If the callable has type annotations, they will be used to infer a strategy for required arguments that were not passed to builds.  You can also tell builds to infer a strategy for an optional argument by passing ... (python:Ellipsis) as a keyword argument to builds, instead of a strategy for that argument to the callable.

If the callable is a class defined with attrs <https://pypi.org/project/attrs/>, missing required arguments will be inferred from the attribute on a best-effort basis, e.g. by checking attrs standard validators. Dataclasses are handled natively by the inference from type hints.

Examples from this strategy shrink by shrinking the argument values to the callable.

hypothesis.strategies.composite(f)

Defines a strategy that is built out of potentially arbitrarily many other strategies.

@composite provides a callable draw as the first parameter to the decorated function, which can be used to dynamically draw a value from any strategy. For example:

from hypothesis import strategies as st, given

@st.composite
def values(draw):
    n1 = draw(st.integers())
    n2 = draw(st.integers(min_value=n1))
    return (n1, n2)

@given(values())
def f(value):
    (n1, n2) = value
    assert n1 <= n2

@composite cannot mix test code and generation code. If you need that, use data().

If @composite is used to decorate a method or classmethod, the draw argument must come before self or cls. While we therefore recommend writing strategies as standalone functions and using register_type_strategy() to associate them with a class, methods are supported and the @composite decorator may be applied either before or after @classmethod or @staticmethod. See issue #2578 <https://github.com/HypothesisWorks/hypothesis/issues/2578> and pull request #2634 <https://github.com/HypothesisWorks/hypothesis/pull/2634> for more details.

Examples from this strategy shrink by shrinking the output of each draw call.

hypothesis.strategies.data()

Provides an object data with a data.draw function which acts like the draw callable provided by @composite, in that it can be used to dynamically draw values from strategies. data() is more powerful than @composite, because it allows you to mix generation and test code.

Here's an example of dynamically generating values using data():

from hypothesis import strategies as st, given

@given(st.data())
def test_values(data):
    n1 = data.draw(st.integers())
    n2 = data.draw(st.integers(min_value=n1))
    assert n1 + 1 <= n2

If the test fails, each draw will be printed with the falsifying example. e.g. the above is wrong (it has a boundary condition error), so will print:

Falsifying example: test_values(data=data(...))
Draw 1: 0
Draw 2: 0

Optionally, you can provide a label to identify values generated by each call to data.draw().  These labels can be used to identify values in the output of a falsifying example.

For instance:

@given(st.data())
def test_draw_sequentially(data):
    x = data.draw(st.integers(), label="First number")
    y = data.draw(st.integers(min_value=x), label="Second number")
    assert x < y

will produce:

Falsifying example: test_draw_sequentially(data=data(...))
Draw 1 (First number): 0
Draw 2 (Second number): 0

Examples from this strategy shrink by shrinking the output of each draw call.

hypothesis.strategies.from_type(thing)

Looks up the appropriate search strategy for the given type.

from_type() is used internally to fill in missing arguments to builds() and can be used interactively to explore what strategies are available or to debug type resolution.

You can use register_type_strategy() to handle your custom types, or to globally redefine certain strategies - for example excluding NaN from floats, or use timezone-aware instead of naive time and datetime strategies.

from_type() looks up a strategy in the following order:

1. If thing is in the default lookup mapping or user-registered lookup, return the corresponding strategy.  The default lookup covers all types with Hypothesis strategies, including extras where possible.
2. If thing is from the python:typing module, return the corresponding strategy (special logic).
3. If thing has one or more subtypes in the merged lookup, return the union of the strategies for those types that are not subtypes of other elements in the lookup.
4. Finally, if thing has type annotations for all required arguments, and is not an abstract class, it is resolved via builds().
5. Because abstract types cannot be instantiated, we treat abstract types as the union of their concrete subclasses. Note that this lookup works via inheritance but not via register, so you may still need to use register_type_strategy().

There is a valuable recipe for leveraging from_type() to generate "everything except" values from a specified type. I.e.

def everything_except(excluded_types):
    return (
        from_type(type)
        .flatmap(from_type)
        .filter(lambda x: not isinstance(x, excluded_types))
    )

For example, everything_except(int) returns a strategy that can generate anything that from_type() can ever generate, except for instances of python:int, and excluding instances of types added via register_type_strategy().

This is useful when writing tests which check that invalid input is rejected in a certain way.

hypothesis.strategies.register_type_strategy(custom_type, strategy)

Add an entry to the global type-to-strategy lookup.

This lookup is used in builds() and @given <#hypothesis.given>.

builds() will be used automatically for classes with type annotations on __init__ , so you only need to register a strategy if one or more arguments need to be more tightly defined than their type-based default, or if you want to supply a strategy for an argument with a default value.

strategy may be a search strategy, or a function that takes a type and returns a strategy (useful for generic types). The function may return NotImplemented to conditionally not provide a strategy for the type (the type will still be resolved by other methods, if possible, as if the function was not registered).

Note that you may not register a parametrised generic type (such as MyCollection[int]) directly, because the resolution logic does not handle this case correctly.  Instead, you may register a function for MyCollection and inspect the type parameters within that function <https://stackoverflow.com/q/48572831>.

hypothesis.strategies.runner(*, default=not_set)

A strategy for getting "the current test runner", whatever that may be. The exact meaning depends on the entry point, but it will usually be the associated 'self' value for it.

If you are using this in a rule for stateful testing, this strategy will return the instance of the RuleBasedStateMachine <#hypothesis.stateful.RuleBasedStateMachine> that the rule is running for.

If there is no current test runner and a default is provided, return that default. If no default is provided, raises InvalidArgument.

Examples from this strategy do not shrink (because there is only one).

hypothesis.strategies.shared(base, *, key=None)

Returns a strategy that draws a single shared value per run, drawn from base. Any two shared instances with the same key will share the same value, otherwise the identity of this strategy will be used. That is:

>>> s = integers()  # or any other strategy
>>> x = shared(s)
>>> y = shared(s)

In the above x and y may draw different (or potentially the same) values. In the following they will always draw the same:

>>> x = shared(s, key="hi")
>>> y = shared(s, key="hi")

Examples from this strategy shrink as per their base strategy.

hypothesis.strategies.functions(*, like=lambda : ..., returns=..., pure=False)

A strategy for functions, which can be used in callbacks.

The generated functions will mimic the interface of like, which must be a callable (including a class, method, or function).  The return value for the function is drawn from the returns argument, which must be a strategy.  If returns is not passed, we attempt to infer a strategy from the return-type annotation if present, falling back to none().

If pure=True, all arguments passed to the generated function must be hashable, and if passed identical arguments the original return value will be returned again - not regenerated, so beware mutable values.

If pure=False, generated functions do not validate their arguments, and may return a different value if called again with the same arguments.

Generated functions can only be called within the scope of the @given which created them.

hypothesis.strategies.slices(size)

Generates slices that will select indices up to the supplied size

Generated slices will have start and stop indices that range from -size to size - 1 and will step in the appropriate direction. Slices should only produce an empty selection if the start and end are the same.

Examples from this strategy shrink toward 0 and smaller values

hypothesis.strategies.uuids(*, version=None, allow_nil=False)

Returns a strategy that generates UUIDs.

If the optional version argument is given, value is passed through to UUID and only UUIDs of that version will be generated.

If allow_nil is True, generate the nil UUID much more often. Otherwise, all returned values from this will be unique, so e.g. if you do lists(uuids()) the resulting list will never contain duplicates.

Examples from this strategy don't have any meaningful shrink order.

hypothesis.strategies.ip_addresses(*, v=None, network=None)

Generate IP addresses - v=4 for IPv4Addresses, v=6 for IPv6Addresses, or leave unspecified to allow both versions.

network may be an IPv4Network or IPv6Network, or a string representing a network such as "127.0.0.0/24" or "2001:db8::/32".  As well as generating addresses within a particular routable network, this can be used to generate addresses from a reserved range listed in the IANA <https://www.iana.org/assignments/iana-ipv4-special-registry/> registries <https://www.iana.org/assignments/iana-ipv6-special-registry/>.

If you pass both v and network, they must be for the same version.

hypothesis.strategies.sampled_from(elements)

Returns a strategy which generates any value present in elements.

Note that as with just(), values will not be copied and thus you should be careful of using mutable data.

sampled_from supports ordered collections, as well as Enum objects.  Flag objects may also generate any combination of their members.

Examples from this strategy shrink by replacing them with values earlier in the list. So e.g. sampled_from([10, 1]) will shrink by trying to replace 1 values with 10, and sampled_from([1, 10]) will shrink by trying to replace 10 values with 1.

It is an error to sample from an empty sequence, because returning nothing() makes it too easy to silently drop parts of compound strategies.  If you need that behaviour, use sampled_from(seq) if seq else nothing().

hypothesis.strategies.permutations(values)

Return a strategy which returns permutations of the ordered collection values.

Examples from this strategy shrink by trying to become closer to the original order of values.

class hypothesis.strategies.DrawFn

This type only exists so that you can write type hints for functions decorated with @composite.

def draw(strategy: SearchStrategy[Ex], label: object = None) -> Ex: ...

@composite
def list_and_index(draw: DrawFn) -> tuple[int, str]:
    i = draw(integers())  # type of `i` inferred as 'int'
    s = draw(text())  # type of `s` inferred as 'str'
    return i, s

class hypothesis.strategies.DataObject

This type only exists so that you can write type hints for tests using the data() strategy.  Do not use it directly!

draw(strategy, label=None)

Like DrawFn.

class hypothesis.strategies.SearchStrategy

A SearchStrategy tells Hypothesis how to generate that kind of input.

This class is only part of the public API for use in type annotations, so that you can write e.g. -> SearchStrategy[Foo] for your function which returns builds(Foo, ...).  Do not inherit from or directly instantiate this class.

example()

Provide an example of the sort of value that this strategy generates.

This method is designed for use in a REPL, and will raise an error if called from inside @given <#hypothesis.given> or a strategy definition.  For serious use, see @composite or data().

filter(condition)

Returns a new strategy that generates values from this strategy which satisfy the provided condition.

Note that if the condition is too hard to satisfy this might result in your tests failing with an Unsatisfiable exception. A basic version of the filtering logic would look something like:

@st.composite
def filter_like(draw, strategy, condition):
    for _ in range(3):
        value = draw(strategy)
        if condition(value):
            return value
    assume(False)

map(pack)

Returns a new strategy which generates a value from this one, and then returns pack(value).  For example, integers().map(str) could generate str(5) == "5".

flatmap(expand)

Old syntax for a special case of @composite:

@st.composite
def flatmap_like(draw, base_strategy, expand):
    value = draw(base_strategy)
    new_strategy = expand(value)
    return draw(new_strategy)

We find that the greater readability of @composite usually outweighs the verbosity, with a few exceptions for simple cases or recipes like from_type(type).flatmap(from_type) ("pick a type, get a strategy for any instance of that type, and then generate one of those").

Hypothesis offers a number of strategies for NumPy <https://numpy.org/> testing, available in the hypothesis[numpy] extra <>. It lives in the hypothesis.extra.numpy package.

The centerpiece is the arrays() strategy, which generates arrays with any dtype, shape, and contents you can specify or give a strategy for. To make this as useful as possible, strategies are provided to generate array shapes and generate all kinds of fixed-size or compound dtypes.

hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(), *, min_size=1, max_size=5, allow_subarrays=False)

Return a strategy for generating array (compound) dtypes, with members drawn from the given subtype strategy.

hypothesis.extra.numpy.array_shapes(*, min_dims=1, max_dims=None, min_side=1, max_side=None)

Return a strategy for array shapes (tuples of int >= 1).

• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to min_dims + 2.
• min_side is the smallest size that a dimension can possess.
• max_side is the largest size that a dimension can possess, defaulting to min_side + 5.

hypothesis.extra.numpy.arrays(dtype, shape, *, elements=None, fill=None, unique=False)

Returns a strategy for generating numpy:numpy.ndarrays.

• dtype may be any valid input to dtype (this includes dtype objects), or a strategy that generates such values.
• shape may be an integer >= 0, a tuple of such integers, or a strategy that generates such values.
• elements is a strategy for generating values to put in the array. If it is None a suitable value will be inferred based on the dtype, which may give any legal value (including eg NaN for floats). If a mapping, it will be passed as **kwargs to from_dtype()
• fill is a strategy that may be used to generate a single background value for the array. If None, a suitable default will be inferred based on the other arguments. If set to nothing() then filling behaviour will be disabled entirely and every element will be generated independently.
• unique specifies if the elements of the array should all be distinct from one another. Note that in this case multiple NaN values may still be allowed. If fill is also set, the only valid values for it to return are NaN values (anything for which numpy:numpy.isnan returns True. So e.g. for complex numbers nan+1j is also a valid fill). Note that if unique is set to True the generated values must be hashable.

Arrays of specified dtype and shape are generated for example like this:

>>> import numpy as np
>>> arrays(np.int8, (2, 3)).example()
array([[-8,  6,  3],
       [-6,  4,  6]], dtype=int8)
>>> arrays(np.float, 3, elements=st.floats(0, 1)).example()
array([ 0.88974794,  0.77387938,  0.1977879 ])

Array values are generated in two parts:

1. Some subset of the coordinates of the array are populated with a value drawn from the elements strategy (or its inferred form).
2. If any coordinates were not assigned in the previous step, a single value is drawn from the fill strategy and is assigned to all remaining places.

You can set fill=nothing() to disable this behaviour and draw a value for every element.

If fill=None, then it will attempt to infer the correct behaviour automatically. If unique is True, no filling will occur by default. Otherwise, if it looks safe to reuse the values of elements across multiple coordinates (this will be the case for any inferred strategy, and for most of the builtins, but is not the case for mutable values or strategies built with flatmap, map, composite, etc) then it will use the elements strategy as the fill, else it will default to having no fill.

Having a fill helps Hypothesis craft high quality examples, but its main importance is when the array generated is large: Hypothesis is primarily designed around testing small examples. If you have arrays with hundreds or more elements, having a fill value is essential if you want your tests to run in reasonable time.

hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0, max_dims=None, allow_newaxis=False, allow_ellipsis=True)

Return a strategy for basic indexes of arrays with the specified shape, which may include dimensions of size zero.

It generates tuples containing some mix of integers, python:slice objects, ... (an Ellipsis), and None. When a length-one tuple would be generated, this strategy may instead return the element which will index the first axis, e.g. 5 instead of (5,).

• shape is the shape of the array that will be indexed, as a tuple of positive integers. This must be at least two-dimensional for a tuple to be a valid index; for one-dimensional arrays use slices() instead.
• min_dims is the minimum dimensionality of the resulting array from use of the generated index. When min_dims == 0, scalars and zero-dimensional arrays are both allowed.
• max_dims is the maximum dimensionality of the resulting array, defaulting to len(shape) if not allow_newaxis else max(len(shape), min_dims) + 2.
• allow_newaxis specifies whether None is allowed in the index.
• allow_ellipsis specifies whether ... is allowed in the index.

hypothesis.extra.numpy.boolean_dtypes()

Return a strategy for boolean dtypes.

hypothesis.extra.numpy.broadcastable_shapes(shape, *, min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for shapes that are broadcast-compatible with the provided shape.

Examples from this strategy shrink towards a shape with length min_dims. The size of an aligned dimension shrinks towards size 1. The size of an unaligned dimension shrink towards min_side.

• shape is a tuple of integers.
• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to max(len(shape), min_dims) + 2.
• min_side is the smallest size that an unaligned dimension can possess.
• max_side is the largest size that an unaligned dimension can possess, defaulting to 2 plus the size of the largest aligned dimension.

The following are some examples drawn from this strategy.

>>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
[(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]

hypothesis.extra.numpy.byte_string_dtypes(*, endianness='?', min_len=1, max_len=16)

Return a strategy for generating bytestring dtypes, of various lengths and byteorder.

While Hypothesis' string strategies can generate empty strings, string dtypes with length 0 indicate that size is still to be determined, so the minimum length for string dtypes is 1.

hypothesis.extra.numpy.complex_number_dtypes(*, endianness='?', sizes=(64, 128))

Return a strategy for complex-number dtypes.

sizes is the total size in bits of a complex number, which consists of two floats.  Complex halves (a 16-bit real part) are not supported by numpy and will not be generated by this strategy.

hypothesis.extra.numpy.datetime64_dtypes(*, max_period='Y', min_period='ns', endianness='?')

Return a strategy for datetime64 dtypes, with various precisions from year to attosecond.

hypothesis.extra.numpy.floating_dtypes(*, endianness='?', sizes=(16, 32, 64))

Return a strategy for floating-point dtypes.

sizes is the size in bits of floating-point number.  Some machines support 96- or 128-bit floats, but these are not generated by default.

Larger floats (96 and 128 bit real parts) are not supported on all platforms and therefore disabled by default.  To generate these dtypes, include these values in the sizes argument.

hypothesis.extra.numpy.from_dtype(dtype, *, alphabet=None, min_size=0, max_size=None, min_value=None, max_value=None, allow_nan=None, allow_infinity=None, allow_subnormal=None, exclude_min=None, exclude_max=None, min_magnitude=0, max_magnitude=None)

Creates a strategy which can generate any value of the given dtype.

Compatible parameters are passed to the inferred strategy function while inapplicable ones are ignored. This allows you, for example, to customise the min and max values, control the length or contents of strings, or exclude non-finite numbers. This is particularly useful when kwargs are passed through from arrays() which allow a variety of numeric dtypes, as it seamlessly handles the width or representable bounds for you.

hypothesis.extra.numpy.integer_array_indices(shape, *, result_shape=array_shapes(), dtype=dtype('int64'))

Return a search strategy for tuples of integer-arrays that, when used to index into an array of shape shape, given an array whose shape was drawn from result_shape.

Examples from this strategy shrink towards the tuple of index-arrays:

len(shape) * (np.zeros(drawn_result_shape, dtype), )

• shape a tuple of integers that indicates the shape of the array, whose indices are being generated.
• result_shape a strategy for generating tuples of integers, which describe the shape of the resulting index arrays. The default is array_shapes().  The shape drawn from this strategy determines the shape of the array that will be produced when the corresponding example from integer_array_indices is used as an index.
• dtype the integer data type of the generated index-arrays. Negative integer indices can be generated if a signed integer type is specified.

Recall that an array can be indexed using a tuple of integer-arrays to access its members in an arbitrary order, producing an array with an arbitrary shape. For example:

>>> from numpy import array
>>> x = array([-0, -1, -2, -3, -4])
>>> ind = (array([[4, 0], [0, 1]]),)  # a tuple containing a 2D integer-array
>>> x[ind]  # the resulting array is commensurate with the indexing array(s)
array([[-4,  0],
       [0, -1]])

Note that this strategy does not accommodate all variations of so-called 'advanced indexing', as prescribed by NumPy's nomenclature.  Combinations of basic and advanced indexes are too complex to usefully define in a standard strategy; we leave application-specific strategies to the user. Advanced-boolean indexing can be defined as arrays(shape=..., dtype=bool), and is similarly left to the user.

hypothesis.extra.numpy.integer_dtypes(*, endianness='?', sizes=(8, 16, 32, 64))

Return a strategy for signed integer dtypes.

endianness and sizes are treated as for unsigned_integer_dtypes().

hypothesis.extra.numpy.mutually_broadcastable_shapes(*, num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for a specified number of shapes N that are mutually-broadcastable with one another and with the provided base shape.

• num_shapes is the number of mutually broadcast-compatible shapes to generate.
• base_shape is the shape against which all generated shapes can broadcast. The default shape is empty, which corresponds to a scalar and thus does not constrain broadcasting at all.
• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to max(len(shape), min_dims) + 2.
• min_side is the smallest size that an unaligned dimension can possess.
• max_side is the largest size that an unaligned dimension can possess, defaulting to 2 plus the size of the largest aligned dimension.

The strategy will generate a python:typing.NamedTuple containing:

• input_shapes as a tuple of the N generated shapes.
• result_shape as the resulting shape produced by broadcasting the N shapes with the base shape.

The following are some examples drawn from this strategy.

>>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
>>> for _ in range(5):
...     print(strat.example())
BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
BroadcastableShapes(input_shapes=((), (), ()), result_shape=())
BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(3,))
BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))


**Use with Generalised Universal Function signatures**

A :doc:`universal function <numpy:reference/ufuncs>` (or ufunc for short) is a function
that operates on ndarrays in an element-by-element fashion, supporting array
broadcasting, type casting, and several other standard features.
A :doc:`generalised ufunc <numpy:reference/c-api/generalized-ufuncs>` operates on
sub-arrays rather than elements, based on the "signature" of the function.
Compare e.g. :obj:`numpy.add() <numpy:numpy.add>` (ufunc) to
:obj:`numpy.matmul() <numpy:numpy.matmul>` (gufunc).

To generate shapes for a gufunc, you can pass the ``signature`` argument instead of
``num_shapes``.  This must be a gufunc signature string; which you can write by
hand or access as e.g. ``np.matmul.signature`` on generalised ufuncs.

In this case, the ``side`` arguments are applied to the 'core dimensions' as well,
ignoring any frozen dimensions.  ``base_shape``  and the ``dims`` arguments are
applied to the 'loop dimensions', and if necessary, the dimensionality of each
shape is silently capped to respect the 32-dimension limit.

The generated ``result_shape`` is the real result shape of applying the gufunc
to arrays of the generated ``input_shapes``, even where this is different to
broadcasting the loop dimensions.

gufunc-compatible shapes shrink their loop dimensions as above, towards omitting
optional core dimensions, and smaller-size core dimensions.

.. code-block:: pycon

    >>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
    >>> for _ in range(3):
    ...     mutually_broadcastable_shapes(signature=np.matmul.signature).example()
    BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
    BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
    BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))

hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(), *, max_leaves=10, max_itemsize=None)

Return the most-general dtype strategy.

Elements drawn from this strategy may be simple (from the subtype_strategy), or several such values drawn from array_dtypes() with allow_subarrays=True. Subdtypes in an array dtype may be nested to any depth, subject to the max_leaves argument.

hypothesis.extra.numpy.scalar_dtypes()

Return a strategy that can return any non-flexible scalar dtype.

hypothesis.extra.numpy.timedelta64_dtypes(*, max_period='Y', min_period='ns', endianness='?')

Return a strategy for timedelta64 dtypes, with various precisions from year to attosecond.

hypothesis.extra.numpy.unicode_string_dtypes(*, endianness='?', min_len=1, max_len=16)

Return a strategy for generating unicode string dtypes, of various lengths and byteorder.

While Hypothesis' string strategies can generate empty strings, string dtypes with length 0 indicate that size is still to be determined, so the minimum length for string dtypes is 1.

hypothesis.extra.numpy.unsigned_integer_dtypes(*, endianness='?', sizes=(8, 16, 32, 64))

Return a strategy for unsigned integer dtypes.

endianness may be < for little-endian, > for big-endian, = for native byte order, or ? to allow either byte order. This argument only applies to dtypes of more than one byte.

sizes must be a collection of integer sizes in bits.  The default (8, 16, 32, 64) covers the full range of sizes.

hypothesis.extra.numpy.valid_tuple_axes(ndim, *, min_size=0, max_size=None)

Return a strategy for generating permissible tuple-values for the axis argument for a numpy sequential function (e.g. numpy:numpy.sum()), given an array of the specified dimensionality.

All tuples will have a length >= min_size and <= max_size. The default value for max_size is ndim.

Examples from this strategy shrink towards an empty tuple, which render most sequential functions as no-ops.

The following are some examples drawn from this strategy.

>>> [valid_tuple_axes(3).example() for i in range(4)]
[(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]

valid_tuple_axes can be joined with other strategies to generate any type of valid axis object, i.e. integers, tuples, and None:

any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)

Hypothesis provides strategies for several of the core pandas data types: pandas.Index, pandas.Series and pandas.DataFrame.

The general approach taken by the pandas module is that there are multiple strategies for generating indexes, and all of the other strategies take the number of entries they contain from their index strategy (with sensible defaults). So e.g. a Series is specified by specifying its numpy.dtype (and/or a strategy for generating elements for it).

class hypothesis.extra.pandas.column(name=None, elements=None, dtype=None, fill=None, unique=False)

Data object for describing a column in a DataFrame.

Arguments:

• name: the column name, or None to default to the column position. Must be hashable, but can otherwise be any value supported as a pandas column name.
• elements: the strategy for generating values in this column, or None to infer it from the dtype.
• dtype: the dtype of the column, or None to infer it from the element strategy. At least one of dtype or elements must be provided.
• fill: A default value for elements of the column. See arrays() for a full explanation.
• unique: If all values in this column should be distinct.

hypothesis.extra.pandas.columns(names_or_number, *, dtype=None, elements=None, fill=None, unique=False)

A convenience function for producing a list of column objects of the same general shape.

The names_or_number argument is either a sequence of values, the elements of which will be used as the name for individual column objects, or a number, in which case that many unnamed columns will be created. All other arguments are passed through verbatim to create the columns.

hypothesis.extra.pandas.data_frames(columns=None, *, rows=None, index=None)

Provides a strategy for producing a pandas.DataFrame.

Arguments:

• columns: An iterable of column objects describing the shape of the generated DataFrame.

• rows: A strategy for generating a row object. Should generate either dicts mapping column names to values or a sequence mapping column position to the value in that position (note that unlike the pandas.DataFrame constructor, single values are not allowed here. Passing e.g. an integer is an error, even if there is only one column).

  At least one of rows and columns must be provided. If both are provided then the generated rows will be validated against the columns and an error will be raised if they don't match.

  Caveats on using rows:

  • In general you should prefer using columns to rows, and only use rows if the columns interface is insufficiently flexible to describe what you need - you will get better performance and example quality that way.
  • If you provide rows and not columns, then the shape and dtype of the resulting DataFrame may vary. e.g. if you have a mix of int and float in the values for one column in your row entries, the column will sometimes have an integral dtype and sometimes a float.

• index: If not None, a strategy for generating indexes for the resulting DataFrame. This can generate either pandas.Index objects or any sequence of values (which will be passed to the Index constructor).

  You will probably find it most convenient to use the indexes() or range_indexes() function to produce values for this argument.

Usage:

The expected usage pattern is that you use column and columns() to specify a fixed shape of the DataFrame you want as follows. For example the following gives a two column data frame:

>>> from hypothesis.extra.pandas import column, data_frames
>>> data_frames([
... column('A', dtype=int), column('B', dtype=float)]).example()
            A              B
0  2021915903  1.793898e+232
1  1146643993            inf
2 -2096165693   1.000000e+07

If you want the values in different columns to interact in some way you can use the rows argument. For example the following gives a two column DataFrame where the value in the first column is always at most the value in the second:

>>> from hypothesis.extra.pandas import column, data_frames
>>> import hypothesis.strategies as st
>>> data_frames(
...     rows=st.tuples(st.floats(allow_nan=False),
...                    st.floats(allow_nan=False)).map(sorted)
... ).example()
               0             1
0  -3.402823e+38  9.007199e+15
1 -1.562796e-298  5.000000e-01

You can also combine the two:

>>> from hypothesis.extra.pandas import columns, data_frames
>>> import hypothesis.strategies as st
>>> data_frames(
...     columns=columns(["lo", "hi"], dtype=float),
...     rows=st.tuples(st.floats(allow_nan=False),
...                    st.floats(allow_nan=False)).map(sorted)
... ).example()
         lo            hi
0   9.314723e-49  4.353037e+45
1  -9.999900e-01  1.000000e+07
2 -2.152861e+134 -1.069317e-73

(Note that the column dtype must still be specified and will not be inferred from the rows. This restriction may be lifted in future).

Combining rows and columns has the following behaviour:

• The column names and dtypes will be used.
• If the column is required to be unique, this will be enforced.
• Any values missing from the generated rows will be provided using the column's fill.
• Any values in the row not present in the column specification (if dicts are passed, if there are keys with no corresponding column name, if sequences are passed if there are too many items) will result in InvalidArgument being raised.

hypothesis.extra.pandas.indexes(*, elements=None, dtype=None, min_size=0, max_size=None, unique=True, name=none())

Provides a strategy for producing a pandas.Index.

Arguments:

• elements is a strategy which will be used to generate the individual values of the index. If None, it will be inferred from the dtype. Note: even if the elements strategy produces tuples, the generated value will not be a MultiIndex, but instead be a normal index whose elements are tuples.
• dtype is the dtype of the resulting index. If None, it will be inferred from the elements strategy. At least one of dtype or elements must be provided.
• min_size is the minimum number of elements in the index.
• max_size is the maximum number of elements in the index. If None then it will default to a suitable small size. If you want larger indexes you should pass a max_size explicitly.
• unique specifies whether all of the elements in the resulting index should be distinct.
• name is a strategy for strings or None, which will be passed to the pandas.Index constructor.

hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None, name=none())

Provides a strategy which generates an Index whose values are 0, 1, ..., n for some n.

Arguments:

• min_size is the smallest number of elements the index can have.
• max_size is the largest number of elements the index can have. If None it will default to some suitable value based on min_size.
• name is the name of the index. If st.none(), the index will have no name.

hypothesis.extra.pandas.series(*, elements=None, dtype=None, index=None, fill=None, unique=False, name=none())

Provides a strategy for producing a pandas.Series.

Arguments:

• elements: a strategy that will be used to generate the individual values in the series. If None, we will attempt to infer a suitable default from the dtype.
• dtype: the dtype of the resulting series and may be any value that can be passed to numpy.dtype. If None, will use pandas's standard behaviour to infer it from the type of the elements values. Note that if the type of values that comes out of your elements strategy varies, then so will the resulting dtype of the series.

• index: If not None, a strategy for generating indexes for the resulting Series. This can generate either pandas.Index objects or any sequence of values (which will be passed to the Index constructor).

  You will probably find it most convenient to use the indexes() or range_indexes() function to produce values for this argument.

• name: is a strategy for strings or None, which will be passed to the pandas.Series constructor.

Usage:

>>> series(dtype=int).example()
0   -2001747478
1    1153062837

There is quite a lot of variation between pandas versions. We only commit to supporting the latest version of pandas, but older minor versions are supported on a "best effort" basis.  Hypothesis is currently tested against and confirmed working with every Pandas minor version from 1.1 through to 2.2.

Releases that are not the latest patch release of their minor version are not tested or officially supported, but will probably also work unless you hit a pandas bug.

Note:

Several array libraries have more library-specific strategies, including Xarray <https://pypi.org/project/xarray/> (via their upstream strategies) and NumPy <https://pypi.org/project/NumPy/> (via its Hypothesis extra). Of course, strategies in the Array API namespace can still be used to test Xarray or NumPy, just like any other array library.

Hypothesis offers strategies for Array API <https://data-apis.org/> adopting libraries in the hypothesis.extra.array_api package. See issue #3037 <https://github.com/HypothesisWorks/hypothesis/issues/3037> for more details.  If you want to test with CuPy <https://pypi.org/project/cupy/>, Dask <https://pypi.org/project/dask/>, JAX <https://pypi.org/project/jax/>, MXNet <https://pypi.org/project/maxnet/>, PyTorch <https://pypi.org/project/torch/>, TensorFlow <https://pypi.org/project/tensorflow/>, or Xarray <https://pypi.org/project/xarray/> - or just NumPy <https://pypi.org/project/numpy/> - this is the extension for you!

hypothesis.extra.array_api.make_strategies_namespace(xp, *, api_version=None)

Creates a strategies namespace for the given array module.

• xp is the Array API library to automatically pass to the namespaced methods.
• api_version is the version of the Array API which the returned strategies namespace should conform to. If None, the latest API version which xp supports will be inferred from xp.__array_api_version__. If a version string in the YYYY.MM format, the strategies namespace will conform to that version if supported.

A python:types.SimpleNamespace is returned which contains all the strategy methods in this module but without requiring the xp argument. Creating and using a strategies namespace for NumPy's Array API implementation would go like this:

>>> xp.__array_api_version__  # xp is your desired array library
'2021.12'
>>> xps = make_strategies_namespace(xp)
>>> xps.api_version
'2021.12'
>>> x = xps.arrays(xp.int8, (2, 3)).example()
>>> x
Array([[-8,  6,  3],
       [-6,  4,  6]], dtype=int8)
>>> x.__array_namespace__() is xp
True

The resulting namespace contains all our familiar strategies like arrays() and from_dtype(), but based on the Array API standard semantics and returning objects from the xp module:

xps.array_shapes(*, min_dims=1, max_dims=None, min_side=1, max_side=None)

Return a strategy for array shapes (tuples of int >= 1).

• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to min_dims + 2.
• min_side is the smallest size that a dimension can possess.
• max_side is the largest size that a dimension can possess, defaulting to min_side + 5.

xps.arrays(dtype, shape, *, elements=None, fill=None, unique=False)

Returns a strategy for arrays <https://data-apis.org/array-api/latest/API_specification/array_object.html>.

• dtype may be a valid dtype <https://data-apis.org/array-api/latest/API_specification/data_types.html> object or name, or a strategy that generates such values.
• shape may be an integer >= 0, a tuple of such integers, or a strategy that generates such values.
• elements is a strategy for values to put in the array. If None then a suitable value will be inferred based on the dtype, which may give any legal value (including e.g. NaN for floats). If a mapping, it will be passed as **kwargs to from_dtype() when inferring based on the dtype.
• fill is a strategy that may be used to generate a single background value for the array. If None, a suitable default will be inferred based on the other arguments. If set to nothing() then filling behaviour will be disabled entirely and every element will be generated independently.
• unique specifies if the elements of the array should all be distinct from one another; if fill is also set, the only valid values for fill to return are NaN values.

Arrays of specified dtype and shape are generated for example like this:

>>> from numpy import array_api as xp
>>> xps.arrays(xp, xp.int8, (2, 3)).example()
Array([[-8,  6,  3],
       [-6,  4,  6]], dtype=int8)

Specifying element boundaries by a python:dict of the kwargs to pass to from_dtype() will ensure dtype bounds will be respected.

>>> xps.arrays(xp, xp.int8, 3, elements={"min_value": 10}).example()
Array([125, 13, 79], dtype=int8)

>>> xps.arrays(xp, xp.float32, 3, elements=floats(0, 1, width=32)).example()
Array([ 0.88974794,  0.77387938,  0.1977879 ], dtype=float32)

Array values are generated in two parts:

1. A single value is drawn from the fill strategy and is used to create a filled array.
2. Some subset of the coordinates of the array are populated with a value drawn from the elements strategy (or its inferred form).

You can set fill to nothing() if you want to disable this behaviour and draw a value for every element.

By default arrays will attempt to infer the correct fill behaviour: if unique is also True, no filling will occur. Otherwise, if it looks safe to reuse the values of elements across multiple coordinates (this will be the case for any inferred strategy, and for most of the builtins, but is not the case for mutable values or strategies built with flatmap, map, composite, etc.) then it will use the elements strategy as the fill, else it will default to having no fill.

Having a fill helps Hypothesis craft high quality examples, but its main importance is when the array generated is large: Hypothesis is primarily designed around testing small examples. If you have arrays with hundreds or more elements, having a fill value is essential if you want your tests to run in reasonable time.

xps.boolean_dtypes()

Return a strategy for just the boolean dtype object.

xps.broadcastable_shapes(shape, *, min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for shapes that are broadcast-compatible with the provided shape.

Examples from this strategy shrink towards a shape with length min_dims. The size of an aligned dimension shrinks towards size 1. The size of an unaligned dimension shrink towards min_side.

• shape is a tuple of integers.
• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to max(len(shape), min_dims) + 2.
• min_side is the smallest size that an unaligned dimension can possess.
• max_side is the largest size that an unaligned dimension can possess, defaulting to 2 plus the size of the largest aligned dimension.

The following are some examples drawn from this strategy.

>>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
[(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]

xps.complex_dtypes(*, sizes=(64, 128))

Return a strategy for complex dtype objects.

sizes contains the complex sizes in bits, defaulting to (64, 128) which covers all valid sizes.

xps.floating_dtypes(*, sizes=(32, 64))

Return a strategy for real-valued floating-point dtype objects.

sizes contains the floating-point sizes in bits, defaulting to (32, 64) which covers all valid sizes.

xps.from_dtype(dtype, *, min_value=None, max_value=None, allow_nan=None, allow_infinity=None, allow_subnormal=None, exclude_min=None, exclude_max=None)

Return a strategy for any value of the given dtype.

Values generated are of the Python scalar which is promotable <https://data-apis.org/array-api/latest/API_specification/type_promotion.html> to dtype, where the values do not exceed its bounds.

• dtype may be a dtype object or the string name of a valid dtype <https://data-apis.org/array-api/latest/API_specification/data_types.html>.

Compatible **kwargs are passed to the inferred strategy function for integers and floats.  This allows you to customise the min and max values, and exclude non-finite numbers. This is particularly useful when kwargs are passed through from arrays(), as it seamlessly handles the width or other representable bounds for you.

xps.indices(shape, *, min_dims=0, max_dims=None, allow_newaxis=False, allow_ellipsis=True)

Return a strategy for valid indices <https://data-apis.org/array-api/latest/API_specification/indexing.html> of arrays with the specified shape, which may include dimensions of size zero.

It generates tuples containing some mix of integers, python:slice objects, ... (an Ellipsis), and None. When a length-one tuple would be generated, this strategy may instead return the element which will index the first axis, e.g. 5 instead of (5,).

• shape is the shape of the array that will be indexed, as a tuple of integers >= 0. This must be at least two-dimensional for a tuple to be a valid index;  for one-dimensional arrays use slices() instead.
• min_dims is the minimum dimensionality of the resulting array from use of the generated index.
• max_dims is the maximum dimensionality of the resulting array, defaulting to len(shape) if not allow_newaxis else max(len(shape), min_dims) + 2.
• allow_ellipsis specifies whether None is allowed in the index.
• allow_ellipsis specifies whether ... is allowed in the index.

xps.integer_dtypes(*, sizes=(8, 16, 32, 64))

Return a strategy for signed integer dtype objects.

sizes contains the signed integer sizes in bits, defaulting to (8, 16, 32, 64) which covers all valid sizes.

xps.mutually_broadcastable_shapes(num_shapes, *, base_shape=(), min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for a specified number of shapes N that are mutually-broadcastable with one another and with the provided base shape.

• num_shapes is the number of mutually broadcast-compatible shapes to generate.
• base_shape is the shape against which all generated shapes can broadcast. The default shape is empty, which corresponds to a scalar and thus does not constrain broadcasting at all.
• min_dims is the smallest length that the generated shape can possess.
• max_dims is the largest length that the generated shape can possess, defaulting to max(len(shape), min_dims) + 2.
• min_side is the smallest size that an unaligned dimension can possess.
• max_side is the largest size that an unaligned dimension can possess, defaulting to 2 plus the size of the largest aligned dimension.

The strategy will generate a python:typing.NamedTuple containing:

• input_shapes as a tuple of the N generated shapes.
• result_shape as the resulting shape produced by broadcasting the N shapes with the base shape.

The following are some examples drawn from this strategy.

>>> # Draw three shapes where each shape is broadcast-compatible with (2, 3)
... strat = mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3))
>>> for _ in range(5):
...     print(strat.example())
BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
BroadcastableShapes(input_shapes=((), (), ()), result_shape=())
BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(3,))
BroadcastableShapes(input_shapes=((1, 2, 3), (3,), ()), result_shape=(1, 2, 3))

xps.numeric_dtypes()

Return a strategy for all numeric dtype objects.

xps.real_dtypes()

Return a strategy for all real-valued dtype objects.

xps.scalar_dtypes()

Return a strategy for all valid dtype <https://data-apis.org/array-api/latest/API_specification/data_types.html> objects.

xps.unsigned_integer_dtypes(*, sizes=(8, 16, 32, 64))

Return a strategy for unsigned integer dtype objects.

sizes contains the unsigned integer sizes in bits, defaulting to (8, 16, 32, 64) which covers all valid sizes.

xps.valid_tuple_axes(ndim, *, min_size=0, max_size=None)

Return a strategy for permissible tuple-values for the axis argument in Array API sequential methods e.g. sum, given the specified dimensionality.

All tuples will have a length >= min_size and <= max_size. The default value for max_size is ndim.

Examples from this strategy shrink towards an empty tuple, which render most sequential functions as no-ops.

The following are some examples drawn from this strategy.

>>> [valid_tuple_axes(3).example() for i in range(4)]
[(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]

valid_tuple_axes can be joined with other strategies to generate any type of valid axis object, i.e. integers, tuples, and None:

any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)

See also:

hypothesis.extra.django.from_model(model, /, **field_strategies)

Return a strategy for examples of model.

Warning:

Hypothesis creates saved models. This will run inside your testing transaction when using the test runner, but if you use the dev console this will leave debris in your database.

model must be an subclass of Model. Strategies for fields may be passed as keyword arguments, for example is_staff=st.just(False).  In order to support models with fields named "model", this is a positional-only parameter.

Hypothesis can often infer a strategy based the field type and validators, and will attempt to do so for any required fields.  No strategy will be inferred for an AutoField, nullable field, foreign key, or field for which a keyword argument is passed to from_model().  For example, a Shop type with a foreign key to Company could be generated with:

shop_strategy = from_model(Shop, company=from_model(Company))

Like for builds(), you can pass ... (python:Ellipsis) as a keyword argument to infer a strategy for a field which has a default value instead of using the default.

hypothesis.extra.django.from_form(form, form_kwargs=None, **field_strategies)

Return a strategy for examples of form.

form must be an subclass of Form. Strategies for fields may be passed as keyword arguments, for example is_staff=st.just(False).

Hypothesis can often infer a strategy based the field type and validators, and will attempt to do so for any required fields.  No strategy will be inferred for a disabled field or field for which a keyword argument is passed to from_form().

This function uses the fields of an unbound form instance to determine field strategies, any keyword arguments needed to instantiate the unbound form instance can be passed into from_form() as a dict with the keyword form_kwargs. E.g.:

shop_strategy = from_form(Shop, form_kwargs={"company_id": 5})

Like for builds(), you can pass ... (python:Ellipsis) as a keyword argument to infer a strategy for a field which has a default value instead of using the default.

hypothesis.extra.django.from_field(field)

Return a strategy for values that fit the given field.

This function is used by from_form() and from_model() for any fields that require a value, or for which you passed ... (python:Ellipsis) to infer a strategy from an annotation.

It's pretty similar to the core from_type() function, with a subtle but important difference: from_field takes a Field instance, rather than a Field subtype, so that it has access to instance attributes such as string length and validators.

hypothesis.extra.django.register_field_strategy(field_type, strategy)

Add an entry to the global field-to-strategy lookup used by from_field().

field_type must be a subtype of django.db.models.Field or django.forms.Field, which must not already be registered. strategy must be a SearchStrategy.

Note:

Strategies in this module require the hypothesis[lark] extra <>, via pip install hypothesis[lark].

This extra can be used to generate strings matching any context-free grammar, using the Lark parser library <https://github.com/lark-parser/lark>.

It currently only supports Lark's native EBNF syntax, but we plan to extend this to support other common syntaxes such as ANTLR and RFC 5234 <https://datatracker.ietf.org/doc/html/rfc5234.html> ABNF. Lark already supports loading grammars <https://lark-parser.readthedocs.io/en/stable/tools.html#importing-grammars-from-nearley-js> from nearley.js <https://nearley.js.org/>, so you may not have to write your own at all.

hypothesis.extra.lark.from_lark(grammar, *, start=None, explicit=None, alphabet=characters(codec='utf-8'))

A strategy for strings accepted by the given context-free grammar.

grammar must be a Lark object, which wraps an EBNF specification. The Lark EBNF grammar reference can be found here <https://lark-parser.readthedocs.io/en/latest/grammar.html>.

from_lark will automatically generate strings matching the nonterminal start symbol in the grammar, which was supplied as an argument to the Lark class.  To generate strings matching a different symbol, including terminals, you can override this by passing the start argument to from_lark.  Note that Lark may remove unreachable productions when the grammar is compiled, so you should probably pass the same value for start to both.

Currently from_lark does not support grammars that need custom lexing. Any lexers will be ignored, and any undefined terminals from the use of %declare will result in generation errors.  To define strategies for such terminals, pass a dictionary mapping their name to a corresponding strategy as the explicit argument.

The hypothesmith <https://pypi.org/project/hypothesmith/> project includes a strategy for Python source, based on a grammar and careful post-processing.

Example grammars, which may provide a useful starting point for your tests, can be found in the Lark repository <https://github.com/lark-parser/lark/tree/master/examples> and in this third-party collection <https://github.com/ligurio/lark-grammars>.

Note:

Strategies in this module require the hypothesis[pytz] extra <>, via pip install hypothesis[pytz].

This module provides pytz <https://pypi.org/project/pytz/> timezones.

If you are unable to use the stdlib zoneinfo module, e.g. via the hypothesis.strategies.timezones() strategy, you can use this strategy with hypothesis.strategies.datetimes() and hypothesis.strategies.times() to produce timezone-aware values.

Warning:

Since zoneinfo was added in Python 3.9, this extra is deprecated.  We intend to remove it after libraries such as Pandas and Django complete their own migrations.

hypothesis.extra.pytz.timezones()

Any timezone in the Olsen database, as a pytz tzinfo object.

This strategy minimises to UTC, or the smallest possible fixed offset, and is designed for use with hypothesis.strategies.datetimes().

Tip:

Prefer the hypothesis.strategies.timezones() strategy, which uses the stdlib zoneinfo module and avoids the many footguns in pytz <https://blog.ganssle.io/articles/2018/03/pytz-fastest-footgun.html>.

Note:

Strategies in this module require the hypothesis[dateutil] extra <>, via pip install hypothesis[dateutil].

This module provides dateutil <https://pypi.org/project/python-dateutil/> timezones.

You can use this strategy to make datetimes() and times() produce timezone-aware values.

Tip:

Consider using the stdlib zoneinfo module, via st.timezones().

hypothesis.extra.dateutil.timezones()

Any timezone from dateutil <https://pypi.org/project/python-dateutil/>.

This strategy minimises to UTC, or the timezone with the smallest offset from UTC as of 2000-01-01, and is designed for use with datetimes().

Note that the timezones generated by the strategy may vary depending on the configuration of your machine. See the dateutil documentation for more information.

Reference for Hypothesis features with a defined interface, but no code API.

Writing tests with Hypothesis frees you from the tedium of deciding on and writing out specific inputs to test.  Now, the hypothesis.extra.ghostwriter module can write your test functions for you too!

The idea is to provide an easy way to start property-based testing, and a seamless transition to more complex test code - because ghostwritten tests are source code that you could have written for yourself.

So just pick a function you'd like tested, and feed it to one of the functions below.  They follow imports, use but do not require type annotations, and generally do their best to write you a useful test.  You can also use our command-line interface:

$ hypothesis write --help
Usage: hypothesis write [OPTIONS] FUNC...

  `hypothesis write` writes property-based tests for you!

  Type annotations are helpful but not required for our advanced
  introspection and templating logic.  Try running the examples below to see
  how it works:

      hypothesis write gzip
      hypothesis write numpy.matmul
      hypothesis write pandas.from_dummies
      hypothesis write re.compile --except re.error
      hypothesis write --equivalent ast.literal_eval eval
      hypothesis write --roundtrip json.dumps json.loads
      hypothesis write --style=unittest --idempotent sorted
      hypothesis write --binary-op operator.add

Options:
  --roundtrip                 start by testing write/read or encode/decode!
  --equivalent                very useful when optimising or refactoring code
  --errors-equivalent         --equivalent, but also allows consistent errors
  --idempotent                check that f(x) == f(f(x))
  --binary-op                 associativity, commutativity, identity element
  --style [pytest|unittest]   pytest-style function, or unittest-style method?
  -e, --except OBJ_NAME       dotted name of exception(s) to ignore
  --annotate / --no-annotate  force ghostwritten tests to be type-annotated
                              (or not).  By default, match the code to test.
  -h, --help                  Show this message and exit.

Tip:

Using a light theme?  Hypothesis respects NO_COLOR <https://no-color.org/> and DJANGO_COLORS=light.

Note:

The ghostwriter requires black <https://pypi.org/project/black/>, but the generated code only requires Hypothesis itself.

Note:

Legal questions?  While the ghostwriter fragments and logic is under the MPL-2.0 license like the rest of Hypothesis, the output from the ghostwriter is made available under the Creative Commons Zero (CC0) <https://creativecommons.org/public-domain/cc0/> public domain dedication, so you can use it without any restrictions.

hypothesis.extra.ghostwriter.magic(*modules_or_functions, except_=(), style='pytest', annotate=None)

Guess which ghostwriters to use, for a module or collection of functions.

As for all ghostwriters, the except_ argument should be an python:Exception or tuple of exceptions, and style may be either "pytest" to write test functions or "unittest" to write test methods and TestCase.

After finding the public functions attached to any modules, the magic ghostwriter looks for pairs of functions to pass to roundtrip(), then checks for binary_operation() and ufunc() functions, and any others are passed to fuzz().

For example, try hypothesis write gzip on the command line!

hypothesis.extra.ghostwriter.fuzz(func, *, except_=(), style='pytest', annotate=None)

Write source code for a property-based test of func.

The resulting test checks that valid input only leads to expected exceptions. For example:

from re import compile, error

from hypothesis.extra import ghostwriter

ghostwriter.fuzz(compile, except_=error)

Gives:

# This test code was written by the `hypothesis.extra.ghostwriter` module
# and is provided under the Creative Commons Zero public domain dedication.
import re

from hypothesis import given, reject, strategies as st

# TODO: replace st.nothing() with an appropriate strategy


@given(pattern=st.nothing(), flags=st.just(0))
def test_fuzz_compile(pattern, flags):
    try:
        re.compile(pattern=pattern, flags=flags)
    except re.error:
        reject()

Note that it includes all the required imports. Because the pattern parameter doesn't have annotations or a default argument, you'll need to specify a strategy - for example text() <#hypothesis.strategies.text> or binary() <#hypothesis.strategies.binary>.  After that, you have a test!

hypothesis.extra.ghostwriter.idempotent(func, *, except_=(), style='pytest', annotate=None)

Write source code for a property-based test of func.

The resulting test checks that if you call func on it's own output, the result does not change.  For example:

from typing import Sequence

from hypothesis.extra import ghostwriter


def timsort(seq: Sequence[int]) -> Sequence[int]:
    return sorted(seq)


ghostwriter.idempotent(timsort)

Gives:

# This test code was written by the `hypothesis.extra.ghostwriter` module
# and is provided under the Creative Commons Zero public domain dedication.

from hypothesis import given, strategies as st


@given(seq=st.one_of(st.binary(), st.binary().map(bytearray), st.lists(st.integers())))
def test_idempotent_timsort(seq):
    result = timsort(seq=seq)
    repeat = timsort(seq=result)
    assert result == repeat, (result, repeat)

hypothesis.extra.ghostwriter.roundtrip(*funcs, except_=(), style='pytest', annotate=None)

Write source code for a property-based test of funcs.

The resulting test checks that if you call the first function, pass the result to the second (and so on), the final result is equal to the first input argument.

This is a very powerful property to test, especially when the config options are varied along with the object to round-trip.  For example, try ghostwriting a test for python:json.dumps() - would you have thought of all that?

hypothesis write --roundtrip json.dumps json.loads

hypothesis.extra.ghostwriter.equivalent(*funcs, allow_same_errors=False, except_=(), style='pytest', annotate=None)

Write source code for a property-based test of funcs.

The resulting test checks that calling each of the functions returns an equal value.  This can be used as a classic 'oracle', such as testing a fast sorting algorithm against the python:sorted() builtin, or for differential testing where none of the compared functions are fully trusted but any difference indicates a bug (e.g. running a function on different numbers of threads, or simply multiple times).

The functions should have reasonably similar signatures, as only the common parameters will be passed the same arguments - any other parameters will be allowed to vary.

If allow_same_errors is True, then the test will pass if calling each of the functions returns an equal value, or if the first function raises an exception and each of the others raises an exception of the same type. This relaxed mode can be useful for code synthesis projects.

hypothesis.extra.ghostwriter.binary_operation(func, *, associative=True, commutative=True, identity=Ellipsis, distributes_over=None, except_=(), style='pytest', annotate=None)

Write property tests for the binary operation func.

While binary operations <https://en.wikipedia.org/wiki/Binary_operation> are not particularly common, they have such nice properties to test that it seems a shame not to demonstrate them with a ghostwriter.  For an operator f, test that:

• if associative <https://en.wikipedia.org/wiki/Associative_property>, f(a, f(b, c)) == f(f(a, b), c)
• if commutative <https://en.wikipedia.org/wiki/Commutative_property>, f(a, b) == f(b, a)
• if identity <https://en.wikipedia.org/wiki/Identity_element> is not None, f(a, identity) == a
• if distributes_over <https://en.wikipedia.org/wiki/Distributive_property> is +, f(a, b) + f(a, c) == f(a, b+c)

For example:

ghostwriter.binary_operation(
    operator.mul,
    identity=1,
    distributes_over=operator.add,
    style="unittest",
)

hypothesis.extra.ghostwriter.ufunc(func, *, except_=(), style='pytest', annotate=None)

Write a property-based test for the array ufunc func.

The resulting test checks that your ufunc or gufunc has the expected broadcasting and dtype casting behaviour.  You will probably want to add extra assertions, but as with the other ghostwriters this gives you a great place to start.

hypothesis write numpy.matmul

Ghostwritten tests are intended as a starting point for human authorship, to demonstrate best practice, help novices past blank-page paralysis, and save time for experts.  They may be ready-to-run, or include placeholders and # TODO: comments to fill in strategies for unknown types.  In either case, improving tests for their own code gives users a well-scoped and immediately rewarding context in which to explore property-based testing.

By contrast, most test-generation tools aim to produce ready-to-run test suites... and implicitly assume that the current behavior is the desired behavior. However, the code might contain bugs, and we want our tests to fail if it does! Worse, tools require that the code to be tested is finished and executable, making it impossible to generate tests as part of the development process.

Fraser 2013 <https://doi.org/10.1145/2483760.2483774> found that evolving a high-coverage test suite (e.g. Randoop <https://homes.cs.washington.edu/~mernst/pubs/feedback-testgen-icse2007.pdf>, EvoSuite <https://www.evosuite.org/wp-content/papercite-data/pdf/esecfse11.pdf>, Pynguin <https://arxiv.org/abs/2007.14049>) "leads to clear improvements in commonly applied quality metrics such as code coverage [but] no measurable improvement in the number of bugs actually found by developers" and that "generating a set of test cases, even high coverage test cases, does not necessarily improve our ability to test software". Invariant detection (famously Daikon <https://plse.cs.washington.edu/daikon/pubs/>; in PBT see e.g. Alonso 2022 <https://doi.org/10.1145/3540250.3559080>, QuickSpec <http://www.cse.chalmers.se/~nicsma/papers/quickspec2.pdf>, Speculate <https://matela.com.br/speculate.pdf>) relies on code execution. Program slicing (e.g. FUDGE <https://research.google/pubs/pub48314/>, FuzzGen <https://www.usenix.org/conference/usenixsecurity20/presentation/ispoglou>, WINNIE <https://www.ndss-symposium.org/wp-content/uploads/2021-334-paper.pdf>) requires downstream consumers of the code to test.

Ghostwriter inspects the function name, argument names and types, and docstrings. It can be used on buggy or incomplete code, runs in a few seconds, and produces a single semantically-meaningful test per function or group of functions. Rather than detecting regressions, these tests check semantic properties such as encode/decode or save/load round-trips <https://zhd.dev/ghostwriter/?q=gzip.compress>, for commutative, associative, and distributive operations <https://zhd.dev/ghostwriter/?q=operator.mul>, equivalence between methods <https://zhd.dev/ghostwriter/?q=operator.add+numpy.add>, array shapes <https://zhd.dev/ghostwriter/?q=numpy.matmul>, and idempotence.  Where no property is detected, we simply check for 'no error on valid input' and allow the user to supply their own invariants.

Evaluations such as the SBFT24 <https://arxiv.org/abs/2401.15189> competition <https://github.com/ThunderKey/python-tool-competition-2024> measure performance on a task which the Ghostwriter is not intended to perform.  I'd love to see qualitative user studies, such as PBT in Practice <https://harrisongoldste.in/papers/icse24-pbt-in-practice.pdf> for test generation, which could check whether the Ghostwriter is onto something or tilting at windmills. If you're interested in similar questions, drop me an email <mailto:zac@zhd.dev?subject=Hypothesis%20Ghostwriter%20research>!

Note:

The Tyche <https://github.com/tyche-pbt/tyche-extension> VSCode extension provides an in-editor UI for observability results generated by Hypothesis. If you want to view observability results, rather than programmatically consume or display them, we recommend using Tyche.

Warning:

This feature is experimental, and could have breaking changes or even be removed without notice.  Try it out, let us know what you think, but don't rely on it just yet!

Understanding what your code is doing - for example, why your test failed - is often a frustrating exercise in adding some more instrumentation or logging (or print() calls) and running it again.  The idea of observability <https://en.wikipedia.org/wiki/Observability_(software)> is to let you answer questions you didn't think of in advance.  In slogan form,

By default, Hypothesis only reports the minimal failing example... but sometimes you might want to know something about all the examples.  Printing them to the terminal by increasing Verbosity <#hypothesis.Verbosity> might be nice, but isn't always enough. This feature gives you an analysis-ready dataframe with useful columns and one row per test case, with columns from arguments to code coverage to pass/fail status.

This is deliberately a much lighter-weight and task-specific system than e.g. OpenTelemetry <https://opentelemetry.io/>.  It's also less detailed than time-travel debuggers such as rr <https://rr-project.org/> or pytrace <https://pytrace.com/>, because there's no good way to compare multiple traces from these tools and their Python support is relatively immature.

If you set the HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY environment variable, Hypothesis will log various observations to jsonlines files in the .hypothesis/observed/ directory.  You can load and explore these with e.g. pd.read_json(".hypothesis/observed/*_testcases.jsonl", lines=True), or by using the sqlite-utils <https://pypi.org/project/sqlite-utils/> and datasette <https://pypi.org/project/datasette/> libraries:

sqlite-utils insert testcases.db testcases .hypothesis/observed/*_testcases.jsonl --nl --flatten
datasette serve testcases.db

If you are experiencing a significant slow-down, you can try setting HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVER instead; this will disable coverage information collection. This should not be necessary on Python 3.12 or later, where coverage collection is very fast.

If you want to record more information about your test cases than the arguments and outcome - for example, was x a binary tree?  what was the difference between the expected and the actual value?  how many queries did it take to find a solution? - Hypothesis makes this easy.

event() <#hypothesis.event> accepts a string label, and optionally a string or int or float observation associated with it.  All events are collected and summarized in Test statistics, as well as included on a per-test-case basis in our observations.

target() <#hypothesis.target> is a special case of numeric-valued events: as well as recording them in observations, Hypothesis will try to maximize the targeted value. Knowing that, you can use this to guide the search for failing inputs.

We dump observations in json lines format <https://jsonlines.org/>, with each line describing either a test case or an information message.  The tables below are derived from this machine-readable JSON schema, to provide both readable and verifiable specifications.

Note that we use python:json.dumps() and can therefore emit non-standard JSON which includes infinities and NaN.  This is valid in JSON5 <https://json5.org/>, and supported by some JSON parsers <https://evanhahn.com/pythons-nonstandard-json-encoding/> including Gson in Java, JSON.parse() in Ruby, and of course in Python.

While the observability format is agnostic to the property-based testing library which generated it, Hypothesis includes specific values in the metadata key for test cases. You may rely on these being present if and only if the observation was generated by Hypothesis.

These additional metadata elements are included in metadata (as e.g. metadata["choice_nodes"] or metadata["choice_spans"]), if and only if OBSERVABILITY_CHOICES <#hypothesis.internal.observability.OBSERVABILITY_CHOICES> is set.

Hypothesis includes a tiny plugin to improve integration with pytest <https://pypi.org/project/pytest/>, which is activated by default (but does not affect other test runners). It aims to improve the integration between Hypothesis and Pytest by providing extra information and convenient access to config options.

• pytest --hypothesis-show-statistics can be used to display test and data generation statistics.
• pytest --hypothesis-profile=<profile name> can be used to load a settings profile (as in load_profile() <#hypothesis.settings.load_profile>).
• pytest --hypothesis-verbosity=<level name> can be used to override the current Verbosity <#hypothesis.Verbosity> setting.
• pytest --hypothesis-seed=<an int> can be used to reproduce a failure with a particular seed (as in @seed <#hypothesis.seed>).
• pytest --hypothesis-explain can be used to temporarily enable Phase.explain <#hypothesis.Phase.explain>.

Finally, all tests that are defined with Hypothesis automatically have @pytest.mark.hypothesis applied to them.  See here for information on working with markers.

Note:

Note:

While test statistics are only available under pytest, you can use the observability interface to view similar information about your tests.

You can see a number of statistics about executed tests by passing the command line argument --hypothesis-show-statistics. This will include some general statistics about the test:

For example if you ran the following with --hypothesis-show-statistics:

from hypothesis import given, strategies as st

@given(st.integers())
def test_integers(i):
    pass

You would see:

- during generate phase (0.06 seconds):
    - Typical runtimes: < 1ms, ~ 47% in data generation
    - 100 passing examples, 0 failing examples, 0 invalid examples
- Stopped because settings.max_examples=100

The final "Stopped because" line tells you why Hypothesis stopped generating new examples. This is typically because we hit max_examples <#hypothesis.settings.max_examples>, but occasionally because we exhausted the search space or because shrinking was taking a very long time. This can be useful for understanding the behaviour of your tests.

In some cases (such as filtered and recursive strategies) you will see events mentioned which describe some aspect of the data generation:

from hypothesis import given, strategies as st

@given(st.integers().filter(lambda x: x % 2 == 0))
def test_even_integers(i):
    pass

You would see something like:

test_even_integers:

  - during generate phase (0.08 seconds):
      - Typical runtimes: < 1ms, ~ 57% in data generation
      - 100 passing examples, 0 failing examples, 12 invalid examples
      - Events:
        * 51.79%, Retried draw from integers().filter(lambda x: x % 2 == 0) to satisfy filter
        * 10.71%, Aborted test because unable to satisfy integers().filter(lambda x: x % 2 == 0)
  - Stopped because settings.max_examples=100

Note:

This feature requires the hypothesis[cli] extra <>, via pip install hypothesis[cli].

$ hypothesis --help
Usage: hypothesis [OPTIONS] COMMAND [ARGS]...

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  codemod  `hypothesis codemod` refactors deprecated or inefficient code.
  fuzz     [hypofuzz] runs tests with an adaptive coverage-guided fuzzer.
  write    `hypothesis write` writes property-based tests for you!

This module requires the click <https://pypi.org/project/click/> package, and provides Hypothesis' command-line interface, for e.g. 'ghostwriting' tests via the terminal. It's also where HypoFuzz <https://hypofuzz.com/> adds the hypothesis fuzz command (learn more about that here <https://hypofuzz.com/docs/quickstart.html>).

Note:

This feature requires the hypothesis[codemods] extra <>, via pip install hypothesis[codemods].

This module provides codemods based on the LibCST <https://pypi.org/project/LibCST/> library, which can both detect and automatically fix issues with code that uses Hypothesis, including upgrading from deprecated features to our recommended style.

You can run the codemods via our CLI:

$ hypothesis codemod --help
Usage: hypothesis codemod [OPTIONS] PATH...

  `hypothesis codemod` refactors deprecated or inefficient code.

  It adapts `python -m libcst.tool`, removing many features and config
  options which are rarely relevant for this purpose.  If you need more
  control, we encourage you to use the libcst CLI directly; if not this one
  is easier.

  PATH is the file(s) or directories of files to format in place, or "-" to
  read from stdin and write to stdout.

Options:
  -h, --help  Show this message and exit.

Alternatively you can use python -m libcst.tool, which offers more control at the cost of additional configuration (adding 'hypothesis.extra' to the modules list in .libcst.codemod.yaml) and some issues on Windows <https://github.com/Instagram/LibCST/issues/435>.

hypothesis.extra.codemods.refactor(code)

Update a source code string from deprecated to modern Hypothesis APIs.

This may not fix all the deprecation warnings in your code, but we're confident that it will be easier than doing it all by hand.

We recommend using the CLI, but if you want a Python function here it is.

Note:

This feature requires the hypothesis[dpcontracts] extra <>, via pip install hypothesis[dpcontracts].

Tip:

For new projects, we recommend using either deal <https://pypi.org/project/deal/> or icontract <https://pypi.org/project/icontract/> and icontract-hypothesis <https://pypi.org/project/icontract-hypothesis/> over dpcontracts <https://pypi.org/project/dpcontracts/>. They're generally more powerful tools for design-by-contract programming, and have substantially nicer Hypothesis integration too!

Warning:

This page documents internal Hypothesis interfaces. Some are fairly stable, while others are still experimental. In either case, they are not subject to our standard deprecation policy <#deprecation-policy>, and we might make breaking changes in minor or patch releases.

This page is intended for people building tools, libraries, or research on top of Hypothesis. If that includes you, please get in touch! We'd love to hear what you're doing, or explore more stable ways to support your use-case.

See also:

class hypothesis.internal.conjecture.providers.PrimitiveProvider(conjecturedata, /)

PrimitiveProvider is the implementation interface of a Hypothesis backend <#alternative-backends>.

A PrimitiveProvider is required to implement the following five draw_* methods:

• draw_integer()
• draw_boolean()
• draw_float()
• draw_string()
• draw_bytes()

Each strategy in Hypothesis generates values by drawing a series of choices from these five methods. By overriding them, a PrimitiveProvider can control the distribution of inputs generated by Hypothesis.

For example, hypothesis-crosshair <https://pypi.org/project/hypothesis-crosshair/> implements a PrimitiveProvider which uses an SMT solver to generate inputs that uncover new branches.

Once you implement a PrimitiveProvider, you can make it available for use through AVAILABLE_PROVIDERS.

lifetime = 'test_function'

The lifetime of a PrimitiveProvider instance. Either test_function or test_case.

If test_function (the default), a single provider instance will be instantiated and used for the entirety of each test function (i.e., roughly one provider per @given <#hypothesis.given> annotation). This can be useful for tracking state over the entirety of a test function.

If test_case, a new provider instance will be instantiated and used for each input Hypothesis generates.

The conjecturedata argument to PrimitiveProvider.__init__ will be None for a lifetime of test_function, and an instance of ConjectureData for a lifetime of test_case.

Third-party providers likely want to set a lifetime of test_function.

avoid_realization = False

Solver-based backends such as hypothesis-crosshair use symbolic values which record operations performed on them in order to discover new paths. If avoid_realization is set to True, hypothesis will avoid interacting with symbolic choices returned by the provider in any way that would force the solver to narrow the range of possible values for that symbolic.

Setting this to True disables some hypothesis features and optimizations. Only set this to True if it is necessary for your backend.

add_observability_callback = False

will never be called by Hypothesis.

The opt-in behavior of observability is because enabling observability might increase runtime or memory usage.

abstractmethod draw_boolean(p=0.5)

Draw a boolean choice.

Parameters

p (float) --

The probability of returning True. Between 0 and 1 inclusive.

Except for 0 and 1, the value of p is a hint provided by Hypothesis, and may be ignored by the backend.

If 0, the provider must return False. If 1, the provider must return True.

abstractmethod draw_integer(min_value=None, max_value=None, *, weights=None, shrink_towards=0)

Draw an integer choice.

Parameters

• min_value (int | None) -- (Inclusive) lower bound on the integer value. If None, there is no lower bound.
• max_value (int | None) -- (Inclusive) upper bound on the integer value. If None, there is no upper bound.
• weights (dict[int, float] | None) -- Maps keys in the range [min_value, max_value] to the probability of returning that key.
• shrink_towards (int) -- The integer to shrink towards. This is not used during generation and can be ignored by backends.

abstractmethod draw_float(*, min_value=-inf, max_value=inf, allow_nan=True, smallest_nonzero_magnitude)

Draw a float choice.

Parameters

• min_value (float) -- (Inclusive) lower bound on the float value.
• max_value (float) -- (Inclusive) upper bound on the float value.
• allow_nan (bool) -- If False, it is invalid to return math.nan.
• smallest_nonzero_magnitude (float) -- The smallest allowed nonzero magnitude. draw_float should not return a float f if abs(f) < smallest_nonzero_magnitude.

abstractmethod draw_string(intervals, *, min_size=0, max_size=10000000000)

Draw a string choice.

Parameters

• intervals (IntervalSet <#hypothesis.internal.intervalsets.IntervalSet>) -- The set of codepoints to sample from.
• min_size (int) -- (Inclusive) lower bound on the string length.
• max_size (int) -- (Inclusive) upper bound on the string length.

abstractmethod draw_bytes(min_size=0, max_size=10000000000)

Draw a bytes choice.

Parameters

• min_size (int) -- (Inclusive) lower bound on the bytes length.
• max_size (int) -- (Inclusive) upper bound on the bytes length.

per_test_case_context_manager()

Returns a context manager which will be entered each time Hypothesis starts generating and executing one test case, and exited when that test case finishes generating and executing, including if any exception is thrown.

In the lifecycle of a Hypothesis test, this is called before generating strategy values for each test case. This is just before any custom executor <#custom-function-execution> is called.

Even if not returning a custom context manager, PrimitiveProvider subclasses are welcome to override this method to know when Hypothesis starts and ends the execution of a single test case.

realize(value, *, for_failure=False)

Called whenever hypothesis requires a concrete (non-symbolic) value from a potentially symbolic value. Hypothesis will not check that value is symbolic before calling realize, so you should handle the case where value is non-symbolic.

The returned value should be non-symbolic.  If you cannot provide a value, raise BackendCannotProceed with a value of "discard_test_case".

If for_failure is True, the value is associated with a failing example. In this case, the backend should spend substantially more effort when attempting to realize the value, since it is important to avoid discarding failing examples. Backends may still raise BackendCannotProceed when for_failure is True, if realization is truly impossible or if realization takes significantly longer than expected (say, 5 minutes).

replay_choices(choices)

Called when Hypothesis has discovered a choice sequence which the provider may wish to enqueue to replay under its own instrumentation when we next ask to generate a test case, rather than generating one from scratch.

This is used to e.g. warm-start hypothesis-crosshair <https://pypi.org/project/hypothesis-crosshair/> with a corpus of high-code-coverage inputs discovered by HypoFuzz <https://hypofuzz.com/>.

observe_test_case()

Called at the end of the test case when observability <#observability> is enabled.

The return value should be a non-symbolic json-encodable dictionary, and will be included in observations as observation["metadata"]["backend"].

observe_information_messages(*, lifetime)

Called at the end of each test case and again at end of the test function.

Return an iterable of {type: info/alert/error, title: str, content: str | dict} dictionaries to be delivered as individual information messages. Hypothesis adds the run_start timestamp and property name for you.

on_observation(observation)

Called at the end of each test case which uses this provider, with the same observation["type"] == "test_case" observation that is passed to other callbacks added via add_observability_callback. This method is not called with observation["type"] in {"info", "alert", "error"} observations.

Important:

For on_observation() to be called by Hypothesis, add_observability_callback must be set to True.

on_observation() is explicitly opt-in, as enabling observability might increase runtime or memory usage.

Calls to this method are guaranteed to alternate with calls to per_test_case_context_manager(). For example:

# test function starts
per_test_case_context_manager()
on_observation()
per_test_case_context_manager()
on_observation()
...
# test function ends

Note that on_observation() will not be called for test cases which did not use this provider during generation, for example during Phase.reuse <#hypothesis.Phase.reuse> or Phase.shrink <#hypothesis.Phase.shrink>, or because Hypothesis switched to the standard Hypothesis backend after this backend raised too many BackendCannotProceed exceptions.

span_start(label, /)

Marks the beginning of a semantically meaningful span of choices.

Spans are a depth-first tree structure. A span is opened by a call to span_start(), and a call to span_end() closes the most recently opened span. So the following sequence of calls:

span_start(label=1)
n1 = draw_integer()
span_start(label=2)
b1 = draw_boolean()
n2 = draw_integer()
span_end()
f1 = draw_float()
span_end()

produces the following two spans of choices:

1: [n1, b1, n2, f1]
2: [b1, n2]

Hypothesis uses spans to denote "semantically meaningful" sequences of choices. For instance, Hypothesis opens a span for the sequence of choices made while drawing from each strategy. Not every span corresponds to a strategy; the generation of e.g. each element in lists() <#hypothesis.strategies.lists> is also marked with a span, among others.

label is an opaque integer, which has no defined semantics. The only guarantee made by Hypothesis is that all spans with the same "meaning" will share the same label. So all spans from the same strategy will share the same label, as will e.g. the spans for lists() <#hypothesis.strategies.lists> elements.

Providers can track calls to span_start() and span_end() to learn something about the semantics of the test's choice sequence. For instance, a provider could track the depth of the span tree, or the number of unique labels, which says something about the complexity of the choices being generated. Or a provider could track the span tree across test cases in order to determine what strategies are being used in what contexts.

It is possible for Hypothesis to start and immediately stop a span, without calling a draw_* method in between. These spans contain zero choices.

Hypothesis will always balance the number of calls to span_start() and span_end(). A call to span_start() will always be followed by a call to span_end() before the end of the test case.

span_start() is called from ConjectureData.start_span() internally.

span_end(discard, /)

Marks the end of a semantically meaningful span of choices.

discard is True when the draw was filtered out or otherwise marked as unlikely to contribute to the input data as seen by the user's test. Note however that side effects can make this determination unsound.

span_end() is called from ConjectureData.stop_span() internally.

hypothesis.internal.conjecture.providers.AVAILABLE_PROVIDERS

Registered Hypothesis backends. This is a dictionary where keys are the name to be used in settings.backend <#hypothesis.settings.backend>. The value of a key can be either:

• A string corresponding to an importable absolute path of a PrimitiveProvider subclass
• A PrimitiveProvider subclass (the class itself, not an instance of the class)

Hypothesis will instantiate the corresponding PrimitiveProvider subclass when the backend is requested by a test's settings.backend <#hypothesis.settings.backend> value.

For example, the default Hypothesis backend is registered as:

from hypothesis.internal.conjecture.providers import AVAILABLE_PROVIDERS

AVAILABLE_PROVIDERS["hypothesis"] = "hypothesis.internal.conjecture.providers.HypothesisProvider"
# or
AVAILABLE_PROVIDERS["hypothesis"] = HypothesisProvider

And can be used with:

from hypothesis import given, settings, strategies as st

@given(st.integers())
@settings(backend="hypothesis")
def f(n):
    pass

Though, as backend="hypothesis" is the default setting, the above would typically not have any effect.

For third-party backend authors, we strongly encourage ensuring that import hypothesis does not automatically import the expensive parts of your package, by:

• setting a string path here, instead of a provider class
• ensuring the registered hypothesis plugin path references a path which just sets AVAILABLE_PROVIDERS and does not import your package

hypothesis.internal.conjecture.provider_conformance.run_conformance_test(Provider, *, context_manager_exceptions=(), settings=None, _realize_objects=st.from_type(object) | st.from_type(type).flatmap(st.from_type))

Test that the given Provider class conforms to the PrimitiveProvider interface.

For instance, this tests that Provider does not return out of bounds choices from any of the draw_* methods, or violate other invariants which Hypothesis depends on.

This function is intended to be called at test-time, not at runtime. It is provided by Hypothesis to make it easy for third-party backend authors to test their provider. Backend authors wishing to test their provider should include a test similar to the following in their test suite:

from hypothesis.internal.conjecture.provider_conformance import run_conformance_test

def test_conformance():
    run_conformance_test(MyProvider)

If your provider can raise control flow exceptions inside one of the five draw_* methods that are handled by your provider's per_test_case_context_manager, pass a list of these exceptions types to context_manager_exceptions. Otherwise, run_conformance_test will treat those exceptions as fatal errors.

class hypothesis.errors.BackendCannotProceed(scope='other', /)

Raised by alternative backends when a PrimitiveProvider cannot proceed. This is expected to occur inside one of the .draw_*() methods, or for symbolic execution perhaps in realize().

The optional scope argument can enable smarter integration:

verified:

Do not request further test cases from this backend.  We may generate more test cases with other backends; if one fails then Hypothesis will report unsound verification in the backend too.

exhausted:

Do not request further test cases from this backend; finish testing with test cases generated with the default backend.  Common if e.g. native code blocks symbolic reasoning very early.

discard_test_case:

This particular test case could not be converted to concrete values; skip any further processing and continue with another test case from this backend.

final class hypothesis.internal.intervalsets.IntervalSet(intervals=())

A compact and efficient representation of a set of (a, b) intervals. Can be treated like a set of integers, in that n in intervals will return True if n is contained in any of the (a, b) intervals, and False otherwise.

hypothesis.internal.observability.add_observability_callback(f, /, *, all_threads=False)

Adds f as a callback for observability <#observability>. f should accept one argument, which is an observation. Whenever Hypothesis produces a new observation, it calls each callback with that observation.

If Hypothesis tests are being run from multiple threads, callbacks are tracked per-thread. In other words, add_observability_callback(f) only adds f as an observability callback for observations produced on that thread.

If all_threads=True is passed, f will instead be registered as a callback for all threads. This means it will be called for observations generated by all threads, not just the thread which registered f as a callback. In this case, f will be passed two arguments: the first is the observation, and the second is the integer thread id from python:threading.get_ident() where that observation was generated.

We recommend against registering f as a callback for both all_threads=True and the default all_threads=False, due to unclear semantics with remove_observability_callback.

hypothesis.internal.observability.remove_observability_callback(f, /)

Removes f from the observability <#observability> callbacks.

If f is not in the list of observability callbacks, silently do nothing.

If running under multiple threads, f will only be removed from the callbacks for this thread.

hypothesis.internal.observability.with_observability_callback(f, /, *, all_threads=False)

A simple context manager which calls add_observability_callback on f when it enters and remove_observability_callback on f when it exits.

hypothesis.internal.observability.observability_enabled()

Returns whether or not Hypothesis considers observability <#observability> to be enabled. Observability is enabled if there is at least one observability callback present.

Callers might use this method to determine whether they should compute an expensive representation that is only used under observability, for instance by alternative backends <#alternative-backends>.

hypothesis.internal.observability.TESTCASE_CALLBACKS = <hypothesis.internal.observability._TestcaseCallbacks object>

Warning:

Deprecated in favor of add_observability_callback, remove_observability_callback, and observability_enabled.

TESTCASE_CALLBACKS remains a thin compatibility shim which forwards .append, .remove, and bool() to those three methods. It is not an attempt to be fully compatible with the previous TESTCASE_CALLBACKS = [], so iteration or other usages will not work anymore. Please update to using the new methods instead.

TESTCASE_CALLBACKS will eventually be removed.

hypothesis.internal.observability.OBSERVABILITY_COLLECT_COVERAGE = True

If False, do not collect coverage information when observability is enabled.

This is exposed both for performance (as coverage collection can be slow on Python 3.11 and earlier) and size (if you do not use coverage information, you may not want to store it in-memory).

hypothesis.internal.observability.OBSERVABILITY_CHOICES = False

If True, include the metadata.choice_nodes and metadata.spans keys in test case observations.

False by default. metadata.choice_nodes and metadata.spans can be a substantial amount of data, and so must be opted-in to, even when observability is enabled.

Warning:

EXPERIMENTAL AND UNSTABLE. We are actively working towards a better interface for this as of June 2025, and this attribute may disappear or be renamed without notice.

We pick reasonable values for these constants, but if you must, you can monkeypatch them. (Hypothesis is not responsible for any performance degradation that may result).

hypothesis.internal.conjecture.engine.MAX_SHRINKS = 500

The maximum number of times the shrinker will reduce the complexity of a failing input before giving up. This avoids falling down a trap of exponential (or worse) complexity, where the shrinker appears to be making progress but will take a substantially long time to finish completely.

hypothesis.internal.conjecture.engine.MAX_SHRINKING_SECONDS = 300

The maximum total time in seconds that the shrinker will try to shrink a failure for before giving up. This is across all shrinks for the same failure, so even if the shrinker successfully reduces the complexity of a single failure several times, it will stop when it hits MAX_SHRINKING_SECONDS of total time taken.

hypothesis.internal.conjecture.engine.BUFFER_SIZE = 8192

The maximum amount of entropy a single test case can use before giving up while making random choices during input generation.

The "unit" of one BUFFER_SIZE does not have any defined semantics, and you should not rely on it, except that a linear increase BUFFER_SIZE will linearly increase the amount of entropy a test case can use during generation.

Note:

See also How not to Die Hard with Hypothesis <https://hypothesis.works/articles/how-not-to-die-hard-with-hypothesis/> and An Introduction to Rule-Based Stateful Testing <https://hypothesis.works/articles/rule-based-stateful-testing/>.

With @given <#hypothesis.given>, your tests are still something that you mostly write yourself, with Hypothesis providing some data. With Hypothesis's stateful testing, Hypothesis instead tries to generate not just data but entire tests. You specify a number of primitive actions that can be combined together, and then Hypothesis will try to find sequences of those actions that result in a failure.

The basic idea of stateful testing is to make Hypothesis choose actions as well as values for your test, and state machines are a great declarative way to do just that.

For simpler cases though, you might not need them at all - a standard test with @given <#hypothesis.given> might be enough, since you can use data() <#hypothesis.strategies.data> in branches or loops.  In fact, that's how the state machine explorer works internally.  For more complex workloads though, where a higher level API comes into it's own, keep reading!

A state machine is very similar to a normal @given <#hypothesis.given> based test in that it takes values drawn from strategies and passes them to a user defined test function, which may use assertions to check the system's behavior. The key difference is that where @given <#hypothesis.given> based tests must be independent, rules can be chained together - a single test run may involve multiple rule invocations, which may interact in various ways.

Rules can take normal strategies as arguments, but normal strategies, with the exception of  runner() <#hypothesis.strategies.runner> and data() <#hypothesis.strategies.data>, cannot take into account the current state of the machine. This is where bundles come in.

A rule can, in place of a normal strategy, take a Bundle <#hypothesis.stateful.Bundle>. A hypothesis.stateful.Bundle <#hypothesis.stateful.Bundle> is a named collection of generated values that can be reused by other operations in the test. They are populated with the results of rules, and may be used as arguments to rules, allowing data to flow from one rule to another, and rules to work on the results of previous computations or actions.

Specifically, a rule that specifies target=a_bundle will cause its return value to be added to that bundle. A rule that specifies an_argument=a_bundle as a strategy will draw a value from that bundle.  A rule can also specify that an argument chooses a value from a bundle and removes that value by using consumes() <#hypothesis.stateful.consumes> as in an_argument=consumes(a_bundle).

Note:

The following rule based state machine example is a simplified version of a test for Hypothesis's example database implementation. An example database maps keys to sets of values, and in this test we compare one implementation of it to a simplified in memory model of its behaviour, which just stores the same values in a Python dict. The test then runs operations against both the real database and the in-memory representation of it and looks for discrepancies in their behaviour.

import shutil
import tempfile
from collections import defaultdict

import hypothesis.strategies as st
from hypothesis.database import DirectoryBasedExampleDatabase
from hypothesis.stateful import Bundle, RuleBasedStateMachine, rule

class DatabaseComparison(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.tempd = tempfile.mkdtemp()
        self.database = DirectoryBasedExampleDatabase(self.tempd)
        self.model = defaultdict(set)

    keys = Bundle("keys")
    values = Bundle("values")

    @rule(target=keys, k=st.binary())
    def add_key(self, k):
        return k

    @rule(target=values, v=st.binary())
    def add_value(self, v):
        return v

    @rule(k=keys, v=values)
    def save(self, k, v):
        self.model[k].add(v)
        self.database.save(k, v)

    @rule(k=keys, v=values)
    def delete(self, k, v):
        self.model[k].discard(v)
        self.database.delete(k, v)

    @rule(k=keys)
    def values_agree(self, k):
        assert set(self.database.fetch(k)) == self.model[k]

    def teardown(self):
        shutil.rmtree(self.tempd)

TestDBComparison = DatabaseComparison.TestCase

In this we declare two bundles - one for keys, and one for values. We have two trivial rules which just populate them with data (k and v), and three non-trivial rules: save saves a value under a key and delete removes a value from a key, in both cases also updating the model of what should be in the database. values_agree then checks that the contents of the database agrees with the model for a particular key.

Note:

We can now integrate this into our test suite by getting a unittest TestCase from it:

TestTrees = DatabaseComparison.TestCase

# Or just run with pytest's unittest support
if __name__ == "__main__":
    unittest.main()

This test currently passes, but if we comment out the line where we call self.model[k].discard(v), we would see the following output when run under pytest:

AssertionError: assert set() == {b''}

------------ Hypothesis ------------

state = DatabaseComparison()
var1 = state.add_key(k=b'')
var2 = state.add_value(v=var1)
state.save(k=var1, v=var2)
state.delete(k=var1, v=var2)
state.values_agree(k=var1)
state.teardown()

Note how it's printed out a very short program that will demonstrate the problem. The output from a rule based state machine should generally be pretty close to Python code - if you have custom repr implementations that don't return valid Python then it might not be, but most of the time you should just be able to copy and paste the code into a test to reproduce it.

You can control the detailed behaviour with a settings object on the TestCase (this is a normal hypothesis settings object using the defaults at the time the TestCase class was first referenced). For example if you wanted to run fewer examples with larger programs you could change the settings to:

DatabaseComparison.TestCase.settings = settings(
    max_examples=50, stateful_step_count=100
)

Which doubles the number of steps each program runs and halves the number of test cases that will be run.

As said earlier, rules are the most common feature used in RuleBasedStateMachine. They are defined by applying the rule() <#hypothesis.stateful.rule> decorator on a function. Note that RuleBasedStateMachine must have at least one rule defined and that a single function cannot be used to define multiple rules (this is to avoid having multiple rules doing the same things). Due to the stateful execution method, rules generally cannot take arguments from other sources such as fixtures or pytest.mark.parametrize - consider providing them via a strategy such as sampled_from() <#hypothesis.strategies.sampled_from> instead.

Initializes are a special case of rules, which are guaranteed to be run exactly once before any normal rule is called. Note if multiple initialize rules are defined, they will all be called but in any order, and that order will vary from run to run.

Initializes are typically useful to populate bundles:

import hypothesis.strategies as st
from hypothesis.stateful import Bundle, RuleBasedStateMachine, initialize, rule

name_strategy = st.text(min_size=1).filter(lambda x: "/" not in x)

class NumberModifier(RuleBasedStateMachine):
    folders = Bundle("folders")
    files = Bundle("files")

    @initialize(target=folders)
    def init_folders(self):
        return "/"

    @rule(target=folders, parent=folders, name=name_strategy)
    def create_folder(self, parent, name):
        return f"{parent}/{name}"

    @rule(target=files, parent=folders, name=name_strategy)
    def create_file(self, parent, name):
        return f"{parent}/{name}"

Initializes can also allow you to initialize the system under test in a way that depends on values chosen from a strategy. You could do this by putting an instance variable in the state machine that indicates whether the system under test has been initialized or not, and then using preconditions (below) to ensure that exactly one of the rules that initialize it get run before any rules that depend on it being initialized.

While it's possible to use assume() <#hypothesis.assume> in RuleBasedStateMachine rules, if you use it in only a few rules you can quickly run into a situation where few or none of your rules pass their assumptions. Thus, Hypothesis provides a precondition() <#hypothesis.stateful.precondition> decorator to avoid this problem. The precondition() <#hypothesis.stateful.precondition> decorator is used on rule-decorated functions, and must be given a function that returns True or False based on the RuleBasedStateMachine instance.

from hypothesis.stateful import RuleBasedStateMachine, precondition, rule

class NumberModifier(RuleBasedStateMachine):
    num = 0

    @rule()
    def add_one(self):
        self.num += 1

    @precondition(lambda self: self.num != 0)
    @rule()
    def divide_with_one(self):
        self.num = 1 / self.num

By using precondition() <#hypothesis.stateful.precondition> here instead of assume() <#hypothesis.assume>, Hypothesis can filter the inapplicable rules before running them. This makes it much more likely that a useful sequence of steps will be generated.

Note that currently preconditions can't access bundles; if you need to use preconditions, you should store relevant data on the instance instead.

Often there are invariants that you want to ensure are met after every step in a process.  It would be possible to add these as rules that are run, but they would be run zero or multiple times between other rules. Hypothesis provides a decorator that marks a function to be run after every step.

from hypothesis.stateful import RuleBasedStateMachine, invariant, rule

class NumberModifier(RuleBasedStateMachine):
    num = 0

    @rule()
    def add_two(self):
        self.num += 2
        if self.num > 50:
            self.num += 1

    @invariant()
    def is_even(self):
        assert self.num % 2 == 0

NumberTest = NumberModifier.TestCase

Invariants can also have precondition() <#hypothesis.stateful.precondition>s applied to them, in which case they will only be run if the precondition function returns true.

Note that currently invariants can't access bundles; if you need to use invariants, you should store relevant data on the instance instead.

If you want to bypass the TestCase infrastructure you can invoke these manually. The stateful module exposes the function run_state_machine_as_test, which takes an arbitrary function returning a RuleBasedStateMachine and an optional settings parameter and does the same as the class based runTest provided.

Hypothesis has minimal dependencies, to maximise compatibility and make installing Hypothesis as easy as possible.

Our integrations with specific packages are therefore provided by extra modules that need their individual dependencies installed in order to work. You can install these dependencies using the setuptools extra feature as e.g. pip install hypothesis[django]. This will check installation of compatible versions.

You can also just install hypothesis into a project using them, ignore the version constraints, and hope for the best.

In general "Which version is Hypothesis compatible with?" is a hard question to answer and even harder to regularly test. Hypothesis is always tested against the latest compatible version and each package will note the expected compatibility range. If you run into a bug with any of these please specify the dependency version.

The following lists the available extras in Hypothesis and their documentation.

• hypothesis[cli] <#hypothesis-cli>
• hypothesis[codemods] <#codemods>
• hypothesis[django] <#hypothesis-django>
• hypothesis[numpy] <#hypothesis-numpy>
• hypothesis[lark] <#hypothesis-lark>
• hypothesis[pytz] <#hypothesis-pytz>
• hypothesis[dateutil] <#hypothesis-dateutil>
• hypothesis[dpcontracts] <#hypothesis-dpcontracts>

This is a record of all past Hypothesis releases and what went into them, in reverse chronological order. All previous releases are still available on PyPI <https://pypi.org/project/hypothesis/>.

Remove some old unused code.

This patch fixes a crash when sys.modules contains unhashable values, such as SimpleNamespace objects (issue #4660 <https://github.com/HypothesisWorks/hypothesis/issues/4660>).

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

This patch fixes several duplicate word typos in comments and documentation.

This patch teaches our pytest plugin to :ref:` find interesting constants <v6.131.1>` when pytest is collecting tests, to avoid arbitrarily attributing the latency to whichever test function happened to be executed first (issue #4627 <https://github.com/HypothesisWorks/hypothesis/issues/4627>).

This patch adjusts how we compute the stopping threshold introduced in version 6.151.3 <changelog.html#v6-151-3>, while still maintaining 99% confidence that <1% of test cases pass.

This patch makes Hypothesis more tolerant of slow-to-satisfy assume() calls. Previously, Hypothesis would give up after max_examples * 10 attempts; now it uses a statistical test to stop only when 99% confident that <1% of examples would pass (issue #4623 <https://github.com/HypothesisWorks/hypothesis/issues/4623>).

Thanks to @ajdavis for this improvement!

Format our code with the latest version of black <https://pypi.org/project/black/>.

Improve internal categorization of test cases when an alternative backend <#alternative-backends> raises BackendCannotProceed <#hypothesis.errors.BackendCannotProceed>.

Add 2025.12 to the list of recognized Array API versions in hypothesis.extra.array_api.

Hypothesis now generates powers of 2 more often when using integers() <#hypothesis.strategies.integers>.

Update some internal type hints.

This patch fixes a bug where recursive() <#hypothesis.strategies.recursive> would fail in cases where the extend= function does not reference it's argument - which was assumed by the recent min_leaves= feature, because the strategy can't actually recurse otherwise.  (issue #4638 <https://github.com/HypothesisWorks/hypothesis/issues/4638>)

Now, the historical behavior is working-but-deprecated, or an error if you explicitly pass min_leaves=.

This release adds a min_leaves argument to recursive() <#hypothesis.strategies.recursive>, which ensures that generated recursive structures have at least the specified number of leaf nodes (issue #4205 <https://github.com/HypothesisWorks/hypothesis/issues/4205>).

Add type hints to an internal class.

This release extends the explain-phase # or any other generated value comments to sub-arguments within builds() <#hypothesis.strategies.builds>, tuples() <#hypothesis.strategies.tuples>, and fixed_dictionaries() <#hypothesis.strategies.fixed_dictionaries>.

Previously, these comments only appeared on top-level test arguments. Now, when the explain phase determines that a sub-argument can vary freely without affecting the test failure, you'll see comments like:

Falsifying example: test_foo(
    obj=MyClass(
        x=0,  # or any other generated value
        y=True,
    ),
    data=(
        '',  # or any other generated value
        42,
    ),
)

This makes it easier to understand which parts of complex inputs actually matter for reproducing a failure.

Clean up an internal helper.

This patch fixes from_type() <#hypothesis.strategies.from_type> to properly handle parameterized type aliases created with Python 3.12+'s PEP 695 <https://peps.python.org/pep-0695/> type statement. For example, st.from_type(A[int]) where type A[T] = list[T] now correctly resolves to lists(integers()) instead of raising a TypeError (issue #4628 <https://github.com/HypothesisWorks/hypothesis/issues/4628>).

Hypothesis now prints a Verbosity.verbose <#hypothesis.Verbosity.verbose> log when we switch away from an alternative backend <#alternative-backends>.

Fixes Ghostwriter <#ghostwriter> output for numpy <https://pypi.org/project/numpy/> >= 2.4.0. Also adds support from_type() <#hypothesis.strategies.from_type> for numpy <https://pypi.org/project/numpy/> 2.5.0 nightly (which has not yet been released).

.example() <#hypothesis.strategies.SearchStrategy.example> no longer emits NonInteractiveExampleWarning <#hypothesis.errors.NonInteractiveExampleWarning> when running a python file directly. This means that e.g. python my_sandbox.py during exploratory work with .example() <#hypothesis.strategies.SearchStrategy.example> will no longer raise warnings.

Add __dict__ and __proto__ to the list of constant strings Hypothesis sometimes generates.

When multiple explicit @example <#hypothesis.example> decorators fail with the same error, Hypothesis now shows only the simplest failing example (by shortlex order) with a note about how many other examples also failed (issue #4520 <https://github.com/HypothesisWorks/hypothesis/issues/4520>).

To see all failing examples, use Verbosity.verbose <#hypothesis.Verbosity.verbose> or higher.

Fix a bug where we persisted symbolics from solver-based alternative backends <#alternative-backends> in event() <#hypothesis.event>.

This patch improves the error message for FlakyStrategyDefinition <#hypothesis.errors.FlakyStrategyDefinition> when the precondition for a rule is flaky (issue #4206 <https://github.com/HypothesisWorks/hypothesis/issues/4206>).

This patch improves the type annotations for basic_indices() <#hypothesis.extra.numpy.basic_indices>. The return type now accurately reflects the allow_ellipsis and allow_newaxis parameters, excluding EllipsisType or None from the union when those index types are disabled (issue #4607 <https://github.com/HypothesisWorks/hypothesis/issues/4607>).

Additionally, assume() <#hypothesis.assume> now has overloaded type annotations: assume(True) returns Literal[True], while assume(False) and assume(None) return NoReturn.

Clean up some internal code.

Document fuzz_one_input() <#hypothesis.core.HypothesisHandle.fuzz_one_input>.

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

Calling register_profile() <#hypothesis.settings.register_profile> from within a test decorated with @settings <#hypothesis.settings> is now deprecated, to avoid confusion about which settings are used as the baseline for the new profile.

This release drops support for nose <https://pypi.org/project/nose/>, which ceased development 9 years ago and does not support Python 3.10 or newer.

Hypothesis still supports nose2 <https://pypi.org/project/nose2/>. While we do not test nose2 in our CI, we will fix any bugs that get reported.

@settings <#hypothesis.settings> now accepts equivalent string representations for settings.verbosity <#hypothesis.settings.verbosity>, settings.phases <#hypothesis.settings.phases>, and settings.suppress_health_check <#hypothesis.settings.suppress_health_check>. For example:

# these two are now equivalent...
settings(verbosity=Verbosity.verbose)
settings(verbosity="verbose")

# ...as are these two...
settings(phases=[Phase.explicit])
settings(phases=["explicit"])

# ...and these two.
settings(suppress_health_check=[HealthCheck.filter_too_much])
settings(suppress_health_check=["filter_too_much"])

This release also changes the canonical value of Verbosity <#hypothesis.Verbosity>, Phase <#hypothesis.Phase>, and HealthCheck <#hypothesis.HealthCheck> members to a string instead of an integer. For example, Phase.reuse.value == "explicit" as of this release, where previously Phase.reuse.value == 1.

Instantiating Verbosity <#hypothesis.Verbosity>, Phase <#hypothesis.Phase>, or HealthCheck <#hypothesis.HealthCheck> with an integer, such as Verbosity(0), is now deprecated.

Refactor some internal logic around strategy definitions.

Hypothesis previously required attrs <https://pypi.org/project/attrs/> as a dependency. This release removes that dependency, so that the only required dependency of Hypothesis is sortedcontainers <https://pypi.org/project/sortedcontainers/>.

All attrs-specific features of Hypothesis, such as using from_type() <#hypothesis.strategies.from_type> with attrs classes, will continue to behave as before.

Tweak how Hypothesis hides internal tracebacks to fix an error under rare conditions (issue #3822 <https://github.com/HypothesisWorks/hypothesis/issues/3822>).

This release adds support for Fraction objects as min_value and max_value bounds in decimals() <#hypothesis.strategies.decimals>, if they can be exactly represented as decimals in the target precision (issue #4466 <https://github.com/HypothesisWorks/hypothesis/issues/4466>).

Bounding decimals() <#hypothesis.strategies.decimals> with other values that cannot be exactly represented is now deprecated; previously the bounds could be off by one.

from_type() <#hypothesis.strategies.from_type> now correctly handles annotated-types <https://pypi.org/project/annotated-types/> annotations on typing.TypedDict fields which are also marked as being ReadOnly, Required, or NotRequired (issue #4474 <https://github.com/HypothesisWorks/hypothesis/issues/4474>).

The extras for NumPy <#hypothesis-numpy> and pandas <#hypothesis-pandas> now support automatically inferring a strategy for dtype="O". Previously, Hypothesis required an explicit elements strategy to be passed, for example nps.arrays("O", shape=(1,), elements=st.just(object())). Now, Hypothesis automatically infers elements=st.from_type(object).

Thanks to Shaun Read for identifying and fixing this!

This patch fixes binary_operation() <#hypothesis.extra.ghostwriter.binary_operation> to include imports for hypothesis.extra.numpy <#module-hypothesis.extra.numpy> strategies such as arrays() <#hypothesis.extra.numpy.arrays>, scalar_dtypes() <#hypothesis.extra.numpy.scalar_dtypes>, and array_shapes() <#hypothesis.extra.numpy.array_shapes> when ghostwriting tests for functions with numpy array parameters (issue #4576 <https://github.com/HypothesisWorks/hypothesis/issues/4576>).

Improve the accuracy of test timing reports, by tracking the start time of each test case closer to when the test is executed.

Fix a recursion error when observability <#observability> is enabled and a test generates an object with a recursive reference, like a = []; a.append(a).

Remove a case where Hypothesis would interact with the global python:random.Random instance if Hypothesis internals were used directly.

Simplify some internal typing logic after dropping Python 3.9.

This release drops support for Python 3.9, which reached end of life in October 2025 <https://devguide.python.org/versions/>.

Fixes an error when using the Ghostwriter <#ghostwriter> with annotations that include python:typing.ForwardRef on Python 3.14 (issue #4565 <https://github.com/HypothesisWorks/hypothesis/issues/4565>).

The from_field() <#hypothesis.extra.django.from_field> and from_form() <#hypothesis.extra.django.from_form> strategies from our Django extra <#hypothesis-django> now support FileField.

Thanks to Arjoonn Sharma for this fix!

Clean up internal @overload type annotations.

Fixes our bundled run_conformance_test() <#hypothesis.internal.conjecture.provider_conformance.run_conformance_test> not respecting avoid_realization <#hypothesis.internal.conjecture.providers.PrimitiveProvider.avoid_realization>.

The automatic switch to the CI settings profile <#hypothesis.settings> now works under tox <https://pypi.org/project/tox/> (for tox >= 4.30.0).

This patch re-enables the warning for incompatible shared() <#hypothesis.strategies.shared> strategies that was first enabled in v6.133.0 <changelog.html#v6-133-0> but disabled in v6.135.15 <changelog.html#v6-135-15>.

characters() <#hypothesis.strategies.characters> now validates that the elements of the exclude_characters and include_characters arguments are single characters, which was always assumed internally. For example, exclude_characters=["a", "b"] is valid while exclude_characters=["ab"] will now raise an error up-front.

Add phase to the hypothesis-specific metadata <#observability-hypothesis-metadata> in observability <#observability>.

Internal refactoring for new lint rules.

Fixed another typo in error message around function-scoped fixtures.

Add get_current_profile_name() <#hypothesis.settings.get_current_profile_name>, which returns the name of the current settings profile.

Fixed typo in error message around function-scoped fixtures.

Improved error message for DeadlineExceeded <#hypothesis.errors.DeadlineExceeded>.

Refactor some stateful testing internals for easier use by third-party libraries.

Patch files written by hypothesis now use a deterministic ordering when multiple @example <#hypothesis.example> decorators are present.

Fix a typo affecting pretty-printing of lambdas with complex default arguments.

Improve automatic detection of the CI profile <#builtin-profiles> on various vendor-specific CI systems.

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

Internal refactor to simplify SearchStrategy <#hypothesis.strategies.SearchStrategy>.

This patch further improves stringification of lambdas, by never returning a lambda source unless it is confirmed to compile to the same code object. This stricter check makes it possible to widen the search for a matching source block, so that it can often be found even if the file has been edited.

Fixes a race condition under threading when using deferred() <#hypothesis.strategies.deferred>.

Improves upon the cache eviction problem workaround of v6.135.12 <changelog.html#v6-135-12>.

Documentation tweaks.

Fixes a race condition under threading for strategies which trigger our filter-rewriting rules, like st.integers().filter(lambda x: abs(x) > 100).

One of our shrinking passes for reducing failing inputs targets failures which require two numbers to add to the same value. This pass previously only worked for positive numbers. This patch fixes that, so it also works for negative numbers.

This patch slightly improves the cache-hit rate for dictionaries() <#hypothesis.strategies.dictionaries> and certain unique lists() <#hypothesis.strategies.lists>.

The type annotations for register_type_strategy() <#hypothesis.strategies.register_type_strategy> now indicate that it accepts registering types created with python:typing.TypeAliasType (aka type MyType = int).

Internal refactoring and cleanup. As a result, hypothesis[cli] and hypothesis[ghostwriter] now require black>=20.8b0 instead of the previous black>=19.10b0.

On Python 3.14, python:memoryview is newly generic. This release adds the ability for from_type() <#hypothesis.strategies.from_type> to resolve generic python:memoryview types on 3.14, like st.from_type(memoryview[CustomBufferClass]) . CustomBufferClass must implement __buffer__, as expected by python:memoryview.

This patch makes the stringification of lambdas, and as a result certain automatic filter rewriting operations, more robust. This fixes issue #4498 <https://github.com/HypothesisWorks/hypothesis/issues/4498>, where a lambda was mistakenly identified as the identity operator due to inspect.getsource() only returning the first line of the lambda definition.

As a result, the repr of strategies filtered or mapped by lambda functions may change slightly.

Add support for Python 3.14, which is currently in release candidate 1 <https://devguide.python.org/versions/>.

Fixes a bug with solver-based alternative backends <#alternative-backends> (like crosshair <https://github.com/pschanely/CrossHair>) where symbolic values passed to event() <#hypothesis.event> would not be realized to concrete values at the end of the test case.

Add the add_observability_callback <#hypothesis.internal.observability.add_observability_callback>, remove_observability_callback <#hypothesis.internal.observability.remove_observability_callback>, with_observability_callback <#hypothesis.internal.observability.with_observability_callback>, and observability_enabled <#hypothesis.internal.observability.observability_enabled> methods to the observability <#observability> interface. The previous TESTCASE_CALLBACKS <#hypothesis.internal.observability.TESTCASE_CALLBACKS> is deprecated.

This release also adds better threading support to observability callbacks. An observability callback will now only be called for observations generated by the same thread.

Fix a threading race condition in one_of() <#hypothesis.strategies.one_of> initialization.

Improve the error messages and documentation for HealthCheck <#hypothesis.HealthCheck>. Among others, the messaging is now more clear that health checks are proactive warnings, not correctness errors.

Improve detection of sys.monitoring to avoid errors on GraalPy.

When a test is executed concurrently from multiple threads, HealthCheck.too_slow <#hypothesis.HealthCheck.too_slow> is now disabled, since the Python runtime may decide to switch away from a thread for arbitrarily long and Hypothesis cannot track execution time per-thread.

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

HealthCheck.differing_executors <#hypothesis.HealthCheck.differing_executors> is no longer raised if a test is executed by different executors from different threads. HealthCheck.differing_executors <#hypothesis.HealthCheck.differing_executors> will still be raised if a test is executed by different executors in the same thread.

When a test is executed concurrently from multiple threads, DeadlineExceeded <#hypothesis.errors.DeadlineExceeded> is now disabled, since the Python runtime may decide to switch away from a thread for longer than settings.deadline <#hypothesis.settings.deadline>, and Hypothesis cannot track execution time per-thread. See issue #4478 <https://github.com/HypothesisWorks/hypothesis/issues/4478>.

@precondition <#hypothesis.stateful.precondition> now errors if used without @rule <#hypothesis.stateful.rule> or @invariant <#hypothesis.stateful.invariant>. Doing so has no effect and is indicative of a user error (issue #4413 <https://github.com/HypothesisWorks/hypothesis/issues/4413>).

Fix on_observation() <#hypothesis.internal.conjecture.providers.PrimitiveProvider.on_observation> being called with observations it wasn't responsible for generating if the test failed.

When a failure found by an alternative backend <#alternative-backends> does not reproduce under the Hypothesis backend, we now raise FlakyBackendFailure <#hypothesis.errors.FlakyBackendFailure> instead of an internal FlakyReplay exception.

Speculative fix for a thread-safety issue in calculating strategy labels.

Improve the thread-safety of strategy validation.

Before this release, Hypothesis did not require that super().__init__() be called in SearchStrategy subclasses. Subclassing SearchStrategy is not supported or part of the public API, but if you are subclassing it anyway, you will need to make sure to call super().__init__() after this release.

Fix a remaining thread-safety issue with the deprecation warning for use of the global random instance (see v6.135.24).

Fix a remaining thread-safety issue with the recursion limit warning Hypothesis issues when an outside caller sets sys.setrecursionlimit (see v6.135.23).

Optimize performance of sampled_from() <#hypothesis.strategies.sampled_from> and internal selection of stateful testing <#stateful> rules.

Optimize the memory and speed of an internal datastructure for compactly storing integers.

Improve thread-safety for stateful @initialize <#hypothesis.stateful.initialize> rules.

Fix a "dictionary changed size during iteration" error that could occur under with register_random() <#hypothesis.register_random> under multiple threads.

Improve thread safety of our sys.monitoring usage (by the Phase.shrink <#hypothesis.Phase.shrink> and Phase.explain <#hypothesis.Phase.explain> phases), as well as the internal computation of strategy labels.

Makes the deprecation warning for using the global random instance thread-safe, as part of our work towards thread safety (issue #4451 <https://github.com/HypothesisWorks/hypothesis/issues/4451>).

In order to de-flake RecursionError failures, Hypothesis sets a deterministic limit on sys.setrecursionlimit. This patch makes the setting of this limit aware of uses by Hypothesis from multiple threads, so it does not produce spurious warnings in multithreaded environments.

Improves the thread safety of caching strategy definitions, as well as usage of strategy transformations like .map() <#hypothesis.strategies.SearchStrategy.map> and .filter() <#hypothesis.strategies.SearchStrategy.filter>.

Fix the thread safety of @rule <#hypothesis.stateful.rule> definitions in RuleBasedStateMachine <#hypothesis.stateful.RuleBasedStateMachine>.

Fixes reproduction_decorator being missing under hypothesis-specific metadata <#observability-hypothesis-metadata> in many observability <#observability> observations, when it should have been present.

Improve threading compatibility of an internal helper for managing deterministic rng seeding.

Remove an internal assertion which could trigger if (1) a lambda was present in the source code of a test, (2) and the source code file was edited on disk between the start of the python process and when Hypothesis runs the property.

Refactor some internals related to the shrinker for better compatibility with free-threading (issue #4451 <https://github.com/HypothesisWorks/hypothesis/issues/4451>).

Fixes an error when the _pytest module is present in sys.modules, but not the _pytest.outcomes or _pytest.fixtures modules. This can happen with code that imports just _pytest, without importing pytest.

Temporarily disable the warning when shared() <#hypothesis.strategies.shared> strategies with the same key draw from different base strategies, due to false alarms. Once we fix the false alarms in a future release, the warning will be re-enabled.

Speed up usages of sampled_from() <#hypothesis.strategies.sampled_from> by deferring evaluation of its repr, and truncating its repr for large collections (over 512 elements). This is especially noticeable when using sampled_from() <#hypothesis.strategies.sampled_from> with large collections. The repr of sampled_from() <#hypothesis.strategies.sampled_from> strategies involving sequence classes with custom reprs may change as a result of this release.

Fixes a substantial performance regression in stateful tests from computing string representations, present since version 6.131.20.

Fix a rare race condition in internal cache eviction logic.

This patch fixes an error when importing our django extra <#hypothesis-django> (via hypothesis.extra.django) if django.contrib.auth was not in INSTALLED_APPS (issue #3716 <https://github.com/HypothesisWorks/hypothesis/issues/3716>).

Thanks to Chris Wesseling for this fix!

Fix a rare race condition in fetch() <#hypothesis.database.ExampleDatabase.fetch>, where we might have read from a non-existent directory.

Refactor some internal code related to patches to make it easier to test.

Add type hints to internal code for patching.

Fixes a race condition in add_listener() <#hypothesis.database.ExampleDatabase.add_listener> for DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase> after version 6.135.1 where the listener might have tried to read a file that doesn't exist.

This patch corrects the f-string formatting of a few array-related error messages.

Improve the error message when applying @given <#hypothesis.given> to a pytest <https://pypi.org/project/pytest/> fixture with pytest 8.4.0.

Further improve the performance of the constants-collection feature introduced in version 6.131.1, by ignoring large files and files with many constants.

This release adds the experimental and unstable OBSERVABILITY_CHOICES <#hypothesis.internal.observability.OBSERVABILITY_CHOICES> option for observability <#observability>. If set, the choice sequence is included in metadata.choice_nodes, and choice sequence spans are included in metadata.choice_spans.

These are relatively low-level implementation detail of Hypothesis, and are exposed in observability for users building tools or research on top of Hypothesis. See PrimitiveProvider <#hypothesis.internal.conjecture.providers.PrimitiveProvider> for more details about the choice sequence and choice spans.

We are actively working towards a better interface for this. Feel free to use OBSERVABILITY_CHOICES <#hypothesis.internal.observability.OBSERVABILITY_CHOICES> to experiment, but don't rely on it yet!

This patch restores compatibility when using the legacy Python 3.9 LL(1) parser <https://docs.python.org/3/whatsnew/3.9.html#new-parser> yet again, because the fix in version 6.131.33 was too brittle.

Thanks to Marco Ricci for this fix!

DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase> now removes empty directories after delete() <#hypothesis.database.ExampleDatabase.delete> is called.

This release adds run_conformance_test() <#hypothesis.internal.conjecture.provider_conformance.run_conformance_test>, for use in testing implementations of alternative backends <#alternative-backends>.

This patch adds hypothesis.extra.django.SimpleTestCase <#hypothesis.extra.django.SimpleTestCase> (issue #4117 <https://github.com/HypothesisWorks/hypothesis/issues/4117>)

Thanks to Chris Wesseling for this contribution!

Internal changes to support hypofuzz <https://hypofuzz.com>.

The to_json hook used internally when writing observability <#observability> reports is now supported on nested dataclasses (in addition to outermost dataclasses).

Warn when shared() <#hypothesis.strategies.shared> strategies with the same key draw from different base strategies. This could lead to subtle failures or lower-than-expected example coverage.

Add on_observation() <#hypothesis.internal.conjecture.providers.PrimitiveProvider.on_observation> to the internal alternative backends <#alternative-backends-internals> interface.

This patch restores compatibility when using the legacy Python 3.9 LL(1) parser <https://docs.python.org/3/whatsnew/3.9.html#new-parser>, which was accidentally broken since version 6.130.13.

Thanks to Marco Ricci for this fix!

fuzz_one_input <#fuzz-one-input> now writes observability reports <#observability> if observability is enabled, bringing it in line with the behavior of other standard ways to invoke a Hypothesis test.

Improve documentation of @example <#hypothesis.example>.

This patch resolves a Pandas FutureWarning (issue #4400 <https://github.com/HypothesisWorks/hypothesis/issues/4400>) caused by indexing with an integer key.

The observations passed to TESTCASE_CALLBACKS <#hypothesis.internal.observability.TESTCASE_CALLBACKS> are now dataclasses, rather than dictionaries. The content written to .hypothesis/observed under HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY remains the same.

Add documentation to some internal APIs.

Add PrimitiveProvider.replay_choices to the alternative backends <#alternative-backends> interface, to support warm-starting e.g. hypothesis-crosshair <https://pypi.org/project/hypothesis-crosshair/> from hypofuzz <https://pypi.org/project/hypofuzz/>.

Improve ExampleDatabase <#hypothesis.database.ExampleDatabase> documentation.

Add some internal type hints.

Improve @settings <#hypothesis.settings> documentation.

This patch adds GITLAB_CI to the environment variables checked when enabling the default CI settings <#hypothesis.settings> profile.

Thanks to Genevieve Mendoza for this contribution!

Include note() <#hypothesis.note> and Phase.explain <#hypothesis.Phase.explain> output in the "representation" field of observability reports <#observability> for failing examples, to more closely match the output produced by Hypothesis.

BackgroundWriteDatabase <#hypothesis.database.BackgroundWriteDatabase> instances now defer creating and starting a thread until first use.

Improve the string representation of characters() <#hypothesis.strategies.characters> in some cases.

Cap the length of bytestrings collected as part of the constants-collection feature introduced in version 6.131.1, as long bytestrings are unlikely to be useful.

All ExampleDatabase <#hypothesis.database.ExampleDatabase> implementations in Hypothesis now implement __eq__.

Further improve the performance of the new features introduced in version 6.131.1, by improving unions of large sets.

Improve performance of an internal method to count possible choices.

Improves output when assigning values to multiple target bundles in a stateful rule (issue #4361 <https://github.com/HypothesisWorks/hypothesis/issues/4361>).

Internal optimization to reduce redundant choice sequence spans.

Add a for_failure: bool = False parameter to provider.realize in alternative backends <#alternative-backends>, so that symbolic-based backends can increase their timeouts when realizing failures, which are more important than regular examples.

Improve type hints for the single-argument form of one_of() <#hypothesis.strategies.one_of>. st.one_of(strategies) now matches the type of st.one_of(*strategies). For instance, st.one_of([st.integers(), st.none()]) now has the correct type of SearchStrategy[int | None] instead of SearchStrategy[Any].

Fix incorrectly reporting alternative backends <#alternative-backends> as unsound in some cases.

Remove more false-positive locations from Phase.explain <#hypothesis.Phase.explain> output, and add a new metadata.reproduction_decorator field in observability reports <#observability> for failing examples.

Fix a BytesWarning after version 6.131.1 if the source code used the same value in both a normal and binary string.

DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase> will now fall back to (potentially non-atomic) copies rather than renames, if the temporary directory used for atomic write-and-rename is on a different filesystem to the configured database location (issue #4335 <https://github.com/HypothesisWorks/hypothesis/issues/4335>).

Further improve the performance of the new features introduced in version 6.131.1.

This patch makes the new features introduced in version 6.131.1 much faster, and fixes an internal RecursionError when working with deeply-nested code.

Fix a rare case where database entries were kept after they were no longer needed when using Phase.target <#hypothesis.Phase.target>.

Internal refactoring of the @settings <#hypothesis.settings> object, with no user-visible change.

Fixes a rare internal error where new code from version 6.131.1 could fail if sys.modules is simultaneously modified, e.g. as a side effect of imports executed from another thread.  Our Thread-Safety Policy <#thread-safety-policy> does not promise that this is supported, but we're happy to take reasonable fixes.

Thanks to Tony Li for reporting and fixing this issue.

The pub-sub change listening interface of the Hypothesis database <#database> now correctly fires events for DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase> if the directory was created after the listener was added.

Also disables on emscripten the constants-extraction feature introduced in 6.131.1 - 2025-04-17, where it caused substantial slowdown.

Hypothesis now looks for constant values in the source code of your program, and sometimes uses them while generating examples. This lets Hypothesis generate interesting inputs that are specific to your program.

Add is_hypothesis_test() <#hypothesis.is_hypothesis_test>, for third-party libraries which want to determine whether a test has been defined with Hypothesis.

Refactor some internals.

Lays some groundwork for future work on collecting interesting literals from the code being tested, for increased bug-finding power (issue #3127 <https://github.com/HypothesisWorks/hypothesis/issues/3127>). There is no user-visible change (yet!)

Fix the caching behavior of sampled_from() <#hypothesis.strategies.sampled_from>, which in rare cases led to failing an internal assertion (issue #4339 <https://github.com/HypothesisWorks/hypothesis/issues/4339>).

This patch deprecates creating a database using the abstract ExampleDatabase() class. Use one of the following instead:

• Replace ExampleDatabase(":memory:") with InMemoryExampleDatabase <#hypothesis.database.InMemoryExampleDatabase>.
• Replace ExampleDatabase("/path/to/dir") with DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase>.
• Replace ExampleDatabase() with either InMemoryExampleDatabase <#hypothesis.database.InMemoryExampleDatabase> or DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase>, depending on your needs. Previously, Hypothesis interpreted ExampleDatabase() as a DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase> in the default .hypothesis directory, with a fallback to InMemoryExampleDatabase <#hypothesis.database.InMemoryExampleDatabase> if that location was not available.

When reporting the always-failing, never-passing lines from the Phase.explain <#hypothesis.Phase.explain> phase, we now sort the reported lines so that local code shows up first, then third-party library code, then standard library code.

Improves the documentation of settings.verbosity <#hypothesis.settings.verbosity> objects.

Rename internal variables for clarity.

Update the documentation link in HealthCheck <#hypothesis.HealthCheck> error messages to their new location in the documentation.

Improve our internal type hints.

Improve an additional interaction between the hypothesis-crosshair <https://pypi.org/project/hypothesis-crosshair/> backend <#alternative-backends> and our observability tools <#observability>.

This patch improves the interaction between the hypothesis-crosshair <https://pypi.org/project/hypothesis-crosshair/> backend <#alternative-backends> and our observability tools <#observability>.

Fix an issue with realizing symbolic values provided by alternative backends <#alternative-backends> when Hypothesis encounters an internal error in its engine.

Improve the documentation for some strategies, including @composite <#hypothesis.strategies.composite> and data() <#hypothesis.strategies.data>.

Nesting @given <#hypothesis.given> inside of @given <#hypothesis.given> is now a HealthCheck <#hypothesis.HealthCheck> failure. Nesting @given <#hypothesis.given> results in quadratic generation and shrinking behavior, and can usually be more cleanly expressed by replacing the inner function with a data() <#hypothesis.strategies.data> parameter on the outer given. For more details, see nested_given <#hypothesis.HealthCheck.nested_given>. (issue #4167 <https://github.com/HypothesisWorks/hypothesis/issues/4167>)

Fixes an internal error when certain alternative backends <#alternative-backends> find a failure on their very first generated example.

nothing() <#hypothesis.strategies.nothing> is now typed as SearchStrategy[Never], because no value can ever be drawn from it. This may help type checkers statically determine that some code is not reachable.

This patch improves the string representation of fixed_dictionaries() <#hypothesis.strategies.fixed_dictionaries>.

Improve how the shrinker checks for unnecessary work, leading to 10% less time spent shrinking on average, with no reduction in quality.

randoms() <#hypothesis.strategies.randoms> no longer produces 1.0, matching the exclusive upper bound of random.Random.random (issue #4297 <https://github.com/HypothesisWorks/hypothesis/issues/4297>).

This release adds a "hypothesis-urandom" backend <#alternative-backends>, which draws randomness from /dev/urandom instead of Python's PRNG. This is useful for users of Antithesis <https://antithesis.com/> who also have Hypothesis tests, allowing Antithesis mutation of /dev/urandom to drive Hypothesis generation. We expect it to be strictly slower than the default backend for everyone else.

It can be enabled with @settings(backend="hypothesis-urandom").

For strategies which draw make recursive draws, including recursive() <#hypothesis.strategies.recursive> and deferred() <#hypothesis.strategies.deferred>, we now generate examples with duplicated subtrees more often. This tends to uncover interesting behavior in tests.

For instance, we might now generate a tree like this more often (though the details depend on the strategy):

               ┌─────┐
        ┌──────┤  a  ├──────┐
        │      └─────┘      │
     ┌──┴──┐             ┌──┴──┐
     │  b  │             │  a  │
     └──┬──┘             └──┬──┘
   ┌────┴────┐         ┌────┴────┐
┌──┴──┐   ┌──┴──┐   ┌──┴──┐   ┌──┴──┐
│  c  │   │  d  │   │  b  │   │ ... │
└─────┘   └─────┘   └──┬──┘   └─────┘
                  ┌────┴────┐
               ┌──┴──┐   ┌──┴──┐
               │  c  │   │  d  │
               └─────┘   └─────┘

Improves input validation for several strategies in our pandas extra <#hypothesis-pandas>, so that they raise a helpful InvalidArgument rather than OverflowError.

Discovered by our recent string generation upgrade.

Rename a few internal classes for clarity.

text() <#hypothesis.strategies.text> now occasionally generates from a preselected list of strings which are likely to find bugs. These include ligatures, right-to-left and top-to-bottom text, emojis, emoji modifiers, strings like "Infinity", "None", and "FALSE", and other interesting things. This is especially useful when testing the full unicode range, where the search space is too large for uniform sampling to be very effective.

Of course, examples generated this way shrink just like they normally would. It was always possible for Hypothesis to generate these strings; it is just more likely after this change. From the outside, it is as if Hypothesis generated the example completely randomly.

Many thanks to the Big List of Naughty Strings <https://github.com/minimaxir/big-list-of-naughty-strings>, Text Rendering Hates You <https://faultlore.com/blah/text-hates-you/>, and Text Editing Hates You Too <https://lord.io/text-editing-hates-you-too/> for forming the basis of this list.

We now provide a better string representation for one_of() <#hypothesis.strategies.one_of> strategies, by flattening consecutive | combinations. For instance:

>>> st.integers() | st.text() | st.booleans()
# previously: one_of(one_of(integers(), text()), booleans())
one_of(integers(), text(), booleans())

Explicit calls to one_of() <#hypothesis.strategies.one_of> remain unflattened, in order to make tracking down complicated one_of() <#hypothesis.strategies.one_of> constructions easier:

>>> st.one_of(st.integers(), st.one_of(st.text(), st.booleans()))
one_of(integers(), one_of(text(), booleans()))

We print one_of in reprs (rather than integers() | text() | ...) for consistency with reprs containing .filter or .map calls, which uses the full one_of to avoid ambiguity.

This patch improves shrinking behavior for values from text() <#hypothesis.strategies.text> and binary() <#hypothesis.strategies.binary> which contain duplicate elements, like "zzzabc". It also improves shrinking for  bugs which require the same character to be drawn from two different text() <#hypothesis.strategies.text> strategies to trigger.

Fix a type-hinting regression from version 6.125.1, where we would no longer guarantee the type of the argument to .filter predicates (issue #4269 <https://github.com/HypothesisWorks/hypothesis/issues/4269>).

# x was previously Unknown, but is now correctly guaranteed to be int
st.integers().filter(lambda x: x > 0)

This patch tweaks the performance of Phase.target <#hypothesis.Phase.target>, avoiding aborting some test cases when it would be better to finish generating them.

This patch fixes a bug where from_type() <#hypothesis.strategies.from_type> would error on certain types involving Protocol (issue #4194 <https://github.com/HypothesisWorks/hypothesis/issues/4194>).

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

Improve shrinking of non-standard NaN float values (issue #4277 <https://github.com/HypothesisWorks/hypothesis/issues/4277>).

Adjust type hints for the pub-sub database implementation in version 6.126.0, and remove a remnant debug print in its implementation.

Improve the clarity of printing counterexamples in stateful testing <#stateful>, by avoiding confusing Bundle <#hypothesis.stateful.Bundle> references with equivalent values drawn from a regular strategy.

For example, we now print:

a_0 = state.add_to_bundle(a=0)
state.unrelated(value=0)

instead of

a_0 = state.add_to_bundle(a=0)
state.unrelated(value=a_0)

if the unrelated rule draws from a regular strategy such as integers() <#hypothesis.strategies.integers> instead of the a bundle.

This releases adds support for type aliases created with the type statement (new in python 3.12) to from_type() <#hypothesis.strategies.from_type> and register_type_strategy() <#hypothesis.strategies.register_type_strategy>.

The Hypothesis database <#database> now supports a pub-sub interface to efficiently listen for changes in the database, via .add_listener and .remove_listener. While all databases that ship with Hypothesis support this interface, implementing it is not required for custom database subclasses. Hypothesis will warn when trying to listen on a database without support.

This feature is currently only used downstream in hypofuzz <https://github.com/zac-hd/hypofuzz>.

Improves sharing of some internal cache behavior.

Optimize performance (improves speed by ~5%) and clarify the wording in an error message.

Fixes a bug since around version 6.124.4 where we might have generated -0.0 for st.floats(min_value=0.0), which is unsound.

Add 2024.12 to the list of recognized Array API versions in hypothesis.extra.array_api.

Registration of experimental Alternative backends for Hypothesis <#alternative-backends> is now done via hypothesis.internal.conjecture.providers.AVAILABLE_PROVIDERS instead of hypothesis.internal.conjecture.data.AVAILABLE_PROVIDERS.

Refactor some internals for better type hinting.

Internal renamings.

More work on internal type hints.

Internal refactoring to make some stateful internals easier to access.

Refactoring of our internal input generation. This shouldn't lead to any changes in the distribution of test inputs. If you notice any, please open an issue!

Some Hypothesis internals now use the number of choices as a yardstick of input size, rather than the entropy consumed by those choices. We don't expect this to cause significant behavioral changes.

Improves our internal caching logic for test cases.

fuzz_one_input <#fuzz-one-input> is now implemented using an alternative backend <#alternative-backends>. This brings the interpretation of the fuzzer-provided bytestring closer to the fuzzer mutations, allowing the mutations to work more reliably. We hope to use this backend functionality to improve fuzzing integration (e.g. atheris issue #20 <https://github.com/google/atheris/issues/20>) in the future!

The Hypothesis example database <#database> now uses a new internal format to store examples. This new format is not compatible with the previous format, so stored entries will not carry over.

The database is best thought of as a cache that may be invalidated at times. Instead of relying on it for correctness, we recommend using @example <#hypothesis.example> to specify explicit examples. When using databases across environments (such as connecting a GitHubArtifactDatabase <#hypothesis.database.GitHubArtifactDatabase> database in CI to your local environment), we recommend using the same version of Hypothesis for each where possible, for maximum reproducibility.

This patch improves certain corner cases for reporting of flaky errors (issue #4183 <https://github.com/HypothesisWorks/hypothesis/issues/4183> and issue #4228 <https://github.com/HypothesisWorks/hypothesis/issues/4228>).

Improves an edge case in one of our integer and float shrinking passes.

Improves one of our shrinking passes for integers which require a constant relative difference to trigger the bug.

Avoid realizing symbolic values from Alternative backends for Hypothesis <#alternative-backends> when Verbosity <#hypothesis.Verbosity> is verbose or higher.

More internal code refactoring.

DirectoryBasedExampleDatabase <#hypothesis.database.DirectoryBasedExampleDatabase> now creates files representing database entries atomically, avoiding a very brief intermediary state where a file could be created but not yet written to.

Internal code refactoring.

Fixes a bug caused by alternative backends <#alternative-backends> raising hypothesis.errors.BackendCannotProceed in certain cases.

Add internal type hints to our pretty printer.

The shrinker contains a pass aimed at integers which are required to sum to a value. This patch extends that pass to floats as well.

Internal type hint additions and refactorings.

@reproduce_failure() <#hypothesis.reproduce_failure> now uses a newer internal interface to represent failures. As a reminder, this representation is not intended to be stable across versions or with respect to changes in the test.

Internal code refactoring for the typed choice sequence (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>). May have some neutral effect on shrinking.

This patch improves shrinking involving long strings or byte sequences whose value is not relevant to the failure.

This release further improves shrinking of strategies using one_of() <#hypothesis.strategies.one_of>, allowing the shrinker to more reliably move between branches of the strategy.

The shrinker now uses the typed choice sequence (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>) when ordering failing examples. As a result, Hypothesis may now report a different minimal failing example for some tests. We expect most cases to remain unchanged.

Our pytest plugin now emits a warning if you set Pytest's norecursedirs config option in such a way that the .hypothesis directory would be searched for tests.  This reliably indicates that you've made a mistake which slows down test collection, usually assuming that your configuration extends the set of ignored patterns when it actually replaces them. (issue #4200 <https://github.com/HypothesisWorks/hypothesis/issues/4200>)

from_type() <#hypothesis.strategies.from_type> can now handle constructors with required positional-only arguments if they have type annotations.  Previously, we only passed arguments by keyword.

This patch lays some groundwork for migrating our internal representation to the typed choice sequence (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>)

This patch cleans up some internal code around clamping floats.

This release improves shrinking in some cases, especially for strategies using one_of() <#hypothesis.strategies.one_of>. This will typically improve shrinking speed and may in some cases improve the end result.

This patch improves generation performance for the provisional domains() <#hypothesis.provisional.domains> strategy, including its derivative strategies urls() <#hypothesis.provisional.urls> and emails() <#hypothesis.strategies.emails>.

This patch improves our error and warning messages.

• Add a warning for st.text("ascii") - you probably meant st.text(st.characters(codec="ascii")). Similarly for "utf-8".
• Recommend remedies in the error message of Unsatisfiable.
• When @given errors because it was given an extra keyword argument, and the keyword matches a setting name like max_examples, recommend @settings(max_examples=...) instead.

This patch updates some outdated external links in our documentation.

Fix from_type() <#hypothesis.strategies.from_type> on collections.abc.Callable returning None.

This release adds .span_start() and .span_end() methods to our internal PrimitiveProvider interface, for use by Alternative backends for Hypothesis <#alternative-backends>.

This patch updates our autoformatting tools, improving our code style without any API changes.

This release brings back the old representation of hypothesis.stateful.Bundle <#hypothesis.stateful.Bundle>, reverting most changes of PR #4124 <https://github.com/HypothesisWorks/hypothesis/pull/4124>.

This release adds BackgroundWriteDatabase <#hypothesis.database.BackgroundWriteDatabase>, a new database backend which defers writes on the wrapped database to a background thread. This allows for low-overhead writes in performance-critical environments like fuzz_one_input <#fuzz-one-input>.

• This release changes our input distribution for low max_examples. Previously, we capped the size of inputs when generating at least the first 10 inputs, with the reasoning that early inputs to a property should be small. However, this meant properties with max_examples=10 would consistent entirely of small inputs. This patch removes the hard lower bound so that inputs to these properties are more representative of the input space.
• When a user requests an interactive input via strategy.example, we generate and cache a batch of 100 inputs, returning the first one. This can be expensive for large strategies or when only a few examples are needed. This release improves the speed of strategy.example by lowering the batch size to 10.

This patch fixes a bug since 6.99.13 - 2024-03-24 where only interactively-generated values (via data.draw) would be reported in the arguments field of our observability output <#observability>. Now, all values are reported.

Hypothesis collects coverage information during the Phase.shrink <#hypothesis.Phase.shrink> and Phase.explain <#hypothesis.Phase.explain> phases in order to show a more informative error message. On 3.12+, this uses sys.monitoring. This patch improves the performance of coverage collection on 3.12+ by disabling events we don't need.

This patch refactors some internals to prepare for future work using our IR (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>).

This patch migrates some more internals (around generating novel inputs) to the IR layer (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>).

This release improves Hypothesis' handling of ExceptionGroup - it's now able to detect marker detections if they're inside a  group and attempts to resolve them. Note that this handling is still a work in progress and might not handle edge cases optimally. Please open issues if you encounter any problems or unexpected behavior with it.

Internal refactorings in preparation for upcoming changes.

Internal renamings.

This patch removes some # type: ignore comments following a mypy <https://pypi.org/project/mypy/> update.

When Hypothesis replays examples from its test database that it knows were previously fully shrunk it will no longer try to shrink them again.

This should significantly speed up development workflows for slow tests, as the shrinking could contribute a significant delay when rerunning the tests.

In some rare cases this may cause minor reductions in example quality. This was considered an acceptable tradeoff for the improved test runtime.

This patch avoids computing some string representations we won't need, giving a small speedup (part of issue #4139 <https://github.com/HypothesisWorks/hypothesis/issues/4139>).

This patch migrates the optimisation algorithm for targeted property-based testing <#targeted> to our IR layer (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>). This should result in moderately different (and hopefully improved) exploration behavior in tests which use hypothesis.target() <#hypothesis.target>.

This patch adds more type hints to internal Hypothesis code.

This patch migrates the Phase.explain <#hypothesis.Phase.explain> phase to our IR layer (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>). This should improve both its speed and precision.

This patch updates some internals around how we determine an input is too large to finish generating.

The urls() <#hypothesis.provisional.urls> strategy no longer generates URLs where the port number is 0.

This change is motivated by the idea that the generated URLs should, at least in theory, be possible to fetch. The port number 0 is special; if a server binds to port 0, the kernel will allocate an unused, and non-zero, port instead. That means that it's not possible for a server to actually be listening on port 0. This motivation is briefly described in the documentation for urls() <#hypothesis.provisional.urls>.

Thanks to @gmacon for fixing issue #4157 <https://github.com/HypothesisWorks/hypothesis/issues/4157>!

This changes the behaviour of settings profiles so that if you reregister the currently loaded profile it will automatically reload it. Previously you would have had to load it again.

In particular this means that if you register a "ci" profile, it will automatically be used when Hypothesis detects you are running on CI.

Hypothesis now detects if it is running on a CI server and provides better default settings for running on CI in this case.

This patch changes the priority order of pretty printing logic so that a user provided pretty printing method will always be used in preference to e.g. printing it like a dataclass.

This patch restores diversity to the outputs of from_type(type) <#hypothesis.strategies.from_type> (issue #4144 <https://github.com/HypothesisWorks/hypothesis/issues/4144>).

This release improves pretty printing of nested classes to include the outer class name in their printed representation.

This patch fixes a regression from version 6.115.2 where generating values from integers() <#hypothesis.strategies.integers> with certain values for min_value and max_value would error.

This release improves integer shrinking by folding the endpoint upweighting for integers() <#hypothesis.strategies.integers> into the weights parameter of our IR (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>).

If you maintain an alternative backend as part of our (for now explicitly unstable) Alternative backends for Hypothesis <#alternative-backends>, this release changes the type of the weights parameter to draw_integer and may be a breaking change for you.

This patch improves the performance of from_type() <#hypothesis.strategies.from_type> with pydantic.types.condate <https://docs.pydantic.dev/latest/api/types/#pydantic.types.condate> (issue #4000 <https://github.com/HypothesisWorks/hypothesis/issues/4000>).

This improves the formatting of dataclasses and attrs classes when printing falsifying examples.

This patch upgrades remaining type annotations to Python 3.9 syntax.

This release drops support for Python 3.8, which reached end of life on 2024-10-07 <https://devguide.python.org/versions/>.

This release adds hypothesis.errors.BackendCannotProceed, an unstable API for use by Alternative backends for Hypothesis <#alternative-backends>.

This release fixes a regression where hypothesis.stateful.Bundle <#hypothesis.stateful.Bundle> did not work properly with .flatmap() <#hypothesis.strategies.SearchStrategy.flatmap> functionality (issue #4128 <https://github.com/HypothesisWorks/hypothesis/issues/4128>).

This patch tweaks the paths in @example(...) patches, so that both git apply and patch will work by default.

This release refactors internals of hypothesis.stateful.Bundle <#hypothesis.stateful.Bundle> to have a more consistent representation internally.

This patch fixes an internal error when the __context__ attribute of a raised exception leads to a cycle (issue #4115 <https://github.com/HypothesisWorks/hypothesis/issues/4115>).

This patch removes a now-incorrect internal assertion about numpy's typing after recent numpy changes (currently only in numpy's nightly release).

This release adds support for variable-width bytes in our IR layer (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>), which should mean improved performance anywhere you use binary() <#hypothesis.strategies.binary>. If you maintain an alternative backend as part of our (for now explicitly unstable) Alternative backends for Hypothesis <#alternative-backends>, this release changes the draw_* interface and may be a breaking change for you.

This patch contains some internal code cleanup.  There is no user-visible change.

This patch improves shrinking in cases involving 'slips' from one strategy to another. Highly composite strategies are the most likely to benefit from this change.

This patch also reduces the range of python:datetime.datetime generated by from_model() <#hypothesis.extra.django.from_model> in order to avoid <https://code.djangoproject.com/ticket/35683>.

Alternative backends for Hypothesis <#alternative-backends> can now implement .observe_test_case() and observe_information_message() methods, to record backend-specific metadata and messages in our observability output <#observability> (issue #3845 <https://github.com/HypothesisWorks/hypothesis/issues/3845> and hypothesis-crosshair#22 <https://github.com/pschanely/hypothesis-crosshair/issues/22>).

Support __default__ field of TypeVar and support the same from typing-extensions <https://pypi.org/project/typing-extensions/> in from_type() <#hypothesis.strategies.from_type>.

Add better error message for TypeIs types in from_type() <#hypothesis.strategies.from_type>.

Support LiteralString in from_type() <#hypothesis.strategies.from_type>.

This patch makes progress towards adding type hints to our internal conjecture engine (issue #3074 <https://github.com/HypothesisWorks/hypothesis/issues/3074>).

This release allows using Annotated and ReadOnly types for TypedDict value types with from_type() <#hypothesis.strategies.from_type>.

This patch fixes compatibility with attrs==24.1.0 <https://pypi.org/project/attrs/> on the nightly build of CPython, 3.14.0 pre-alpha (issue #4067 <https://github.com/HypothesisWorks/hypothesis/issues/4067>).

This patch removes an assertion which was in fact possible in rare circumstances involving a small number of very large draws.

This patch improves our example generation performance by adjusting our internal cache implementation.

This patch improves our pretty-printer for unusual numbers.

• Signalling NaNs are now represented by using the struct module to show the exact value by converting from a hexadecimal integer
• CPython limits integer-to-string conversions <https://docs.python.org/3/library/stdtypes.html#integer-string-conversion-length-limitation> to mitigate DDOS attacks.  We now use hexadecimal for very large integers, and include underscore separators for integers with ten or more digits.

This patch improves generation speed in some cases by avoiding pretty-printing overhead for non-failing examples.

This patch fixes a rare internal error when using integers() <#hypothesis.strategies.integers> with a high number of examples and certain {min, max}_value parameters (pull request #4059 <https://github.com/HypothesisWorks/hypothesis/pull/4059>).

This patch addresses the issue of hypothesis potentially accessing mocked time.perf_counter during test execution (issue #4051 <https://github.com/HypothesisWorks/hypothesis/issues/4051>).

Minor internal-only cleanups to some error-handling and reporting code.

This patch disables hypothesis.target() <#hypothesis.target> on alternative backends where it would not work.

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

This patch changes most Flaky errors to use an ExceptionGroup, which makes the representation of these errors easier to understand.

The alphabet= argument to from_regex() <#hypothesis.strategies.from_regex> now accepts unions of characters() <#hypothesis.strategies.characters> and sampled_from() <#hypothesis.strategies.sampled_from> strategies, in addition to accepting each individually.

This patch also fixes a bug where text(...).filter(re.compile(...).match) could generate non-matching instances if the regex pattern contained | (issue #4008 <https://github.com/HypothesisWorks/hypothesis/issues/4008>).

This patch improves our pretty-printer (issue #4037 <https://github.com/HypothesisWorks/hypothesis/issues/4037>).

It also fixes the codemod for HealthCheck.all() from version 6.72, which was instead trying to fix Healthcheck.all() - note the lower-case c! Since our tests had the same typo, it all looked good... until issue #4030 <https://github.com/HypothesisWorks/hypothesis/issues/4030>.

This release improves support for unions of numpy <https://pypi.org/project/numpy/> dtypes such as np.float64 | np.complex128 in from_type() <#hypothesis.strategies.from_type> and arrays() <#hypothesis.extra.numpy.arrays> (issue #4041 <https://github.com/HypothesisWorks/hypothesis/issues/4041>).

This patch improves the reporting of certain flaky errors.

This patch iterates on our experimental support for alternative backends (Alternative backends for Hypothesis <#alternative-backends>). See pull request #4029 <https://github.com/HypothesisWorks/hypothesis/pull/4029> for details.

This release improves support for Django 5.0, and drops support for end-of-life Django versions (< 4.2).

Thanks to Joshua Munn for this contribution.

Clean up internal cache implementation.

This patch updates our autoformatting tools, improving our code style without any API changes.

This patch fixes an issue when realizing symbolics with our experimental backend <#hypothesis.settings.backend> setting.

Improves internal test coverage.

This release adds strategies for Django's ModelChoiceField and ModelMultipleChoiceField (issue #4010 <https://github.com/HypothesisWorks/hypothesis/issues/4010>).

Thanks to Joshua Munn for this contribution.

Fixes and reinstates full coverage of internal tests, which was accidentally disabled in pull request #3935 <https://github.com/HypothesisWorks/hypothesis/pull/3935> (issue #4003 <https://github.com/HypothesisWorks/hypothesis/issues/4003>).

This release prevents a race condition inside internal cache implementation.

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

This patch improves our deduplication tracking across all strategies (pull request #4007 <https://github.com/HypothesisWorks/hypothesis/pull/4007>). Hypothesis is now less likely to generate the same input twice.

Account for time spent in garbage collection during tests, to avoid flaky DeadlineExceeded errors as seen in issue #3975 <https://github.com/HypothesisWorks/hypothesis/issues/3975>.

Also fixes overcounting of stateful run times, a minor observability bug dating to version 6.98.9 (pull request #3890 <https://github.com/HypothesisWorks/hypothesis/pull/3890>).

This release migrates the shrinker to our new internal representation, called the IR layer (pull request #3962 <https://github.com/HypothesisWorks/hypothesis/pull/3962>). This improves the shrinker's performance in the majority of cases. For example, on the Hypothesis test suite, shrinking is a median of 1.38x faster.

It is possible this release regresses performance while shrinking certain strategies. If you encounter strategies which reliably shrink more slowly than they used to (or shrink slowly at all), please open an issue!

You can read more about the IR layer at issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>.

This patch fixes one of our shrinking passes getting into a rare O(n) case instead of O(log(n)).

This patch fixes some introspection errors new in Python 3.11.9 and 3.13.0b1, for the Ghostwriter and from_type() <#hypothesis.strategies.from_type>.

Internal developer documentation, no user-visible changes.

This patch improves our shrinking of unique collections, such as  dictionaries() <#hypothesis.strategies.dictionaries>, sets() <#hypothesis.strategies.sets>, and lists() <#hypothesis.strategies.lists> with unique=True.

This patch fixes a rare internal error when generating very large elements from strategies (issue #3874 <https://github.com/HypothesisWorks/hypothesis/issues/3874>).

This patch fixes an overly strict internal type assertion.

This release improves our support for the annotated-types <https://pypi.org/project/annotated-types/> iterable GroupedMetadata protocol.  In order to treat the elements "as if they had been unpacked", if one such element is a SearchStrategy <#hypothesis.strategies.SearchStrategy> we now resolve to that strategy.  Previously, we treated this as an unknown filter predicate.

We expect this to be useful for libraries implementing custom metadata - instead of requiring downstream integration, they can implement the protocol and yield a lazily-created strategy.  Doing so only if Hypothesis is in sys.modules gives powerful integration with no runtime overhead or extra dependencies.

The from_model() <#hypothesis.extra.django.from_model> function currently tries to create a strategy for AutoField fields if they don't have auto_created set to True.  The docs say it's supposed to skip all AutoField fields, so this patch updates the code to do what the docs say (issue #3978 <https://github.com/HypothesisWorks/hypothesis/issues/3978>).

This patch adds some internal type annotations (issue #3074 <https://github.com/HypothesisWorks/hypothesis/issues/3074>). Thanks to Andrew Sansom for his contribution!

This patch fixes a rare internal error when using integers() <#hypothesis.strategies.integers> with a high max_examples setting (issue #3974 <https://github.com/HypothesisWorks/hypothesis/issues/3974>).

This patch improves our internal caching logic. We don't expect it to result in any performance improvements (yet!).

This patch turns off a check in register_random() <#hypothesis.register_random> for possibly unreferenced RNG instances on the free-threaded build of CPython 3.13 because this check has a much higher false positive rate in the free-threaded build (issue #3965 <https://github.com/HypothesisWorks/hypothesis/issues/3965>).

Thanks to Nathan Goldbaum for this patch.

This patch turns off a warning for functions decorated with typing.overload() and then composite() <#hypothesis.strategies.composite>, although only in that order (issue #3970 <https://github.com/HypothesisWorks/hypothesis/issues/3970>).

This patch fixes a significant slowdown when using the precondition() <#hypothesis.stateful.precondition> decorator in some cases, due to expensive repr formatting internally (issue #3963 <https://github.com/HypothesisWorks/hypothesis/issues/3963>).

Explicitly cast numpy.finfo.smallest_normal to builtin float in preparation for the numpy==2.0 <https://pypi.org/project/numpy/> release (issue #3950 <https://github.com/HypothesisWorks/hypothesis/issues/3950>)

This patch improve a rare error message for flaky tests (issue #3940 <https://github.com/HypothesisWorks/hypothesis/issues/3940>).

The from_dtype() <#hypothesis.extra.numpy.from_dtype> function no longer generates NaT ("not-a-time") values for the datetime64 or timedelta64 dtypes if passed allow_nan=False (issue #3943 <https://github.com/HypothesisWorks/hypothesis/issues/3943>).

This patch includes the backend <#hypothesis.settings.backend> setting in the how_generated field of our observability output <#observability>.

If you were running Python 3.13 (currently in alpha) with pytest-xdist <https://pypi.org/project/pytest-xdist/> and then attempted to pretty-print a lambda functions which was created using the eval() builtin, it would have raised an AssertionError. Now you'll get "lambda ...: <unknown>", as expected.

This release improves an internal invariant.

This patch fixes Hypothesis sometimes raising a Flaky error when generating collections of unique floats containing nan. See issue #3926 <https://github.com/HypothesisWorks/hypothesis/issues/3926> for more details.

This patch continues our work on refactoring the shrinker (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>).

This patch continues our work on refactoring shrinker internals (issue #3921 <https://github.com/HypothesisWorks/hypothesis/issues/3921>).

This release resolves PermissionError that come from creating databases on inaccessible paths.

This patch starts work on refactoring our shrinker internals. There is no user-visible change.

This patch fixes a longstanding performance problem in stateful testing (issue #3618 <https://github.com/HypothesisWorks/hypothesis/issues/3618>), where state machines which generated a substantial amount of input for each step would hit the maximum amount of entropy and then fail with an Unsatisfiable error.

We now stop taking additional steps when we're approaching the entropy limit, which neatly resolves the problem without touching unaffected tests.

Fix regression caused by using PEP 696 <https://peps.python.org/pep-0696/> default in TypeVar with Python 3.13.0a3.

This patch further improves the type annotations in hypothesis.extra.numpy <#module-hypothesis.extra.numpy>.

Simplify the type annotation of column() <#hypothesis.extra.pandas.column> and columns() <#hypothesis.extra.pandas.columns> by using PEP 696 <https://peps.python.org/pep-0696/> to avoid overloading.

This patch implements type annotations for column() <#hypothesis.extra.pandas.column>.

This release adds the experimental and unstable backend <#hypothesis.settings.backend> setting.  See Alternative backends for Hypothesis <#alternative-backends> for details.

This patch fixes issue #3900 <https://github.com/HypothesisWorks/hypothesis/issues/3900>, a performance regression for arrays() <#hypothesis.extra.numpy.arrays> due to the interaction of 6.98.12 - 2024-02-25 and 6.97.1 - 2024-01-27.

This patch improves the type annotations in hypothesis.extra.numpy <#module-hypothesis.extra.numpy>, which makes inferred types more precise for both mypy <https://pypi.org/project/mypy/> and pyright <https://pypi.org/project/pyright/>, and fixes some strict-mode errors on the latter.

Thanks to Jonathan Plasse for reporting and fixing this in pull request #3889 <https://github.com/HypothesisWorks/hypothesis/pull/3889>!

This patch paves the way for future shrinker improvements. There is no user-visible change.

This release adds support for the Array API's 2023.12 release <https://data-apis.org/array-api/2023.12/> via the api_version argument in make_strategies_namespace() <#hypothesis.extra.array_api.make_strategies_namespace>. The API additions and modifications in the 2023.12 spec do not necessitate any changes in the Hypothesis strategies, hence there is no distinction between a 2022.12 and 2023.12 strategies namespace.

This patch adjusts the printing of bundle values to correspond with their names when using stateful testing.

This patch implements filter-rewriting for text() <#hypothesis.strategies.text> and binary() <#hypothesis.strategies.binary> with the search(), match(), or fullmatch() method of a re.compile()d regex.

This patch implements filter-rewriting for most length filters on some additional collection types (issue #3795 <https://github.com/HypothesisWorks/hypothesis/issues/3795>), and fixes several latent bugs where unsatisfiable or partially-infeasible rewrites could trigger internal errors.

This patch makes stateful testing somewhat less likely to get stuck when there are only a few possible rules.

This patch adds a note <https://peps.python.org/pep-0678/> to errors which occur while drawing from a strategy, to make it easier to tell why your test failed in such cases.

This patch ensures that observability <#observability> outputs include an informative repr for RuleBasedStateMachine <#hypothesis.stateful.RuleBasedStateMachine> stateful tests, along with more detailed timing information.

This patch improves the Ghostwriter <#ghostwriter> for binary operators.

This patch improves import-detection in the Ghostwriter <#ghostwriter> (issue #3884 <https://github.com/HypothesisWorks/hypothesis/issues/3884>), particularly for from_type() <#hypothesis.strategies.from_type> and strategies from hypothesis.extra.*.

This patch clarifies the documentation on stateful testing (issue #3511 <https://github.com/HypothesisWorks/hypothesis/issues/3511>).

This patch improves argument-to-json conversion for observability <#observability> output.  Checking for a .to_json() method on the object before a few other options like dataclass support allows better user control of the process (issue #3880 <https://github.com/HypothesisWorks/hypothesis/issues/3880>).

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

This patch fixes an error when generating observability <#observability> reports involving large (n > 1e308) integers.

This patch refactors some internals. There is no user-visible change.

This release improves our distribution of generated values for all strategies, by doing a better job of tracking which values we have generated before and avoiding generating them again.

For example, st.lists(st.integers()) previously generated ~5 each of [] [0] in 100 examples. In this release, each of [] and [0] are generated ~1-2 times each.

This release deprecates use of the global random number generator while drawing from a strategy, because this makes test cases less diverse and prevents us from reporting minimal counterexamples (issue #3810 <https://github.com/HypothesisWorks/hypothesis/issues/3810>).

If you see this new warning, you can get a quick fix by using randoms() <#hypothesis.strategies.randoms>; or use more idiomatic strategies sampled_from() <#hypothesis.strategies.sampled_from>, floats() <#hypothesis.strategies.floats>, integers() <#hypothesis.strategies.integers>, and so on.

Note that the same problem applies to e.g. numpy.random, but for performance reasons we only check the stdlib random module - ignoring even other sources passed to register_random() <#hypothesis.register_random>.

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

This patch adds some observability information <#observability> about how many times predicates in assume() <#hypothesis.assume> or precondition() <#hypothesis.stateful.precondition> were satisfied, so that downstream tools can warn you if some were never satisfied by any test case.

This patch improves formatting and adds some cross-references to our docs.

Internal test refactoring.

This patch slightly changes how we replay examples from the database <#database>: if the behavior of the saved example has changed, we now keep running the test case instead of aborting at the size of the saved example.  While we know it's not the same example, we might as well continue running the test!

Because we now finish running a few more examples for affected tests, this might be a slight slowdown - but correspondingly more likely to find a bug.

We've also applied similar tricks to the Phase.target <#hypothesis.Phase.target> phase, where they are a pure performance improvement for affected tests.

Improves the performance of the arrays() <#hypothesis.extra.numpy.arrays> strategy when generating unique values.

Changes the distribution of sampled_from() <#hypothesis.strategies.sampled_from> when sampling from a Flag. Previously, no-flags-set values would never be generated, and all-flags-set values would be unlikely for large enums. With this change, the distribution is more uniform in the number of flags set.

This patch slightly refactors some internals. There is no user-visible change.

This patch fixes a spurious warning about slow imports when HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY was set.

This patch refactors some more internals, continuing our work on supporting alternative backends (issue #3086 <https://github.com/HypothesisWorks/hypothesis/issues/3086>). There is no user-visible change.

Fix a spurious warning seen when running pytest's test suite, caused by never realizing we got out of initialization due to imbalanced hook calls.

Warns when constructing a repr that is overly long. This can happen by accident if stringifying arbitrary strategies, and is expensive in time and memory. The associated deferring of these long strings in sampled_from() <#hypothesis.strategies.sampled_from> should also lead to improved performance.

This release adds the ability to pass any object to note() <#hypothesis.note>, instead of just strings. The pretty-printed representation of the object will be used.

See also issue #3843 <https://github.com/HypothesisWorks/hypothesis/issues/3843>.

This release avoids creating a .hypothesis directory when using register_type_strategy() <#hypothesis.strategies.register_type_strategy> (issue #3836 <https://github.com/HypothesisWorks/hypothesis/issues/3836>), and adds warnings for plugins which do so by other means or have other unintended side-effects.

This patch improves observability <#observability> reports by moving timing information from metadata to a new timing key, and supporting conversion of additional argument types to json rather than string reprs via a .to_json() method (including e.g. Pandas dataframes).

Additionally, the too_slow <#hypothesis.HealthCheck.too_slow> health check will now report which strategies were slow, e.g. for strategies a, b, c, ...:

    count | fraction |    slowest draws (seconds)
a |    3  |     65%  |      --      --      --   0.357,  2.000
b |    8  |     16%  |   0.100,  0.100,  0.100,  0.111,  0.123
c |    3  |      8%  |      --      --   0.030,  0.050,  0.200
(skipped 2 rows of fast draws)

This patch refactors some internals, continuing our work on supporting alternative backends (issue #3086 <https://github.com/HypothesisWorks/hypothesis/issues/3086>). There is no user-visible change.

The from_lark() <#hypothesis.extra.lark.from_lark> strategy now accepts an alphabet= argument, which is passed through to from_regex() <#hypothesis.strategies.from_regex>, so that you can e.g. constrain the generated strings to a particular codec.

In support of this feature, from_regex() <#hypothesis.strategies.from_regex> will avoid generating optional parts which do not fit the alphabet.  For example, from_regex(r"abc|def", alphabet="abcd") was previously an error, and will now generate only 'abc'.  Cases where there are no valid strings remain an error.

This patch refactors some internals, continuing our work on supporting alternative backends (issue #3086 <https://github.com/HypothesisWorks/hypothesis/issues/3086>). There is no user-visible change.

This patch adds a test statistics <#statistics> event when a generated example is rejected via assume <#hypothesis.assume>.

This may also help with distinguishing gave_up examples in observability <#observability> (issue #3827 <https://github.com/HypothesisWorks/hypothesis/issues/3827>).

This introduces the rewriting of length filters on some collection strategies (issue #3791 <https://github.com/HypothesisWorks/hypothesis/issues/3791>).

Thanks to Reagan Lee for implementing this feature!

If a test uses sampled_from() <#hypothesis.strategies.sampled_from> on a sequence of strategies, and raises a TypeError, we now add a note <https://peps.python.org/pep-0678/> asking whether you meant to use one_of() <#hypothesis.strategies.one_of>.

Thanks to Vince Reuter for suggesting and implementing this hint!

This patch registers explicit strategies for a handful of builtin types, motivated by improved introspection in PyPy 7.3.14 triggering existing internal warnings. Thanks to Carl Friedrich Bolz-Tereick for helping us work out what changed!

This patch fixes an error when writing observability <#observability> reports without a pre-existing .hypothesis directory.

This patch adds a new environment variable HYPOTHESIS_EXPERIMENTAL_OBSERVABILITY_NOCOVER, which turns on observability <#observability> data collection without collecting code coverage data, which may be faster on Python 3.11 and earlier.

Thanks to Harrison Goldstein for reporting and fixing issue #3821 <https://github.com/HypothesisWorks/hypothesis/issues/3821>.

This patch updates our vendored list of top-level domains <https://www.iana.org/domains/root/db>, which is used by the provisional domains() <#hypothesis.provisional.domains> strategy.

This patch fixes a bug introduced in version 6.92.0, where using the data() <#hypothesis.strategies.data> strategy would fail to draw a dataclass() with a defaultdict field.  This was due to a bug in the standard library which was fixed in 3.12 <https://github.com/python/cpython/pull/32056>, so we've vendored the fix.

This release adds an experimental observability <https://en.wikipedia.org/wiki/Observability_(software)> mode.  You can read the docs about it here <#observability>.

This patch refactors some more internals, continuing our work on supporting alternative backends (issue #3086 <https://github.com/HypothesisWorks/hypothesis/issues/3086>). There is no user-visible change.

This patch fixes an issue where builds() <#hypothesis.strategies.builds> could not be used with attrs <https://pypi.org/project/attrs/> objects that defined private attributes (i.e. attributes with a leading underscore). See also issue #3791 <https://github.com/HypothesisWorks/hypothesis/issues/3791>.

This patch also adds support more generally for using builds() <#hypothesis.strategies.builds> with attrs' alias parameter, which was previously unsupported.

This patch increases the minimum required version of attrs to 22.2.0.

This release adds an optional payload argument to hypothesis.event() <#hypothesis.event>, so that you can clearly express the difference between the label and the value of an observation.  Test statistics <#statistics> will still summarize it as a string, but future observability options can preserve the distinction.

This patch supports assigning settings = settings(...) as a class attribute on a subclass of a .TestCase attribute of a RuleBasedStateMachine <#hypothesis.stateful.RuleBasedStateMachine>. Previously, this did nothing at all.

# works as of this release
class TestMyStatefulMachine(MyStatefulMachine.TestCase):
    settings = settings(max_examples=10000)

# the old way still works, but it's more verbose.
MyStateMachine.TestCase.settings = settings(max_examples=10000)

class TestMyStatefulMachine(MyStatefulMachine.TestCase):
    pass

Thanks to Joey Tran for reporting these settings-related edge cases in stateful testing.

This release makes it an error to assign settings = settings(...) as a class attribute on a RuleBasedStateMachine <#hypothesis.stateful.RuleBasedStateMachine>. This has never had any effect, and it should be used as a decorator instead:

class BadMachine(RuleBasedStateMachine):
    """This doesn't do anything, and is now an error!"""

    settings = settings(derandomize=True)

@settings(derandomize=True)
class GoodMachine(RuleBasedStateMachine):
    """This is the right way to do it :-)"""

This patch refactors some internals.  There is no user-visible change, but we hope to improve performance and unlock support for alternative backends such as symbolic execution with crosshair <https://pypi.org/project/crosshair-tool/> in future (issue #3086 <https://github.com/HypothesisWorks/hypothesis/issues/3086>).

Thanks to Liam DeVoe for this fantastic contribution!