If you have an OpenAPI or GraphQL schema, Schemathesis can turn it into thousands of test cases without you writing a single assertion. It reads the spec, generates wild and valid inputs, fires them at your API, and reports where the responses break the contract. This guide explains what Schemathesis is, how to install and run it, what property-based testing actually means, and how it fits next to a functional and contract testing workflow in Apidog.
What is Schemathesis?
Schemathesis is an open-source Python tool that generates API tests automatically from a schema. You point it at an OpenAPI or GraphQL definition, and it builds test cases from the types, formats, and constraints you already declared. It’s built on top of Hypothesis, the property-based testing library for Python, and it’s distributed under the MIT license.
The core idea is simple. Your spec already describes what a valid request and response look like. Schemathesis treats that description as a source of truth and checks whether your running API actually honors it. When the API returns a 500, sends a response that violates the declared schema, or accepts input it should have rejected, Schemathesis flags it.
You run it from the command line with schemathesis run (or the shorter alias st run). There’s also a Python integration if you want to drive it from pytest instead of the CLI. Most teams start with the CLI because it needs almost no setup.
What property-based testing means
Most API tests are example-based. You write a request, you assert on a specific response, and the test passes or fails on that one case. That’s useful, but you only catch the bugs you thought to write a test for.
Property-based testing flips the approach. Instead of one fixed example, you describe a property that should always hold, then let the tool generate hundreds of inputs to try to break it. For Schemathesis, the property is roughly: “no valid request should ever crash the server or produce a response that violates the schema.”
Schemathesis generates inputs from the constraints in your spec. If a field is an integer with a minimum of 1, it’ll try 1, large numbers, boundary values, and the kinds of inputs that trip up naive validation. When a test fails, Hypothesis shrinks the input down to the smallest case that still reproduces the bug, so you get a minimal, readable repro instead of a 4KB random payload.
This is different from monkey testing, which throws random input at an interface to see what falls over. Property-based testing is structured. It uses the schema to generate inputs that are plausible and targeted, and it knows what a correct outcome looks like because the spec told it.
Installing and running Schemathesis
Schemathesis is a Python package, so you need Python 3 and pip. Install it the usual way:
pip install schemathesis
You can also run it without a permanent install using uvx schemathesis if you have uv set up. Once installed, the basic command points at a schema and a base URL:
st run http://127.0.0.1:8000/openapi.json
If your schema lives on disk rather than behind a URL, pass the file path and tell Schemathesis where the live API is:
st run ./openapi.yaml --url http://127.0.0.1:8000
For an authenticated API, add a header:
st run http://127.0.0.1:8000/openapi.json \
--header 'Authorization: Bearer your-token'
Schemathesis discovers every operation in the spec, generates cases for each, and prints a summary of which checks passed and which failed. Failures come with the exact request it sent, so you can replay them with curl or in any API client. Flag names and defaults shift between major versions, so check st run --help against your installed version rather than trusting a blog snippet, including this one.
What Schemathesis catches
The default checks target the gap between what your spec promises and what your server does. Here’s what tends to surface.
| Issue | What it looks like |
|---|---|
| Server errors | A request triggers a 500 instead of a handled 4xx or a valid response |
| Schema violations | The response body doesn’t match the schema you declared for that status code |
| Status code mismatches | The API returns a status code the spec never documented |
| Content-type drift | The response content type doesn’t match what the spec advertises |
| Stateful bugs | An operation works alone but fails inside a realistic multi-step workflow |
That last one is worth a closer look. Schemathesis can run stateful tests, where it chains operations together based on links in your spec, for example create a resource, then fetch it, then delete it. Bugs that only appear in sequence, like a resource that reports the wrong state after an update, are hard to find with single-request tests. The stateful phase drives those sequences as a state machine.
You can extend the default behavior with hooks. Hooks are Python functions that let you customize data generation, add your own checks, or filter which operations get tested. That’s how teams adapt Schemathesis to APIs with auth flows or non-obvious constraints the spec can’t express.
When to use Schemathesis
Schemathesis earns its place when you have a reasonably accurate OpenAPI or GraphQL spec and you want broad, cheap coverage of edge cases. It’s strong at finding the failures you’d never write a manual test for: unhandled inputs, undocumented error shapes, and contract drift between spec and implementation.
It’s a good fit when:
- You maintain an OpenAPI specification and want it enforced against the real server.
- You’re worried about unhandled 500s and want a fuzzing layer in CI.
- Your API has stateful workflows where ordering matters.
- Your team already works in Python and is comfortable with
pipandpytest.
It’s a weaker fit when your spec is thin or out of date, since Schemathesis can only test what the schema describes. Garbage in, garbage out. It also won’t replace your example-based functional tests. Knowing that an endpoint never crashes is not the same as knowing it returns the right value for a known input. You want both.
Schemathesis and Apidog: where each fits
Apidog and Schemathesis solve different problems, and they work well side by side. Apidog is an all-in-one platform for designing, debugging, testing, mocking, and documenting APIs. Schemathesis is a focused property-based fuzzer. Being honest about the boundary matters here.
Apidog does not do property-based fuzzing. Its closest adjacent feature is monkey testing, which sends random input to surface crashes. That’s useful, but it is not the same as schema-driven property-based testing. Schemathesis generates inputs from your spec’s constraints and checks responses against the schema. So if property-based fuzzing is what you need, reach for Schemathesis, not Apidog.
Where Apidog fits is the rest of the lifecycle around that fuzzing pass.
| Workflow stage | Schemathesis | Apidog |
|---|---|---|
| Property-based fuzzing from a spec | Yes, core feature | No |
| Visual API design and spec editing | No | Yes |
| Example-based functional tests | Limited | Yes, visual test builder |
| Contract testing against a spec | Partial, via schema checks | Yes, dedicated workflow |
| Mock servers from a schema | No | Yes, smart mock |
| CI test runner | Yes, st run |
Yes, apidog run |
| Auto-generated docs | No | Yes |
A practical pairing looks like this. You design and maintain the spec in Apidog, generate functional test cases and a mock server from it, and run those in CI with apidog run. Then you add a Schemathesis pass that fuzzes the same spec for edge-case crashes and contract violations. The two layers catch different bug classes. Apidog confirms the API does the right thing for known cases; Schemathesis hunts for the unknown cases that break it.
If your goal is turning a spec into a runnable functional suite rather than a fuzzing run, Apidog can generate API test collections from your OpenAPI specs directly, and you can download Apidog to try that flow.
Frequently asked questions
Is Schemathesis free?
Yes. Schemathesis is open source under the MIT license, and the CLI is free to install and run via pip install schemathesis. There’s also a hosted commercial offering for teams that want a managed experience, but the core tool you run locally and in CI costs nothing.
What’s the difference between schemathesis run and st run?
They’re the same command. st is a short alias for schemathesis, so st run and schemathesis run do exactly the same thing. Use whichever you prefer. Both take a schema path or URL plus options like --url and --header.
Can Schemathesis replace my functional API tests?
No, and it isn’t meant to. Schemathesis checks that your API doesn’t crash and that responses match the schema. It doesn’t verify business logic, like whether a discount calculation is correct. You still want example-based functional and contract testing for that, which you can build and run in Apidog. Treat Schemathesis as an additional fuzzing layer, not a replacement.
Does Schemathesis work with GraphQL?
Yes. Schemathesis supports both OpenAPI and GraphQL schemas. For GraphQL, it generates queries from the schema’s type definitions and checks the responses the same way it does for REST APIs defined in OpenAPI.
Conclusion
Schemathesis is a sharp tool for one job: taking your OpenAPI or GraphQL spec and stress-testing your live API against it with property-based generation. It finds the crashes and contract violations that example-based tests miss, and it costs you almost nothing to add to CI. What it doesn’t do is design, mock, document, or verify business logic.
That’s where Apidog complements it. Design the spec, generate functional tests and mocks, run them in CI with apidog run, and document the result, all in one place, then layer Schemathesis on top for fuzzing. Download Apidog to build the design and functional side of that workflow, and let Schemathesis handle the edge-case hunt.
