Artillery Load Testing: A Practical Guide for APIs

Artillery is an open-source, Node.js load-testing toolkit that drives high-concurrency traffic at your API from a simple YAML script. You define load phases and request flows, run artillery run script.yml, and read back latency percentiles, request rates, and error counts. This guide walks you through installing Artillery v2, writing a real test script, running it, capturing results the current v2 way, and wiring it into CI.

What Artillery Is and When to Reach for It

Artillery generates virtual users (VUs) that hit your endpoints and measures how the system holds up under sustained traffic. A virtual user is a simulated client that runs through a scenario, one request after another, just like a real caller would.

You reach for Artillery when you need answers to scale questions. How does p95 latency behave at 50 requests per second? At what arrival rate do errors start appearing? Does the API stay stable for five minutes of sustained load, or does it degrade?

Artillery is good at this because the test is declarative. You describe the load shape in YAML instead of hand-coding a concurrency loop. It runs anywhere Node.js runs, so the same script works on your laptop and in CI.

Artillery is one of several options in this space. If you are still comparing tools, the top load testing tools roundup and this load testing software comparison cover the trade-offs across k6, JMeter, Gatling, and others.

Install Artillery (v2)

The package name is exactly artillery, and the current major version is v2. Install it globally with npm, then verify the version.

npm install -g artillery@latest
artillery version

You need a recent LTS release of Node.js. Artillery runs on Windows, macOS, and Linux.

If you would rather not install anything globally, run it on demand with npx.

npx artillery@latest run script.yml

Writing an Artillery Test Script

An Artillery test is a YAML file with two top-level sections. The config section defines the target and the load profile. The scenarios section defines what each virtual user does.

Here is a complete script that warms up, ramps to peak, then holds a sustained load.

config:
  target: "https://api.example.com"
  phases:
    - name: "Warm up"
      duration: 60
      arrivalRate: 5
    - name: "Ramp to peak"
      duration: 120
      arrivalRate: 5
      rampTo: 50
    - name: "Sustained load"
      duration: 300
      arrivalRate: 50
      maxVusers: 500
  # Inline variables (or use a CSV via config.payload)
  variables:
    productId:
      - "1001"
      - "1002"

scenarios:
  - name: "Browse and create order"
    flow:
      - get:
          url: "/v1/products/{{ productId }}"
      - post:
          url: "/v1/orders"
          json:
            productId: "{{ productId }}"
            quantity: 2

Understanding the config section

config.target is the base host every request runs against. Each step in a scenario appends its url to this base.

config.phases is an array of load phases that run in order. The keys you will use most:

• duration: how long the phase lasts, in seconds or a human-readable string like "5m".
• arrivalRate: how many new virtual users start each second.
• rampTo: linearly ramps the arrival rate from arrivalRate up to this value across the phase.
• arrivalCount: a fixed number of VUs spread across the phase, instead of a per-second rate.
• maxVusers: a cap on how many virtual users can run concurrently.
• name: a label that shows up in the output.

One detail trips people up. A phase’s duration controls how long Artillery keeps spawning virtual users, not the total wall-clock time of the test. If a VU starts near the end of a phase and its scenario takes a while, the run continues until that user finishes.

Understanding the scenarios section

scenarios is an array. Each scenario has a flow, which is the ordered list of steps a virtual user runs. Optional keys include name and weight, where weight sets the relative probability that Artillery picks this scenario for a given VU.

Flow steps use HTTP-verb keys: get, post, put, delete, and patch. Each takes a url, and request bodies go under json. The double-curly syntax, {{ productId }}, pulls in a variable.

Driving requests from a CSV file

Hard-coding values is fine for a smoke test. For realistic load, feed data from a CSV with config.payload. Each virtual user picks a row, and the column names become variables.

config:
  target: "https://api.example.com"
  payload:
    path: "./users.csv"
    fields:
      - "email"
      - "password"
  phases:
    - duration: 120
      arrivalRate: 20
scenarios:
  - flow:
      - post:
          url: "/login"
          json:
            email: "{{ email }}"
            password: "{{ password }}"

Running the Test

The basic command points Artillery at your script.

artillery run script.yml

# Override target without editing the script:
artillery run --target https://staging.example.com script.yml

# Pass variables as JSON:
artillery run -v '{ "productId": ["1001","1002"] }' script.yml

A few flags are worth knowing. --target (or -t) overrides config.target so you can point the same script at staging or production. --environment (or -e) selects a named block under config.environments. --config (or -c) loads config from a separate file. --insecure (or -k) skips TLS verification for self-signed certs in test environments.

Reading the Results

While the test runs, Artillery prints aggregate metrics roughly every 10 seconds. When it finishes, you get a summary report. The numbers that matter most:

• Request rate: requests per second the run actually achieved.
• Latency percentiles: p50 (median), p95, and p99 response times. The p95 tells you the experience of the slowest 5 percent of requests, which is usually where pain shows up first.
• Error counts: failed requests, timeouts, and non-2xx responses, grouped by type.

Watch the tail latencies, not just the average. An average can look healthy while p99 quietly climbs into multi-second territory. If errors appear only during the sustained phase, you have likely found a saturation point worth investigating. For a deeper treatment of which metrics to track and why, see this API performance testing guide.

Generating a Report in Artillery v2

Reporting changed across Artillery versions, so this is where outdated tutorials lead you astray. Older guides tell you to run artillery run --output report.json and then artillery report report.json to produce an HTML file. The first half still works. The second half does not.

The --output flag still writes a machine-readable JSON results file.

# Write machine-readable JSON results (still supported):
artillery run --output report.json script.yml

The artillery report command, the JSON-to-HTML generator, has been removed from the Artillery CLI. The official docs state it “is no longer supported and has been removed from the Artillery CLI.” The HTML reporting code went unmaintained, was deprecated, and then dropped, with no plans to bring it back. Do not write artillery report report.json; it will not work on current v2.

You have three current options instead.

First, parse the JSON yourself. This is ideal for CI, where you want to assert against a threshold. Pull the aggregate p95 latency with jq:

jq '.aggregate.summaries["http.response_time"].p95' report.json

Second, use Artillery Cloud for a hosted dashboard. This is the official replacement for the old HTML report. Pass --record with your API key.

artillery run --record --key $ARTILLERY_CLOUD_API_KEY script.yml

Third, push metrics to your own monitoring stack with the publish-metrics plugin or OpenTelemetry, so latency and error rates land in the same dashboards you already use for production.

Running Artillery in CI

Because Artillery is just a Node.js CLI, it slots into any pipeline. Here is a GitHub Actions workflow that installs Artillery, runs the test, and uploads the JSON report as a build artifact.

name: Load test
on: [workflow_dispatch]
jobs:
  artillery:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "lts/*"
      - run: npm install -g artillery@latest
      - run: artillery run --output report.json script.yml
      - uses: actions/upload-artifact@v4
        with:
          name: artillery-report
          path: report.json

This example runs on manual dispatch. Heavy load tests are usually triggered on demand or on a schedule rather than on every commit, since they take minutes and consume real bandwidth. Once the JSON report exists, you can add a jq step that fails the job if p95 exceeds your budget.

Where Apidog Fits: Functional Testing and CI Gating

Artillery answers “can the API survive this much traffic?” That is load and performance testing. A different question runs alongside it: “does the API still return correct responses after this code change?” That is functional and regression testing, and it is where Apidog fits.

Apidog is an all-in-one API platform for design, debugging, mocking, documentation, and automated testing. Its test scenarios group endpoints into logical steps with conditions like if, for, and foreach, so you can validate response bodies, status codes, and contracts. You run those scenarios in CI with the Apidog CLI to gate merges after code changes.

Be clear about the boundary. Apidog does include a performance-testing feature, but it is capped at up to 100 virtual users. That is enough to spot obvious regressions, and it is not a substitute for Artillery at high concurrency. For distributed, code-modeled load at massive scale, Artillery is the right tool. This same honest framing shows up in our writeup on load-testing APIs without Python, and the mechanics of Apidog’s 100-VU feature are covered in API performance testing in Apidog.

So use both. Load-test with Artillery for scale. Run functional and regression checks in CI with the Apidog CLI to catch broken behavior before it ships.

The Apidog CLI installs from npm, and apidog run is flag-only.

# Apidog CLI: functional/regression run in CI (flag-only, no positional file)
npm install -g apidog-cli
apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t <TEST_SCENARIO_ID> \
  -e <ENVIRONMENT_ID> \
  -r cli,junit \
  --out-dir ./apidog-reports

The -t flag is the test scenario ID, -e is the required environment ID, and -r cli,junit emits both console output and a JUnit XML report that CI systems can read. For a step-by-step walkthrough, see the Apidog CLI tutorial, and for pipeline design patterns, see these CI/CD best practices for API testing.

Want functional and contract tests gating your CI alongside your Artillery load runs? Download Apidog for free and build your first test scenario.

Frequently Asked Questions

What is Artillery load testing?

Artillery load testing is the practice of using the open-source Artillery toolkit to simulate many concurrent virtual users hitting your API. You describe the load shape and request flow in a YAML script, run it, and measure latency percentiles, request rates, and errors to see how your system behaves under stress.

Is Artillery free and open source?

Yes. The core Artillery CLI is free and open source, distributed on npm as the artillery package. There is also a paid hosted offering, Artillery Cloud, which provides a dashboard for results, but you can run full load tests locally and in CI without it.

How do you run an Artillery load test?

Install it with npm install -g artillery@latest, write a YAML script with a config block (target and phases) and a scenarios block (the request flow), then run artillery run script.yml. Artillery prints live metrics every 10 seconds and a summary at the end.

How do you generate an Artillery report?

Run artillery run --output report.json script.yml to write a JSON results file. The old artillery report command that produced HTML has been removed from the CLI. Instead, parse the JSON with a tool like jq, use Artillery Cloud via --record --key, or push metrics out with the publish-metrics or OpenTelemetry plugin.

Artillery vs k6 or JMeter: which should you use?

All three handle high-scale load. Artillery uses declarative YAML and Node.js, which suits teams already in the JavaScript ecosystem. k6 scripts in JavaScript with a code-first model. JMeter is GUI-driven and Java-based with a long plugin history. The Gatling vs JMeter comparison covers the trade-offs in more depth. Pick the one whose scripting model matches your team, then pair it with functional CI tests for full coverage.