> ## Documentation Index
> Fetch the complete documentation index at: https://codspeed.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# CodSpeed CLI

> Run performance tests from your terminal

The CodSpeed CLI allows you to benchmark **any executable program** and collect
performance data.

It is open source, and its source code is available on
[GitHub](https://github.com/CodSpeedHQ/codspeed).

## Getting Started

### Installation

Install the CodSpeed CLI using the installation script:

```bash theme={null}
curl -fsSL https://codspeed.io/install.sh | sh
```

After installation, authenticate with your CodSpeed account by running
`codspeed auth login`.

> [accordion] Installing a specific version
>
> To install a specific version:
>
> ```bash theme={null}
> curl -fsSL https://codspeed.io/v5.1.0/install.sh | sh
> ```
>
> Replace `v5.1.0` with the version you want to install.
>
> You can find all available versions on the
> [releases page](https://github.com/CodSpeedHQ/codspeed/releases?q=codspeed-runner).

> [!NOTE]
>
> The CodSpeed CLI officially supports Ubuntu 22.04, 24.04, 26.04 and Debian 12.
> Other Linux distributions can be used as long as the required tools for your
> chosen instruments are [manually installed](#manual-tool-installation) and
> available in your `PATH`.

### Run your first command

Let's run a simple benchmark using the walltime instrument:

```shellsession title=terminal icon="square-terminal" theme={null}
$ codspeed exec -- sleep 1

►►► Running the benchmarks
Executing: sleep 1
Completed 2 warmup rounds
Warmup done, now performing 3 rounds

►►► Uploading results
Linked repository: [...]
Performance data uploaded

►►► Benchmark results

sleep 1: 1.01 s

To see the full report, visit: https://app.codspeed.io/[...]
```

## Configuration

You can define benchmarks and options in a `codspeed.yml` configuration file.

Most importantly, items in the `benchmarks` list describe the commands that are
run when using `codspeed run`.

> [accordion] Other supported filenames
>
> The CodSpeed CLI looks for configuration files with the following names, by order of priority:
>
> 1. `codspeed.yml`
> 2. `codspeed.yaml`
> 3. `.codspeed.yml`
> 4. `.codspeed.yaml`
>
> You can also specify a custom path with `--config <path>`.

### Benchmark fields

**exec** `string` *required*

The command to execute. Can include arguments and flags.

**name** `string`

A descriptive name for the benchmark. This is used to identify the benchmark
in reports and results.

When omitted, the command is used as the benchmark name.

**options** `object`

Override global instrument options for this specific benchmark.

See the [Instruments](#instruments) section for available options for each
instrument and their effects.

### Running benchmarks

To run all benchmarks defined in your configuration file, specify the instrument
with the `-m` flag:

```shellsession title=terminal icon="square-terminal" theme={null}
$ codspeed run -m walltime

►►► Running the benchmarks

Executing: <commmand 1>
Completed 13 warmup rounds
Warmup done, now performing 20 rounds

Executing: <command 2>
A single warmup execution (517.ms) exceeded or met max_time (200ms).

Executing: <command 3>
Completed 23 warmup rounds
Warmup done, now performing 68 rounds

►►► Uploading results
Linked repository: [...]
Performance data uploaded

►►► Benchmark results

┌──────────────┬─────────────┐
│ Benchmark    │ Measurement │
├──────────────┼─────────────┤
│ <command 1>  │ 517.23 ms   │
├──────────────┼─────────────┤
│ <command 2>  │ 14.10 ms    │
├──────────────┼─────────────┤
│ <command 3>  │ 6.91 ms     │
└──────────────┴─────────────┘

To see the full report, visit: https://app.codspeed.io/[...]

```

You can also use `codspeed exec` to run a single command:

```shellsession wrap title=terminal icon="square-terminal" theme={null}
$ codspeed exec -m walltime --warmup-time 1ms --max-time 2s -- ./my-binary --arg1 value --arg2

►►► Running the benchmarks
Executing: ./my-binary --arg1 value --arg2

Completed 1 warmup rounds
Warmup done, now performing 115 rounds


►►► Uploading results
Linked repository: [...]
Performance data uploaded

►►► Benchmark results

./my-binary --arg1 value --arg2: 14.01 ms

To see the full report, visit: https://app.codspeed.io/[...]
```

### Example configuration

```yaml codspeed.yml theme={null}
$schema: https://raw.githubusercontent.com/CodSpeedHQ/codspeed/refs/heads/main/schemas/codspeed.schema.json

# Global options applied to all benchmarks
options:
  warmup-time: "0.2s"
  max-time: 1s

# List of benchmarks to run
benchmarks:
  - name: "<command 1>"
    exec: ./my_binary --arg1 value --arg2
    # Override global options for this benchmark
    options:
      max-rounds: 20

  - name: "<command 2>"
    exec: ./my_binary2
    options:
      max-time: 200ms

  - name: "<command 3>"
    exec: ./my_binary3 --flag
    options:
      min-time: 200ms
```

> [!NOTE]
>
> As displayed in the above example, we provide a schema for the configuration file
> [here](https://github.com/CodSpeedHQ/codspeed/blob/main/schemas/codspeed.schema.json).
>
> If your editor supports it, you can use the `$schema` field to enable
> autocompletion and validation for the `codspeed.yml` file.
>
> Please note that by default, the VSCode yaml language server does not support
> declaring the schema through the `$schema` field. One of the ways to declare the
> schema as is to add
>
> ```yaml codspeed.yaml theme={null}
> # yaml-language-server: $schema=https://raw.githubusercontent.com/CodSpeedHQ/codspeed/refs/heads/main/schemas/codspeed.schema.json
> ```
>
> More information in the
> [yaml-language-server documentation](https://github.com/redhat-developer/yaml-language-server?tab=readme-ov-file#using-inlined-schema).

## Instruments

CodSpeed supports three instruments for measuring different aspects of your
program's performance.

### Walltime

The walltime instrument measures **real-world execution time**. It runs your
benchmark multiple times and collects timing data. See the
[Walltime documentation](https://codspeed.io/docs/instruments/walltime.md) for more details.

#### How it works

1. **Warmup phase**: The benchmark runs repeatedly until the warmup time is
   reached. This allows the system to reach a steady state.

2. **Measurement phase**: Based on the average time from warmup, the CodSpeed
   CLI calculates how many rounds to run within the configured time bounds, then
   collects timing data for each round.

> [!NOTE]
>
> The walltime instrument requires `sudo` privileges to run because it needs to
> configure the kernel so that [linux
> perf](https://en.wikipedia.org/wiki/Perf_%28Linux%29) can collect all the
> necessary events.
>
> We plan to fine tune permissions in the future to reduce permissions to the
> strict minimum.

> [!NOTE]
>
> Profiling for interpreted languages (e.g., Python, JavaScript) is not
> currently supported with the walltime instrument.
>
> We are currently working on it and it's coming soon!

#### Walltime Options

**warmup\_time** `duration` (default: 1s)

Time spent in warmup runs. Set to `0` to disable warmup.

**min\_time** `duration`

Minimum time spent running measurement rounds.

**max\_time** `duration` (default: 3s)

Maximum time spent running measurement rounds.

**min\_rounds** `integer`

Minimum number of measurement rounds.

**max\_rounds** `integer`

Maximum number of measurement rounds.

> [!NOTE]
>
> If there is a conflict between time and round constraints, the CodSpeed CLI
> prioritizes satisfying the maximum bounds first.

#### Example

> [tab] CLI
>
> ```shellsession title=terminal icon="square-terminal" theme={null}
> $ codspeed exec -m walltime --warmup-time 100ms --max-time 1s -- <command>
>
> ►►► Running the benchmarks
> Executing: <command>
>
> Completed 7 warmup rounds
> Warmup done, now performing 63 rounds
>
>
> ►►► Uploading results
> Linked repository: [...]
> Performance data uploaded
>
> ►►► Benchmark results
>
> <command>: 14.15 ms
>
> To see the full report, visit:
> https://app.codspeed.io/[...]
> ```

> [tab] Configuration file
>
> ```yaml codspeed.yml theme={null}
> benchmarks:
>   - name: My command
>     exec: <command>
>     options:
>       warmup_time: 100ms
>       max_time: 1s
> ```
>
> ```shellsession title=terminal icon="square-terminal" theme={null}
> $ codspeed run
>
> ►►► Running the benchmarks
> Executing: My command
>
> Completed 7 warmup rounds
> Warmup done, now performing 63 rounds
>
>
>
> ►►► Uploading results
> Linked repository: [...]
> Performance data uploaded
>
> ►►► Benchmark results
>
> <command>: 14.15 ms
>
> To see the full report, visit:
> https://app.codspeed.io/[...]
> ```

### Memory

The memory instrument tracks **heap allocations** during benchmark execution.
Unlike walltime, the benchmark runs **only once** while memory usage is
recorded. See the [Memory documentation](https://codspeed.io/docs/instruments/memory.md) for more details.

It measures:

* Peak memory usage
* Total allocated memory
* Allocation count
* Average allocation size

See [Supported Allocators](https://codspeed.io/docs/instruments/memory.md#supported-allocators) for the
list of allocators that can be tracked.

> [!NOTE]
>
> The memory instrument requires `sudo` privileges to run because it uses
> [eBPF](https://en.wikipedia.org/wiki/EBPF) to track memory allocations.
>
> We plan to fine tune permissions in the future to reduce permissions to the
> strict minimum.

#### Example

```shellsession title=terminal icon="square-terminal" theme={null}
$ codspeed exec -m memory -- <command>

►►► Running the benchmarks
Executing: <command>


►►► Uploading results
Linked repository: [...]
Performance data uploaded

►►► Benchmark results

<command>: 7.8 MB

To see the full report, visit: https://app.codspeed.io/[...]
```

### Simulation

The simulation instrument uses **CPU simulation** to measure performance. The
benchmark runs **only once** and CPU behavior is simulated, providing consistent
measurements independent of system load. See the
[CPU Simulation documentation](https://codspeed.io/docs/instruments/cpu.md) for more details.

This is the same instrument used by
[CodSpeed integrations](https://codspeed.io/docs/benchmarks/overview.md) when running in CI.

> [!NOTE]
>
> Profiling for interpreted languages (e.g., Python, JavaScript) is not
> currently supported with the simulation instrument.
>
> We are currently working on it and it's coming soon!

#### Limitations

Statically linked executables are not currently supported with the simulation
instrument. If this is a limitation for your use case, please
[open a GitHub issue](https://github.com/CodSpeedHQ/codspeed/issues/new).

Additionally, the simulation instrument currently does not follow child
processes. Support for this is planned for a future release.

#### Example

```shellsession title=terminal icon="square-terminal" theme={null}
$ codspeed exec -m simulation -- <my_command>

►►► Running the benchmarks
Executing: <my_command>

►►► Uploading results
Linked repository: [...]
Performance data uploaded

►►► Benchmark results

<my_command>: 26.74 ms

To see the full report, visit: https://app.codspeed.io/[...]
```

### Setting instrument with `codspeed use`

If you do not want to always explicitly specify the instrument with `-m` when
running benchmarks, you can set an instrument for your whole shell session by
using

```shellsession title=terminal icon="square-terminal" theme={null}
$ codspeed use simulation
```

This will set the simulation instrument as the default for all subsequent
`codspeed run` and `codspeed exec` commands in the current shell session.

You can check the currently selected instrument with

```shellsession title=terminal icon="square-terminal" theme={null}
$ codspeed show
Simulation
```

> [!TIP]
>
> Even if you ran `codspeed use <instrument>`, you can still override the
> instrument for a specific command by using the `-m` flag with `codspeed run`
> or `codspeed exec`.

## Running the benchmarks in your CI

To generate performance reports, you need to run the benchmarks in your CI. This
allows CodSpeed to automatically run benchmarks and warn you about regressions
during development.

> [!TIP]
>
> If you want more details on how to configure the CodSpeed action, you can check
> out the [Continuous Reporting section](https://codspeed.io/docs/integrations/ci.md).

Here is an example of a GitHub Actions workflow that runs the benchmarks and
reports the results to CodSpeed on every push to the `main` branch and every
pull request:

```yaml .github/workflows/codspeed.yml icon="github" theme={null}
name: CodSpeed Benchmarks

on:
  push:
    branches:
      - "main" # or "master"
  pull_request:
  # `workflow_dispatch` allows CodSpeed to trigger backtest
  # performance analysis in order to generate initial data.
  workflow_dispatch:

jobs:
  benchmarks:
    name: Run benchmarks
    runs-on: ubuntu-latest
    permissions: # optional for public repositories
      contents: read # required for actions/checkout
      id-token: write # required for OIDC authentication with CodSpeed
    steps:
      - uses: actions/checkout@v7
      # ...
      # Setup your environment here:
      #  - Configure your Python/Rust/Node version
      #  - Install your dependencies
      #  - Build your executable (if using a compiled language)
      # ...
      - name: Run the benchmarks
        uses: CodSpeedHQ/action@v5
        with:
          mode: simulation
```

> [!NOTE]
>
> Contrary to other CI usages, **the `run` input is intentionally omitted** here.
>
> This is what defines whether the action will run benchmarks defined in your
> configuration file, or a benchmark that makes use of one of the
> [CodSpeed integrations](https://codspeed.io/docs/benchmarks/overview.md).

## Manual tool installation

On Ubuntu and Debian, the CodSpeed CLI automatically installs the tools required
by each instrument. On other distributions, you need to install them manually.

#### Walltime

The walltime instrument uses
[linux perf](https://en.wikipedia.org/wiki/Perf_%28Linux%29) for profiling,
which is enabled by default. Refer to your distribution's documentation for how
to install the `perf` tool.

If you don't need profiling data, you can disable it by setting the
`CODSPEED_PERF_ENABLED` environment variable to `false`.

> [!NOTE]
>
> On macOS, no additional tool installation is needed. Note that Apple system
> binaries cannot be profiled, see [Profiling System Processes on
> macOS](https://codspeed.io/docs/instruments/walltime/macos-profiling.md).

#### Simulation

The simulation instrument requires
[CodSpeed's fork of Valgrind](https://github.com/CodSpeedHQ/valgrind-codspeed).
You need to build and install it from source.

**Prerequisites**: `git`, `autoconf`, `automake`, `make`, and a C compiler.

```bash theme={null}
git clone https://github.com/CodSpeedHQ/valgrind-codspeed.git
cd valgrind-codspeed
./autogen.sh
./configure
make -j
make install
```

After installation, verify that the CodSpeed version of Valgrind is in your
`PATH`:

```shellsession theme={null}
$ valgrind --version
valgrind-3.26.0.codspeed
```

The `valgrind` binary in your `PATH` must be the CodSpeed version for the
simulation instrument to work correctly.

#### Memory

The memory instrument requires a Linux kernel with eBPF support. No additional
tool installation is needed, the CLI will download and load the necessary tools
automatically.

## Next Steps

> [card] [Profiling](https://codspeed.io/docs/features/profiling.md)
>
> Learn how to use flamegraphs and profiling data to optimize your code

> [card] [Instruments](https://codspeed.io/docs/instruments.md)
>
> Dive into instruments and learn how to measure different aspects of your
> program's performance


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.