> ## Documentation Index
> Fetch the complete documentation index at: https://codspeed.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Writing Benchmarks in Java

> Create benchmarks for your Java codebase using Java Microbenchmark Harness

> [!NOTE]
>
> The JVM integration is still in early development, and **only the [walltime
> instrument](https://codspeed.io/docs/instruments/walltime.md) is currently supported**.
>
> If you have any feedback, please reach out to us via
> [Discord](https://discord.gg/MxpaCfKSqF) or
> [email our support](mailto:contact@codspeed.io).

Integrating CodSpeed into your Java project works through a fork of
[JMH](https://github.com/openjdk/jmh) (Java Microbenchmark Harness). You write
standard JMH benchmarks and swap in the CodSpeed JMH fork as a dependency. When
running in CI with CodSpeed, the results are automatically collected and
reported.

## Installation

CodSpeed provides a fork of JMH that collects walltime results and sends them to
CodSpeed.

> [tab] Gradle
>
> Add the CodSpeed JMH fork as a Git submodule:
>
> ```bash theme={null}
> git submodule add https://github.com/CodSpeedHQ/codspeed-jvm.git third-party/codspeed-jvm
> git submodule update --init --recursive -- third-party/codspeed-jvm
> ```
>
> Then include it as a composite build in your `settings.gradle.kts`, with
> dependency substitution to redirect JMH dependencies to the CodSpeed fork:
>
> ```kotlin settings.gradle.kts theme={null}
> includeBuild("third-party/codspeed-jvm/jmh-fork") {              // [!code ++]
>     dependencySubstitution {                          // [!code ++]
>         substitute(module("org.openjdk.jmh:jmh-core"))           // [!code ++]
>             .using(project(":jmh-core"))              // [!code ++]
>         substitute(module("org.openjdk.jmh:jmh-generator-annprocess")) // [!code ++]
>             .using(project(":jmh-generator-annprocess")) // [!code ++]
>     }                                                 // [!code ++]
> }                                                     // [!code ++]
> ```
>
> We recommend using the
> [JMH Gradle Plugin](https://github.com/melix/jmh-gradle-plugin) as it handles
> benchmark compilation and provides the `jmh` task. Add it to your
> `build.gradle.kts`:
>
> ```kotlin build.gradle.kts theme={null}
> plugins {
>     java
>     id("me.champeau.jmh") version "0.7.3"// [!code ++]
> }
> ```

> [tab] Maven
>
> Add the CodSpeed JMH fork as a Git submodule and publish it to your local Maven
> repository:
>
> ```bash theme={null}
> git submodule add https://github.com/CodSpeedHQ/codspeed-jvm.git third-party/codspeed-jvm
> git submodule update --init --recursive -- third-party/codspeed-jvm
> cd third-party/codspeed-jvm
> ./gradlew -p jmh-fork publishToMavenLocal
> ```
>
> We're planning to publish to Maven Central in the future. If you need this,
> please reach out via [Discord](https://discord.gg/MxpaCfKSqF) or
> [email](mailto:contact@codspeed.io).
>
> Then replace the JMH dependencies in your `pom.xml` with the CodSpeed fork.
> Maven resolves from `~/.m2/repository` by default. The repository entry below is
> only necessary if your local repository is at a non-default location:
>
> ```xml pom.xml theme={null}
> <repositories>                                        // [!code ++]
>     <repository>                                      // [!code ++]
>         <id>local</id>                                // [!code ++]
>         <url>file://${user.home}/.m2/repository</url> // [!code ++]
>     </repository>                                     // [!code ++]
> </repositories>                                       // [!code ++]
>
> <dependencies>
>     <dependency>
>         <groupId>org.openjdk.jmh</groupId>// [!code --]
>         <version>1.37</version>// [!code --]
>         <groupId>io.codspeed.jmh</groupId>// [!code ++]
>         <version>0.2.1</version>// [!code ++]
>         <artifactId>jmh-core</artifactId>
>     </dependency>
>     <dependency>
>         <groupId>org.openjdk.jmh</groupId>// [!code --]
>         <version>1.37</version>// [!code --]
>         <groupId>io.codspeed.jmh</groupId>// [!code ++]
>         <version>0.2.1</version>// [!code ++]
>         <artifactId>jmh-generator-annprocess</artifactId>
>     </dependency>
> </dependencies>
> ```
>
> > [!NOTE]
> >
> > Your Maven project must be configured to produce an executable benchmark JAR.
> > See the [official JMH setup
> > guide](https://github.com/openjdk/jmh?tab=readme-ov-file#preferred-usage-command-line)
> > for instructions.

## Creating benchmarks

Write your benchmarks using standard JMH annotations:

```java FibBenchmark.java theme={null}
package bench;

import org.openjdk.jmh.annotations.*;

public class FibBenchmark {

  @Benchmark
  public long fib() {
    return fib(30);
  }

  private static long fib(int n) {
    if (n <= 1) return n;
    return fib(n - 1) + fib(n - 2);
  }
}
```

> [!NOTE]
>
> JMH benchmarks must return their result or use `Blackhole.consume()` to
> prevent the JVM from eliminating dead code. All examples on this page return
> the computed value.

> [!TIP]
>
> For an in-depth tutorial on JMH, see the [How to Benchmark Java with
> JMH](https://codspeed.io/docs/guides/how-to-benchmark-java-with-jmh.md) guide.

## Testing the benchmarks locally

To run the benchmarks with CodSpeed locally, first install the `codspeed`
runner:

```bash theme={null}
curl -fsSL https://codspeed.io/install.sh | sh
```

Then run your benchmarks with CodSpeed:

> [tab] Gradle
>
> \$ codspeed run --mode walltime -- ./gradlew jmh
> ►►► Running the benchmarks
> \# JMH version: 0.2.1
> \# VM version: JDK 21.0.6, OpenJDK 64-Bit Server VM, 21.0.6+7
> \# Warmup: 5 iterations, 10 s each
> \# Measurement: 5 iterations, 10 s each
> \# Benchmark: bench.FibBenchmark.fib
>
> \# Run progress: 0.00% complete, ETA 00:01:40
> \# Warmup Iteration   1: 112.541 ns/op
> ...
>
> Benchmark              Mode  Cnt      Score      Error  Units
> FibBenchmark.fib       avgt    5    110.342 ±    2.153  ns/op

> [tab] Maven
>
> \$ mvn package -q
> \$ codspeed run --mode walltime -- java -jar target/benchmarks.jar
> ►►► Running the benchmarks
> \# JMH version: 0.2.1
> \# VM version: JDK 21.0.6, OpenJDK 64-Bit Server VM, 21.0.6+7
> \# Warmup: 5 iterations, 10 s each
> \# Measurement: 5 iterations, 10 s each
> \# Benchmark: bench.FibBenchmark.fib
>
> \# Run progress: 0.00% complete, ETA 00:01:40
> \# Warmup Iteration   1: 112.541 ns/op
> ...
>
> Benchmark              Mode  Cnt      Score      Error  Units
> FibBenchmark.fib       avgt    5    110.342 ±    2.153  ns/op

## Running the benchmarks in your CI

To generate performance reports, you need to run the benchmarks in your CI. This
allows CodSpeed to automatically run benchmarks and warn you about regressions
during development.

> [!TIP]
>
> If you want more details on how to configure the CodSpeed action, you can check
> out the [Continuous Reporting section](https://codspeed.io/docs/integrations/ci.md).

Here is an example of a GitHub Actions workflow that runs the benchmarks and
reports the results to CodSpeed on every push to the `main` branch and every
pull request:

> [tab] Gradle
>
> ```yaml .github/workflows/codspeed.yml icon="github" theme={null}
> name: CodSpeed Benchmarks
>
> on:
>   push:
>     branches:
>       - "main" # or "master"
>   pull_request:
>   # `workflow_dispatch` allows CodSpeed to trigger backtest
>   # performance analysis in order to generate initial data.
>   workflow_dispatch:
>
> jobs:
>   benchmarks:
>     name: Run benchmarks
>     runs-on: codspeed-macro-arm64-graviton-ubuntu-22-04
>     permissions: # optional for public repositories
>       contents: read # required for actions/checkout
>       id-token: write # required for OIDC authentication with CodSpeed
>     steps:
>       - uses: actions/checkout@v7
>         with:
>           submodules: recursive
>       - uses: actions/setup-java@v4
>         with:
>           distribution: temurin
>           java-version: 21
>       - name: Run the benchmarks
>         uses: CodSpeedHQ/action@v5
>         with:
>           mode: walltime
>           run: ./gradlew jmh
> ```

> [tab] Maven
>
> ```yaml .github/workflows/codspeed.yml icon="github" theme={null}
> name: CodSpeed Benchmarks
>
> on:
>   push:
>     branches:
>       - "main" # or "master"
>   pull_request:
>   # `workflow_dispatch` allows CodSpeed to trigger backtest
>   # performance analysis in order to generate initial data.
>   workflow_dispatch:
>
> jobs:
>   benchmarks:
>     name: Run benchmarks
>     runs-on: codspeed-macro-arm64-graviton-ubuntu-22-04
>     permissions: # optional for public repositories
>       contents: read # required for actions/checkout
>       id-token: write # required for OIDC authentication with CodSpeed
>     steps:
>       - uses: actions/checkout@v7
>         with:
>           submodules: recursive
>       - uses: actions/setup-java@v4
>         with:
>           distribution: temurin
>           java-version: 21
>       - name: Install CodSpeed JMH fork
>         run: cd third-party/codspeed-jvm && ./gradlew -p jmh-fork publishToMavenLocal
>       - name: Build benchmark JAR
>         run: mvn package -q
>       - name: Run the benchmarks
>         uses: CodSpeedHQ/action@v5
>         with:
>           mode: walltime
>           run: java -jar target/benchmarks.jar
> ```

## Advanced usage

JMH provides many features for writing expressive benchmarks. Below is a
selection that can be useful in CodSpeed benchmarks.

### Parameterized benchmarks

Use `@Param` to run the same benchmark with different input values:

```java ParamBenchmark.java theme={null}
package bench;

import org.openjdk.jmh.annotations.*;

@State(Scope.Benchmark)
public class ParamBenchmark {

  @Param({"10", "20", "30"})
  int n;

  @Benchmark
  public long fib() {
    return fib(n);
  }

  private static long fib(int n) {
    if (n <= 1) return n;
    return fib(n - 1) + fib(n - 2);
  }
}
```

### Shared state

Use `@State` to share setup logic across benchmarks and control the scope of the
state object:

```java StateBenchmark.java theme={null}
package bench;

import org.openjdk.jmh.annotations.*;
import java.util.ArrayList;
import java.util.List;

@State(Scope.Benchmark)
public class StateBenchmark {

  List<Integer> list;

  @Setup
  public void setup() {
    list = new ArrayList<>();
    for (int i = 0; i < 1000; i++) {
      list.add(i);
    }
  }

  @Benchmark
  public int sum() {
    return list.stream().mapToInt(Integer::intValue).sum();
  }
}
```

For a deeper dive into JMH, see the dedicated guide:

> [card] [How to Benchmark Java with JMH](https://codspeed.io/docs/guides/how-to-benchmark-java-with-jmh.md)
>
> An in-depth guide to writing JMH benchmarks: project setup, annotations,
> parameters, common pitfalls, and CodSpeed CI integration.

## Compatibility

* **JDK 21 or later** is required.
* Benchmark classes must not be final, since JMH subclasses them to generate the
  measurement harness.
* All standard JMH annotations are supported.
* Only the [Walltime instrument](https://codspeed.io/docs/instruments/walltime.md) is supported.
* CodSpeed uses a custom mode for all benchmarks to collect statistically
  significant results. Any `@BenchmarkMode` annotations in your code are
  ignored.

If you run into issues or require certain features, please
[open an issue](https://github.com/CodSpeedHQ/codspeed-jvm/issues) or
[join our Discord](https://discord.com/invite/MxpaCfKSqF) to get help.

## Next steps

> [card] [Example repository](https://github.com/CodSpeedHQ/codspeed-jvm)
>
> The CodSpeed JVM repository with example JMH benchmarks.

> [card] [How to Benchmark Java with JMH](https://codspeed.io/docs/guides/how-to-benchmark-java-with-jmh.md)
>
> An in-depth guide to writing JMH benchmarks with CodSpeed.

> [card] [Walltime instrument](https://codspeed.io/docs/instruments/walltime.md)
>
> Learn more about the Walltime instrument and how to use it.

> [card] [Dive into performance changes](https://codspeed.io/docs/features/profiling.md)
>
> Learn more about profiling and how to read flamegraphs.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.