Automated SQL Server Benchmarking with HammerDB and Docker: A Complete Testing Framework

I’m excited to announce the release of a new open-source project that fully automates HammerDB benchmarking for SQL Server using Docker. If you’ve ever needed to run TPC-C or TPC-H benchmarks multiple times, you know how time-consuming the manual setup can be. This project removes the hassle and gets you up and running a single command: ./loadtest.sh.

Why I Built This

In my work, I frequently benchmark SQL Server configurations, whether I’m comparing versions, testing new hardware, or validating performance tuning changes. Setting up HammerDB manually each time became a significant time bottleneck (see what I did there! ;). I needed an automated solution that would work consistently across different environments and reduce the time required to get test results.

What This Project Does

This project provides a complete benchmarking environment that automates the entire HammerDB workflow:

# Run everything with one command
./loadtest.sh

What happens when you run the command: 0. Starts up a SQL Server 2025 container

  1. Builds the TPC-C schema with your specified number of warehouses.
  2. Builds the TPC-H schema according to your scale factor.
  3. Runs the TPC-C OLTP workload.
  4. Runs the TPC-H analytical queries.
  5. Parses and saves all results.

The Architecture

Here’s a breakdown of what’s under the hood:

hammerdb/
├── hammerdb.env                   # All your settings in one place—edit this file based on the hardware you want to test.
├── docker-compose.yaml            # Orchestrates the containers.
├── loadtest.sh                    # Control script.
├── scripts/
│   ├── build_schema_tprocc.tcl    # Builds the TPC-C schema.
│   ├── build_schema_tproch.tcl    # Builds the TPC-H schema.
│   ├── load_test_tprocc.tcl       # Runs the TPC-C test.
│   ├── load_test_tproch.tcl       # Runs the TPC-H test.
│   ├── parse_output_tprocc.tcl    # Parses results for TPC-C.
│   └── parse_output_tproch.tcl    # Parses results for TPC-H.
└── output/                        # Results are stored here.

Every setting is parameterized through the hammerdb.env file, no more digging through TCL scripts or using a GUI to change configurations!

Getting Started: The 5-Minute Setup

# Clone the repository
git clone https://github.com/nocentino/hammerdb.git
cd hammerdb

# Configure for your environment
cp hammerdb.env.example hammerdb.env
# Edit hammerdb.env with your SQL Server connection details.

# Run everything!
./loadtest.sh

That’s it! This framework automates the entire process of building schemas, running tests, and collecting results. Once everything is set up, you can begin experimenting with more complex test scenarios to truly push your SQL Server to its limits. I have provided some examples and guidance in the GitHub repository. Just adjust the parameters based on your hardware and point the SQL_SERVER_HOST configuration hammerdb.env at your SQL Server an go!

Understanding the Results

The framework automatically extracts key metrics, including:

TPC-C Output:

  • Transactions Per Minute (TPM)
  • New Orders Per Minute (NOPM)
  • Response time percentiles

TPC-H Output:

  • Individual query execution times
  • Total runtime for all 22 queries
  • Query-specific metrics

Results are saved in both raw format (logs) and parsed format in the output/ directory.

Get the Code

The complete project is available on GitHub: https://github.com/nocentino/hammerdb. I’ve included extensive documentation and examples to help you get started quickly. The README contains additional details on customization options and advanced configurations.

A special thanks to Andrew Pruski, my partner in SQL Server benchmarking, for his contributions to this project. I couldn’t have done it without you!