mirror of
https://github.com/yhirose/cpp-httplib.git
synced 2026-08-12 05:11:24 +00:00
Fix benchmark-run reporting success without measuring anything
The first run of this workflow exposed three problems. Crow's amalgamated header includes <asio.hpp>, which no runner provides, so the build died immediately. It compiled locally only because CPATH happened to point at Homebrew's include directory. Install asio explicitly, and on macOS pass its include path through CROW_CXXFLAGS. The macos runner image has no Go, so bombardier could not be installed. Add actions/setup-go, which also pins a known toolchain on Linux. Worst of all, the ubuntu job reported success. `make ... | tee` returns tee's status, so the failed build was invisible. Enable pipefail. That alone is not enough: every recipe in benchmark/Makefile ends in `kill`, so make still exits 0 when bombardier itself fails to run. Assert that the expected number of "Reqs/sec" lines came out.
This commit is contained in:
37
.github/workflows/benchmark_run.yaml
vendored
37
.github/workflows/benchmark_run.yaml
vendored
@@ -43,17 +43,54 @@ jobs:
|
||||
- name: checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
# macos runners ship without Go.
|
||||
- name: setup Go
|
||||
uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version: stable
|
||||
|
||||
- name: install bombardier
|
||||
run: go install github.com/codesenberg/bombardier@latest
|
||||
|
||||
# crow_all.h includes <asio.hpp>, which no runner has out of the box.
|
||||
- name: install asio
|
||||
if: ${{ inputs.crow }}
|
||||
run: |
|
||||
if [ "$RUNNER_OS" = "Linux" ]; then
|
||||
sudo apt-get update && sudo apt-get install -y libasio-dev
|
||||
else
|
||||
brew install asio
|
||||
fi
|
||||
|
||||
- name: run benchmark
|
||||
run: |
|
||||
# Without pipefail the `tee` below swallows a build failure and the
|
||||
# job reports success having measured nothing.
|
||||
set -o pipefail
|
||||
export PATH="$(go env GOPATH)/bin:$PATH"
|
||||
CROW_FLAGS="-std=c++17"
|
||||
if [ "$RUNNER_OS" = "macOS" ]; then
|
||||
CROW_FLAGS="$CROW_FLAGS -I$(brew --prefix asio)/include"
|
||||
fi
|
||||
if [ "${{ inputs.crow }}" = "true" ]; then TARGET=bench-all; else TARGET=bench; fi
|
||||
make -C benchmark "$TARGET" \
|
||||
CROW_CXXFLAGS="$CROW_FLAGS" \
|
||||
BENCH="bombardier -c ${{ inputs.connections }} -d ${{ inputs.duration }} localhost:8080" \
|
||||
2>&1 | tee /tmp/bench.txt
|
||||
|
||||
# pipefail only catches a failed build. Each Makefile recipe ends in
|
||||
# `kill`, so a bombardier that never ran still leaves make happy — check
|
||||
# that the measurements are actually there.
|
||||
- name: check results were produced
|
||||
run: |
|
||||
expected=1
|
||||
if [ "${{ inputs.crow }}" = "true" ]; then expected=2; fi
|
||||
got=$(grep -c "Reqs/sec" /tmp/bench.txt || true)
|
||||
if [ "$got" -lt "$expected" ]; then
|
||||
echo "::error::expected $expected benchmark result(s), found $got"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: record results
|
||||
if: always()
|
||||
run: |
|
||||
|
||||
Reference in New Issue
Block a user