14 Commits

Author	SHA1	Message	Date
yhirose	29ceccecd6	Add A/B throughput benchmark for comparing two refs ... The existing test_benchmark asserts a single request completes within 5ms, which catches gross connection-setup regressions but cannot see throughput changes: the effects we care about are tens of microseconds per request, a hundredth of that resolution. There was no way to answer "does this patch make the server faster" other than measuring by hand. Absolute req/s is not usable for that. Running the same binary five times on an idle 8-core machine gave 53.9k to 74.6k req/s, and shared CI runners are noisier still, so a number printed per push says nothing. Build both refs and measure them alternately in one session, flipping the order each round to cancel ordering bias, then report only the ratio of the medians. Whether that ratio means anything is decided by an exact permutation test rather than by comparing the change against the min/max spread - a single slow round is enough to make a spread check give up, while the rank test rides it out. Validated against a patch that removes a redundant poll() per request: individual measurements ranged 41.7k-93.9k req/s, yet nine rounds resolved a 1.244x speedup at p = 0.019. The same data truncated to five rounds was inconclusive, so the workflow defaults to nine. Manual dispatch only, Linux and non-SSL for now, and it never fails the build - this is a measurement, not a test.	2026-07-30 19:56:03 -04:00
yhirose	78ff94e604	Wait for server startup before running benchmark	2026-06-09 20:55:03 -04:00
yhirose	f29bb15f9d	Performance improvement!	2026-02-22 07:53:23 -05:00
yhirose	0c08c378d7	Simplify benchmark	2025-07-05 00:11:59 -04:00
yhirose	0bda3a7d1a	Update benchmark	2025-03-13 21:54:05 -04:00
yhirose	929dfbd348	Update copyright year	2025-01-20 00:32:10 -05:00
yhirose	a61b2427b0	Update benchmark base vertion to 0.18.0	2024-09-13 20:34:30 -04:00
yhirose	af4ece3d5f	Update benchmark/Makefile	2024-09-12 20:41:56 -04:00
yhirose	7b18ae6f16	Update benchmark	2024-09-08 10:56:13 -04:00
yhirose	3d6e315a4c	Fix #1923	2024-09-08 08:38:36 -04:00
yhirose	c43c51362a	Add monitor tool	2024-09-07 10:16:15 -04:00
yhirose	dfa641ca41	Misc	2024-09-05 22:54:48 -04:00
yhirose	adf65cfe61	Target C++11 for benchmark	2024-09-05 17:44:51 -04:00
yhirose	c75d071615	Add benchmark tool	2024-09-05 12:22:46 -04:00