diff --git a/README.md b/README.md index f7abebb..59ba8d3 100644 --- a/README.md +++ b/README.md @@ -447,6 +447,8 @@ svr.set_pre_compression_logger([](const httplib::Request& req, const httplib::Re The pre-compression logger is only called when compression would be applied. For responses without compression, only the access logger is called. +For a static file response (see [Static file compression](#static-file-compression)), `res.body` is empty when the logger runs. The bytes are still on disk at that point, not in memory. + #### Error Logging Error loggers capture failed requests and connection issues. Unlike access loggers, error loggers only receive the Error and Request information, as errors typically occur before a meaningful Response can be generated. @@ -1466,6 +1468,32 @@ The server can apply compression to the following MIME type contents: - application/protobuf - application/xhtml+xml +### Static file compression + +Responses served from a file, whether through `set_mount_point()` or `Response::set_file_content()`, are sent as is by default. Turn compression on for them with: + +```c++ +svr.set_static_file_compression(true); +``` + +Only files within a size range are compressed, and both ends of it can be moved: + +```c++ +svr.set_static_file_compression_min_length(512); +svr.set_static_file_compression_max_length(1024 * 1024); +``` + +The lower bound defaults to 1400 bytes. A response that already fits in a single 1500-byte MTU is not delivered any faster for being smaller, and a file of a few bytes comes back larger than it went in, since gzip's header and trailer outweigh what deflate saves. `0` compresses everything down to a single byte, and `CPPHTTPLIB_STATIC_FILE_COMPRESSION_MIN_LENGTH` sets the default at compile time. An empty file is never compressed regardless. + +The upper bound defaults to 4MB, and exists for a different reason: the file is compressed per request, and the compressed bytes are held in memory until the response has been written, so the peak cost scales with the number of requests in flight. It is a bound on what one request can cost, not a statement about how well large files compress, which is why raising it is reasonable when the files are known and the traffic is not. `0` removes the limit, and `CPPHTTPLIB_STATIC_FILE_COMPRESSION_MAX_LENGTH` sets the default at compile time. + +A compressed response keeps its `Content-Length`, so `HEAD` still reports the size a `GET` would return. Two details are worth knowing: + +- Range requests are answered from the uncompressed representation, so `Content-Range` keeps naming the file's own bytes. +- The `ETag` carries the coding it belongs to (`W/"...-gzip"`), so a client that cached the compressed form revalidates against the right validator. + +Content providers registered with `set_content_provider()` are not covered. Feeding one through a compressor would hold each write back until the compressor's window filled, which breaks providers that produce their body incrementally. Use `set_chunked_content_provider()` to compress a generated body. + ### Zlib Support 'gzip' compression is available with `CPPHTTPLIB_ZLIB_SUPPORT`. `libz` should be linked. diff --git a/docs-src/pages/en/cookbook/s08-compress-response.md b/docs-src/pages/en/cookbook/s08-compress-response.md index a03877e..9cc67a0 100644 --- a/docs-src/pages/en/cookbook/s08-compress-response.md +++ b/docs-src/pages/en/cookbook/s08-compress-response.md @@ -48,6 +48,31 @@ svr.Get("/events", [](const httplib::Request &req, httplib::Response &res) { }); ``` -> **Note:** Tiny responses barely benefit from compression and just waste CPU time. cpp-httplib skips compression for bodies that are too small to bother with. +## Static files need to be opted in + +Files served as they are, through `set_mount_point()` or `Response::set_file_content()`, are not compressed by default. Turn it on with: + +```cpp +svr.set_static_file_compression(true); +``` + +Only files within a size range are compressed, and both ends of it can be moved: + +```cpp +svr.set_static_file_compression_min_length(512); +svr.set_static_file_compression_max_length(1024 * 1024); +``` + +The lower bound defaults to 1400 bytes. A response that already fits in a single 1500-byte MTU is not delivered any faster for being smaller, and a file of a few bytes comes back larger than it went in, because gzip's header and trailer outweigh what deflate saves. + +The upper bound defaults to 4MB and exists for a different reason: the file is compressed on every request, and the compressed bytes stay in memory until the response has been written, so the peak cost scales with the number of requests in flight. It bounds what a single request can cost, and says nothing about how well large files compress, so raising it is reasonable when the files are known and the traffic is not. + +Either bound takes `0` to turn it off, and each has a compile-time default (`CPPHTTPLIB_STATIC_FILE_COMPRESSION_MIN_LENGTH`, `CPPHTTPLIB_STATIC_FILE_COMPRESSION_MAX_LENGTH`). + +A compressed response keeps its `Content-Length`, so `HEAD` reports the same size a `GET` would. Two details to know: Range requests are answered from the uncompressed representation, and the `ETag` carries the coding it belongs to, as in `W/"...-gzip"`. + +Content providers registered with `set_content_provider()` are not covered. Running one through a compressor holds each write back until the internal buffer fills, which stalls providers that build their body a piece at a time. To compress a generated body, use `set_chunked_content_provider()`. + +> **Note:** The size range covers static files only. A body passed to `set_content()` is compressed whenever the client accepts it and the MIME type is compressible, however small it is, so a response of a few bytes ends up larger than it started. Decide in the handler if you want to avoid that. > For the client-side counterpart, see [C15. Enable compression](../c15-compression). diff --git a/docs-src/pages/ja/cookbook/s08-compress-response.md b/docs-src/pages/ja/cookbook/s08-compress-response.md index 26100a1..cf56861 100644 --- a/docs-src/pages/ja/cookbook/s08-compress-response.md +++ b/docs-src/pages/ja/cookbook/s08-compress-response.md @@ -48,6 +48,31 @@ svr.Get("/events", [](const httplib::Request &req, httplib::Response &res) { }); ``` -> **Note:** 小さなレスポンスは圧縮しても効果が薄く、むしろCPU時間を無駄にすることがあります。cpp-httplibは小さすぎるボディは圧縮をスキップします。 +## 静的ファイルは明示的に有効にする + +`set_mount_point()`や`Response::set_file_content()`でファイルをそのまま返す場合、デフォルトでは圧縮されません。有効にするには次を呼びます。 + +```cpp +svr.set_static_file_compression(true); +``` + +圧縮の対象になるのは一定のサイズ範囲に収まるファイルだけで、上下どちらの境界も変更できます。 + +```cpp +svr.set_static_file_compression_min_length(512); +svr.set_static_file_compression_max_length(1024 * 1024); +``` + +下限のデフォルトは1400バイトです。1500バイトのMTUに収まるレスポンスは、小さくしたところで到達が速くなるわけではありません。さらに数バイトのファイルは、gzipのヘッダとトレーラがdeflateの削減分を上回るため、かえって大きくなって返ります。 + +上限のデフォルトは4MBで、こちらは理由が違います。リクエストのたびに圧縮が走り、圧縮後のバイト列はレスポンスを書き終えるまでメモリに載るため、ピーク時のコストが同時処理中のリクエスト数に比例するからです。つまり1リクエストあたりのコストを抑えるための値であって、大きいファイルは圧縮しても無駄だという意味ではありません。配信するファイルが分かっていてトラフィックがそれほど多くないなら、引き上げて構いません。 + +どちらの境界も`0`で無効にできます。コンパイル時のデフォルトは`CPPHTTPLIB_STATIC_FILE_COMPRESSION_MIN_LENGTH`と`CPPHTTPLIB_STATIC_FILE_COMPRESSION_MAX_LENGTH`で決まります。 + +圧縮しても`Content-Length`は付いたままなので、`HEAD`は`GET`と同じサイズを返します。細かい挙動として、Rangeリクエストは非圧縮の表現から切り出して返し、`ETag`には`W/"...-gzip"`のように使われた圧縮方式が入ります。 + +なお`set_content_provider()`で登録したコンテンツプロバイダは対象外です。圧縮器を通すと、内部バッファが埋まるまで書き込みが送出されず、ボディを少しずつ生成するプロバイダが止まってしまうためです。生成したボディを圧縮したい場合は`set_chunked_content_provider()`を使ってください。 + +> **Note:** サイズ範囲が効くのは静的ファイルだけです。`set_content()`に渡したボディは、圧縮対象のMIMEタイプでクライアントが受け入れていれば、大きさによらず圧縮されます。数バイトのレスポンスはgzipのヘッダ分だけかえって大きくなるので、避けたい場合はハンドラ側で判断してください。 > クライアント側の挙動は[C15. 圧縮を有効にする](../c15-compression)を参照してください。 diff --git a/httplib.h b/httplib.h index 6866085..3e855bc 100644 --- a/httplib.h +++ b/httplib.h @@ -134,6 +134,16 @@ #define CPPHTTPLIB_FORM_URL_ENCODED_PAYLOAD_MAX_LENGTH 8192 #endif +#ifndef CPPHTTPLIB_STATIC_FILE_COMPRESSION_MIN_LENGTH +// 1400 rather than a round number: a body that already fits in one 1500-byte +// MTU gains nothing from being made smaller. +#define CPPHTTPLIB_STATIC_FILE_COMPRESSION_MIN_LENGTH 1400 +#endif + +#ifndef CPPHTTPLIB_STATIC_FILE_COMPRESSION_MAX_LENGTH +#define CPPHTTPLIB_STATIC_FILE_COMPRESSION_MAX_LENGTH (4 * 1024 * 1024) // 4MB +#endif + #ifndef CPPHTTPLIB_RANGE_MAX_COUNT #define CPPHTTPLIB_RANGE_MAX_COUNT 1024 #endif @@ -1733,6 +1743,14 @@ struct Request { #endif }; +namespace detail { + +// Declared up here, away from the rest of the compression helpers, because +// `Response` stores one. +enum class EncodingType { None = 0, Gzip, Brotli, Zstd }; + +} // namespace detail + struct Response { std::string version; int status = -1; @@ -1798,6 +1816,11 @@ struct Response { bool content_provider_success_ = false; std::string file_content_path_; std::string file_content_content_type_; + + // Content coding chosen for a file-backed content provider, decided once + // where the file is opened so that the ETag and the body cannot disagree. + // `EncodingType::None` for every other kind of response. + detail::EncodingType file_content_encoding_ = detail::EncodingType::None; }; enum class Error { @@ -2200,6 +2223,10 @@ public: Server &set_payload_max_length(size_t length); + Server &set_static_file_compression(bool on); + Server &set_static_file_compression_min_length(size_t length); + Server &set_static_file_compression_max_length(size_t length); + Server &set_websocket_ping_interval(time_t sec); template Server &set_websocket_ping_interval( @@ -2270,6 +2297,11 @@ protected: time_t idle_interval_sec_ = CPPHTTPLIB_IDLE_INTERVAL_SECOND; time_t idle_interval_usec_ = CPPHTTPLIB_IDLE_INTERVAL_USECOND; size_t payload_max_length_ = CPPHTTPLIB_PAYLOAD_MAX_LENGTH; + bool static_file_compression_ = false; + size_t static_file_compression_min_length_ = + CPPHTTPLIB_STATIC_FILE_COMPRESSION_MIN_LENGTH; + size_t static_file_compression_max_length_ = + CPPHTTPLIB_STATIC_FILE_COMPRESSION_MAX_LENGTH; time_t websocket_ping_interval_sec_ = CPPHTTPLIB_WEBSOCKET_PING_INTERVAL_SECOND; int websocket_max_missed_pongs_ = CPPHTTPLIB_WEBSOCKET_MAX_MISSED_PONGS; @@ -2326,6 +2358,10 @@ private: const HandlersForContentReader &handlers) const; bool parse_request_line(const char *s, Request &req) const; + detail::EncodingType static_file_encoding(const Request &req, + const std::string &content_type, + size_t length) const; + bool apply_static_file_compression(const Request &req, Response &res) const; void apply_ranges(const Request &req, Response &res, std::string &content_type, std::string &boundary) const; bool write_response(Stream &strm, bool close_connection, Request &req, @@ -3625,7 +3661,7 @@ ssize_t send_socket(socket_t sock, const void *ptr, size_t size, int flags); ssize_t read_socket(socket_t sock, void *ptr, size_t size, int flags); -enum class EncodingType { None = 0, Gzip, Brotli, Zstd }; +EncodingType encoding_type(const Request &req, const std::string &content_type); EncodingType encoding_type(const Request &req, const Response &res); @@ -5075,7 +5111,8 @@ inline std::string from_i_to_hex(size_t n) { return ret; } -inline std::string compute_etag(const FileStat &fs) { +inline std::string compute_etag(const FileStat &fs, + const std::string &suffix = std::string()) { if (!fs.is_file()) { return std::string(); } // If mtime cannot be determined (negative value indicates an error @@ -5089,7 +5126,7 @@ inline std::string compute_etag(const FileStat &fs) { auto size = fs.size(); return std::string("W/\"") + from_i_to_hex(mtime) + "-" + - from_i_to_hex(size) + "\""; + from_i_to_hex(size) + suffix + "\""; } // Format time_t as HTTP-date (RFC 9110 Section 5.6.7): "Sun, 06 Nov 1994 @@ -7449,10 +7486,9 @@ inline bool parse_quality(const char *b, const char *e, std::string &token, return !invalid; } -inline EncodingType encoding_type(const Request &req, const Response &res) { - if (!can_compress_content_type(res.get_header_value("Content-Type"))) { - return EncodingType::None; - } +inline EncodingType encoding_type(const Request &req, + const std::string &content_type) { + if (!can_compress_content_type(content_type)) { return EncodingType::None; } auto s = get_combined_header_value(req.headers, "Accept-Encoding"); if (s.empty()) { return EncodingType::None; } @@ -7506,6 +7542,10 @@ inline EncodingType encoding_type(const Request &req, const Response &res) { return best; } +inline EncodingType encoding_type(const Request &req, const Response &res) { + return encoding_type(req, res.get_header_value("Content-Type")); +} + inline std::unique_ptr make_compressor(EncodingType type) { #ifdef CPPHTTPLIB_ZLIB_SUPPORT if (type == EncodingType::Gzip) { @@ -8531,6 +8571,67 @@ write_content_without_length(Stream &strm, // down } +// Runs a known-length content provider to completion and compresses what it +// writes into `out`. Nothing is buffered in identity form: a provider backed +// by an mmap hands the compressor a pointer straight into the mapping. +inline bool compress_content_provider(const ContentProvider &content_provider, + size_t length, compressor &cmp, + std::string &out) { + size_t offset = 0; + auto ok = true; + auto finished = false; + DataSink data_sink; + + auto append = [&](const char *data, size_t data_len) { + out.append(data, data_len); + return true; + }; + + data_sink.write = [&](const char *d, size_t l) -> bool { + if (!ok) { return false; } + offset += l; + if (l > 0 && !cmp.compress(d, l, false, append)) { ok = false; } + return ok; + }; + + // The body is framed by `length`, so a provider that reports itself done + // early has truncated it; the short-body check below turns that into a + // failure rather than calling the provider again forever. + data_sink.done = [&]() { finished = true; }; + + while (offset < length && !finished) { + auto prev_offset = offset; + if (!content_provider(offset, length - offset, data_sink) || !ok) { + return false; + } + // No Stream to block on here, so a provider that keeps returning true + // without writing would spin. Treat a pass that made no progress as a + // failure. + if (offset == prev_offset) { return false; } + } + + if (offset != length) { return false; } + + return cmp.compress(nullptr, 0, true, append); +} + +// Serves `m` as the response body. `set_content_provider()` clears the coding, +// so recording it has to come after; keeping both here means a third +// file-serving path cannot get that order wrong. +inline void set_file_content_provider(Response &res, + const std::shared_ptr &m, + const std::string &content_type, + EncodingType encoding) { + res.set_content_provider( + m->size(), content_type, + [m](size_t offset, size_t length, DataSink &sink) -> bool { + sink.write(m->data() + offset, length); + return true; + }); + + res.file_content_encoding_ = encoding; +} + template inline bool write_content_chunked(Stream &strm, const ContentProvider &content_provider, @@ -11392,6 +11493,7 @@ inline void Response::set_content(const char *s, size_t n, auto rng = headers.equal_range("Content-Type"); headers.erase(rng.first, rng.second); set_header("Content-Type", content_type); + file_content_encoding_ = detail::EncodingType::None; } inline void Response::set_content(const std::string &s, @@ -11406,6 +11508,7 @@ inline void Response::set_content(std::string &&s, auto rng = headers.equal_range("Content-Type"); headers.erase(rng.first, rng.second); set_header("Content-Type", content_type); + file_content_encoding_ = detail::EncodingType::None; } inline void Response::set_content_provider( @@ -11416,6 +11519,7 @@ inline void Response::set_content_provider( if (in_length > 0) { content_provider_ = std::move(provider); } content_provider_resource_releaser_ = std::move(resource_releaser); is_chunked_content_provider_ = false; + file_content_encoding_ = detail::EncodingType::None; } inline void Response::set_content_provider( @@ -11426,6 +11530,7 @@ inline void Response::set_content_provider( content_provider_ = detail::ContentProviderAdapter(std::move(provider)); content_provider_resource_releaser_ = std::move(resource_releaser); is_chunked_content_provider_ = false; + file_content_encoding_ = detail::EncodingType::None; } inline void Response::set_chunked_content_provider( @@ -11436,6 +11541,7 @@ inline void Response::set_chunked_content_provider( content_provider_ = detail::ContentProviderAdapter(std::move(provider)); content_provider_resource_releaser_ = std::move(resource_releaser); is_chunked_content_provider_ = true; + file_content_encoding_ = detail::EncodingType::None; } inline void Response::set_file_content(const std::string &path, @@ -12883,6 +12989,21 @@ inline Server &Server::set_payload_max_length(size_t length) { return *this; } +inline Server &Server::set_static_file_compression(bool on) { + static_file_compression_ = on; + return *this; +} + +inline Server &Server::set_static_file_compression_min_length(size_t length) { + static_file_compression_min_length_ = length; + return *this; +} + +inline Server &Server::set_static_file_compression_max_length(size_t length) { + static_file_compression_max_length_ = length; + return *this; +} + inline Server &Server::set_websocket_max_missed_pongs(int count) { websocket_max_missed_pongs_ = count; return *this; @@ -13339,7 +13460,29 @@ inline bool Server::handle_file_request(Request &req, Response &res) { res.set_header(kv.first, kv.second); } - auto etag = detail::compute_etag(stat); + auto content_type_of = [&]() { + return detail::find_content_type( + path, file_extension_and_mimetype_map_, default_file_mimetype_); + }; + + // Only the ETag needs the content type this early, and only to name + // the coding. Deciding it here would otherwise put a regex in front + // of the 304 below, which serving a file never used to pay for. + std::string content_type; + auto encoding = detail::EncodingType::None; + if (static_file_compression_) { + content_type = content_type_of(); + encoding = static_file_encoding(req, content_type, stat.size()); + } + + // The ETag names the representation actually sent, so a client that + // cached the compressed form revalidates against the compressed ETag + // and still gets a 304, while one that took identity keeps the plain + // ETag. + auto etag = detail::compute_etag( + stat, encoding == detail::EncodingType::None + ? std::string() + : std::string("-") + detail::encoding_name(encoding)); if (!etag.empty()) { res.set_header("ETag", etag); } auto mtime = stat.mtime(); @@ -13359,14 +13502,9 @@ inline bool Server::handle_file_request(Request &req, Response &res) { return false; } - res.set_content_provider( - mm->size(), - detail::find_content_type(path, file_extension_and_mimetype_map_, - default_file_mimetype_), - [mm](size_t offset, size_t length, DataSink &sink) -> bool { - sink.write(mm->data() + offset, length); - return true; - }); + if (!static_file_compression_) { content_type = content_type_of(); } + + detail::set_file_content_provider(res, mm, content_type, encoding); if (req.method != "HEAD" && file_request_handler_) { file_request_handler_(req, res); @@ -13745,9 +13883,88 @@ inline bool Server::dispatch_request(Request &req, Response &res, return false; } +// Decides the content coding for a response served straight from a file. Both +// the ETag, which has to name the representation actually sent, and +// `apply_static_file_compression()` go through this, so the two cannot drift +// apart. +inline detail::EncodingType Server::static_file_encoding( + const Request &req, const std::string &content_type, size_t length) const { + if (!static_file_compression_) { return detail::EncodingType::None; } + + // Nothing to compress, and an empty file already answers with + // `Content-Length: 0`. Checked on its own so that a zero floor still cannot + // turn an empty body into a 20-byte gzip stream. + if (length == 0) { return detail::EncodingType::None; } + + // A file that already fits in a single packet gains nothing from being made + // smaller, since it still travels in that one segment, and a file of a few + // bytes comes out larger than it went in. + if (length < static_file_compression_min_length_) { + return detail::EncodingType::None; + } + + // RFC 9110 applies Range to the representation after content coding, so a + // compressed 206 would mean compressing the whole file and then slicing it. + // Serve ranges from the identity representation instead. + if (!req.ranges.empty()) { return detail::EncodingType::None; } + + if (static_file_compression_max_length_ > 0 && + length > static_file_compression_max_length_) { + return detail::EncodingType::None; + } + + return detail::encoding_type(req, content_type); +} + +// Compresses a file-backed content provider into `res.body` and takes over the +// framing headers. Returns false when the response is left untouched. +inline bool Server::apply_static_file_compression(const Request &req, + Response &res) const { + auto type = res.file_content_encoding_; + if (type == detail::EncodingType::None || !res.content_provider_) { + return false; + } + + auto compressor = detail::make_compressor(type); + if (!compressor) { return false; } + + output_pre_compression_log(req, res); + + std::string compressed; + if (!detail::compress_content_provider(res.content_provider_, + res.content_length_, *compressor, + compressed)) { + return false; + } + + res.body.swap(compressed); + + // The provider was consumed in full, so a resource releaser registered with + // it should hear about a success when the response goes away. + res.content_provider_success_ = true; + res.content_provider_ = nullptr; + res.content_length_ = 0; + res.file_content_encoding_ = detail::EncodingType::None; + + res.set_header("Content-Encoding", detail::encoding_name(type)); + res.set_header("Vary", "Accept-Encoding"); + res.set_header("Content-Length", std::to_string(res.body.size())); + + return true; +} + inline void Server::apply_ranges(const Request &req, Response &res, std::string &content_type, std::string &boundary) const { + // A known-length content provider leaves `res.body` empty, so the compressor + // at the end of this function never runs for one (issue #2545). A file-backed + // provider is fully readable right here, so compress it and answer with an + // ordinary body: `Content-Length` and HEAD keep working, and the response + // takes the same path as `set_content()` from here on. Range requests never + // get a content coding, so `Content-Range` still names identity bytes and + // none of the framing below applies. + if (apply_static_file_compression(req, res)) { return; } + if (req.ranges.size() > 1 && res.status == StatusCode::PartialContent_206) { auto it = res.headers.find("Content-Type"); if (it != res.headers.end()) { @@ -14180,12 +14397,9 @@ Server::process_request(Stream &strm, const std::string &remote_addr, path, file_extension_and_mimetype_map_, default_file_mimetype_); } - res.set_content_provider( - mm->size(), content_type, - [mm](size_t offset, size_t length, DataSink &sink) -> bool { - sink.write(mm->data() + offset, length); - return true; - }); + detail::set_file_content_provider( + res, mm, content_type, + static_file_encoding(req, content_type, mm->size())); } } diff --git a/test/test.cc b/test/test.cc index 083cc2d..95aadb1 100644 --- a/test/test.cc +++ b/test/test.cc @@ -8858,6 +8858,327 @@ TEST_F(ServerTest, MultipartFormDataGzip) { } #endif +#ifdef CPPHTTPLIB_ZLIB_SUPPORT +// Static file compression is opt-in, so these run their own server rather than +// flipping it on for the shared ServerTest fixture, which would silently +// rewrite what every other static file test is asserting. +class StaticFileCompressionTest : public ::testing::Test { +protected: + void TearDown() override { + if (t_.joinable()) { + svr_.stop(); + t_.join(); + } + } + + int start(const std::function &configure = nullptr) { + svr_.set_mount_point("/", "./www"); + + svr_.Get("/file_content", [](const Request & /*req*/, Response &res) { + res.set_file_content("./www/dir/index.html", "text/html"); + }); + + svr_.Get("/streamed", [](const Request & /*req*/, Response &res) { + res.set_content_provider( + 6, "text/plain", + [](size_t offset, size_t /*length*/, DataSink &sink) { + sink.write(offset < 3 ? "aaa" : "bbb", 3); + return true; + }); + }); + + svr_.Get("/slow-stream", [](const Request & /*req*/, Response &res) { + res.set_content_provider( + 1000, "text/plain", + [](size_t offset, size_t /*length*/, DataSink &sink) { + if (offset < 100) { + std::string data(100, 'A'); + sink.write(data.data(), data.size()); + return true; + } + std::this_thread::sleep_for(std::chrono::seconds(2)); + std::string data(900, 'B'); + sink.write(data.data(), data.size()); + return true; + }); + }); + + if (configure) { configure(svr_); } + + auto port = svr_.bind_to_any_port(HOST); + t_ = std::thread([&]() { svr_.listen_after_bind(); }); + svr_.wait_until_ready(); + return port; + } + + static void enable(Server &svr) { svr.set_static_file_compression(true); } + + // Every file under ./www except 1MB.txt is smaller than the default + // 1400-byte floor, so a test that needs a small file compressed has to lower + // it out of the way. + static void enable_without_floor(Server &svr) { + svr.set_static_file_compression(true); + svr.set_static_file_compression_min_length(0); + } + + Server svr_; + std::thread t_; +}; + +TEST_F(StaticFileCompressionTest, DisabledByDefault) { + auto port = start(); + + Client cli(HOST, port); + auto res = cli.Get("/dir/1MB.txt", Headers{{"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_FALSE(res->has_header("Content-Encoding")); + EXPECT_EQ("1048576", res->get_header_value("Content-Length")); + EXPECT_EQ(1048576U, res->body.size()); +} + +TEST_F(StaticFileCompressionTest, MountPoint) { + auto port = start(enable); + + Client cli(HOST, port); + cli.set_decompress(false); + auto res = cli.Get("/dir/1MB.txt", Headers{{"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_EQ("text/plain", res->get_header_value("Content-Type")); + EXPECT_EQ("gzip", res->get_header_value("Content-Encoding")); + EXPECT_EQ("Accept-Encoding", res->get_header_value("Vary")); + // The response stays self-delimiting: a length, not a chunked body. + EXPECT_FALSE(res->has_header("Transfer-Encoding")); + EXPECT_EQ(std::to_string(res->body.size()), + res->get_header_value("Content-Length")); + EXPECT_LT(res->body.size(), 1048576U); + EXPECT_FALSE(res->body.empty()); +} + +TEST_F(StaticFileCompressionTest, MountPointDecompressed) { + auto port = start(enable); + + Client cli(HOST, port); + auto res = cli.Get("/dir/1MB.txt", Headers{{"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_EQ("gzip", res->get_header_value("Content-Encoding")); + EXPECT_EQ(1048576U, res->body.size()); +} + +TEST_F(StaticFileCompressionTest, FileContent) { + auto port = start(enable_without_floor); + + Client cli(HOST, port); + auto res = cli.Get("/file_content", Headers{{"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_EQ("text/html", res->get_header_value("Content-Type")); + EXPECT_EQ("gzip", res->get_header_value("Content-Encoding")); + EXPECT_EQ(104U, res->body.size()); +} + +TEST_F(StaticFileCompressionTest, Head) { + auto port = start(enable); + + Client cli(HOST, port); + cli.set_decompress(false); + auto get = cli.Get("/dir/1MB.txt", Headers{{"Accept-Encoding", "gzip"}}); + ASSERT_TRUE(get) << "Error: " << to_string(get.error()); + + auto res = cli.Head("/dir/1MB.txt", Headers{{"Accept-Encoding", "gzip"}}); + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_EQ("gzip", res->get_header_value("Content-Encoding")); + EXPECT_FALSE(res->has_header("Transfer-Encoding")); + // HEAD still reports the size a GET would return. + EXPECT_EQ(get->get_header_value("Content-Length"), + res->get_header_value("Content-Length")); + EXPECT_TRUE(res->body.empty()); +} + +TEST_F(StaticFileCompressionTest, RangeStaysIdentity) { + auto port = start(enable); + + Client cli(HOST, port); + auto res = cli.Get("/dir/test.abcde", Headers{make_range_header({{2, 3}}), + {"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::PartialContent_206, res->status); + EXPECT_FALSE(res->has_header("Content-Encoding")); + EXPECT_EQ("2", res->get_header_value("Content-Length")); + EXPECT_EQ("bytes 2-3/5", res->get_header_value("Content-Range")); + EXPECT_EQ("cd", res->body); +} + +TEST_F(StaticFileCompressionTest, NotCompressibleType) { + auto port = start(enable); + + Client cli(HOST, port); + auto res = cli.Get("/file", Headers{{"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_EQ("application/octet-stream", res->get_header_value("Content-Type")); + EXPECT_FALSE(res->has_header("Content-Encoding")); + EXPECT_EQ("5", res->get_header_value("Content-Length")); +} + +TEST_F(StaticFileCompressionTest, EmptyFile) { + auto port = start(enable); + + Client cli(HOST, port); + auto res = cli.Get("/empty_file", Headers{{"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_FALSE(res->has_header("Content-Encoding")); + EXPECT_EQ("0", res->get_header_value("Content-Length")); + EXPECT_TRUE(res->body.empty()); +} + +TEST_F(StaticFileCompressionTest, MinLength) { + auto port = start(enable); + + Client cli(HOST, port); + + // 104 bytes, well under the default floor. Compressing it would add the + // gzip header and trailer to a body that already fits in one packet. + auto res = cli.Get("/dir/index.html", Headers{{"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_FALSE(res->has_header("Content-Encoding")); + EXPECT_EQ("104", res->get_header_value("Content-Length")); +} + +TEST_F(StaticFileCompressionTest, MinLengthLowered) { + auto port = start([](Server &svr) { + svr.set_static_file_compression(true); + svr.set_static_file_compression_min_length(1); + }); + + Client cli(HOST, port); + cli.set_decompress(false); + + auto res = cli.Get("/dir/test.html", Headers{{"Accept-Encoding", "gzip"}}); + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ("gzip", res->get_header_value("Content-Encoding")); + + // A 9-byte file comes back larger than it went in, because gzip's header and + // trailer outweigh anything deflate can save. That is the end of the range + // the floor exists to keep out. + EXPECT_GT(res->body.size(), 9U); +} + +TEST_F(StaticFileCompressionTest, MaxLength) { + auto port = start([](Server &svr) { + svr.set_static_file_compression(true); + svr.set_static_file_compression_min_length(0); + svr.set_static_file_compression_max_length(1024); + }); + + Client cli(HOST, port); + + // Over the limit: served as is. Not named `big`/`small`: + // defines `small` as a macro on Windows. + auto over = cli.Get("/dir/1MB.txt", Headers{{"Accept-Encoding", "gzip"}}); + ASSERT_TRUE(over) << "Error: " << to_string(over.error()); + EXPECT_FALSE(over->has_header("Content-Encoding")); + EXPECT_EQ("1048576", over->get_header_value("Content-Length")); + + // Under it: compressed. + auto under = cli.Get("/dir/index.html", Headers{{"Accept-Encoding", "gzip"}}); + ASSERT_TRUE(under) << "Error: " << to_string(under.error()); + EXPECT_EQ("gzip", under->get_header_value("Content-Encoding")); +} + +TEST_F(StaticFileCompressionTest, EtagPerEncoding) { + auto port = start(enable_without_floor); + + Client cli(HOST, port); + + auto identity = cli.Get("/dir/index.html", Headers{{"Accept-Encoding", ""}}); + ASSERT_TRUE(identity) << "Error: " << to_string(identity.error()); + EXPECT_FALSE(identity->has_header("Content-Encoding")); + auto identity_etag = identity->get_header_value("ETag"); + ASSERT_FALSE(identity_etag.empty()); + + auto gzipped = + cli.Get("/dir/index.html", Headers{{"Accept-Encoding", "gzip"}}); + ASSERT_TRUE(gzipped) << "Error: " << to_string(gzipped.error()); + EXPECT_EQ("gzip", gzipped->get_header_value("Content-Encoding")); + auto gzip_etag = gzipped->get_header_value("ETag"); + ASSERT_FALSE(gzip_etag.empty()); + + // Two representations, two validators. + EXPECT_NE(identity_etag, gzip_etag); + + // Each one revalidates against the request that would produce it... + auto fresh = + cli.Get("/dir/index.html", Headers{{"Accept-Encoding", "gzip"}, + {"If-None-Match", gzip_etag}}); + ASSERT_TRUE(fresh) << "Error: " << to_string(fresh.error()); + EXPECT_EQ(StatusCode::NotModified_304, fresh->status); + + auto fresh_identity = + cli.Get("/dir/index.html", Headers{{"Accept-Encoding", ""}, + {"If-None-Match", identity_etag}}); + ASSERT_TRUE(fresh_identity) << "Error: " << to_string(fresh_identity.error()); + EXPECT_EQ(StatusCode::NotModified_304, fresh_identity->status); + + // ...and not against the other representation. + auto crossed = + cli.Get("/dir/index.html", + Headers{{"Accept-Encoding", ""}, {"If-None-Match", gzip_etag}}); + ASSERT_TRUE(crossed) << "Error: " << to_string(crossed.error()); + EXPECT_EQ(StatusCode::OK_200, crossed->status); +} + +// The opt-in covers responses the server reads off disk itself. A caller's own +// known-length provider keeps its Content-Length and, more importantly, keeps +// delivering incrementally: routing it through a compressor would hold every +// write back until zlib's window filled. +TEST_F(StaticFileCompressionTest, KnownLengthProviderUnaffected) { + auto port = start(enable); + + Client cli(HOST, port); + auto res = cli.Get("/streamed", Headers{{"Accept-Encoding", "gzip"}}); + + ASSERT_TRUE(res) << "Error: " << to_string(res.error()); + EXPECT_EQ(StatusCode::OK_200, res->status); + EXPECT_FALSE(res->has_header("Content-Encoding")); + EXPECT_EQ("6", res->get_header_value("Content-Length")); + EXPECT_EQ("aaabbb", res->body); +} + +TEST_F(StaticFileCompressionTest, IncrementalProviderUnaffected) { + auto port = start(enable); + + Client cli(HOST, port); + cli.set_read_timeout(1, 0); + + auto handle = cli.open_stream("GET", "/slow-stream", Params{}, + Headers{{"Accept-Encoding", "gzip"}}); + ASSERT_TRUE(handle.is_valid()); + + // The provider writes 100 bytes and then stalls for longer than the read + // timeout. Those first bytes have to arrive anyway: run the response through + // a compressor and zlib holds them until its window fills, so the read would + // come back empty instead. + char buf[256]; + auto n = handle.read(buf, sizeof(buf)); + ASSERT_GT(n, 0) << "first read should return the bytes already written"; + EXPECT_EQ(std::string(100, 'A'), std::string(buf, static_cast(n))); +} +#endif + #ifdef CPPHTTPLIB_BROTLI_SUPPORT TEST_F(ServerTest, Brotli) { Headers headers;