1
0
Fork 0
mirror of https://github.com/NixOS/nix synced 2025-06-25 10:41:16 +02:00

Add 'download-buffer-size' setting

We are piping curl downloads into `unpackTarfileToSink()`, but the
latter is typically slower than the former if you're on a fast
connection. So the download could appear unnecessarily slow. (There is
even a risk that if the Git import is *really* slow for whatever
reason, the TCP connection could time out.)

So let's make the download buffer bigger by default - 64 MiB is big
enough for the Nixpkgs tarball. Perhaps in the future, we could have
an unlimited buffer that spills data to disk beyond a certain
threshold, but that's probably overkill.
This commit is contained in:
Eelco Dolstra 2024-07-24 20:10:45 +02:00
parent caf4e98f0c
commit 8ffea0a018
2 changed files with 7 additions and 1 deletions

View file

@ -858,7 +858,7 @@ void FileTransfer::download(
buffer). We don't wait forever to prevent stalling the buffer). We don't wait forever to prevent stalling the
download thread. (Hopefully sleeping will throttle the download thread. (Hopefully sleeping will throttle the
sender.) */ sender.) */
if (state->data.size() > 1024 * 1024) { if (state->data.size() > fileTransferSettings.downloadBufferSize) {
debug("download buffer is full; going to sleep"); debug("download buffer is full; going to sleep");
state.wait_for(state->request, std::chrono::seconds(10)); state.wait_for(state->request, std::chrono::seconds(10));
} }

View file

@ -47,6 +47,12 @@ struct FileTransferSettings : Config
Setting<unsigned int> tries{this, 5, "download-attempts", Setting<unsigned int> tries{this, 5, "download-attempts",
"How often Nix will attempt to download a file before giving up."}; "How often Nix will attempt to download a file before giving up."};
Setting<size_t> downloadBufferSize{this, 64 * 1024 * 1024, "download-buffer-size",
R"(
The size of Nix's internal download buffer during `curl` transfers. If data is
not processed quickly enough to exceed the size of this buffer, downloads may stall.
)"};
}; };
extern FileTransferSettings fileTransferSettings; extern FileTransferSettings fileTransferSettings;