1
0
Fork 0
mirror of https://github.com/NixOS/nix synced 2025-06-29 14:53:16 +02:00
Commit graph

391 commits

Author SHA1 Message Date
Eelco Dolstra
432a3a18d2 Move isUri() and resolveUri() out of filetransfer.cc
These are purely related to NIX_PATH / -I command line parsing, so put
them in libexpr.
2022-09-12 15:37:09 +02:00
Eelco Dolstra
71b155b9e6 Move download stuff into tarball.hh 2022-08-01 16:05:58 +02:00
Eelco Dolstra
1790698a74 Rename fetch() -> fetchToStore(), lazyFetch() -> getAccessor() 2022-07-25 16:30:50 +02:00
Eelco Dolstra
7a64bb7f1c Rename Input::fetch() to fetchToStore() 2022-07-25 15:48:31 +02:00
Eelco Dolstra
4f8f52ae58 Remove Tree datatype 2022-07-25 15:21:37 +02:00
Eelco Dolstra
a18b3c665a Store a ref to InputAccessor in SourcePath 2022-07-06 17:25:15 +02:00
Eelco Dolstra
0d14ffbcba Merge remote-tracking branch 'origin/master' into lazy-trees 2022-06-28 16:29:42 +02:00
Guillaume Girol
e8109cf405 fetchGit: document shallow argument 2022-06-26 12:00:00 +00:00
Eelco Dolstra
5f1340219b
Add builtins.patch
This replaces the 'patches' argument to builtins.fetchTree with
something more generic. So instead of 'builtins.fetchTree { patches =
... }' you can do 'builtins.patch { src = builtins.fetchTree { ... };
patchFiles = ... }'.
2022-06-10 13:10:22 +02:00
Eelco Dolstra
dfbb9a997f
Merge remote-tracking branch 'origin/master' into lazy-trees 2022-05-26 14:11:46 +02:00
Eelco Dolstra
91b7d5373a
Style tweaks 2022-05-25 12:32:22 +02:00
Eelco Dolstra
31d8f3369d
Fix fetchGit 2022-05-19 13:17:05 +02:00
Eelco Dolstra
066ed1c830
Restore previous fetchGit behaviour
For compatibility, it returns a store path again, rather than a
SourcePath.
2022-05-19 12:29:17 +02:00
Eelco Dolstra
a71f209330
Add CanonPath wrapper to represent canonicalized paths 2022-05-16 23:27:04 +02:00
Eelco Dolstra
84c273c503
Fix path concatenation 2022-05-12 14:48:25 +02:00
Eelco Dolstra
30ca717558 Merge remote-tracking branch 'origin/master' into lazy-trees 2022-05-09 11:26:39 +02:00
Ben Burdette
2c9fafdc9e trying debugThrow 2022-05-06 08:47:21 -06:00
Ben Burdette
6e19947993 Merge branch 'master' into debug-merge-master 2022-04-28 12:32:57 -06:00
pennae
8775be3393 store Symbols in a table as well, like positions
this slightly increases the amount of memory used for any given symbol, but this
increase is more than made up for if the symbol is referenced more than once in
the EvalState that holds it. on average every symbol should be referenced at
least twice (once to introduce a binding, once to use it), so we expect no
increase in memory on average.

symbol tables are limited to 2³² entries like position tables, and similar
arguments apply to why overflow is not likely: 2³² symbols would require as many
string instances (at 24 bytes each) and map entries (at 24 bytes or more each,
assuming that the map holds on average at most one item per bucket as the docs
say). a full symbol table would require at least 192GB of memory just for
symbols, which is well out of reach. (an ofborg eval of nixpks today creates
less than a million symbols!)
2022-04-21 21:56:31 +02:00
pennae
6526d1676b replace most Pos objects/ptrs with indexes into a position table
Pos objects are somewhat wasteful as they duplicate the origin file name and
input type for each object. on files that produce more than one Pos when parsed
this a sizeable waste of memory (one pointer per Pos). the same goes for
ptr<Pos> on 64 bit machines: parsing enough source to require 8 bytes to locate
a position would need at least 8GB of input and 64GB of expression memory. it's
not likely that we'll hit that any time soon, so we can use a uint32_t index to
locate positions instead.
2022-04-21 21:46:06 +02:00
Ben Burdette
27d45f9eb3 minor cleanup 2022-04-08 15:46:12 -06:00
Ben Burdette
b8b8ec7101 move throw to preverve Error type; turn off debugger for tryEval 2022-04-08 12:34:27 -06:00
Ben Burdette
1a93ac8133 Merge remote-tracking branch 'upstream/master' into upstream-merge 2022-04-07 13:42:01 -06:00
Eelco Dolstra
589f6f267b fetchClosure: Don't allow URL query parameters
Allowing this is a potential security hole, since it allows the user
to specify parameters like 'local-nar-cache'.
2022-04-06 11:52:51 +02:00
Eelco Dolstra
06c1edf889 Checkpoint 2022-03-29 11:01:14 +02:00
Eelco Dolstra
4b313ceb9e fetchTree: Support applying patches
You can now write

  fetchTree {
    type = "github";
    owner = "NixOS";
    repo = "nixpkgs";
    rev = "0f316e4d72daed659233817ffe52bf08e081b5de";
    patches = [ ./thunderbird-1.patch ./thunderbird-2.patch ];
  };

to apply a list of patches to a tree. These are applied lazily - the
patched tree is not materialized unless you do something that causes
the entire tree to be copied to the store (like 'src = fetchTree {
... }'). The equivalent of '-p1' is implied.

File additions/deletions/renames are not yet handled.

Issue #3920.
2022-03-29 11:01:14 +02:00
Eelco Dolstra
3ec83565b1 Checkpoint 2022-03-29 11:01:14 +02:00
Eelco Dolstra
86b05ccd54 Only provide builtin.{getFlake,fetchClosure} is the corresponding experimental feature is enabled
This allows writing fallback code like

  if builtins ? fetchClosure then
    builtins.fetchClose { ... }
  else
    builtins.storePath ...
2022-03-25 14:04:18 +01:00
Eelco Dolstra
f902f3c2cb Add experimental feature 'fetch-closure' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
e5f7029ba4 nix store make-content-addressed: Support --from / --to 2022-03-24 21:33:33 +01:00
Eelco Dolstra
98658ae9d2 Document fetchClosure 2022-03-24 21:33:33 +01:00
Eelco Dolstra
28186b7044 Add a test for fetchClosure and 'nix store make-content-addressed' 2022-03-24 21:33:33 +01:00
Eelco Dolstra
4120930ac1 fetchClosure: Only allow some "safe" store types 2022-03-24 21:33:33 +01:00
Eelco Dolstra
7ffda0af6e fetchClosure: Skip makeContentAddressed() if toPath is already valid 2022-03-24 21:33:33 +01:00
Eelco Dolstra
545c2d0d8c fetchClosure: Allow a path to be rewritten to CA on the fly
The advantage is that the resulting closure doesn't need to be signed,
so you don't need to configure any binary cache keys on the client.
2022-03-24 21:33:33 +01:00
Eelco Dolstra
7f6fe8ca1d Rename 2022-03-24 21:33:33 +01:00
Eelco Dolstra
41659418cf fetchClosure: Require a CA path in pure mode 2022-03-24 21:33:33 +01:00
Eelco Dolstra
f4bafc412f Add builtins.fetchClosure
This allows closures to be imported at evaluation time, without
requiring the user to configure substituters. E.g.

  builtins.fetchClosure {
    storePath = /nix/store/f89g6yi63m1ywfxj96whv5sxsm74w5ka-python3.9-sqlparse-0.4.2;
    from = "https://cache.ngi0.nixos.org";
  }
2022-03-24 21:33:33 +01:00
Eelco Dolstra
e4ff430866
Merge pull request #6237 from obsidiansystems/store-path-string-context
Decode string context straight to using StorePaths
2022-03-22 10:29:46 +01:00
John Ericson
4d6a3806d2 Decode string context straight to using StorePaths
I gather decoding happens on demand, so I hope don't think this should
have any perf implications one way or the other.
2022-03-18 15:36:11 +00:00
John Ericson
197feed51d Clean up DerivationOutput, and headers
1. `DerivationOutput` now as the `std::variant` as a base class. And the
   variants are given hierarchical names under `DerivationOutput`.

   In 8e0d0689be @matthewbauer and I
   didn't know a better idiom, and so we made it a field. But this sort
   of "newtype" is anoying for literals downstream.

   Since then we leaned the base class, inherit the constructors trick,
   e.g. used in `DerivedPath`. Switching to use that makes this more
   ergonomic, and consistent.

2. `store-api.hh` and `derivations.hh` are now independent.

   In bcde5456cc I swapped the dependency,
   but I now know it is better to just keep on using incomplete types as
   much as possible for faster compilation and good separation of
   concerns.
2022-03-17 22:35:53 +00:00
Ben Burdette
eaecaaa00b more debug_throw coverage of EvalErrors 2022-03-14 11:39:53 -06:00
John Ericson
91adfb8894 Create some type aliases for string Contexts 2022-03-11 22:30:10 +00:00
Eelco Dolstra
0cb5af5000
Merge pull request #6185 from hercules-ci/fetchTree-reuse-local-paths
fetchTree: Use isValidPath, add comment
2022-03-01 13:15:42 +01:00
Robert Hensing
b6deca7c0d fetchTree: Use isValidPath, add comment 2022-03-01 12:11:10 +01:00
Eelco Dolstra
d974d2ad59 fetch{url,Tarball}: Remove 'narHash' attribute
This was introduced in #6174. However fetch{url,Tarball} are legacy
and we shouldn't have an undocumented attribute that does the same
thing as one that already exists ('sha256').
2022-03-01 11:30:26 +01:00
Robert Hensing
ee019d0afc Add EvalState::allowAndSetStorePathString helper
This switches addPath from `printStorePath` to `toRealPath`.
2022-02-28 21:37:49 +01:00
Robert Hensing
158280d8e9 fetchTree: Do not re-fetch paths already present 2022-02-28 21:37:49 +01:00
Eelco Dolstra
df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Eelco Dolstra
8518cebfce libfetchers: Rename immutable -> locked
This is more consistent with flake terminology.
2022-02-24 18:09:00 +01:00