John Ericson
ff2a8ccfe1
Merge branch 'path-info' into ca-drv-exotic
2022-03-25 19:40:52 +00:00
John Ericson
0dc2974930
Merge remote-tracking branch 'upstream/master' into path-info
2022-03-25 19:25:08 +00:00
Eelco Dolstra
fc35b11a7c
Fix mismatched tag warning on clang
2022-03-25 15:22:22 +01:00
Eelco Dolstra
8c363eb3eb
Document getFlake
...
Fixes #5523 .
2022-03-25 14:19:55 +01:00
Eelco Dolstra
86b05ccd54
Only provide builtin.{getFlake,fetchClosure} is the corresponding experimental feature is enabled
...
This allows writing fallback code like
if builtins ? fetchClosure then
builtins.fetchClose { ... }
else
builtins.storePath ...
2022-03-25 14:04:18 +01:00
Eelco Dolstra
f902f3c2cb
Add experimental feature 'fetch-closure'
2022-03-24 21:33:33 +01:00
Eelco Dolstra
e5f7029ba4
nix store make-content-addressed: Support --from / --to
2022-03-24 21:33:33 +01:00
Eelco Dolstra
98658ae9d2
Document fetchClosure
2022-03-24 21:33:33 +01:00
Eelco Dolstra
28186b7044
Add a test for fetchClosure and 'nix store make-content-addressed'
2022-03-24 21:33:33 +01:00
Eelco Dolstra
4120930ac1
fetchClosure: Only allow some "safe" store types
2022-03-24 21:33:33 +01:00
Eelco Dolstra
7ffda0af6e
fetchClosure: Skip makeContentAddressed() if toPath is already valid
2022-03-24 21:33:33 +01:00
Eelco Dolstra
545c2d0d8c
fetchClosure: Allow a path to be rewritten to CA on the fly
...
The advantage is that the resulting closure doesn't need to be signed,
so you don't need to configure any binary cache keys on the client.
2022-03-24 21:33:33 +01:00
Eelco Dolstra
7f6fe8ca1d
Rename
2022-03-24 21:33:33 +01:00
Eelco Dolstra
41659418cf
fetchClosure: Require a CA path in pure mode
2022-03-24 21:33:33 +01:00
Eelco Dolstra
f4bafc412f
Add builtins.fetchClosure
...
This allows closures to be imported at evaluation time, without
requiring the user to configure substituters. E.g.
builtins.fetchClosure {
storePath = /nix/store/f89g6yi63m1ywfxj96whv5sxsm74w5ka-python3.9-sqlparse-0.4.2;
from = "https://cache.ngi0.nixos.org ";
}
2022-03-24 21:33:33 +01:00
Eelco Dolstra
d67fe90375
Merge pull request #6305 from flox/genericClosure_doc
...
docs: genericClosure
2022-03-24 14:02:58 +01:00
Tom Bereknyei
0736f3651d
docs: genericClosure
2022-03-24 08:03:59 -04:00
Sergei Trofimovich
9174d884d7
lexer: add error location to lexer errors
...
Before the change lexter errors did not report the location:
$ nix build -f. mc
error: path has a trailing slash
(use '--show-trace' to show detailed location information)
Note that it's not clear what file generates the error.
After the change location is reported:
$ src/nix/nix --extra-experimental-features nix-command build -f ~/nm mc
error: path has a trailing slash
at .../pkgs/development/libraries/glib/default.nix:54:18:
53| };
54| src = /tmp/foo/;
| ^
55|
(use '--show-trace' to show detailed location information)
Here we see both problematic file and the string itself.
2022-03-24 08:16:14 +00:00
Eelco Dolstra
a0259a21a4
Don't hide repeated values while generating manifest.nix
...
Fixes #6243 .
2022-03-22 13:18:56 +01:00
Eelco Dolstra
732296ddc0
printValue(): <REPEAT> -> «repeated»
...
This ensures that it doesn't get parsed as a valid Nix expression.
2022-03-22 13:00:27 +01:00
Eelco Dolstra
e4ff430866
Merge pull request #6237 from obsidiansystems/store-path-string-context
...
Decode string context straight to using StorePaths
2022-03-22 10:29:46 +01:00
John Ericson
4d6a3806d2
Decode string context straight to using StorePaths
...
I gather decoding happens on demand, so I hope don't think this should
have any perf implications one way or the other.
2022-03-18 15:36:11 +00:00
John Ericson
a544ed7684
Generalize DerivationType in preparation for impure derivations
2022-03-18 14:59:56 +00:00
John Ericson
049fae155a
Avoid some pointless copying of drvs
2022-03-18 14:59:56 +00:00
John Ericson
8496be7def
Use Deferred when building an input-addressed drv
...
Easier than using dummy path with input addressed.
2022-03-18 14:59:56 +00:00
Guillaume Maudoux
c2b620f3ad
Try to fix issues with macos clang, v3
2022-03-18 15:35:24 +01:00
Guillaume Maudoux
726f5836d8
Try to fix issues with macos clang, v2
2022-03-18 15:22:25 +01:00
Guillaume Maudoux
37e84316c2
Try to fix issues with macos clang
2022-03-18 14:48:49 +01:00
Guillaume Maudoux
9c42c00570
Fix some error kind mismatches
2022-03-18 10:22:47 +01:00
Guillaume Maudoux
ad3fadb95a
fixup! Merge remote-tracking branch 'origin/master' into coerce-string
2022-03-18 10:11:36 +01:00
Guillaume Maudoux
ca5c3e86ab
Merge remote-tracking branch 'origin/master' into coerce-string
2022-03-18 01:25:55 +01:00
Guillaume Maudoux
1942fed6d9
Revert extra colon at end os strings
2022-03-18 01:10:04 +01:00
Guillaume Maudoux
e6d07e0d89
Refactor to use more traces and less string manipulations
2022-03-18 00:58:09 +01:00
John Ericson
197feed51d
Clean up DerivationOutput, and headers
...
1. `DerivationOutput` now as the `std::variant` as a base class. And the
variants are given hierarchical names under `DerivationOutput`.
In 8e0d0689bebcde5456cc
2022-03-17 22:35:53 +00:00
Théophane Hufschmitt
a0b517de57
Merge pull request #6242 from ncfavier/print-output-names
...
nix-env: always print output names in JSON and XML
2022-03-17 10:55:22 +01:00
Naïm Favier
5736661922
nix-env: always print output names in JSON and XML
...
The current `--out-path` flag has two disadvantages when one is only
concerned with querying the names of outputs:
- it requires evaluating every output's `outPath`, which takes
significantly more resources and runs into more failures
- it destroys the information of the order of outputs so we can't tell
which one is the main output
This patch makes the output names always present (replacing paths with
`null` in JSON if `--out-path` isn't given), and adds an `outputName`
field.
2022-03-16 21:26:19 +01:00
Ben Burdette
88a54108eb
formatting
2022-03-16 12:09:47 -06:00
Ben Burdette
3dfab6e534
have only one debuggerHook declaration
2022-03-14 11:58:11 -06:00
Ben Burdette
eaecaaa00b
more debug_throw coverage of EvalErrors
2022-03-14 11:39:53 -06:00
John Ericson
91adfb8894
Create some type aliases for string Contexts
2022-03-11 22:30:10 +00:00
John Ericson
0948b8e94d
Reduce variants for derivation hash modulo
...
This changes was taken from dynamic derivation (#4628 ). It` somewhat
undoes the refactors I first did for floating CA derivations, as the
benefit of hindsight + requirements of dynamic derivations made me
reconsider some things.
They aren't to consequential, but I figured they might be good to land
first, before the more profound changes @thufschmitt has in the works.
2022-03-11 21:20:37 +00:00
Eelco Dolstra
aee56e0f89
Merge remote-tracking branch 'origin/eval-suggestions'
2022-03-11 12:02:26 +01:00
Eelco Dolstra
31a392dfe2
Merge pull request #5865 from pennae/memory-friendliness
...
be more memory friendly
2022-03-11 11:52:39 +01:00
John Ericson
938650700f
Merge branch 'path-info' into ca-drv-exotic
2022-03-10 16:20:01 +00:00
John Ericson
8ba089597f
Merge remote-tracking branch 'upstream/master' into path-info
2022-03-10 15:48:14 +00:00
pennae
4d629c4f7a
add HAVE_BOEHMGC guards to batched allocation functions
2022-03-09 00:18:50 +01:00
pennae
47baa9d43c
make Pos smaller
...
reduces peak hep memory use on eval of our test system from 264.4MB to 242.3MB,
possibly also a slight performance boost.
theoretically memory use could be cut down by another eight bytes per Pos on
average by turning it into a tuple containing an index into a global base
position table with row and column offsets, but that doesn't seem worth the
effort at this point.
2022-03-08 23:30:18 +01:00
pennae
c96460f352
force-inline a few much-used functions
...
these functions are called a whole lot, and they're all comparatively small.
always inlining them gives ~0.7% performance boost on eval.
before:
Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
Time (mean ± σ): 6.935 s ± 0.052 s [User: 5.852 s, System: 0.853 s]
Range (min … max): 6.808 s … 7.026 s 20 runs
Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 329.8 ms ± 2.7 ms [User: 299.0 ms, System: 30.8 ms]
Range (min … max): 326.6 ms … 336.5 ms 20 runs
Benchmark 3: nix flakes eval --raw --impure --file expr.nix
Time (mean ± σ): 2.655 s ± 0.038 s [User: 2.364 s, System: 0.220 s]
Range (min … max): 2.574 s … 2.737 s 20 runs
after:
Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
Time (mean ± σ): 6.912 s ± 0.036 s [User: 5.823 s, System: 0.856 s]
Range (min … max): 6.849 s … 6.980 s 20 runs
Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 325.1 ms ± 2.5 ms [User: 293.2 ms, System: 31.8 ms]
Range (min … max): 322.2 ms … 332.8 ms 20 runs
Benchmark 3: nix flakes eval --raw --impure --file expr.nix
Time (mean ± σ): 2.636 s ± 0.024 s [User: 2.352 s, System: 0.226 s]
Range (min … max): 2.574 s … 2.681 s 20 runs
2022-03-08 23:30:18 +01:00
pennae
60ed4e908a
cache singleton Envs just like Values
...
vast majority of envs is this size.
before:
Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
Time (mean ± σ): 6.946 s ± 0.041 s [User: 5.875 s, System: 0.835 s]
Range (min … max): 6.834 s … 7.005 s 20 runs
Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 330.3 ms ± 2.5 ms [User: 299.2 ms, System: 30.9 ms]
Range (min … max): 327.5 ms … 337.7 ms 20 runs
Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 2.671 s ± 0.035 s [User: 2.370 s, System: 0.232 s]
Range (min … max): 2.597 s … 2.749 s 20 runs
after:
Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
Time (mean ± σ): 6.935 s ± 0.052 s [User: 5.852 s, System: 0.853 s]
Range (min … max): 6.808 s … 7.026 s 20 runs
Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 329.8 ms ± 2.7 ms [User: 299.0 ms, System: 30.8 ms]
Range (min … max): 326.6 ms … 336.5 ms 20 runs
Benchmark 3: nix flakes eval --raw --impure --file expr.nix
Time (mean ± σ): 2.655 s ± 0.038 s [User: 2.364 s, System: 0.220 s]
Range (min … max): 2.574 s … 2.737 s 20 runs
2022-03-08 23:30:18 +01:00
pennae
4b2b0d3a55
remove GC_PTR_STORE_AND_DIRTY
...
turns out it's only necessary for MANUAL_VDB, which nix doesn't use. omitting
them gives a slight performance improvement on eval.
before:
Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
Time (mean ± σ): 6.988 s ± 0.061 s [User: 5.935 s, System: 0.845 s]
Range (min … max): 6.865 s … 7.075 s 20 runs
Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 332.6 ms ± 3.9 ms [User: 299.6 ms, System: 32.9 ms]
Range (min … max): 328.1 ms … 339.1 ms 20 runs
Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 2.681 s ± 0.049 s [User: 2.382 s, System: 0.228 s]
Range (min … max): 2.607 s … 2.776 s 20 runs
after:
Benchmark 1: nix flakes search --no-eval-cache --offline ../nixpkgs hello
Time (mean ± σ): 6.946 s ± 0.041 s [User: 5.875 s, System: 0.835 s]
Range (min … max): 6.834 s … 7.005 s 20 runs
Benchmark 2: nix flakes eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 330.3 ms ± 2.5 ms [User: 299.2 ms, System: 30.9 ms]
Range (min … max): 327.5 ms … 337.7 ms 20 runs
Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 2.671 s ± 0.035 s [User: 2.370 s, System: 0.232 s]
Range (min … max): 2.597 s … 2.749 s 20 runs
2022-03-08 23:30:18 +01:00
