This patch adds an EvalProfiler and MultiEvalProfiler that can be used
to insert hooks into the evaluation for the purposes of function tracing
(what function-trace currently does) or for flamegraph/tracy profilers.
See the following commits for how this is supposed to be integrated into
the evaluator and performance considerations.
This trades off some executable size for measurable lexer performance
improvements.
Note on the explicitly enabling 8bit scanner.
This is needed due to the default behavior of flex (excerpt from the manual [1]):
> Flex’s default behavior is to generate an 8-bit scanner unless you
> use the ‘-Cf’ or ‘-CF’, in which case flex defaults to generating
> 7-bit scanners unless your site was always configured to generate 8-bit
> scanners.
Some quantifyable metrics:
Nixpkgs revision: a6e3f45acf4e817532a861ab0eda4ab5485fecc1
Parsing the largest file in nixpkgs: pkgs/development/haskell-modules/hackage-packages.nix.
(Before this patch)
```
$ nix build github:nixos/nix/9fe3077d4#nix-expr
$ du --apparent-size result/lib/libnixexpr.so
2518 result/lib/libnixexpr.so
$ nix build github:nixos/nix/9fe3077d4#nix-cli
$ taskset -c 2,3 hyperfine "GC_INITIAL_HEAP_SIZE=16g \
result/bin/nix-instantiate --parse \
../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix > /dev/null"
Time (mean ± σ): 375.5 ms ± 6.3 ms [User: 316.9 ms, System: 56.7 ms]
Range (min … max): 368.5 ms … 388.3 ms 10 runs
```
(After the patch)
```
$ nix build .#nix-expr
$ du --apparent-size result/lib/libnixexpr.so
2685 result/lib/libnixexpr.so
$ nix build .#nix-cli
$ taskset -c 2,3 hyperfine "GC_INITIAL_HEAP_SIZE=16g \
result/bin/nix-instantiate --parse \
../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix > /dev/null"
Time (mean ± σ): 326.8 ms ± 4.9 ms [User: 269.5 ms, System: 55.3 ms]
Range (min … max): 319.7 ms … 335.5 ms 10 runs
```
Overall, the change is roughly:
- 2518KiB -> 2685KiB ~ 150 KiB of machine code
- 375ms -> 325ms ~ 50ms
The perf uplift for eval-heavy test cases is obviously less noticeable,
but it doesn't make sense not to take this free perf win.
[1]: https://westes.github.io/flex/manual/Options-Affecting-Scanner-Behavior.html#Options-Affecting-Scanner-Behavior
Revert most of "Hack together a fix for the public headers"
- The `libmain` change is kept, and one more libmain change is made.
(Need to update Meson and Nix per the package alike).
- The S3 situation is fixed in a different way: the variable is public
now, used in the header, and fixed accordingly.
- Fix TODO for `HAVE_EMBEDDED_SANDBOX_SHELL`
This reverts commit 2b51250534.
For example, instead of doing
#include "nix/store-config.hh"
#include "nix/derived-path.hh"
Now do
#include "nix/store/config.hh"
#include "nix/store/derived-path.hh"
This was originally planned in the issue, and also recent requested by
Eelco.
Most of the change is purely mechanical. There is just one small
additional issue. See how, in the example above, we took this
opportunity to also turn `<comp>-config.hh` into `<comp>/config.hh`.
Well, there was already a `nix/util/config.{cc,hh}`. Even though there
is not a public configuration header for libutil (which also would be
called `nix/util/config.{cc,hh}`) that's still confusing, To avoid any
such confusion, we renamed that to `nix/util/configuration.{cc,hh}`.
Finally, note that the libflake headers already did this, so we didn't
need to do anything to them. We wouldn't want to mistakenly get
`nix/flake/flake/flake.hh`!
Progress on #7876
There are two big changes:
1. Public and private config is now separated. Configuration variables
that are only used internally do not go in a header which is
installed.
(Additionally, libutil has a unix-specific private config header,
which should only be used in unix-specific code. This keeps things a
bit more organized, in a purely private implementation-internal way.)
2. Secondly, there is no more `-include`. There are very few config
items that need to be publically exposed, so now it is feasible to
just make the headers that need them just including the (public)
configuration header.
And there are also a few more small cleanups on top of those:
- The configuration files have better names.
- The few CPP variables that remain exposed in the public headers are
now also renamed to always start with `NIX_`. This ensures they should
not conflict with variables defined elsewhere.
- We now always use `#if` and not `#ifdef`/`#ifndef` for our
configuration variables, which helps avoid bugs by requiring that
variables must be defined in all cases.
The short answer for why we need to do this is so we can consistently do
`#include "nix/..."`. Without this change, there are ways to still make
that work, but they are hacky, and they have downsides such as making it
harder to make sure headers from the wrong Nix library (e..g.
`libnixexpr` headers in `libnixutil`) aren't being used.
The C API alraedy used `nix_api_*`, so its headers are *not* put in
subdirectories accordingly.
Progress on #7876
We resisted doing this for a while because it would be annoying to not
have the header source file pairs close by / easy to change file
path/name from one to the other. But I am ameliorating that with
symlinks in the next commit.
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.
Fix a footgun. In my case, I had a couple of build ("output")
directories sitting around.
rm -rf build-*
Was confused for a bit why a meson.build file was missing.
Probably also helps with autocompletion.
I tried meson-build-support first, but I had to add something like
a nix- prefix, in order to make meson happy. They've reserved the
meson- prefix.
This reduces the amount of boilerplate. More importantly, it provides
a place to add compiler flags (such as -O3) without having to add it
to every subproject (and the risk of forgetting to include it).
