wroclaw-git-backups/nix
1
0
Fork 0
mirror of https://github.com/NixOS/nix synced 2025-07-05 12:21:48 +02:00
Code Activity

Revert flake-schemas for now

Browse source
This commit is contained in:
Eelco Dolstra 2024-10-29 15:18:48 +01:00
parent 6ff6baaa35
commit ffcc42faf4
25 changed files with 827 additions and 730 deletions
Download patch file Download diff file Expand all files Collapse all files

1
Makefile.config.in
View file

@ -37,7 +37,6 @@ checkbindir = @checkbindir@
checklibdir = @checklibdir@ checklibdir = @checklibdir@
datadir = @datadir@ datadir = @datadir@
datarootdir = @datarootdir@ datarootdir = @datarootdir@
default_flake_schemas = @default_flake_schemas@
docdir = @docdir@ docdir = @docdir@
embedded_sandbox_shell = @embedded_sandbox_shell@ embedded_sandbox_shell = @embedded_sandbox_shell@
exec_prefix = @exec_prefix@ exec_prefix = @exec_prefix@

6
configure.ac
View file

@ -428,12 +428,6 @@ if test "$embedded_sandbox_shell" = yes; then
AC_DEFINE(HAVE_EMBEDDED_SANDBOX_SHELL, 1, [Include the sandbox shell in the Nix binary.]) AC_DEFINE(HAVE_EMBEDDED_SANDBOX_SHELL, 1, [Include the sandbox shell in the Nix binary.])
fi fi
AC_ARG_WITH(default-flake-schemas, AS_HELP_STRING([--with-default-flake-schemas=PATH],[path of the default flake schemas flake]),
default_flake_schemas=$withval,
[AC_MSG_FAILURE([--with-default-flake-schemas is missing])])
AC_SUBST(default_flake_schemas)
]) ])

1
doc/manual/src/SUMMARY.md.in
View file

@ -114,7 +114,6 @@
- [Store Path Specification](protocols/store-path.md) - [Store Path Specification](protocols/store-path.md)
- [Nix Archive (NAR) Format](protocols/nix-archive.md) - [Nix Archive (NAR) Format](protocols/nix-archive.md)
- [Derivation "ATerm" file format](protocols/derivation-aterm.md) - [Derivation "ATerm" file format](protocols/derivation-aterm.md)
- [Flake Schemas](protocols/flake-schemas.md)
- [C API](c-api.md) - [C API](c-api.md)
- [Glossary](glossary.md) - [Glossary](glossary.md)
- [Development](development/index.md) - [Development](development/index.md)

64
doc/manual/src/protocols/flake-schemas.md
View file

@ -1,64 +0,0 @@
# Flake Schemas
Flake schemas are a mechanism to allow tools like `nix flake show` and `nix flake check` to enumerate and check the contents of a flake
in a generic way, without requiring built-in knowledge of specific flake output types like `packages` or `nixosConfigurations`.
A flake can define schemas for its outputs by defining a `schemas` output. `schemas` should be an attribute set with an attribute for
every output type that you want to be supported. If a flake does not have a `schemas` attribute, Nix uses a built-in set of schemas (namely https://github.com/DeterminateSystems/flake-schemas).
A schema is an attribute set with the following attributes:
| Attribute | Description | Default |
| :---------- | :---------------------------------------------------------------------------------------------- | :------ |
| `version` | Should be set to 1 | |
| `doc` | A string containing documentation about the flake output type in Markdown format. | |
| `allowIFD` | Whether the evaluation of the output attributes of this flake can read from derivation outputs. | `true` |
| `inventory` | A function that returns the contents of the flake output (described [below](#inventory)). | |
# Inventory
The `inventory` function returns a _node_ describing the contents of the flake output. A node is either a _leaf node_ or a _non-leaf node_. This allows nested flake output attributes to be described (e.g. `x86_64-linux.hello` inside a `packages` output).
Non-leaf nodes must have the following attribute:
| Attribute | Description |
| :--------- | :------------------------------------------------------------------------------------- |
| `children` | An attribute set of nodes. If this attribute is missing, the attribute is a leaf node. |
Leaf nodes can have the following attributes:
| Attribute | Description |
| :----------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `derivation` | The main derivation of this node, if any. It must evaluate for `nix flake check` and `nix flake show` to succeed. |
| `evalChecks` | An attribute set of Boolean values, used by `nix flake check`. Each attribute must evaluate to `true`. |
| `isFlakeCheck` | Whether `nix flake check` should build the `derivation` attribute of this node. |
| `shortDescription` | A one-sentence description of the node (such as the `meta.description` attribute in Nixpkgs). |
| `what` | A brief human-readable string describing the type of the node, e.g. `"package"` or `"development environment"`. This is used by tools like `nix flake show` to describe the contents of a flake. |
Both leaf and non-leaf nodes can have the following attributes:
| Attribute | Description |
| :----------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `forSystems` | A list of Nix system types (e.g. `["x86_64-linux"]`) supported by this node. This is used by tools to skip nodes that cannot be built on the user's system. Setting this on a non-leaf node allows all the children to be skipped, regardless of the `forSystems` attributes of the children. If this attribute is not set, the node is never skipped. |
# Example
Here is a schema that checks that every element of the `nixosConfigurations` flake output evaluates and builds correctly (meaning that it has a `config.system.build.toplevel` attribute that yields a buildable derivation).
```nix
outputs = {
schemas.nixosConfigurations = {
version = 1;
doc = ''
The `nixosConfigurations` flake output defines NixOS system configurations.
'';
inventory = output: {
children = builtins.mapAttrs (configName: machine:
{
what = "NixOS configuration";
derivation = machine.config.system.build.toplevel;
}) output;
};
};
};
```

29
flake.lock generated
View file

@ -36,21 +36,6 @@
"type": "github" "type": "github"
} }
}, },
"flake-schemas": {
"locked": {
"lastModified": 1719857163,
"narHash": "sha256-wM+8JtoKBkahHiKn+EM1ikurMnitwRQrZ91hipJIJK8=",
"owner": "DeterminateSystems",
"repo": "flake-schemas",
"rev": "61a02d7183d4241962025e6c6307a22a0bb72a21",
"type": "github"
},
"original": {
"owner": "DeterminateSystems",
"repo": "flake-schemas",
"type": "github"
}
},
"git-hooks-nix": { "git-hooks-nix": {
"inputs": { "inputs": {
"flake-compat": [], "flake-compat": [],
@ -63,11 +48,11 @@
] ]
}, },
"locked": { "locked": {
"lastModified": 1721042469, "lastModified": 1729104314,
"narHash": "sha256-6FPUl7HVtvRHCCBQne7Ylp4p+dpP3P/OYuzjztZ4s70=", "narHash": "sha256-pZRZsq5oCdJt3upZIU4aslS9XwFJ+/nVtALHIciX/BI=",
"owner": "cachix", "owner": "cachix",
"repo": "git-hooks.nix", "repo": "git-hooks.nix",
"rev": "f451c19376071a90d8c58ab1a953c6e9840527fd", "rev": "3c3e88f0f544d6bb54329832616af7eb971b6be6",
"type": "github" "type": "github"
}, },
"original": { "original": {
@ -79,16 +64,15 @@
"libgit2": { "libgit2": {
"flake": false, "flake": false,
"locked": { "locked": {
"lastModified": 1715853528, "lastModified": 1730025633,
"narHash": "sha256-J2rCxTecyLbbDdsyBWn9w7r3pbKRMkI9E7RvRgAqBdY=", "narHash": "sha256-HcL9fW5crHeLpP7C7vShO+j5fwY8z95Plr1c+hIwFRQ=",
"owner": "libgit2", "owner": "libgit2",
"repo": "libgit2", "repo": "libgit2",
"rev": "36f7e21ad757a3dacc58cf7944329da6bc1d6e96", "rev": "b363ea4b9e761fed7942eef4bbc735ccf16f9fed",
"type": "github" "type": "github"
}, },
"original": { "original": {
"owner": "libgit2", "owner": "libgit2",
"ref": "v1.8.1",
"repo": "libgit2", "repo": "libgit2",
"type": "github" "type": "github"
} }
@ -145,7 +129,6 @@
"inputs": { "inputs": {
"flake-compat": "flake-compat", "flake-compat": "flake-compat",
"flake-parts": "flake-parts", "flake-parts": "flake-parts",
"flake-schemas": "flake-schemas",
"git-hooks-nix": "git-hooks-nix", "git-hooks-nix": "git-hooks-nix",
"libgit2": "libgit2", "libgit2": "libgit2",
"nixpkgs": "nixpkgs", "nixpkgs": "nixpkgs",

8
flake.nix
View file

@ -5,8 +5,7 @@
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2"; inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446"; inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; }; inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
inputs.libgit2 = { url = "github:libgit2/libgit2/v1.8.1"; flake = false; }; inputs.libgit2 = { url = "github:libgit2/libgit2"; flake = false; };
inputs.flake-schemas.url = "github:DeterminateSystems/flake-schemas";
# dev tooling # dev tooling
inputs.flake-parts.url = "github:hercules-ci/flake-parts"; inputs.flake-parts.url = "github:hercules-ci/flake-parts";
@ -19,7 +18,8 @@
inputs.git-hooks-nix.inputs.flake-compat.follows = ""; inputs.git-hooks-nix.inputs.flake-compat.follows = "";
inputs.git-hooks-nix.inputs.gitignore.follows = ""; inputs.git-hooks-nix.inputs.gitignore.follows = "";
outputs = inputs@{ self, nixpkgs, nixpkgs-regression, libgit2, flake-schemas, ... }: outputs = inputs@{ self, nixpkgs, nixpkgs-regression, libgit2, ... }:
let let
inherit (nixpkgs) lib; inherit (nixpkgs) lib;
@ -156,8 +156,6 @@
}; };
in { in {
schemas = flake-schemas.schemas;
# A Nixpkgs overlay that overrides the 'nix' and # A Nixpkgs overlay that overrides the 'nix' and
# 'nix-perl-bindings' packages. # 'nix-perl-bindings' packages.
overlays.default = overlayFor (p: p.stdenv); overlays.default = overlayFor (p: p.stdenv);

3
package.nix
View file

@ -38,8 +38,6 @@
, busybox-sandbox-shell ? null , busybox-sandbox-shell ? null
, flake-schemas
# Configuration Options # Configuration Options
#: #:
# This probably seems like too many degrees of freedom, but it # This probably seems like too many degrees of freedom, but it
@ -261,7 +259,6 @@ in {
(lib.enableFeature enableMarkdown "markdown") (lib.enableFeature enableMarkdown "markdown")
(lib.enableFeature installUnitTests "install-unit-tests") (lib.enableFeature installUnitTests "install-unit-tests")
(lib.withFeatureAs true "readline-flavor" readlineFlavor) (lib.withFeatureAs true "readline-flavor" readlineFlavor)
"--with-default-flake-schemas=${flake-schemas}"
] ++ lib.optionals (!forDevShell) [ ] ++ lib.optionals (!forDevShell) [
"--sysconfdir=/etc" "--sysconfdir=/etc"
] ++ lib.optionals installUnitTests [ ] ++ lib.optionals installUnitTests [

13
packaging/dependencies.nix
View file

@ -152,16 +152,5 @@ scope: {
inherit resolvePath filesetToSource; inherit resolvePath filesetToSource;
mkMesonDerivation = f: let mkMesonDerivation = f: stdenv.mkDerivation (lib.extends localSourceLayer f);
exts = [
miscGoodPractice
bsdNoLinkAsNeeded
localSourceLayer
];
in stdenv.mkDerivation
(lib.extends
(lib.foldr lib.composeExtensions (_: _: {}) exts)
f);
inherit (inputs) flake-schemas;
} }

2
packaging/hydra.nix
View file

@ -28,8 +28,6 @@ let
test-daemon = daemon; test-daemon = daemon;
doBuild = false; doBuild = false;
inherit (inputs) flake-schemas;
}; };
# Technically we could just return `pkgs.nixComponents`, but for Hydra it's # Technically we could just return `pkgs.nixComponents`, but for Hydra it's

14
src/libcmd/installable-flake.cc
View file

@ -43,6 +43,20 @@ std::vector<std::string> InstallableFlake::getActualAttrPaths()
return res; return res;
} }
Value * InstallableFlake::getFlakeOutputs(EvalState & state, const flake::LockedFlake & lockedFlake)
{
auto vFlake = state.allocValue();
callFlake(state, lockedFlake, *vFlake);
auto aOutputs = vFlake->attrs()->get(state.symbols.create("outputs"));
assert(aOutputs);
state.forceValue(*aOutputs->value, aOutputs->value->determinePos(noPos));
return aOutputs->value;
}
static std::string showAttrPaths(const std::vector<std::string> & paths) static std::string showAttrPaths(const std::vector<std::string> & paths)
{ {
std::string s; std::string s;

2
src/libcmd/installable-flake.hh
View file

@ -53,6 +53,8 @@ struct InstallableFlake : InstallableValue
std::vector<std::string> getActualAttrPaths(); std::vector<std::string> getActualAttrPaths();
Value * getFlakeOutputs(EvalState & state, const flake::LockedFlake & lockedFlake);
DerivedPathsWithInfo toDerivedPaths() override; DerivedPathsWithInfo toDerivedPaths() override;
std::pair<Value *, PosIdx> toValue(EvalState & state) override; std::pair<Value *, PosIdx> toValue(EvalState & state) override;

5
src/libcmd/installables.cc
View file

@ -449,6 +449,11 @@ ref<eval_cache::EvalCache> openEvalCache(
: std::nullopt; : std::nullopt;
auto rootLoader = [&state, lockedFlake]() auto rootLoader = [&state, lockedFlake]()
{ {
/* For testing whether the evaluation cache is
complete. */
if (getEnv("NIX_ALLOW_EVAL").value_or("1") == "0")
throw Error("not everything is cached, but evaluation is not allowed");
auto vFlake = state.allocValue(); auto vFlake = state.allocValue();
flake::callFlake(state, *lockedFlake, *vFlake); flake::callFlake(state, *lockedFlake, *vFlake);

6
src/libexpr/eval-cache.cc
View file

@ -368,12 +368,6 @@ Value * EvalCache::getRootValue()
{ {
if (!value) { if (!value) {
debug("getting root value"); debug("getting root value");
/* For testing whether the evaluation cache is
complete. */
if (getEnv("NIX_ALLOW_EVAL").value_or("1") == "0")
throw Error("not everything is cached, but evaluation is not allowed");
value = allocRootValue(rootLoader()); value = allocRootValue(rootLoader());
} }
return *value; return *value;

7
src/libexpr/eval-cache.hh
View file

@ -34,11 +34,7 @@ class EvalCache : public std::enable_shared_from_this<EvalCache>
friend struct CachedEvalError; friend struct CachedEvalError;
std::shared_ptr<AttrDb> db; std::shared_ptr<AttrDb> db;
public:
EvalState & state; EvalState & state;
private:
typedef std::function<Value *()> RootLoader; typedef std::function<Value *()> RootLoader;
RootLoader rootLoader; RootLoader rootLoader;
RootValue value; RootValue value;
@ -93,10 +89,7 @@ class AttrCursor : public std::enable_shared_from_this<AttrCursor>
friend class EvalCache; friend class EvalCache;
friend struct CachedEvalError; friend struct CachedEvalError;
public:
ref<EvalCache> root; ref<EvalCache> root;
private:
typedef std::optional<std::pair<std::shared_ptr<AttrCursor>, Symbol>> Parent; typedef std::optional<std::pair<std::shared_ptr<AttrCursor>, Symbol>> Parent;
Parent parent; Parent parent;
RootValue _value; RootValue _value;

36
src/libflake/flake/flake.cc
View file

@ -204,7 +204,7 @@ static std::map<FlakeId, FlakeInput> parseFlakeInputs(
return inputs; return inputs;
} }
Flake readFlake( static Flake readFlake(
EvalState & state, EvalState & state,
const FlakeRef & originalRef, const FlakeRef & originalRef,
const FlakeRef & resolvedRef, const FlakeRef & resolvedRef,
@ -338,16 +338,20 @@ static LockFile readLockFile(
: LockFile(); : LockFile();
} }
/* Compute an in-memory lock file for the specified top-level flake,
and optionally write it to file, if the flake is writable. */
LockedFlake lockFlake( LockedFlake lockFlake(
const Settings & settings, const Settings & settings,
EvalState & state, EvalState & state,
const FlakeRef & topRef, const FlakeRef & topRef,
const LockFlags & lockFlags, const LockFlags & lockFlags)
Flake flake,
FlakeCache & flakeCache)
{ {
FlakeCache flakeCache;
auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries); auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries);
auto flake = getFlake(state, topRef, useRegistries, flakeCache);
if (lockFlags.applyNixConfig) { if (lockFlags.applyNixConfig) {
flake.config.apply(settings); flake.config.apply(settings);
state.store->setOptions(); state.store->setOptions();
@ -738,30 +742,6 @@ LockedFlake lockFlake(
} }
} }
LockedFlake lockFlake(
const Settings & settings,
EvalState & state,
const FlakeRef & topRef,
const LockFlags & lockFlags)
{
FlakeCache flakeCache;
auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries);
return lockFlake(settings, state, topRef, lockFlags, getFlake(state, topRef, useRegistries, flakeCache), flakeCache);
}
LockedFlake lockFlake(
const Settings & settings,
EvalState & state,
const FlakeRef & topRef,
const LockFlags & lockFlags,
Flake flake)
{
FlakeCache flakeCache;
return lockFlake(settings, state, topRef, lockFlags, std::move(flake), flakeCache);
}
void callFlake(EvalState & state, void callFlake(EvalState & state,
const LockedFlake & lockedFlake, const LockedFlake & lockedFlake,
Value & vRes) Value & vRes)

19
src/libflake/flake/flake.hh
View file

@ -203,31 +203,12 @@ struct LockFlags
std::set<InputPath> inputUpdates; std::set<InputPath> inputUpdates;
}; };
Flake readFlake(
EvalState & state,
const FlakeRef & originalRef,
const FlakeRef & resolvedRef,
const FlakeRef & lockedRef,
const SourcePath & rootDir,
const InputPath & lockRootPath);
/**
* Compute an in-memory lock file for the specified top-level flake,
* and optionally write it to file, if the flake is writable.
*/
LockedFlake lockFlake( LockedFlake lockFlake(
const Settings & settings, const Settings & settings,
EvalState & state, EvalState & state,
const FlakeRef & flakeRef, const FlakeRef & flakeRef,
const LockFlags & lockFlags); const LockFlags & lockFlags);
LockedFlake lockFlake(
const Settings & settings,
EvalState & state,
const FlakeRef & topRef,
const LockFlags & lockFlags,
Flake flake);
void callFlake( void callFlake(
EvalState & state, EvalState & state,
const LockedFlake & lockedFlake, const LockedFlake & lockedFlake,

43
src/nix/call-flake-schemas.nix
View file

@ -1,43 +0,0 @@
/* The flake providing default schemas. */
defaultSchemasFlake:
/* The flake whose contents we want to extract. */
flake:
let
# Helper functions.
mapAttrsToList = f: attrs: map (name: f name attrs.${name}) (builtins.attrNames attrs);
in
rec {
outputNames = builtins.attrNames flake.outputs;
allSchemas = (flake.outputs.schemas or defaultSchemasFlake.schemas) // schemaOverrides;
schemaOverrides = {}; # FIXME
schemas =
builtins.listToAttrs (builtins.concatLists (mapAttrsToList
(outputName: output:
if allSchemas ? ${outputName} then
[{ name = outputName; value = allSchemas.${outputName}; }]
else
[ ])
flake.outputs));
inventory =
builtins.mapAttrs
(outputName: output:
if schemas ? ${outputName} && schemas.${outputName}.version == 1
then
{ output = schemas.${outputName}.inventory output;
inherit (schemas.${outputName}) doc;
}
else
{ unknown = true; }
)
flake.outputs;
}

58
src/nix/flake-check.md
View file

@ -18,20 +18,56 @@ R""(
# Description # Description
This command verifies that the flake specified by flake reference This command verifies that the flake specified by flake reference
*flake-url* can be evaluated and built successfully according to its *flake-url* can be evaluated successfully (as detailed below), and
`schemas` flake output. For every flake output that has a schema that the derivations specified by the flake's `checks` output can be
definition, `nix flake check` uses the schema to extract the contents built successfully.
of the output. Then, for every item in the contents:
* It evaluates the elements of the `evalChecks` attribute set returned
by the schema for that item, printing an error or warning for every
check that fails to evaluate or that evaluates to `false`.
* It builds `derivation` attribute returned by the schema for that
item, if the item has the `isFlakeCheck` attribute.
If the `keep-going` option is set to `true`, Nix will keep evaluating as much If the `keep-going` option is set to `true`, Nix will keep evaluating as much
as it can and report the errors as it encounters them. Otherwise it will stop as it can and report the errors as it encounters them. Otherwise it will stop
at the first error. at the first error.
# Evaluation checks
The following flake output attributes must be derivations:
* `checks.`*system*`.`*name*
* `defaultPackage.`*system*
* `devShell.`*system*
* `devShells.`*system*`.`*name*
* `nixosConfigurations.`*name*`.config.system.build.toplevel`
* `packages.`*system*`.`*name*
The following flake output attributes must be [app
definitions](./nix3-run.md):
* `apps.`*system*`.`*name*
* `defaultApp.`*system*
The following flake output attributes must be [template
definitions](./nix3-flake-init.md):
* `defaultTemplate`
* `templates.`*name*
The following flake output attributes must be *Nixpkgs overlays*:
* `overlay`
* `overlays.`*name*
The following flake output attributes must be *NixOS modules*:
* `nixosModule`
* `nixosModules.`*name*
The following flake output attributes must be
[bundlers](./nix3-bundle.md):
* `bundlers.`*name*
* `defaultBundler`
In addition, the `hydraJobs` output is evaluated in the same way as
Hydra's `hydra-eval-jobs` (i.e. as a arbitrarily deeply nested
attribute set of derivations). Similarly, the
`legacyPackages`.*system* output is evaluated like `nix-env --query --available `.
)"" )""

224
src/nix/flake-schemas.cc
View file

@ -1,224 +0,0 @@
#include "flake-schemas.hh"
#include "eval-settings.hh"
#include "fetch-to-store.hh"
#include "memory-source-accessor.hh"
#include "strings-inline.hh"
namespace nix::flake_schemas {
using namespace eval_cache;
using namespace flake;
static LockedFlake getBuiltinDefaultSchemasFlake(EvalState & state)
{
auto accessor = make_ref<MemorySourceAccessor>();
accessor->setPathDisplay("«builtin-flake-schemas»");
accessor->addFile(
CanonPath("flake.nix"),
#include "builtin-flake-schemas.nix.gen.hh"
);
// FIXME: remove this when we have lazy trees.
auto storePath = fetchToStore(*state.store, {accessor}, FetchMode::Copy);
state.allowPath(storePath);
// Construct a dummy flakeref.
auto flakeRef = parseFlakeRef(
fetchSettings,
fmt("tarball+https://builtin-flake-schemas?narHash=%s",
state.store->queryPathInfo(storePath)->narHash.to_string(HashFormat::SRI, true)));
auto flake = readFlake(state, flakeRef, flakeRef, flakeRef, state.rootPath(state.store->toRealPath(storePath)), {});
return lockFlake(flakeSettings, state, flakeRef, {}, flake);
}
std::tuple<ref<EvalCache>, ref<eval_cache::AttrCursor>>
call(EvalState & state, std::shared_ptr<flake::LockedFlake> lockedFlake, std::optional<FlakeRef> defaultSchemasFlake)
{
auto fingerprint = lockedFlake->getFingerprint(state.store);
std::string callFlakeSchemasNix =
#include "call-flake-schemas.nix.gen.hh"
;
auto lockedDefaultSchemasFlake = defaultSchemasFlake
? flake::lockFlake(flakeSettings, state, *defaultSchemasFlake, {})
: getBuiltinDefaultSchemasFlake(state);
auto lockedDefaultSchemasFlakeFingerprint = lockedDefaultSchemasFlake.getFingerprint(state.store);
std::optional<Fingerprint> fingerprint2;
if (fingerprint && lockedDefaultSchemasFlakeFingerprint)
fingerprint2 = hashString(
HashAlgorithm::SHA256,
fmt("app:%s:%s:%s",
hashString(HashAlgorithm::SHA256, callFlakeSchemasNix).to_string(HashFormat::Base16, false),
fingerprint->to_string(HashFormat::Base16, false),
lockedDefaultSchemasFlakeFingerprint->to_string(HashFormat::Base16, false)));
// FIXME: merge with openEvalCache().
auto cache = make_ref<EvalCache>(
evalSettings.useEvalCache && evalSettings.pureEval ? fingerprint2 : std::nullopt,
state,
[&state, lockedFlake, callFlakeSchemasNix, lockedDefaultSchemasFlake]() {
auto vCallFlakeSchemas = state.allocValue();
state.eval(
state.parseExprFromString(callFlakeSchemasNix, state.rootPath(CanonPath::root)), *vCallFlakeSchemas);
auto vFlake = state.allocValue();
flake::callFlake(state, *lockedFlake, *vFlake);
auto vDefaultSchemasFlake = state.allocValue();
if (vFlake->type() == nAttrs && vFlake->attrs()->get(state.symbols.create("schemas")))
vDefaultSchemasFlake->mkNull();
else
flake::callFlake(state, lockedDefaultSchemasFlake, *vDefaultSchemasFlake);
auto vRes = state.allocValue();
Value * args[] = {vDefaultSchemasFlake, vFlake};
state.callFunction(*vCallFlakeSchemas, 2, args, *vRes, noPos);
return vRes;
});
return {cache, cache->getRoot()->getAttr("inventory")};
}
/* Derive the flake output attribute path from the cursor used to
traverse the inventory. We do this so we don't have to maintain a
separate attrpath for that. */
std::vector<Symbol> toAttrPath(ref<AttrCursor> cursor)
{
auto attrPath = cursor->getAttrPath();
std::vector<Symbol> res;
auto i = attrPath.begin();
assert(i != attrPath.end());
++i; // skip "inventory"
assert(i != attrPath.end());
res.push_back(*i++); // copy output name
if (i != attrPath.end())
++i; // skip "outputs"
while (i != attrPath.end()) {
++i; // skip "children"
if (i != attrPath.end())
res.push_back(*i++);
}
return res;
}
std::string toAttrPathStr(ref<AttrCursor> cursor)
{
return concatStringsSep(".", cursor->root->state.symbols.resolve(toAttrPath(cursor)));
}
void forEachOutput(
ref<AttrCursor> inventory,
std::function<void(Symbol outputName, std::shared_ptr<AttrCursor> output, const std::string & doc, bool isLast)> f)
{
// FIXME: handle non-IFD outputs first.
// evalSettings.enableImportFromDerivation.setDefault(false);
auto outputNames = inventory->getAttrs();
for (const auto & [i, outputName] : enumerate(outputNames)) {
auto output = inventory->getAttr(outputName);
try {
auto isUnknown = (bool) output->maybeGetAttr("unknown");
Activity act(*logger, lvlInfo, actUnknown, fmt("evaluating '%s'", toAttrPathStr(output)));
f(outputName,
isUnknown ? std::shared_ptr<AttrCursor>() : output->getAttr("output"),
isUnknown ? "" : output->getAttr("doc")->getString(),
i + 1 == outputNames.size());
} catch (Error & e) {
e.addTrace(nullptr, "while evaluating the flake output '%s':", toAttrPathStr(output));
throw;
}
}
}
void visit(
std::optional<std::string> system,
ref<AttrCursor> node,
std::function<void(ref<AttrCursor> leaf)> visitLeaf,
std::function<void(std::function<void(ForEachChild)>)> visitNonLeaf,
std::function<void(ref<AttrCursor> node, const std::vector<std::string> & systems)> visitFiltered)
{
Activity act(*logger, lvlInfo, actUnknown, fmt("evaluating '%s'", toAttrPathStr(node)));
/* Apply the system type filter. */
if (system) {
if (auto forSystems = node->maybeGetAttr("forSystems")) {
auto systems = forSystems->getListOfStrings();
if (std::find(systems.begin(), systems.end(), system) == systems.end()) {
visitFiltered(node, systems);
return;
}
}
}
if (auto children = node->maybeGetAttr("children")) {
visitNonLeaf([&](ForEachChild f) {
auto attrNames = children->getAttrs();
for (const auto & [i, attrName] : enumerate(attrNames)) {
try {
f(attrName, children->getAttr(attrName), i + 1 == attrNames.size());
} catch (Error & e) {
// FIXME: make it a flake schema attribute whether to ignore evaluation errors.
if (node->root->state.symbols[toAttrPath(node)[0]] != "legacyPackages") {
e.addTrace(nullptr, "while evaluating the flake output attribute '%s':", toAttrPathStr(node));
throw;
}
}
}
});
}
else
visitLeaf(ref(node));
}
std::optional<std::string> what(ref<AttrCursor> leaf)
{
if (auto what = leaf->maybeGetAttr("what"))
return what->getString();
else
return std::nullopt;
}
std::optional<std::string> shortDescription(ref<AttrCursor> leaf)
{
if (auto what = leaf->maybeGetAttr("shortDescription")) {
auto s = trim(what->getString());
if (s != "")
return s;
}
return std::nullopt;
}
std::shared_ptr<AttrCursor> derivation(ref<AttrCursor> leaf)
{
return leaf->maybeGetAttr("derivation");
}
MixFlakeSchemas::MixFlakeSchemas()
{
addFlag(
{.longName = "default-flake-schemas",
.description = "The URL of the flake providing default flake schema definitions.",
.labels = {"flake-ref"},
.handler = {&defaultFlakeSchemas},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, getStore(), prefix);
}}});
}
std::optional<FlakeRef> MixFlakeSchemas::getDefaultFlakeSchemas()
{
if (!defaultFlakeSchemas)
return std::nullopt;
else
return parseFlakeRef(fetchSettings, *defaultFlakeSchemas, absPath("."));
}
}

45
src/nix/flake-schemas.hh
View file

@ -1,45 +0,0 @@
#include "eval-cache.hh"
#include "flake/flake.hh"
#include "command.hh"
namespace nix::flake_schemas {
using namespace eval_cache;
std::tuple<ref<eval_cache::EvalCache>, ref<AttrCursor>>
call(EvalState & state, std::shared_ptr<flake::LockedFlake> lockedFlake, std::optional<FlakeRef> defaultSchemasFlake);
std::vector<Symbol> toAttrPath(ref<AttrCursor> cursor);
std::string toAttrPathStr(ref<AttrCursor> cursor);
void forEachOutput(
ref<AttrCursor> inventory,
std::function<void(Symbol outputName, std::shared_ptr<AttrCursor> output, const std::string & doc, bool isLast)> f);
typedef std::function<void(Symbol attrName, ref<AttrCursor> attr, bool isLast)> ForEachChild;
void visit(
std::optional<std::string> system,
ref<AttrCursor> node,
std::function<void(ref<AttrCursor> leaf)> visitLeaf,
std::function<void(std::function<void(ForEachChild)>)> visitNonLeaf,
std::function<void(ref<AttrCursor> node, const std::vector<std::string> & systems)> visitFiltered);
std::optional<std::string> what(ref<AttrCursor> leaf);
std::optional<std::string> shortDescription(ref<AttrCursor> leaf);
std::shared_ptr<AttrCursor> derivation(ref<AttrCursor> leaf);
/* Some helper functions for processing flake schema output. */
struct MixFlakeSchemas : virtual Args, virtual StoreCommand
{
std::optional<std::string> defaultFlakeSchemas;
MixFlakeSchemas();
std::optional<FlakeRef> getDefaultFlakeSchemas();
};
}

907
src/nix/flake.cc
View file

@ -17,7 +17,6 @@
#include "eval-cache.hh" #include "eval-cache.hh"
#include "markdown.hh" #include "markdown.hh"
#include "users.hh" #include "users.hh"
#include "flake-schemas.hh"
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
#include <iomanip> #include <iomanip>
@ -166,6 +165,31 @@ struct CmdFlakeLock : FlakeCommand
} }
}; };
static void enumerateOutputs(EvalState & state, Value & vFlake,
std::function<void(std::string_view name, Value & vProvide, const PosIdx pos)> callback)
{
auto pos = vFlake.determinePos(noPos);
state.forceAttrs(vFlake, pos, "while evaluating a flake to get its outputs");
auto aOutputs = vFlake.attrs()->get(state.symbols.create("outputs"));
assert(aOutputs);
state.forceAttrs(*aOutputs->value, pos, "while evaluating the outputs of a flake");
auto sHydraJobs = state.symbols.create("hydraJobs");
/* Hack: ensure that hydraJobs is evaluated before anything
else. This way we can disable IFD for hydraJobs and then enable
it for other outputs. */
if (auto attr = aOutputs->value->attrs()->get(sHydraJobs))
callback(state.symbols[attr->name], *attr->value, attr->pos);
for (auto & attr : *aOutputs->value->attrs()) {
if (attr.name != sHydraJobs)
callback(state.symbols[attr.name], *attr.value, attr.pos);
}
}
struct CmdFlakeMetadata : FlakeCommand, MixJSON struct CmdFlakeMetadata : FlakeCommand, MixJSON
{ {
std::string description() override std::string description() override
@ -296,7 +320,7 @@ struct CmdFlakeInfo : CmdFlakeMetadata
} }
}; };
struct CmdFlakeCheck : FlakeCommand, flake_schemas::MixFlakeSchemas struct CmdFlakeCheck : FlakeCommand
{ {
bool build = true; bool build = true;
bool checkAllSystems = false; bool checkAllSystems = false;
@ -337,26 +361,16 @@ struct CmdFlakeCheck : FlakeCommand, flake_schemas::MixFlakeSchemas
auto state = getEvalState(); auto state = getEvalState();
lockFlags.applyNixConfig = true; lockFlags.applyNixConfig = true;
auto flake = std::make_shared<LockedFlake>(lockFlake()); auto flake = lockFlake();
auto localSystem = std::string(settings.thisSystem.get()); auto localSystem = std::string(settings.thisSystem.get());
auto [cache, inventory] = flake_schemas::call(*state, flake, getDefaultFlakeSchemas());
std::vector<DerivedPath> drvPaths;
std::set<std::string> uncheckedOutputs;
std::set<std::string> omittedSystems;
std::function<void(ref<eval_cache::AttrCursor> node)> visit;
bool hasErrors = false; bool hasErrors = false;
auto reportError = [&](const Error & e) { auto reportError = [&](const Error & e) {
try { try {
throw e; throw e;
} catch (Error & e) { } catch (Error & e) {
if (settings.keepGoing) { if (settings.keepGoing) {
logError({.msg = e.info().msg}); ignoreException();
hasErrors = true; hasErrors = true;
} }
else else
@ -364,70 +378,428 @@ struct CmdFlakeCheck : FlakeCommand, flake_schemas::MixFlakeSchemas
} }
}; };
visit = [&](ref<eval_cache::AttrCursor> node) std::set<std::string> omittedSystems;
{
flake_schemas::visit(
checkAllSystems ? std::optional<std::string>() : localSystem,
node,
[&](ref<eval_cache::AttrCursor> leaf) // FIXME: rewrite to use EvalCache.
{
if (auto evalChecks = leaf->maybeGetAttr("evalChecks")) {
auto checkNames = evalChecks->getAttrs();
for (auto & checkName : checkNames) {
// FIXME: update activity
auto cursor = evalChecks->getAttr(checkName);
auto b = cursor->getBool();
if (!b)
reportError(Error("Evaluation check '%s' failed.", flake_schemas::toAttrPathStr(cursor)));
}
}
if (auto drv = flake_schemas::derivation(leaf)) { auto resolve = [&] (PosIdx p) {
if (auto isFlakeCheck = leaf->maybeGetAttr("isFlakeCheck")) { return state->positions[p];
if (isFlakeCheck->getBool()) {
auto drvPath = drv->forceDerivation();
drvPaths.push_back(DerivedPath::Built {
.drvPath = makeConstantStorePathRef(drvPath),
.outputs = OutputsSpec::All { },
});
}
}
}
},
[&](std::function<void(flake_schemas::ForEachChild)> forEachChild)
{
forEachChild([&](Symbol attrName, ref<eval_cache::AttrCursor> node, bool isLast)
{
visit(node);
});
},
[&](ref<eval_cache::AttrCursor> node, const std::vector<std::string> & systems) {
for (auto & s : systems)
omittedSystems.insert(s);
});
}; };
flake_schemas::forEachOutput(inventory, [&](Symbol outputName, std::shared_ptr<eval_cache::AttrCursor> output, const std::string & doc, bool isLast) auto argHasName = [&] (Symbol arg, std::string_view expected) {
{ std::string_view name = state->symbols[arg];
if (output) { return
visit(ref(output)); name == expected
} else || name == "_"
uncheckedOutputs.insert(std::string(state->symbols[outputName])); || (hasPrefix(name, "_") && name.substr(1) == expected);
}); };
if (!uncheckedOutputs.empty()) auto checkSystemName = [&](std::string_view system, const PosIdx pos) {
warn("The following flake outputs are unchecked: %s.", // FIXME: what's the format of "system"?
concatStringsSep(", ", uncheckedOutputs)); // FIXME: quote if (system.find('-') == std::string::npos)
reportError(Error("'%s' is not a valid system type, at %s", system, resolve(pos)));
};
auto checkSystemType = [&](std::string_view system, const PosIdx pos) {
if (!checkAllSystems && system != localSystem) {
omittedSystems.insert(std::string(system));
return false;
} else {
return true;
}
};
auto checkDerivation = [&](const std::string & attrPath, Value & v, const PosIdx pos) -> std::optional<StorePath> {
try {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking derivation %s", attrPath));
auto packageInfo = getDerivation(*state, v, false);
if (!packageInfo)
throw Error("flake attribute '%s' is not a derivation", attrPath);
else {
// FIXME: check meta attributes
auto storePath = packageInfo->queryDrvPath();
if (storePath) {
logger->log(lvlInfo,
fmt("derivation evaluated to %s",
store->printStorePath(storePath.value())));
}
return storePath;
}
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking the derivation '%s'", attrPath));
reportError(e);
}
return std::nullopt;
};
std::vector<DerivedPath> drvPaths;
auto checkApp = [&](const std::string & attrPath, Value & v, const PosIdx pos) {
try {
#if 0
// FIXME
auto app = App(*state, v);
for (auto & i : app.context) {
auto [drvPathS, outputName] = NixStringContextElem::parse(i);
store->parseStorePath(drvPathS);
}
#endif
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking the app definition '%s'", attrPath));
reportError(e);
}
};
auto checkOverlay = [&](std::string_view attrPath, Value & v, const PosIdx pos) {
try {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking overlay '%s'", attrPath));
state->forceValue(v, pos);
if (!v.isLambda()) {
throw Error("overlay is not a function, but %s instead", showType(v));
}
if (v.payload.lambda.fun->hasFormals()
|| !argHasName(v.payload.lambda.fun->arg, "final"))
throw Error("overlay does not take an argument named 'final'");
// FIXME: if we have a 'nixpkgs' input, use it to
// evaluate the overlay.
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking the overlay '%s'", attrPath));
reportError(e);
}
};
auto checkModule = [&](std::string_view attrPath, Value & v, const PosIdx pos) {
try {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking NixOS module '%s'", attrPath));
state->forceValue(v, pos);
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking the NixOS module '%s'", attrPath));
reportError(e);
}
};
std::function<void(std::string_view attrPath, Value & v, const PosIdx pos)> checkHydraJobs;
checkHydraJobs = [&](std::string_view attrPath, Value & v, const PosIdx pos) {
try {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking Hydra job '%s'", attrPath));
state->forceAttrs(v, pos, "");
if (state->isDerivation(v))
throw Error("jobset should not be a derivation at top-level");
for (auto & attr : *v.attrs()) {
state->forceAttrs(*attr.value, attr.pos, "");
auto attrPath2 = concatStrings(attrPath, ".", state->symbols[attr.name]);
if (state->isDerivation(*attr.value)) {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking Hydra job '%s'", attrPath2));
checkDerivation(attrPath2, *attr.value, attr.pos);
} else
checkHydraJobs(attrPath2, *attr.value, attr.pos);
}
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking the Hydra jobset '%s'", attrPath));
reportError(e);
}
};
auto checkNixOSConfiguration = [&](const std::string & attrPath, Value & v, const PosIdx pos) {
try {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking NixOS configuration '%s'", attrPath));
Bindings & bindings(*state->allocBindings(0));
auto vToplevel = findAlongAttrPath(*state, "config.system.build.toplevel", bindings, v).first;
state->forceValue(*vToplevel, pos);
if (!state->isDerivation(*vToplevel))
throw Error("attribute 'config.system.build.toplevel' is not a derivation");
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking the NixOS configuration '%s'", attrPath));
reportError(e);
}
};
auto checkTemplate = [&](std::string_view attrPath, Value & v, const PosIdx pos) {
try {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking template '%s'", attrPath));
state->forceAttrs(v, pos, "");
if (auto attr = v.attrs()->get(state->symbols.create("path"))) {
if (attr->name == state->symbols.create("path")) {
NixStringContext context;
auto path = state->coerceToPath(attr->pos, *attr->value, context, "");
if (!path.pathExists())
throw Error("template '%s' refers to a non-existent path '%s'", attrPath, path);
// TODO: recursively check the flake in 'path'.
}
} else
throw Error("template '%s' lacks attribute 'path'", attrPath);
if (auto attr = v.attrs()->get(state->symbols.create("description")))
state->forceStringNoCtx(*attr->value, attr->pos, "");
else
throw Error("template '%s' lacks attribute 'description'", attrPath);
for (auto & attr : *v.attrs()) {
std::string_view name(state->symbols[attr.name]);
if (name != "path" && name != "description" && name != "welcomeText")
throw Error("template '%s' has unsupported attribute '%s'", attrPath, name);
}
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking the template '%s'", attrPath));
reportError(e);
}
};
auto checkBundler = [&](const std::string & attrPath, Value & v, const PosIdx pos) {
try {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking bundler '%s'", attrPath));
state->forceValue(v, pos);
if (!v.isLambda())
throw Error("bundler must be a function");
// TODO: check types of inputs/outputs?
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking the template '%s'", attrPath));
reportError(e);
}
};
{
Activity act(*logger, lvlInfo, actUnknown, "evaluating flake");
auto vFlake = state->allocValue();
flake::callFlake(*state, flake, *vFlake);
enumerateOutputs(*state,
*vFlake,
[&](std::string_view name, Value & vOutput, const PosIdx pos) {
Activity act(*logger, lvlInfo, actUnknown,
fmt("checking flake output '%s'", name));
try {
evalSettings.enableImportFromDerivation.setDefault(name != "hydraJobs");
state->forceValue(vOutput, pos);
std::string_view replacement =
name == "defaultPackage" ? "packages.<system>.default" :
name == "defaultApp" ? "apps.<system>.default" :
name == "defaultTemplate" ? "templates.default" :
name == "defaultBundler" ? "bundlers.<system>.default" :
name == "overlay" ? "overlays.default" :
name == "devShell" ? "devShells.<system>.default" :
name == "nixosModule" ? "nixosModules.default" :
"";
if (replacement != "")
warn("flake output attribute '%s' is deprecated; use '%s' instead", name, replacement);
if (name == "checks") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
const auto & attr_name = state->symbols[attr.name];
checkSystemName(attr_name, attr.pos);
if (checkSystemType(attr_name, attr.pos)) {
state->forceAttrs(*attr.value, attr.pos, "");
for (auto & attr2 : *attr.value->attrs()) {
auto drvPath = checkDerivation(
fmt("%s.%s.%s", name, attr_name, state->symbols[attr2.name]),
*attr2.value, attr2.pos);
if (drvPath && attr_name == settings.thisSystem.get()) {
drvPaths.push_back(DerivedPath::Built {
.drvPath = makeConstantStorePathRef(*drvPath),
.outputs = OutputsSpec::All { },
});
}
}
}
}
}
else if (name == "formatter") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
const auto & attr_name = state->symbols[attr.name];
checkSystemName(attr_name, attr.pos);
if (checkSystemType(attr_name, attr.pos)) {
checkApp(
fmt("%s.%s", name, attr_name),
*attr.value, attr.pos);
};
}
}
else if (name == "packages" || name == "devShells") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
const auto & attr_name = state->symbols[attr.name];
checkSystemName(attr_name, attr.pos);
if (checkSystemType(attr_name, attr.pos)) {
state->forceAttrs(*attr.value, attr.pos, "");
for (auto & attr2 : *attr.value->attrs())
checkDerivation(
fmt("%s.%s.%s", name, attr_name, state->symbols[attr2.name]),
*attr2.value, attr2.pos);
};
}
}
else if (name == "apps") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
const auto & attr_name = state->symbols[attr.name];
checkSystemName(attr_name, attr.pos);
if (checkSystemType(attr_name, attr.pos)) {
state->forceAttrs(*attr.value, attr.pos, "");
for (auto & attr2 : *attr.value->attrs())
checkApp(
fmt("%s.%s.%s", name, attr_name, state->symbols[attr2.name]),
*attr2.value, attr2.pos);
};
}
}
else if (name == "defaultPackage" || name == "devShell") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
const auto & attr_name = state->symbols[attr.name];
checkSystemName(attr_name, attr.pos);
if (checkSystemType(attr_name, attr.pos)) {
checkDerivation(
fmt("%s.%s", name, attr_name),
*attr.value, attr.pos);
};
}
}
else if (name == "defaultApp") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
const auto & attr_name = state->symbols[attr.name];
checkSystemName(attr_name, attr.pos);
if (checkSystemType(attr_name, attr.pos) ) {
checkApp(
fmt("%s.%s", name, attr_name),
*attr.value, attr.pos);
};
}
}
else if (name == "legacyPackages") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
checkSystemName(state->symbols[attr.name], attr.pos);
checkSystemType(state->symbols[attr.name], attr.pos);
// FIXME: do getDerivations?
}
}
else if (name == "overlay")
checkOverlay(name, vOutput, pos);
else if (name == "overlays") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs())
checkOverlay(fmt("%s.%s", name, state->symbols[attr.name]),
*attr.value, attr.pos);
}
else if (name == "nixosModule")
checkModule(name, vOutput, pos);
else if (name == "nixosModules") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs())
checkModule(fmt("%s.%s", name, state->symbols[attr.name]),
*attr.value, attr.pos);
}
else if (name == "nixosConfigurations") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs())
checkNixOSConfiguration(fmt("%s.%s", name, state->symbols[attr.name]),
*attr.value, attr.pos);
}
else if (name == "hydraJobs")
checkHydraJobs(name, vOutput, pos);
else if (name == "defaultTemplate")
checkTemplate(name, vOutput, pos);
else if (name == "templates") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs())
checkTemplate(fmt("%s.%s", name, state->symbols[attr.name]),
*attr.value, attr.pos);
}
else if (name == "defaultBundler") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
const auto & attr_name = state->symbols[attr.name];
checkSystemName(attr_name, attr.pos);
if (checkSystemType(attr_name, attr.pos)) {
checkBundler(
fmt("%s.%s", name, attr_name),
*attr.value, attr.pos);
};
}
}
else if (name == "bundlers") {
state->forceAttrs(vOutput, pos, "");
for (auto & attr : *vOutput.attrs()) {
const auto & attr_name = state->symbols[attr.name];
checkSystemName(attr_name, attr.pos);
if (checkSystemType(attr_name, attr.pos)) {
state->forceAttrs(*attr.value, attr.pos, "");
for (auto & attr2 : *attr.value->attrs()) {
checkBundler(
fmt("%s.%s.%s", name, attr_name, state->symbols[attr2.name]),
*attr2.value, attr2.pos);
}
};
}
}
else if (
name == "lib"
|| name == "darwinConfigurations"
|| name == "darwinModules"
|| name == "flakeModule"
|| name == "flakeModules"
|| name == "herculesCI"
|| name == "homeConfigurations"
|| name == "homeModule"
|| name == "homeModules"
|| name == "nixopsConfigurations"
)
// Known but unchecked community attribute
;
else
warn("unknown flake output '%s'", name);
} catch (Error & e) {
e.addTrace(resolve(pos), HintFmt("while checking flake output '%s'", name));
reportError(e);
}
});
}
if (build && !drvPaths.empty()) { if (build && !drvPaths.empty()) {
Activity act(*logger, lvlInfo, actUnknown, Activity act(*logger, lvlInfo, actUnknown,
fmt("running %d flake checks", drvPaths.size())); fmt("running %d flake checks", drvPaths.size()));
store->buildPaths(drvPaths); store->buildPaths(drvPaths);
} }
if (hasErrors) if (hasErrors)
throw Error("some errors were encountered during the evaluation"); throw Error("some errors were encountered during the evaluation");
@ -438,7 +810,7 @@ struct CmdFlakeCheck : FlakeCommand, flake_schemas::MixFlakeSchemas
"Use '--all-systems' to check all.", "Use '--all-systems' to check all.",
concatStringsSep(", ", omittedSystems) concatStringsSep(", ", omittedSystems)
); );
} };
}; };
}; };
@ -723,7 +1095,7 @@ struct CmdFlakeArchive : FlakeCommand, MixJSON, MixDryRun
} }
}; };
struct CmdFlakeShow : FlakeCommand, MixJSON, flake_schemas::MixFlakeSchemas struct CmdFlakeShow : FlakeCommand, MixJSON
{ {
bool showLegacy = false; bool showLegacy = false;
bool showAllSystems = false; bool showAllSystems = false;
@ -756,158 +1128,267 @@ struct CmdFlakeShow : FlakeCommand, MixJSON, flake_schemas::MixFlakeSchemas
void run(nix::ref<nix::Store> store) override void run(nix::ref<nix::Store> store) override
{ {
evalSettings.enableImportFromDerivation.setDefault(false);
auto state = getEvalState(); auto state = getEvalState();
auto flake = std::make_shared<LockedFlake>(lockFlake()); auto flake = std::make_shared<LockedFlake>(lockFlake());
auto localSystem = std::string(settings.thisSystem.get()); auto localSystem = std::string(settings.thisSystem.get());
auto [cache, inventory] = flake_schemas::call(*state, flake, getDefaultFlakeSchemas()); std::function<bool(
eval_cache::AttrCursor & visitor,
const std::vector<Symbol> &attrPath,
const Symbol &attr)> hasContent;
if (json) { // For frameworks it's important that structures are as lazy as possible
std::function<void(ref<eval_cache::AttrCursor> node, nlohmann::json & obj)> visit; // to prevent infinite recursions, performance issues and errors that
// aren't related to the thing to evaluate. As a consequence, they have
// to emit more attributes than strictly (sic) necessary.
// However, these attributes with empty values are not useful to the user
// so we omit them.
hasContent = [&](
eval_cache::AttrCursor & visitor,
const std::vector<Symbol> &attrPath,
const Symbol &attr) -> bool
{
auto attrPath2(attrPath);
attrPath2.push_back(attr);
auto attrPathS = state->symbols.resolve(attrPath2);
const auto & attrName = state->symbols[attr];
visit = [&](ref<eval_cache::AttrCursor> node, nlohmann::json & obj) auto visitor2 = visitor.getAttr(attrName);
{
flake_schemas::visit(
showAllSystems ? std::optional<std::string>() : localSystem,
node,
[&](ref<eval_cache::AttrCursor> leaf) try {
{ if ((attrPathS[0] == "apps"
obj.emplace("leaf", true); || attrPathS[0] == "checks"
|| attrPathS[0] == "devShells"
if (auto what = flake_schemas::what(leaf)) || attrPathS[0] == "legacyPackages"
obj.emplace("what", what); || attrPathS[0] == "packages")
&& (attrPathS.size() == 1 || attrPathS.size() == 2)) {
if (auto shortDescription = flake_schemas::shortDescription(leaf)) for (const auto &subAttr : visitor2->getAttrs()) {
obj.emplace("shortDescription", shortDescription); if (hasContent(*visitor2, attrPath2, subAttr)) {
return true;
if (auto drv = flake_schemas::derivation(leaf)) }
obj.emplace("derivationName", drv->getAttr(state->sName)->getString()); }
return false;
// FIXME: add more stuff
},
[&](std::function<void(flake_schemas::ForEachChild)> forEachChild)
{
auto children = nlohmann::json::object();
forEachChild([&](Symbol attrName, ref<eval_cache::AttrCursor> node, bool isLast)
{
auto j = nlohmann::json::object();
try {
visit(node, j);
} catch (EvalError & e) {
// FIXME: make it a flake schema attribute whether to ignore evaluation errors.
if (node->root->state.symbols[flake_schemas::toAttrPath(node)[0]] == "legacyPackages")
j.emplace("failed", true);
else
throw;
}
children.emplace(state->symbols[attrName], std::move(j));
});
obj.emplace("children", std::move(children));
},
[&](ref<eval_cache::AttrCursor> node, const std::vector<std::string> & systems)
{
obj.emplace("filtered", true);
});
};
auto res = nlohmann::json::object();
flake_schemas::forEachOutput(inventory, [&](Symbol outputName, std::shared_ptr<eval_cache::AttrCursor> output, const std::string & doc, bool isLast)
{
auto j = nlohmann::json::object();
if (!showLegacy && state->symbols[outputName] == "legacyPackages") {
j.emplace("skipped", true);
} else if (output) {
j.emplace("doc", doc);
auto j2 = nlohmann::json::object();
visit(ref(output), j2);
j.emplace("output", std::move(j2));
} else
j.emplace("unknown", true);
res.emplace(state->symbols[outputName], j);
});
logger->cout("%s", res.dump());
}
else {
logger->cout(ANSI_BOLD "%s" ANSI_NORMAL, flake->flake.lockedRef);
std::function<void(
ref<eval_cache::AttrCursor> node,
const std::string & headerPrefix,
const std::string & prevPrefix)> visit;
visit = [&](
ref<eval_cache::AttrCursor> node,
const std::string & headerPrefix,
const std::string & prevPrefix)
{
flake_schemas::visit(
showAllSystems ? std::optional<std::string>() : localSystem,
node,
[&](ref<eval_cache::AttrCursor> leaf)
{
auto s = headerPrefix;
if (auto what = flake_schemas::what(leaf))
s += fmt(": %s", *what);
if (auto drv = flake_schemas::derivation(leaf))
s += fmt(ANSI_ITALIC " [%s]" ANSI_NORMAL, drv->getAttr(state->sName)->getString());
logger->cout(s);
},
[&](std::function<void(flake_schemas::ForEachChild)> forEachChild)
{
logger->cout(headerPrefix);
forEachChild([&](Symbol attrName, ref<eval_cache::AttrCursor> node, bool isLast)
{
visit(node,
fmt(ANSI_GREEN "%s%s" ANSI_NORMAL ANSI_BOLD "%s" ANSI_NORMAL, prevPrefix,
isLast ? treeLast : treeConn, state->symbols[attrName]),
prevPrefix + (isLast ? treeNull : treeLine));
});
},
[&](ref<eval_cache::AttrCursor> node, const std::vector<std::string> & systems)
{
logger->cout(fmt("%s " ANSI_WARNING "omitted" ANSI_NORMAL " (use '--all-systems' to show)", headerPrefix));
});
};
flake_schemas::forEachOutput(inventory, [&](Symbol outputName, std::shared_ptr<eval_cache::AttrCursor> output, const std::string & doc, bool isLast)
{
auto headerPrefix = fmt(
ANSI_GREEN "%s" ANSI_NORMAL ANSI_BOLD "%s" ANSI_NORMAL,
isLast ? treeLast : treeConn, state->symbols[outputName]);
if (!showLegacy && state->symbols[outputName] == "legacyPackages") {
logger->cout(headerPrefix);
logger->cout(
ANSI_GREEN "%s" "%s" ANSI_NORMAL ANSI_ITALIC "%s" ANSI_NORMAL,
isLast ? treeNull : treeLine,
treeLast,
"(skipped; use '--legacy' to show)");
} else if (output) {
visit(ref(output), headerPrefix, isLast ? treeNull : treeLine);
} else {
logger->cout(headerPrefix);
logger->cout(
ANSI_GREEN "%s" "%s" ANSI_NORMAL ANSI_ITALIC "%s" ANSI_NORMAL,
isLast ? treeNull : treeLine,
treeLast,
"(unknown flake output)");
} }
});
} if ((attrPathS.size() == 1)
&& (attrPathS[0] == "formatter"
|| attrPathS[0] == "nixosConfigurations"
|| attrPathS[0] == "nixosModules"
|| attrPathS[0] == "overlays"
)) {
for (const auto &subAttr : visitor2->getAttrs()) {
if (hasContent(*visitor2, attrPath2, subAttr)) {
return true;
}
}
return false;
}
// If we don't recognize it, it's probably content
return true;
} catch (EvalError & e) {
// Some attrs may contain errors, e.g. legacyPackages of
// nixpkgs. We still want to recurse into it, instead of
// skipping it at all.
return true;
}
};
std::function<nlohmann::json(
eval_cache::AttrCursor & visitor,
const std::vector<Symbol> & attrPath,
const std::string & headerPrefix,
const std::string & nextPrefix)> visit;
visit = [&](
eval_cache::AttrCursor & visitor,
const std::vector<Symbol> & attrPath,
const std::string & headerPrefix,
const std::string & nextPrefix)
-> nlohmann::json
{
auto j = nlohmann::json::object();
auto attrPathS = state->symbols.resolve(attrPath);
Activity act(*logger, lvlInfo, actUnknown,
fmt("evaluating '%s'", concatStringsSep(".", attrPathS)));
try {
auto recurse = [&]()
{
if (!json)
logger->cout("%s", headerPrefix);
std::vector<Symbol> attrs;
for (const auto &attr : visitor.getAttrs()) {
if (hasContent(visitor, attrPath, attr))
attrs.push_back(attr);
}
for (const auto & [i, attr] : enumerate(attrs)) {
const auto & attrName = state->symbols[attr];
bool last = i + 1 == attrs.size();
auto visitor2 = visitor.getAttr(attrName);
auto attrPath2(attrPath);
attrPath2.push_back(attr);
auto j2 = visit(*visitor2, attrPath2,
fmt(ANSI_GREEN "%s%s" ANSI_NORMAL ANSI_BOLD "%s" ANSI_NORMAL, nextPrefix, last ? treeLast : treeConn, attrName),
nextPrefix + (last ? treeNull : treeLine));
if (json) j.emplace(attrName, std::move(j2));
}
};
auto showDerivation = [&]()
{
auto name = visitor.getAttr(state->sName)->getString();
if (json) {
std::optional<std::string> description;
if (auto aMeta = visitor.maybeGetAttr(state->sMeta)) {
if (auto aDescription = aMeta->maybeGetAttr(state->sDescription))
description = aDescription->getString();
}
j.emplace("type", "derivation");
j.emplace("name", name);
if (description)
j.emplace("description", *description);
} else {
logger->cout("%s: %s '%s'",
headerPrefix,
attrPath.size() == 2 && attrPathS[0] == "devShell" ? "development environment" :
attrPath.size() >= 2 && attrPathS[0] == "devShells" ? "development environment" :
attrPath.size() == 3 && attrPathS[0] == "checks" ? "derivation" :
attrPath.size() >= 1 && attrPathS[0] == "hydraJobs" ? "derivation" :
"package",
name);
}
};
if (attrPath.size() == 0
|| (attrPath.size() == 1 && (
attrPathS[0] == "defaultPackage"
|| attrPathS[0] == "devShell"
|| attrPathS[0] == "formatter"
|| attrPathS[0] == "nixosConfigurations"
|| attrPathS[0] == "nixosModules"
|| attrPathS[0] == "defaultApp"
|| attrPathS[0] == "templates"
|| attrPathS[0] == "overlays"))
|| ((attrPath.size() == 1 || attrPath.size() == 2)
&& (attrPathS[0] == "checks"
|| attrPathS[0] == "packages"
|| attrPathS[0] == "devShells"
|| attrPathS[0] == "apps"))
)
{
recurse();
}
else if (
(attrPath.size() == 2 && (attrPathS[0] == "defaultPackage" || attrPathS[0] == "devShell" || attrPathS[0] == "formatter"))
|| (attrPath.size() == 3 && (attrPathS[0] == "checks" || attrPathS[0] == "packages" || attrPathS[0] == "devShells"))
)
{
if (!showAllSystems && std::string(attrPathS[1]) != localSystem) {
if (!json)
logger->cout(fmt("%s " ANSI_WARNING "omitted" ANSI_NORMAL " (use '--all-systems' to show)", headerPrefix));
else {
logger->warn(fmt("%s omitted (use '--all-systems' to show)", concatStringsSep(".", attrPathS)));
}
} else {
if (visitor.isDerivation())
showDerivation();
else
throw Error("expected a derivation");
}
}
else if (attrPath.size() > 0 && attrPathS[0] == "hydraJobs") {
if (visitor.isDerivation())
showDerivation();
else
recurse();
}
else if (attrPath.size() > 0 && attrPathS[0] == "legacyPackages") {
if (attrPath.size() == 1)
recurse();
else if (!showLegacy){
if (!json)
logger->cout(fmt("%s " ANSI_WARNING "omitted" ANSI_NORMAL " (use '--legacy' to show)", headerPrefix));
else {
logger->warn(fmt("%s omitted (use '--legacy' to show)", concatStringsSep(".", attrPathS)));
}
} else if (!showAllSystems && std::string(attrPathS[1]) != localSystem) {
if (!json)
logger->cout(fmt("%s " ANSI_WARNING "omitted" ANSI_NORMAL " (use '--all-systems' to show)", headerPrefix));
else {
logger->warn(fmt("%s omitted (use '--all-systems' to show)", concatStringsSep(".", attrPathS)));
}
} else {
if (visitor.isDerivation())
showDerivation();
else if (attrPath.size() <= 2)
// FIXME: handle recurseIntoAttrs
recurse();
}
}
else if (
(attrPath.size() == 2 && attrPathS[0] == "defaultApp") ||
(attrPath.size() == 3 && attrPathS[0] == "apps"))
{
auto aType = visitor.maybeGetAttr("type");
if (!aType || aType->getString() != "app")
state->error<EvalError>("not an app definition").debugThrow();
if (json) {
j.emplace("type", "app");
} else {
logger->cout("%s: app", headerPrefix);
}
}
else if (
(attrPath.size() == 1 && attrPathS[0] == "defaultTemplate") ||
(attrPath.size() == 2 && attrPathS[0] == "templates"))
{
auto description = visitor.getAttr("description")->getString();
if (json) {
j.emplace("type", "template");
j.emplace("description", description);
} else {
logger->cout("%s: template: " ANSI_BOLD "%s" ANSI_NORMAL, headerPrefix, description);
}
}
else {
auto [type, description] =
(attrPath.size() == 1 && attrPathS[0] == "overlay")
|| (attrPath.size() == 2 && attrPathS[0] == "overlays") ? std::make_pair("nixpkgs-overlay", "Nixpkgs overlay") :
attrPath.size() == 2 && attrPathS[0] == "nixosConfigurations" ? std::make_pair("nixos-configuration", "NixOS configuration") :
(attrPath.size() == 1 && attrPathS[0] == "nixosModule")
|| (attrPath.size() == 2 && attrPathS[0] == "nixosModules") ? std::make_pair("nixos-module", "NixOS module") :
std::make_pair("unknown", "unknown");
if (json) {
j.emplace("type", type);
} else {
logger->cout("%s: " ANSI_WARNING "%s" ANSI_NORMAL, headerPrefix, description);
}
}
} catch (EvalError & e) {
if (!(attrPath.size() > 0 && attrPathS[0] == "legacyPackages"))
throw;
}
return j;
};
auto cache = openEvalCache(*state, flake);
auto j = visit(*cache->getRoot(), {}, fmt(ANSI_BOLD "%s" ANSI_NORMAL, flake->flake.lockedRef), "");
if (json)
logger->cout("%s", j.dump());
} }
}; };

6
src/nix/local.mk
View file

@ -55,9 +55,3 @@ $(d)/main.cc: \
$(d)/profile.cc: $(d)/profile.md $(d)/profile.cc: $(d)/profile.md
$(d)/profile.md: $(d)/profiles.md.gen.hh $(d)/profile.md: $(d)/profiles.md.gen.hh
src/nix/flake.cc: src/nix/call-flake-schemas.nix.gen.hh src/nix/builtin-flake-schemas.nix.gen.hh
src/nix/builtin-flake-schemas.nix: $(default_flake_schemas)/flake.nix
$(trace-gen) cp $^ $@
@chmod +w $@

11
tests/functional/flakes/check.sh
View file

@ -16,6 +16,17 @@ EOF
nix flake check $flakeDir nix flake check $flakeDir
cat > $flakeDir/flake.nix <<EOF
{
outputs = { self }: {
overlay = finalll: prev: {
};
};
}
EOF
(! nix flake check $flakeDir)
cat > $flakeDir/flake.nix <<EOF cat > $flakeDir/flake.nix <<EOF
{ {
outputs = { self, ... }: { outputs = { self, ... }: {

43
tests/functional/flakes/show.sh
View file

@ -15,9 +15,9 @@ nix flake show --json > show-output.json
nix eval --impure --expr ' nix eval --impure --expr '
let show_output = builtins.fromJSON (builtins.readFile ./show-output.json); let show_output = builtins.fromJSON (builtins.readFile ./show-output.json);
in in
assert show_output.packages.output.children.someOtherSystem.filtered; assert show_output.packages.someOtherSystem.default == {};
assert show_output.packages.output.children.${builtins.currentSystem}.children.default.derivationName == "simple"; assert show_output.packages.${builtins.currentSystem}.default.name == "simple";
assert show_output.legacyPackages.skipped; assert show_output.legacyPackages.${builtins.currentSystem} == {};
true true
' '
@ -26,8 +26,8 @@ nix flake show --json --all-systems > show-output.json
nix eval --impure --expr ' nix eval --impure --expr '
let show_output = builtins.fromJSON (builtins.readFile ./show-output.json); let show_output = builtins.fromJSON (builtins.readFile ./show-output.json);
in in
assert show_output.packages.output.children.someOtherSystem.children.default.derivationName == "simple"; assert show_output.packages.someOtherSystem.default.name == "simple";
assert show_output.legacyPackages.skipped; assert show_output.legacyPackages.${builtins.currentSystem} == {};
true true
' '
@ -36,7 +36,34 @@ nix flake show --json --legacy > show-output.json
nix eval --impure --expr ' nix eval --impure --expr '
let show_output = builtins.fromJSON (builtins.readFile ./show-output.json); let show_output = builtins.fromJSON (builtins.readFile ./show-output.json);
in in
assert show_output.legacyPackages.output.children.${builtins.currentSystem}.children.hello.derivationName == "simple"; assert show_output.legacyPackages.${builtins.currentSystem}.hello.name == "simple";
true
'
# Test that attributes are only reported when they have actual content
cat >flake.nix <<EOF
{
description = "Bla bla";
outputs = inputs: rec {
apps.$system = { };
checks.$system = { };
devShells.$system = { };
legacyPackages.$system = { };
packages.$system = { };
packages.someOtherSystem = { };
formatter = { };
nixosConfigurations = { };
nixosModules = { };
};
}
EOF
nix flake show --json --all-systems > show-output.json
nix eval --impure --expr '
let show_output = builtins.fromJSON (builtins.readFile ./show-output.json);
in
assert show_output == { };
true true
' '
@ -56,7 +83,7 @@ nix flake show --json --legacy --all-systems > show-output.json
nix eval --impure --expr ' nix eval --impure --expr '
let show_output = builtins.fromJSON (builtins.readFile ./show-output.json); let show_output = builtins.fromJSON (builtins.readFile ./show-output.json);
in in
assert show_output.legacyPackages.output.children.${builtins.currentSystem}.children.AAAAAASomeThingsFailToEvaluate.failed; assert show_output.legacyPackages.${builtins.currentSystem}.AAAAAASomeThingsFailToEvaluate == { };
assert show_output.legacyPackages.output.children.${builtins.currentSystem}.children.simple.derivationName == "simple"; assert show_output.legacyPackages.${builtins.currentSystem}.simple.name == "simple";
true true
' '

4
tests/functional/fmt.sh
View file

@ -32,6 +32,4 @@ cat << EOF > flake.nix
EOF EOF
nix fmt ./file ./folder | grep 'Formatting: ./file ./folder' nix fmt ./file ./folder | grep 'Formatting: ./file ./folder'
nix flake check nix flake check
nix flake show | grep -P "package 'formatter'"
clearStore
nix flake show | grep -P "package.*\[formatter\]"