mirror of
https://github.com/NixOS/nix
synced 2025-06-25 06:31:14 +02:00
manual: Edit
This commit is contained in:
parent
2aa6e0f084
commit
1e00d14c29
8 changed files with 55 additions and 27 deletions
|
@ -20,7 +20,7 @@ For a local machine to forward a build to a remote machine, the remote machine m
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
To test connecting to a remote Nix instance (in this case `mac`), run:
|
To test connecting to a remote [Nix instance] (in this case `mac`), run:
|
||||||
|
|
||||||
```console
|
```console
|
||||||
nix store info --store ssh://username@mac
|
nix store info --store ssh://username@mac
|
||||||
|
@ -106,3 +106,5 @@ file included in `builders` via the syntax `@/path/to/file`. For example,
|
||||||
|
|
||||||
causes the list of machines in `/etc/nix/machines` to be included.
|
causes the list of machines in `/etc/nix/machines` to be included.
|
||||||
(This is the default.)
|
(This is the default.)
|
||||||
|
|
||||||
|
[Nix instance]: @docroot@/glossary.md#gloss-nix-instance
|
|
@ -1,5 +1,13 @@
|
||||||
# Glossary
|
# Glossary
|
||||||
|
|
||||||
|
- [build system]{#gloss-build-system}
|
||||||
|
|
||||||
|
Generic term for software that facilitates the building of software by automating the invocation of compilers, linkers, and other tools.
|
||||||
|
|
||||||
|
Nix can be used as a generic build system.
|
||||||
|
It has no knowledge of any particular programming language or toolchain.
|
||||||
|
These details are specified in [derivation expressions](#gloss-derivation-expression).
|
||||||
|
|
||||||
- [content address]{#gloss-content-address}
|
- [content address]{#gloss-content-address}
|
||||||
|
|
||||||
A
|
A
|
||||||
|
@ -19,6 +27,10 @@
|
||||||
|
|
||||||
Besides content addressing, the Nix store also uses [input addressing](#gloss-input-addressed-store-object).
|
Besides content addressing, the Nix store also uses [input addressing](#gloss-input-addressed-store-object).
|
||||||
|
|
||||||
|
- [content-addressed storage]{#gloss-content-addressed-store}
|
||||||
|
|
||||||
|
The industry term for storage and retrieval systems using [content addressing](#gloss-content-address). A Nix store also has [input addressing](#gloss-input-addressed-store-object), and metadata.
|
||||||
|
|
||||||
- [store derivation]{#gloss-store-derivation}
|
- [store derivation]{#gloss-store-derivation}
|
||||||
|
|
||||||
A single build task.
|
A single build task.
|
||||||
|
@ -88,6 +100,12 @@
|
||||||
|
|
||||||
[store]: #gloss-store
|
[store]: #gloss-store
|
||||||
|
|
||||||
|
- [Nix instance]{#gloss-nix-instance}
|
||||||
|
<!-- ambiguous -->
|
||||||
|
1. An installation of Nix, which includes the presence of a [store], and the Nix package manager which operates on that store.
|
||||||
|
A local Nix installation and a [remote builder](@docroot@/advanced-topics/distributed-builds.md) are two examples of Nix instances.
|
||||||
|
2. A running Nix process, such as the `nix` command.
|
||||||
|
|
||||||
- [binary cache]{#gloss-binary-cache}
|
- [binary cache]{#gloss-binary-cache}
|
||||||
|
|
||||||
A *binary cache* is a Nix store which uses a different format: its
|
A *binary cache* is a Nix store which uses a different format: its
|
||||||
|
@ -220,7 +238,7 @@
|
||||||
directly or indirectly “reachable” from that store path; that is,
|
directly or indirectly “reachable” from that store path; that is,
|
||||||
it’s the closure of the path under the *references* relation. For
|
it’s the closure of the path under the *references* relation. For
|
||||||
a package, the closure of its derivation is equivalent to the
|
a package, the closure of its derivation is equivalent to the
|
||||||
build-time dependencies, while the closure of its output path is
|
build-time dependencies, while the closure of its [output path] is
|
||||||
equivalent to its runtime dependencies. For correct deployment it
|
equivalent to its runtime dependencies. For correct deployment it
|
||||||
is necessary to deploy whole closures, since otherwise at runtime
|
is necessary to deploy whole closures, since otherwise at runtime
|
||||||
files could be missing. The command `nix-store --query --requisites ` prints out
|
files could be missing. The command `nix-store --query --requisites ` prints out
|
||||||
|
|
|
@ -53,7 +53,7 @@ where
|
||||||
method of content addressing store objects,
|
method of content addressing store objects,
|
||||||
if the hash algorithm is [SHA-256].
|
if the hash algorithm is [SHA-256].
|
||||||
Just like in the "Text" case, we can have the store objects referenced by their paths.
|
Just like in the "Text" case, we can have the store objects referenced by their paths.
|
||||||
Additionally, we can have an optional `:self` label to denote self reference.
|
Additionally, we can have an optional `:self` label to denote self-reference.
|
||||||
|
|
||||||
- ```ebnf
|
- ```ebnf
|
||||||
| "output:" id
|
| "output:" id
|
||||||
|
|
|
@ -54,7 +54,7 @@ The [`builder`](./derivation/index.md#builder) is executed as follows:
|
||||||
it’s `out`.)
|
it’s `out`.)
|
||||||
|
|
||||||
- If an output path already exists, it is removed. Also, locks are
|
- If an output path already exists, it is removed. Also, locks are
|
||||||
acquired to prevent multiple Nix instances from performing the same
|
acquired to prevent multiple [Nix instances][Nix instance] from performing the same
|
||||||
build at the same time.
|
build at the same time.
|
||||||
|
|
||||||
- A log of the combined standard output and error is written to
|
- A log of the combined standard output and error is written to
|
||||||
|
@ -95,3 +95,6 @@ If the builder exited successfully, the following steps happen in order to turn
|
||||||
|
|
||||||
Nix also scans for references to other outputs' paths in the same way, because outputs are allowed to refer to each other.
|
Nix also scans for references to other outputs' paths in the same way, because outputs are allowed to refer to each other.
|
||||||
If the outputs' references to each other form a cycle, this is an error, because the references of store objects much be acyclic.
|
If the outputs' references to each other form a cycle, this is an error, because the references of store objects much be acyclic.
|
||||||
|
|
||||||
|
|
||||||
|
[Nix instance]: @docroot@/glossary.md#gloss-nix-instance
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
# Store Derivation and Deriving Path
|
# Store Derivation and Deriving Path
|
||||||
|
|
||||||
Besides functioning as a [content addressed store] the Nix store layer works as a [build system].
|
Besides functioning as a [content-addressed store], the Nix store layer works as a [build system].
|
||||||
Other system (like Git or IPFS) also store and transfer immutable data, but they don't concern themselves with *how* that data was created.
|
Other systems (like Git or IPFS) also store and transfer immutable data, but they don't concern themselves with *how* that data was created.
|
||||||
|
|
||||||
This is where Nix distinguishes itself.
|
This is where Nix distinguishes itself.
|
||||||
*Derivations* represent individual build steps, and *deriving paths* are needed to refer to the *outputs* of those build steps before they are built.
|
*Derivations* represent individual build steps, and *deriving paths* are needed to refer to the *outputs* of those build steps before they are built.
|
||||||
|
@ -42,6 +42,8 @@ A derivation consists of:
|
||||||
[args]: #args
|
[args]: #args
|
||||||
[env]: #env
|
[env]: #env
|
||||||
[system]: #system
|
[system]: #system
|
||||||
|
[content-addressed store]: @docroot@/glossary.md#gloss-content-addressed-store
|
||||||
|
[build system]: @docroot@/glossary.md#gloss-build-system
|
||||||
|
|
||||||
### Referencing derivations {#derivation-path}
|
### Referencing derivations {#derivation-path}
|
||||||
|
|
||||||
|
@ -78,7 +80,7 @@ type DerivingPath = ConstantPath | OutputPath;
|
||||||
```
|
```
|
||||||
|
|
||||||
Deriving paths are necessary because, in general and particularly for [content-addressing derivations][content-addressing derivation], the [store path] of an [output] is not known in advance.
|
Deriving paths are necessary because, in general and particularly for [content-addressing derivations][content-addressing derivation], the [store path] of an [output] is not known in advance.
|
||||||
We can use an output deriving path to refer to such an out, instead of the store path which we do not yet know.
|
We can use an output deriving path to refer to such an output, instead of the store path which we do not yet know.
|
||||||
|
|
||||||
[deriving path]: #deriving-path
|
[deriving path]: #deriving-path
|
||||||
[validity]: @docroot@/glossary.md#gloss-validity
|
[validity]: @docroot@/glossary.md#gloss-validity
|
||||||
|
@ -89,7 +91,7 @@ A derivation is constructed from the parts documented in the following subsectio
|
||||||
|
|
||||||
### Inputs {#inputs}
|
### Inputs {#inputs}
|
||||||
|
|
||||||
The inputs are a set of [deriving paths][deriving path], refering to all store objects needed in order to perform this build step.
|
The inputs are a set of [deriving paths][deriving path], referring to all store objects needed in order to perform this build step.
|
||||||
|
|
||||||
The [process creation fields] will presumably include many [store paths][store path]:
|
The [process creation fields] will presumably include many [store paths][store path]:
|
||||||
|
|
||||||
|
@ -102,12 +104,13 @@ But rather than somehow scanning all the other fields for inputs, Nix requires t
|
||||||
|
|
||||||
The system type on which the [`builder`](#attr-builder) executable is meant to be run.
|
The system type on which the [`builder`](#attr-builder) executable is meant to be run.
|
||||||
|
|
||||||
A necessary condition for Nix to schedule a given derivation on some Nix instance is for the "system" of that derivation to match that instance's [`system` configuration option].
|
A necessary condition for Nix to schedule a given derivation on some [Nix instance] is for the "system" of that derivation to match that instance's [`system` configuration option] or [`extra-platforms` configuration option].
|
||||||
|
|
||||||
By putting the `system` in each derivation, Nix allows *heterogenous* build plans, where not all steps can be run on the same machine or same sort of machine.
|
By putting the `system` in each derivation, Nix allows *heterogenous* build plans, where not all steps can be run on the same machine or same sort of machine.
|
||||||
Nix can schedule builds such that it automatically builds on other platforms by [forwarding build requests](@docroot@/advanced-topics/distributed-builds.md) to other Nix instances.
|
Nix can schedule builds such that it automatically builds on other platforms by [forwarding build requests](@docroot@/advanced-topics/distributed-builds.md) to other Nix instances.
|
||||||
|
|
||||||
[`system` configuration option]: @docroot@/command-ref/conf-file.md#conf-system
|
[`system` configuration option]: @docroot@/command-ref/conf-file.md#conf-system
|
||||||
|
[`extra-platforms` configuration option]: @docroot@/command-ref/conf-file.md#conf-extra-platforms
|
||||||
|
|
||||||
[content-addressing derivation]: @docroot@/glossary.md#gloss-content-addressing-derivation
|
[content-addressing derivation]: @docroot@/glossary.md#gloss-content-addressing-derivation
|
||||||
[realise]: @docroot@/glossary.md#gloss-realise
|
[realise]: @docroot@/glossary.md#gloss-realise
|
||||||
|
@ -240,14 +243,14 @@ That works because we've implicitly assumed that all derivations are created *st
|
||||||
But what if derivations could also be created dynamically within Nix?
|
But what if derivations could also be created dynamically within Nix?
|
||||||
In other words, what if derivations could be the outputs of other derivations?
|
In other words, what if derivations could be the outputs of other derivations?
|
||||||
|
|
||||||
:::{.note}
|
> **Note**
|
||||||
In the parlance of "Build Systems à la carte", we are generalizing the Nix store layer to be a "Monadic" instead of "Applicative" build system.
|
>
|
||||||
:::
|
> In the parlance of "Build Systems à la carte", we are generalizing the Nix store layer to be a "Monadic" instead of "Applicative" build system.
|
||||||
|
|
||||||
How should we refer to such derivations?
|
How should we refer to such derivations?
|
||||||
A deriving path works, the same as how we refer to other derivation outputs.
|
A deriving path works, the same as how we refer to other derivation outputs.
|
||||||
But what about a dynamic derivations output?
|
But what about a dynamic derivations output?
|
||||||
(i.e. how do we refer to the output of an output of a derivation?)
|
(i.e. how do we refer to the output of a derivation, which is itself an output of a derivation?)
|
||||||
For that we need to generalize the definition of deriving path, replacing the store path used to refer to the derivation with a nested deriving path:
|
For that we need to generalize the definition of deriving path, replacing the store path used to refer to the derivation with a nested deriving path:
|
||||||
|
|
||||||
```diff
|
```diff
|
||||||
|
@ -295,3 +298,5 @@ The result of this is that it is possible to have a chain of `^<output-name>` at
|
||||||
> |------------------------------------------------------------| |-----|
|
> |------------------------------------------------------------| |-----|
|
||||||
> innermost constant store path (usual encoding) output name
|
> innermost constant store path (usual encoding) output name
|
||||||
> ```
|
> ```
|
||||||
|
|
||||||
|
[Nix instance]: @docroot@/glossary.md#gloss-nix-instance
|
||||||
|
|
|
@ -12,7 +12,7 @@ Given the method, the output's name (computed from the derivation name and outpu
|
||||||
|
|
||||||
## Fixed-output content-addressing {#fixed}
|
## Fixed-output content-addressing {#fixed}
|
||||||
|
|
||||||
In this case the content-address of the *fixed* in advanced by the derivation itself.
|
In this case the content address of the *fixed* in advanced by the derivation itself.
|
||||||
In other words, when the derivation has finished [building](@docroot@/store/building.md), and the provisional output' content-address is computed as part of the process to turn it into a *bona fide* store object, the calculated content address must much that given in the derivation, or the build of that derivation will be deemed a failure.
|
In other words, when the derivation has finished [building](@docroot@/store/building.md), and the provisional output' content-address is computed as part of the process to turn it into a *bona fide* store object, the calculated content address must much that given in the derivation, or the build of that derivation will be deemed a failure.
|
||||||
|
|
||||||
The output spec for an output with a fixed content addresses additionally contains:
|
The output spec for an output with a fixed content addresses additionally contains:
|
||||||
|
@ -159,7 +159,7 @@ A *determinstic* content-addressing derivation should produce outputs with the s
|
||||||
|
|
||||||
### Floating versus Fixed
|
### Floating versus Fixed
|
||||||
|
|
||||||
While the destinction between content- and input-addressing is one of *mechanism*, the distinction between fixed and floating content addression is more one of *policy*.
|
While the distinction between content- and input-addressing is one of *mechanism*, the distinction between fixed and floating content addressing is more one of *policy*.
|
||||||
A fixed output that passes its content address check is just like a floating output.
|
A fixed output that passes its content address check is just like a floating output.
|
||||||
It is only in the potential for that check to fail that they are different.
|
It is only in the potential for that check to fail that they are different.
|
||||||
|
|
||||||
|
|
|
@ -3,7 +3,7 @@
|
||||||
[input addressing]: #input-addressing
|
[input addressing]: #input-addressing
|
||||||
|
|
||||||
"Input addressing" means the address the store object by the *way it was made* rather than *what it is*.
|
"Input addressing" means the address the store object by the *way it was made* rather than *what it is*.
|
||||||
That is to say, an input-addressed output's store path is a function not of the output itself, but the derivation that produced it.
|
That is to say, an input-addressed output's store path is a function not of the output itself, but of the derivation that produced it.
|
||||||
Even if two store paths have the same contents, if they are produced in different ways, and one is input-addressed, then they will have different store paths, and thus guaranteed to not be the same store object.
|
Even if two store paths have the same contents, if they are produced in different ways, and one is input-addressed, then they will have different store paths, and thus guaranteed to not be the same store object.
|
||||||
|
|
||||||
<!---
|
<!---
|
||||||
|
|
|
@ -28,7 +28,7 @@ With all currently-supported store object content-addressing methods, the file s
|
||||||
|
|
||||||
### References
|
### References
|
||||||
|
|
||||||
#### References to other store object#### References to other store objectss
|
#### References to other store objects
|
||||||
|
|
||||||
With all currently supported store object content addressing methods,
|
With all currently supported store object content addressing methods,
|
||||||
other objects are referred to by their regular (string-encoded-) [store paths][Store Path].
|
other objects are referred to by their regular (string-encoded-) [store paths][Store Path].
|
||||||
|
@ -44,28 +44,28 @@ Self-references however cannot be referred to by their path, because we are in t
|
||||||
> which is computationally infeasible.
|
> which is computationally infeasible.
|
||||||
> As far as we know, this is equivalent to finding a hash collision.
|
> As far as we know, this is equivalent to finding a hash collision.
|
||||||
|
|
||||||
Instead we have a "has self reference" boolean, which end up affecting the digest:
|
Instead we have a "has self-reference" boolean, which ends up affecting the digest:
|
||||||
In all currently-supported store object content-addressing methods, when hashing the file system object data, any occurence of store objects own store path in the digested data is replaced with a [sentinal value](https://en.wikipedia.org/wiki/Sentinel_value).
|
In all currently-supported store object content-addressing methods, when hashing the file system object data, any occurence of store object's own store path in the digested data is replaced with a [sentinel value](https://en.wikipedia.org/wiki/Sentinel_value).
|
||||||
The hashes of these modified input streams are used instead.
|
The hashes of these modified input streams are used instead.
|
||||||
|
|
||||||
When validating the content-address of a store object after the fact, the above process works as written.
|
When validating the content address of a store object after the fact, the above process works as written.
|
||||||
However, when first creating the store object we don't know the store object's store path, as explained just above.
|
However, when first creating the store object we don't know the store object's store path, as explained just above.
|
||||||
We therefore, strictly speaking, do not know what value we will be replacing with the sentinental value in the inputs to hash functions.
|
We therefore, strictly speaking, do not know what value we will be replacing with the sentinental value in the inputs to hash functions.
|
||||||
What instead happens is that the provisional store object --- the data from which we wish to create a store object --- is paired with a provisional "scratch" store path (that presumably was choosen when the data was created).
|
What instead happens is that the provisional store object --- the data from which we wish to create a store object --- is paired with a provisional "scratch" store path (that presumably was chosen when the data was created).
|
||||||
That provisional store path is instead what is replaced with the sentinal value, rather than the final store object which we do not yet know.
|
That provisional store path is instead what is replaced with the sentinel value, rather than the final store object which we do not yet know.
|
||||||
|
|
||||||
> **Design note**
|
> **Design note**
|
||||||
>
|
>
|
||||||
> It is an informal property of content-addressed store objects that the choice of provisional store path should not matter.
|
> It is an informal property of content-addressed store objects that the choice of provisional store path should not matter.
|
||||||
> In other words, if a provisional store object is prepared in the same way except for the choice of provision store path, the provisional data need not be identical.
|
> In other words, if a provisional store object is prepared in the same way except for the choice of provision store path, the provisional data need not be identical.
|
||||||
> But, after the sentinal value is substituted in place of each provisional store object's provision store path, the final so-normalized data *should* be identifical.
|
> But, after the sentinel value is substituted in place of each provisional store object's provision store path, the final so-normalized data *should* be identical.
|
||||||
>
|
>
|
||||||
> If, conversely, the data after this normalization process is still different, we'll compute a different content-address.
|
> If, conversely, the data after this normalization process is still different, we'll compute a different content-address.
|
||||||
> The method of preparing the provisional self-referenced data has *failed* to be deterministic in the sense of not *leaking* the choice of provisional store path --- a choice which is supposed to be arbitrary --- into the final store object.
|
> The method of preparing the provisional self-referenced data has *failed* to be deterministic in the sense of not *leaking* the choice of provisional store path --- a choice which is supposed to be arbitrary --- into the final store object.
|
||||||
>
|
>
|
||||||
> This property is informal because at this stage, we are just described store objects, which have no formal notion of their origin.
|
> This property is informal because at this stage, we are just described store objects, which have no formal notion of their origin.
|
||||||
> Without such a formal notion, there is nothing to formally accuse of being insufficiently deterministic.
|
> Without such a formal notion, there is nothing to formally accuse of being insufficiently deterministic.
|
||||||
> Later in this chapter, when we cover [derivations](@docroot@/store/derivation/index.md), we will have a chance to make this a formal property, not of content-addressed store objects themselves, but of derivations that *produce* content-addressed store objects.
|
> Where we cover [derivations](@docroot@/store/derivation/index.md), we will have a chance to make this a formal property, not of content-addressed store objects themselves, but of derivations that *produce* content-addressed store objects.
|
||||||
|
|
||||||
### Name and Store Directory
|
### Name and Store Directory
|
||||||
|
|
||||||
|
@ -88,7 +88,7 @@ References are not supported: store objects with flat hashing *and* references c
|
||||||
|
|
||||||
This also uses the corresponding [Flat](../file-system-object/content-address.md#serial-flat) method of file system object content addressing.
|
This also uses the corresponding [Flat](../file-system-object/content-address.md#serial-flat) method of file system object content addressing.
|
||||||
|
|
||||||
References to other store objects are supported, but self references are not.
|
References to other store objects are supported, but self-references are not.
|
||||||
|
|
||||||
This is the only store-object content-addressing method that is not named identically with a corresponding file system object method.
|
This is the only store-object content-addressing method that is not named identically with a corresponding file system object method.
|
||||||
It is somewhat obscure, mainly used for "drv files"
|
It is somewhat obscure, mainly used for "drv files"
|
||||||
|
@ -99,7 +99,7 @@ Prefer another method if possible.
|
||||||
|
|
||||||
This uses the corresponding [Nix Archive](../file-system-object/content-address.md#serial-nix-archive) method of file system object content addressing.
|
This uses the corresponding [Nix Archive](../file-system-object/content-address.md#serial-nix-archive) method of file system object content addressing.
|
||||||
|
|
||||||
References (to other store objects and self references alike) are supported so long as the hash algorithm is SHA-256, but not (neither kind) otherwise.
|
References (to other store objects and self-references alike) are supported so long as the hash algorithm is SHA-256, but not (neither kind) otherwise.
|
||||||
|
|
||||||
### Git { #method-git }
|
### Git { #method-git }
|
||||||
|
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue