1
0
Fork 0
mirror of https://github.com/NixOS/nix synced 2025-06-28 05:21:16 +02:00
Commit graph

143 commits

Author SHA1 Message Date
Eelco Dolstra
3197c19a31 Add link to tracking issue 2025-01-17 12:34:38 +01:00
Eelco Dolstra
12e14956e2 Warn against the use of relative 'git+file:' flake inputs 2025-01-16 17:23:31 +01:00
Bryan Lai
96bd9bad2f fetchers/git: make path absolute for local repo 2025-01-10 11:37:29 +08:00
Eelco Dolstra
4077aa43a8 ParsedURL: Remove base field 2025-01-07 14:52:00 +01:00
Eelco Dolstra
33852ead6b Optimisation 2024-12-17 12:56:14 +01:00
Eelco Dolstra
7ba933e989 Cache calls to GitRepo::getWorkdirInfo()
A command like `nix flake metadata` was causing about 4 calls to
getWorkdirInfo(), which is slow for large repos (even when they're not
dirty).
2024-12-17 12:56:14 +01:00
Eelco Dolstra
d044a05197 Don't fingerprint dirty repos with submodules for now
Fixes nixpkgsLibTests.
2024-12-17 12:56:14 +01:00
Eelco Dolstra
331bf3e261 Git fetcher: Calculate a fingerprint for dirty workdirs
This restores evaluation caching for dirty Git workdirs.
2024-12-17 12:56:14 +01:00
Sergei Zimmerman
756758d968 chore: get rid of dead code and unused variables where appropriate
Looks like some cruft has been left over from previous refactorings.
This removes dead variables, which should not have side effects in their
constructors. In cases where the variable initialization has a purpose
[[maybe_unused]] is inserted to silence compiler warnings.
2024-11-22 18:05:53 +03:00
Eelco Dolstra
15e5684884 experimental-features.hh: Don't include json-utils.hh
This caused nlohmann/json.hpp to leak into a lot of compilation units,
which is slow (when not using precompiled headers).

Cuts build time from 46m24s to 42m5s (real time with -j24: 2m42s to
2m24s).
2024-10-04 15:59:35 +02:00
Eelco Dolstra
1c2508f176
Merge pull request #11466 from DeterminateSystems/git-mtime-warning
Git fetcher: Don't update mtime of ref file if fetching by rev
2024-09-11 17:34:46 +02:00
Noam Yorav-Raphael
38bfbb297c
Use envvars NIX_CACHE_HOME, NIX_CONFIG_HOME, NIX_DATA_HOME, NIX_STATE_HOME if defined (#11351) 2024-09-11 10:36:46 +00:00
Eelco Dolstra
b80b091bac Git fetcher: Don't update mtime of ref file if fetching by rev
This fixes the warning

  $ nix eval --store /tmp/nix --expr 'builtins.fetchTree { type = "git"; url = "https://github.com/DeterminateSystems/attic"; ref = "fixups-for-magic-nix-cache"; rev = "635753a2069d4b8228e846dc5c09ad361c75cd1a"; }'
  warning: could not update mtime for file '/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache': error: changing modification time of '"/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache"': No such file or directory

When we're fetching by rev, that file doesn't necessarily exist, and we
don't care about it anyway.
2024-09-09 19:52:21 +02:00
John Ericson
3fc77f281e No global settings in libnixfetchers and libnixflake
Progress on #5638

There are still a global fetcher and eval settings, but they are pushed
down into `libnixcmd`, which is a lot less bad a place for this sort of
thing.

Continuing process pioneered in
52bfccf8d8.
2024-07-12 08:50:28 -04:00
Brian McKenna
5be44d235a Guard uses of lutimes, for portability 2024-06-24 17:35:34 -04:00
John Ericson
e0b159549b Misc Windows fixes
1. Fix build by making the legacy SSH Storey's secret `logFD` setting
   not a setting on Windows. (It doesn't make sense to specify `void *`
   handles by integer cross-proccess, I don't think.)

2. Move some files that don't need to be Unix-only anymore back to their
   original locations.
2024-06-01 19:19:35 -04:00
John Ericson
02fa20622f Start factoring out Unix assumptions
This splits files and adds new identifiers in preperation for supporting
windows, but no Windows-specific code is actually added yet.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-02 14:43:38 -04:00
Bouke van der Bijl
60c2d15f5a git fetcher: use resolveRef for getting revision of reference
* Add regression test
* Fix 'no repo' test so it doesn't succeed if the data is still in cache
* Use git_revparse_single inside git-utils instead of reimplementing the same logic.
2024-03-14 14:04:51 +01:00
Eelco Dolstra
0e07f81d2b Fetcher cleanups
* Convert all InputScheme::fetch() methods to getAccessor().

* Add checkLocks() method for checking lock attributes.

* Rename fetch() to fetchToStore().
2024-03-04 22:24:55 +01:00
Eelco Dolstra
071dd2b3a4 Input: Replace 'locked' bool by isLocked() method
It's better to just check whether the input has all the attributes
needed to consider itself locked (e.g. whether a Git input has an
'rev' attribute).

Also, the 'locked' field was actually incorrect for Git inputs: it
would be set to true even for dirty worktrees. As a result, we got
away with using fetchTree() internally even though fetchTree()
requires a locked input in pure mode. In particular, this allowed
'--override-input' to work by accident.

The fix is to pass a set of "overrides" to call-flake.nix for all the
unlocked inputs (i.e. the top-level flake and any --override-inputs).
2024-02-20 16:59:15 +01:00
Eelco Dolstra
9e762454cf Support empty Git repositories / workdirs
Fixes #10039.
2024-02-20 11:40:02 +01:00
John Ericson
4687beecef Get rid of CanonPath::fromCwd
As discussed in the last Nix team meeting (2024-02-95), this method
doesn't belong because `CanonPath` is a virtual/ideal absolute path
format, not used in file systems beyond the native OS format for which a
"current working directory" is defined.

Progress towards #9205
2024-02-08 11:01:41 -05:00
Eelco Dolstra
a6737b7e17 CanonPath, SourcePath: Change operator + to /
This is less confusing and makes it more similar to std::filesystem::path.
2024-02-05 15:17:39 +01:00
DavHau
bc00fa4647 fetchTree/fetchGit: re-enable shallow fetching
Add several tests for git fetching:
- shallow-cache-separation: can fetch the same repo shallowly and non-shallowly
- shallow-ignore-ref: ensure that ref gets ignored when shallow=true is set
- ssh-shallow: can fetch a git repo via ssh using shallow=1
2024-01-19 20:30:47 +07:00
Robert Hensing
692e9197bc fetchTree: Disallow combination of submodules and exportIgnore for now 2024-01-12 15:31:53 +01:00
Robert Hensing
99bd12f0b1 fetchGit/fetchTree: Improve exportIgnore, submodule interaction
Also fingerprint and some preparatory improvements.

Testing is still not up to scratch because lots of logic is duplicated
between the workdir and commit cases.
2024-01-12 15:31:53 +01:00
Robert Hensing
7774eff10e libfetchers/git: Move workdir accessor into GitRepo::getAccessor 2024-01-12 15:31:53 +01:00
Robert Hensing
ce6d58a97c git fetcher: Add exportIgnore parameter
Enabled for fetchGit, which historically had this behavior,
among other behaviors we do not want in fetchGit.

fetchTree disables this parameter by default. It can choose the
simpler behavior, as it is still experimental.

I am not confident that the filtering implementation is future
proof. It should reuse a source filtering wrapper, which I believe
Eelco has already written, but not merged yet.
2024-01-12 15:31:52 +01:00
Eelco Dolstra
359990dfdc
Merge pull request #9324 from 9999years/fix-8854-take-2
Don't attempt to `git add` ignored files
2024-01-05 14:57:32 +01:00
John Ericson
dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
Robert Hensing
d4f6b1d38b
Merge pull request #9497 from edolstra/move-access-control
Move restricted/pure-eval access control out of the evaluator and into the accessor
2023-12-08 22:21:50 +01:00
Peter Kolloch
fc6f29053a Renamed HashFormat::Base32 to HashFormat::Nix32
...and also adjusted parsing accordingly.

Also added CLI completion for HashFormats.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch
5334c9c792 HashType: Rename to HashAlgorithm
To be consistent with CLI, nix API
and many other references.

As part of this, we also converted it to a scoped enum.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Eelco Dolstra
8cafc754d8 Move access control from FSInputAccessor to FilteringInputAccessor 2023-11-30 21:54:53 +01:00
Rebecca Turner
be30c2ea8d
Don't attempt to git add ignored files
This uses `git check-ignore` to determine if files are ignored before
attempting to add them in `putFile`.

We also add a condition to the `fetchFromWorkdir` filter to always add
the `flake.lock` file, even if it's not tracked. This is necessary to
resolve inputs.

This fixes #8854 without `git add --force`.
2023-11-30 10:26:13 -08:00
Théophane Hufschmitt
6a94755b12 Allow user input in git commit
We occasionnally commit to git repositories (like with `nix flake update --commit-lock-file`).
This shells out to `git commit`, which might wait for user input (for a signing key passphrase for instance).
Disable the progress bar while this is running to make sure that the
user can enter it.
2023-11-24 11:45:37 +01:00
Eelco Dolstra
38844943d0
Merge pull request #9413 from edolstra/fetchToStore-cache
Persistently cache `InputAccessor::fetchToStore()`
2023-11-22 12:19:25 +01:00
Robert Hensing
9cd69e1c39
Merge pull request #9411 from edolstra/path-display
Improve SourceAccessor path display
2023-11-21 18:12:01 +01:00
Eelco Dolstra
99d5204baa Persistently cache InputAccessor::fetchToStore()
This avoids repeated copying of the same source tree between Nix
invocations. It requires the accessor to have a "fingerprint" (e.g. a
Git revision) that uniquely determines its contents.
2023-11-20 20:04:37 +01:00
Eelco Dolstra
a0162d5732 Improve SourceAccessor path display
Backported from lazy-trees. This allows SourceAccessors to show the
origin of the accessor. E.g. we now get

  copying '«git+https://github.com/blender/blender.git?ref=refs/heads/main&rev=4edc1389337dd3679ff66969c332d2aff52e1992»/' to the store

instead of

  copying '/' to the store
2023-11-20 18:54:36 +01:00
Eelco Dolstra
e4066c0444 Fetch specific Git revisions
This is more efficient, and necessary when using shallow=1 with a rev.
2023-11-20 17:25:40 +01:00
Eelco Dolstra
7ab91e7238 Implement shallow fetching 2023-11-15 15:00:17 +01:00
Eelco Dolstra
5dd4ae8687 Remove unused cacheType field 2023-11-15 14:08:34 +01:00
Eelco Dolstra
2890999911 Show Git fetch progress 2023-11-15 13:57:20 +01:00
Eelco Dolstra
2964a9f562 Fix relative submodule handling
Tested on

  nix flake prefetch 'git+https://github.com/blender/blender.git?rev=4ed8a360e956daf2591add4d3c9ec0719e2628fe&submodules=1'
2023-11-14 16:00:21 +01:00
Eelco Dolstra
6ec6b8aa36 Improve git submodule error reporting 2023-11-14 15:52:18 +01:00
Eelco Dolstra
7f576f5dfe Rename UnionInputAccessor to MountedInputAccessor 2023-11-14 14:02:57 +01:00
Eelco Dolstra
38b07d6347
src/libfetchers/git.cc: Apply suggestion
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-14 13:38:03 +01:00
Eelco Dolstra
98a120b8b8 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-09 16:48:41 +01:00
John Ericson
ac89bb064a Split up util.{hh,cc}
All OS and IO operations should be moved out, leaving only some misc
portable pure functions.

This is useful to avoid copious CPP when doing things like Windows and
Emscripten ports.

Newly exposed functions to break cycles:

 - `restoreSignals`
 - `updateWindowSize`
2023-11-05 12:20:02 -05:00