wroclaw-git-backups/nix
1
0
Fork 0
mirror of https://github.com/NixOS/nix synced 2025-07-16 02:01:59 +02:00
Code Activity

Harden tests' bash

Browse source 
Use `set -u` and `set -o pipefail` to catch accidental mistakes and
failures more strongly.

 - `set -u` catches the use of undefined variables
 - `set -o pipefail` catches failures (like `set -e`) earlier in the
   pipeline.

This makes the tests a bit more robust. It is nice to read code not
worrying about these spurious success paths (via uncaught) errors
undermining the tests. Indeed, I caught some bugs doing this.

There are a few tests where we run a command that should fail, and then
search its output to make sure the failure message is one that we
expect. Before, since the `grep` was the last command in the pipeline
the exit code of those failing programs was silently ignored. Now with
`set -o pipefail` it won't be, and we have to do something so the
expected failure doesn't accidentally fail the test.

To do that we use `expect` and a new `expectStderr` to check for the
exact failing exit code. See the comments on each for why.

`grep -q` is replaced with `grepQuiet`, see the comments on that
function for why.

`grep -v` when we just want the exit code is replaced with `grepInverse,
see the comments on that function for why.

`grep -q -v` together is, surprise surprise, replaced with
`grepQuietInverse`, which is both combined.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
This commit is contained in:
John Ericson 2021-12-09 15:26:46 +00:00
parent 0159dfad3f
commit c11836126b
51 changed files with 300 additions and 177 deletions
Download patch file Download diff file Expand all files Collapse all files

12
tests/repl.sh
View file

@ -33,14 +33,14 @@ testRepl () {
nix repl "${nixArgs[@]}" <<< "$replCmds" || fail "nix repl does not work twice with the same inputs"
# simple.nix prints a PATH during build
echo "$replOutput" | grep -qs 'PATH=' || fail "nix repl :log doesn't output logs"
echo "$replOutput" | grepQuiet -s 'PATH=' || fail "nix repl :log doesn't output logs"
local replOutput="$(nix repl "${nixArgs[@]}" <<< "$replFailingCmds" 2>&1)"
echo "$replOutput"
echo "$replOutput" | grep -qs 'This should fail' \
echo "$replOutput" | grepQuiet -s 'This should fail' \
|| fail "nix repl :log doesn't output logs for a failed derivation"
local replOutput="$(nix repl --show-trace "${nixArgs[@]}" <<< "$replUndefinedVariable" 2>&1)"
echo "$replOutput"
echo "$replOutput" | grep -qs "while evaluating the file" \
echo "$replOutput" | grepQuiet -s "while evaluating the file" \
|| fail "nix repl --show-trace doesn't show the trace"
nix repl "${nixArgs[@]}" --option pure-eval true 2>&1 <<< "builtins.currentSystem" \
@ -58,7 +58,7 @@ testReplResponse () {
local commands="$1"; shift
local expectedResponse="$1"; shift
local response="$(nix repl "$@" <<< "$commands")"
echo "$response" | grep -qs "$expectedResponse" \
echo "$response" | grepQuiet -s "$expectedResponse" \
|| fail "repl command set:
$commands
@ -121,5 +121,5 @@ sed -i 's/beforeChange/afterChange/' flake/flake.nix
echo ":reload"
echo "changingThing"
) | nix repl ./flake --experimental-features 'flakes repl-flake')
echo "$replResult" | grep -qs beforeChange
echo "$replResult" | grep -qs afterChange
echo "$replResult" | grepQuiet -s beforeChange
echo "$replResult" | grepQuiet -s afterChange