Pipe your nix-build output through the nix-output-monitor a.k.a nom to get additional information while building.
Go to file
maralorn 06f4728ed2
Some checks failed
Nix build / nix-build (default) (push) Successful in 9s
Nix build / nix-flake-check (push) Successful in 31s
Nix build / cabal-test (push) Failing after 3m5s
Add cabal-test
2023-11-27 10:25:22 +01:00
.forgejo/workflows Add cabal-test 2023-11-27 10:25:22 +01:00
bench Autopopulate before benchmark 2023-11-26 14:27:45 +01:00
completions Add nom-build with zsh completion 2022-03-16 13:52:47 +01:00
exe Use CPS Writer Monad 2023-11-22 22:05:31 +01:00
lib Remove redundant imports 2023-11-26 14:08:08 +01:00
test Improve test output 2023-11-23 00:04:47 +01:00
.envrc Add .envrc 2022-03-08 02:06:21 +01:00
.gitignore git ignore recorded profiles 2022-09-29 13:35:23 +02:00
.hlint.yaml Revert "Lookup storePath producers and better activity reporting" 2023-11-26 13:42:07 +01:00
cabal.project.local tweak configure script 2022-10-18 02:41:55 +02:00
CHANGELOG.md Release 2.1.1 2023-11-27 10:18:16 +01:00
default.nix Release 2.1.1 2023-11-27 10:18:16 +01:00
example-screenshot.png Update screenshot 2022-10-22 01:28:50 +02:00
flake.lock flake.lock: Update 2023-11-08 22:36:01 +01:00
flake.nix Only use -Werror in local package 2023-11-27 10:17:29 +01:00
fourmolu.yaml Strictify 2022-11-21 00:19:58 +01:00
hie.yaml Check in hie.yaml 2022-03-05 20:21:48 +01:00
LICENSE Initial commit 2020-10-03 16:06:03 +02:00
nix-output-monitor.cabal Release 2.1.1 2023-11-27 10:18:16 +01:00
README.md Update README 2023-11-26 15:44:25 +01:00
weeder.dhall Weeder 2022-11-20 22:08:51 +01:00

nix-output-monitor

Pipe your nix-build output through the nix-output-monitor (aka nom) to get additional information while building.

While your build runs, nom will draw something like this at the bottom of your build log:

(note that to reduce clutter nom only shows timers over 1s build or download time.)

Packaging status

Status

This was an experimental fun project, which proved to be useful to quite a lot of people. The purpose of it is to write something fun and useful in Haskell. You are free and very welcome to contribute feedback, issues or PRs. I do not commit to maintain this project over a long time period but it doesnt look like I am stopping anytime soon.

By now, nom is quite fully featured with support for nix v1 commands (e.g. nix-build) and nix v2 command (e.g. nix build).

Issues and pull requests are welcome at https://github.com/maralorn/nix-output-monitor.

Starting from version 2.1.0, nom uses SemVer. The versioning applies to the behavior of the executable. There are no stability guarantees for the library component in the cabal project.

Support

If your question is not answered in this README you can ask it in #nix-output-monitor:maralorn.de on matrix or open an issue on github.

Installing

  • nixpkgs: nom is in nixpkgs. Just install pkgs.nix-output-monitor in the usual way. You might want to install it from nixos-unstable to get the newest version.
  • cabal: Install cabal-install and run cabal install in the checked out repo.
  • nix: or use nix build or nix-env or include the flake output of this repo in your nixos config.

Running

The Easy Way

Warning: The displayed build tree might be incomplete with new-style commands like nix build for nix versions <2.10.

The nom binary (starting from version 2.0) behaves as a nix drop in, with much more colorful output, but only for the following commands:

nom build <args>: Behaves like nix build <args>.
nom shell <args>: Behaves like nix shell <args>.
nom develop <args>: Behaves like nix develop <args>.

The latter two commands work by calling nix shell or nix develop twice, the first time with overridden --run exit and monitoring the output, the second time passing output through to the user. This will incur a performance cost by doubling eval time.

Furthermore when called via the corresponding provided symlinks, nom is also a drop-in for the following commands:
nom-build <args>: Behaves like nix-build <args>.
nom-shell <args>: Behaves like nix-shell <args>.

All aliases internally use the json-based approach (see next section) and propagate error codes. If you want nom support for other nix commands please open an issue.

The Flexible Way

JSON based

nix-build --log-format internal-json -v |& nom --json

Warning: Dont forget to redirect stderr. That's what the &, does.

Human readable log parsing

It his highly recommended to always append --log-format internal-json -v (or use the above mentioned aliases.) and call nom with --json. That will give you much more informative output.

If you are in a situation, where you cant use the json based nix output you can still use

nix-build |& nom

Warning: Dont forget to redirect stderr. That's what the &, does.

This has the advantage to also work with other commands like nixos-rebuild or home-manager, where it is not trivial to pass in the --log-format internal-json -v flag. nom will pass everything it reads through, if it does not understand it. This makes it ideal to attach it to scripts which output more then just nix output.

Preserving Colored Text

Colored text will work as expected in json-mode.

In human-readable log mode you can preserve the color of the redirected text by using the unbuffer command from the expect package.

unbuffer nix-build |& nom

Explanation

Legend

Nom tries to convey information via symbols and colors

  • , yellow: running builds
  • , green: completed builds
  • ⏳︎︎︎, blue: planned builds
  • , red: failed builds
  • ↓ ⏵, yellow: running downloads
  • ↑ ⏵, yellow: running uploads
  • ↓ ✔, green: completed downloads
  • ↑ ✔, green: completed uploads
  • ↓ ⏳︎︎︎, blue: waiting downloads
  • : a moving average over past builds of this derivation
  • ⏱︎: running time
  • : a summary over all packages and hosts

If you cant see all icons you maybe need another terminal font. I recommend any font from pkgs.nerdfonts e.g. "JetBrainsMono Nerd Font". Also different terminals might work differently well. I recommend: pkgs.foot.

How to Read the Dependency Graph

  • Every entry in the nom tree stands for one derivation.
  • Children of a node are direct dependencies.
  • nom will try to show you the most relevant part of the dependency tree, roughly aiming to fill a third of your terminal
  • No build will be printed twice in the tree, it will only be shown for the lower most dependency.
  • nom will do its best to print all running or failed builds, downloads and uploads, but it does not print every direct child of a node.
  • Use the colors from above to read the summary

Example Runs

An example remote build: asciicast

An example with a lot of downloads: asciicast

Implementation

Right now nom uses four sources of information:

  1. The parsed nix-build output (json or human-readable)
  2. it checks if build results exist in the nix-store (only in human-readable mode)
  3. it queries .drv files for information about the out output path.
  4. It caches build times in $XDG_CACHE_HOME/nix-output-monitor/build-reports.csv.

Limitations

  • This will fail in unexpected and expected ways.
  • Luckily I dont think this program screws up anything more than your terminal.
  • remote builds will sometimes be shown as running even when they are actually still waiting for uploads or downloads. This is how nix reports it.
  • Terminal clearing and reprinting is brittle. It might fail with your terminal or terminal width. But at this point Ive invested some effort to make it usable.
  • This program also makes assumptions like your nix-store is at "/nix/store".

For human-readable log parsing mode:

  • nix-output-monitor receives most it's information from parsing nix-build output. The parser might be to strict or to loose for use cases I didnt think of. Then the numbers displayed will be off!
  • nix-build does not show info when a download or upload is finished, so we currently cannot differentiate between started and completed downloads.
  • For completed build detection we assume that every derivation has an output called "out".