maralorn 0e327a78da
All checks were successful
Nix build / nix-build (default) (push) Successful in 9s
Nix build / nix-flake-check (push) Successful in 31s
2023-11-26 15:44:25 +01:00

151 lines
7 KiB
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# nix-output-monitor
Pipe your nix-build output through the nix-output-monitor (aka nom) to get additional information while building.
While your build runs, nom will draw something like this at the bottom of your build log:
*(note that to reduce clutter nom only shows timers over 1s build or download time.)*
[![Packaging status](](
## Status
This was an experimental fun project, which proved to be useful to quite a lot of people.
The purpose of it is to write something fun and useful in Haskell.
You are free and very welcome to contribute feedback, issues or PRs.
I do not commit to maintain this project over a long time period but it doesnt look like I am stopping anytime soon.
By now, nom is quite fully featured with support for nix v1 commands (e.g. `nix-build`) and nix v2 command (e.g. `nix build`).
Issues and pull requests are welcome at
Starting from version 2.1.0, nom uses [SemVer](https://semver).
The versioning applies to the behavior of the executable.
There are no stability guarantees for the library component in the cabal project.
## Support
If your question is not answered in this README you can ask it in []( on matrix or open an issue on github.
## Installing
* nixpkgs: nom is in nixpkgs. Just install `pkgs.nix-output-monitor` in the usual way. You might want to install it from nixos-unstable to get the newest version.
* cabal: Install `cabal-install` and run `cabal install` in the checked out repo.
* nix: or use `nix build` or `nix-env` or include the flake output of this repo in your nixos config.
## Running
### The Easy Way
**Warning:** The displayed build tree might be incomplete with new-style commands like `nix build` for nix versions <2.10.
The `nom` binary (starting from version 2.0) behaves as a `nix` drop in, with much more colorful output, but **only** for the following commands:
`nom build <args>`: Behaves like `nix build <args>`.
`nom shell <args>`: Behaves like `nix shell <args>`.
`nom develop <args>`: Behaves like `nix develop <args>`.
The latter two commands work by calling `nix shell` or `nix develop` twice, the first time with overridden `--run exit` and monitoring the output, the second time passing output through to the user. This will incur a performance cost by doubling eval time.
Furthermore when called via the corresponding provided symlinks, nom is also a drop-in for the following commands:
`nom-build <args>`: Behaves like `nix-build <args>`.
`nom-shell <args>`: Behaves like `nix-shell <args>`.
All aliases internally use the json-based approach (see next section) and propagate error codes.
If you want nom support for other nix commands please open an issue.
### The Flexible Way
#### JSON based
nix-build --log-format internal-json -v |& nom --json
**Warning:** Dont forget to redirect stderr. That's what the `&`, does.
#### Human readable log parsing
It his highly recommended to always append `--log-format internal-json -v` (or use the above mentioned aliases.) and call `nom` with `--json`. That will give you much more informative output.
If you are in a situation, where you cant use the json based nix output you can still use
nix-build |& nom
**Warning:** Dont forget to redirect stderr. That's what the `&`, does.
This has the advantage to also work with other commands like `nixos-rebuild` or `home-manager`, where it is not trivial to pass in the `--log-format internal-json -v` flag. nom will pass everything it reads through, if it does not understand it. This makes it ideal to attach it to scripts which output more then just `nix` output.
### Preserving Colored Text
Colored text will work as expected in json-mode.
In human-readable log mode you can preserve the color of the redirected text by using the `unbuffer` command from the `expect` package.
unbuffer nix-build |& nom
## Explanation
### Legend
Nom tries to convey information via symbols and colors
* `⏵`, yellow: running builds
* `✔`, green: completed builds
* `⏳︎︎︎`, blue: planned builds
* `⚠`, red: failed builds
* `↓ ⏵`, yellow: running downloads
* `↑ ⏵`, yellow: running uploads
* `↓ ✔`, green: completed downloads
* `↑ ✔`, green: completed uploads
* `↓ ⏳︎︎︎`, blue: waiting downloads
* `∅`: a moving average over past builds of this derivation
* `⏱︎`: running time
* `∑`: a summary over all packages and hosts
If you cant see all icons you maybe need another terminal font.
I recommend any font from `pkgs.nerdfonts` e.g. `"JetBrainsMono Nerd Font"`.
Also different terminals might work differently well. I recommend: `pkgs.foot`.
### How to Read the Dependency Graph
* Every entry in the nom tree stands for one derivation.
* Children of a node are direct dependencies.
* nom will try to show you the most relevant part of the dependency tree, roughly aiming to fill a third of your terminal
* No build will be printed twice in the tree, it will only be shown for the lower most dependency.
* nom will do its best to print all running or failed builds, downloads and uploads, but it does not print every direct child of a node.
* Use the colors from above to read the summary
## Example Runs
An example remote build:
An example with a lot of downloads:
## Implementation
Right now nom uses four sources of information:
1. The parsed nix-build output (json or human-readable)
2. it checks if build results exist in the nix-store (only in human-readable mode)
3. it queries `.drv` files for information about the `out` output path.
4. It caches build times in `$XDG_CACHE_HOME/nix-output-monitor/build-reports.csv`.
## Limitations
* This will fail in unexpected and expected ways.
* Luckily I dont think this program screws up anything more than your terminal.
* remote builds will sometimes be shown as running even when they are actually still waiting for uploads or downloads. This is how nix reports it.
* Terminal clearing and reprinting is brittle. It might fail with your terminal or terminal width. But at this point Ive invested some effort to make it usable.
* This program also makes assumptions like your nix-store is at "/nix/store".
### For human-readable log parsing mode:
* nix-output-monitor receives most it's information from parsing nix-build output. The parser might be to strict or to loose for use cases I didnt think of. Then **the numbers displayed will be off**!
* nix-build does not show info when a download or upload is finished, so we currently cannot differentiate between started and completed downloads.
* For completed build detection we assume that every derivation has an output called "out".