Announcing gfold 3

Tuesday, January 4, 2022

Today marks the release of gfold 3 🎉!

It’s a bit surreal, given that this project started in a private repository from my early days of learning Rust.

I began to learn Rust in the middle of 2019, for many of the reasons popular among newcomers: safe concurrency, compile-time reliability, great community, etc. Leading into 2020, my Rust code steadily improved, I started iterating on public projects (as well as many more private projects), and gfold’s codebase aged quickly.

All of my other Rust projects were better designed, better written, and more interesting in a technical sense than gfold. Yet, I continued to use it more than any of those projects. Despite the state of the codebase, my favorite part about gfold was that I was using it daily, and multiple times throughout the day. Thus, I decided to open source what I had in early April 2020.

Many of you decided to try it out since then! At least, it seems as such because, as of version 3.0.0-rc.4, the project has seen the following:

• 📦 ~1,600 downloads from crates.io
• 📚 rank #21 on lib.rs for command-line-utilities
• ⭐️ 90 stars on GitHub

For a project that was not originally intended to live past my earliest days of learning Rust, that’s pretty great! I’m thankful for all users of gfold, and I am excited for you to try out version 3.0.0.

Design

gfold 3.0.0 is centered around a domain-driven refactor. Its focus areas include the following:

• Targets: focus on generating targets with a given config
• Reports: generate reports given generated targets
• Displays: display results for generated reports

These focus areas contain separate entities that try to be self-contained. However, gfold is a small, “single-execution” binary, so these focus areas may intersect with one another in practice. For example, it might be optimal to combine and parallelize everything:

    ┌──────────────────────────┐
    │ perform a directory walk │
┌───► with each "dive" opened  │
│   │ as a (green) thread      │
│   └─────────────┬────────────┘
│                 │
│   ┌─────────────▼─────────────────┐
│   │    determine if the           │
│   │    child path is a target     │
│   └────────┬───────────────────┬──┘
│            │                   │
│          false               true
│            │                   │
│            │  ┌────────────────▼──┐
│            │  │ generate a report │
│            │  │ with that path    │
│            │  └────────────────┬──┘
│            │                   │
│   ┌────────▼───────────────┐   │
│   │ create another (green) │   │
└───┤ thread with that path  │   │
    └────────────────────────┘   │
                                 │
               ┌─────────────────▼─────┐
               │ display that report   │
               │ immediately to STDOUT │
               └───────────────────────┘

That could result in incredibly fast execution, but you would run into a number of other problems:

• Do we have to worry about a maximum number of threads?
• What if certain display settings need to be context-aware and pad with whitespaces according to other reports?
• Should we parallelize report generation, and keep directory walking sequential, if the latter is much less expensive than executing several libgit2 or git subcommand calls?

All of these questions were similar to those that arose during gfold 3.0.0 development.

The final architecture diagram contained more discrete parts; making each focus area a little more extendable. While they might not be co-joined, trading minor performance hits for extensibility was well worth it in the final design. Overall, the goal was to strike a balance between being a fast, glorified script runner and being a slower, complex application.

New Features

Aside from the design philosophy change and overall refactor, there are a number of new features. They include, but are not limited to, the following:

• Ability to store options in an optional config file
  • Ability to ignore the config file and only use CLI options
  • Ability to merge CLI options with config file options (default behavior if config file exists)
• Brand new display mode
  • This is the new default display mode
  • The old display mode is available via a flag and can be used by default if set in the config file
  • Primary motivation for the display mode change was to minimize line length (classic display mode can result in very long lines)
  • Repositories are not grouped by parent directory, unlike the classic display mode
• Ability to print merged config options (CLI options + config file options)
  • Since the config is printed as valid JSON, you can direct the output in your config file (e.g. gfold --classic ~/src --print > gfold.json)
• Ability to specify path to git
  • By default, gfold tries to find git in your PATH, but you provide another git binary by its path
  • Since version 2.0.0, libgit2 is not used in order to reduce binary size, minimize divergent behavior, reduce attack surface, and remove a redundant dependency (i.e. you likely have git installed if you are using gfold)
• Ability to store default path target in config file
  • By default, the the target is (and has been) the current working directory
  • If you’d like to target the same directory from anywhere, you can provide it in the config file

There have been numerous performance improvements, a new CLI parsing structure, and other smaller changes, but the above features encompass the end user goals for this release.

Closing

Thank you for reading and enjoy gfold 3!

A special thank you to those who have helped along the way:

• @jrcichra for adding multi-OS support to the very early CI pipeline
• @orhun for adding gfold to the AUR, which caused a significant increase in users
• @wezm for including gfold in the “One Hundred Rust Binaries” series, which caused a significant increase in users
• @yaahc for helping with a very early refactor through Awesome Rust Mentors