The Nine Circles of Dependency Hell (and a roadmap out)

Your project has been overwhelmed by the complex web of its software dependencies to the point of stoppage. We spend more time fixing these dependency issues than writing code most of the time. Every developer has been there.

Welcome to the Nine Circles of Dependency Hell; I’ll be your Virgil.

The First Circle: Limbo. Are my dependencies even correct?

Each circle represents a more evil transgression of package management. In the first circle are those who committed updated packages without recording them.

Most language package managers suggest that you don’t check in a node_modules or vendor folder anymore. But there can still be inconsistencies between the packages in use and a package manifest like packages.json or go.mod—a developer uses a new dependency without explicitly adding it, or removes one without removing it from the manifest.

Make sure you’re running a check against this in your presubmit. Before the pull request is merged is also a great time to vet new dependencies—for licenses or security issues. For example, the bouk/monkey package on GitHub has a license that explicitly forbids anyone from using it!

The Second Circle: Lust. You will stop at nothing to depend on that new package.

Here wander those who will chase any dependency without vetting it. It might not have documentation, it might not have been updated in years, but for some reason, it calls your name to call its functions. We all want that new function, that latest version of the library.

Package lust is not without punishment. When the author of the left-pad JavaScript library decided to remove the package from npm, thousands of projects broke, including large ones like Node and Babel. What if the author had uploaded malicious code instead? Sometimes a little duplication is better than a bit of dependency.

The Third Circle: Gluttony. Bloated bundles. Too many dependencies.

One more dependency won’t hurt until it does. Slow builds and massive repositories are symptoms of too many dependencies.

Instead of eating more dependencies, try to snack on the low-hanging fruit of removing unused dependencies (First Circle) as a good starting point. Then, use static code analysis tools to determine where your most significant dependencies come from and determine whether or not you can slim them down or remove them. One example is to only import what you need from the code (e.g., no star (*) imports)). Importing only what you need might not always reduce your bundle size, but will almost certainly clear up your global namespace so that you avoid potential conflicts.

The Fourth Circle: Greed. Multiple package managers.

Your data scientist loves to use anaconda, so now there’s a conda configuration file checked in alongside the pip requirements.txt. Two’s company.

The only option here is to have a single source of truth per language. Of course, you’ll most likely need multiple package managers along the way: OS-level package management like apt, pacman, and different package managers for other languages. How do you manage all these layers of packages? Tools like Docker can help capture all of these in one place, all in a mostly reproducible manner.

The Fifth Circle: Wrath. The package or version you need isn’t in your package manager.

Now that you think about it, you’re using Ubuntu Trusty Tahr. Didn’t it reach end of life in 2019 (it resides in a different place in Dependency Hell)? So, where are the package owner gremlins that are supposed to add it?

There’s a simple way for most language-level package managers to reference a specific Git repository and commit.

For npm, you can simply do

shell npm install git+https://github.com/organization/repository.git#commit

For Go modules, you can do

go go get github.com/organization/repository@commit

If you can, point to a tag. For operating system packages, you can be a good open source citizen for operating system packages and contribute the package to your package manager of choice. If that’s untenable, many projects simply copy the code into a special folder (at Google, it’s usually called third_party) along with the licenses. Make sure you have a way of capturing the versioning information—what commit was the code pulled? Where does the upstream code live?

The Sixth Circle: Heresy. Monkey patching a dependency.

Why won’t this open source project take my specific and untested patch? Guess I’ll just monkey patch it.

Usually, monkey patching is a terrible idea. First, it makes code difficult to upgrade. Second, patches are difficult for others to discover (how do you know what’s been changed?). If you really must do this, using the third_party folder is one potential escape hatch. Just make sure you don’t use the bouk/monkey library from the First Circle!

The Seventh Circle: Violence. Breaking changes on a minor or patch version.

In theory, it should follow major.minor.patch. In practice, the developer used a pseudo-random number generator. Version is open to interpretation—a popular Python package, html5lib, previously versioned its packages asymptotically: 0.99, 0.999, 0.9999, and so on.