Dependencies in bpt

In bpt a project or library can declare dependencies on libraries in external packages. A dependency consists of an external package name, a range of acceptable versions, and some set of libraries to use from that external package.

Dependencies can be set using the dependencies property on either the top-level of bpt.yaml, or as a property attached to an invidual library within the project. The similar test-dependencies key specifies dependencies that are required to build and execute the project’s tests.

These dependency properties should be an array of dependency specifiers.

Dependency Specifiers

The dependencies and test-dependencies arrays in bpt.yaml are used to declare dependencies on libraries in external packages:

dependencies:
  - dep-name@1.2.3

The above specifier requests a package named “dep-name” with a version range of greater-or-equal-to-1.2.3 and less-than 2.0.0, and uses the default library from that project (The library with the same name as the package itself).

If a package provides more than one library, or does not use a default library, the dependency specifier can include a using clause to name one or more comma-separated library names to use from the package:

dependencies:
  - acme@1.2.3 using widgets, gadgets

If the using clause is omitted an implicit “using [pkg-name]” is assumed, with [pkg-name] being the name of the depended-on package.

More formally: A dependency specifier (dep-spec) is a string of the following format:

dep-spec    ::=  dep_name dep_range ["using" lib_name ("," lib_name)+]
dep_name    ::=  name
lib_name    ::=  name
dep_range   ::=  range_sym dep_version
range_sym   ::=  "@" | "^" | "~" | "=" | "+"
dep_version ::=  version

alternatively:

<dep_name>{@,^,~,=,+}<version> [using <lib_name> [, <lib_name> [...]]]

Compatible Range Specifiers

When specifying a dependency on a package, one will want to specify which versions of the dependency are supported.

Note

Unlike other packaging tools, bpt will find a solution with the lowest possible version that satisfies the given requirements for each package. This decision is not incidental: It’s entirely intentional. Refer to: Why Pull the Lowest Matching Version?.

bpt compatible-version ranges use similar syntax to other tools. There are four version range kinds available, listed in order of most-to-least restrictive:

Exact: =1.2.3

Specifies an exact requirement. The dependency must match the named version exactly or it is considered incompatible.

Minor: ~1.2.3

Specifies a minor requirement. The version of the dependency should be at least the given version, but not as new or newer than the next minor revision. In this example, it represents the half-open version range [1.2.3, 1.3.0).

Major: ^1.2.3 or @1.2.3

Specifies a major requirement. The version must be at least the same given version, but not any newer than the the next major version. In the example, this is the half-open range [1.2.3, 2.0.0).

Note

This is the recommended default option to reach for, as it matches the intended behavior of Semantic Versioning.

At-least: +1.2.3

Specifies an at least requirement. The version must be at least the given version, but any newer version is acceptable.

Why Pull the Lowest Matching Version?

When resolving dependencies, bpt will pull the version of the dependency that is the lowest version that satisfies the given range. In most cases, this will be the same version that is the base of the version range.

Imagine a scenario where we did select the “latest-matching-version”:

Suppose we are developing a library Gadgets, and we wish to make use of Widgets. The latest version is 1.5.2, and they promise Semantic Versioning compatibility, so we select a dependency statement of Widgets^1.5.2.

Suppose a month passes, and Widgets@1.6.0 is published. A few things happen:

  1. Our CI builds now switch from 1.5.2 to 1.6.0 without any code changes. Should be okay, right? I mean… it’s still compatible, yeah?

  2. Bugs in Widgets@1.6.0 will now appear in all CI builds, and won’t be reproducible locally unless we re-pull our dependencies and obtain the new version of Widgets. This requires that we be conscientious enough to realize what is actually going on.

  3. Even if Widgets@1.6.0 introduces no new bugs, a developer re-pulling their dependencies will suddenly be developing against 1.6.0, and may not even realize it. In fact, this may continue for weeks or months until everyone is developing against 1.6.0 without realizing that they actually only require 1.5.2 in their dependency declarations.

  4. Code in our project is written that presupposes features or bugfixes added in 1.6.0, and thus makes the dependency declaration on Widgets^1.5.2 a lie.

Pulling the lowest-matching-version has two huge benefits:

  1. No automatic CI upgrades. The code built today will produce the same result when built a year from now.

  2. Using a feature/fix beyond our minimum requirement becomes a compile error, and we catch these up-front rather than waiting for a downstream user discovering them for us.

Isn’t this what lockfiles are for?

Somewhat. Lockfiles will prevent automatic upgrades, but they will do nothing to stop accidental reliance on new versions. There are other useful features of lockfiles, but preventing automatic upgrades can be a non-issue by simply using lowest-matching-version.

So, if this is the case, why use ranges at all?

In short: Your compatibility ranges are not for you. They are for your users.

Suppose package A requires B^1.0.0, and B requires C^1.2.0. Now let us suppose that A wishes to use a newer feature of C, and thus declares a dependency on C^1.3.0. B and A have different compatibility ranges on C, but this will work perfectly fine as long as the compatible version ranges of A and B have some overlap.

That final qualification is the reason we use compatibility ranges: To support our downstream users to form dependency graphs that would otherwise form conflicts if we required exact versions for everything. In the above example, C@1.3.0 will be selected for the build of A.

Now, if another downstream user wants to use A, they will get C@1.3.0. But they discover that they actually need a bugfix in C, so they place their own requirement on C ^1.3.1. Thus, they get C@1.3.1, which still satisfies the compatibility ranges of A and B. Everyone gets along just fine!

Dependency Compatibility and the using Specifier

Besides requiring that a candidate for dependency resolution meet the version requirements, the candidate must also provide all of the libraries named by the using specifier on the dependency statement.

Note

If you omit the using specifier, it is equivalent to using a library with the same name as the package. (This would be the default library of the package.)

A Simple Example

For example, suppose that the following packages are available:

  • acme-libs@1.2.0 - Provides one library: widgets.

  • acme-libs@1.3.0 - Provides libraries widgets and gadgets.

  • acme-libs@1.4.0 - Provides libraries gadgets, and gizmos

Suppose now I have a project bpt.yaml:

name: my-code
version: 4.2.4

dependencies:
  - acme-libs@1.0.0 using gadgets, widgets

Our package contains a single dependency statement \(R_1\) of acme-libs@1.0.0 using gadgets, widgets. When bpt does dependency resolution, it sees the requirement on acme-libs and seeks out a compatilbe version. Since bpt prefers to find the lowest-matching-version, it begins by considering acme-libs@1.2.0. Good news: This matches the version requirement of \(R_1\)! Bad news: Our dependency \(R_1\) has a using gadgets, widgets, and the 1.2.0 version of acme-libs does not provide the required gadgets library.

bpt will mark acme-libs@1.2.0 as incompatible with \(R_1\) and move on to the next candidate: acme-libs@1.3.0. Great news: This version both matches the version requirement \(R_1\) and provides both libraries required by \(R_1\). Thus, acme-libs@1.3.0 will be selected to solve \(R_1\).

Since \(R_1\) is the only dependency statement, we have a complete dependency solution with just selecting acme-libs@1.3.0.

Getting More Complicated

Suppose now that there are additional packages available for use:

  • gandalf@6.3.0

    • Provides library wizard which depends on acme-libs@1.2.0 using gizmos.

  • gandalf@6.4.0

    • Provides wizard, which depends on acme-libs@1.2.0 using gadgets

Let’s update our bpt.yaml to use this package:

name: my-code
version: 4.3.0

dependencies:
  - acme-libs@1.0.0 using gadgets, widgets
  - gandalf@6.0.0 using wizard

In addition to our previous dependency \(R_1\) of acme-libs@1.0.0 using gadgets, widgets, we now have an additional requirement \(R_2\) of gadgalf@6.0.0 using wizard. Dependency resolution now becomes more complex:

  1. In solving \(R_2\), we first check gandalf@6.3.0

    1. This looks okay at first: This package matches our version requirement in \(R_2\) and it also provides the wizard library that we are using in \(R_2\).

    2. bpt will speculatively select this package as part of the solution.

  2. bpt will now validate the new package against the “partial solution” that we are working with.

  3. The used wizard library of the selected gandalf@6.3.0 has its own dependency \(R_g1\) of acme-libs@1.2.0 using gizmos. We can take the intersection \(R_x = R_1 \cap R_g1\) of our existing requirement \(R_1\) on acme-libs to form a new derived requirement \(R_x =\) acme-libs@1.0.0 using gadgets, widgets, gizmos

    This \(R_x\) is the dependency intersection of \(R_g1\) and \(R_1\) because any selection that satisfies \(R_x\) will necessarily also satisfy \(R_g1\) and \(R_1\).

  4. bpt must now seek a version of acme-libs that satisfies \(R_x\), but a cursory glance reveals that \(R_x\) is unsatisfiable: There is no acme-libs package that provides gadgets and widgets and gizmos. (bpt encodes this fact with a special requirement acme-libs@[⊥], which is unsatisfiable by definition. This notation may appear in diagnostics during dependency resolution failure.)

  5. Because our partial solution contains an unsatisfiable derived requirement, the entire partial solution is invalid, and bpt must backtrack to find the speculative decision that caused the failure. In this case, the speculative selection of gandalf@6.3.0 caused the creation of an unsatisfiable partial solution, so we transitively mark gandalf@6.3.0 as incompatible with the partial solution that led to its selection.

  6. With gandalf@6.3.0 ruled out, we need to find another package to satisfy \(R_2\). Fortunately, we have one: gandalf@6.4.0. This is speculatively selected for the solution.

  7. The gandalf@6.4.0 library wizard contains a new requirement \(R_g2\) of acme-libs@1.2.0 using gadgets, which we will check against our existing partial solution.

  8. The speculated selection of acme-libs@1.3.0 satisfies R_g2, so the partial solution is okay.

  9. There are no remaining unsatisfied requirements, so we select acme-libs@1.3.0 and gandalf@6.4.0 as our dependency solution.

A Note on Library Removal

In the above example, the acme-libs@1.4.0 simltaneously added the gizmos library and removed the widgets library.

The removal of a library from a package is necessarily a breaking change, and is especially troublesome here in that there is no version of the package that contains all of widgets, gadgets, and gizmos. Any dependency statement (derived or direct) that requests all three libraries will be unsatisfiable. It is very highly recommended to refrain from removing libraries as part of a minor version change, thus reducing the likelihood of such conflicts.