YANG versioning and backwards (in)compatibility

YANG is defined in RFC6020, which has a section on what changes are allowed between revisions of a YANG model.

The whole idea with having a model describing data is to leave a sort of "contract" to another party on what data is accepted. The rules in RFC6020 are defined such that you cannot break that contract.

For example, the first revision of model foo has the leaf /bar. If we realise this leaf isn't actually needed we cannot remove it in the second revision of the model since anyone with the first revision of the model might try to set a value for the /bar leaf. Deleting the leaf would break the "contract".

I won't list all the rules here but they essentially boil down to only allowing additions. You cannot remove nodes or make changes that reduce the value space for a leaf.

I think one of the more clear examples of YANG compatibility breakage is Cisco's IOS XR as can be witnessed by the XR YANG models published on GitHub.

Cisco doesn't even try to hide that they are breaking YANG rules (kudos for that, I suppose), in fact there is a section in the README file of each subdirectory that reads:

It should be noted that some of the modules released in
IOX-XR 6.0.1 break the backwards compatibility guidelines
defined in RFC 6020 when compared to the same modules
released in IOS-XR 6.0.1. This is because the "native" YANG
modules for IOS-XR are generated from internal schema files
that are an integral part of the implementation, and, as
such, these can change in ways that break backwards
compatibility per RFC 6020 guidelines when new features are
introduced or when bugs are fixed. Thus, while we rigorously
review the changes that impact the external YANG schema,
Cisco cannot guarantee full backwards compatibility of these
modules across releases.

However, when new versions of the native models are
released, the check-models.sh script, in conjunction with
pyang 1.6, can be used to determine what technically
incompatible changes have occurred. Please run check.sh from
this directory with pyang 1.5 or greater on your path thus:

If you run the suggested commands you will see that there are incompatible changes between the majority of revisions:

• 5.3.0 -> 5.3.1 fails
• 5.3.1 -> 5.3.2 fails
• 5.3.2 -> 5.3.3 fails
• 5.3.3 -> 6.0.0 fails
• 6.0.0 -> 6.0.1 works ok, I'm guessing because they didn't make any (or much) changes to the models at all

If you happen to be using Cisco NSO (formerly Tail-F NCS) you might have noticed that you can't load the same module / namespace multiple times. You are supposed to load the latest revision, which will work (to a certain extent) with older models, or if you are using an older revision of the model you are still able to work with the leaves it contains. However, this only holds true if you stick to the YANG revision compatibility rules, which Cisco aren't.

With NSO you compile the YANG models into .fxs files which are then loaded by NSO. ncsc is the NSO compiler that takes care of the compilation step and it actually has an option relevant to this, namely --lax-revision-merge;

--lax-revision-merge    When we have multiple revisions of
  the same module, the ncsc command to import the module
  will fail if a YANG module does not follow the YANG
  module upgrade rules. See RFC 6020. This option makes
  ncsc ignore those strict rules. Use with extreme care,
  the end result may be that NCS is incompatible with the
  managed devices.

However, in the case of merging XR 5.1 and XR 5.3 models, it does not work as warned by the man page.

It's rather ironic how two Cisco products aren't compatible with each other while other products, such as JUNOS, works great together with Cisco NSO, even across different versions. Never buy more than one product from Cisco? ;)

The simple solution is of course that Cisco stops breaking YANG rules. However, I can understand it's difficult with their development process and sometimes you just have to make backwards incompatible changes. Besides, it's an industry wide problem and not just related to Cisco.

Let's first look at semver and then how we might merge the concept of semver with YANG versioning scheme that allows for backwards incompatibility.

Semantic versioning (semver for short) is a fairly widely accepted versioning scheme where the version is described as MAJOR.MINOR.PATCH. In essence, increment the:

• MAJOR version when you make incompatible API changes,
• MINOR version when you add functionality in a backwards-compatible manner, and
• PATCH version when you make backwards-compatible bug fixes.

YANG however doesn't follow semver but I believe the same concept can be applied.

The suggestion is simple:

Let the MAJOR version form part of the model name and namespace and let the revisions of a YANG model represent the MINOR and PATCH versions.

Following semver this means all the backwards compatible changes are made within the same YANG model with the addition of new revisions. Backwards incompatible changes however results in the MAJOR version being bumped and as the MAJOR version is part of the model name and namespace it means we effectively create a new YANG model.

• for backwards compatible changes
  • keep MAJOR version, thus:
    • we keep the YANG module name & namespace
  • bump MINOR or PATCH version, thus:
    • add new revision in current YANG module
• for backwards incompatible changes
  • bump MAJOR version, thus:
    • create a new YANG module name & namespace based on new MAJOR version
  • add a new revision to the new YANG module
  • let the old YANG module remain with the older MAJOR version, thus maintaining backwards compatibility

Let's say we have a module we wish to name foo. The first public release of it, keeping in line with semver, is v1.0.0. The module would thus be called foo-v1 and with a namespace that also includes the -v1 part, like http://example.com/ns/yang/foo-v1.

A bug fix is made, fixing a regexp for one of the leaves, but keeping inline with RFC6020 rules it does not decrease the allowed value space. Since it's a bug fix, we increase to semver v1.0.1. The YANG module will still be called foo-v1 but we add a revision explaining the bug fix.

Similarly if we add a leaf, it is classified as a minor feature, thus bumping the semver MINOR version to v1.1.0. We still keep the module name foo-v1 and add a revision, just as for the bug fix.

Now, if we decide to completely restructure a part of the module, including removing some containers and their leaves, we have made a backwards incompatible change and must bump the MAJOR version. The module will now be v2.0.0 and thus the name changes to foo-v2 and the same change is reflected in the namespace. For the sake of clarity we can keep all the revisions from the v1 "line" of the module. We now have two modules; foo-v1 and foo-v2.

Further bug fixes and minor feature additions can be made to both, or if you choose to only one (likely the v2). The v1 line will eventuelly be deprecated and the exact timeline for that is mosty the result of how long your organisation can or want to maintain multiple versions in parallel.

The "label" used in the revision statement in YANG models is an ISO 8601 date. This could potentially be replaced with a semver version but I don't believe it obviates the need to put the major version in the model name. If a backwards incompatible change is made we might want to support both MAJOR version 1 and 2 at the same time and thus need to load two versions of the model. How are we to do that if both versions have the same name? How do we store two different YANG modules on disk if the filename doesn't reflect the MAJOR version?

There are probably more aspects to consider associated with this suggestion. For example, YANG models can refer to each other, through imports or submodules. If a large number of models are "intertwined" through reference it might not be apparent which of these models can or should be upgraded in a MAJOR version bump.

Do you have any thoughts? Don't hesitate in reaching out to me (contact details in the page footer).

Incidentally, just a few days after I wrote the first draft of this post I noticed that Brocade had put a -v1 in the name of their YANG models. I'm not sure about their rationale but I suspect it is rather similar to what I've outlined here.