Bazel for Open-source C / C++ Libraries Distribution

0

In the previous a few days, I’ve been experimenting with Bazel as a library distribution mechanism for ccv.

I’m relatively accustomed to hermetic manufacture systems at this level. My predominant files comes from Buck dating 8 years lend a hand. For the time being, it never befell to me this form of manufacture system could perhaps perhaps sooner or later be a library distribution mechanism. For the duration of the equal 8 years, NPM has taken over the field. Original language-dependent equipment managers comparable to Whisk module, Cargo and Swift Bundle Manager popularized the theorem of the usage of the final public repositories (GitHub) as the dependency references. Languages sooner than this duration, essentially C / C++ are moving to this route, slowly.

ccv has a easy autoconf based mostly fully mostly characteristic detection / configuration system. It’s seemingly you’ll perhaps perhaps perhaps presumably request the equipment to work when ./configure && construct. Nonetheless, it never made any serious attempt to be too neat. My initial expertise with monorepos at corporations strongly influenced the resolution to have a easy manufacture system. I fully request that serious consumers will seller the library into their monorepo the usage of their very have manufacture systems.

This has been correct for the previous a few years. Nonetheless as I’m completing nnc and an increasing number of the usage of that for a amount of closed-provide inside most projects, keeping a closed-provide monorepo setup for my inside most projects whereas upstreaming fixes in all equity an hideous expertise. On the varied hand, nnc from the starting up intended to be a low-level implementation. I’m expected to have excessive-level language bindings at some level. On condition that I’m doing more utility-connected construction with nnc in closed-provide structure now, it appears like the positive time.

Even supposing there is no one-correct library distribution mechanism for C / C++, there are contenders. From the genuine-worn genuine / rpm, to Conan, which has obtained some mind-half within the originate-provide world currently.

The replacement of Bazel isn’t very accidental. I’ve been doing some Swift construction with Bazel and the expertise has been obvious. Furthermore, the replacement of excessive-level binding language for nnc, I figured, would be Swift.

Configure

ccv’s manufacture direction of, as much as I could perhaps reasonably not, is host-dependent. I exhaust autoconf to detect system-extensive libraries comparable to libjpeg and libpng, to configure aesthetic compiler recommendations. Even supposing ccv could perhaps perhaps be worn with zero dependency, in that configuration, it will from time to time be gradual.

Coming from the monorepo background, Bazel doesn’t have many utilities which could perhaps perhaps be as readily available as in autoconf. It’s seemingly you’ll perhaps perhaps be in a plan to write automatic configurations in Starlark as repository ideas, nevertheless there isn’t very any genuine documentation on how to write unprecedented ones.

I ended up letting whoever exhaust ccv to resolve how they’ll permit sure sides. For things like CUDA, such configuration isn’t very tenable. I ended up copying over TensorFlow’s CUDA ideas.

Dependencies

Precise worn C / C++ libraries are notoriously indifferent to libraries dependencies v.s. toolchains. Autoconf detects both toolchain configurations moreover available libraries. A amount of those host dependencies construct tainted-compilation a skill in itself.

Bazel is incredible for in-tree dependencies. For out-tree dependencies on the other hand, there isn’t very any established mechanism. The popular system is to write a repository ideas to load relevant dependencies.

This if truth be told works effectively for me. It is versatile ample to tackle circumstances that have Bazel integrations and enjoy not have any Bazel integrations.

Spend Bazel Dependencies

Consumption of the packaged Bazel dependencies then becomes as easy as at the side of git_repository to the WORKSPACE and make contact with aesthetic _deps() repository rule.

After packaging ccv with Bazel, now Swift for nnc can relish the packaged dependency.

Semantic Versioning Challenges

While the Bazel-supplied library distribution mechanism works effectively for my case, it’s miles simplistic. For one, there is certainly no genuine system to entire semantic versioning. It is understandable. Coming from a monorepo culture, it’s miles bright for anyone to dive into dependency hells of library versioning. A a small varied story took plan to Whisk a whereas lend a hand as effectively.

It is messy whenever you occur to are looking to pin a particular version of the library whereas your dependencies are not agreeing with you. That is going to be messy regardless in C / C++ world, unless you prelink these extraordinarily pretty. Bazel’s philosophy from what I can sight, seems largely on keeping the trunk working facet. It is working to this level, nevertheless one has to surprise whether or not this will perhaps perhaps scale if more libraries adopted Bazel as the distribution mechanism.

Closing Phrases

The previous a few months expertise with Bazel has been delectable. While I could perhaps continue to make exhaust of language particular tools (pip, Whisk modules, Cargo, NPM) when doing construction in that individual language, Bazel is a well-behaved replacement for me when doing tainted-language construction. Ideas comparable to workspace, git_repository, http_archive match effectively throughout the larger originate-provide ecosystem. And most surprisingly, it works for heaps of-repo setup whenever you occur to ever have to.

Read More

Leave A Reply

Your email address will not be published.