Librsvg's build infrastructure: Autotools and Rust

- autotools, gnome, librsvg, rust

Today I released librsvg 2.41.1, and it's a big release! Apart from all the Rust goodness, and the large number of bug fixes, I am very happy with the way the build system works these days. I've found it invaluable to have good examples of Autotools incantations to copy&paste, so hopefully this will be useful to someone else.

There are some subtleties that a "good" autotools setup demands, and so far I think librsvg is doing well:

  • The configure script checks for cargo and rustc.

  • "make distcheck" works. This means that the build can be performed with builddir != srcdir, and also that make check runs the available tests and they all pass.

  • The rsvg_internals library is built with Rust, and our Makefile.am calls cargo build with the correct options. It is able to handle debug and release builds.

  • "make clean" cleans up the Rust build directories as well.

  • If you change a .rs file and type make, only the necessary stuff gets rebuilt.

  • Etcetera. I think librsvg feels like a normal autotool'ed library. Let's see how this is done.

Librsvg's basic autotools setup

Librsvg started out with a fairly traditional autotools setup with a configure.ac and Makefile.am. For historical reasons the .[ch] source files live in the toplevel librsvg/ directory, not in a src subdirectory or something like that.

librsvg
├ configure.ac
├ Makefile.am
├ *.[ch]
├ src/
├ doc/
├ tests/
└ win32/

Adding Rust to the build

The Rust source code lives in librsvg/rust; that's where Cargo.toml lives, and of course there is the conventional src subdirectory with the *.rs files.

librsvg
├ configure.ac
├ Makefile.am
├ *.[ch]
├ src/
├ rust/         <--- this is new!
│ ├ Cargo.toml
│ └ src/
├ doc/
├ tests/
└ win32/

Detecting the presence of cargo and rustc in configure.ac

This goes in configure.ac:

AC_CHECK_PROG(CARGO, [cargo], [yes], [no])
AS_IF(test x$CARGO = xno,
    AC_MSG_ERROR([cargo is required.  Please install the Rust toolchain from https://www.rust-lang.org/])
)
AC_CHECK_PROG(RUSTC, [rustc], [yes], [no])
AS_IF(test x$RUSTC = xno,
    AC_MSG_ERROR([rustc is required.  Please install the Rust toolchain from https://www.rust-lang.org/])
)

These two try to execute cargo and rustc, respectively, and abort with an error message if they are not present.

Supporting debug or release mode for the Rust build

One can call cargo like "cargo build --release" to turn on expensive optimizations, or normally like just "cargo build" to build with debug information. That is, the latter is the default: if you don't pass any options, cargo does a debug build.

Autotools and C compilers normally work a bit differently; one must call the configure script like "CFLAGS='-g -O0' ./configure" for a debug build, or "CFLAGS='-O2 -fomit-frame-pointer' ./configure" for a release build.

Linux distros already have all the infrastructure to pass the appropriate CFLAGS to configure. We need to be able to pass the appropriate flag to Cargo. My main requirement for this was:

  • Distros shouldn't have to substantially change their RPM specfiles (or whatever) to accomodate the Rust build.
  • I assume that distros will want to make release builds by default.
  • I as a developer am comfortable with passing extra options to make debug builds on my machine.

The scheme in librsvg lets you run "configure --enable-debug" to make it call a plain cargo build, or a plain "configure" to make it use cargo build --release instead. The CFLAGS are passed as usual through an environment variable. This way, distros don't have to change their packaging to keep on making release builds as usual.

This goes in configure.ac:

dnl Specify --enable-debug to make a development release.  By default,
dnl we build in public release mode.

AC_ARG_ENABLE(debug,
              AC_HELP_STRING([--enable-debug],
                             [Build Rust code with debugging information [default=no]]),
              [debug_release=$enableval],
              [debug_release=no])

AC_MSG_CHECKING(whether to build Rust code with debugging information)
if test "x$debug_release" = "xyes" ; then
    AC_MSG_RESULT(yes)
    RUST_TARGET_SUBDIR=debug
else
    AC_MSG_RESULT(no)
    RUST_TARGET_SUBDIR=release
fi
AM_CONDITIONAL([DEBUG_RELEASE], [test "x$debug_release" = "xyes"])

AC_SUBST([RUST_TARGET_SUBDIR])

This defines an Automake conditional called DEBUG_RELEASE, which we will use in Makefile.am later.

It also causes @RUST_TARGET_SUBDIR@ to be substituted in Makefile.am with either debug or release; we will see what these are about.

Adding Rust source files

The librsvg/rust/src directory has all the *.rs files, and cargo tracks their dependencies and whether they need to be rebuilt if one changes. However, since that directory is not tracked by make, it won't rebuild things if a Rust source file changes! So, we need to tell our Makefile.am about those files:

RUST_SOURCES =                   \
        rust/build.rs            \
        rust/Cargo.toml          \
        rust/src/aspect_ratio.rs \
        rust/src/bbox.rs         \
        rust/src/cnode.rs        \
        rust/src/color.rs        \
        ...

RUST_EXTRA +=                    \
        rust/Cargo.lock

EXTRA_DIST += $(RUST_SOURCES) $(RUST_EXTRA)

It's a bit unfortunate that the change tracking is duplicated in the Makefile, but we are already used to listing all the C source files in there, anyway.

Most notably, the rust subdirectory is not listed in the SUBDIRS in Makefile.am, since there is no rust/Makefile at all!

Cargo release or debug build?

if DEBUG_RELEASE
CARGO_RELEASE_ARGS=
else
CARGO_RELEASE_ARGS=--release
endif

We will call cargo build with that argument later.

Verbose or quiet build?

Librsvg uses AM_SILENT_RULES([yes]) in configure.ac. This lets you just run "make" for a quiet build, or "make V=1" to get the full command lines passed to the compiler. Cargo supports something similar, so let's add it to Makefile.am:

CARGO_VERBOSE = $(cargo_verbose_$(V))
cargo_verbose_ = $(cargo_verbose_$(AM_DEFAULT_VERBOSITY))
cargo_verbose_0 =
cargo_verbose_1 = --verbose

This expands the V variable to empty, 0, or 1. The result of expanding that gives us the final command-line argument in the CARGO_VERBOSE variable.

What's the filename of the library we are building?

RUST_LIB=@abs_top_builddir@/rust/target/@RUST_TARGET_SUBDIR@/librsvg_internals.a

Remember our @RUST_TARGET_SUBDIR@ from configure.ac? If you call plain "cargo build", it will put the binaries in rust/target/debug. But if you call "cargo build --release", it will put the binaries in rust/target/release.

With the bit above, the RUST_LIB variable now has the correct path for the built library. The @abs_top_builddir@ makes it work when the build directory is not the same as the source directory.

Okay, so how do we call cargo?

@abs_top_builddir@/rust/target/@RUST_TARGET_SUBDIR@/librsvg_internals.a: $(RUST_SOURCES)
    cd $(top_srcdir)/rust && \
    CARGO_TARGET_DIR=@abs_top_builddir@/rust/target cargo build $(CARGO_VERBOSE) $(CARGO_RELEASE_ARGS)

We make the funky library filename depend on $(RUST_SOURCES). That's what will cause make to rebuild the Rust library if one of the Rust source files changes.

We override the CARGO_TARGET_DIR with Automake's preference, and call cargo build with the correct arguments.

Linking into the main C library

librsvg_@RSVG_API_MAJOR_VERSION@_la_LIBADD = \
        $(LIBRSVG_LIBS)                      \
        $(LIBM)                              \
        $(RUST_LIB)

This expands our $(RUST_LIB) from above into our linker line, along with librsvg's other dependencies.

make check

This is our hook so that make check will cause cargo test to run:

check-local:
        cd $(srcdir)/rust && \
        CARGO_TARGET_DIR=@abs_top_builddir@/rust/target cargo test

make clean

Same thing for make clean and cargo clean:

clean-local:
        cd $(top_srcdir)/rust && \
        CARGO_TARGET_DIR=@abs_top_builddir@/rust/target cargo clean

Vendoring dependencies

Linux distros probably want Rust packages to come bundled with their dependencies, so that they can replace them later with newer/patched versions.

Here is a hook so that make dist will cause cargo vendor to be run before making the tarball. That command will creates a rust/vendor directory with a copy of all the Rust crates that librsvg depends on.

RUST_EXTRA += rust/cargo-vendor-config

dist-hook:
    (cd $(distdir)/rust && \
    cargo vendor -q && \
    mkdir .cargo && \
    cp cargo-vendor-config .cargo/config)

The tarball needs to have a rust/.cargo/config to know where to find the vendored sources (i.e. the embedded dependencies), but we don't want that in our development source tree. Instead, we generate it from a rust/cargo-vendor-config file in our source tree:

# This is used after `cargo vendor` is run from `make dist`.
#
# In the distributed tarball, this file should end up in
# rust/.cargo/config

[source.crates-io]
registry = 'https://github.com/rust-lang/crates.io-index'
replace-with = 'vendored-sources'

[source.vendored-sources]
directory = './vendor'

One last thing

If you put this in your Cargo.toml, release binaries will be a lot smaller. This turns on link-time optimizations (LTO), which removes unused functions from the binary.

[profile.release]
lto = true

Summary and thanks

I think the above is some good boilerplate that you can put in your configure.ac / Makefile.am to integrate a Rust sub-library into your C code. It handles make-y things like make clean and make check; debug and release builds; verbose and quiet builds; builddir != srcdir; all the goodies.

I think the only thing I'm missing is to check for the cargo-vendor binary. I'm not sure how to only check for that if I'm the one making tarballs... maybe an --enable-maintainer-mode flag?

This would definitely not have been possible without prior work. Thanks to everyone who figured out Autotools before me, so I could cut&paste your goodies: