Today, I was working on optimizing my travis builds to speed them up, especially
in the direction of caching. Travis allows you to set
cache: cargo in your
to enable caching for rust projects and caching is cool: it avoids having to rebuild all
dependencies all the time, speeding up the builds.
Or does it?
The travis documentation states this about caching:
Note that this makes our cache not network-local, it is still bound to network bandwidth and DNS resolutions. That impacts what you can and should store in the cache. If you store archives larger than a few hundred megabytes in the cache, it is unlikely that you’ll see a big speed improvement.
Out of curiosity, I ran
du -sh in the
target/ directory of my local version of
wayland-rs. The result was not quite what I expected: 4.2 GB. The kind of things
that likely does not play well with travis caching. And no doubt, the travis caches of
my project were enourmous: between 5GB and 7GB each.
Why are these caches so large? It's because travis unconditionnaly caches the
~/.cargo/ directories, which tend to grow very big, and that for two main reasons:
First of all, cargo and rustc, especially since incremental compilation, generate a lot of build artifacts. These artifacts are stored by the travis cache, and are mostly useless, as in your travis build a large part of them will likely be rebuilt anyway.
Secondly, outdated dependencies accumulate. Both dependencies that have released a new version, and build artifacts that were created using a previous version of the compiler. Their data is never deleted, and just sits around, making the caches bigger and bigger over time.
As a result of this observation, I have drastically changed by caching policy:
target/is not worth caching: it is huge, and only build artifacts of not-outdated dependencies are useful in it. I don't know of any way to selectively keep only the relevant part from it, so I can't affort caching it.
~/.cargo/bin/is worth caching. I really don't want to recompile the cargo subcommands I use in my builds every time as they are generally very long to compile. Combined with the use of cargo-update I can make sure I recompile them only when an update is needed. But for this to work the file
~/.cargo/.crates.tomlmust be cached too.
~/.cargo/registry/is not worth caching either: it accumulate a lot of dead weight as dependencies are updated, and downloading its contents from crates.io rather than the travis cache will likely hardly make a difference.
With that the cache portion of my
.travis.yml now contains the following,
all my builds run between 25% and 50% faster than with
cache: cargo, and my travis
caches now only weight 68MB.
# Need to cache the whole `.cargo` directory to keep .crates.toml for # cargo-update to work cache: directories: - /home/travis/.cargo # But don't cache the cargo registry before_cache: - rm -rf /home/travis/.cargo/registry
For sure, what applies to my project does not necessarily applies to yours, but this is
a quick analysis that might be worth doing before adding
cache: cargo to your travis