Universal tinderbox

Tinderbox's main purpose would be as a developer tool to assist in FreeBSD port maintainers in getting various software packages available to FreeBSD users, but can it be used for more than that - yes. It is based on the scripts used by pointyhat to constantly compile ports to test for errors as well as create packages that are available to download through the various mirror sites. Which means it also gives you the ability to compile your own packages repository. If you want to make it do other things you would need to get into the scripts and adjust it to suit. Basically it is a set of scripts that sets up a clean system environment then compiles ports within it. That is a series of preset commands with some runtime options. There is no reason it can't be adjusted to run any automated task that you want to test within multiple clean systems.

So why would you want to build your own packages repository? Maybe you want your packages built with extra options turned on. Maybe you want to compile your ports with -O3 or CPUTYPE=corei7 to get a bit more performance. Or you support a range of servers and desktops that are not all the same OS versions and company policy prevents binary downloads from external sources being installed on your systems. Or you just want to build your own ports but want to know everything builds successfully before changing anything on your base system.

That last one comes in handy if you want to compile everything with clang. While effort is being made to move away from gcc and have everything built with clang, we aren't there yet. When you build your own ports in your base system various dependencies are built and installed before the port that needs them. This leads to newer libraries installed that may not work well with older versions of packages that use them and this mixed bag gets left that way when one port breaks. So it can be useful to know that everything compiles that is needed before anything in the base system gets changed. If you are using zfs then you could also use snapshots to rollback after a bad upgrade, but that's another story.

If you make changes to your ports tree to fix build issues then share the fixes with others by submitting a problem report. Chances are others would like to use the same fixes you made.

But tinderbox also lets you build against system versions that you don't have installed. How does that help you? Maybe you have an old 7.4 machine that you want to upgrade, you can build all the ports you want for that machine against FreeBSD 9.0 so they are all ready before you upgrade. It is often recommended that you recompile your ports after major system version updates. But then what if your upgrade goes bad and you want to go back to your old system? With tinderbox you could also build your ports against an older version and downgrade with confidence. While no-one would recommend you do that, it can give you peace of mind knowing that you have a way to fall back when changes don't work the way you want. If you want the option to fallback you may want to build both package versions before you upgrade.

In previous posts I have talked about setting up builds with a variety of system versions and configuring the environment options for each so I won't repeat that here. If you are wanting to build your own ports then you may want to expand the environment settings a bit. While you don't get the flexibility that you have with the /etc/make.conf you can add most of the options you would normally use there. The main limitation is you don't get conditional sections so anything you setup will be applied to every port you build. I previously showed you how to set the cputype used and you should be able to figure out how to add your own CFLAGS to the environment settings.

The recent options changes within the ports infrastructure (referred to as optionsng) does provide a way to set options for every port - tinderbox limitations prevent you using that within the environment settings for every port, but you can set some global options or turn on a few important options if that is all you want to do. Options set in the environment file will be applied to each port build, whether you choose to build with non-default options or not. If you want to configure options for a lot of ports then use the standard option config dialogs to set them up. My compileport script from the previous post allows you to use the -O option to use non-default options for each port.

So what are some options that you may want to add to your environment?

Well PAPERSIZE seems obvious. QT4_OPTIONS shows that any values with a space needs to be quoted, without the quotes only the first item is recognized. Within the environment file you can't use variable+=addvalue to set your variables gradually - you only get variable=value. The OPTIONS_SET and OPTIONS_UNSET provide a way to globally turn on or off common options. While a lot of options are unique to each port there is some consistency with naming that may also be expanded over time, with optionsng already adding some common ones like THREADS and DOCS to replace the NOPORTDOCS option that was previously available. To set an option only for a specific port use <uniquename>_SET. To get the ports uniquename use make -V UNIQUENAME eg -

While there is some consistency you can't always guess the ports uniquename from the portname. You may also want to look in /var/db/ports/<portname>/options for installed ports to see what values you have already used and what is available.

By default tinderbox isn't setup to store the port options you configure. You need to do that manually. To make it work you need to create a directory called options at the top of the tinderbox directory and inside that create a directory for each buildname you want options saved. As standard in unix the directory can be a link to another so you can use your existing options. To compile the ports with non-default options you need to build the ports with the -O option (or -o option to clear existing options and then set options)

So now all there is to do is create a build environment for your ports and start building. If you've been following along and have the base jails setup then -

And if you are setting up your own packages to distribute through your network of FreeBSD machines you have a couple of options. To use them with the standard pkg_add you can either set PKG_PATH or PACKAGESITE to point to your packages. See man pkg_add for more details. If you use portmaster then you can set LOCAL_PACKAGEDIR to point to /usr/local/tinderbox/packages/9-mypackages-amd64-clang (or a common network mount) as well as PM_PACKAGES=first and PM_PACKAGES_LOCAL=pmp_local. Other package management tools should offer similar config options.