Adventures in tinderbox 2


If you stumbled into this post first you may want to have a look over this previous post as we are building upon steps covered there.

So far we have got our world binaries built and these are the foundation of the environments we setup for testing our port builds. From here we create a tinderbox build. Within tinderbox a build refers to a combination of world binaries (the tinderbox jails we setup before) and a version of the ports files. It is probably a rare case but it would appear that you could setup a build using say netbsd pkgsrc system or similar, the most likely alternative to the official ports system would be an internal development branch of the ports source.

The first step is to setup the ports files to be used. A cvs checkout is shown in the docs and you could adjust the svn usage from yesterday if you have your own local svn repo (official ports are still in cvs at this point). The examples in the docs show a source checkout that is unique to your tinderbox setup. I think using the system installed ports will be enough for what I want and will bypass the confusion of three copies of the ports files. Personally I use portsnap to update my system ports tree and create or update a port there to test and develop it. Then I have a copy within my home folder that is an svn checkout from my redports account that I transfer changes to that I want to keep, and commit as desired from there. While my personal folder is just the ports I make changes to, I don't want a third copy just for test builds, so what I will use here is a user defined update method that will run portsnap anytime I want to update the ports tree. We have two options here, 1. create a portsTree with no update method leaving us to run portsnap manually as desired. 2. create a portsTree with a user update method. The first option is straight forward and acceptable for most but I want to look at option 2 here.

Reading the docs you may jump straight in and use a command like cd /usr/local/tinderbox/scripts ./tc createPortsTree -p FreeBSD -u USER -m /usr/ports and get errors and no ports tree setup. The docs don't help here so I went digging through the scripts and found the answer. When you create a portstree with a user defined update method it looks for the update script when it is first setup. The solution is simple. If we are creating a portstree called FreeBSD then we create a file in tinderbox/portstrees/FreeBSD and call it update.sh and for this example I want it to run portsnap so I use the following contents - #!/bin/sh portsnap fetch update Then run the createPortsTree command with the -u USER option. Simple and a good basis of any custom update you want to setup. You could even use this to run portsnap and then merge files from your personal ports repo automatically. This setup script is run with the command ./tc updatePortsTree

You may also want to share your local distfiles with tinderbox. Pretty easy and the one setup is shared across all portstree configs -

cd /usr/local/tinderbox/scripts ./tc configDistfile -c /usr/ports/distfiles

Now that we have our jails and portsTree setup it is time for the build. Remember a tinderbox build is a combination of a specific jail and portsTree and is straight forward -

cd /usr/local/tinderbox/scripts ./tc createBuild -b 10-CURRENT-clang -j 10-CURRENT -p FreeBSD -d "10-CURRENT building with clang"

The name I used there may raise a few questions but first lets consider what variations of builds we want to use.

The aim of using tinderbox is to test building under a variety of conditions, in this case primarily different system versions. To that end we started by creating 2 jails with different versions of world binaries, you may have extended that number and I encourage you to do so but I use two in these examples. From FreeBSD 9.0 we are moving away from a  dependency on gcc as the main compiler and now include clang within the base system. Being in the early transition of this is a bit awkward as not all ports build with the old gcc v4.2 that is included with base and not all ports build with clang either. Well ports changes need to be tested with each compiler so we may get a full ports build happening on any system install. One option we have is to specify what version of gcc is used to build our port. Easy till we get linking issues with something like boost installed with gcc42 and not linking with a gcc46 built port (that is my early guess and may not be the location of a solution). So anyway I am suggesting that a number of builds are setup to test building with clang as well as gcc. This will only apply to 9.0 and higher, 8.3 has just entered beta1 and doesn't appear to include clang. The other question you may have is that I only put clang in the name, how does it know to build with clang? It doesn't yet - that comes in a minute.

Well using the two jails we built yesterday lets setup the builds -

cd /usr/local/tinderbox/scripts ./tc createBuild -b 10-CURRENT-clang -j 10-CURRENT -p FreeBSD -d "10-CURRENT building with clang" ./tc createBuild -b 10-CURRENT-gcc -j 10-CURRENT -p FreeBSD -d "10-CURRENT building with gcc" ./tc createBuild -b 9-STABLE-clang -j 9-STABLE -p FreeBSD -d "9-STABLE building with clang" ./tc createBuild -b 9-STABLE-gcc -j 9-STABLE -p FreeBSD -d "9-STABLE building with gcc"

Now we have four builds setup to test our ports in. As I suggested before you should also consider having at least an 8-STABLE jail setup and maybe even a 7-STABLE one as well if not more. As clang is not included in 7 or 8 there is no need to setup two builds for each, I would still use gcc in the build name though.

At this point you will find a few directories have been created. The first you will probably see is tinderbox/{BUILDNAME} so if your following along you will have four there corresponding to each build you setup. The next ones are inside tinderbox/builds and have the same names as the four we just looked at. The dirs inside builds contain a Makefile with all the package  dependencies and although you may think make.0 and make.1 are makefiles they are in fact logs containing the log of the last compile done, each port compile is run in two phases, and a more than likely empty duds file. The top level dirs are where all the work is done. For each port you make, tinderbox will extract the tarball found in the corresponding jail build and mount the various points needed for portstrees/distfiles/devfs/procfs. From that you can work out that each build starts up in a unique environment (it uses chroot) completely isolated for your test make. Each environment is cleaned up after each test make and the log files will include files left behind if you get your pkg-plist incorrect.

Now to configuring the compiler to use. When you first test making your port and look at a log file you may notice that the timestamp at the start is all wrong. That will be a wrong timezone setting and gets us close to the compiler settings. First lets set the timezone, so edit the file tinderbox/scripts/etc/env/GLOBAL and add the following line (adjusted to match your timezone of course)

export TZ=Australia/Adelaide

You can see that it uses a bash style environment setting. Within the same folder we want to create a config file for each build we setup using the name build.{BUILDNAME} and have it contain the CC options that we wish to use. Before you go thinking about copying all your /etc/make.conf settings consider the purpose of this - testing against a minimum system config to ensure your port is as compatible as possible. Those with a custom edited make.conf need to handle their own port build conflicts with the options they want to use. So here all we want to do is set the compiler to use. To make this easy let's make a script, I will put this inside tinderbox/scripts and lets call it gencompilerenvfiles.sh

#!/bin/sh cd /usr/local/tinderbox/scripts/etc/env for BNAME in `ls ../../../builds | grep clang` do if [ ! -e build.$BNAME ] then cat > build.$BNAME <<CCOPTIONS export CC=clang export CXX=clang++ export CPP=clang-cpp CCOPTIONS fi done for BNAME in `ls ../../../builds | grep gcc` do if [ ! -e build.$BNAME ] then cat > build.$BNAME <<CCOPTIONS export CC=gcc export CXX=g++ export CPP=cpp CCOPTIONS fi done

Pretty straight forward if you know some shell scripting, if it's a bit beyond what you know it simply gets a list of dirs from within builds and outputs three lines to a corresponding config file for each one. The script won't touch files that already exist, so delete them if you want a clean slate or adjust if you want to add to existing files. The first loop uses grep to get clang named builds the second gcc. Is it overkill specifying gcc for the gcc builds - yep - but it keeps things consistent and also gets the CC setting output to the log file to remove any confusion when looking over them later, or more to the point when you get someone else to look over them for some help.

Now your all setup to use tinderbox to test your ports building. While the docs say you can do an adhoc build with just the tinderbuild command I find that not to be the case, the two command you use are

cd /usr/local/tinderbox/scripts ./tc addPort -b 10-CURRENT-clang -d graphics/blender ./tc tinderbuild -nullfs -b 10-CURRENT-clang graphics/blender

The first time you build a port it may take a while - it will build each  dependency needed, basically running make package-recursive for your port. Next time it will just pkg_add each dependancy and build the port you ask it to. The -nullfs option is needed if you setup the nullfs mount for portstree as shown above (or maybe it is for distfiles?). This adds some more dirs for you as well.

tinderbox/packages/{BUILDNAME} will contain all the packages created by that build. You will find a folder for each category containing links to the corresponding package inside the All dir.

tinderbox/logs/{BUILDNAME} will contain log files of each port compiled for that build.

tinderbox/errors/{BUILDNAME} will contain copies of the log files that had errors - this is probably all your really interested in.

That's about all you need to get up and running with tinderbox. I think I will do one more post covering some housekeeping and other notes. To close off I'll show a script I use to automate running any number of ports inside every build you have configured, I save it in tinderbox/scripts and call it testbuilds.sh

#!/bin/sh ## auto build passed ports for every tinderbox build setup ## works on assumption that /usr/ports is used for tinderbox portstree ## to test that the portname is correct cd /usr/local/tinderbox/scripts if [ ! $# -gt 0 ] then echo "Usage: ${0##*/} [category/portname ...]" echo "Multiple ports can be listed - space separated list" exit 1 fi for TESTPORT in "$@" do if [ ! -e "/usr/ports/$TESTPORT" ] then echo "********************" echo " /usr/ports/$TESTPORT doesn't exist" echo "********************" else for BNAME in `ls ../builds` do echo "***** Building $TESTPORT" ./tc addPort -b $BNAME -d ${TESTPORT} ./tc tinderbuild -nullfs -b $BNAME ${TESTPORT} done fi done

So to make use of the script

cd /usr/local/tinderbox/scripts ./testbuilds.sh graphics/blender graphics/luxrender ports-mgmt/tinderbox

and it will go through and compile each port with every tinderbox build you have setup.


11286 comments

France's picture

The principle drawback to e mail advertising is that some customers might consider the emails to be spam
and hold it in opposition to a brand's reputation. If prospects really
feel they are being bombarded by undesirable emails from a enterprise, they
are going to be much less likely to grow to be new
or repeat clients.

Maxwell's picture

But that does not imply we should turn a blind eye to the darker aspect of this expertise.
Indeed, we need to contemplate it very rigorously.

And simply maybe, with a sustained effort by shoppers,
producers, and researchers, the photovoltaics industry
will sooner or later be actually, not just symbolically, inexperienced.

Window Tinting Hamilton's picture

That's the value for warmth rejecter tints," Aburumuh says.

Audra's picture

We are dedicated to providing you with the best level of service,
so you will always select us and come again.

erotica's picture

Hello, i think that i noticed you visited my weblog so i came to return the
desire?.I am trying to find issues to enhance
my website!I guess its good enough to make use of some of your ideas!!

Glenn's picture

The great thing about protein is that it'll help build lean body mass.

Look into my web page :: grass fed whey protein isolate powder (Glenn)

check this site out's picture

My brother suggested I might like this blog.
He used to be totally right. This publish truly made my day.
You cann't believe just how so much time I had spent for this info!
Thanks!

Shaunte's picture

The replace will also include stuff like Deathclaws and mole rats;
in different phrases, more stuff to maintain the
sport interesting.

gorgonzola's picture

Hi! Do you know if they make any plugins to
help with Search Engine Optimization? I'm
trying to get my blog to rank for some targeted keywords
but I'm not seeing very good success. If you know of any please share.

Thanks!

Texas Craft Beer's picture

The UK craft beer subscription neighborhood is ever rising with Flavourly people from the
UK are able to discover new craft beers very like their counterparts
over the pond within the USA.

cummins workshop manual's picture

The other father assured me that his boys were wixe old campers having spent the summers with their aunt camping all
over the northwest. Article Source: Snow Performance MPG-MAX water methanol injection kit can be used
in lieu of diesel DPF Delete to legally increase horsepower and fuel economy in your lazte modeel diesel
truck. Not all Bilingual programs out there are the same or even beneficial.

Kitchen Remodeling St. Louis's picture

When purchasing for a new sink, have a look at options like extensions, sprayers and
soap dispensers.

www.hoolatata.fr's picture

Just beneficial info is provided by this blog and
I am subscribing to it this instant! Thank you folks!

Subway surfers generator's picture

Some games are too difficult to deal with and as a gamer,
you possibly can either stop it or hack your means by the game.

Using hack tools is only a method to hold yourself active
and excited throughout.

Asana Health Products Canada's picture

Just desire to say your article is as astonishing. The clearness in your post is just excellent
and i could assume you are an expert on this subject. Well with your permission allow me to grab your feed to keep up to date with forthcoming post.

Thanks a million and please carry on the enjoyable
work.

http://www.thenewcivilrightsmovement.com/coldwell-banker-'s picture

We are a group of volunteers and opening a new scheme in our community.
Your web site provided us with valuable info to work on. You've done a formidable job and our
whole community will be thankful to you.

joshua9dba3jblog.edublogs.org's picture

Things get higher then worse, then better and worse again...and when it seems like my skin is
lastly clearing up for good, I breakout once again with
purple, infected cysts throughout my face.

Glenna's picture

They will slip and slide, even by a tiny amount.

When you maintain your car in the best condition,
you can avoid unexpected breakdowns and costly repairs. Another consideration in choosing
a car repair service is the type of services offered.

Darcy's picture

Conflict of Clans is free to obtain and play, however some
game items can also be purchased for real money.

If you do not need to make use of this feature,
please arrange password safety for purchases within the settings of your Google Play
Retailer app.

picks for guitar's picture

This is the right webpage for everyone who really wants to find
out about this topic. You realize a whole lot its
almost hard to argue with you (not that I personally would want
to...HaHa). You definitely put a brand new spin on a topic that has been discussed for many years.
Wonderful stuff, just great!

perfect biotics reviews's picture

We are a group of volunteers and opening a new scheme in our community.
Your site offered us with helpful information to
work on. You've done a formidable activity and our whole group will be grateful to you.

Feel free to surf to my site - perfect biotics reviews

Kristen's picture

Other than that, I'm not really bought on remote residence monitoring.

https://www.facebook.com/TrucosCOChack/'s picture

Los duendes nunca tienen el escudo activo y atacarles
no tte quita ell tuyo.

alkaline bottled water's picture

Hi, I check your blog daily. Your writing style is awesome, keep it up!

sick's picture

I like it whenever people come together and share ideas.
Great site, stick with it!

Wireless Clocks for Businesses's picture

As the admin of this web page is working, no question very quickly
it will be famous, due to its feature contents.

Website's picture

The other day, while I was at work, my sister stole
my iphone and tested to see if it can survive a 25 foot drop, just so she
can be a youtube sensation. My apple ipad is now destroyed and she has 83 views.
I know this is completely off topic but I had to share it
with someone!

Full Article's picture

Hi there friends, fastidious article and nice arguments commented at this place, I am truly enjoying by these.

matched betting tutorial's picture

I've been exploring for a little for any high-quality articles or weblog posts in this kind of
house . Exploring in Yahoo I eventually stumbled upon this
website. Reading this info So i'm happy to convey that I've an incredibly good
uncanny feeling I found out exactly what I needed. I so much without a doubt will make
sure to do not forget this website and give it a glance regularly.

wireless synchronized time clocks's picture

This paragraph is truly a pleasant one it assists
new internet users, who are wishing in favor of blogging.

vitamin c for face's picture

It's very effortless to find out any topic on net
as compared to books, as I found this article at this
website.

cursos de cerrajeros en Jaen's picture

Hi there! Do you know if they make any plugins to protect against hackers?
I'm kinda paranoid about losing everything I've worked hard on. Any
tips?

tumblr blogs on chat up lines's picture

Very nice post. I just stumbled upon your blog
and wanted to say that I have really enjoyed surfing around your blog posts.
In any case I'll be subscribing to your feed and I hope you write again very soon!

football's picture

What's up, after reading this awesome post i am as well delighted to share my experience here
with mates.

deadpool online's picture

Armed together with his new skills and a dark, twisted humorousness, Deadpool hunts down the man who nearly
destroyed his life.

Town car services in Los Angeles's picture

A key function of the Gett out-of-dwelling campaign is
Posterscope's use of actual-time buying for digital screens in elevators and bars.

Danelle's picture

The by no means ending pursuit of artwork. I wish to work with the shopper to help them realize their vision via my design.

Reynaldo's picture

Nonetheless, you possibly can certainly ask what it may cost
to have your legal work carried out and different questions to find
out if the lawyer could also be a great fit for you.

Ugly Christmas Sweater T-Shirt's picture

I have read so many articles about the blogger lovers except this piece of writing
is in fact a pleasant paragraph, keep it up.

Full Article's picture

This is my first time visit at here and i am really impressed to read everthing at single
place.

masajista en lima's picture

Ahaa, its nice dialogue on the topic of this article at this
place at this website, I have read all that,
so at this time me also commenting here.

[主页]'s picture

I really like what you guys tend to be up too.

This kind of clever work and reporting! Keep up the wonderful works guys
I've incorporated you guys to my own blogroll.

LED Light's picture

Terrific post however , I was wanting to know if you could write a
litte more on this subject? I'd be very grateful if you could elaborate a little bit more.
Thank you!

smart member 2.0 discount's picture

I don't know whether it's just me or if everyone else
experiencing problems with your website. It seems like some of the text within your content
are running off the screen. Can somebody else please comment and
let me know if this is happening to them as well? This may be a issue with my web browser because I've had this happen before.
Kudos

nfc tags tasker's picture

Purchase NFC tag stickers that may be easily integrated
into promotional marketing collateral similar to posters
, duratrans , window clings , retail shows, signage, product packaging, junk
mail and quite a few different print marketing choices.

Smart Member 2's picture

Woah! I'm really enjoying the template/theme of this website.
It's simple, yet effective. A lot of times it's difficult to get that "perfect balance" between user friendliness and visual appearance.

I must say you have done a awesome job with this.
Additionally, the blog loads very fast for me on Firefox.
Outstanding Blog!

earn extra money's picture

My spouse and I stumbled over here different website and thought I
should check things out. I like what I see so now i am following you.
Look forward to checking out your web page for a second time.

useful reference's picture

Hello! I know this is kinda off topic however I'd figured I'd ask.
Would you be interested in trading links or maybe guest writing a blog
article or vice-versa? My site addresses a lot of the same topics as yours and I feel we could greatly benefit from
each other. If you're interested feel free to shoot me an e-mail.

I look forward to hearing from you! Wonderful blog by the way!

tumblr blogs on chat up lines's picture

Excellent items from you, man. I have bear in mind your stuff previous
to and you are simply too great. I really like what you've acquired right here,
certainly like what you're stating and the way through which you say it.
You make it enjoyable and you continue to care for to keep it smart.
I can not wait to read much more from you. That is really a wonderful
site.

格安スマホ's picture

Ri Ben Lu Xing notokiniXi Dai Dian Hua woLi Yong surutokiniDian Hua Liao woAn kusiteShi eruFang Fa nihaGe An SIMgaarudesiyou.

MVNOtohasumahotoUSIMwoCha Ru sitenetutositarisite,gawanoGou Ru Liao Jin ya,Mei Yue noTong Xin Liao woAn Jia niYi erukotogadekimasu.

Pages

Add new comment