Adventures in tinderbox 2


If you stumbled into this post first you may want to have a look over this previous post as we are building upon steps covered there.

So far we have got our world binaries built and these are the foundation of the environments we setup for testing our port builds. From here we create a tinderbox build. Within tinderbox a build refers to a combination of world binaries (the tinderbox jails we setup before) and a version of the ports files. It is probably a rare case but it would appear that you could setup a build using say netbsd pkgsrc system or similar, the most likely alternative to the official ports system would be an internal development branch of the ports source.

The first step is to setup the ports files to be used. A cvs checkout is shown in the docs and you could adjust the svn usage from yesterday if you have your own local svn repo (official ports are still in cvs at this point). The examples in the docs show a source checkout that is unique to your tinderbox setup. I think using the system installed ports will be enough for what I want and will bypass the confusion of three copies of the ports files. Personally I use portsnap to update my system ports tree and create or update a port there to test and develop it. Then I have a copy within my home folder that is an svn checkout from my redports account that I transfer changes to that I want to keep, and commit as desired from there. While my personal folder is just the ports I make changes to, I don't want a third copy just for test builds, so what I will use here is a user defined update method that will run portsnap anytime I want to update the ports tree. We have two options here, 1. create a portsTree with no update method leaving us to run portsnap manually as desired. 2. create a portsTree with a user update method. The first option is straight forward and acceptable for most but I want to look at option 2 here.

Reading the docs you may jump straight in and use a command like cd /usr/local/tinderbox/scripts ./tc createPortsTree -p FreeBSD -u USER -m /usr/ports and get errors and no ports tree setup. The docs don't help here so I went digging through the scripts and found the answer. When you create a portstree with a user defined update method it looks for the update script when it is first setup. The solution is simple. If we are creating a portstree called FreeBSD then we create a file in tinderbox/portstrees/FreeBSD and call it update.sh and for this example I want it to run portsnap so I use the following contents - #!/bin/sh portsnap fetch update Then run the createPortsTree command with the -u USER option. Simple and a good basis of any custom update you want to setup. You could even use this to run portsnap and then merge files from your personal ports repo automatically. This setup script is run with the command ./tc updatePortsTree

You may also want to share your local distfiles with tinderbox. Pretty easy and the one setup is shared across all portstree configs -

cd /usr/local/tinderbox/scripts ./tc configDistfile -c /usr/ports/distfiles

Now that we have our jails and portsTree setup it is time for the build. Remember a tinderbox build is a combination of a specific jail and portsTree and is straight forward -

cd /usr/local/tinderbox/scripts ./tc createBuild -b 10-CURRENT-clang -j 10-CURRENT -p FreeBSD -d "10-CURRENT building with clang"

The name I used there may raise a few questions but first lets consider what variations of builds we want to use.

The aim of using tinderbox is to test building under a variety of conditions, in this case primarily different system versions. To that end we started by creating 2 jails with different versions of world binaries, you may have extended that number and I encourage you to do so but I use two in these examples. From FreeBSD 9.0 we are moving away from a  dependency on gcc as the main compiler and now include clang within the base system. Being in the early transition of this is a bit awkward as not all ports build with the old gcc v4.2 that is included with base and not all ports build with clang either. Well ports changes need to be tested with each compiler so we may get a full ports build happening on any system install. One option we have is to specify what version of gcc is used to build our port. Easy till we get linking issues with something like boost installed with gcc42 and not linking with a gcc46 built port (that is my early guess and may not be the location of a solution). So anyway I am suggesting that a number of builds are setup to test building with clang as well as gcc. This will only apply to 9.0 and higher, 8.3 has just entered beta1 and doesn't appear to include clang. The other question you may have is that I only put clang in the name, how does it know to build with clang? It doesn't yet - that comes in a minute.

Well using the two jails we built yesterday lets setup the builds -

cd /usr/local/tinderbox/scripts ./tc createBuild -b 10-CURRENT-clang -j 10-CURRENT -p FreeBSD -d "10-CURRENT building with clang" ./tc createBuild -b 10-CURRENT-gcc -j 10-CURRENT -p FreeBSD -d "10-CURRENT building with gcc" ./tc createBuild -b 9-STABLE-clang -j 9-STABLE -p FreeBSD -d "9-STABLE building with clang" ./tc createBuild -b 9-STABLE-gcc -j 9-STABLE -p FreeBSD -d "9-STABLE building with gcc"

Now we have four builds setup to test our ports in. As I suggested before you should also consider having at least an 8-STABLE jail setup and maybe even a 7-STABLE one as well if not more. As clang is not included in 7 or 8 there is no need to setup two builds for each, I would still use gcc in the build name though.

At this point you will find a few directories have been created. The first you will probably see is tinderbox/{BUILDNAME} so if your following along you will have four there corresponding to each build you setup. The next ones are inside tinderbox/builds and have the same names as the four we just looked at. The dirs inside builds contain a Makefile with all the package  dependencies and although you may think make.0 and make.1 are makefiles they are in fact logs containing the log of the last compile done, each port compile is run in two phases, and a more than likely empty duds file. The top level dirs are where all the work is done. For each port you make, tinderbox will extract the tarball found in the corresponding jail build and mount the various points needed for portstrees/distfiles/devfs/procfs. From that you can work out that each build starts up in a unique environment (it uses chroot) completely isolated for your test make. Each environment is cleaned up after each test make and the log files will include files left behind if you get your pkg-plist incorrect.

Now to configuring the compiler to use. When you first test making your port and look at a log file you may notice that the timestamp at the start is all wrong. That will be a wrong timezone setting and gets us close to the compiler settings. First lets set the timezone, so edit the file tinderbox/scripts/etc/env/GLOBAL and add the following line (adjusted to match your timezone of course)

export TZ=Australia/Adelaide

You can see that it uses a bash style environment setting. Within the same folder we want to create a config file for each build we setup using the name build.{BUILDNAME} and have it contain the CC options that we wish to use. Before you go thinking about copying all your /etc/make.conf settings consider the purpose of this - testing against a minimum system config to ensure your port is as compatible as possible. Those with a custom edited make.conf need to handle their own port build conflicts with the options they want to use. So here all we want to do is set the compiler to use. To make this easy let's make a script, I will put this inside tinderbox/scripts and lets call it gencompilerenvfiles.sh

#!/bin/sh cd /usr/local/tinderbox/scripts/etc/env for BNAME in `ls ../../../builds | grep clang` do if [ ! -e build.$BNAME ] then cat > build.$BNAME <<CCOPTIONS export CC=clang export CXX=clang++ export CPP=clang-cpp CCOPTIONS fi done for BNAME in `ls ../../../builds | grep gcc` do if [ ! -e build.$BNAME ] then cat > build.$BNAME <<CCOPTIONS export CC=gcc export CXX=g++ export CPP=cpp CCOPTIONS fi done

Pretty straight forward if you know some shell scripting, if it's a bit beyond what you know it simply gets a list of dirs from within builds and outputs three lines to a corresponding config file for each one. The script won't touch files that already exist, so delete them if you want a clean slate or adjust if you want to add to existing files. The first loop uses grep to get clang named builds the second gcc. Is it overkill specifying gcc for the gcc builds - yep - but it keeps things consistent and also gets the CC setting output to the log file to remove any confusion when looking over them later, or more to the point when you get someone else to look over them for some help.

Now your all setup to use tinderbox to test your ports building. While the docs say you can do an adhoc build with just the tinderbuild command I find that not to be the case, the two command you use are

cd /usr/local/tinderbox/scripts ./tc addPort -b 10-CURRENT-clang -d graphics/blender ./tc tinderbuild -nullfs -b 10-CURRENT-clang graphics/blender

The first time you build a port it may take a while - it will build each  dependency needed, basically running make package-recursive for your port. Next time it will just pkg_add each dependancy and build the port you ask it to. The -nullfs option is needed if you setup the nullfs mount for portstree as shown above (or maybe it is for distfiles?). This adds some more dirs for you as well.

tinderbox/packages/{BUILDNAME} will contain all the packages created by that build. You will find a folder for each category containing links to the corresponding package inside the All dir.

tinderbox/logs/{BUILDNAME} will contain log files of each port compiled for that build.

tinderbox/errors/{BUILDNAME} will contain copies of the log files that had errors - this is probably all your really interested in.

That's about all you need to get up and running with tinderbox. I think I will do one more post covering some housekeeping and other notes. To close off I'll show a script I use to automate running any number of ports inside every build you have configured, I save it in tinderbox/scripts and call it testbuilds.sh

#!/bin/sh ## auto build passed ports for every tinderbox build setup ## works on assumption that /usr/ports is used for tinderbox portstree ## to test that the portname is correct cd /usr/local/tinderbox/scripts if [ ! $# -gt 0 ] then echo "Usage: ${0##*/} [category/portname ...]" echo "Multiple ports can be listed - space separated list" exit 1 fi for TESTPORT in "$@" do if [ ! -e "/usr/ports/$TESTPORT" ] then echo "********************" echo " /usr/ports/$TESTPORT doesn't exist" echo "********************" else for BNAME in `ls ../builds` do echo "***** Building $TESTPORT" ./tc addPort -b $BNAME -d ${TESTPORT} ./tc tinderbuild -nullfs -b $BNAME ${TESTPORT} done fi done

So to make use of the script

cd /usr/local/tinderbox/scripts ./testbuilds.sh graphics/blender graphics/luxrender ports-mgmt/tinderbox

and it will go through and compile each port with every tinderbox build you have setup.


11286 comments

bateau-popop.com's picture

Hey! I know this is somewhat off topic but I was wondering
which blog platform are you using for this
site? I'm getting fed up of Wordpress because I've had issues with hackers and I'm
looking at alternatives for another platform. I would be great if you could point
me in the direction of a good platform.

Marijuana Addiction Symptoms's picture

Marijuana Addiction

college loan student's picture

Wow, this post is good, my younger sister is analyzing these things, so I am going to convey her.

organic crib sheets's picture

Right now it appears like Drupal is the preferred blogging platform out there right now.

(from what I've read) Is that what you're using on your blog?

Babyphone's picture

We stumbled over here coming from a different web address and thought I might
as well check things out. I like what I see so now i'm following you.
Look forward to exploring your web page again.

imitation jewellery kundan's picture

The other day, while I was at work, my sister stole my iphone and tested to see if it
can survive a 40 foot drop, just so she can be a youtube sensation. My iPad is now destroyed and she
has 83 views. I know this is totally off topic but I had to share it with someone!

ד&quot;ר פוליש's picture

Emotion free is the key here.

depot prices's picture

pharmaeurope.org Its the only legit online site to buying steroids...dont listen to
these fools...

成功するitエンジニア's picture

It is appropriate time to make some plans for the longer term and it
is time to be happy. I've read this post and if I may just I desire to counsel you some fascinating issues or advice.
Maybe you could write subsequent articles referring to this article.
I wish to learn more issues about it!

Streaming Jav Online Free - JavPlay's picture

Fabulous, what a website it is! This webpage provides helpful facts
to us, keep it up.

why not try here's picture

Excellent goods from you, man. I've understand
your stuff previous to and you are just extremely great.

I actually like what you've acquired here, certainly like
what you're saying and the way in which you say it.
You make it enjoyable and you still care for to keep it wise.

I can not wait to read far more from you. This is actually a
great site.

my singing monsters hack android's picture

You might get slapped (and this completely depends on your gender).
He made several appearances at sci-fi conventions throughout his career.
), and then you will find that there has a new character,
"Whorm Loathsom".

21 day fix reviews's picture

Thank you for every other informative site. Where else could I am getting that kind of info written in such
an ideal manner? I've a project that I'm simply now operating on, and I
have been at the glance out for such info.

Also visit my web page - 21 day fix reviews

car insurace's picture

Traditionally, as your insurance policy expiration date approaches, your insurance broker(s) will want to
schedule sales interviews with you to deliver their quotations.
Should the insured default on the loan the rights within the policy would revert to the lender.
The money saved from not opening an office will be passed on to
you by lowering the cost of your car insurance
policy.

fifa 2015 hack's picture

Harvard admissions are even more challenging though, with under
6% of applicants accepted. When fishing this bait it is so important to
fish it very slow bouncing it off the bottom occasionally or
giving it a little twitch when encountering cover.
Let's put it this way, the article is a high-quality synopsis,
if the investor is brand new to the stock market, but
if the investor is profoundly experienced, the chances are that the investor may already be
doing just this kind of system and is currently making money with it.

Janet's picture

We have all been wired from birth to seek this out.

real racing 3 cheats's picture

None of these tasks ever change no matter how much you get promoted.
Whether you're trying to sell toothpaste or supplements,
there will be someone trying to find your type of items so you should get out there and make pitches.
The Boomette emits a higher-pitched gurgling noise than the
male and sounds more swine-like.

Summoners War Hack's picture

You're so awesome! I don't believe I have read through anything like this before.

So great to discover another person with unique thoughts
on this subject. Seriously.. thanks for starting this up.

This web site is something that's needed on the internet, someone with a
little originality!

las chicas más lindas del mundo's picture

What's up, after reading this amazing paragraph i am too happy to share my knowledge here with friends.

daily cleanse ingredients's picture

These are really enormous ideas in regarding
blogging. You have touched some good things here.

Any way keep up wrinting.

Max Garcinia's picture

I loved as much as you will receive carried out right here.
The sketch is tasteful, your authored material stylish.
nonetheless, you command get bought an impatience over that you wish be delivering the following.
unwell unquestionably come more formerly again since exactly the
same nearly a lot often inside case you shield this hike.

смотреть кхл онлайн's picture

Mnogie ee znayut, kak nastoyashchuyu otorvu.
smotret khl onlayn

clash of clans free hack's picture

I absolutely love your blog and find the majority of
your post's to be exactly what I'm looking for.
Do you offer guest writers to write content in your case?

I wouldn't mind producing a post or elaborating on a lot of the subjects you write with
regards to here. Again, awesome weblog!

Ebony's picture

It's remarkable to pay a visit this web page and reading the views of all colleagues on the topic of this piece of writing, while I am also
zealous of getting knowledge.

miracle skin transformer reviews's picture

I absolutely love your site.. Very nice colors & theme.

Did you develop this site yourself? Please reply back as I'm hoping to create my own site and would like to learn where you got this from or what the theme is called.
Thanks!

nu colon pro's picture

I'm really enjoying the theme/design of your weblog.

Do you ever run into any web browser compatibility problems?

A few of my blog readers have complained about my blog not working correctly in Explorer but looks great in Firefox.
Do you have any suggestions to help fix this issue?

pure body cleanse's picture

With havin so much content do you ever run into any problems of plagorism
or copyright violation? My website has a lot of exclusive content I've either created myself or outsourced but
it appears a lot of it is popping it up all over the web without my permission. Do you know any solutions to help reduce content from
being ripped off? I'd certainly appreciate it.

http://www.prnewswire.com's picture

Hi there to all, the contents present at this web site are really awesome for people
knowledge, well, keep uup the good work fellows.

ずわい蟹's picture

Hey there fantastic blog! Does running a blog similar
to this take a great deal of work? I have no knowledge of programming
but I had been hoping to start my own blog in the near future.
Anyhow, should you have any ideas or techniques for new blog owners please share.

I know this is off topic nevertheless I just wanted to ask.
Cheers!

Watch spy Online's picture

They might give you recommendations on how to deal with them.

facebook application quiz's picture

Fantastic items from you, man. I've be mindful your stuff previous to and you're just extremely excellent.
I actually like what you've got right here, certainly like what you are stating and the
way in which by which you say it. You're making it entertaining and you still take care of to
keep it wise. I can not wait to learn much more from you.
That is actually a wonderful site.

t shirts custom's picture

What's Taking place i am new to this, I stumbled upon this I
have found It absolutely helpful and it has aided me out loads.
I am hoping to give a contribution & assist other customers like its aided me.
Good job.

toko buku online's picture

Along using your copy in the best-selling "Secret Recipes Of Restaurants" cookbook you
can use delights in your individual house.
This program allows your user to choose quite a few different places on thee computer to set up the ebook,
to locate a shortcut on hiis or her desktop, and also to add thhe ebook, when they choose, on the Start Programs menu.
Congratulations, you've written your ebook also known as being
a digital or electronic bok plus the time has arrived at get the word out around the worldwiide web.

com uma galera muito animada's picture

Bate papo cam
O melhor bate papo gratis.

Quer conhecer pessoas engracadas?Quer ter uma boa conversa?

Quer encontrar o amor da sua vida? Quer um encontro?
Nao hesite, este e o Bate papo cam
O melhor bate papo gratis que voce estava procurando,
onde voce pode ver e conversar com milhares de pessoas ao redor do mundo
todos os dias.

Luke's picture

This will help users better find content relevant to their search.

To manipulate buttons in your portfolio, you'll need to layer functions.
If you are looking for easy way to make money online then there is no
other way than google adsense.

stowe landscape gardens's picture

It's genuinely very difficult in this active life to listen news
on TV, thus I simply use web for that purpose, and obtain the most recent
information.

neuro3d.fr's picture

Somebody necessarily lend a hand to make severely articles I might state.

This is the first time I frequented your website page and up to now?
I surprised with the analysis you made to create this actual submit extraordinary.
Great activity!

daliol.com's picture

This is very fascinating, You are a very skilled blogger.

I have joined your feed and look forward to in quest of more of your fantastic post.
Also, I have shared your website in my social networks

trancheuse's picture

Hello, Neat post. There is an issue along with your website in web explorer, could check this?
IE nonetheless is the marketplace chief and a good component
to other people will pass over your magnificent writing because of this
problem.

migliori passeggini's picture

You actually make it seem so easy along with your presentation however I find this matter to be actually something that I believe I'd by no means
understand. It kind of feels too complex and very broad for me.
I'm looking forward for your subsequent post, I will try to get the grasp of it!

humidificateur's picture

Hello There. I found your blog using msn. This is a really well written article.
I'll make sure to bookmark it and come back to read more
of your useful information. Thanks for the post. I will definitely return.

Silke's picture

This post is invaluable. When can I find out more?

lisseu vapeur's picture

Hola! I've been reading your weblog for a while now and finally
got the courage to go ahead and give you a shout out from
Austin Texas! Just wanted to tell you keep up the good work!

buy steroids online's picture

Only after you have thought about the following supplements would
we advise that you think about steroids.

Milos Forman's picture

I pay a quick visit each day some web pages and blogs to read articles or reviews, however this
web site offers feature based content.

http://fasttechbuzz.com/category/windows/'s picture

After I originally commented I appear to have clicked on the -Notify me when new comments are added- checkbox and now each time
a comment is added I receive four emails with the same comment.
Is there a means you are able to remove me from that service?
Kudos!

dien dan rao vat sai gon's picture

Hello to all, since I am actually eager of reading this weblog's post to be updated regularly.
It contains nice information.

Nike Free Run's picture

I am officially pleased with this website and I wished to take the
time to thanks from my heart's base!

Delia's picture

Cuando Alquiler Coches Mallorca Aeropuerto Ceja El Estiercol Abonable

gta 5 money glitch after patch 1.26's picture

Wow, this piece of writing is pleasant, my sister is analyzing
these kinds of things, so I am going to tell her.

Here is my homepage: gta 5 money glitch after patch 1.26

Pages

Add new comment