The latest release of Tinderbox can be downloaded from http://tinderbox.marcuscom.com.
Tinderbox lives in the MarcusCom CVS repository under the portstools module.
Recent FreeBSD (development happens on -CURRENT only, but Tinderbox is known to work on RELENG_6 and RELENG_7, too), Perl 5.8 or later (lang/perl5.8), DBD::Mysql Perl module (databases/p5-DBD-mysql41), and MySQL 4.1 or later (databases/mysql41-server).
The web front-ends require php4 (lang/php4), php4-mysql extension (databases/php4-mysql), and PEAR::DB (databases/pear-DB).
The standard web front-end (found in the www subdirectory) requires you have "register_globals=on" in your php.ini. The experimental web front-end (found in the www-exp subdirectory) does not require register_globals to be on, but does require the php4-session extension (www/php4-session).
By default, Tinderbox expects to do all of its work in /space. This is completely configurable, however. Hence forth, /space will be referred to as ${pb} (think: package build). If you are using an alternative Tinderbox root, just substitute ${pb} with that root directory.
NOTE: The create script use cvsup12 as
their default cvsup mirror. If you would like to use
another server, enable cvsup compression, or use an
alternatecvsup program use the -H, -C, and -P
command line arguments to the create script respectively.
For example, to use cvsup2.freebsd.org, enable
cvsup compression, and use csup instead of cvsup for all
Jail updates, use the following command:
# cd ${pb}/scripts && ./create Jail -j
6-STABLE -d "FreeBSD 6-STABLE" -t RELENG_6 -u
CVSUP -C -H cvsup2.freebsd.org -P
/usr/local/bin/csup
Tinderbox may undergo a variety of changes between versions. In
order to make sure your Tinderbox stays fully operational after
copying over a new version's distribution, you should run the
${pb}/scripts/upgrade.sh command:
# cd ${pb}/scripts && ./upgrade.sh
Note: This command may require administrative access to the database. If such access is not available from your Tinderbox host, you may have to load an upgrade schema file manually. The upgrade script will provide such instructions if needed.
To run a Tinderbox build, and track the progress in the
database, you must first add the port you wish to build to the
database using the ${pb}/scripts/tc
application:
# cd ${pb}/scripts && ./tc addPort -b
{BUILD} -d {PORT DIRECTORY} -r
Where {BUILD} is the name of the Build for which this
port should be built, and {PORT DIRECTORY} is the directory
within the PortsTree where this port can be found. For example, to
build the GNOME 2 Desktop port for the Build
6.3-FreeBSD:
# cd ${pb}/scripts && ./tc addPort -b
6.3-FreeBSD -d x11/gnome2 -r
Note: a port does not have to be added to the database for Tinderbox to build it. If you just want to do a quick ad hoc port build, forgo the previously mentioned step.
To start a Tinderbox build, use the
${pb}/scripts/tinderbuild command:
# cd ${pb}/scripts && ./tinderbuild -b
{BUILD} {PORT DIRECTORY}
For example, to build the GNOME 2 Desktop for the Build
6.3-FreeBSD:
# cd ${pb}/scripts && ./tinderbuild -b
6.3-FreeBSD x11/gnome2
TIP: The example above will run the build in the foreground with all messages and errors echoing to the terminal. To capture all of this, it is recommended to redirect tinderbuild output to a log file. For example:
The tinderbuild script also accepts some additional command line arguments:
| -init | updates the Jail then updates the Build |
| -nullfs | uses nullfs instead of NFS to mount Jail and PortsTree file systems |
| -cleanpackages | removes all packages already built for the specified Build |
| -updateports | updates the Build's PortsTree (NOTE: dangerous if doing parallel runs with the same PortsTree) |
| -skipmake | skips the Makefile generation stage (NOTE: only use this option if a good Makefile already exists) |
| -noduds | skips the duds file generation stage (NOTE: packages which are forbidden or ignored will be built) |
| -noclean | does not clean up the Build hierarchy after the port build completes |
| -plistcheck | makes any plist verification problems (e.g. leftover files) fatal |
| -cleandistfiles | removes all files and directories in the distfile cache prior to starting the build |
| -fetch-original | ignores the distfile cache, and fetches all distfiles from their respective sources |
| -nolog | disables log analysis code |
| -trybroken | builds ports marked as BROKEN (this does NOT require -noduds) |
| -jobs | starts n number of parallel port builds (NOTE: the default is 1, and for best results should not exceed the number of physical CPUs in the Tinderbox host) |
| -onceonly | only performs one build pass (i.e. tinderbuild Phase 1) |
| -norebuild | do not force a rebuild of packages specified on the command line |
To update existing Jails, use the
${pb}/scripts/mkjail command. For example:
# cd ${pb}/scripts && ./mkjail
6.3
The output of the Jail build will go to stdout. The output of the update command (e.g. cvsup) will go to ${pb}/jails/{JAIL}/update.log (where {JAIL} is the name of the Jail in question).
To update existing PortsTrees, use the tc (Tinderbox
Controller) application with the updatePortsTree command.
For example:
# cd ${pb}/scripts && ./tc updatePortsTree -p
FreeBSD
Over time, Builds may become cluttered with old log files and
packages. To cleanup old, unreferenced, files, use the
${pb}/scripts/tc application with the
tbcleanup command:
# cd ${pb}/scripts && ./tc tbcleanup
If you want to terminate a running tinderbuild, run the
command:
# cd ${pb}/scripts && ./tbkill.sh -b
BUILD
This will gracefully terminate a running tinderbuild for Build
BUILD. If you want to force the tinderbuild to die, then
specify the kill signal:
# cd ${pb}/scripts && ./tbkill.sh -b BUILD -s
9
Note: It may take a few minutes after executing a graceful termination of the tinderbuild before all processes exit. This is because the processes are cleaning up the build environment. If, after five minutes, the build is still running, then you should consider killing it with signal 9.
If you encounter problems with Tinderbox, it helps to see what is going on inside a Build. Tinderbox operational logs can be found under ${pb}/builds/{BUILD} (where {BUILD} is the Build name). This is where tinderbuild output should be redirected (see Using Tinderbox above). The make.0 and make.1 logs contain the initial build setup for each port. The reason there are two logs is that tinderbuild runs in two phases. The second build phase is identical to the first, and is run to catch any transient problems that may have occurred in the first phase.
The full build log of each port will be copied to ${pb}/logs/{BUILD} (where {BUILD} is the Build name). If the port failed to build successfully, the log will also be copied to ${pb}/errors/{BUILD} (where {BUILD} is the Build name).
Sometimes, the log alone is not sufficient for figuring out why a port failed to build. In such cases, one must also see the port's work directory. To have Tinderbox save this, create an empty file called .keep in the port's directory, and the work directory will be tarred, compressed, and copied to ${pb}/wrkdirs/{BUILD} (where {BUILD} is the Build name).
When it becomes too difficult to figure out the problem based on
the wrkdir, it may become necessary to access the Build itself. To
do this, touch a file called .sleepme in the port's
directory. The moment the .sleepme file is detected by the
build system, the port build will suspend just before executing
make build. You can access the Build with the
command:
# cd ${pb}/scripts && ./enterbuild -b
{BUILD} -d {PORT_DIRECTORY}
When you have finished, remove the .sleepme file, and the port build will continue.
If you want to mount /ports inside your
PortsTree or /src inside your Jail via
nullfs or NFS from another location, use -m switch to
create.
Example for NFS:
./create PortsTree -p FreeBSD -m
server:/directory
./create Jail -j 6-FreeBSD -m
server:/directory
Example for nullfs:
./create PortsTree -p FreeBSD -m
/directory
./create Jail -j 6-FreeBSD -m
/directory
Tinderbuild will ensure that these file systems are correctly mounted so you do not need to mount them by your own before calling tinderbuild.
If you want to change this settings later, use:
./tc setPortsMount -p <portstreename> -m
<mount path>
./tc setSrcMount -j <jailname> -m <mount
path>
Caching distfiles in a local repository can greatly improve
build times. As long as there is sufficient disk space, enabling a
local (or NFS) distfile cache is easy. Just use the following
command:
./tc configDistfile -c <mount point>
Where <mount point> is either an NFS specification, or fully qualified path (in the case of nullfs) in which to store downloaded distfiles. For example:
NFS:
./tc configDistfile -c
localhost:/space/distfiles
nullfs:
./tc configDistfile -c /d/distfiles
Another excellent way of accelerating builds is to use the compiler cache, ccache. To use ccache support, you must first create a tar file with ccache and various symlinks within a /opt directory. Your tarball contents should be:
opt opt/ccache opt/gcc -> ccache opt/cc -> ccache opt/g++ -> ccache opt/c++ -> ccache
This tarball must be called ccache.tar, and be placed in the Jail directory for each Jail that will use ccache (e.g. ${pb}/jails/6.3).
Once the tarball is created, run (-e for enabling,
-d for disabling, -c specifies directory,
-s maximal size):
./tc configCcache -e -c /ccache -s 2G
Then run your builds as you normally would. To debug ccache, add -l /ccache.log switch to the command.
Then, in the root of each build directory, there will be a ccache.log that will let you know if the cache is working.
It is possible to export environment variables on a per-Jail, per-PortsTree, and/or per-Build basis. This done by creating a file named jail.env, portstree.env, or build.env respectively, and placing it in the root of the Jail (${pb}/jails/{JAIL}), PortsTree (${pb}/portstrees/{PORTSTREE}), or Build (${pb}/builds/{BUILD}) in question.
For example, if you want a particular Build (e.g. 6.X-Perl56) to use Perl 5.6.2 instead of Perl 5.8.x, create a file named build.env in ${pb}/builds/6.X-Perl56 that contains the following:
export PERL_VER=5.6.2 export PERL_VERSION=5.6.2
Likewise, if you want to enable debugging for a particular PortsTree (e.g. FreeBSD-debug), create the following portstree.env in ${pb}/portstrees/FreeBSD-debug:
export CFLAGS="-O -g -pipe" export STRIP=
It is possible to manipulate port OPTIONS on a per-Build basis. To do this, create a directory under which all build OPTIONS subdirectories will go (e.g. ${pb}/options). For each Build that you wish to use to use OPTIONS, create a subdirectory named for that build (e.g. 6-STABLE-FreeBSD). This directory should look like /var/db/ports in that it contains subdirectories for each OPTIONS-supporting port.
For example, if you wanted to build net/wireshark with RTP support for the Build 6-STABLE-FreeBSD, you would have the following directory structure:
${pb}/options/6-STABLE-FreeBSD/wireshark/options
The contents of the wireshark/options file word be:
_OPTIONS_READ=wireshark-0.99.4 WITH_RTP=true WITH_SNMP=true WITH_ADNS=true WITH_PCRE=true
Once this structure is setup, then enable OPTIONS support in Tinderbox, and specify the path to the OPTIONS directory structure:
# cd ${pb}/scripts
# ./tc configOptions -e
# ./tc configOptions -o /options
The configOptions command takes the standard host arguments as well as -e (enable OPTIONS support), -d (disable OPTIONS support), and -o (set OPTIONS source directory tree).
If you would like to interactively choose OPTIONS for your ports, specify the -o option to ./tc addPort. This will perform a make config on all ports, displaying the ncurses OPTIONS dialog. All OPTIONS settings will be automatically saved to the correct location.
A hook is a callout which gives Tinderbox the ability to run custom code at pre-determined times. For example, you can establish a Hook to run just before a Port is built, after a Build is extracted, before a PortsTree is updated, etc. Pre-condition hooks can even cause an operation to terminate if so desired. To see a list of all available Hooks, run ./tc listHooks. To add a new command for an existing Hook, use the command ./tc updateHookCmd -h <hook> -c <command>. If you wish to disable a Hook, run ./tc updateHookCmd -h <hook> -c (note: do not specify a command after the -c option).
If you want to use tinderbox to test many different ports one
after the other you probably want tinderd.
tinderd runs as a daemon and looks to see if something was
added to the ports to build queue. You can add different
ports for different builds for different hosts with different
priorities. tinderd will automatically pick up the port
with the highest priority for its host and starts building it. That
repeats until the queue is empty. After the queue empties
tinderd will sleep for a configurable amount of time
(default: 120 seconds) thereafter it starts searching for new queue
entries again. If you need tinderd to check the queue before the
sleep timer has expired, send the tinderd process a SIGHUP:
# kill -HUP {PID of tinderd}
Where {PID of tinderd} is the process ID of the tinderd script as seen in the output of ps(1)
To set it up you must first add a Tinderbox host to the
database:
# cd ${pb}/scripts && ./tc addHost
Right after that you can start tinderd (it will stay in
foreground by default).If you want to run tinderd
automatically when the system boots, copy the include
${pb}/etc/rc.d/tinderd.sh script to
/usr/local/etc/rc.d. Be sure to check out the various
rc.conf variables documented in this script before using
it.
Now use:
# cd ${pb}/scripts && ./tc
addBuildPortsQueueEntry -b {BUILD} -d {PORT
DIRECTORY}
To add a port to the queue. tinderd will automatically pick it up, run a tinderbuild on it, and will delete the entry after tinderbuild completed. (Where {BUILD} is a Build name, and {PORT DIRECTORY} is a directory under {BUILDS}'s PortsTree (e.g. x11/gnome2).)
NOTE: You must be running the www-exp web front-end to take advantage of the following.
To use tinderd scheduling via the web you must first
create a User:
# cd ${pb}/scripts && ./tc addUser -u
{USER} -e {EMAIL} -p {PASSWORD} -w
(Where {USER} is a username, {EMAIL} is the user's email address, and {PASSWORD} is the user's password for Tinderbox web access.)
If you want to enable web access for a previously created user,
you must update that user's account to give them a password as well
as web access:
# cd ${pb}/scripts && ./tc updateUser -u
{USER} -e {EMAIL} -p {PASSWORD} -w
(Where {USER} is a username, {EMAIL} is the user's email address, and {PASSWORD} is the user's password for Tinderbox web access.)
Then you have to define one web administrator who has full
rights on all Hosts/Builds and is the only account that can add
other users:
# cd ${pb}/scripts && ./tc setWwwAdmin -u
{USER}
(Where {USER} is the web administrator's username.)
After that, just browse to the Tinderbox web site with your web browser and login with {USER} and {PASSWORD}. You can now create and modify other users easily by using the Add User or Modify User links.
$MCom: tinderweb/README.html,v 1.17 2008/06/02 06:36:41 ade Exp $