Basic Installation And Usage Instructions

Obtaining Tinderbox

The latest release of Tinderbox can be downloaded from http://tinderbox.marcuscom.com.

Tinderbox lives in the MarcusCom CVS repository under the portstools module.

Requirements

Recent FreeBSD (development happens on -CURRENT only, but Tinderbox is known to work on RELENG_6 and RELENG_7, too), Perl 5.8 or later (lang/perl5.8), DBD::Mysql Perl module (databases/p5-DBD-mysql41), and MySQL 4.1 or later (databases/mysql41-server).

The web front-ends require php4 (lang/php4), php4-mysql extension (databases/php4-mysql), and PEAR::DB (databases/pear-DB).

The standard web front-end (found in the www subdirectory) requires you have "register_globals=on" in your php.ini. The experimental web front-end (found in the www-exp subdirectory) does not require register_globals to be on, but does require the php4-session extension (www/php4-session).

Installation

By default, Tinderbox expects to do all of its work in /space. This is completely configurable, however. Hence forth, /space will be referred to as ${pb} (think: package build). If you are using an alternative Tinderbox root, just substitute ${pb} with that root directory.

  1. Create directories ${pb} and ${pb}/scripts.
  2. Extract the Tinderbox distribution into ${pb}/scripts.
  3. Run ${pb}/scripts/setup.sh to setup configuration files and initialize the Tinderbox database.
    # cd ${pb}/scripts && ./setup.sh
    If your Tinderbox host does not have administrative access to the database server, you must perform the next few steps by hand. Else, skip to step 7.
  4. Edit ${pb}/scripts/ds.ph for your setup (if you are using the standard web front-end, also edit www/inc_ds.php).
  5. Create the database and database user on the host that you defined within ds.ph. Note: the database can live on the same server as Tinderbox. Just set $DB_HOST to localhost in ds.ph. The Tinderbox user must be granted the following permissions on the Tinderbox database (for security purposes no other permissions should be granted):
  6. Populate the database with the Tinderbox schema:
    mysql -uroot -p -h {$DB_HOST} {$DB_NAME} &< tinderbox.schema
    Where {$DB_HOST} and {$DB_NAME} are the values of $DB_HOST and $DB_NAME from ds.ph respectively. Note: do not do this if you are upgrading! If you do, you will overwrite all of your previous data. If schema changes are required for upgrading, a separate upgrade schema file will be included, and instructions will be available at
    http://tinderbox.marcuscom.com.
  7. Edit ${pb}/scripts/tinderbox.ph for your environment.
  8. Initialize the Tinderbox:
    # cd ${pb}/scripts && ./tc init
  9. Tinderbox can use either NFS or nullfs to mount the required file systems within the build chroots (called Builds in Tinderbox). If you wish to use nullfs, skip to step 12.
  10. Setup the Tinderbox server as an NFS server by adding the following to /etc/exports:
    ${pb} -alldirs -maproot=0:0 localhost
    Note: ${pb} CANNOT be a symlink. It should be a real, fully-qualified path.
  11. Add the following to /etc/rc.conf to enable the NFS client and server:
    nfs_client_enable="YES"
    nfs_server_flags="-u -t -n 20"
    rpcbind_enable="YES"
    nfs_server_enable="YES"
    nfs_reserved_port_only="YES"
  12. Create the required Jails using the ${pb}/scripts/create command. A Jail is nothing more than a version of FreeBSD. For example, to create a jail for FreeBSD 6.3-RELEASE:
    # cd ${pb}/scripts && ./create Jail -j 6.3 -d "FreeBSD 6.3-RELEASE" -t RELENG_6_3_0_RELEASE -u CVSUP
    Note: all Jail names MUST begin with their FreeBSD major version number. That is, the following is an illegal jail name: FreeBSD-6.3.
  13. Create the required PortsTrees using the ${pb}/scripts/create command. A PortsTree is a set of ports you wish to build. A PortsTree does not have to be a complete FreeBSD ports tree. However, all ports within a tree must have all of their dependencies within the same tree.
    For example, to create a portstree that tracks the full FreeBSD ports tree:
    # cd ${pb}/scripts && ./create PortsTree -p FreeBSD -d "FreeBSD ports tree" -w http://www.freebsd.org/cgi/cvsweb.cgi/ports/
  14. Create the required Builds using the ${pb}/scripts/create command. A Build is a combination of a Jail and a PortsTree. The build is the object in which packages are created. To create a build that combines a 6.3-RELEASE Jail with the FreeBSD ports tree:
    # cd ${pb}/scripts && ./create Build -b 6.3-FreeBSD -j 6.3 -p FreeBSD -d "6.3-RELEASE with FreeBSD ports tree"
    Note: the recommended way to name Builds is Jail-PortsTree. All builds must also begin with their FreeBSD major version number.

NOTE: The create script use cvsup12 as their default cvsup mirror. If you would like to use another server, enable cvsup compression, or use an alternatecvsup program use the -H, -C, and -P command line arguments to the create script respectively. For example, to use cvsup2.freebsd.org, enable cvsup compression, and use csup instead of cvsup for all Jail updates, use the following command:
# cd ${pb}/scripts && ./create Jail -j 6-STABLE -d "FreeBSD 6-STABLE" -t RELENG_6 -u CVSUP -C -H cvsup2.freebsd.org -P /usr/local/bin/csup

Upgrading

Tinderbox may undergo a variety of changes between versions. In order to make sure your Tinderbox stays fully operational after copying over a new version's distribution, you should run the ${pb}/scripts/upgrade.sh command:
# cd ${pb}/scripts && ./upgrade.sh

Note: This command may require administrative access to the database. If such access is not available from your Tinderbox host, you may have to load an upgrade schema file manually. The upgrade script will provide such instructions if needed.

Using Tinderbox

To run a Tinderbox build, and track the progress in the database, you must first add the port you wish to build to the database using the ${pb}/scripts/tc application:
# cd ${pb}/scripts && ./tc addPort -b {BUILD} -d {PORT DIRECTORY} -r

Where {BUILD} is the name of the Build for which this port should be built, and {PORT DIRECTORY} is the directory within the PortsTree where this port can be found. For example, to build the GNOME 2 Desktop port for the Build 6.3-FreeBSD:
# cd ${pb}/scripts && ./tc addPort -b 6.3-FreeBSD -d x11/gnome2 -r

Note: a port does not have to be added to the database for Tinderbox to build it. If you just want to do a quick ad hoc port build, forgo the previously mentioned step.

To start a Tinderbox build, use the ${pb}/scripts/tinderbuild command:
# cd ${pb}/scripts && ./tinderbuild -b {BUILD} {PORT DIRECTORY}

For example, to build the GNOME 2 Desktop for the Build 6.3-FreeBSD:
# cd ${pb}/scripts && ./tinderbuild -b 6.3-FreeBSD x11/gnome2

TIP: The example above will run the build in the foreground with all messages and errors echoing to the terminal. To capture all of this, it is recommended to redirect tinderbuild output to a log file. For example:

The tinderbuild script also accepts some additional command line arguments:

-init updates the Jail then updates the Build
-nullfs uses nullfs instead of NFS to mount Jail and PortsTree file systems
-cleanpackages removes all packages already built for the specified Build
-updateports updates the Build's PortsTree (NOTE: dangerous if doing parallel runs with the same PortsTree)
-skipmake skips the Makefile generation stage (NOTE: only use this option if a good Makefile already exists)
-noduds skips the duds file generation stage (NOTE: packages which are forbidden or ignored will be built)
-noclean does not clean up the Build hierarchy after the port build completes
-plistcheck makes any plist verification problems (e.g. leftover files) fatal
-cleandistfiles removes all files and directories in the distfile cache prior to starting the build
-fetch-original ignores the distfile cache, and fetches all distfiles from their respective sources
-nolog disables log analysis code
-trybroken builds ports marked as BROKEN (this does NOT require -noduds)
-jobs starts n number of parallel port builds (NOTE: the default is 1, and for best results should not exceed the number of physical CPUs in the Tinderbox host)
-onceonly only performs one build pass (i.e. tinderbuild Phase 1)
-norebuild do not force a rebuild of packages specified on the command line

Maintenance

To update existing Jails, use the ${pb}/scripts/mkjail command. For example:
# cd ${pb}/scripts && ./mkjail 6.3

The output of the Jail build will go to stdout. The output of the update command (e.g. cvsup) will go to ${pb}/jails/{JAIL}/update.log (where {JAIL} is the name of the Jail in question).

To update existing PortsTrees, use the tc (Tinderbox Controller) application with the updatePortsTree command. For example:
# cd ${pb}/scripts && ./tc updatePortsTree -p FreeBSD

Over time, Builds may become cluttered with old log files and packages. To cleanup old, unreferenced, files, use the ${pb}/scripts/tc application with the tbcleanup command:
# cd ${pb}/scripts && ./tc tbcleanup

If you want to terminate a running tinderbuild, run the command:
# cd ${pb}/scripts && ./tbkill.sh -b BUILD

This will gracefully terminate a running tinderbuild for Build BUILD. If you want to force the tinderbuild to die, then specify the kill signal:
# cd ${pb}/scripts && ./tbkill.sh -b BUILD -s 9

Note: It may take a few minutes after executing a graceful termination of the tinderbuild before all processes exit. This is because the processes are cleaning up the build environment. If, after five minutes, the build is still running, then you should consider killing it with signal 9.

Troubleshooting

If you encounter problems with Tinderbox, it helps to see what is going on inside a Build. Tinderbox operational logs can be found under ${pb}/builds/{BUILD} (where {BUILD} is the Build name). This is where tinderbuild output should be redirected (see Using Tinderbox above). The make.0 and make.1 logs contain the initial build setup for each port. The reason there are two logs is that tinderbuild runs in two phases. The second build phase is identical to the first, and is run to catch any transient problems that may have occurred in the first phase.

The full build log of each port will be copied to ${pb}/logs/{BUILD} (where {BUILD} is the Build name). If the port failed to build successfully, the log will also be copied to ${pb}/errors/{BUILD} (where {BUILD} is the Build name).

Sometimes, the log alone is not sufficient for figuring out why a port failed to build. In such cases, one must also see the port's work directory. To have Tinderbox save this, create an empty file called .keep in the port's directory, and the work directory will be tarred, compressed, and copied to ${pb}/wrkdirs/{BUILD} (where {BUILD} is the Build name).

When it becomes too difficult to figure out the problem based on the wrkdir, it may become necessary to access the Build itself. To do this, touch a file called .sleepme in the port's directory. The moment the .sleepme file is detected by the build system, the port build will suspend just before executing make build. You can access the Build with the command:
# cd ${pb}/scripts && ./enterbuild -b {BUILD} -d {PORT_DIRECTORY}

When you have finished, remove the .sleepme file, and the port build will continue.

Advanced Topics

Alternative Mounting

If you want to mount /ports inside your PortsTree or /src inside your Jail via nullfs or NFS from another location, use -m switch to create.
Example for NFS:
./create PortsTree -p FreeBSD -m server:/directory
./create Jail -j 6-FreeBSD -m server:/directory

Example for nullfs:
./create PortsTree -p FreeBSD -m /directory
./create Jail -j 6-FreeBSD -m /directory

Tinderbuild will ensure that these file systems are correctly mounted so you do not need to mount them by your own before calling tinderbuild.

If you want to change this settings later, use:
./tc setPortsMount -p <portstreename> -m <mount path>
./tc setSrcMount -j <jailname> -m <mount path>

Distfile Caching

Caching distfiles in a local repository can greatly improve build times. As long as there is sufficient disk space, enabling a local (or NFS) distfile cache is easy. Just use the following command:
./tc configDistfile -c <mount point>

Where <mount point> is either an NFS specification, or fully qualified path (in the case of nullfs) in which to store downloaded distfiles. For example:

NFS:
./tc configDistfile -c localhost:/space/distfiles

nullfs:
./tc configDistfile -c /d/distfiles

Using Ccache

Another excellent way of accelerating builds is to use the compiler cache, ccache. To use ccache support, you must first create a tar file with ccache and various symlinks within a /opt directory. Your tarball contents should be:

opt
opt/ccache
opt/gcc -> ccache
opt/cc -> ccache
opt/g++ -> ccache
opt/c++ -> ccache

This tarball must be called ccache.tar, and be placed in the Jail directory for each Jail that will use ccache (e.g. ${pb}/jails/6.3).

Once the tarball is created, run (-e for enabling, -d for disabling, -c specifies directory, -s maximal size):
./tc configCcache -e -c /ccache -s 2G

Then run your builds as you normally would. To debug ccache, add -l /ccache.log switch to the command.

Then, in the root of each build directory, there will be a ccache.log that will let you know if the cache is working.

Customizing the Environment

It is possible to export environment variables on a per-Jail, per-PortsTree, and/or per-Build basis. This done by creating a file named jail.env, portstree.env, or build.env respectively, and placing it in the root of the Jail (${pb}/jails/{JAIL}), PortsTree (${pb}/portstrees/{PORTSTREE}), or Build (${pb}/builds/{BUILD}) in question.

For example, if you want a particular Build (e.g. 6.X-Perl56) to use Perl 5.6.2 instead of Perl 5.8.x, create a file named build.env in ${pb}/builds/6.X-Perl56 that contains the following:

export PERL_VER=5.6.2
export PERL_VERSION=5.6.2

Likewise, if you want to enable debugging for a particular PortsTree (e.g. FreeBSD-debug), create the following portstree.env in ${pb}/portstrees/FreeBSD-debug:

export CFLAGS="-O -g -pipe"
export STRIP=

Configuring port OPTIONS

It is possible to manipulate port OPTIONS on a per-Build basis. To do this, create a directory under which all build OPTIONS subdirectories will go (e.g. ${pb}/options). For each Build that you wish to use to use OPTIONS, create a subdirectory named for that build (e.g. 6-STABLE-FreeBSD). This directory should look like /var/db/ports in that it contains subdirectories for each OPTIONS-supporting port.

For example, if you wanted to build net/wireshark with RTP support for the Build 6-STABLE-FreeBSD, you would have the following directory structure:

${pb}/options/6-STABLE-FreeBSD/wireshark/options

The contents of the wireshark/options file word be:

_OPTIONS_READ=wireshark-0.99.4
WITH_RTP=true
WITH_SNMP=true
WITH_ADNS=true
WITH_PCRE=true

Once this structure is setup, then enable OPTIONS support in Tinderbox, and specify the path to the OPTIONS directory structure:

# cd ${pb}/scripts
# ./tc configOptions -e
# ./tc configOptions -o /options

The configOptions command takes the standard host arguments as well as -e (enable OPTIONS support), -d (disable OPTIONS support), and -o (set OPTIONS source directory tree).

If you would like to interactively choose OPTIONS for your ports, specify the -o option to ./tc addPort. This will perform a make config on all ports, displaying the ncurses OPTIONS dialog. All OPTIONS settings will be automatically saved to the correct location.

Using Hooks

A hook is a callout which gives Tinderbox the ability to run custom code at pre-determined times. For example, you can establish a Hook to run just before a Port is built, after a Build is extracted, before a PortsTree is updated, etc. Pre-condition hooks can even cause an operation to terminate if so desired. To see a list of all available Hooks, run ./tc listHooks. To add a new command for an existing Hook, use the command ./tc updateHookCmd -h <hook> -c <command>. If you wish to disable a Hook, run ./tc updateHookCmd -h <hook> -c (note: do not specify a command after the -c option).

Automating/Queuing Port Builds

If you want to use tinderbox to test many different ports one after the other you probably want tinderd. tinderd runs as a daemon and looks to see if something was added to the ports to build queue. You can add different ports for different builds for different hosts with different priorities. tinderd will automatically pick up the port with the highest priority for its host and starts building it. That repeats until the queue is empty. After the queue empties tinderd will sleep for a configurable amount of time (default: 120 seconds) thereafter it starts searching for new queue entries again. If you need tinderd to check the queue before the sleep timer has expired, send the tinderd process a SIGHUP:
# kill -HUP {PID of tinderd}

Where {PID of tinderd} is the process ID of the tinderd script as seen in the output of ps(1)

To set it up you must first add a Tinderbox host to the database:
# cd ${pb}/scripts && ./tc addHost

Right after that you can start tinderd (it will stay in foreground by default).If you want to run tinderd automatically when the system boots, copy the include ${pb}/etc/rc.d/tinderd.sh script to /usr/local/etc/rc.d. Be sure to check out the various rc.conf variables documented in this script before using it.
Now use:
# cd ${pb}/scripts && ./tc addBuildPortsQueueEntry -b {BUILD} -d {PORT DIRECTORY}

To add a port to the queue. tinderd will automatically pick it up, run a tinderbuild on it, and will delete the entry after tinderbuild completed. (Where {BUILD} is a Build name, and {PORT DIRECTORY} is a directory under {BUILDS}'s PortsTree (e.g. x11/gnome2).)

NOTE: You must be running the www-exp web front-end to take advantage of the following.

To use tinderd scheduling via the web you must first create a User:
# cd ${pb}/scripts && ./tc addUser -u {USER} -e {EMAIL} -p {PASSWORD} -w

(Where {USER} is a username, {EMAIL} is the user's email address, and {PASSWORD} is the user's password for Tinderbox web access.)

If you want to enable web access for a previously created user, you must update that user's account to give them a password as well as web access:
# cd ${pb}/scripts && ./tc updateUser -u {USER} -e {EMAIL} -p {PASSWORD} -w

(Where {USER} is a username, {EMAIL} is the user's email address, and {PASSWORD} is the user's password for Tinderbox web access.)

Then you have to define one web administrator who has full rights on all Hosts/Builds and is the only account that can add other users:
# cd ${pb}/scripts && ./tc setWwwAdmin -u {USER}

(Where {USER} is the web administrator's username.)

After that, just browse to the Tinderbox web site with your web browser and login with {USER} and {PASSWORD}. You can now create and modify other users easily by using the Add User or Modify User links.

$MCom: tinderweb/README.html,v 1.17 2008/06/02 06:36:41 ade Exp $