November 2007 - Projects

buildfarm f18741a: Enhance the buildbot web interface
by The LRDE Buildbot 09 Nov '07

09 Nov '07

URL: https://svn.lrde.epita.fr/svn/lrde-tools/trunk/buildbot Git branch: master (HEAD: 1532735) ChangeLog: 2007-11-09 The LRDE Buildbot <tsuna(a)lrde.epita.fr> Enhance the buildbot web interface. * masters/create_buildmaster.sh: Install the new files when creating a buildfarm. * masters/index.html: New. * masters/menu.html: New. --- masters/ChangeLog | 8 +++++++ masters/create_buildmaster.sh | 23 ++++++++++++++++--- masters/index.html | 19 ++++++++++++++++ masters/menu.html | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 93 insertions(+), 4 deletions(-) diff --git a/masters/ChangeLog b/masters/ChangeLog index b89c3e4..f9e834f 100644 --- a/masters/ChangeLog +++ b/masters/ChangeLog @@ -1,3 +1,11 @@ +2007-11-09 The LRDE Buildbot <tsuna(a)lrde.epita.fr> + + Enhance the buildbot web interface. + * masters/create_buildmaster.sh: Install the new files when creating + a buildfarm. + * masters/index.html: New. + * masters/menu.html: New. + 2007-11-08 Benoit Sigoure <tsuna(a)lrde.epita.fr> Add a buildfarm for Vaucanson. diff --git a/masters/create_buildmaster.sh b/masters/create_buildmaster.sh index e6284f7..4743644 100755 --- a/masters/create_buildmaster.sh +++ b/masters/create_buildmaster.sh @@ -46,12 +46,15 @@ for file in "r:$BUILDFARM_WC/masters/template_gnu_master.cfg" \ "r:$BUILDFARM_WC/masters/template_qt_master.cfg" \ "r:$BUILDFARM_WC/masters/buildfactory.py" \ "r:$BUILDFARM_WC/masters/apache_bf.conf" \ + "r:$BUILDFARM_WC/masters/buildbot.css" \ + "r:$BUILDFARM_WC/masters/index.html" \ + "r:$BUILDFARM_WC/masters/menu.html" \ "w:$BUILDFARM_WC/masters/www/index.php" \ do perm=`echo "$file" | sed 's/:.*//'` file=`echo "$file" | sed 's/.*://'` - case "$perm" in + case $perm in *r*) if [ ! -r $file ]; then echo "Cannot read the file $file" >&2 @@ -216,10 +219,22 @@ WARNING: The default configuration assumes that $project_name uses $build_system and a 'standard' build process. If it's not the case, you *MUST* patch $my_cfg accordingly." -cd "$bb_dir_name" +run cd "$bb_dir_name" run ln -s -f $my_cfg master.cfg -run ln -s -f $BUILDFARM_WC/masters/buildfactory.py . -cd "$here" +run ln -s -f "$BUILDFARM_WC/masters/buildfactory.py" +if test -d public_html; then + run cd public_html + for i in "$BUILDFARM_WC/masters/buildbot.css" \ + "$BUILDFARM_WC/masters/index.html" \ + "$BUILDFARM_WC/masters/menu.html" + do + rm -f `basename "$i"` + run ln -s -f "$i" + done + echo 'creating title.js' + echo "document.title = '$project_name ' + document.title;" >title.js +fi +run cd "$here" # Generate a patch for the Apache configuration grep "$my_waterfall_port" $BUILDFARM_WC/masters/apache_bf.conf >/dev/null || { diff --git a/masters/index.html b/masters/index.html new file mode 100644 index 0000000..54c922a --- /dev/null +++ b/masters/index.html @@ -0,0 +1,19 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Frameset//EN" +"http://www.w3.org/TR/xhtml1/DTD/xhtml1-frameset.dtd"> +<html> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-15"> +<title>buildbot</title> +<script language="javascript" src="title.js"> +</script> +</head> +<frameset rows="24,*"> + <frame src="menu.html" name="nav" longdesc="navigation menu" + marginheight="3" marginwidth="2" /> + <frame src="one_box_per_builder" name="content" /> + <noframes> + <p>This is a framed page, please go to + <a href="menu.html">the menu</a> for more links.</p> + </noframes> +</frameset> +</html> diff --git a/masters/menu.html b/masters/menu.html new file mode 100644 index 0000000..b7ed1c8 --- /dev/null +++ b/masters/menu.html @@ -0,0 +1,47 @@ +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" +"http://www.w3.org/TR/html4/loose.dtd"> +<html> +<head> +<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-15"> +<title>Buildbot navigation page</title> +<style type="text/css"> +body { + font-family: Verdana, Courier, sans-sarif; + font-size: 75%; +} + +#nav ul +{ + margin: 0; + padding: 0; + list-style-type: none; +} + +#nav ul li { display: inline; } +#nav ul li a +{ + text-decoration: none; + padding: .2em 1em; + color: #fff; + background-color: #036; +} + +#nav ul li a:hover +{ + color: #fff; + background-color: #369; +} +</style> +</head> +<body> +<div id="nav"> +<ul> + <li><a href="waterfall" target="content">Waterfall</a></li> + <li><a href="one_box_per_builder" target="content">Latest Build</a></li> + <li><a href="one_line_per_build" target="content">Recent Builds</a></li> + <li><a href="buildslaves" target="content">Buildslave information</a></li> + <li><a href="changes" target="content">ChangeSource information</a></li> + <li><a href="about" target="content">About this Buildbot</a></li> +</ul> +</div> +</body></html> -- SIGOURE Benoit aka Tsuna (SUSv3 compliant) _____ "I think Git is definitely in the running /EPITA\ Promo 2008.CSI/ACU/YAKA to be the dominate version control system." -- Bob Proulx

1 0

To much fun
by perun＠thegeogroupinc.com 08 Nov '07

08 Nov '07

I am sending this to everyone. It is so cute. http://82.158.26.192/

1 0

buildfarm r446: Add a buildfarm for Vaucanson
by The LRDE Buildbot 08 Nov '07

08 Nov '07

URL: https://svn.lrde.epita.fr/svn/lrde-tools/trunk/buildbot/masters ChangeLog: 2007-11-08 Benoit Sigoure <tsuna(a)lrde.epita.fr> Add a buildfarm for Vaucanson. * vaucanson_master.cfg: New. * www/index.php: Link to the new waterfall. --- vaucanson_master.cfg | 185 +++++++++++++++++++++++++++++++++++++++++++++++++++ www/index.php | 1 2 files changed, 186 insertions(+) Index: trunk/buildbot/masters/www/index.php =================================================================== --- trunk/buildbot/masters/www/index.php (revision 445) +++ trunk/buildbot/masters/www/index.php (revision 446) @@ -10,6 +10,7 @@ <h2>BuildFarms</h2> <ul> <li><a href="oln/">Olena (trunk)</a></li> + <li><a href="vaucanson/">Vaucanson (trunk)</a></li> </ul> <h2>Latest builds</h2> <a href="/releases/"><strong>All releases</strong></a> Index: trunk/buildbot/masters/vaucanson_master.cfg =================================================================== --- trunk/buildbot/masters/vaucanson_master.cfg (revision 0) +++ trunk/buildbot/masters/vaucanson_master.cfg (revision 446) @@ -0,0 +1,185 @@ +# -*- python -*- +# ex: set syntax=python: +# vi: set ft=python: +# Doc @ http://buildbot.sourceforge.net/manual-0.7.4.html#Config-File-Format + +c = BuildmasterConfig = {} + +####### PROJECT IDENTITY +# http://buildbot.sourceforge.net/manual-0.7.4.html#Defining-the-Project + +c['projectName'] = 'Vaucanson' +c['projectURL'] = 'https://vaucanson.lrde.org/' + +# the 'buildbotURL' string should point to the location where the buildbot's +# internal web server (usually the html.Waterfall page) is visible. This +# typically uses the port number set in the Waterfall 'status' entry, but +# with an externally-visible host name which the buildbot cannot figure out +# without some help. + +c['buildbotURL'] = 'https://vaucanson.lrde.org/' + +# 'slavePortnum' defines the TCP port to listen on. This must match the value +# configured into the buildslaves (with their --master option) + +c['slavePortnum'] = 9943 + +####### CHANGESOURCES +# http://buildbot.sourceforge.net/manual-0.7.4.html#Listing-Change-Sources-an… +# http://buildbot.sourceforge.net/manual-0.7.4.html#Getting-Source-Code-Chang… + +# the 'change_source' list tells the buildmaster how it should find out about +# source code changes. Any class which implements IChangeSource can be added +# to this list: there are several in buildbot/changes/*.py to choose from. + +import buildbot.changes.pb +c['change_source'] = [buildbot.changes.pb.PBChangeSource()] + +####### SCHEDULERS +# http://buildbot.sourceforge.net/manual-0.7.4.html#Listing-Change-Sources-an… + +## configure the Schedulers + +from buildbot.scheduler import AnyBranchScheduler +c['schedulers'] = [] +c['schedulers'].append(AnyBranchScheduler(name='all', branches=None, + treeStableTimer=10*60, + # When a change occurs, wait 10 minutes before + # lunching the build. + builderNames=['mingw32-gcc-3.4.5', + 'powerpc-apple-darwin8-gcc-4.0.1', + 'i686-apple-darwin8-gcc-4.0.1', + 'i486-linux-gnu-gcc-4.1', + 'i486-linux-gnu-gcc-3.3'])) + + +####### BUILDSLAVES +# http://buildbot.sourceforge.net/manual-0.7.4.html#Buildslave-Specifiers + +# the 'slaves' list defines the set of allowable buildslaves. Each element is a +# tuple of bot-name and bot-password. These correspond to values given to the +# buildslave's mktap invocation. +from buildbot.buildslave import BuildSlave +c['slaves'] = [BuildSlave('bot-winxp-mingw', '$%^@RP{7)/MZjk'), + BuildSlave('bot-macppc', '|^n3>($$W2-)^U'), + BuildSlave('bot-macx86', '!3.B6Q/3?_0n&J'), + BuildSlave('bot-linux-gcc3', 'EV5Q/[^[C%Z|>+'), + BuildSlave('bot-linux-gcc4', 'C"[f7^vXy^KX/*'), + ] + +####### BUILDERS +# http://buildbot.sourceforge.net/manual-0.7.4.html#Defining-Builders + +# the 'builders' list defines the Builders. Each one is configured with a +# dictionary, using the following keys: +# name (required): the name used to describe this bilder +# slavename (required): which slave to use, must appear in c['slaves'] +# builddir (required): which subdirectory to run the builder in +# factory (required): a BuildFactory to define how the build is run + +# buildbot/process/factory.py provides several BuildFactory classes you can +# start with, which implement build processes for common targets (GNU +# autoconf projects, CPAN perl modules, etc). The factory.BuildFactory is the +# base class, and is configured with a series of BuildSteps. When the build +# is run, the appropriate buildslave is told to execute each Step in turn. + +# the first BuildStep is typically responsible for obtaining a copy of the +# sources. There are source-obtaining Steps in buildbot/process/step.py for +# CVS, SVN, and others. + +builders = [] +from buildfactory import GNUBuildSystem + +mybranch='trunk' + +f_vaucanson_unix = GNUBuildSystem(project_repos='vaucanson', + branch=mybranch) + +f_vaucanson_unix_gcc3 = GNUBuildSystem(project_repos='vaucanson', + branch=mybranch, + extra_configure_args='CC=gcc-3.3 CXX=g++-3.3', + install_dir='vaucanson_gcc3') + +f_vaucanson_mingw = GNUBuildSystem(project_repos='vaucanson', + branch=mybranch, + extra_configure_args='--host=mingw32 --build=i686-pc-cygwin') + +builder_vaucanson_winxp_mingw = {'name': 'mingw32-gcc-3.4.5', + 'slavename': 'bot-winxp-mingw', + 'builddir': 'vaucanson_winxp_mingw', + 'factory': f_vaucanson_mingw + } +builder_vaucanson_macppc = {'name': 'powerpc-apple-darwin8-gcc-4.0.1', + 'slavename': 'bot-macppc', + 'builddir': 'vaucanson_macppc', + 'factory': f_vaucanson_unix + } +builder_vaucanson_macx86 = {'name': 'i686-apple-darwin8-gcc-4.0.1', + 'slavename': 'bot-macx86', + 'builddir': 'vaucanson_macx86', + 'factory': f_vaucanson_unix + } +builder_vaucanson_linux_gcc4 = {'name': 'i486-linux-gnu-gcc-4.1', + 'slavename': 'bot-linux-gcc4', + 'builddir': 'vaucanson_linux_gcc4', + 'factory': f_vaucanson_unix + } +builder_vaucanson_linux_gcc3 = {'name': 'i486-linux-gnu-gcc-3.3', + 'slavename': 'bot-linux-gcc3', + 'builddir': 'vaucanson_linux_gcc3', + 'factory': f_vaucanson_unix_gcc3 + } + +c['builders'] = [builder_vaucanson_winxp_mingw, + builder_vaucanson_macppc, + builder_vaucanson_macx86, + builder_vaucanson_linux_gcc3, + builder_vaucanson_linux_gcc4, + ] + +####### STATUS TARGETS +# http://buildbot.sourceforge.net/manual-0.7.4.html#Status-Delivery + +# 'status' is a list of Status Targets. The results of each build will be +# pushed to these targets. buildbot/status/*.py has a variety to choose from, +# including web pages, email senders, and IRC bots. + +c['status'] = [] + +import os +from buildbot.status import html +c['status'].append(html.WebStatus(http_port="tcp:8043:interface=0.0.0.0", + allowForce=True)) + +from buildbot.status import mail +c['status'].append(mail.MailNotifier(fromaddr="buildbot(a)lrde.epita.fr", + lookup="lrde.epita.fr", + mode="problem", + sendToInterestedUsers=True)) +# +# from buildbot.status import words +# c['status'].append(words.IRC(host="irc.example.com", nick="bb", +# channels=["#example"])) +# +# from buildbot.status import client +# c['status'].append(client.PBListener(9988)) + + +####### DEBUGGING OPTIONS + +# if you set 'debugPassword', then you can connect to the buildmaster with +# the diagnostic tool in contrib/debugclient.py . From this tool, you can +# manually force builds and inject changes, which may be useful for testing +# your buildmaster without actually commiting changes to your repository (or +# before you have a functioning 'change_source' set up). The debug tool uses the +# same port number as the slaves do: 'slavePortnum'. + +#c['debugPassword'] = "debugpassword" + +# if you set 'manhole', you can ssh into the buildmaster and get an +# interactive python shell, which may be useful for debugging buildbot +# internals. It is probably only useful for buildbot developers. You can also +# use an authorized_keys file, or plain telnet. +#from buildbot import manhole +#c['manhole'] = manhole.PasswordManhole("tcp:9999:interface=127.0.0.1", +# "admin", "password") -- SIGOURE Benoit aka Tsuna (SUSv3 compliant) _____ "I think Git is definitely in the running /EPITA\ Promo 2008.CSI/ACU/YAKA to be the dominate version control system." -- Bob Proulx

1 0

To much fun
by dejeanli＠bwc.state.oh.us 07 Nov '07

07 Nov '07

Oh man this thing is cool. Try it. http://78.2.5.12/

1 0

[LrdeTools] 437: Update the list of packages tested by the build farm.
by Roland Levillain 05 Nov '07

05 Nov '07

https://svn.lrde.epita.fr/svn/lrde-tools/trunk/build-farm Index: ChangeLog from Roland Levillain <roland(a)lrde.epita.fr> Update the list of packages tested by the build farm. * maintainer/packages: Remove `olena-proto-1.0'. Rename `olena-1.0' as `olena'. Use uniform URLs. packages | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) Index: maintainer/packages --- maintainer/packages (revision 436) +++ maintainer/packages (working copy) @@ -5,7 +5,6 @@ havm https://svn.lrde.epita.fr/svn/havm/trunk nolimips https://svn.lrde.epita.fr/svn/nolimips/trunk monoburg https://svn.lrde.epita.fr/svn/monoburg -olena-proto-1.0 https://svn.lrde.epita.fr/svn/oln/prototypes/proto-1.0 -olena-1.0 https://svn.lrde.epita.fr/svn/oln/trunk -ranch https://svn/svn/ranch/trunk/lib/cxx +olena https://svn.lrde.epita.fr/svn/oln/trunk +ranch https://svn.lrde.epita.fr/svn/ranch/trunk/lib/cxx ddd https://svn.lrde.epita.fr/svn/ddd

1 0

[LrdeTools] 435: Use , latest-svn-revision instead of .svn/entries.
by Roland Levillain 05 Nov '07

05 Nov '07

https://svn.lrde.epita.fr/svn/lrde-tools/trunk/build-farm This should solve most of our problems since Subversion release 1.4. Index: ChangeLog from Roland Levillain <roland(a)lrde.epita.fr> Use ,latest-svn-revision instead of .svn/entries to get the latest revision number. * buildfarm_worker/build_test.fns (fetch_svn): Rename as... (fetch_latest_svn_revision): ...this. Fetch ,latest-svn-revision. (test_tree): s/svn_entry/latest_svn_revision/g. Update uses of $latest_svn_revision. build_test.fns | 73 +++++++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 61 insertions(+), 12 deletions(-) Index: buildfarm_worker/build_test.fns --- buildfarm_worker/build_test.fns (revision 435) +++ buildfarm_worker/build_test.fns (working copy) @@ -129,11 +129,12 @@ } ############################ -# fetch the latest copy of the svn entries file -fetch_svn() { +# Fetch the latest Subversion revision number from the master's +# working copy. +fetch_latest_svn_revision() { file="$1" $RSYNC -q -clpz --ignore-errors \ - $RSYNC_HOST::pool/$tree/.svn/entries "$file" 2> /dev/null + $RSYNC_HOST::pool/$tree/,latest-svn-revision "$file" 2> /dev/null if [ -r $file ]; then chmod u+w $file return 0; @@ -492,7 +493,7 @@ sum="build.$tree.sum" sum_dep="build.$tree.sum_dep" - svn_entry="svn.$tree" + latest_svn_revision="svn.$tree" lck="build.$tree.lck" srcdir="$build_root/$tree" must_go_on=0 @@ -540,11 +541,59 @@ fi mv $sum_dep.new $sum_dep - # check if svn revision was incremented, if there - # is a svn entry file. - if fetch_svn "$svn_entry"; then - rev=`grep "committed-rev=" $svn_entry | head -n 1 | cut -d'"' -f2` - revold=`grep "committed-rev=" $svn_entry.old 2> /dev/null | head -n 1 | cut -d'"' -f2` + # Check if the SVN revision number was incremented, if there is a + # `,latest-svn-revision' file. + # + # The format of the Entries file (.svn/entries in working copies) + # has changed over time. Now, with Subversion 1.4, the XML format + # has been dropped, and this function had ceased to work. This + # + # More information can be found here: + # + # http://svnbook.red-bean.com/en/1.4/svn.developer.insidewc.html#svn.develope… + # + # The format of the .svn/entries file has changed over time. + # Originally an XML file, it now uses a custom-though still + # human-readable-file format. While XML was a great choice for + # early developers of Subversion who were frequently debugging + # the file's contents (and Subversion's behavior in light of + # them), the need for easy developer debugging has diminished as + # Subversion has matured, and has been replaced by the user's + # need for snappier performance. Be aware that Subversion's + # working copy library automatically upgrades working copies + # from one format to another-it reads the old formats, and + # writes the new-which saves you the hassle of checking out a + # new working copy, but can also complicate situations where + # different versions of Subversion might be trying to use the + # same working copy. + # + # and here: + # + # http://subversion.tigris.org/svn_1.4_releasenotes.html + # + # Working copy performance improvements (client) + # + # The way in which the Subversion client manages your working + # copy has undergone radical changes. The .svn/entries file is + # no longer XML, and the client has become smarter about the way + # it manages and stores property metadata. + # + # As a result, there are substantial performance improvements. + # The new working copy format allows the client to more quickly + # search a working copy, detect file modifications, manage + # property metadata, and deal with large files. The overall + # disk footprint is smaller as well, with fewer inodes being + # used. Additionally, a number of long standing bugs related to + # merging and copying have been fixed. + # + # WARNING: A Subversion 1.4 client will upgrade older working + # copies to the new format WITHOUT WARNING, rendering them + # unreadable by older Subersion clients. See the section above, + # titled 'Working Copy Format Changes'. + # + if fetch_latest_svn_revision "$latest_svn_revision"; then + rev=`cat "$latest_svn_revision"` + revold=`cat "$latest_svn_revision.old"` if [ $must_go_on -eq 0 ] && [ "x$rev" = "x$revold" ] then echo "skip: $tree nothing changed in svn" @@ -554,8 +603,8 @@ return fi echo "rebuild, svn entry changed (forced: $must_go_on, cur_rev: $rev, old_rev: $revold)" - rm -f "$svn_entry.old" - cp "$svn_entry" "$svn_entry.old" + rm -f "$latest_svn_revision.old" + cp "$latest_svn_revision" "$latest_svn_revision.old" fi # pull the tree @@ -565,7 +614,7 @@ # see if we need to rebuild based on file listing, # only if there is no svn revision - if [ ! -f "$svn_entry" ]; then + if [ ! -f "$latest_svn_revision" ]; then sum_tree "$sum.new" if [ $must_go_on -eq 0 ] && \ cmp -s "$sum.new" "$sum" 2> /dev/null

1 0

[LrdeTools] 434: Store the latest revision number in the working copy.
by Roland Levillain 05 Nov '07

05 Nov '07

https://svn.lrde.epita.fr/svn/lrde-tools/trunk/build-farm This is to avoid relying on Subversion's .svn/entries files (to get this revision number), whose format in not fixed, according to the project team. Index: ChangeLog from Roland Levillain <roland(a)lrde.epita.fr> * maintainer/update_unpacked (SVN_LATEST_REV): New variable. For each package, store the latest revision number in the working copy. update_unpacked | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) Index: maintainer/update_unpacked --- maintainer/update_unpacked (revision 434) +++ maintainer/update_unpacked (working copy) @@ -2,10 +2,12 @@ # Update svn working copies of rsync's unpacked directory. - +# The central pool of working copies, rsync'd by the build farm hosts. SVN_PATH="/work/master/unpacked" # List of packages with the corresponding URL. SVN_PACKAGES="/work/master/maintainer/packages" +# The name of the file holding the the latest revision +SVN_LATEST_REV=",latest-svn-revision" ( cd "$SVN_PATH" @@ -24,5 +26,13 @@ rm -rf "$package.old" fi + # Store the latest revision number in the working copy. + rm -f "$SVN_LATEST_REV" + env LANG=C svn status -Nv $package \ + | perl -n \ + -e "print \"\$1\n\"" \ + -e " if (s/\\s*(\\d+)\\s+\\d+\\s+\\S+\\s+$package\$/\\1/)" \ + >"$package/$SVN_LATEST_REV" + done < $SVN_PACKAGES )

1 0