CVS & Bugzilla integration | Steve McIntyre
CVS & Bugzilla integration
The most up-to-date page is at here
- 1. Introduction
- 2. CVS configuration
- 3 Bugzilla configuration
- 4 ViewCVS configuration
- 5. MoinMoin configuration
1. Introduction
In the software group at Plasmon
(my employer), we have settled on a very useful combination of free
software to help us manage development. We use
- CVS (the Concurrent Versions
System) for source control; - Bugzilla for tracking bugs;
- ViewCVS for easy
access to our CVS repository via a web browser; - and a much-hacked version of the MoinMoin Wiki to help in
documentation etc. - All of these run on top of the
We have made some small changes to these packages locally to
help integrate them into our development process. With these small
changes, we now have diverse open source tools linked almost
seamlessly:
- CVS check in messages are automatically appended to bug
reports, and also bounced by email to the development team. This
ensures all changes are logged. - The checkin messages in bugzilla are then further linked into
the CVS repository, so that anyone browsing bug reports can not
only see the commit messages, but also the exact changes that were
committed; i.e. the changes that were made to fix a particular
bug.
The work we have done here did not take very long, but was quite
fiddly. We hope that others find it useful - please feel free to
NOTE! I have used example.com throughout
these pages. Be aware that the links I'm using here won't work!
Example.com is reserved for use in documentation and not available for
registration. See RFC 2606, Section 3. You'll (obviously) need to
change config files to make this lot work for you!
2. CVS configuration
CVS is (probably) the world's
most commonly-used software for source code control. It is a
tried-and-tested stable piece of open source software that has been
continuously developed and maintained since 1989.
On our internal network, we use the pserver access
method, as this is most easily configured for the range of client
machines that we commonly use (Linux, FreeBSD and Windows). For
external access, we use the ext method and cvs over ssh -
this is useful as ssh provides a simple secure connection that is both
authenticated and encrypted.
Most of the configuration of CVS is done through the files in the
CVSROOT module, which is itself managed through CVS (of
course!).
2.1 loginfo
The first thing to do is make grab the logs from CVS checkin
messages. To do this, modify the file
CVSROOT/loginfo. The version we use looks (something)
like:
^genstr (echo ""; echo ; date; cat) | mail -s "CVS commit -- genstr" developer1 developer2
^twedmon (echo ""; echo ; date; cat) | mail -s "CVS commit -- twedmon" developer1 developer2
^udo_demo (echo ""; echo ; date; cat) | mail -s "CVS commit -- udo_demo" developer2 developer3
^bugzilla-test (echo ""; echo ; date; cat) | mail -s "CVS commit -- bugzilla-test" developer4
^moinmoin (echo ""; echo ; date; cat) | mail -s "CVS commit -- moinmoin" developer2 developer1 developer4
CVSROOT (echo ""; echo ; date; cat) | mail -s "CVS commit -- CVSROOT" cvs-watchers
DEFAULT (echo ""; echo ; date; cat) | mail -s "CVS commit -- Other modules" developer1
ALL (echo ""; echo ; date; cat) >> /CVSROOT/commitlog
ALL (echo ""; echo "Date: "`date`; cat) | /CVSROOT/bugzilla-watcher %{SVv}
This warrants a little explanation!
- The first column of each line is a regular expression.
- Each file checked in is compared against each regexp.
- For the first one that matches, the rest of that line is
executed, with the commit message passed in through stdin. - There are two special entries:
DEFAULTmatches any commit that has not yet
been matchedALLis executed for all
checkins.
In the above list, project members are mailed about checkins on the
various modules that they work on (genstr, twedmon, udo_demo,
bugzilla-test and moinmoin). CVSROOT is special, and the
commits are sent to a local mailing list called cvs-watcheers. The
first ALL line simply appends all commit messages to a
logfile that we find helpful for searching.
The last line is most important here. It passes all commit messages
through a locally-written script called
bugzilla-watcher. The variable will
evaluate to the name of the user running the script (i.e. the user
running the checkin). The macro %{SVv} expands to a
comma-separated list of files checked in, of the form:
<DIRECTORY> "<FILE1>",<OLDVER1>,<NEWVER1> "<FILE2>",<OLDVER2>,<NEWVER2> ... "<FILEn>",<OLDVERn>,<NEWVERn>
From this information, the bugzilla-watcher script can (with some
work) extract all the information it needs to log all the changes
(filename, old version number and new version number) along with their
commitlog entries. On additions and removals, the version number shows
up as "NONE".
NOTE! The %S entry in the macro is not
standard in CVS! We have added it in to our version of CVS (patch for 1.11 ; patch for 1.12.7). It behaves exactly as the %s
macro, but it adds quotes around the file name which makes dealing
with silly filenames containing spaces much easier. As I'm
also the Debian CVS maintainer, current Debian CVS packages also have
this patch applied. If you can't or don't want to apply this patch and
build your own CVS binary, use %s instead and modify the
bugzilla-watcher script - you'll just have problems with those silly
filenames. See the
documentation for more information about the macros available.
2.2 rcsinfo
CVSROOT/rcsinfo controls templates used on commits and
imports. The format of the rcsinfo file is similar to the loginfo file
- the first column is a regexp; when it matches the checkin directory,
use the filename on the rest of the line as a template. We have added
a simple single template called rcstemplate; if you want
different messages for different modules, this is easy enough. Our
rcsinfo file looks like:
DEFAULT /CVSROOT/rcstemplate
2.3 rcstemplate
CVSROOT/rcstemplate is the simple template that is
given to the user when they are asked for a CVS commit
message. What we use is:
BugID: CVS: Please fill in the above fields with information about your checkin. CVS: Make sure that you know for certain what you are checking in.
As in the default CVS commit message, lines beginning with "CVS:" are
stripped from the message before checkin. We use the
BugID: tag later in the bugzilla-watcher script;
developers are told to add the Bugzilla bug number(s) on that line,
and this is how we track where the commit log messages are meant to
go.
NOTE! If people check in using cvs commit, then they will not see this template. Also, the
-m "message"
templates will only be applied to new checkouts; existing checkouts
will not start to use it, even if people cvs update.
2.4 checkoutlist
CVSROOT/checkoutlist contains a list of files that should
be tracked in the CVSROOT special modules alongside the
files that CVS knows about internally. Ours lists
rcstemplate bugzilla-watcher
so that these local files are tracked too.
2.5 bugzilla-watcher
CVSROOT/bugzilla-watcher is a locally-written perl
script. It accepts the information about each changed file that is
passed in from CVS loginfo, and reformats that information into
something more useful. It then packages this up into an email which is
sent to the bugzilla system. I didn't write the script, and it's
grotty perl :-). It's here.
This outputs email like:
To: bugzilla-interface@example.com Subject: [Bug 522] CVS Checkin From: CVS User <user@example.com> CVS COMMIT LOG MESSAGE: Make the stripspaces code work in preview mode too. Needs changes to send_editor() - modify the contents of the edit box send_page() - modify the contents of the preview pane do_savepage() - make sure we use the stripspaces option on preview too BRANCH: HEAD FILES CHECKED IN: moinmoin/Page.py 1.12 1.13 moinmoin/PageEditor.py 1.6 1.7 moinmoin/wikiaction.py 1.5 1.6
which is sent to the bugzilla system, ready to be further parsed and
dealt with.
3. Bugzilla configuration
Bugzilla is the defect-tracking
system, or bug tracker, that was written to support the development of
Netscape products and more recently
tool that compares very favourably with the expensive proprietary
systems available today. It is especially popular among users and
developers of other Open Source projects, but also has a growing
following in commercial environments.
Bugzilla is a large perl application that implements a web interface
on top of a backend SQL database. As well as the web frontend that
most users are aware of, there has also been some work done to add an
email interface that can be used to manipulate bugs. In the contrib
directory of the bugzilla distribution, read the file README.Mailif
for instructions on how to set this up. We have done exactly as
specified here, and set up a user account with access to the bugzilla
database. Procmail for that user pipes incoming mail into the
bugzilla_email_append.pl script. We don't use the
bug_email.pl script normally.
For reference, there is a bug open against bugzilla right now that
covers the patches here, see
3.1 bugzilla_email_append.pl
We have slightly modified the input script
bugzilla_email_append.pl from the original. We have added
the capability to enter the same commit message against multiple bug
numbers. We also cope with the case where the user checking in does
not have a bugzilla account (e.g. on a build machine with a common
login used by many users). The bugzilla_email_append.pl
will simply append the body of any email it receives to the bug
number(s) specified in the Subject: line, which is quite
straightforward.
Our version of this script is
3.2 globals.pl
Now that we have CVS commit messages added into the appropriate bug
reports, the logical next step is to link those commit messages back
into CVS itself, so that it is possible to easily view the changes
committed at each step. We have ViewCVS installed
on the CVS server, which is a simple-to-use web front end for CVS. By
making bugzilla parse the comments it prints when viewing bug reports,
we can turn the files and versions in the checkin comments into links
into the CVS repository. The code that talks to the database and turns
bug reports into HTML is in globals.pl. We hook into the
code in quoteUrls() that already parses the comments, and
add a chunk of code that converts the format in the email above into
something like:
CVS COMMIT LOG MESSAGE: Make the stripspaces code work in preview mode too. Needs changes to send_editor() - modify the contents of the edit box send_page() - modify the contents of the preview pane do_savepage() - make sure we use the stripspaces option on preview too BRANCH: HEAD FILES CHECKED IN:
| moinmoin/Page.py | 1.12 | 1.13 | View diff |
| moinmoin/PageEditor.py | 1.6 | 1.7 | View diff |
| moinmoin/wikiaction.py | 1.5 | 1.6 | View diff |
Simply clicking on the links here shows you all the changes that were
committed for a given bug. The patch is
quite small, but not especially pretty. (patch for Bugzilla 2.16.5 ; patch for Bugzilla 2.17 and later)
4 ViewCVS configuration
ViewCVS is a Python
package that allows web access to CVS repositories. Configuration is
quite easy, so I won't cover it here. We have made one
converts source and logs into HTML for printing, we have added a new
regexp that matches the BugID tag we use and converts the
bug numbers following it into links back to bugzilla.
HINT:We use several different Apache virtual host
configurations in the system described here. This is thoroughly
recommended, as it makes life much easier for users and admins
alike. Users only have to remember service names (and no trailing
/cgi-bin/foo/bar?code=wibble etc.), and it becomes
possible to rearrange services onto multiple different machines easily
without end-users noticing if real machine names and paths are not
hard-coded everywhere. We currently use one machine to host all of the
services here, but it is aliased as
- www. (normal web pages)
- bugs. (bugzilla interface)
- cvs. (viewcvs interface)
- wiki. (moinmoin wiki interface)
- mail. (internal mailing lists)
5 MoinMoin configuration
See the MoinMoin project page
for now. I have quite a lot of local changes that I'd like to be
merged upstream, but for now I don't have the time to fully document
them here:
- Diffs betweeen any two versions of a file
- PDF output (great for documentation)
- Minor integration with bugzilla - linkify Bug#xxx
- Support for multi-coloured comments in pages to aid in discussions
Steve McIntyre
steve@einval.com
"Can't keep my eyes from the circling sky,
Tongue-tied & twisted, Just an earth-bound misfit, I..."
What did I say then?
French Finance Minister Christine Lagarde tells the Journal it will take much s...
