Developer Blog
allocPSA Source Code Workflow with Darcs
2008-08-04 by Alex Lance
We use darcs for source code version control because it provides spontaneous branching and
cherry-picking. Functionality that is usually considered to be an "add-on" for most DVCS', we
consider to be a necessary and fundamental part of running a distributed development project.
There are two main problems that continually confront me as technical lead on allocPSA:
development and deployment.
Development
* I develop functionality for alloc, and;
* I also have developers that submit patches to me.
My needs are simple. I need a good SCM tool that allows me to review and test patches that are
submitted, and provides me with the ability to revert, or hopefully apply and send changes on to a
central development repository.
Most DVCS can help you here, but darcs is particularly effective because of its solid email
hooks. For example, the workflow that we have adopted is to have developers commit their work
locally and then `darcs send` their patches to a mailing list.
Once I review their patch, I can apply it to a local branch of mine for further
testing[1], and then send it on to a central development repository[2], where it gets automatically
applied to the canonical development branch[3] and becomes publically available for download.
This mechanism has the following advantages:
* all source code patches that are sent to the mailing list are archived by the mailing list in
perpetuity.
* I am not required to have access to, or even know the locations of, my
fellow developers source code branches.
* Having patches sent to a mailing list provides visibility on development
to people that might not necessarily know what's going on otherwise, i.e. managers.
* This workflow allows a clear distinction between patches that have made it "in"
and patches that are "pending".
* And of course it promotes further discussion about the patches on the mailing list with a
consistant subject line etc.
Deployment
The process of developing and testing patches as they get pushed into the development tree is
continual. The stability of the development tree is always a bit of an unknown. This is why we
have a devel alloc site, a staging alloc site, and also a production alloc site.
This allows managers the ability to try out functionality on alloc_devel and say, "hey we like
this feature, can we get it pushed up to alloc_staging?". And once it has been on staging for a
couple of weeks we push the new feature out to production alloc.
Deploying particular changes to these trees used to be an arduous process. The problem is that
developers put patches into the tree in a different order than the order that the managers would
like to see the features promoted in.
As a loose example, if your developers throw 20 patches into devel, and someone would like to
see a feature deployed up to staging that is interleaved within those 20 patches, eg patches 5, 10
and 19, then it is a royal pain in the arse getting just those patches extrapolated up to a higher
branch, i.e. staging.
I mean how do you do it? hg has transplant, but what is that really? It doesn't just pull the
patches across, it creates *new* patches with new hashes, that just happen to have the same contents
as the original patches. That is not good! I would also like to point out that it is a non-trivial
exercise to transplant say 10 particular patches across. I.e. lots of commands, lots of searching
through logs. Ditto with bzr. Ditto with git.
With darcs the solution we have implemented is as follows, since all work that is done must
relate to an issue number in our issue tracker, we prefix each commit log with the issue ID of the
feature.
This means you might have a stack of patches in devel that look like this:
* 11522 Added code for product module.
* 11598 Added taxTfID to db_data.sql to bring DB in line with patches
* 11598 Removed permissionIDs from db_data.sql
* 13203 Nuked colspan=3 from Project->Task List. Renamed parenTaskDropdown to parentTaskDropdown.
* 13153 Added Project, Parent Task and Task Type to en masse Task editing. Added clicks to the table rows.
* 12946 Repeating expenses should no longer go into an infinite loop.
* 11589 Engineers can now see a list of project managers on the project details page.
* 12469 'Potential' projects can now have tasks added to them.
* 13153 Smoothed over UI for en masse task editing. Added new task editing options.
* 12209 Time sheet items for task that have overrun their estimates are now really obviously coloured red.
* 12209 Time estimate overruns are now notified on "timesheet waiting for approval" emails.
* 12586 Expansion of auto-resizing comment boxes is now capped.
* 8532 Project import/export facility for GNOME Planner and CSV
* 13153 The task list can now be used to make changes to tasks en masse.
* 11406 Allow comments and emails with attachments on time sheets.
If I want to promote all the patches that relate to issue number 13153 up to staging, I just go
over to alloc_stage and pull up the changes that I want from devel:
$ darcs pull -p 13153
(it remembers that we last pulled changes from alloc_devel, so it uses that as the default
source branch again)
That's one command. Done. No additional commits or merging required. No newly generated hashes.
If for some reason feature 13153 doesn't work out, it is equally trivial to backout the change:
$ darcs unpull -p 13153
Bang. Presto. That change that was spread across three patches has now been removed.
This unbelievably useful feature makes darcs worth its weight in gold to us. It is IMO
"killer-app" funtionality, and none of the other DVCS' have anywhere near that ease of use for
deployment.
Summary
There are many other reasons to like darcs, the consistency and predictability of the commands.
The slick, interactive UI. The simplicity and transparency of the actions that you are actually
performing. Darcs amend, the hooks, the interactive-committing-by-default, the high value it
encourages you to place on commits of quality - i.e. small commits that do exactly what their commit
log say they do.
There are one or two downsides to using darcs, for instance for some repositories it runs
slowly, although I've never had a problem. Another downside is that the darcs development community
appears to be quite small. I do wonder how important that is though, do I need a new grep every few
months? Do I need a continually updated bash? Darcs works for me, simply and effectively. I don't
need it to jump through hoops, I just need it to keep doing what it says it will do on the box.
Notes
[1] I use the email client mutt, if you put the following lines into your ~/.muttrc it will
allow you to view patches, and also apply them to a local branch with one keystroke:
auto_view text/x-patch text/x-darcs-patch
macro pager A "<pipe-entry>umask 022;darcs apply --verbose --mark-conflicts --repodir ~/dev"
[2] You can configure `darcs send` to digitally sign a patch provided you have GPG set up, and
you place the following two lines into the _darcs/prefs/defaults/ file:
send sign
send to incomingpatches@example.org
[3] I use darcs send on my local testing repository to send the patches, on to the canonical
development repository which has a procmail rule like this:
:0
* ^To: incomingpatches@example.org
|(umask 022; darcs apply --dont-allow-conflicts --repodir /dev/ --verify /dev/allowed_gpg_keys)
This procmail rule checks that the patch has been signed with my special GPG key and if so,
applies it to the devel branch. The devel branch has a hook in its _darcs/prefs/defaults file which
looks like this:
apply posthook echo "$DARCS_PATCHES" | mutt -x -s "new patches applied" em@example.org
apply run-posthook
Which basically gets run after new patches are successfully applied to the devel tree. This
command sends an email back to the mailing list that lets all the developers know that a new patch
has been applied to the development tree.
