Developer Blog

allocPSA Source Code Workflow with Darcs

2008-08-04 by Alex Lance We use darcs for source code version control because it provides spontaneous branching and cherry-picking. Functionality that is usually considered to be an "add-on" for most DVCS', we consider to be a necessary and fundamental part of running a distributed development project. There are two main problems that continually confront me as technical lead on allocPSA: development and deployment. Development * I develop functionality for alloc, and; * I also have developers that submit patches to me. My needs are simple. I need a good SCM tool that allows me to review and test patches that are submitted, and provides me with the ability to revert, or hopefully apply and send changes on to a central development repository. Most DVCS can help you here, but darcs is particularly effective because of its solid email hooks. For example, the workflow that we have adopted is to have developers commit their work locally and then `darcs send` their patches to a mailing list. Once I review their patch, I can apply it to a local branch of mine for further testing[1], and then send it on to a central development repository[2], where it gets automatically applied to the canonical development branch[3] and becomes publically available for download. This mechanism has the following advantages: * all source code patches that are sent to the mailing list are archived by the mailing list in perpetuity. * I am not required to have access to, or even know the locations of, my fellow developers source code branches. * Having patches sent to a mailing list provides visibility on development to people that might not necessarily know what's going on otherwise, i.e. managers. * This workflow allows a clear distinction between patches that have made it "in" and patches that are "pending". * And of course it promotes further discussion about the patches on the mailing list with a consistant subject line etc. Deployment The process of developing and testing patches as they get pushed into the development tree is continual. The stability of the development tree is always a bit of an unknown. This is why we have a devel alloc site, a staging alloc site, and also a production alloc site. This allows managers the ability to try out functionality on alloc_devel and say, "hey we like this feature, can we get it pushed up to alloc_staging?". And once it has been on staging for a couple of weeks we push the new feature out to production alloc. Deploying particular changes to these trees used to be an arduous process. The problem is that developers put patches into the tree in a different order than the order that the managers would like to see the features promoted in. As a loose example, if your developers throw 20 patches into devel, and someone would like to see a feature deployed up to staging that is interleaved within those 20 patches, eg patches 5, 10 and 19, then it is a royal pain in the arse getting just those patches extrapolated up to a higher branch, i.e. staging. I mean how do you do it? hg has transplant, but what is that really? It doesn't just pull the patches across, it creates *new* patches with new hashes, that just happen to have the same contents as the original patches. That is not good! I would also like to point out that it is a non-trivial exercise to transplant say 10 particular patches across. I.e. lots of commands, lots of searching through logs. Ditto with bzr. Ditto with git. With darcs the solution we have implemented is as follows, since all work that is done must relate to an issue number in our issue tracker, we prefix each commit log with the issue ID of the feature. This means you might have a stack of patches in devel that look like this: * 11522 Added code for product module. * 11598 Added taxTfID to db_data.sql to bring DB in line with patches * 11598 Removed permissionIDs from db_data.sql * 13203 Nuked colspan=3 from Project->Task List. Renamed parenTaskDropdown to parentTaskDropdown. * 13153 Added Project, Parent Task and Task Type to en masse Task editing. Added clicks to the table rows. * 12946 Repeating expenses should no longer go into an infinite loop. * 11589 Engineers can now see a list of project managers on the project details page. * 12469 'Potential' projects can now have tasks added to them. * 13153 Smoothed over UI for en masse task editing. Added new task editing options. * 12209 Time sheet items for task that have overrun their estimates are now really obviously coloured red. * 12209 Time estimate overruns are now notified on "timesheet waiting for approval" emails. * 12586 Expansion of auto-resizing comment boxes is now capped. * 8532 Project import/export facility for GNOME Planner and CSV * 13153 The task list can now be used to make changes to tasks en masse. * 11406 Allow comments and emails with attachments on time sheets. If I want to promote all the patches that relate to issue number 13153 up to staging, I just go over to alloc_stage and pull up the changes that I want from devel: $ darcs pull -p 13153 (it remembers that we last pulled changes from alloc_devel, so it uses that as the default source branch again) That's one command. Done. No additional commits or merging required. No newly generated hashes. If for some reason feature 13153 doesn't work out, it is equally trivial to backout the change: $ darcs unpull -p 13153 Bang. Presto. That change that was spread across three patches has now been removed. This unbelievably useful feature makes darcs worth its weight in gold to us. It is IMO "killer-app" funtionality, and none of the other DVCS' have anywhere near that ease of use for deployment. Summary There are many other reasons to like darcs, the consistency and predictability of the commands. The slick, interactive UI. The simplicity and transparency of the actions that you are actually performing. Darcs amend, the hooks, the interactive-committing-by-default, the high value it encourages you to place on commits of quality - i.e. small commits that do exactly what their commit log say they do. There are one or two downsides to using darcs, for instance for some repositories it runs slowly, although I've never had a problem. Another downside is that the darcs development community appears to be quite small. I do wonder how important that is though, do I need a new grep every few months? Do I need a continually updated bash? Darcs works for me, simply and effectively. I don't need it to jump through hoops, I just need it to keep doing what it says it will do on the box. Notes [1] I use the email client mutt, if you put the following lines into your ~/.muttrc it will allow you to view patches, and also apply them to a local branch with one keystroke: auto_view text/x-patch text/x-darcs-patch macro pager A "<pipe-entry>umask 022;darcs apply --verbose --mark-conflicts --repodir ~/dev" [2] You can configure `darcs send` to digitally sign a patch provided you have GPG set up, and you place the following two lines into the _darcs/prefs/defaults/ file: send sign send to [3] I use darcs send on my local testing repository to send the patches, on to the canonical development repository which has a procmail rule like this: :0 * ^To: |(umask 022; darcs apply --dont-allow-conflicts --repodir /dev/ --verify /dev/allowed_gpg_keys) This procmail rule checks that the patch has been signed with my special GPG key and if so, applies it to the devel branch. The devel branch has a hook in its _darcs/prefs/defaults file which looks like this: apply posthook echo "$DARCS_PATCHES" | mutt -x -s "new patches applied" apply run-posthook Which basically gets run after new patches are successfully applied to the devel tree. This command sends an email back to the mailing list that lets all the developers know that a new patch has been applied to the development tree.
© 2007 ALSG