Posts for the month of December 2011

Bitbucket Mercurial Patch Queues

I use Mercurial. It use it for most of my open source development, and it is feature rich and powerful yet simple and intuitive. What I use most is the MQ (Mercurial Queues) feature for making patch queues (stacks really). With much of my work being Trac related, I can combine the Trac mirror at Bitbucket and share my patch queues - and work with others on their development.

There are many sources online for learning about Mercurial patch queues (1 2 3), but this post is really a plain how-to written for fellow Trac developers that need to work with my patch queues. Hopefully this live example will also explain the basics of why & how it works - and use hg help mq and help <command> for each to learn more.


Pre-history: I've created https://bitbucket.org/osimons/trac-t10425/src as a start to try to unwind a complicated set of features into a stack of simpler incremental changes. This was simply done by going to the main Trac repository at Bitbucket (https://bitbucket.org/edgewall/trac) and hitting the 'patch queue' link (right next to 'fork'). Provide details of your login if needed, and the name you want for the patch repository ('trac-t10425' is the name I used this time, combining 'trac' and the ticket number it corresponds to).

The work is started by adding a first default_columns.diff patch and committed it to the patch repository. The next would be to start attacking the remaining parts of the patch, adding each feature in its own patch. Here is the short guide for how to pick this up:

  1. Clone the repository; hg qclone https://bitbucket.org/osimons/trac-t10425 which actually clones BOTH the full Trac repository and the patch queue (hidden inside in .hg/patches).
  1. The basics of the patch queue:
    > cd trac-t10425
    > python -m trac.ticket.tests.query
    Ran 44 tests in 0.577s
    > hg qseries
    > hg qpush
    applying default_columns.diff
    now at: default_columns.diff
    > python -m trac.ticket.tests.query
    Ran 48 tests in 0.660s
    > hg qpop
    popping default_columns.diff
    patch queue now empty
  2. So, you pop and push patches and they can be stacked of course - which is the whole idea for this set of changes. Get to the top of the patch queue, and make a new patch (named after feature, but can be renamed later if needed):
    > hg qpush
    applying default_columns.diff
    now at: default_columns.diff
    > hg qnew the_next_thing.diff
    > hg qseries # prints the patch stack
    > cat .hg/patches/the_next_thing.diff
    # HG changeset patch
    # Parent e4075428584eea0ca3b6c63984a1f4445d1f9814
  3. Hack away at the code for "the next thing", and continually refresh the patch if needed (useful for comparing and reverting baby steps). Finally the set of changes can be committed to the patch repository. The patch queue is a full repository, so any number of changes to any patches in the queue may be committed.
    > hg qrefresh # updates the current patch, repeat at any time
    > hg commit --mq -m "The next thing is now OK."
    > hg outgoing --mq
    comparing with https://bitbucket.org/osimons/trac-t10425/.hg/patches
    searching for changes
    changeset:   1:xxxxxxxxx
  4. For anyone wanting write access to this patch queue, just contact me with your Bitbucket user, and I'll add write permission for you. Then you can:
    > hg push --mq
    pushing to https://bitbucket.org/osimons/trac-t10425/.hg/patches
    searching for changes
    remote: adding changesets
    remote: adding manifests
    remote: adding file changes
    remote: added 1 changesets with 3 changes to 3 files
  5. Updating changes from others follows same principle:
    hg pull --mq --update

That's it for the basics. It is just a nested repository inside the regular Trac repository, and each can be updated, committed, pulled and pushed as needed. Update Trac with new changes and reapply patches in series one-by-one, adjusting them if needed to make sure they apply cleanly.

Bonus insight

For sporadic work on single patch queues, it becomes a bit cumbersome to keep so many full copies of the full Trac repository laying around - it wastes space, and it is a burden to continually keep reinstalling and hooking things up to Apache development server and more. Instead, it is possible to reuse one checkout of Trac with many patch queues - Mercurial supports multiple patch queues, and just like branches you just switch:

> hg qqueue
t10425-bb (active)
> hg qq --create my-new-q
> hg qq
my-new-q (active)

By default patch queues are unversioned, so if you want to share or follow changes over time you can init a repository for the new active patch queue:

> hg init --mq
> hg status --mq
A .hgignore
A my_patch.diff
A series
> hg ci --mq -m "Adding stuff."

After working in a new patch queue, this patch queue can also be hooked up to Bitbucket. When making a new patch queue from a repository, just select to NOT create a series file - seeing you will provide your own full patch repository, so a dummy first changeset with an empty series file will just complicated things.

Edit .hg/patches-my-new-q/.hg/hgrc and add the patch to the patch repository. The trick here is that the public URL shown for the Bitbucket patch queue is not exactly what you want - you want to directly match the two nested patch repositories (local and remote). So, make the patch repos hgrc look like this instead:

default = https://bitbucket.org/osimons/trac-t10425/.hg/patches

With paths matched, it just becomes a matter of pushing patch queue changes to the empty Bitbucket repository:

> hg push --mq
pushing to...

The other way around is very easy too. If you find a patch queue of interest, just make a checkout of it to your main project .hg directory - but be sure to name it patches-<queue-name>. And update the .hg/patches.queues to add queue-name to the list of available queues.

> cd .hg
> hg clone https://bitbucket.org/osimons/trac-t10425/.hg/patches patches-t10425
> echo "t10425" >> patches.queues
> hg qq
patches (active)
> hg qq t10425
> hg qpush --all
applying default_columns.diff
now at: default_columns.diff


Trac Template Debug

Here is a simplified template for debugging rendering issues or helping out when developing your own Trac / plugin templates. All it does is to find the main information available for rendering, and provide a simple (big) printout of the information at the end of each HTML page.

Drop it into your Trac project as templates/site.html, or alternatively if you already have a site.html then just copy & paste the main py:match section into your own file.

It is currently restricted to TRAC_ADMIN permission (<div py:if="'TRAC_ADMIN' in req.perm"...), but please use with caution for anything production related! There is no telling what information it may print, and there are no further checks. Be warned.

<html xmlns="http://www.w3.org/1999/xhtml" 

<!--! A new debug information <div> at the bottom of all pages -->
<py:match path="body" once="True">
<body py:attrs="select('@*')">
  <div py:if="'TRAC_ADMIN' in req.perm"
       style="width: 98%; margin: 5px; border: 2px solid green; padding: 10px; font-family: courier;"
       py:with="b_dir = globals()['__builtins__'].dir">
    <div style="text-indent: -30px; padding-left: 30px;">
      <!--! Some potentially very long lists... -->
      <p style="">perm for ${perm.username}: ${repr(perm.permissions())}</p>  
      <p>project: ${repr(project)}</p>  
      <p>trac: ${repr(trac or 'not defined')}</p>
      <p>context: ${repr(context)}</p>  
      <p>context members: ${repr(b_dir(context))}</p>
      <p><strong>context __dict__:</strong>
        <div py:for="item in sorted(context.__dict__.keys())">
            ${item}: ${repr(context.__dict__[item])}</div></p>
        <div py:for="item in sorted(req.environ.keys())">
            ${item}: ${repr(req.environ[item])}</div></p>
      <p><strong>req members:</strong> ${repr(b_dir(req))}</p>
      <p><strong>req __dict__:</strong>
        <div py:for="item in sorted(req.__dict__.keys())">
            ${to_unicode(item)}: ${to_unicode(repr(req.__dict__[item]))}</div></p>
      <p><strong>all objects from locals().['__data__']:</strong>
        <div py:for="item in sorted(locals()['__data__'].keys())">
            ${to_unicode(item)}: ${to_unicode(repr(locals()['__data__'][item]))}</div></p>
        <div py:for="key in sorted(globals()['__builtins__'].keys())">
            ${key}: ${repr(globals()['__builtins__'][key])}</div></p>
      <p py:with="sys = __import__('sys')">
        <strong>sys.path:</strong><br />


The debug template also allows you to play around with the information and try it out interactively. Here are some examples:

<p>Try using req.hef(): ${req.href('wiki')}</p>
<p>Test fetching an element: ${select('div[@id="mainnav"]')}</p>

If you want to avoid having to reload the Trac process for each template change, just turn on auto_reload to have it picked up automatically:

auto_reload = True


  • Posted: 2011-12-13 13:06 (Updated: 2012-07-10 00:18)
  • Categories: trac
  • Comments (1)