caurea.org

Then source code of MacFUSE, the FUSE port for MacOSX is quite a mess. For one, the SVN repository contains a tarball with the FUSE userspace libary which was originally written for Linux and a patch which is applied to the extracted files to make the source compile on MacOSX. The tarball seems to contain fuse version 2.7.3 which is almost two years old now. And because the patch is against the tarball and not the source, most of it are changes to autoconf/automake generated files. That makes it hard to see what the actual changes are. Today I have tried to forward-port the patch to the latest version.

The basic idea was to first get the patch into git, and then use cherry-pick to forward-port it. Obviously the patch would apply cleanly to the extracted tarball, so first thing I did was to create a new Git repository, extract the tarball into it and apply the patch in a separate commit:

tar -xvzf ../macfuse/core/10.5/libfuse/fuse-current.tar.gz
mv fuse-2.7.3/* . && rmdir fuse-2.7.3
git add . && git commit -m 'Import fuse-current.tar.gz'

git apply < ../macfuse/core/10.5/libfuse/fuse-current-macosx.patch
git add . && git commit -m 'Apply the macfuse patch'

Now that the patch is recorded in the Git repository as a commit, I can use cherry-pick to apply it to latest version. FUSE still uses CVS but thanks to git cvsimport it was no problem to import the source into Git. I checked out the latest version and cherry-picked the patch:

git checkout upstream-cvs-source
git cherry-pick master

As expected, cherry-pick failed because the patch didn't apply cleanly as there have been many changes since 2.7.3 and the latest version. After I had fixed some of the issues and tried to compile the code, it failed because upstream has removed a field (tree_lock) from a structure that the cherry-picked code was using. Using git log -Stree_lock I was quickly able to find out in which commit the field was removed and using git show I saw what exactly the developers had changed. And that all did not take more than ten seconds! After a fixing the remaining issues, the code compiled cleanly.

The lesson learned? Just get your source somehow into Git. Once your code is there, you'll find it so much easier to navigate it. Who introduced a certain string? Easy, use git log -S<string>. Who wrote this line? git annotate <file>. All these commands take only a fraction of a second to get you the result that you need. And this is where you'll learn to appreciate the power of Git.

This morning I imported ccache into a Git repository (available now on github). I often do that with software for which there is no existing Git repository. The source of ccache is only available in tarballs, but that's not a big deal because Git comes with a script to import a set of tarballs (contrib/fast-import/import-tars.perl). However, the script generates rather ugly commit messages. Editing commit messages, even of commits which are far back in the history, is not a problem for Git, but editing the root commit can be tricky. This is what I used:

# find the root commit sha1 and check it out
# (this detaches HEAD but don't worry about it)
git checkout <root commit>

# amend/edit the commit as you want
git commit --amend

# go back to the branch you were before
git checkout -

# now rebase all the remaining commits on top of this new root
git rebase --onto HEAD@{1} <root commit>

You could even use interactive rebase in the very last command if you wanted to edit the remaining messages. Or you can use git rebase -i <new root commit> at any point later.

The subtree merge strategy can be used if you want to merge one project into a subdirectory of another project, and the subsequently keep the subproject up to date. It is an alternative to git submodules. The octopus merge strategy can be used to merge three or more branches. The normal strategy can merge only two branches and if you try to merge more than that, git automatically falls back to the octopus strategy. The problem is that you can choose only one strategy. But I wanted to combine the two in order to get a clean history in which the whole repository is atomically updated to a new version.

I have a superproject, let's call it projectA, and a subproject, projectB, that I merged into a subdirectory of projectA. I'm also maintaining a few local commits. ProjectA is regularly updated, projectB has a new version every couple days or weeks and usually depends on a particular version of projectA. When I decide to update both projects, I don't simply pull from projectA and projectB as that would create two commits for what should be an atomic update of the whole project. Instead, I create a single merge commit which combines projectA, projectB and my local commits. The tricky part here is that this is an octopus merge (three heads), but projectB needs to be merged with the subtree strategy. So this is what I do:

# Merge projectA with the default strategy:
git merge projectA/master

# Merge projectB with the subtree strategy:
git merge -s subtree projectB/master

# Now the index contains the final tree that I want,
# save the tree id for later use:
tree=$(git write-tree)

# Reset to before the two merges:
git reset --hard HEAD@{2}

# Read the tree into the index
git read-tree $tree

# Pretend that we just did an octopus merge with three heads:
echo $(git rev-parse projectA/master) > .git/MERGE_HEAD
echo $(git rev-parse projectB/master) >> .git/MERGE_HEAD

# And finally do the commit:
git commit

You should end up with a single merge commit of the three heads. You can check with git cat-file -p HEAD that the commit really contains three parents.

I'm working on a project with a few friends. It's a server software that will run on one of my servers. There will be two versions running in parallel, a live version and a test version. We are using git to manage the source and to make it easier for my friends to update the test server, I've set up a post-receive hook that triggers a build. Issuing make from within the post-receive hook is a bad idea, because it would block the git client that is trying to push the code to wait until the build completes. Therefore I'm using a message bus to pass messages between the post-receive hook and the build script. This way the post-receive script can return quickly and not block the git client. Another advantage is that the build script can run under a different user account than the git server process (I'll probably run it from a screen session so I can see what it is doing). After finishing the build process, the script automatically restarts the test server and sends me an email. What follows here are some of the script involved:

The post-receive script executes the trigger for each updated ref:

#!/bin/sh
while read old new ref; do
   /var/git/hooks/trigger.rb "ProjectName" "$old" "$new" "$ref"
done

The trigger is a tiny ruby script that does nothing more than sending a message to the stomp message bus. I've decided to use a separate queue for each git project. The message body contains the ref that was updated, so that the receiving can decide whether to build the source or not.

#!/usr/bin/env ruby

require 'rubygems'
require 'stomp'

client = Stomp::Client.open "stomp://localhost:61613"
client.send("/git/repos/#{ARGV[0]}", ARGV[3])

The ruby script on the other end of the stomp queues waits for trigger messages which tell us that refs/heads/master was updated. It then executes a shell script that does all the heavy work. All output is captured and sent in an email.

#!/usr/bin/env ruby

require 'rubygems'
require 'stomp'
require 'net/smtp'
require 'session'

client = Stomp::Client.open "stomp://localhost:61613"

client.subscribe("/git/repos/ProjectName") do |msg|
   if msg.body == "refs/heads/master"

      shell, out = Session::Shell.new, StringIO.new
      shell.execute "/path/to/build-script.sh", :stdout => out, :stderr => out

      Net::SMTP.start('localhost') do |smtp|
         smtp.open_message_stream('git-build@domain.tld', ['to@domail.tld']) do |f|
            f.puts 'From: git-build@domain.tld'
            f.puts 'To: to@domail.tld'
            f.puts 'Subject: git build finished'
            f.puts out.string
         end
      end
   end
end

client.join

The build script is a fairly standard make; make install script (with a few project specific commands added before and after the make process), so I won't post it here.