Monday, 20 July 2009

Breaching The Castle Walls

For those unfamiliar with horn which I am guessing is pretty much everyone, horn is an effort to try and take the pain out of both getting and building open source projects. This also includes resolving the sometimes complex maze of dependencies that exists between projects.

My wish is at some stage to have the ability to be able to drop the introduction to horn because it has a presence in the .NET world. That day if it exists is sadly in the future.

The goal of horn is a very worthy and just cause. Anybody familiar with the pain of updating an open source stack containing Nhibernate, castle and Rhino will know exactly what I mean. For those who do not, it is worth mentioning that there is a very contrived dependency map that exists between these projects that makes this an upgrade path fraught with danger. Think of horn as a ruby gems for .NET or an appget.

The basic workflow of horn is this:

The user happily punches an instruction into the command window that on the surface seems to be a very innocent, uncomplicated and a reasonable request.

The request we are going to dissect is horn -install:castle.facilities.

This awakens the infamous horn.exe into life tasked with the mission of not only retrieving and building the castle.facilities source code but also any dependant components that horn might know about. You may ask yourself “how does horn know” about these dependencies? This would be a reasonable question and somebody asked me recently if the code programmatically did a scan of the project files and auto magically worked out what dependencies went where. Sadly this is not the case. There are many reasons why this is not the case but generally the build scripts (nant) that accompany something like castle contain much more than just compiling instructions that make it practically impossible to simply use csc or MSBuild.

Horn has the notion of build descriptors that are housed in what has been lovingly christened the horn package tree. The realisation of this is a directory of horn DSL files that contain build instructions of how to build not only the requested package which in this case is castle.facilities but also any dependencies. I have previously blogged about the DSL here here and here. The first step for horn is to locate the requested package which in this case is castle.facilities. In my last blog post, I wrote about the challenges faced with splitting the castle code base which is a behemoth of a solution.

I have spent the last week trying to build the descriptors for castle. I now consider that I have a good first iteration of the castle build descriptors which is why I have named this blog post so.

Below is the castle.facilities.boo DSL instance file that horn will use to build the requested package.

1install castle.facilities:
2    description "A castle facility augments the container with new functionality."
3
4    prebuild:
5        cmd "xcopy /s /y \"../Patch\" ."
6
7    include:
8         repository(castle, part("SharedLibs"), to("SharedLibs"))
9         repository(castle, part("Facilities"), to("Facilities"))
10        repository(castle, part("common.xml"), to("common.xml"))
11        repository(castle, part("common-project.xml"), to("common-project.xml"))
12        repository(castle, part("CastleKey.snk"), to("CastleKey.snk"))
13
14    build_with nant, buildfile("Facilities/facilities.build"), FrameworkVersion35
15
16    switches:
17        parameters "sign=true","common.testrunner.enabled=false", "common.silverlight=false"
18
19    shared_library "SharedLibs/net/2.0"
20    output "build/net-3.5/debug"
21
22dependencies:
23    depend "castle.windsor" >> "Castle.Core"
24    depend "castle.windsor" >> "Castle.DynamicProxy2"
25    depend "castle.windsor" >> "Castle.MicroKernel"
26    depend "castle.windsor" >> "Castle.Windsor"
27    depend "castle.activerecord" >> "Castle.ActiveRecord"
28    depend "castle.services" >> "Castle.Services.Transaction"
29    depend "castle.services" >> "Castle.Services.Logging.Log4netIntegration"
30    depend "castle.services" >> "Castle.Services.Logging.NLogIntegration"
31    depend "nhibernate" >> "2.1" >> "NHibernate"
32    depend "nhibernate" >> "2.1" >> "Iesi.Collections"
33
34package.homepage = "http://www.castleproject.org/"
35package.forum = "http://groups.google.co.uk/group/castle-project-users?hl=en"

The main point of interest is the dependencies section (lines 22 to 32). Each line of the dependencies section contains 2 or more parts. The first part will point to another horn package which will have its own DSL file and the second part is the name of the .dll we want to retrieve. In the case of Nhibernate we are able to give a specific version number. At the time of writing horn will go for the trunk build by default but I am not sure if this is practical for the long term practicality of the project. We will have to wait and see.

Horn builds an in memory dependency tree that is devoid of duplication and is ordered in such a way that the package with the least dependencies is built first. The in memory dependency tree is constructed by retrieving each of the dependencies found in the DSL descriptor .boo file and then parsing out any dependencies that may be in this dependency file and so on. Obviously this is a recursive process. If you look at the dependencies section on line 22 of the castle.facilities.boo file listing above, the horn process will load the first dependency file it finds in the dependency section, which in this case is the castle.tools.boo file and parse out any dependencies that might exist in this file. Care is taken not to parse the same file more than once and checks are in place to throw an exception in the case of a circular reference. Another horn contributor Craig Nicol recently blogged about the parsing of the dependency tree here.

An interesting dependency map can be illustrated with the castle.facilities.boo listing above that should hopefully illustrate some of the challenges that are faced when resolving dependencies. When the horn process reaches line 27 of the above code listing:

27    depend "castle.activerecord" >> "Castle.ActiveRecord"

The horn process will load the castle.activerecord.boo file into memory which has a dependency section like this:

1dependencies:
2    depend "castle.tools" >> "Castle.Core"
3    depend "castle.tools" >> "Castle.DynamicProxy2"
4    depend "castle.components" >> "Castle.Components.Validator"
5    depend "nhibernate.search" >> "NHibernate.Search"
6    depend "nhibernate" >> "2.1" >> "NHibernate"
7    depend "nhibernate" >> "2.1" >> "Iesi.Collections"

This will cause horn to load and parse out the build descriptors for each of these dependencies and so on. The data in the build descriptors is parsed into an in memory domain model of build metadata objects that will know how to build their particular package.

Castle facilities is by far the most complex package we have built so far and below is the build list we are left with after horn has created it’s in memory dependency tree of build metadata domain objects:
Here is the actual listing of the horn packages in their exact build order.

  • castle.tools

  • castle.services

  • castle.windsor

  • nhibernate

  • castle.components

  • nhibernate.search

  • castle.activerecord

  • castle.facilities



I must stress that horn is very much a work in progress and we are not at beta quality yet. Horn will download a lot of souce code to your file system given half the chance. Things can and will go wrong.

In my next post, I will explain how horn achieves the task of building the source and resolving the dependencies of a typical horn package.

If any of this is of interest to you then please join the Horn user group for updates or check out the source here.

No comments:

Post a Comment