The Horn Dsl

An introduction to the horn project can be found here.

In my last post, I finished up by mentioning that the first task was to outline a DSL syntax containing metadata and build instructions for each package dependency.

We are basing our package manager on the Portage build system and hope to replicate their metaphor of the Portage Tree.  The DSL build instructions will reside in our package tree structure in the form of a build.boo file.

The choice of language for the DSL was an obvious one and we went with Boo which is a statically typed language for the CLI with a python inspired syntax and a special focus on language and compiler extensibility.

Below is the current DSL syntax for defining the package metadata and build instructions:

install horn:
    description "This is a description of horn"
    get_from svn("https://scotaltdotnet.googlecode.com/svn/trunk/")
    build_with msbuild, buildfile("source/Horn/horn.sln"), frameworkVersion35


This is our first iteration DSL and I expect the above prose to go through multiple metamorphic transformations as the ebb and flow of horn twists and turns through its development evolutionary cycle. I am also expecting that this is a fairly readable grammar and should explain what the purpose is.

First we are defining some basic metadata by way of the description of the install package.  I want to further open the metadata details out to contain other information points like the project’s home page URL and a category definition containing key words to make the packages searchable through the horn application.  I have pencilled in lucene.net as a proposed means of indexing the horn package tree.

Next we have the following line:

get_from svn("https://scotaltdotnet.googlecode.com/svn/trunk/")

Hopefully this sentence structure is perfectly readable and it is indeed clear that the sentence is a command that gets the source from subversion at the parenthesisised URI.

I should add at this point that we are using the excellent Rhino.DSL tool developed by Oren Eini. It makes getting started with Boo DSL much easier and I highly recommend its use to develop your own DSL.  The steps that were completed to allow the horn Boo DSL syntax to be parsed are as follows:

• A subclass was christened from the DslEngine base class of Rhino.DSL named ConfigReaderEngine. Then an AnonymousBaseClassCompilerStep was added to the compiler pipeline.


• created an abstract class named BaseConfigReader that the AnonymousBaseClassCompilerStep will use to parse the DSL syntax into.

Below is the listing of the horn install Dsl’s AnonymousBaseClassCompilerStep:

public class ConfigReaderEngine : DslEngine
{
    protected override void CustomizeCompiler(Boo.Lang.Compiler.BooCompiler compiler, Boo.Lang.Compiler.CompilerPipeline pipeline, string[] urls)
    {
        pipeline.Insert(1, new ImplicitBaseClassCompilerStep(typeof(BooConfigReader), "Prepare", "Horn.Core.Dsl"));
        pipeline.InsertBefore(typeof(ProcessMethodBodiesWithDuckTyping), new RightShiftToMethodCompilerStep());
        pipeline.Insert(2, new UnderscorNamingConventionsToPascalCaseCompilerStep());
        pipeline.Insert(3, new UseSymbolsStep());
 
    }
}

I will now attempt and I must stress attempt to explain how the build_with grammar can both be constructed and parsed out with regular C# code.  Let us remind ourselves of the syntax we want to parse:

build_with msbuild, buildfile("source/Horn/horn.sln"), frameworkVersion35
The BaseConfigReader abstract class mentioned previously will be entrusted with the job of making sense of our install grammar. Let us now start breaking the sentence structure down into it’s constituent parts.

The first part of our sentence is the build_with construct and this verb will be used to inform horn which build engine to use, where the build file is situated relative to where the package’s position is in the package tree directory hive and lastly which .NET framework version to build against.

A programmatic view of how to view this sentence is to observe build_with as the method name and the other parts as parameters delimited by commas that will be passed to the method. One of the reasons that boo is such a good fit for authoring DSLs is the absence of parenthesis needed for defining methods.

The following C# method is charged with making sense of the boo statement:

[Meta]
public static Expression build_with(ReferenceExpression builder, MethodInvocationExpression build, ReferenceExpression frameWorkVersion)
{
    var targetName = builder.Name;
 
    return new MethodInvocationExpression(
            new ReferenceExpression(targetName),
            build.Arguments[0],
            new StringLiteralExpression(frameWorkVersion.Name)
        );
}

The first thing of note is the Boo.Lang Meta attribute.  Static methods tagged with the Meta attribute should be perceived as a shortcut to the compiler that accepts AST (Abstract Syntax Tree) Nodes and returns an AST node.

An AST is the most common way for computers to work with structured text.  They turn the structured text into a graph on which operations can be performed against instead of manual parsing.

If we can return to the boo sentence under examination:
build_with msbuild, buildfile("source/Horn/horn.sln"), frameworkVersion35
Each sentence part after the build_with expression can be viewed as an AST node that will be passed into the buildWith static meta method in the form of the boo language construct of an expression.

• The msbuild snippet is translated as a ReferenceExpression. A ReferenceExpression can be viewed as string literal without the double quotes.


• The buildFile node is a MethodInvocationExpression. A MethodInvocationExpression contains the method name we want to invoke an the arguments if any that are needed to complete the method call.


• The frameWorkVersion35 is another ReferenceExpression.

The return call of the meta method will invoke the following method.

protected void msbuild(string buildFile, string frameWorkVersion)
{
    var version = (FrameworkVersion)Enum.Parse(typeof(FrameworkVersion), frameWorkVersion);
 
    BuildEngine = new BuildEngine(new MSBuildBuildTool(), buildFile, version);
}

We construct the call from the ‘msbuild’ reference expression.

This leaves the door open to call other build engines like nant for example.  We then satisfy the method call by selecting the argument from the buildFile method invocation and the framework version reference expression.

While constructing this DSL, I found the boo documentation pretty non existent but was able to extract knowledge both from Ayendes’s DSL book and the Rhino.DSL test fixtures.

If any of this is of interest to you then please join the Horn user group for updates or check out the source here.