Boost.Spirit is cool

I recently needed to parse simple material definition scripts for a work-in-progress rendering system, so I decided it was high time I learn how to use Spirit. For those unfamiliar with the library, Spirit facilitates the creation of small, fast parsers by letting you define EBNF grammars directly in C++ code. Obviously, that's a very "nutshell" description of the library, but you can check out the Spirit homepage if you want to know more. Anyways, lets demonstrate the library's wizadry using my simple material script use-case:

// A (very) simple material type
struct Material
{
    std::string name;
    std::string shaderProgramName;
    std::vector<std::string> textureNames;
};

// Make our material type usable as a Fusion tuple
BOOST_FUSION_ADAPT_STRUCT(
    Material,
    (std::string, name)
    (std::string, shaderProgramName)
    (std::vector<std::string>, textureNames) )

So here we have a simple structure containing two strings and a vector of strings. Now lets see an example material definition script that would be used to populate this structure:

 NAME sample_material
 SHADER some_shader_name

 TEXTURES
 {
     texture1
     texture2
     texture3
 }

So how do we go about using this cool Spirit library to parse this thing into a Material object? Furthermore, lets say we want to be able to arbitrarily order the NAME, SHADER, and TEXTURES terms in the script, while requiring each one to be present. With these requirements in mind, we'll first define a grammar that describes a valid material definition:

template< typename Iterator >
struct MaterialDefinition : boost::spirit::qi::grammar< Iterator, Material() >
{
    MaterialDefinition()
        : MaterialDefinition::base_type( start )
    {
        using namespace boost::spirit::qi;

        start = &(requiredTermCount) >> (name ^ shader ^ textureList);
        name = *space >> lit("NAME") >> +space >> value >> *space;
        shader = *space >> lit("SHADER") >> +space >> value >> *space;
        textureList = *space >> lit("TEXTURES") >> *space >> lit('{') >> *space >> (value % +space) >> *space >> lit('}') >> *space;
        value = +char_("a-zA-Z_0-9");
        requiredTerm = name | shader | textureList;
        requiredTermCount = requiredTerm >> requiredTerm >> requiredTerm;
        space = lit(' ') | lit('\n') | lit('\t');
    }

    boost::spirit::qi::rule< Iterator, Material() > start;
    boost::spirit::qi::rule< Iterator, std::string() > name;
    boost::spirit::qi::rule< Iterator, std::string() > shader;
    boost::spirit::qi::rule< Iterator, std::vector<std::string>() > textureList;
    boost::spirit::qi::rule< Iterator, std::string() > value;
    boost::spirit::qi::rule< Iterator > requiredTerm;
    boost::spirit::qi::rule< Iterator > requiredTermCount;
    boost::spirit::qi::rule< Iterator > space;
};

Sure it looks scary at first (it sure did to me), but just stare at it for a few seconds and compare it to the script above. If you're at all familiar with EBNF grammars, the grammar definition above should look familiar. Aside from a few superficial differences, it's an EBNF grammar written in standard C++. Now how do we actually use this thing to create a Material object from the script above? Well that's easy:

std::string script; // Contains script to be parsed

Material material;
if( !boost::spirit::qi::parse(script.begin(),
                              script.end(),
                              MaterialDefinition<std::string::iterator>(),
                              material) )
{
    throw SyntaxError( script );
}

That's a fairly self-documenting peice of code if I ever saw one. We even get easy error handling :)

Comments

hmmm...

Boost.Property_tree (NFO parser)ftw... nuff said ;P
Oh hello there ken :)

RE: hmmm...

Right you are, although I was using Boost 1.40 when I wrote this, so Property Tree wasn't yet available in the official release :) That, and I just wanted to learn Spirit. Besides, Property Tree uses Spirit internally for its parsing duties IIRC. That being said, I'm planning a significant redesign of the rendering code that was the impetus behind me finally learning Spirit, so I may use Property Tree yet. As is often the case when implementing things you've never implemented before, I've made rendering code that works, but not in a very elegant way due to not understanding the whole problem. Also, I've learned a bit more about Spirit since writing this example, and will rewrite it in a better way in a future post.