Building a Templating Language in Node.js - Part III: Implementing Our First Template Tag

Now that we have our work flow defined and development system set up, we can finally write some application code! We’ll start with the count tag, which will be used to count the number of elements in a collection (ie. an array or object).

A Specification for count

Before we do anything else, let’s define in as much detail as possible how count should function.

count will:

  1. Replace itself with an integer.
  2. Accept a collection attribute.
    • The collection attribute will take in the name of a collection within the current scope of the count tag.
    • The collection attribute will accept dot notation to access nested collections.
  3. Output error text if collection is undefined.
  4. Emit an error if no collection is defined.
  5. Output error text if collection can’t be found.
  6. Emit an error if the collection can’t be found.
  7. Output error text if collection is not an Object or Array.
  8. Emit an error if collection is not an Object or Array.
  9. Output the length property if collection is an Array.
  10. Output the length property of Object.keys(collection) if collection is an Object.

With this specification, not only have we defined a clear development goal, but we have also defined most of the tests we’ll need. How convenient!

A Critical Fork in the Road

Now comes a critical point in the development of our project: we need to determine how to parse our template files. There are many factors to consider at this point, but the two I’m most concerned with are speed and implementation difficulty. Since we’re dealing with HTML, we’ll probably want to use a pre-existing, node-based HTML parser. It would be even better if we could find an HTML parser that allowed us to easily query for elements in our template strings.

Enter Cheerio

Thankfully, Cheerio just so happens to be an easy to use, node-based HTML parser that offers a jQuery-like interface for accessing passed in HTML strings. Perfect! I do have two concerns with Cheerio, however:

  1. Speed. I have no idea how much time processing a large amount of template files might take.
  2. Memory usage. If we need to process a large template file, or generate a large amount of HTML, it is quite possible that we’ll run into memory issues.

I’m confident we can overcome these issues, so let’s not be premature with any optimizations. If you’re pulling your hair out right now because I’m making a horrible decision, let me know!

Our First Test

As I mentioned in my previous post, we will be writing tests before writing any application code, so without further ado, let’s get to it!

Well, almost. First, let’s create a new git branch for count.

git checkout -b count

Next, let’s create a file in our ./tests/ directory named 02-count.js. In this file we’ll bring in the necessary modules and describe our first test.

var sumo = require('../');
var assert = require('assert');

describe('<count />', function () {
  it('should output error text if `collection` is not supplied.', function () {
    // Write test here
  });
});

Great! What do we want our error text to look like? How about an HTML comment describing the error? Something like:

<!-- Sumo Count Error: No collection provided -->

Looks good to me. Let’s flesh out our test.

it('should output error text if `collection` is not supplied', function () {
  var actual = sumo.compile(
    '<count />', 
    { test: [1, 2, 3, 4] }
  );

  var expected = '<!-- Sumo Count Error: No collection provided -->';

  assert.equal(
    actual, 
    expected, 
    'Error text should be output if no collection is provided'
  );
});

Let’s run our tests and see what Mocha says.

  <count />
    1) should output error text if `collection` is not supplied.


  6 passing (3ms)
  1 failing

  1) <count /> should output error text if `collection` is not supplied:

      AssertionError: Error text should be output if no collection is supplied
      + expected - actual

      +<!-- Sumo Count Error: No collection provided -->
      -<count />

Perfect, our test is failing, and shows the value it expects and the value we are providing. We now just need to make them match.

Passing the Test

Now that we have a test in place, let’s move over to index.js and implement the code we need for our test to pass. We’re going to need to bring in the cheerio module, but this time we’ll want to save it as a production dependency, so make sure to use the --save flag, not --save-dev.

The first thing we’ll want to do is flesh out sumo.compile and add some boilerplate for our process functions.

var cheerio = require('cheerio');

var sumo = {
  /**
   * `elements` is a list of all of the template tags 
   * we will be searching for and processing. 
   */
  elements: [
    'count'
  ],

  /**
   * `process` will house all of the functions that will 
   * process the template tags above.
   */
  process: {}
};

/**
 * Let's cache a reference to `sumo.process`.
 */
var process = sumo.process;

sumo.compile = function (templateStr) {

  /**
   * First, we'll want to create an array of all tags in 
   * `templateStr` that need to processed.
   */
  var toProcess = this.elements.filter(function (element) {

    /**
     * To do this, we'll filter out all elements that
     * don't occur in the template string.
     */
    var filter = new RegExp('<' + element + '.*?>');

    return filter.test(templateStr);
  });

  /**
   * If `toProcess` isn't empty, then we know we have 
   * template tags to process.
   */
  if (toProcess.length) {

    /**
     * Let's load `templateStr` into `cheerio` and bind 
     * the result to `$`, so we can use a familiar 
     * jQuery like syntax.
     *
     * We need to set xmlMode to true to better handle 
     * self closing tags like `<count />`
     */
    var $ = cheerio.load(templateStr, { 
      xmlMode: true 
    });

    /**
     * Now we can process each tag in the template
     * string.
     */
    toProcess.forEach(function (element) {

      /**
       * `element` is the name of the tag we are looking 
       * to process. If there is a function with the same 
       * name in the `process` object, then we'll execute 
       * it and pass in our HTML via the Cheerio object.
       */
      if (typeof process[element] === 'function') {
        process[element]($);
      }
    });

    /**
     * Finally, we will return the contents of `$` as a
     * string.
     */
    return $.html();
  }

  /**
   * If there are no tags to process, just return
   * the template string unchanged.
   */
  return templateStr;
};

module.exports = sumo;

Next, let’s implement process.count.

process.count = function ($) {

  /**
   * First, we'll iterate through each of the `count` 
   * elements in our parsed template string
   */
  $('count').each(function () {

    /**
     * Then cache a reference to the current `count` 
     * element.
     */
    var $el = $(this);

    /**
     * Now we can look for the `collection` attribute.
     */
    var collection = $el.attr('collection');

    /**
     * If there is no `collection` attribute supplied, 
     * then we'll output error text.
     */
    if (collection === undefined) {
      $el.replaceWith(
        '<!-- Sumo Count Error: No collection supplied -->'
      );
    }
  });
};

In theory, our test should now pass. Let’s see what gulp test has to say.

  <count />
    ✓ should output error text if `collection` is not supplied 


  7 passing (8ms)

Success!

Emitting Error Events

Our next task is to emit an error event in addition to writing the error text above. We’ll also want to make sure the correct error message is being passed along with the event.

it('should emit an error event if `collection` is not supplied', function () {
  var eventFired = false;

  /**
   * We'll use `once` instead of `on` to create a
   * single use event handler.
   */
  sumo.once('error', function (err) {
    var actual = err;
    var expected = 'Count: No collection supplied';

    /**
     * Let's make sure the error message is correct
     */
    assert.equal(
      actual, 
      expected, 
      'Error message should be: No collection supplied'
    );
    eventFired = true;
  });

  sumo.compile('<count />');

  /**
   * We also need to make sure the event actually fired
   */
  assert(eventFired, 'Error event should be emitted');
});

Running our tests should produce one error:

  <count />
    ✓ should output error text if `collection` is not supplied 
    1) should emit an error event if `collection` is not supplied.


  7 passing (8ms)
  1 failing

Implementing an event system

In order to get this test passing, we’ll need to implement an event system. Thankfully node comes with the events module which contains all of the functionality we need.

var cheerio = require('cheerio');

/**
 * We'll use the `EventEmitter` to handle all of our 
 * event related needs.
 */
var EventEmitter = require('events').EventEmitter;

/**
 * Since we don't need a constructor function, let's just 
 * make `sumo` an event emitter. This will allow us bind
 * events like so: 
 * 
 * sumo.on('error', function () {})
 */
var sumo = new EventEmitter();

/**
 * Let's cache a reference to `sumo.process` and 
 * `sumo.elements`
 */
var process = sumo.process = {};
var elements = sumo.elements = [
  'count'
];

Now we can modify process.count() to emit an error event.

process.count = function ($) {
  $('count').each(function () {
    var $el = $(this);
    var collectionName = $el.attr('collection');
    if (collectionName === undefined) {

      /**
       * Emit `error` event and supply a message.
       */
      sumo.emit('error', 'Count: No collection supplied');

      $el.replaceWith(
        '<!-- Sumo Count Error: No collection supplied -->'
      );
    }
  });
};

Now all of our tests should pass… right?

  <count />
  [gulp] 'test' errored after 76 ms Count: No collection supplied
    1) should output error text if `collection` is not supplied
    ✓ should emit an error event if `collection` is not supplied. 


  7 passing (37ms)
  1 failing

  1) <count /> should output error text if `collection` is not supplied:
     Count: No collection supplied

Oh no, our first test is failing! Not to worry, this has more to do with Mocha than our code. Because we are emitting an error event, we need to make sure we catch it before it reaches Mocha, or else our test will fail. Let’s add a catch all event handler to prevent this from happening.

describe('<count />', function () {
  /**
   * Catch all error handler
   */
  sumo.on('error', function () {});

  /**
   * ... Tests here ...
   */
});

Let’s see if this worked.

  <count />
    ✓ should output error text if `collection` is not supplied 
    ✓ should emit an error event if `collection` is not supplied. 
    ✓ should supply correct error message to error event if `collection` not supplied. 


  8 passing (10ms)

Perfect!

Let’s commit our changes and continue.

git add .
git commit -m "Adds tests for and implements `no collection supplied` error"

Note: In order to reduce the length of this post, I have cut out most of the git commits. In general, you should commit after each passing test.

Now that we have our first few error tests set up, we can reuse a lot of the code for the subsequent error tests. Because of this, I won’t go over them in this post, but you can view all of the tests here and the implementation here.

Counting

Now that we’ve taken care of all of our expected error cases, let’s actually count some collections.

var testString = '<count collection="test" />';
it('should output the correct count of Array', function () {
  var collection = [1, 2, 3, 4];
  assert.equal(
    sumo.compile(testString, { test: collection }),
    '4',
    'Count of array is incorrect'
  );
});

it('should output the correct count of Object', function () {
  var collection = {
    1: 1,
    2: 2,
    3: 3,
    4: 4
  };
  assert.equal(
    sumo.compile(testString, { test: collection }),
    '4',
    'Count of object is incorrect'
  );
});

To make these tests pass, we’ll first need to pass the supplied template data into the compile and process functions.

/**
 * Pass in the `data` parameter
 */
sumo.compile = function (templateStr, data) {
  var toProcess = elements.filter(function (element) {
    var filter = new RegExp('<' + element + '.*?>');
    return filter.test(templateStr);
  });

  if (toProcess.length) {
    var $ = cheerio.load(templateStr, { xmlMode: true });

    toProcess.forEach(function (element) {

      /**
       * Pass in the `data` parameter
       */
      if (typeof process[element] === 'function') {
        process[element]($, data);
      }

    });

    return $.html();
  }

  return templateStr;
};

Now we actually have something to count. Back in our count function, let’s get our first test to pass.

process.count = function ($, data) {
  $('count').each(function () {
    var $el = $(this);
    var collectionName = $el.attr('collection');
    var collection = data[collectionName];

    /**
     * ...abbreviated...
     */

    /**
     * Get the length of the passed in array. We need to 
     * make sure `count` is a string, or else we'll run into
     * issues with Cheerio
     */
    var count = collection.length.toString();

    $el.replaceWith(count);
  });
};

This should do it for the first test. After watching it pass we’ll move on to the second.

process.count = function ($, data) {
  $('count').each(function () {
    var $el = $(this);
    var collectionName = $el.attr('collection');
    var collection = data[collectionName];

    /**
     * ...abbreviated...
     */

    /**
     * If the collection is an array
     */
    var count = Array.isArray(collection) ?

      /**
       * Get its length
       */
      collection.length.toString() :

      /**
       * Else, create an array of the object's
       * keys and get the length of that.
       */
      Object.keys(collection).length.toString();

    $el.replaceWith(count);
  });
};

If we run our tests, we should see that the count tag is correctly counting collections!

The Final Test

We’re getting close! Now that we can correctly count objects and arrays, let’s write a test for handling dot notation in the collection attribute.

it('should support dot notation in collection identifier', function () {
  var testString = '<count collection="test.arr" />';
  var collection = {
    arr: [1, 2, 3]
  };

  assert.equal(
    sumo.compile( testString, { test: collection } ),
    '3',
    'Incorrect count with dot notation'
  );
});

Let’s make this pass.

process.count = function ($, data) {
  $('count').each(function () {
    var $el = $(this);
    var collectionName = $el.attr('collection');
    var collection = data[collectionName];

    /**
     * ...abbreviated...
     */

    /**
     * If the collection name contains `.`, then split the 
     * name into an array.
     */
    if (/\./.test(collectionName)) {
      collection = collectionName.split('.')
        /**
         * We'll use the `Array#reduce` method to crawl 
         * down the correct branch of the object tree. The 
         * first time `reduce` fires, `prev` will reference 
         * `data`. After that, `prev` will be a reference 
         * to the next branch of the `data` object.
         */
        .reduce(function (prev, curr) {
          /**
           * If we encounter `undefined` anywhere in the 
           * process, then we will just return `undefined`. 
           * This will allow us to emit a 
           * 'collection not found' error below.
           */
          if (prev === undefined || prev[curr] === undefined) {
            return undefined;
          }
          /**
           * Else, we'll traverse the rest of the object. 
           * What we return here will become `prev` in the 
           * next iteration of `reduce`. If there are no 
           * iterations left, this is what will be returned 
           * to `collection`.
           */
          return prev[curr];
        }, data);
    }

    /**
     * ...abbreviated...
     */
  });
};

Bam! We now have a functioning <count /> tag! We might have to come back and revisit our code and our tests after we implement other tags, but we’ve successfully implemented everything in our specification!

Integrating Our Changes

There is one final task to complete before we end this post, and that is integrating our changes into the master branch. Let’s make sure all of our changes are commited, then let’s push to the count branch in our remote repo.

git push origin count

Now let’s log into GitHub and submit a pull request to our development branch. If you don’t have a development branch yet, you can create one through the github interface.

To submit a pull request, click on the pull request tab on the right side of the page, then click New pull request. On this page, you’ll want to select development as your base, and count as the branch to compare it to. Then click Create pull request. Since we’re the only ones on this project, we’ll also go ahead and merge our changes.

If we login to Travis CI, we can watch our tests run, or we can just wait for Travis to shoot us an email that our tests have passed. After the tests have successfully passed, we can use the same process for merging development into master.

This system may seem tedious and unnecessary, and it is for this particular project, but it a good habit to practice, as this system and others like it become vital when working with large teams.

Conclusion

That’s it for this post! As always, feel free to contact me with any questions, feedback, or concerns! In the next post, we will come back and refactor some of this code, as we need to figure out a way to make the foundation more modular, extensible, and maintainable.

Back To Top