It’s my third-last week in Hacker School, and people seem to play around a lot with ASTs (mostly in Python with provides run-time AST modification - how cool is that!?!). No wonder, after Paul Tagliamonte demonstrated how much fun it is!

I’ve been using them to create my learning tool that overloads JS operators with more transparent versions modified from V8 code.

This is a simplified walkthrough of the essential parts of the overloading process.

The REPL is still under construction but I’ll link it to this article when I get it to a stable state.

Goal

For my REPL I need to overload javascript operators and functions with my own functions. Since JS doesn’t have operator overloading, I’ll need to take the code in, replace the things I want to overload, and run the code.

Other possible uses for similar pipeline are syntax checking, adding logging magically to all functions in the codebase, code minification / obfuscation lisp/c-like macros etc. So this is very powerful for most kind of JS preprocessing.

Say I’d like to transform

1

var a = 4 + "foo";

to

1

var a = ADD(4, foo);

Seems clear? Let’s see how we’ll go about doing it.

Summary of actions

1. Create an abstract syntax tree from the code
2. Replace the function calls (from the tree)
3. Re-create the JS-code (from the syntax tree)
4. Run the modified code (good old eval)

How

Getting ready

First thing is to include esprima.js, which you can get through bower. Esprima parses your JS code to an abstract syntax tree (AST). This is a tree-representation of your program, which can be manipulated more easily and systematically than text-shaped code. You can play with the esprima demo and see what kind of ASTs your code produces.

To get it running on your own machine, first step is naturally to install it.

1

bower install esprima

Include in your JS, and you’re ready to dabble!

1. Creating abstract syntax trees

I want to see the syntax trees my different rows generate, so my test is

1
2

var ast = esprima.parse("var answer = 34 + 8"),
    ast2 = esprima.parse("var answer2 = ADD(34, 8)")

Which means I can see how the syntax trees differ for my source and target tree.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

{
  "type": "Program",
  "body": [
    {
      "type": "VariableDeclaration",
      "declarations": [
        {
          "type": "VariableDeclarator",
          "id": {
            "type": "Identifier",
            "name": "answer"
          },
          "init": {
            "type": "BinaryExpression",
            "operator": "+",
            "left": {
              "type": "Literal",
              "value": 34,
              "raw": "34"
            },
            "right": {
              "type": "Literal",
              "value": 8,
              "raw": "8"
            }
          }
        }
      ],
      "kind": "var"
    }
  ]
}

The ast2 part seems otherwise the same, but init-part is different..

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  "init": {
    "type": "CallExpression",
    "callee": {
      "type": "Identifier",
      "name": "ADD"
    },
    "arguments": [
      {
        "type": "Literal",
        "value": 34,
        "raw": "34"
      },
      {
        "type": "Literal",
        "value": 8,
        "raw": "8"
      }
    ]
  }

2. Replace the function calls

So I’ll make a function that overwrites the former with the latter:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

function replacePlusByADD(node) {
    // get operands from +
    var a = node.left,
        b = node.right;

    node.type = "CallExpression";
    node.callee = {
      "type": "Identifier",
      "name": "ADD"
    };
    node.arguments = [a, b];

    // reset unnecessary properties
    node.left = null;
    node.right = null;
    node.operator = null;

    return node;
}

I’ll use estraverse for the AST traversal

1
2
3
4
5
6
7
8

var ast  = esprima.parse(code);
estraverse.traverse(ast, {
    enter: function(node) {
        if (node.type === "BinaryExpression") {
            replacePlusByADD(node);
        }
    }
});

3 & 4. Re-create the JS-code and run

This is really just

1
2
3

var modified_code = escodegen.generate(ast);

eval(modified_code);

And Voilà! We have just created a JS pre-processor.

I found playing around with these tools, very much fun, in addition to helping me make pretty complex things in very few lines of code.

More resources: Esprima tutorial Fun with Esprima