Abstract syntax trees (ASTs)

ASTs are tree-based data structures that represent source code. ASTs are useful for source code analysis, transformation, and generation.

Table of contents

• AST Explorer
• Popular JavaScript AST parsers
• Specifications
  • ESTree
    • ESTree's AST descriptor syntax
    • ESTree's philosophy
    • ESTree resources
  • ESPree
  • Shift
    • Shift Resources
• Theses
  • 3.1. Graph-Based Source Code Analysis of JavaScript Repositories
    • 3.1.1. Abstract

AST Explorer

AST Explorer is an indispensable browser-based tool for exploring ASTs generated by popular parsers.

AST Explorer not only includes 14 popular JavaScript parsers, but also parsers for CSS, GraphQL, Handlebars, HTML, ICU, JSON, Lua, Markdown, PHP, Regular Expressions, Scala, SQL, WebIDL, and YAML.

Popular JavaScript AST parsers

AST Explorer currently supports these ECMAScript parsers.

1. acorn: A small, fast, JavaScript-based JavaScript parser.
2. acorn-jsx: An Acorn plugin for parsing React.js JSX.
3. babel-eslint: A Babel parser built with ESLint for JavaScript using types (Flow) or experimental features not yet supported in ESLint.
4. babylon: A JavaScript compiler that transforms the latest JavaScript syntax into browser-compatible source code.
5. esformatter: An ECMAScript code beautifier/formatter.
6. espree: An Esprima-compatible JavaScript parser built on top of Acorn's modular architecture for backwards-compatible ASTs.
7. esprima: A high performance, standard-compliant ECMAScript parser written in ECMAScript.
8. flow-parser: A JavaScript parser written in OCaml that produces an AST that conforms to SpiderMonkey's Parser API (and that mostly matches esprima ASTs).
9. recast: A JavaScript syntax tree transformer, nondestructive pretty-printer, and automatic source map generator.
10. shift: An ECMAScript parser that produces a Shift format AST.
11. traceur: A JavaScript.next compiler focused on testing new and proposed ECMAScript features in order to inform the standards process.
12. typescript-eslint-parser: A parser that converts TypeScript into an ESTree-compatible form so it can be used in ESLint.
13. typescript: A strongly-typed superset of JavaScript that compiles to clean JavaScript output.
14. uglify-js: A JavaScript parser, minifier, compressor and beautifier toolkit.

Specifications

ESTree

ESTree defines core AST node types that support the ESn grammar based on the SpiderMonkey engine's JavaScript parser.

ESTree's AST descriptor syntax

extend interface Program {
    sourceType: "script" | "module";
    body: [ Statement | ModuleDeclaration ];
}

ESTree's philosophy

Suggested additions and modifications must follow these guidelines:

1. Backwards compatible: Non-additive modifications to existing constructs will not be considered unless immense support is in favor of such changes. (eg. #65)
2. Contextless: Nodes should not retain any information about their parent. ie. a FunctionExpression should not be aware of if it's a concise method. (eg. #5)
3. Unique: Information should not be duplicated. ie. a kind property should not be present on Literal if the type can be discerned from the value. (eg. #61)
4. Extensible: New nodes should be specced to easily allow future spec additions. This means expanding the coverage of node types. ie. MetaProperty over NewTarget to cover future meta properties. (eg. #32)

ESTree resources

View the estree / estree repository on GitHub.

ESPree

An Esprima-compatible JavaScript parser that conforms to the ESTree specification, based on the Acorn parser.

Shift

A JavaScript parser that abandons backward-compatibility with SpiderMonkey/ESTree ASTs in favor of more concise nodes. It's much cleaner, but has a much smaller community and infrequent updates.

Shift Resources

1. Shift AST
2. Shift AST Specification (ES6)
3. A Technical Comparison of the Shift and SpiderMonkey AST Formats
4. Why Composition is Harder with Classes

Theses

3.1. Graph-Based Source Code Analysis of JavaScript Repositories

Graph-Based Source Code Analysis of JavaScript Repositories, by Dániel Stein.

3.1.1. Abstract

We are surrounded by more and more complex software that operate in mission-critical systems. Even small errors in these software can lead to serious consequences that may be too expensive to let happen. Static analysis is a proven approach for detecting mistakes in the source code early in the development cycle. Since static analysis does not compile or run the code, it can be applied at an early state of development. With static analysis it is possible to check whether the software conforms to the coding rules and requirements, and to locate potential errors.

While multiple static analysis tools exist for general purpose programming languages and these are generally part of the continuous integration systems, this is not the case with JavaScript. Due to the dynamically typed nature of this language there are only a few tools available for JavaScript codebases. Also, there are currently no tools available jointly providing lower level and global static analysis, finding control flows, and providing integration points for continuous integration systems.

In this thesis I design, implement and evaluate a framework extending the continuous integration workflow of large and frequently changing JavaScript repositories with configurable static analysis tools and techniques. Due to the novel approach of the framework, its users can express requirements easier and they are able to check global level requirements more efficiently.