Manipulate comments with TypeScript API

3 min readJan 26, 2020

Background

I needed a TypeScript custom transformer to rewrite webpack magic comment a few months ago, such as:

Input source:

const Hoge = import(/* webpackPrefetch: true, webPackChunkName: "XXXXX" */ "./hoge");

Expected output:

const Hoge = import(/* webpackPrefetch: true, webPackChunkName: "ZZZ" */ "./hoge");

There are no difference except a little comment change. However, manipulating comments was much harder than I imaged.

How to manipulate comments with TS API ?

Read comments

Basically, comment is not AST node nor token but trivia attached with node. Trivia is trivial text, e.g. whitespace, Git conflict merkers.

The following underlined text is leading trivia of a = 1; node:

/* hoge */ a = 1;
~~~~~~~~~~~const content = `/* hoge */ a = 1`;
const src = ts.createSourceFile("main.ts", content, ts.ScriptTarget.Latest, true);assert.equal(src.getFullText().slice(0, src.getLeadingTriviaWidth()), "/* hoge */ ");

It’s important that we must extract comment value from source file text.

Also, TS provides other API to get location of comments, getLeadingCommentRanges and getTrailingCommentRanges.

const content = `/* hoge */  a = 1;`;
const src = ts.createSourceFile("main.ts", content, ts.ScriptTarget.Latest, true);const commentRange = ts.getLeadingCommentRanges(src.getFullText(), src.getFullStart())![0];
assert.equal(commentRange.kind, ts.SyntaxKind.MultiLineCommentTrivia);
assert.equal(src.getFullText().slice(commentRange.pos, commentRange.end), "/* hoge */");

const content = `"hoge"; // TODO`;
const src = ts.createSourceFile("main.ts", content, ts.ScriptTarget.Latest, true);

const node = src.statements[0] as ts.ExpressionStatement;
const commentRange = ts.getTrailingCommentRanges(src.getFullText(), node.getEnd())![0];
assert.equal(commentRange.kind, ts.SyntaxKind.SingleLineCommentTrivia);
assert.equal(src.getFullText().slice(commentRange.pos, commentRange.end), "// TODO");

Which node has the comment ?

We can’t determine an unique owner node for a comment.

For example, a comment in /* hoge */ a = 1; is accessible by 4 nodes whose getStart() return same value.

SourceFile (top level node, src)
ExpressionStatement (src.statements[0])
BinaryExpression (src.statements[0].expression)
Identifier (src.statements[0].expression.left)

const content = `/* hoge */ a = 1;`;
const src = ts.createSourceFile("main.ts", content, ts.ScriptTarget.Latest, true);let node: ts.Node;
let commentRange: ts.CommentRange;
node = src.statements[0];
commentRange = ts.getLeadingCommentRanges(src.getFullText(), node.getFullStart())![0];
assert.equal(src.getFullText().slice(commentRange.pos, commentRange.end), "/* hoge */");node = (node as ts.ExpressionStatement).expression;
commentRange = ts.getLeadingCommentRanges(src.getFullText(), node.getFullStart())![0];
assert.equal(src.getFullText().slice(commentRange.pos, commentRange.end), "/* hoge */");const left = (node as ts.BinaryExpression).left;
commentRange = ts.getLeadingCommentRanges(src.getFullText(), left.getFullStart())![0];
assert.equal(src.getFullText().slice(commentRange.pos, commentRange.end), "/* hoge */");

This means that we should be careful when we count the number of comments with visiting AST(e.g. ts.forEachChild, ts.visitEachChild).

Add comment

To write comment is easier than to read comment. We just use addSyntheticLeadingComment or addSyntheticTrailingComment.

const statement = ts.createExpressionStatement(ts.createIdentifier("debugger"));
ts.addSyntheticLeadingComment(statement, ts.SyntaxKind.MultiLineCommentTrivia, "eslint-disable", true);
assert.equal(printNode(statement), "/*eslint-disable*/\ndebugger;");function printNode(node: ts.Node) {
  const printer = ts.createPrinter({
    newLine: ts.NewLineKind.LineFeed,
    removeComments: false,
  });
  if (ts.isSourceFile(node)) return printer.printFile(node);
  return printer.printNode(ts.EmitHint.Unspecified, node, ts.createSourceFile("", "", ts.ScriptTarget.Latest));
}

What does synthetic mean?

Comment added by addSyntheticLeadingComment or addSyntheticTrailingComment has no real text. It's similar that a node created ts.createIdentifier has no parent node. They are not written by us but generated by our code artificially.

We can’t extract synthetic comment via ts.getLeadingCommentRanges because this characteristic. Instead, use ts.getSyntheticLeadingComments API.

const statement = ts.createExpressionStatement(ts.createIdentifier("debugger"));
ts.addSyntheticLeadingComment(statement, ts.SyntaxKind.MultiLineCommentTrivia, "eslint-disable", true);
const src = ts.updateSourceFileNode(ts.createSourceFile("main.ts", "", ts.ScriptTarget.Latest, true), [statement]);
assert(printNode(src).indexOf("/*eslint-disable*/") !== -1);
const commentRange = ts.getLeadingCommentRanges(src.getFullText(), src.getFullStart());// We can not access synthetic comments with getLeadingCommentRanges/getTrailingCommentRanges
assert.equal(commentRange, undefined);// Use getSyntheticLeadingComments to read synthetic comments
assert.equal(ts.getSyntheticLeadingComments(src.statements[0])![0].text, "eslint-disable");// Unlike leading trivia, getSyntheticLeadingComments does not return comment ranges for parents of the node attached comments
assert.equal(ts.getSyntheticLeadingComments(src), undefined);

Update comment within transformer

The following is a trivial example transformer to rewrite comments. It just only makes comments’ text upper case.

const content = "// todo\nconsole.log('hoge');";
const originalSrc = ts.createSourceFile("main.ts", content, ts.ScriptTarget.Latest, true);
const transformer = (ctx: ts.TransformationContext) => {
  return (src: ts.SourceFile) => {
    const text = src.getFullText();
    let previousCommentPos = -1;
    const visit = <T extends ts.Node>(node: T): T => {
      if (ts.isSourceFile(node)) return ts.visitEachChild(node, visit, ctx);
      ts.forEachLeadingCommentRange(text, node.getFullStart(), (pos, end, kind, hasTrailingNewLine) => {
        if (pos === previousCommentPos) return;
        const commentContent =
          kind === ts.SyntaxKind.MultiLineCommentTrivia ? text.slice(pos + 2, end - 2) : text.slice(pos + 2, end);
        src.text = text.slice(0, pos).padEnd(end, " ") + text.slice(end);
        ts.addSyntheticLeadingComment(node, kind, commentContent.toUpperCase(), hasTrailingNewLine);
        previousCommentPos = pos;
      });
      return ts.visitEachChild(node, visit, ctx);
    };
    return visit(src);
  };
};
const { transformed } = ts.transform(originalSrc, [transformer], ts.getDefaultCompilerOptions());
assert.equal(
  ts.createPrinter({ newLine: ts.NewLineKind.LineFeed, removeComments: false }).printFile(transformed[0]),
  "// TODO\nconsole.log('hoge');\n",
);

As I wrote above, we already have way to read/write comments. Oh, there is a missing piece. We need to remove existing comments in order to rewrite. As far as I know, there are no API to do that unfortunately.

But remember “comment is just trivia”. We can replace them by not AST method but text manipulation. The following line mutates the original source file text to replace a comment text to whitespaces whose length is same to the comment’s length.

src.text = text.slice(0, pos).padEnd(end, " ") + text.slice(end);

Summary

Use getLeadingCommentRanges to read comments from parsed sources
Use addSyntheticLeadingComment to write comments
TypeScript provides API to remove comments(both trivia and synthetic)

References

https://basarat.gitbook.io/typescript/overview/ast/ast-trivia