transformExpression
Developer Interface to Aim for and Current Challenges
First, take a look at this component.
<script>
import { ref } from 'chibivue'
export default {
setup() {
const count = ref(0)
const increment = () => {
count.value++
}
return { count, increment }
},
}
</script>
<template>
<div>
<button :onClick="increment">count + count is: {{ count + count }}</button>
</div>
</template>
There are several issues with this component.
Since this component is written in SFC, the with
statement is not used.
In other words, the bindings are not working properly.
Let's take a look at the compiled code.
const _sfc_main = {
setup() {
const count = ref(0)
const increment = () => {
count.value++
}
return { count, increment }
},
}
function render(_ctx) {
const { h, mergeProps, normalizeProps, normalizeClass, normalizeStyle } =
ChibiVue
return h('div', null, [
'\n ',
h('button', normalizeProps({ onClick: increment }), [
'count + count is: ',
_ctx.count + count,
]),
'\n ',
])
}
export default { ..._sfc_main, render }
- Issue 1: The
increment
registered as the event handler cannot access_ctx
.
This is because the prefix was not added in the previous implementation ofv-bind
. - Issue 2: The expression
count + count
cannot access_ctx
.
Regarding the mustache syntax, it only adds_ctx.
to the beginning, and cannot handle other identifiers.
Therefore, all identifiers that appear in the middle of an expression need to be prefixed with_ctx.
. This applies to all parts, not just mustaches.
It seems that a process is needed to add _ctx.
to the identifiers that appear in the expressions.
Desired Compilation Result
const _sfc_main = {
setup() {
const count = ref(0)
const increment = () => {
count.value++
}
return { count, increment }
},
}
function render(_ctx) {
const { h, mergeProps, normalizeProps, normalizeClass, normalizeStyle } =
ChibiVue
return h('div', null, [
'\n ',
h('button', normalizeProps({ onClick: _ctx.increment }), [
'count + count is: ',
_ctx.count + _ctx.count,
]),
'\n ',
])
}
export default { ..._sfc_main, render }
WARNING
Actually, the original implementation takes a slightly different approach.
As you can see below, in the original implementation, anything bound from the setup
function is resolved through $setup
.
However, implementing this is a bit difficult, so we will simplify it and implement it by adding _ctx.
. (All props and setup will be resolved from _ctx
)
Implementation Approach
To put it simply, what we want to do is "add _ctx.
to the beginning of every Identifier (name) on the ExpressionNode".
Let me explain it in a bit more detail.
As a review, a program is represented as an AST by being parsed.
And the AST representing the program has two main types of nodes: Expression and Statement.
These are commonly known as expressions and statements.
1 // This is an Expression
ident // This is an Expression
func() // This is an Expression
ident + func() // This is an Expression
let a // This is a Statement
if (!a) a = 1 // This is a Statement
for (let i = 0; i < 10; i++) a++ // This is a Statement
What we want to consider here is Expression.
There are various types of expressions. Identifier is one of them, which is an expression represented by an identifier.
(You can think of it as a variable name in general)
Identifier appears in various places in an expression.
1 // None
ident // ident --- (1)
func() // func --- (2)
ident + func() // ident, func --- (3)
In this way, Identifier appears in various places in an expression.
You can observe various Identifiers on the ExpressionNode by entering the program on the following site, which allows you to observe the AST.
https://astexplorer.net/#/gist/670a1bee71dbd50bec4e6cc176614ef8/9a9ff250b18ccd9000ed253b0b6970696607b774
Searching for Identifiers
Now that we know what we want to do, how do we implement it?
It seems very difficult, but it is actually simple. We will use a library called estree-walker.
https://github.com/Rich-Harris/estree-walker
We will use this library to walk through the AST obtained by parsing with babel.
The usage is very simple. Just pass the AST to the walk
function and describe the processing for each Node as the second argument.
This walk
function walks through the AST node by node, and the processing at the point when it reaches that Node is done with the enter
option.
In addition to enter
, there are also options such as leave
to process at the end of that Node. We will only use enter
this time.
Create a new file called compiler-core/babelUtils.ts
and implement utility functions that can perform operations on Identifiers.
First, install estree-walker.
npm install estree-walker
npm install -D @babel/types # Also install this
import { Identifier, Node } from '@babel/types'
import { walk } from 'estree-walker'
export function walkIdentifiers(
root: Node,
onIdentifier: (node: Identifier) => void,
) {
;(walk as any)(root, {
enter(node: Node) {
if (node.type === 'Identifier') {
onIdentifier(node)
}
},
})
}
Then, generate the AST for the expression and pass it to this function to perform the transformation while rewriting the nodes.
Implementation of transformExpression
Changes to AST and Parser for InterpolationNode
We will implement the main body of the transformation process, transformExpression.
First, we will modify InterpolationNode so that it has a SimpleExpressionNode instead of a string as its content.
export interface InterpolationNode extends Node {
type: NodeTypes.INTERPOLATION
content: string
content: ExpressionNode
}
With this change, we also need to modify parseInterpolation.
function parseInterpolation(
context: ParserContext,
): InterpolationNode | undefined {
// .
// .
// .
return {
type: NodeTypes.INTERPOLATION,
content: {
type: NodeTypes.SIMPLE_EXPRESSION,
isStatic: false,
content,
loc: getSelection(context, innerStart, innerEnd),
},
loc: getSelection(context, start),
}
}
Implementation of the transformer (main body)
To make the expression transformation usable in other transformers, we will extract it as a function called processExpression
. In transformExpression, we will process the ExpressionNode of INTERPOLATION and DIRECTIVE.
export const transformExpression: NodeTransform = node => {
if (node.type === NodeTypes.INTERPOLATION) {
node.content = processExpression(node.content as SimpleExpressionNode)
} else if (node.type === NodeTypes.ELEMENT) {
for (let i = 0; i < node.props.length; i++) {
const dir = node.props[i]
if (dir.type === NodeTypes.DIRECTIVE) {
const exp = dir.exp
const arg = dir.arg
if (exp && exp.type === NodeTypes.SIMPLE_EXPRESSION) {
dir.exp = processExpression(exp)
}
if (arg && arg.type === NodeTypes.SIMPLE_EXPRESSION && !arg.isStatic) {
dir.arg = processExpression(arg)
}
}
}
}
}
export function processExpression(node: SimpleExpressionNode): ExpressionNode {
// TODO:
}
Next, let's explain the implementation of processExpression. First, we will implement a function called rewriteIdentifier to rewrite Identifier within node. If node is a single Identifier, we simply apply this function and return it.
One thing to note is that this processExpression is specific to SFC (Single File Component) cases (cases without using the with statement). In other words, if the isBrowser flag is set, we implement it to simply return the node. We modify the implementation to receive the flag via ctx.
Also, I want to leave literals like true and false as they are, so I'll create a whitelist for literals.
export function processExpression(
node: SimpleExpressionNode,
ctx: TransformContext,
): ExpressionNode {
if (ctx.isBrowser) {
// Do nothing for the browser
return node
}
const rawExp = node.content
const rewriteIdentifier = (raw: string) => {
return `_ctx.${raw}`
}
if (isSimpleIdentifier(rawExp)) {
node.content = rewriteIdentifier(rawExp)
return node
}
// TODO:
}
makeMap
is a helper function for existence checking implemented in vuejs/core, which returns a boolean indicating whether it matches the string defined with comma separation.
export function makeMap(
str: string,
expectsLowerCase?: boolean,
): (key: string) => boolean {
const map: Record<string, boolean> = Object.create(null)
const list: Array<string> = str.split(',')
for (let i = 0; i < list.length; i++) {
map[list[i]] = true
}
return expectsLowerCase ? val => !!map[val.toLowerCase()] : val => !!map[val]
}
The problem lies in the next step, which is how to transform the SimpleExpressionNode (not a simple Identifier) and transform the node. In the following discussion, please pay attention to the fact that we will be dealing with two different ASTs: the JavaScript AST generated by Babel and the AST defined by chibivue. To avoid confusion, we will refer to the former as estree and the latter as AST in this chapter.
The strategy is divided into two stages.
- Replace the estree node while collecting the node
- Build the AST based on the collected node
First, let's start with stage 1. This is relatively simple. If we can parse the original SimpleExpressionNode content (string) with Babel and obtain the estree, we can pass it through the utility function we created earlier and apply rewriteIdentifier. At this point, we collect the estree node.
import { parse } from '@babel/parser'
import { Identifier } from '@babel/types'
import { walkIdentifiers } from '../babelUtils'
interface PrefixMeta {
start: number
end: number
}
export function processExpression(
node: SimpleExpressionNode,
ctx: TransformContext,
): ExpressionNode {
// .
// .
// .
const ast = parse(`(${rawExp})`).program // ※ This ast refers to estree.
type QualifiedId = Identifier & PrefixMeta
const ids: QualifiedId[] = []
walkIdentifiers(ast, node => {
node.name = rewriteIdentifier(node.name)
ids.push(node as QualifiedId)
})
// TODO:
}
One thing to note is that up to this point, we have only manipulated the estree and have not manipulated the ast node.
CompoundExpression
Next, let's move on to stage 2. Here, we will define a new AST Node called CompoundExpressionNode
. Compound implies "combination" or "complexity". This Node has children, which take slightly special values. First, let's take a look at the definition of AST.
export interface CompoundExpressionNode extends Node {
type: NodeTypes.COMPOUND_EXPRESSION
children: (
| SimpleExpressionNode
| CompoundExpressionNode
| InterpolationNode
| TextNode
| string
)[]
}
Children takes an array like the one shown above. To understand what children in this Node represent, it would be easier to see specific examples, so let's give some examples.
The following expression will be parsed into the following CompoundExpressionNode:
count * 2
{
"type": 7,
"children": [
{
"type": 4,
"isStatic": false,
"content": "_ctx.count"
},
" * 2"
]
}
It's quite a strange feeling. The reason why "children" takes the type of string is because it takes this form.
In CompoundExpression, the Vue compiler divides it into the necessary granularity and expresses it partially as a string or partially as a Node.
Specifically, in cases like this where an Identifier existing in Expression is rewritten, only the Identifier part is divided into another SimpleExpressionNode.
In other words, what we are going to do is to generate this CompoundExpression based on the collected estree's Identifier Node and source.
The following code is the implementation for that.
export function processExpression(node: SimpleExpressionNode): ExpressionNode {
// .
// .
// .
const children: CompoundExpressionNode['children'] = []
ids.sort((a, b) => a.start - b.start)
ids.forEach((id, i) => {
const start = id.start - 1
const end = id.end - 1
const last = ids[i - 1]
const leadingText = rawExp.slice(last ? last.end - 1 : 0, start)
if (leadingText.length) {
children.push(leadingText)
}
const source = rawExp.slice(start, end)
children.push(
createSimpleExpression(id.name, false, {
source,
start: advancePositionWithClone(node.loc.start, source, start),
end: advancePositionWithClone(node.loc.start, source, end),
}),
)
if (i === ids.length - 1 && end < rawExp.length) {
children.push(rawExp.slice(end))
}
})
let ret
if (children.length) {
ret = createCompoundExpression(children, node.loc)
} else {
ret = node
}
return ret
}
The Node parsed by Babel has start and end (location information of where it corresponds to the original string), so we extract the corresponding part from rawExp based on that and divide it diligently.
Please take a close look at the source code for more details. If you understand the policy so far, you should be able to read it. (Also, please take a look at the implementation of advancePositionWithClone, etc., as they are newly implemented.)
Now that we can generate CompoundExpressionNode, let's also support it in Codegen.
function genInterpolation(
node: InterpolationNode,
context: CodegenContext,
option: Required<CompilerOptions>,
) {
genNode(node.content, context, option)
}
function genCompoundExpression(
node: CompoundExpressionNode,
context: CodegenContext,
option: Required<CompilerOptions>,
) {
for (let i = 0; i < node.children!.length; i++) {
const child = node.children![i]
if (isString(child)) {
// If it is a string, push it as it is
context.push(child)
} else {
// For anything else, generate codegen for the Node
genNode(child, context, option)
}
}
}
(genInterpolation has become just genNode, but I'll leave it for now.)
Try it out
Now that we have implemented this far, let's complete the compiler and try running it!
// Add transformExpression
export function getBaseTransformPreset(): TransformPreset {
return [[transformElement], { bind: transformBind }]
return [[transformExpression, transformElement], { bind: transformBind }]
}
import { createApp, defineComponent, ref } from 'chibivue'
const App = defineComponent({
setup() {
const count = ref(3)
const getMsg = (count: number) => `Count: ${count}`
return { count, getMsg }
},
template: `
<div class="container">
<p> {{ 'Message is "' + getMsg(count) + '"'}} </p>
</div>
`,
})
const app = createApp(App)
app.mount('#app')
Source code up to this point: GitHub