Implementing a Template Compiler

Implementation Approach

The basic approach is to manipulate the string passed through the template option to generate specific functions.
Let's divide the compiler into three elements.

Parsing

Parsing involves extracting necessary information from the given string. You can think of it like this:

const { tag, props, textContent } = parse(`<p class="hello">Hello World</p>`)
console.log(tag) // "p"
console.log(prop) // { class: "hello" }
console.log(textContent) // "Hello World"

Code Generation

Code generation generates code (strings) based on the result of parsing.

const code = codegen({ tag, props, textContent })
console.log(code) // "h('p', { class: 'hello' }, ['Hello World']);"

Function Object Generation

Function object generation creates executable functions based on the code (strings) generated by codegen.
In JavaScript, you can generate functions from strings using the Function constructor.

const f = new Function('return 1')
console.log(f()) // 1

// If you want to define arguments, you can do it like this
const add = new Function('a', 'b', 'return a + b')
console.log(add(1, 1)) // 2

We will use this to generate functions.
One thing to note here is that the generated function can only handle variables defined within it, so we need to include the import of functions like the h function in it.

import * as runtimeDom from './runtime-dom'
const render = new Function('ChibiVue', code)(runtimeDom)

By doing this, we can receive runtimeDom as ChibiVue and include the h function in the codegen stage as follows:

const code = codegen({ tag, props, textContent })
console.log(code) // "return () => { const { h } = ChibiVue; return h('p', { class: 'hello' }, ['Hello World']); }"

In other words, earlier we said that we would convert it like this:

;`<p class="hello">Hello World</p>`
// ↓
h('p', { class: 'hello' }, ['Hello World'])

But to be precise, we convert it like this:

;`<p class="hello">Hello World</p>`

// ↓

ChibiVue => {
  return () => {
    const { h } = ChibiVue
    return h('p', { class: 'hello' }, ['Hello World'])
  }
}

And pass runtimeDom to generate the render function.
The responsibility of codegen is to generate the following string:

const code = `
  return () => {
      const { h } = ChibiVue;
      return h("p", { class: "hello" }, ["Hello World"]);
  };
`

Implementation

Once you understand the approach, let's implement it.
Create a directory called compiler-core in ~/packages and create index.ts, parse.ts, and codegen.ts in it.

pwd # ~/
mkdir packages/compiler-core
touch packages/compiler-core/index.ts
touch packages/compiler-core/parse.ts
touch packages/compiler-core/codegen.ts

index.ts is only used for exporting as usual.

Now let's start implementing from parse. packages/compiler-core/parse.ts

export const baseParse = (
  content: string,
): { tag: string; props: Record<string, string>; textContent: string } => {
  const matched = content.match(/<(\w+)\s+([^>]*)>([^<]*)<\/\1>/)
  if (!matched) return { tag: '', props: {}, textContent: '' }

  const [_, tag, attrs, textContent] = matched

  const props: Record<string, string> = {}
  attrs.replace(/(\w+)=["']([^"']*)["']/g, (_, key: string, value: string) => {
    props[key] = value
    return ''
  })

  return { tag, props, textContent }
}

Although it is a very simple parser using regular expressions, it is sufficient for the first implementation.

Next, let's generate the code. Implement it in codegen.ts.
packages/compiler-core/codegen.ts

export const generate = ({
  tag,
  props,
  textContent,
}: {
  tag: string
  props: Record<string, string>
  textContent: string
}): string => {
  return `return () => {
  const { h } = ChibiVue;
  return h("${tag}", { ${Object.entries(props)
    .map(([k, v]) => `${k}: "${v}"`)
    .join(', ')} }, ["${textContent}"]);
}`
}

Now, let's implement a function that generates a function string from a template by combining these.
Create a new file called packages/compiler-core/compile.ts.

packages/compiler-core/compile.ts

import { generate } from './codegen'
import { baseParse } from './parse'

export function baseCompile(template: string) {
  const parseResult = baseParse(template)
  const code = generate(parseResult)
  return code
}

This shouldn't be too difficult. In fact, the responsibility of compiler-core ends here.

Runtime Compiler and Build Process Compiler

In fact, Vue has two types of compilers.
One is the compiler that runs on the runtime (in the browser), and the other is the compiler that runs in the build process (such as Node.js).
Specifically, the runtime compiler is responsible for compiling the template option or the template provided as HTML, while the build process compiler is responsible for compiling SFC (or JSX).
The template option we are currently implementing falls into the former category.

const app = createApp({ template: `<p class="hello">Hello World</p>` })
app.mount('#app')

html

<div id="app"></div>

The template provided as HTML is a developer interface where you write Vue templates in HTML.
(It is convenient for quickly incorporating it into HTML via CDN, etc.)

const app = createApp()
app.mount('#app')

html

<div id="app">
  <p class="hello">Hello World</p>
  <button @click="() => alert('hello')">click me!</button>
</div>

Both of these need to be compiled, but the compilation is performed in the browser.

On the other hand, SFC compilation is performed during the project build, and only the compiled code exists on the runtime.
(You need to set up a bundler such as Vite or webpack in your development environment.)

vue

<!-- App.vue -->
<script>
export default {}
</script>

<template>
  <p class="hello">Hello World</p>
  <button @click="() => alert("hello")">click me!</button>
</template>

import App from 'App.vue'
const app = createApp(App)
app.mount('#app')

html

<div id="app"></div>

The important point to note is that both compilers share common processing.
The source code for this common part is implemented in the compiler-core directory.
And the runtime compiler and SFC compiler are implemented in the compiler-dom and compiler-sfc directories, respectively.
Please take a look at this diagram again.

https://github.com/vuejs/core/blob/main/.github/contributing.md#package-dependencies

Continued Implementation

We've jumped ahead a bit, but let's continue with the implementation.
Although I would like to implement package/index.ts, there is some preparation work to be done, so let's do that first.
The preparation work is to implement a variable in package/runtime-core/component.ts to hold the compiler itself, and a registration function.

package/runtime-core/component.ts

type CompileFunction = (template: string) => InternalRenderFunction
let compile: CompileFunction | undefined

export function registerRuntimeCompiler(_compile: any) {
  compile = _compile
}

Now, let's generate the function in package/index.ts and register it.

import { compile } from './compiler-dom'
import { InternalRenderFunction, registerRuntimeCompiler } from './runtime-core'
import * as runtimeDom from './runtime-dom'

function compileToFunction(template: string): InternalRenderFunction {
  const code = compile(template)
  return new Function('ChibiVue', code)(runtimeDom)
}

registerRuntimeCompiler(compileToFunction)

export * from './runtime-core'
export * from './runtime-dom'
export * from './reactivity'

※ Don't forget to export the h function from runtime-dom because it needs to be included in runtimeDom.

export { h } from '../runtime-core'

Now that the compiler is registered, let's actually perform the compilation.
Since the template is required in the component options type, let's add the template for now.

export type ComponentOptions = {
  props?: Record<string, any>
  setup?: (
    props: Record<string, any>,
    ctx: { emit: (event: string, ...args: any[]) => void },
  ) => Function
  render?: Function
  template?: string // Added
}

Now, let's compile the important part.

const mountComponent = (initialVNode: VNode, container: RendererElement) => {
  const instance: ComponentInternalInstance = (initialVNode.component =
    createComponentInstance(initialVNode))

  // ----------------------- From here
  const { props } = instance.vnode
  initProps(instance, props)
  const component = initialVNode.type as Component
  if (component.setup) {
    instance.render = component.setup(instance.props, {
      emit: instance.emit,
    }) as InternalRenderFunction
  }
  // ----------------------- To here

  setupRenderEffect(instance, initialVNode, container)
}

We will extract the above part in package/runtime-core/component.ts.

package/runtime-core/component.ts

export const setupComponent = (instance: ComponentInternalInstance) => {
  const { props } = instance.vnode
  initProps(instance, props)

  const component = instance.type as Component
  if (component.setup) {
    instance.render = component.setup(instance.props, {
      emit: instance.emit,
    }) as InternalRenderFunction
  }
}

package/runtime-core/renderer.ts

const mountComponent = (initialVNode: VNode, container: RendererElement) => {
  // prettier-ignore
  const instance: ComponentInternalInstance = (initialVNode.component = createComponentInstance(initialVNode));
  setupComponent(instance)
  setupRenderEffect(instance, initialVNode, container)
}

Now, let's perform the compilation inside the setupComponent function.

export const setupComponent = (instance: ComponentInternalInstance) => {
  const { props } = instance.vnode
  initProps(instance, props)

  const component = instance.type as Component
  if (component.setup) {
    instance.render = component.setup(instance.props, {
      emit: instance.emit,
    }) as InternalRenderFunction
  }

  // ------------------------ Here
  if (compile && !component.render) {
    const template = component.template ?? ''
    if (template) {
      instance.render = compile(template)
    }
  }
}

Now, we should be able to compile simple HTML using the template option.
Let's try it in the playground!

const app = createApp({ template: `<p class="hello">Hello World</p>` })
app.mount('#app')

simple_template_compiler

It seems to be working fine.
Let's try making some changes to see if they are reflected.

const app = createApp({
  template: `<b class="hello" style="color: red;">Hello World!!</b>`,
})
app.mount('#app')

simple_template_compiler2

It appears to be implemented correctly!

Source code up to this point:
chibivue (GitHub)

Implementing a Template Compiler ​

Implementation Approach ​

Parsing ​

Code Generation ​

Function Object Generation ​

Implementation ​

Runtime Compiler and Build Process Compiler ​

Continued Implementation ​

Implementing a Template Compiler

Implementation Approach

Parsing

Code Generation

Function Object Generation

Implementation

Runtime Compiler and Build Process Compiler

Continued Implementation