Introduction
This post is part of the F# Advent Calendar in English 2018. I’m very grateful to be a part of this and I hope you will enjoy this one as much as I have enjoyed the others.
Today I would like to show you how you can create VS Code extension from scratch using Fable. Along the way we write some code that will extend the Markdown syntax.
Project setup
Code extension
To get started we will follow the general approach of scaffolding a new extension. Using the yeoman generator we will scaffold a new JavaScript extension.
Adding Fable
In order to add Fable we need two main pieces: a couple of npm modules and some NuGet packages.
First, we will add them with yarn.
PS> cd fable-markdown-extension
PS> yarn add -D “fable-compiler” “fable-splitter”
We won’t be using webpack as we don’t really need a bundle in this scenario. When a plugin is installed all JavaScript is executed in a node environment. All files are on disk so there is no need to bundle in order to spare some network requests.
- fable-compiler? The piece of Fable that compiles your F# source code. Don’t be confused by the fact that this is a npm package. It contains a
.NETexecutable and a compiled version ofFable.Core. Although it is not really that important to know all this. - fable-splitter? This npm package can compile multiple F# files linked by a
.fsprojwhen configured correctly. We will use the splitter to compile F# files to JavaScript and nothing more. Under the hoodfable-splitterusesfable-compiler.
Next we need a F# project. This can be a simple netstandard classlib.
PS> dotnet new classlib -lang F# -o src -n AdventExtension
This created a src folder with a Library.fs file in it. Let us try and compile this to JavaScript using fable-splitter.
In our package.json we can add a new script called build.
"scripts": {
"postinstall": "node ./node_modules/vscode/bin/install",
"build": "fable-splitter --config splitter.config.js"
}
And as you would expect we will also need a splitter.config.js file.
The configuration looks like this:
const path = require("path");
function resolve(relativePath) {
return path.join(__dirname, relativePath);
}
module.exports = {
entry: resolve("src/AdventExtension.fsproj"),
outDir: resolve("out"),
allFiles: true
};
Execute by running yarn build. This is the equivalent of running npm run build. Yarn just doesn’t need the run word.
We see that all the files in the project are compiled to the out folder as specified in splitter.config.js.
Now let us rename Library.fs and change it to Extension.fs, compile again and try to run our extension.
The main property of our package.json is pointing to ./extension, so we will need to change that into out/Extension.js.
However if we look at the existing extension.js file it seems to expose an activate function.
// The module 'vscode' contains the VS Code extensibility API
// Import the module and reference it with the alias vscode in your code below
const vscode = require('vscode');
// this method is called when your extension is activated
// your extension is activated the very first time the command is executed
function activate(context) {
// Use the console to output diagnostic information (console.log) and errors (console.error)
// This line of code will only be executed once when your extension is activated
console.log('Congratulations, your extension "fable-markdown-extension" is now active!');
// The command has been defined in the package.json file
// Now provide the implementation of the command with registerCommand
// The commandId parameter must match the command field in package.json
let disposable = vscode.commands.registerCommand('extension.sayHello', function () {
// The code you place here will be executed every time your command is executed
// Display a message box to the user
vscode.window.showInformationMessage('Hello World!');
});
context.subscriptions.push(disposable);
}
exports.activate = activate;
// this method is called when your extension is deactivated
function deactivate() {
}
exports.deactivate = deactivate;
So we will need this in our Fable code as well.
Changing Extension.fs to
module AdventExtension.Extension
let activate _ =
printfn "Fable extension is activated!"
should do the trick.
At this point we have had to compile for a couple times already. It would be easier if we could compile continuously.
This can be achieved by means of another fable-splitter, so let’s add a new build script watch:
"scripts": {
"postinstall": "node ./node_modules/vscode/bin/install",
"build": "fable-splitter --config splitter.config.js",
"watch": "yarn run build --watch"
}
yarn watch
Our compiled code looks like
import { toConsole, printf } from "./fable-library.2.1.8/String";
export function activate(_arg1) {
toConsole(printf("Fable extension is activated!"));
}
and we can launch the extension in code by hitting the play button in the Debug panel.
However in our current setup we want our extension to run when VS Code starts.
Change the activationEvents to * in the package.json.
"activationEvents": [
"*"
]
and start debugging.
This opens a new instance of vscode and should contain our extension.
Yikes, this isn’t really doing anything at all!
We even see a crash when we toggle the developer tools.
What code is telling us here is that it doesn’t know how to deal with the import and export keywords.
I believe this whole thing is running in a node context, so we should be using module.exports instead.
One small Babel plugin can do this for us. We need to download it with yarn and tell fable-splitter to use it as well.
PS> yarn add -D “@babel/plugin-transform-modules-commonjs” “@babel/core”
splitter.config.js becomes
const path = require("path");
function resolve(relativePath) {
return path.join(__dirname, relativePath);
}
module.exports = {
entry: resolve("src/AdventExtension.fsproj"),
outDir: resolve("out"),
babel: {
plugins: ["@babel/plugin-transform-modules-commonjs"]
},
allFiles: true
};
The JavaScript output now looks like
"use strict";
Object.defineProperty(exports, "__esModule", {
value: true
});
exports.activate = activate;
var _String = require("./fable-library.2.1.8/String");
function activate(_arg1) {
(0, _String.toConsole)((0, _String.printf)("Fable extension is activated!"));
}
and our extension now runs!
F# dependencies
Ok, after some trial and error we have the basics in place. However, a little intelliSense would not hurt.
There is a NuGet package for that: Fable.Import.VSCode.
We could install it into our fsproj with dotnet add, but I would prefer to download it with paket instead.
Paket can easily be added to a project using:
PS> dotnet tool install —tool-path “.paket” Paket —add-source https://api.nuget.org/v3/index.json
This downloads the .NETCore version of Paket into a .paket folder. This should be familiar to most of us.
Next we enter .paket\paket.exe init.
Open the paket.dependencies file and add
source https://www.nuget.org/api/v2
storage:none
framework: netstandard2.0
# nuget Fable.Import.Jest // Use when ready for Fable2
github nojaf/fable-jest:fable2 fable/Bindings.fs
github nojaf/fable-jest:fable2 fable/Exports.fs
github nojaf/fable-jest:fable2 fable/Matchers.fs
nuget FSharp.Core redirects:force
nuget Fable.Import.VSCode
nuget Fable.Parsimmon
It will download VS Code bindings, Jest bindings and Fable.Parsimmon. I’ll explain later why we need those.
Then, create a paket.references file next to the fsproj.
Add
Fable.Import.VSCode
Fable.Parsimmon
File:Bindings.fs Jest
File:Exports.fs Jest
File:Matchers.fs Jest
Run .paket\paket.exe install, which should add everything on .NET side.
Those last npm packages
To wrap up the setup of the project, we still need a few npm packages.
yarn add -D “jest” “markdown-it”
That should be it. Thanks for sticking around if you are still here 😊.
Now we can finally start coding.
Revelations
I feel like we have been through a lot already, and I haven’t really revealed the master plan of all this.
I mentioned that we would extend Markdown. In this blogpost we will add two new features:
- Fable logo
- Fable font
Fable logo
Whenever we type :fable: in our Markdown file, we would like to the Fable logo to show in our preview.
Fable text
If we add text between two 🐉 emoji, it should rendered in a different font.
Both features should also be highlighted in the editor itself.
Test-driven development
Now that we know what we want, we can start thinking about solving the problem.
Without considering VS Code we know that we will need a way to detect:fable: and text between 🐉.
What better way of parsing text than using Fable.Parsimmon?
If you have any experience with FParsec, the API is very similar.
In order to have a short feedback loop, we can easily do some TDD instead of trying things out in the extension itself.
We already installed Jest and the accompanying F# bindings. Note that these bindings were installed with paket through github instead of nuget. This has to do with the fact that Fable.Jest doesn’t work 100% with Fable 2.X.
My fork of Fable.Jest contains a workaround for this.
Here you can already see that using paket instead of NuGet was a smart move.
Unit testing in Fable is described to some extent in the FAQ. In our case everything is almost fully set up. We need to change the test script in our package.json.
"scripts": {
"postinstall": "node ./node_modules/vscode/bin/install",
"build": "fable-splitter --config splitter.config.js",
"watch": "yarn run build --watch",
"test": "jest"
},
Add a file that matches the following convention *.test.fs to our project, and the *.test.js should be picked up by Jest.
Parser.test.fs with a simple unit test should be enough for now.
module AdventExtension.Tests.Parser
open Fable.Import.Jest
open Fable.Import.Jest.Matchers
test "simple Jest test should be picked up" <| fun () ->
"42" == "the answer of the Universe"
Execute with
PS> yarn build
PS> yarn test
Coolio, we can unit test now! The way I like to proceed now is having two console windows. One to run fable-splitter with yarn watch, and other to run tests with yarn test --watchAll.
Fable.Parsimmon
Since we are going to parse text, we need a type to capture the result of our parsing.
type ParserResult =
| Text of string
| FableFontText of NodeResult<string>
| FableLogo of NodeResult<string>
We may now have found some text, text between dragons (🐉) or :fable:.
When we have found one or both of the latter two, we would need to know where in our text we have found them.
This is where Parsimmon.node comes in handy, as it returns a NodeResult<'t>.
type TokenPosition =
{ offset: int
line: int
column: int }
type NodeResult<'t> =
{ name: string
value: 't
start: TokenPosition
``end``: TokenPosition }
Next we need a function that takes a string and returns Fable.Parsimmon.ParseResult<ParserResult array>.
let parseText input : ParseResult<ParserResult array> = failwith "Nothing here yet."
And we can quickly write some tests.
test "basic fable font" <| fun () ->
let input = "🐉vibes🐉"
let parsed = Parser.parseText input
parsed.status == true
test "no fable font" <| fun () ->
let input = "fableless vibes"
let parsed = Parser.parseText input
parsed.status == true
test "fable logo is found" <| fun () ->
let input = "Once upon a :fable:..."
let parsed = Parser.parseText input
parsed.status == true
match parsed.value with
| [| Text _; FableLogo logo; Text _|] ->
logo.start.offset == 12
logo.value.Length == 7
| _ -> failwith "Expected logo in center"
Notice that == is a shorthand for Fable.Import.Jest.Exports.expect.Invoke("foo").toBe("bar").
To make these tests pass we’ll write some custom parsers, mostly based on regular expressions.
module AdventExtension.Parser
open Fable.Parsimmon
open Fable.Core
open System.Text.RegularExpressions
type ParserResult =
| Text of string
| FableFontText of NodeResult<string>
| FableLogo of NodeResult<string>
// no /g flag
[<Emit("new RegExp($0)")>]
let createRegex pattern : Regex = jsNative
let dragonParser = Parsimmon.str "🐉"
let noDragonParser =
Parsimmon.regex (createRegex @"[^🐉]+")
let fableFontParser =
Parsimmon.between dragonParser dragonParser noDragonParser
|> Parsimmon.node "FableFont"
|> Parsimmon.map (ParserResult.FableFontText)
let halfDragon =
"🐉".ToCharArray()
|> Array.head
|> (string)
let noDragonsParser =
Parsimmon.satisfy (fun token -> token <> halfDragon && token <> ":")
|> Parsimmon.atLeastOneOrMany
|> Parsimmon.concat
|> Parsimmon.map (ParserResult.Text)
let fableLogoParser =
Parsimmon.str ":fable:"
|> Parsimmon.node "FableLogo"
|> Parsimmon.map (FableLogo)
let parseText input : ParseResult<ParserResult array> =
Parsimmon.choose [
fableLogoParser
noDragonsParser
fableFontParser
]
|> fun parser -> parser.many().parse(input)
I won’t go into detail about exactly how this works but I do want to highlight two things:
"🐉".Lengthreturns2. AndParsimmon.satisfyonly compares a single token at a time.- Creating a
Regexvia the .NET API compiles to a JavaScript Regex that uses the/gflag. This is something we don’t want in this case, so I’ve added a little helper that emits aRegex.
Presumably this code is more or less readable, and we can continue 😅.
Highlighting code
Highlighting in VS Code is surprisingly easily. All we need to do is tell code which range should be decorated with which style.
Let’s first create some TextDecorationEditorStyle objects.
module AdventExtension.Extension
open Fable.Core.JsInterop
open Fable.Import.vscode
let createFontOptions color backgroundColor fontWeight : DecorationRenderOptions =
jsOptions (fun options ->
options.color <- Some color
options?fontWeight <- fontWeight
options.backgroundColor <- backgroundColor
)
let activate _ =
let fableFontStyle = window.createTextEditorDecorationType(createFontOptions "#87c5fd" (Some "white") None)
let fableLogoStyle = window.createTextEditorDecorationType(createFontOptions "#87c5fd" None (Some "bold"))
printfn "Fable extension activated!"
Now we need to subscribe to certain events.
let activate (context: ExtensionContext) =
let fableFontStyle = window.createTextEditorDecorationType(createFontOptions "#87c5fd" (Some "white") None)
let fableLogoStyle = window.createTextEditorDecorationType(createFontOptions "#87c5fd" None (Some "bold"))
// Capture the active editor.
let mutable activeEditor = window.activeTextEditor
let updateDecorations() = failwithf "Nothing here yet."
let mutable timeoutKey = None;
let triggerUpdateDecorations _ =
timeoutKey
|> Option.iter (JS.clearTimeout)
timeoutKey <- Some (JS.setTimeout updateDecorations 500)
// User is typing
window.onDidChangeActiveTextEditor.Invoke((fun editor ->
activeEditor <- Some editor
triggerUpdateDecorations()
null
))
|> context.subscriptions.Add
// Change of document
workspace.onDidChangeTextDocument.Invoke((fun event ->
match activeEditor with
| Some aEditor when (event.document = aEditor.document) ->
triggerUpdateDecorations()
| _ -> ()
null
))
|> context.subscriptions.Add
activeEditor
|> Option.iter (triggerUpdateDecorations)
printfn "Fable extension activated!"
Notice that the event handlers call triggerUpdateDecorations, which prevents us from re-rendering too much.
// Parsimmon ranges are 1-based, VS Code works with 0-based indexing.
let nodeResultToRange<'t> (nr: NodeResult<'t>) =
let zeroBasedFloat r = r - 1 |> (float)
vscode.Range(zeroBasedFloat nr.start.line,
zeroBasedFloat nr.start.column,
zeroBasedFloat nr.``end``.line,
zeroBasedFloat nr.``end``.column)
let updateDecorations() =
activeEditor
|> Option.iter (fun aEditor ->
if aEditor.document.languageId = "markdown" then
let text = aEditor.document.getText()
let parsed = Parser.parseText text
if parsed.status then
parsed.value
|> Array.map (fun node ->
match node with
| ParserResult.FableLogo logo ->
Some (fableLogoStyle, nodeResultToRange logo)
| ParserResult.FableFontText fableText ->
Some (fableFontStyle, nodeResultToRange fableText)
| _ ->
None
)
|> Array.choose id
|> Array.groupBy (fun (style,_) -> style.key)
|> Array.iter (fun (styleKey, ranges) ->
let r =
Array.map snd ranges
|> ResizeArray
let style = if fableFontStyle.key = styleKey then fableFontStyle else fableLogoStyle
aEditor.setDecorations(style, !^ r)
)
)
updateDecorations is rather straightforward as well. We map the result of parseText and make ranges out of it. The result looks as follows:
Preview mode
When we switch to preview mode of the Markdown file, we don’t really see any changes.
This can be changed by writing our own markdown-it extension.
To be fair, it took me a while to figure out how this works. Even after reading the documentation, I was still somewhat puzzled.
What markdown-it does is it parses the Markdown content to a token stream.
Ex.
This is some **markdown text**
Will be parsed to
[
{
"type": "paragraph_open",
"tag": "p",
},
{
"type": "inline",
"tag": "",
"map": [
0,
1
],
"nesting": 0,
"level": 1,
"children": [
{
"type": "text",
"tag": "",
"content": "This is some ",
},
{
"type": "strong_open",
"tag": "strong",
},
{
"type": "text",
"tag": "",
"content": "markdown text",
},
{
"type": "strong_close",
"tag": "strong",
},
{
"type": "text",
"tag": "",
}
],
"content": "This is some **markdown text**",
},
{
"type": "paragraph_close",
"tag": "p",
}
]
Note that this is a simplified version. We need to break down any tokens of type:"text" and replace them with our own token type (fable-logo and fable-text).
Once we’ve updated the tree, we can hook into the render engine by adding custom rules.
F# bindings
I found that there are TypeScript bindings on npm. With ts2fable I cooked up some minimal F# bindings which we can use.
Unit tests
Since this is a rather tricky situation, we should add some tests first.
module AdventExtension.Tests.MarkdownPlugin
open Fable.Import.Jest
open AdventExtension.MarkdownPlugin
jest.unmock("markdown-it")
let md = MarkdownItModule.Exports.Invoke().``use``(fableMarkdownPlugin)
test "fable img is rendered" <| fun () ->
let input = ":fable:"
let parsed = md.render input
expect.Invoke(parsed.StartsWith("<p><img class='fable-logo' src")).toBeTruthy()
test "fable font is rendered" <| fun () ->
let input = "Over the seas 🐉we shall rise🐉!"
let parsed = md.render input
expect.Invoke(parsed.Contains("<span class='fable'>we shall rise</span>")).toBeTruthy()
The plugin function fableMarkdownPlugin looks like
module AdventExtension.MarkdownPlugin
open MarkdownItModule
let fableMarkdownPlugin (md: MarkdownIt) (options: Options) = failwith "Nothing yet"
Inside this function, we need to replace the tokens and add custom render rules.
module AdventExtension.MarkdownPlugin
open MarkdownItModule
open Fable.Core
open AdventExtension.Parser
[<Emit("new $0.Token($1,$2,$3)")>]
let createToken state name tag nesting: Token = jsNative
[<Emit("$0[$1] = $2")>]
let addRenderRule rules name fn: unit = jsNative
let fableMarkdownPlugin (md: MarkdownIt) (options: Options) =
// Phase one: replace text tokens to custom tokens
let replaceTokens (state: State) =
let parseTextToken (parentToken: Token) (textToken: Token) =
let textContent = textToken.content
let parsedContent = parseText textContent
let updatedChildren =
if not parsedContent.status then
Array.empty
else
parsedContent.value
|> Array.map (fun node ->
match node with
| ParserResult.Text text ->
let t = createToken state "text" "" 0
t.content <- text
t
| ParserResult.FableLogo _ ->
createToken state "fable-logo" "img" 0
| ParserResult.FableFontText fontText ->
let t = createToken state "fable-font" "span" 0
t.content <- fontText.value.Replace("🐉", "")
t
)
parentToken.children <- new ResizeArray<Token>(updatedChildren)
let rec parseToken (parentToken: Token) (token: Token) =
if token.children <> null && not(Seq.isEmpty token.children) then
token.children
|> Seq.iter (fun childToken ->
if childToken.``type`` = "inline" then
parseToken token childToken
else
parseTextToken token childToken
)
if token.``type`` = "text" then
parseTextToken parentToken token
state.tokens
|> Seq.filter (fun t -> t.``type`` = "inline" && t.children <> null && not (Seq.isEmpty t.children))
|> Seq.iter (fun token ->
token.children
|> Seq.iter (parseToken token)
)
md.core.ruler.push("fable", replaceTokens)
This code might not look like much, but we are traversing the tree, replacing the children of a parent node that contains text.
Our unit tests remain unaffected, as the render part will just ignore the fable-logo and fable-text tokens and render them as regular text.
Part two determines how our own nodes should render:
[<Emit("$0[$1] = $2")>]
let addRenderRule rules name fn: unit = jsNative
let fableMarkdownPlugin (md: MarkdownIt) (options: Options) = ...
// Phase one ...
// Phase two
let renderFableLogo tokens idx = "<img class='fable-logo' src='data:image/png;base64,iVBORw0KGgoAAAANSUhEU...' />"
let renderFableText (tokens: Token array) (idx: int) =
let token = tokens.[idx]
sprintf "<span class='fable'>%s</span>" (token.content.Replace("🐉", ""))
addRenderRule md.renderer.rules "fable-logo" (new System.Func<Token array,int,string>(fun tokens idx -> renderFableLogo tokens idx))
addRenderRule md.renderer.rules "fable-font" (new System.Func<Token array,int,string>(fun tokens idx -> renderFableText tokens idx))
We need to wrap our render functions inside a System.Func<_,_,_> in order to preserve compatibility with JavaScript.
The reason for this is that Fable compiles these into functions that can be curried, and markdown-it doesn’t like this.
Once more our tests show green, and the times has come to shove this into our VS Code extension.
Tag on code
To see all this in code we need to extend the contributes section in our package.json
"contributes": {
"markdown.markdownItPlugins": true,
"markdown.previewStyles": [
"./fable.css"
]
}
Create a fable.css file
// this works in code, I was somewhat surprised by this. Should be a local url in real world scenario.
@import url('https://fonts.googleapis.com/css?family=Josefin+Slab:400,600i');
.fable {
font-family: 'Josefin Slab', serif;
font-weight: 600;
font-style: italic;
color: #87c5fd;
font-size: 18px;
}
.fable-logo {
display: inline-block;
width: 24px;
}
And make the final change to Extension.fs by adding
// Add render plugin
createObj [
"extendMarkdownIt" ==> (fun md -> md?``use``(MarkdownPlugin.fableMarkdownPlugin))
]
at the end.
Final outcome:
The Source
You can find all code on my github.
Remarks
- The extension has not been published in the store, nor do I have the intention to so.
- This was my first VS Code and markdown-it plugin, so there is some room for improvement.
fable-splitteris very fast. I didn’t missfable-loaderandwebpackfor a single moment.- Many thanks to Zaid Ajaj for the help with Fable.Parsimmon.
Final words
I hope you enjoyed this blogpost and it all makes sense to you. If you have any suggestions or questions, please leave a comment.
Yours truly,
nojaf
Photo by Massimo Rinaldi