📈 Chartem Ipsum
14 Sep, 2020Integrating as many declarative visualization libraries as I can into Markdown.
Markdown is great for its declarative syntax, but I want to extend it to allow charts and diagrams to be embedded. I’ve played with this previously, but I used custom code or only a couple libraries.
This is an attempt to embed all the libraries I’m aware of that can handle declarative charting in a browser.
While I am using shortcodes for fencing (since Markdown doesn’t have a generic container syntax yet), there is no raw HTML or JavaScript in the file this page is generated from. Also, as a note, most examples are taken from the gallery/documentation of the library they come from. If so, they’re either from the home page of the library’s website, or I linked the example.
Complex Libraries
These libraries are big and verbose, but effective at visualizing large amounts of data and providing complex interactions. If you’re trying to display quantitative data, these are the ones you’re almost certainly going to use for declarative charting.
Vega
Vega is a visualization grammar that describes how data is processed and rendered. Vega can be rather verbose, but it allows for an extremely high amount of control over how data is processed and allows complex interactions, while still being delarative.
Here is the Airport Connections Example to show both rendering and interaction of data:
Code
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"description": "Interactive map of U.S. airport connections in 2008.",
"width": 900,
"height": 560,
"padding": {"top": 25, "left": 0, "right": 0, "bottom": 0},
"autosize": "none",
"signals": [
{
"name": "scale", "value": 1200,
"bind": {"input": "range", "min": 500, "max": 3000}
},
{
"name": "translateX", "value": 450,
"bind": {"input": "range", "min": -500, "max": 1200}
},
{
"name": "translateY", "value": 260,
"bind": {"input": "range", "min": -300, "max": 700}
},
{
"name": "shape", "value": "line",
"bind": {"input": "radio", "options": ["line", "curve"]}
},
{
"name": "hover",
"value": null,
"on": [
{"events": "@cell:mouseover", "update": "datum"},
{"events": "@cell:mouseout", "update": "null"}
]
},
{
"name": "title",
"value": "U.S. Airports, 2008",
"update": "hover ? hover.name + ' (' + hover.iata + ')' : 'U.S. Airports, 2008'"
},
{
"name": "cell_stroke",
"value": null,
"on": [
{"events": "dblclick", "update": "cell_stroke ? null : 'brown'"},
{"events": "mousedown!", "update": "cell_stroke"}
]
}
],
"data": [
{
"name": "states",
"url": "https://vega.github.io/vega/data/us-10m.json",
"format": {"type": "topojson", "feature": "states"},
"transform": [
{
"type": "geopath",
"projection": "projection"
}
]
},
{
"name": "traffic",
"url": "https://vega.github.io/vega/data/flights-airport.csv",
"format": {"type": "csv", "parse": "auto"},
"transform": [
{
"type": "aggregate",
"groupby": ["origin"],
"fields": ["count"], "ops": ["sum"], "as": ["flights"]
}
]
},
{
"name": "airports",
"url": "https://vega.github.io/vega/data/airports.csv",
"format": {"type": "csv","parse": "auto"
},
"transform": [
{
"type": "lookup",
"from": "traffic", "key": "origin",
"fields": ["iata"], "as": ["traffic"]
},
{
"type": "filter",
"expr": "datum.traffic != null"
},
{
"type": "geopoint",
"projection": "projection",
"fields": ["longitude", "latitude"]
},
{
"type": "filter",
"expr": "datum.x != null && datum.y != null"
},
{
"type": "voronoi", "x": "x", "y": "y"
},
{
"type": "collect", "sort": {
"field": "traffic.flights",
"order": "descending"
}
}
]
},
{
"name": "routes",
"url": "https://vega.github.io/vega/data/flights-airport.csv",
"format": {"type": "csv", "parse": "auto"},
"transform": [
{
"type": "filter",
"expr": "hover && hover.iata == datum.origin"
},
{
"type": "lookup",
"from": "airports", "key": "iata",
"fields": ["origin", "destination"], "as": ["source", "target"]
},
{
"type": "filter",
"expr": "datum.source && datum.target"
},
{
"type": "linkpath",
"shape": {"signal": "shape"}
}
]
}
],
"projections": [
{
"name": "projection",
"type": "albersUsa",
"scale": {"signal": "scale"},
"translate": [{"signal": "translateX"}, {"signal": "translateY"}]
}
],
"scales": [
{
"name": "size",
"type": "linear",
"domain": {"data": "traffic", "field": "flights"},
"range": [16, 1000]
}
],
"marks": [
{
"type": "path",
"from": {"data": "states"},
"encode": {
"enter": {
"fill": {"value": "#dedede"},
"stroke": {"value": "white"}
},
"update": {
"path": {"field": "path"}
}
}
},
{
"type": "symbol",
"from": {"data": "airports"},
"encode": {
"enter": {
"size": {"scale": "size", "field": "traffic.flights"},
"fill": {"value": "steelblue"},
"fillOpacity": {"value": 0.8},
"stroke": {"value": "white"},
"strokeWidth": {"value": 1.5}
},
"update": {
"x": {"field": "x"},
"y": {"field": "y"}
}
}
},
{
"type": "path",
"name": "cell",
"from": {"data": "airports"},
"encode": {
"enter": {
"fill": {"value": "transparent"},
"strokeWidth": {"value": 0.35}
},
"update": {
"path": {"field": "path"},
"stroke": {"signal": "cell_stroke"}
}
}
},
{
"type": "path",
"interactive": false,
"from": {"data": "routes"},
"encode": {
"enter": {
"path": {"field": "path"},
"stroke": {"value": "black"},
"strokeOpacity": {"value": 0.35}
}
}
},
{
"type": "text",
"interactive": false,
"encode": {
"enter": {
"x": {"value": 895},
"y": {"value": 0},
"fill": {"value": "black"},
"fontSize": {"value": 20},
"align": {"value": "right"}
},
"update": {
"text": {"signal": "title"}
}
}
}
]
}
Vega Lite
Vega-Lite is a higher-level grammar on top of Vega which can map properties and automatically produce many interactive components.
The same airport visualization above also has an example in Vega-Lite, and the code, while still somewhat verbose, is 40% the length (91 lines vs 227 lines):
Code
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "An interactive visualization of connections among major U.S. airports in 2008. Based on a U.S. airports example by Mike Bostock.",
"layer": [
{
"mark": {
"type": "geoshape",
"fill": "#ddd",
"stroke": "#fff",
"strokeWidth": 1
},
"data": {
"url": "https://vega.github.io/vega/data/us-10m.json",
"format": {"type": "topojson", "feature": "states"}
}
},
{
"mark": {"type": "rule", "color": "#000", "opacity": 0.35},
"data": {"url": "https://vega.github.io/vega/data/flights-airport.csv"},
"transform": [
{"filter": {"selection": "single"}},
{
"lookup": "origin",
"from": {
"data": {"url": "https://vega.github.io/vega/data/airports.csv"},
"key": "iata",
"fields": ["latitude", "longitude"]
}
},
{
"lookup": "destination",
"from": {
"data": {"url": "https://vega.github.io/vega/data/airports.csv"},
"key": "iata",
"fields": ["latitude", "longitude"]
},
"as": ["lat2", "lon2"]
}
],
"encoding": {
"latitude": {"field": "latitude"},
"longitude": {"field": "longitude"},
"latitude2": {"field": "lat2"},
"longitude2": {"field": "lon2"}
}
},
{
"mark": {"type": "circle"},
"data": {"url": "https://vega.github.io/vega/data/flights-airport.csv"},
"transform": [
{"aggregate": [{"op": "count", "as": "routes"}], "groupby": ["origin"]},
{
"lookup": "origin",
"from": {
"data": {"url": "https://vega.github.io/vega/data/airports.csv"},
"key": "iata",
"fields": ["state", "latitude", "longitude"]
}
},
{"filter": "datum.state !== 'PR' && datum.state !== 'VI'"}
],
"selection": {
"single": {
"type": "single",
"on": "mouseover",
"nearest": true,
"fields": ["origin"],
"empty": "none"
}
},
"encoding": {
"latitude": {"field": "latitude"},
"longitude": {"field": "longitude"},
"size": {
"field": "routes",
"type": "quantitative",
"scale": {"rangeMax": 1000},
"legend": null
},
"order": {
"field": "routes",
"sort": "descending"
}
}
}
],
"projection": {"type": "albersUsa"},
"width": 900,
"height": 500,
"config": {"view": {"stroke": null}}
}
To allow a fair comparison later, here’s the Simple Bar Chart example to show that basic charts are pretty easy:
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"description": "A simple bar chart with embedded data.",
"data": {
"values": [
{"a": "A", "b": 28}, {"a": "B", "b": 55}, {"a": "C", "b": 43},
{"a": "D", "b": 91}, {"a": "E", "b": 81}, {"a": "F", "b": 53},
{"a": "G", "b": 19}, {"a": "H", "b": 87}, {"a": "I", "b": 52}
]
},
"mark": "bar",
"encoding": {
"x": {"field": "a", "type": "nominal", "axis": {"labelAngle": 0}},
"y": {"field": "b", "type": "quantitative"}
}
}
ECharts
ECharts is a project in the Apache incubator. While it’s not explicitly declarative, it can do many types of charts in a declarative nature.
Basic charts are fairly simple, such as the Simple Bar Chart example:
{
"xAxis": {
"type": "category",
"data": ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
},
"yAxis": {
"type": "value"
},
"series": [
{
"data": [120, 200, 150, 80, 70, 110, 130],
"type": "bar"
}
]
}
If you’re willing to have a really long entry in your Markdown, it is technically possible to have a complex graph, such as the Sunburst Demo:
ECharts seems to be a little simpler than Vega-Lite if you want basic charts, particularly if you want a little out-of-the-box functionality. However, you get limited fast if you want to add interaction or load data from the web if you want to still have your chart be declarative; unlike Vega/Vega-Lite, ECharts does not provide any built-in configuration for common interactions or loading data, you have to either use what’s automatically set or do it via JavaScript.
Simple Libraries
Mermaid
Mermaid is a diagram library. It has many integrations, either natively or via a plugin.
pie title Pets adopted by volunteers
"Dogs" : 386
"Cats" : 85
"Rats" : 15
Sequence Diagram:
sequenceDiagram
Alice->>Bob: Says Hello
Note right of Bob: Bob thinks about it
Bob-->>Alice: How are you?
Alice->>Bob: I am good thanks!
Flowchart:
graph TD
st([Start])
op1[My Operation]
sub1[[My Subroutine]]
cond{Yes or No?}
io[/catch something.../]
para[Parallel Tasks]
e([End])
st-->op1
op1-->cond
cond-->|yes|io-->e
cond-->|no|para-->sub1-->op1
I feel it’s worth noting that Mermaid can be a bit rough around the edges sometimes. For example, arrow syntax is different for almost every diagram type and diagrams don’t always follow the max width (altough it looks like there was a PR merged to fix the latter). That said, it’s continually getting better and it’s still a good library regardless.
js-sequence-disgrams
js-sequence-diagrams is, as the name implies, a sequence diagram library. The basic syntax is very similar to Mermaid’s, which makes sense as Mermaid credits it for the grammer. That said, Mermaid has a lot more options albeit not being quite as clean in rendering.
Alice->Bob: Says Hello
Note right of Bob: Bob thinks\nabout it
Bob-->Alice: How are you?
Alice->>Bob: I am good thanks!
It also supports a hand-drawn theme and titles:
Title: Here is a title
A->B: Normal line
B-->C: Dashed line
C->>D: Open arrow
D-->>A: Dashed open arrow
Flowchart.js
Flowchart.js is, as the name implies, a flowchart libraries.
st=>start: Start
e=>end
op1=>operation: My Operation
sub1=>subroutine: My Subroutine
cond=>condition: Yes
or No?
io=>inputoutput: catch something...
para=>parallel: parallel tasks
st->op1->cond
cond(yes)->io->e
cond(no)->para
para(path1, bottom)->sub1(right)->op1
para(path2, top)->op1
While flowchart.js does support color styling, it takes it as a separate parameter so I haven’t added it here. The styling data is JSON though, so it shouldn’t be terrible to do, but would require some finessing of the implementation (I’d rather make it possible to move to Markdown code block render syntax in the future than add extra parameters to a shortcode).
WaveDrom
WaveDrom is a diagram libary for waveforms and circuit/logic diagrams. Unlike some of my friends, I’m not an electrical engineer, so I’m going to just post a couple examples and not even try to pretend I understand them (actually, I do know circuit/logic diagrams a little, but not much).
{ signal: [
{ name: "CK", wave: "P.......", period: 2 },
{ name: "CMD", wave: "x.3x=x4x=x=x=x=x", data: "RAS NOP CAS NOP NOP NOP NOP", phase: 0.5 },
{ name: "ADDR", wave: "x.=x..=x........", data: "ROW COL", phase: 0.5 },
{ name: "DQS", wave: "z.......0.1010z." },
{ name: "DQ", wave: "z.........5555z.", data: "D0 D1 D2 D3" }
]}
{ assign:[
["out",
["|",
["&", ["~", "a"], "b"],
["&", ["~", "b"], "a"]
]
]
]}
nomnoml
nomnoml is a UML diagram library.
I find the basic example amusing so I’m going to include it as-is:
[Pirate|eyeCount: Int|raid();pillage()|
[beard]--[parrot]
[beard]-:>[foul mouth]
]
[<table>mischief | bawl | sing || yell | drink]
[<abstract>Marauder]<:--[Pirate]
[Pirate]- 0..7[mischief]
[jollyness]->[Pirate]
[jollyness]->[rum]
[jollyness]->[singing]
[Pirate]-> *[rum|tastiness: Int|swig()]
[Pirate]->[singing]
[singing]<->[rum]
[<start>st]->[<state>plunder]
[plunder]->[<choice>more loot]
[more loot]->[st]
[more loot] no ->[<end>e]
[<actor>Sailor] - [<usecase>shiver me;timbers]
Other Mentions
Here’s a list of libraries that I tried to implement but appear unmaintained:
- yUML
- dagre-d3 (supports Graphviz DOT language)
- viz.js (Graphviz compiled for JS)
- There seems to be an active fork, but the work to use seems a bit high for simple charting
PlantUML is also worth a mention for declarative charting, but you have to encode your UML and send it to a Java-based server for rendering.
Personally, if the point is to handle it as an embed (such as from a .puml file) then I don’t mind it, but for inline Markdown I’m not too keen on it.
The requirement for another server may also be a deal-breaker for people who want a simple solution and have requirements around where their data is allowed to be sent.
Ending Thoughts
Declarative charts and diagrams are exciting because of how they can extend Markdown without requiring server-side code, complex code, or databases.
If you just want a basic sequence diagram or flowchart, js-sequence-diagram and flowchart.js are really good, simple options. However, I personally think Mermaid and Vega-Lite can cover most use cases and are probably the most practical to work with.
Right now, you have to use shortcodes for these libraries in Hugo; eventually I’d like to see render syntax support for Markdown in general and Hugo support for code block render templates. However, many Markdown editors have implemented support for at least Mermaid, which is exciting.