A place for musings, observations, design notes, quick tips...
thought gists, if you like.
Surely you're `jq`ing
Today I read through the jq
manual cover-to-cover. For those unaware, jq
is a popular CLI tool to query and manipulate JSON. It’s also a Turing-complete mini-language with nice functional semantics that fits well into the ethos of composable CLI tools.
It was an exemplar of well-written technical documentation. Concise, well-written, littered with examples, and linking to an interactive playground to test-and-learn.
Some learnings:
- It’s surprisingly functional! You can implement recursive functions and use higher-order functions! For example, here’s factorial in
jq
:
jq '[.,1]|until(.[0] < 1; [.[0] - 1, .[1] * .[0]])|.[1]'
- It supports string interpolation - this is really nice if you’re piping stuff from JSON into a string. Coupled with format strings this becomes frictionless:
$ echo '{"search":"hello; world"}' | jq -r '@uri "https://www.google.com/search?q=\(.search)"'
# https://www.google.com/search?q=hello%3B%20world
- You can define functions that accept functions1, and control structures that allow labelling.
echo '[[1,2],[10,20]]' | jq -r 'def addvalue(f): . + [f]; map(addvalue(.[0]))'
[[1,2,1], [10,20,10]]
- You can traverse complex data structures with first-class pathing support. And you can easily modify nested structures to extend objects.
- For the category theorists/polyglots, there’s a denotational semantics paper written about
jq
. - Bonus: You can build a Brainfuck interpreter in
jq
; and you can build ajq
interpreter injq
- how’s that for bootstrapping!
Side note: This is one of my goals for 2025 - read through documentation end-to-end to develop mastery over tools. I’m trying to prioritise selectively depth over breadth.
-
The line between “functions” and “filters” is a little blurry to me. ↩︎
Shellshocked? Brace yourselves!
I just discovered that to capture multiple lines of stdout
from a shell script and redirect them to a file, you can simply wrap them in braces!
For example, my “Create a blog post via a GitHub Action triggered on an Issue creation” workflow uses this snippet:
{
echo "---"
jq 'del(.content)' "parsed_issue.json" | yq -P
echo "---"
echo ''
# Inline "content" key for the body
jq -r '.content' "parsed_issue.json"
} > content/micro-blog/"$FILENAME"
Typography in Parasite
This is awesome! Typographic Hangul from the title scene of Parasite
Developer Ergonomics
“I wonder how much it is insightful to watch someone doing a workflow and to note when discomfort kicks in. That’s a really insightful thing to realize what matters from bitter experience, right? … Experience tells you when to worry about something and when not to worry about it” - Ben Sparks
That is - the rising discomfort of a programmer when employing a new tool, framework, or library is a good window into the ergonomics of how one uses your tool, framework, or library. Source: How I animate 3Blue1Brown | A Manim demo with Ben Sparks. The whole video is worth checking out! It’s a masterclass on how to construct a programatic-animation library and demonstrate how to work within it.
When I get a chance, I’d love to unpack the source code for how sprite-to-sprite transforms (part of 3B1B’s signature look) and the vectorised rendering engine are implemented with OpenGL.
Audiolising SwiftUI Graphs
This demonstration of an auditory representation of a (visual) graph in Swift Charts is wild: Vectorized and function plots in Swift Charts at WWDC24.
Week 22
- I noted that there’s a difference between Anomaly detection and Outlier detection.
- Word of the week ~ enjambement
enjambement: (in verse) the continuation of a sentence without a pause beyond the end of a line, couplet, or stanza.
- As an aside, I find Rust as a language fiendishly complex at a first glance, and I’m amazed by people who find it so straightforward. Having said that, I’d much rather have lived in a universe where Rust was the first ’low-level’ language I learnt instead of C.
Week 21
- Discovered the concept of similarity graphs through a talk on Uniform Manifold Approximation and Projection (UMAP) for Dimension Reduction at SciPy 2018.
- Word of the week ~ grimoire, from French, an alteration of “grammaire” (grammar)
grimoire: a book of magic spells and invocations.
- Also, in the world of books: The Cossacks by Leo Tolstoy.