When you first learned to program, you might have started out with something super simple. The most popular, or at least stereotypical “first program,” depending on the language you used, probably looked something like console.log('Hello World'); , print('Hello World'), or maybe System.out.println("Hello world"). If you wrote it in JavaScript, you might have run it directly from the browser console.
Next, you likely graduated to writing simple programs. You might have learned about some data structures, common algorithms, and, in general, how to convert system requirements into code.
Continuing on your journey, your next step may have been creating and publishing a library or package. You’ve been programming for a while and maybe noticed a gap in existing packages where you frequently find yourself repeating code. This is a great place to be, as you’re skilled enough to not only write code but also to begin to move beyond just translating requirements into code to a level where you’re starting to build ecosystems.
The next layer down goes beyond code abstraction into framework building. Rather than abstracting away code to prevent repeating boilerplate, your goal is now to make things “just work” for others. The developer using what you’re writing doesn’t have to know how it works; they just need to follow the instructions you give them. Frameworks work “automagically.” At this level, you truly enter the territory of one hour of work saving others seven years.
But how does this actually work?
That’s a question I’ve asked myself since learning to program. A lot of frameworks are so far removed from the normal vanilla language that it is easy to almost forget that you’re writing the underlying language rather than the framework.
In this post, I want to explore the topic of automagical programming, and by that, I simply mean programming techniques that feel like magic because of all of the behind-the-scenes work being done. I’m only covering a few techniques, but even knowing these has helped me become a better developer.
Build-time Transformations
How does JSX work? This clearly isn’t HTML, but it also isn’t JavaScript.
function Welcome({ name }) {
return <div className="greeting">Hello, {name}!</div>;
}Your build tool, Vite, Webpack, etc. acts almost like a compiler. It bundles your code, transforming it from JSX into JavaScript. The output of the above might look something like this:
function Welcome({ name }) {
return React.createElement('div', { className: 'greeting' }, 'Hello, ', name, '!');
}At build time, the higher-level syntax is translated into a lower-level syntax that can be understood by the browser. This is super powerful because it lets developers write code in a way that feels more natural. JSX feels like a superpowered cross between HTML and JavaScript almost similar to PHP. Browsers have no clue what JSX is but after JSX in bundled at build time into JavaScript is automagically works.
Another example is file-based routing in frameworks like Next.js. With file-based routing, you can simply create files like:
./about.tsx./blog/index.tsx./blog/[slug].tsx
Gives us:
example.com/aboutexample.com/blogexample.com/blog/todays-post
You don’t even have to think about routing. It just works.
These types of features work by basically scanning and parsing your project, then generating actual code based on the simplified syntaxes you used. I started a project a few months back to learn how to build a static site generator; it works in a similar way. Anything with a build step is able to do super powerful transformations to your code without adding additional code to the build artifacts, and in turn, reducing the amount of code shipped to the client.
The key distinction here is that, rather than relying on clever tricks and simply abstracting away code (as seen in the other examples in this post), build-time transformations enable intelligent hyper-optimization. You can literally hard-code in logic and data because it takes place after the developer’s code, but before being put on the server.
This is maybe the most automagical programming method because it is so far hidden from how the code is written. This unlocks many capabilities but also shifts some responsibility. Since your framework code is making decisions on behalf of the developer, you need to follow best practices and clearly communicate what exactly it does so that it isn’t misused.
Runtime Abstractions
This is in the world of libraries, but often used by frameworks to create an integrated experience. A runtime abstraction is code that appears to do one thing, but behind the scenes is doing something more complex. Examples include database ORMs, state management features, and event systems.
One of the best examples of this is React’s virtual DOM. If you’re like me, you didn’t start off writing React. The first websites I created using JavaScript manually interacted with the DOM, or Document Object Model. If you’re not familiar, that’s a big phrase, but with a simple-to-understand purpose. JavaScript is a language that makes use of a concept called object-oriented programming. Objects are a high-level data structure that allows you to access methods and attributes/properties. HTML, or HyperText Markup Language, which is the language used to define the structure and content of documents, isn’t easily represented as an object. So, JavaScript abstracts access to the document through an interface called the Document Object Model.
React has an issue, though. Modifying the DOM directly isn’t efficient for the number of changes that a React app needs to make. The solution to this is to create a virtual DOM that acts as a staging area for changes. A side effect of this approach is that optimizations can be done to improve the changes made to the real DOM before wasting energy and time to do them. The full inner workings of the virtual DOM is a big topic, so I won’t go into too much detail.
As users of React, we might write something like this setCount(count + 1); which in normal JavaScript world would look more like count += 1;. There’s clearly something more going on. This is what is going on behind the scenes:
- React creates a new virtual DOM tree (just JavaScript objects)
- Compares it to the previous virtual DOM tree
- Calculates the minimal set of actual DOM changes needed
- Applies only those specific changes to the real DOM
You’re not manipulating the DOM directly; instead, you’re manipulating a JavaScript representation of it, which then optimizes and batches the real DOM updates.
Implementing this from scratch in every project would be crazy. Instead, experts have created a highly optimized magical system that just works.
Metaprogramming
This is one of the most interesting concepts because it breaks your thinking. A language doesn’t have to do what it is supposed to, it can be bent to your will. Metaprogramming allows you to use code in code.
One example that I’ve been working with recently is enhancing the features of the Cloudflare Zaraz Managed Component framework. This framework lets you route fetch requests either to be run in the Worker or in the client’s browser. This is pretty useful for offloading requests from the browser. One problem is that the server-side fetch in the framework waits until after the program completes to send off the requests. This is okay if you’re looking to maximize performance, but a deal breaker if you need to process the response. Metaprogramming using Proxy allows me to intelligently either process the fetch and return the response or lazily wait until later.
Because Metaprogramming lets you reprogram the language, you can do some super-powerful stuff. In JavaScript, one of the most popular features is Proxy. Proxy allows you to intercept and redefine how objects work. Here’s an example from the MDN web docs:
const target = {
message1: "hello",
message2: "everyone",
};
const handler3 = {
get(target, prop, receiver) {
if (prop === "message2") {
return "world";
}
return Reflect.get(...arguments);
},
};
const proxy3 = new Proxy(target, handler3);
console.log(proxy3.message1); // hello
console.log(proxy3.message2); // worldSee how that worked? It works for other things, too. Here we are intercepting a fetch request and logging the parameters. Notice how when we call fetch, we’re not calling proxiedFetch or something like that. We’re calling actual fetch, then using Reflect to pass through the default behavior after we log the arguments.
(async () => {
const fetchHandler = {
apply(target, thisArg, args) {
console.log('Fetch called with:', args);
return Reflect.apply(target, thisArg, args);
}
};
globalThis.fetch = new Proxy(fetch, fetchHandler);
const response = await fetch('https://mackenly.com');
console.log('Status:', response.status);
})();Results in:
Fetch called with: ['https://mackenly.com']
Promise {<pending>}
VM972:12 Status: 200That’s pretty awesome. You just rewrote a language feature to do what you wanted. If you’re new to this concept, you’re probably now thinking, “Oh, so that’s how logging tools work without me having to manually log everything!”
Speaking of logging tools. Why would you want to use a custom logging function when we can just hook into console.log? It is pretty easy to do the exact same thing for console methods.
(() => {
const originalConsole = { ...console };
globalThis.console = new Proxy(console, {
get(target, prop, receiver) {
const original = Reflect.get(target, prop, receiver);
if (typeof original === 'function') {
return new Proxy(original, {
apply(targetFn, thisArg, args) {
// you can instead send to your logging system here!
originalConsole.log('[proxy:log]', ...args);
return Reflect.apply(targetFn, thisArg, args);
}
});
}
return original;
}
});
console.log("This should have a proxy prefix");
})();So in the past 60 seconds or so, we just built the core of our own log monitoring system that doesn’t require using special functions or manual setup. Super easy.
I don’t have time to get into any more Metaprogramming features in this post (let me know if you’re interested in that!), but if you’re looking, check out the docs for these for some things I found interesting:
- Proxy
- Reflect
- Symbol
- Function.name
- eval - This one can be dangerous
- Tag Functions
- Object.freeze
Summary
I hope you found this post interesting and perhaps learned something new. Exploring language features and de-black boxing various tools and technologies is something I really enjoy. Although it sometimes takes away the “automagical” feeling, it helps developers like us to be better when we’re operating on a level deeper than just following the docs, and where we know our tools well enough to color outside the lines when needed. If you enjoyed this post, definitely sign up to get new posts delivered by email. I never spam. If anything, I don’t write enough. By signing up, you get the full post directly in your inbox, and let me know that you like this type of content.
Liked this? Get the next one.
No algorithms or news feeds. Add the RSS feed to your reader and new posts show up as soon as they're live.