PersonalDevelopment

Abusing virtual modules for a blog

Mats
Mats
The king
1/27/20258 minutes read
Abusing virtual modules for a blog

Two things have always been true about me:

  • I love sharing things I work on
  • I am incredibly lazy and drop anything that has the slightest bit of friction

And sadly, these two things are not compatible. So I decided to build a blog. But not just any blog, a blog that is so overengineered that there's a really fair chance that I spent more time building it than I will ever spend writing on it.

In a perfect world, I would just have a folder where I could quack out some markdown files and have them automagically appear on the internet. And there are many tools that do just that, leagues better than what I could ever build. But that's not the point. The point is to build something myself, and to learn from it. The last site I built with this idea (the OpenAudioMc docs) required me to manually register markdown files in a JSON file, provide metadata, and then painfully wait for Next.js to rebuild the site. It was a pain, but it worked. And it was fun to build.

Naturally, I asked myself: "How can I make this even more painful?". The answer was simple: "Let's build a vite plugin that does this". Why should vite have anything to do with this? Because fuck it, that's why.

Drafting

I started building the site normally, quick router, sprinkle in some ReactMarkdown, hardcode a text blob, and, then, well shit, what now?

Well, markdown "supports" comments, not every parser respects them, but you can abuse this mechanism to embed metadata in a markdown file itself! A quick

---
title: Overbuilding a blog
date: '2024-01-27'
excerpt: Probably the worst way to build a blog
tags:
  - Personal
coverImage: /blog/articles/welcome/scheveningen.png
author:
  name: Mats
  avatar: https://github.com/Mindgamesnl.png
  role: The king
---

then, pull it through Gray Matter (or any other yaml parser), and congratulations! You are now the proud owner of some goofy ahh metadata system!

Indexing

Like I said, I didn't want to write yucky manual indexes for my files, but I also didn't want to setup a full CMS or crud (I want this to be a dumb static site, no fancy serverless nonsense).

Vite already bundles my garbage JS code, so why not also abuse it to bundle my markdown? And with that, I don't mean bundling it as huge fuckass strings/blobs, but building a custom plugin that reads markdown files, parses it's embedded metadata, listens for changes, and then makes a virtual module to export all this data. So I did just that.

What are Virtual Modules?

Virtual Modules are a concept by the Module government, to sell you more modules without needing to write out a file. You can dynamically generate them in a plugin, and then access them through a special namespace/prefix in your code.

We can implement the resolve method in our vite plugin, to tell it that an import statement should actually be using our virtual module instead!

resolveId(id) {
    if (id === 'virtual:blog-data') {
        return '\0virtual:blog-data';
    }
}

Vite will then later call the load method, where we can return the actual content of our virtual module.

load(id) {
    if (id === '\0virtual:blog-data') {
        return `export const exampleData = ${JSON.stringify({ someObject: 'thingy' })};`;
    }
}

Implementing a virtual blog module

Now that we know that we can dynamically generate bundles, we can make one that:

  • Lists all my markdown files
  • Parses its content and metadata yaml blob
  • is pretty slow but I don't care yet
const processArticle = (filePath) => {
  const source = fs.readFileSync(filePath, 'utf-8');
  const { data: frontmatter, content } = matter(source);
  const slug = path.basename(filePath, '.md');

  return {
    slug,
    content,
    ...frontmatter,
    title: frontmatter.title || 'Untitled',
    date: frontmatter.date || new Date().toISOString(),
    excerpt: frontmatter.excerpt || '',
    readTime: calculateReadTime(content),
    author: {
      name: frontmatter.author?.name || 'Mats',
      avatar: frontmatter.author?.avatar || 'https://github.com/Mindgamesnl.png',
      role: frontmatter.author?.role || '',
      bio: frontmatter.author?.bio || ''
    },
    tags: frontmatter.tags || [],
    coverImage: frontmatter.coverImage || '/blog/articles/cover.png'
  };
};

const loadAllArticles = () => {
  const blogDir = path.resolve('src/content/blog/articles');
  const files = fs.readdirSync(blogDir)
    .filter(file => file.endsWith('.md'));

  const articles = files.map(file => {
    const filePath = path.join(blogDir, file);
    return processArticle(filePath);
  });

  articles.sort((a, b) => new Date(b.date) - new Date(a.date));
  return articles;
};

which we can then use in our module, where we'll embed it as a json object (passed as string, because vite doesn't like objects). You could also provide function implementations here, and import them like you would any other function from any other module, but this already feels incredibly dirty (it is), and even I have some standards.

So, with that, our load method will look like this:

load(id) {
  if (id === '\0virtual:blog-data') {
    const articles = loadAllArticles();

    return `
      const articles = ${JSON.stringify(articles, null, 2)};
      export { articles };
    `;
  }
}

Consuming the data

Now that we have our data, we can import it in our React components, and use it to render our blog posts. I wrote a quick little hook that fetches the data from our virtual module, and returns it as a normal object.

import { useMemo } from 'react';
import { articles } from 'virtual:blog-data';

export const useBlogArticles = () => {
  return articles;
};

export const useBlogArticle = (slug) => {
  return useMemo(() => {
    if (!slug) return undefined;
    const article = articles.find(article => article.slug === slug);
    return article || null;
  }, [slug]);
};

And that's it! this will work just like any other hook. But I don't like it. This is forgetting my main goal: to let myself be as lazy as possible.

I want hot reloading, I want to be able to write a markdown file, and see it appear on my site without needing to refresh the page. And I want to do this without needing to write a single line of code or god forbid, tab out of my editor.

Self-reloading

Making the plugin listen for changes is the easy part, we add markdown files, sprinkle in a listener, and we're gaming

const blogDir = path.resolve('src/');

server.watcher.add(path.resolve(blogDir, '**/*.md'));

server.watcher.on('change', (file) => {
  if (file.endsWith('.md')) {
    console.log(`[blog] File changed: ${file}`);
    const article = processArticle(file);
  }
});

but how do we then get this to the client? We don't have to! Our magnificent vite overlords got us covered here too! Vite already runs a websocket server under the hood which is attached and available to us in our plugin and hook, all we need to do is send a custom event, catch it in the hook, and we're absolutely golden!

server.watcher.on('change', (file) => {
  if (file.endsWith('.md')) {
    console.log(`[blog] File changed: ${file}`);
    const article = processArticle(file);
    server.ws.send({
      type: 'custom',
      event: 'blog-article-update',
      data: {
        slug: article.slug,
        event: 'blog-article-update',
        article
      }
    });
  }
});

End result

And that's it! I can now build my blog by writing markdown files, and have them automagically appear on my site, even while running a development instance with hot reloading. All it took was this (final) vite blog plugin:

import fs from 'fs';
import path from 'path';
import matter from 'gray-matter';

const calculateReadTime = (content) => {
  const wordsPerMinute = 200;
  const words = content.trim().split(/\s+/).length;
  const minutes = Math.ceil(words / wordsPerMinute);
  return minutes === 1 ? '1 minute read' : minutes < 1 ? 'Quick read' : `${minutes} minutes read`;
};

export default function blogPlugin() {
  const virtualModuleId = 'virtual:blog-data';
  const resolvedVirtualModuleId = '\0' + virtualModuleId;

  const processArticle = (filePath) => {
    const source = fs.readFileSync(filePath, 'utf-8');
    const { data: frontmatter, content } = matter(source);
    const slug = path.basename(filePath, '.md');

    return {
      slug,
      content,
      ...frontmatter,
      title: frontmatter.title || 'Untitled',
      date: frontmatter.date || new Date().toISOString(),
      excerpt: frontmatter.excerpt || '',
      readTime: calculateReadTime(content),
      author: {
        name: frontmatter.author?.name || 'Mats',
        avatar: frontmatter.author?.avatar || 'https://github.com/Mindgamesnl.png',
        role: frontmatter.author?.role || '',
        bio: frontmatter.author?.bio || ''
      },
      tags: frontmatter.tags || [],
      coverImage: frontmatter.coverImage || '/blog/articles/cover.png'
    };
  };

  const loadAllArticles = () => {
    const blogDir = path.resolve('src/content/blog/articles');
    const files = fs.readdirSync(blogDir)
      .filter(file => file.endsWith('.md'));

    const articles = files.map(file => {
      const filePath = path.join(blogDir, file);
      return processArticle(filePath);
    });

    articles.sort((a, b) => new Date(b.date) - new Date(a.date));
    return articles;
  };

  return {
    name: 'vite-plugin-blog-data',
    enforce: 'pre',

    configureServer(server) {
      const blogDir = path.resolve('src/');
      server.watcher.add(path.resolve(blogDir, '**/*.md'));

      server.watcher.on('change', (file) => {
        if (file.endsWith('.md')) {
          console.log(`[blog] File changed: ${file}`);
          const article = processArticle(file);
          server.ws.send({
            type: 'custom',
            event: 'blog-article-update',
            data: {
              slug: article.slug,
              event: 'blog-article-update',
              article
            }
          });
        }
      });
    },

    resolveId(id) {
      if (id === virtualModuleId) {
        return resolvedVirtualModuleId;
      }
    },

    load(id) {
      if (id === resolvedVirtualModuleId) {
        const articles = loadAllArticles();

        return `
          const articles = ${JSON.stringify(articles, null, 2)};
          export { articles };
        `;
      }
    }
  };
}

and this hook:

import { useState, useEffect, useMemo } from 'react';
import { articles as initialArticles } from 'virtual:blog-data';

// Shared state between hooks
let currentArticles = [...initialArticles];

export function useBlogArticles() {
  const [articles, setArticles] = useState(currentArticles);

  useEffect(() => {
    // Only set up hot reloading in development
    if (!import.meta.hot) return;

    const handleUpdate = (data) => {
      if (data.event === 'blog-article-update') {
        const updatedArticle = data.article;
        const index = currentArticles.findIndex(
          article => article.slug === updatedArticle.slug
        );
        
        if (index !== -1) {
          currentArticles[index] = updatedArticle;
        } else {
          currentArticles.push(updatedArticle);
        }

        // Update component state
        setArticles([...currentArticles]);
      }
    };

    import.meta.hot.on('blog-article-update', handleUpdate);
    return () => import.meta.hot.off('blog-article-update', handleUpdate);
  }, []);

  return articles;
}

export function useBlogArticle(slug) {
  const articles = useBlogArticles();

  return useMemo(() => {
    if (!slug) return undefined;
    return articles.find(article => article.slug === slug) || null;
  }, [slug, articles]);
}

if (import.meta.env.DEV) {
  window.__blogArticles = currentArticles;
}