kjv

In Depth: Autosave

vue work

First, some background

I work on the tooling that our editorial staff use to write and and publish their stories; aka I work on a content management system called Chorus. The UI of Chorus was rebuilt about 8 years ago using a Javascript framework called Ractive (no not React, and no you’ve likely never heard of it). It has your typical 2 way data binding with global events, components, routing, and so forth, a very typical early days reactive JS framework.

The big drawbacks are that no one knows what it is, it’s not being updated, and it also has many of the typical problems of earlier frameworks: it uses webpack, the state system is less than ideal, there are no types, and so on. It was great for what it was, but the time came for The Big Refactor.

Before I joined the team, a plan was put in place to move to Vue.js and the team refactored the story dashboard. That route has the least complexity of any other, since it’s just a list of stories and search, for the most part. It should be noted that almost no one on the team worked on the original product, rather they had worked on our video application before taking over the editor.

Autosave Complexities

The most complex part of the app—and the least understood—is its very excellent autosave and multi-user editing features. While some parts of ractive and vue are similiar, state and global events are one of the areas where they are very different, and both are critical to these features.

The autosave pipeline itself essentially runs in a loop, capturing local changes via operational transforms, sends a packet of changes to the remote server, checks for remote changes from other users, diffs everything together, compares packet sequences, and updates local state. It also saves changes to the user’s local storage in the case of a missed packet or loss of connection that can be merged back in once the user regains their connection.

Not only is this pipeline complex, it is also business critical, given that we are in the business of publishing content.

Into Vue

As you can imagine, I spent a lot of time just reading code, reading old documentation, and drawing diagrams of the system. We wanted to maintain the reliability of the system, as well as the ease-of-use. It was pretty easy to trigger an autosave from anywhere in the system. However, we could not depend on global event triggers, and we wanted the data to flow through the system in the way that Vue would expect.

I proposed putting the entire pipeline into Vuex (we were on Vue 2 at the time). The logic was split up into various pieces: Actions triggered events like local saves and parsing incoming remote updates, mutations updated state, and a vuex plugin handles the pipeline logic—setting changes into queues, sending the updates to the server, composing changes together, and rescue logic.

The entire pipeline is kicked off via mutation—when a document is set as “dirty”, that means a change has been queued and the cycle should begin. The plugin itself subscribes to store mutations to begin the pipeline, as well as an initialization process when the doc first loads in state.

store.subscribe(mutation => {
      switch (mutation.type) {
      case SET_DIRTY:
        sendIfReadyThrottled();
        break;
      case SET_DOC:
        initDoc(mutation.payload);
        break;
      }
    });

I will admit, I don’t love this. It feels very side-effect-y, and I tend towards a more functional approach. I don’t want magic, I want clarity. But at least in Vue 2 land, this felt like the cleanest way forward.

The actual state itself contains all of the document content, so when local or remote updates happen, the UI everywhere is updated immediately using vuex state and getters. For throttled and batched updates, the user can see the saving and saved states.

Any component can dispatch a save —

this.$store.dispatch('localUpdate', { docId, delta });

which makes it very easy to spin up new autosaving components. The delta is based on json0 transforms, which includes a path (where in the document to save the data), and an array of operations to make. These changes can look like “at character number 15, insert hello", or “delete list item number 4”.

Results

This work has been in production for a year and a half without issue, and it unlocked the rest of the Big Rewrite project. The initial implementation of this went live with our updated layout tooling, so it mostly consisted of clicking a few buttons to trigger the save. But since then, we have deployed the compose screen for Quick Posts for the Verge and are well underway for other composition and publishing screens.

While we are now on Vue 3, we have not made any major moves as far as using the composition API or using Pinia. At the moment, I don’t see any major advantage to moving in that direction besides Typescript, although honestly VS code does a good job of giving hints even without TS.