All writing

A Draft Is a Graph of Identifiers

Designing cloud drafts for a video editor: checkpoints, content-addressed assets, and a sync model that behaves like version control

The Ask: A Draft That Follows You

The requirement arrived as one sentence: a user edits a video on their phone, and later opens the same draft on another device and keeps editing.

It is worth separating this from a feature it superficially resembles. Cloud backup only has to preserve bytes; if the same build of the same app restores the same file, the job is done. A cloud draft has to preserve meaning. The draft will be opened by a different device, possibly a different platform, possibly a different version of the editor, and the user expects their timeline, not a file.

The first design decision, the one everything else hangs on, is refusing to treat the draft as a file at all. A draft is two very different things wearing one name:

  • a project document: tracks, clips, in and out points, keyframes, effect parameters, text. Structured data, a few hundred kilobytes;
  • the assets: the source videos, music, and images the document points at. Gigabytes.

The naive design zips both into one archive and uploads it. It fails on first contact: edit one caption and you re-upload two gigabytes, nothing deduplicates, nothing resumes meaningfully, and the other device cannot open anything until everything has arrived. The correct shape is the one below, and the rest of this article is the consequences of choosing it.

Anatomy of a cloud draft: a small draft document holds the timeline and references assets by content hash; assets live once in a content-addressed store, each with original, proxy, and thumbnail representations

An earlier post on this blog told the story of passing a single asset identifier across an app boundary instead of copying the file. A cloud draft is that idea under load: the thing being synchronized is not one identifier but a versioned graph of them.

The Local Half: Editing Is a Write-Ahead Log

Before anything touches the network, the draft has to survive the device itself, and the enemy on the device is the crash.

The rule that makes crash handling tractable sounds backwards at first: do nothing at crash time. A crash handler runs with a possibly corrupted heap, where only trivially safe operations are permitted; serializing a project there risks overwriting the last good file with a broken one. Worse, the two most common ways a video editor dies produce no callback at all. Out-of-memory kills arrive with no signal, and a decoder pipeline chewing 4K frames makes them routine; watchdog terminations for a blocked main thread are equally silent. Any design that depends on saving during the crash misses exactly the crashes that matter.

So the protection lives before and after the crash instead:

  • A journal. Every edit operation appends one small record to an append-only log, each entry checksummed. Tens of bytes per gesture, sequential writes, cheap enough to run on every operation. A crash loses at most the final gesture, not the final five minutes.
  • Rolling snapshots. Every so often the full document is serialized, written to a temporary file, and atomically renamed over a generation slot. At any instant the disk holds one complete, readable, last-good document, plus the journal entries newer than it. A memory warning is treated as a snapshot trigger, because the step after a memory warning is often the silent kill.
  • A dirty sentinel. A marker file is created when an editing session opens and removed on clean exit. It is written once, in advance, precisely so that the crash itself has nothing left to do.

The visible machinery sits at the edges of this. In the app delegate, entering the background commits the session, wrapped in a background task because the system grants only seconds. And on the next launch, a surviving sentinel means the last session did not end cleanly, so the app rebuilds the state (snapshot plus journal replay, truncating any entry that fails its checksum) and asks the user whether to restore it.

That restore dialog carries one more defense that is easy to forget: a counter. If the crash was caused by the draft data, automatically restoring it re-crashes the app in a loop. Restore attempts are counted and the count is cleared only on success; after a few failures the app stops offering automatic restore, quarantines the data, and lets the user in through a safe path.

Out of all this falls the concept the whole cloud design stands on: the checkpoint. A checkpoint is a complete, consistent, user-acknowledged version of the draft: produced by a clean background commit, an explicit save, or a restore the user confirmed. Everything else, the journal, the rolling snapshots, the quarantined recovery data, is private working state of one device.

Checkpoints, Not Streams

The sync model follows from one product decision we made early and defended often: this feature is continuity across devices, not simultaneous editing. Real-time collaborative editing of a video timeline is a different product with a different engine; two edits to the same clip's trim points do not merge the way concurrent text edits do, and pretending otherwise produces merges that are technically consistent and visually wrong.

Declining that problem bought a drastically simpler contract: only checkpoints travel.

The checkpoint pipeline: an editing session journals every operation on device; a background commit, explicit save, or confirmed restore produces a checkpoint, which is quota-prechecked, has its missing blobs uploaded, and becomes visible when the version pointer flips; forks produce conflict copies and over-quota saves enter a pending state

Because every uploaded version is a complete checkpoint, conflict handling collapses into two rules:

  1. Every upload names the cloud revision it was based on.
  2. If the server sees that the draft has moved past that base revision, the upload does not overwrite anything. It becomes a conflict copy, a sibling draft labeled with its origin, and the user chooses.

Conflicts are rare in this model, because checkpoints are coarse: they happen when a session ends, not on every keystroke. Editing on the phone in the morning and on the tablet at night never conflicts. Leaving the draft open on both and exiting both does, and then the system forks instead of guessing. The one thing the sync layer must never do is silently discard an hour of cutting.

One corollary deserves its own sentence: recovery data that the user has not confirmed never uploads. A crash produces an offer to restore, not a new cloud version. Half-finished state stays on the device that made it, which is what keeps a bad session on one device from contaminating every other.

The Asset Layer: One Store, Many Drafts

The document references assets by content hash, and that single choice does most of the asset layer's work.

Deduplication is free. The same source clip used in five drafts hashes to the same identifier and is stored once. Uploading a checkpoint means diffing its manifest against what the store already holds and sending only the missing blobs, resumably; a re-used asset costs nothing.

Local identifiers are translated at the edge. On device, an asset usually arrives as a PHAsset local identifier, and that name is device-local; it means nothing on the user's other phone. Import is therefore also a naming ceremony: bytes are hashed, and from then on the draft speaks only in content addresses. The failure case where the user deleted the original from Photos before the asset was ever uploaded is detected here, at the boundary, and surfaced as a missing-asset placeholder rather than discovered as a mystery on another device.

Every asset is several representations. Opening a draft on a new device cannot wait for gigabytes of originals. Each blob carries a family: the original for export, an editing proxy at a fraction of the bitrate, a thumbnail for browsing. The other device opens the draft against proxies within seconds and fetches originals only when the user actually exports. The proxy tier is simultaneously the performance story, the cellular story, and the storage-cost story.

Visibility is atomic. A checkpoint's document must never be visible while its assets are still uploading, or another device opens a draft full of dangling references. The order is fixed: blobs first, then the version pointer flips. The pointer flip is the commit.

Deletion is garbage collection. With cross-draft dedup, no draft owns an asset; deleting a draft decrements references, and blobs are collected later, with a grace window because version history and restore flows may still reach them.

Schema Migration Happens Before Upload

The checkpoint model assumes that any device can open any checkpoint. That assumption is false the moment two app versions coexist, which is to say always. The document carries a schema_version, every release that adds a timeline feature bumps it, and the fleet at any moment spans half a dozen versions across two platforms. The question "can this device open this draft?" has to be answered by a rule, not by hoping.

The rule we landed on treats the schema like a ladder with three bands, and the bands are decided by two numbers each client knows about itself and one number the cloud publishes.

The schema ladder: a document below the cloud's minimum must be migrated forward on device before upload; a document within the client's readable range opens, in compatibility mode if it is newer than what the client writes; a document above the client's readable ceiling is refused with an update promptMigrations are pure functions from one schema to the next and only ever run forward. The cloud never migrates; it stores whatever schema it was given and enforces a floor.

Each client knows what it writes and what it can read. writeSchema is the version this build produces. readCeiling is the newest version it can open, which is higher than writeSchema because of the tolerance rule from the end of this article: unknown nodes are preserved opaquely, so a client can usually read a document one or two features ahead of itself. Above the ceiling, tolerance breaks down, typically because the track model itself changed, and the only honest response is to refuse and ask for an update.

The cloud publishes a floor. minimumCloudSchema is the oldest version that every supported client can read. It moves up slowly, on the cadence of dropping old app versions, never on the cadence of releases.

Those numbers produce three cases, and the first one is the one that surprised the team:

  1. Below the floor: migrate forward, on device, before upload. A draft created two years ago on a version-3 document cannot be uploaded as version 3, because the newest tablet client has not been able to read version 3 for a year. The client runs the migration chain, 3 → 4 → 5 → … → writeSchema, on its own copy and uploads the result. Each step is a pure function of the document tree with a fixture pair in the test suite, and the chain only ever goes forward. The cloud never migrates anything. It is a blob store with a version pointer, and keeping it that dumb is what makes every other property of the design hold.
  2. Within the readable range: open. If the document is at or below writeSchema, the client migrates it up on open, same chain, and edits normally. If it is above writeSchema but at or below readCeiling, the client opens it in compatibility mode: unknown nodes are carried untouched, rendered as placeholders, and written back exactly as they arrived. The one thing compatibility mode must never do is lower the schema_version on save, because that would tell the next device the opaque nodes are gone when they are not.
  3. Above the ceiling: refuse. Show the update prompt. Do not open read-only, do not render what can be rendered; a user who trims a clip in a draft whose structure the client cannot fully see will eventually save, and that save is corruption with a progress bar.

Why migrate on the client rather than on the server, where it would run once instead of on every device? Because migrations are not purely structural. Bumping from one schema to the next frequently means resolving a renamed effect against the current effect catalog, filling in a default that the old feature never had, or re-expressing a keyframe curve in a new parameterization. The client is the only process that has the catalog, can render the result, and can tell the user what changed. A server-side migration would be guessing at all three.

There is one more wrinkle, and it is a product decision disguised as an engineering one. Migrating a local draft forward is irreversible for that draft: once it is at version 8, the older app version still installed on the same device cannot open it. So a draft below the floor is not migrated silently. The draft list shows it as needs upgrade to sync, the user taps, the migration runs, and only then does the checkpoint upload. A user who never taps keeps a local-only draft, visibly marked as such, and nothing else in the system changes. The same visible-state principle from the quota section applies: a draft that is not in the cloud must never look like it is.

The Transport Underneath

Everything above is about what moves between devices. Moving it reliably on a phone is its own discipline, and most of the rules are consequences of one fact: the app will be suspended, killed, or offline at some point during any upload large enough to matter.

Background sessions, not foreground tasks. Blob uploads run through a background URLSession. Those sessions are file-based by design, which is why the checkpoint pipeline stages every blob to a file before enqueueing: the system can continue the transfer after the app is suspended and relaunch the app to deliver the result. The delegate reattaches on launch through the background-session completion path, and the sync state machine is rebuilt from the session's outstanding tasks rather than from memory, because memory is exactly what was lost.

Chunking, resumption, and idempotency come from the content hash. Large originals go through the company's shared media transfer layer, which splits a file into parts, uploads each part independently with its own retry, and resumes by asking the server which parts it already holds. The part of that design that belongs to the draft system is the key: every blob is identified by its content hash, so a retry after a lost acknowledgment uploads nothing the server did not already accept, and two drafts referencing the same clip can never race to upload it twice.

Retry is classified, not blind. Network-layer failures and timeouts retry with exponential backoff and jitter. Permanent failures (an expired upload token, a quota rejection, a 4xx the server means) do not retry; they move the checkpoint to pending with a reason, and the reason is what the UI shows. Treating every failure as transient is how an upload loops silently for a day.

Downloads start with the smallest representation. Opening a draft on a new device fetches the document, then the proxies, then nothing else until export. Proxies and thumbnails are served from CDN edge nodes, and the document and the blob manifest are the only requests that must reach the origin. This is the second half of the proxy-tier argument from the asset section: the tier was designed for playback, and it turned out to be the entire cold-open performance story as well.

One network stack, so regional rules apply once. Every request, origin or edge, upload or download, goes through the app's single networking layer. That matters beyond tidiness: the base hostnames per region live in one configuration sheet, and the data-residency constraint described in an earlier post about regional isolation is enforced there, not in the draft code. The sync layer never constructs a URL from a string.

Instrumentation is part of the feature. Each checkpoint reports the funnel it moved through: committed, prechecked, blobs uploaded, pointer flipped, visible on the other device. Success rate per stage and time-to-synced are the two numbers that turned out to predict support tickets, and both regressed in ways that no crash metric would have caught: a CDN configuration change that slowed proxy downloads showed up as a jump in time-to-synced two days before the first user wrote in.

Quota Turns Billing into Failure Semantics

Cloud drafts shipped behind an entitlement: a small free tier, a paid tier above it. That is a product decision, but it lands in the sync layer as a set of failure states that have to be designed, not discovered.

The upload pipeline prechecks the entire checkpoint delta against the remaining quota before transferring anything. The alternative, discovering the limit halfway through the blob list, strands orphaned assets in the store: they consume the user's quota and no visible draft explains why. All or nothing, decided up front.

When the check fails, the checkpoint enters a pending state: safely committed locally, visibly not synced, retried automatically after the user upgrades or frees space. The one unacceptable outcome is the user believing something is in the cloud when it is not, so the sync state machine (local, uploading, synced, pending, conflicted) is user-visible per draft rather than an internal flag.

Dedup complicates the accounting in a way worth deciding early: when one blob backs two drafts, whose quota holds it? Charging it once makes "I deleted a draft and got no space back" a support ticket; charging it per reference gives the savings of dedup to the operator instead of the user. Either answer works. Not choosing one is the only wrong option, because the refcounting design depends on it.

What I Would Underline

Looking back at the whole system, the parts that consumed design effort were not the parts that looked hard at kickoff. Upload and download were the easy 80 percent. The three things I would tell a past team to over-invest in:

  1. Schema evolution across platforms. The document outlives every client version that writes it. A client that opens a draft containing nodes it does not understand must preserve them untouched, render placeholders, and save without stripping them; the alternative is the oldest client in the fleet quietly deleting every newer feature's work. The ladder of floor, write version, and read ceiling is what turns that tolerance from a convention into a rule, and the forward-only migration chain is what keeps the cloud out of the business of understanding documents. This is the versioned-protocol discipline from our bridge and share-SDK projects, hardened for data at rest.
  2. Render consistency. Cross-device continuity is judged by eyes, not checksums. Fonts, color pipelines, effect implementations, and keyframe interpolation must agree across platforms, or the same document is a different video on each device, and the user calls that corruption.
  3. The checkpoint boundary. Every simplification in this design traces back to one invariant: nothing incomplete ever crosses a boundary, whether that boundary is a crash, a device, or a billing tier.

Takeaway

Strip the domain words away and the shape underneath is familiar. The draft document is a commit. The assets are content-addressed blobs. Uploading is push, opening on a new device is pull, the base revision check is a compare-and-swap, and the conflict copy is a branch the user merges by choosing. We did not set out to build a small version control system with a media-sized object store, but that is what a cloud draft is, and every place our design was clean is a place it agreed with that shape.

That earlier post ended with passing one identifier instead of one file. This project taught the sequel: when the identifiers multiply into a graph that changes over time, a handoff is no longer enough. You need history, atomic visibility, and a rule for divergence. Which is to say: once your references outnumber your copies, you are not designing a transfer anymore. You are designing version control.