Skip to content

Sync and the firehose

The com.atproto.sync methods: getRepo, getLatestCommit and subscribeRepos, the firehose. What every method shares, and how the server is checked, is in The PDS: accounts and their records; the repository they serve is as The repository's records says.

What the reference does that the specs leave open

Section titled “What the reference does that the specs leave open”
  • getRepo and getLatestCommit of a repository the server does not hold are 400 RepoNotFound.
  • getRepo with since, a rev, answers a CAR file whose root is the latest commit and whose blocks are those the repository holds now that commits after that rev added: the latest commit, and every record and MST node of the current tree that the tree at that rev did not have. Nothing outside the current tree is among them, so a record written after since and then replaced or deleted is not. A block the tree had at that rev is not among them either, unless a later commit removed it and another put it back, or wrote it again, as a record whose content another key holds: such a block may be among them or not. There are no blocks at all when since is the latest rev. A since that is not a TID is 400 InvalidRequest.
  • subscribeRepos with a cursor that is not an integer sends an error frame, InvalidRequest, and closes.

The server also serves com.atproto.sync.subscribeRepos, as its lexicon, atproto's Event Stream spec and atproto's Sync spec define it: a WebSocket at /xrpc/com.atproto.sync.subscribeRepos, each message a DAG-CBOR header and body.

  • Every commit is a #commit event, in the order the commits were made, with seq increasing. Its blocks are a CAR file whose root is the signed commit, holding the record blocks it wrote and MST nodes enough for two things, which are what is checked: every op proved against the commit (each created or updated key present with its CID, each deleted key absent), and the ops undone on the tree in the event, in reverse, giving the tree prevData names (atproto's Sync spec, the inversion it describes). The server may choose which other blocks it holds, and their order. Ops name prev, the CID a record had, for updates and deletes. since is the previous commit’s rev, and prevData its data. rebase and tooBig are false: a commit has at most 200 ops and 2000000 bytes of blocks, and one that would have more is refused (Limits).
  • A new account’s stream starts as the reference’s does: #identity, #account, a #commit of the empty repository with no ops, since or prevData, then #sync.
  • An account’s changes of state are #account events, with #identity and #sync when it is activated, and its deletion drops its earlier events from what the stream replays (Account lifecycle).
  • An account moving here (Moving an account between servers): createAccount with a did, and importRepo while the account is deactivated, make no events. submitPlcOperation, in any state, is #identity with the DID and no handle, and an active account’s importRepo is #sync of its new head.
  • An applyWrites commit has one op per write, in the batch’s order, not one per key changed: a create then an update of the same key are two ops. An op’s prev is the CID the record had before the commit, whatever the action and whatever earlier writes in the batch did: an update of a record the batch created has none, and a create after the batch deleted the same record has the deleted record’s.
  • With cursor, the stream first replays every event after that seq, not from it as atproto's Event Stream spec says, then goes on live. Without one, it is live only. A cursor later than the last event is refused as the reference refuses it: an error frame, FutureCursor. A cursor whose next event is out of the backfill window, PDS_REPO_BACKFILL_LIMIT_MS, gets an #info message, OutdatedCursor, first, then the events of the window (The store: what is kept, the backfill window, and the bounds).
  • The client sends nothing: a message from it is ignored, up to 4096 bytes, and a longer one closes the connection. One client address may hold 16 subscriptions, and the server 1024 (Limits).
  • Events are kept with everything else: after a restart, seq carries on from the last event, and a cursor from before it in the backfill window replays without a gap (Durability, The store: what is kept, the backfill window, and the bounds).

node compare.mjs --firehose ... subscribes to both servers from the start. At each check it waits for the event of each side’s latest commit, checks every new event the way a consumer would (seq, signature, the chain of since and prevData, undoing the ops, and the reference library’s verifyProofs), and compares the commits since the last check with the reference’s by their ops, ignoring their order, since writes sent at once may be committed in either. Sometimes it also subscribes again from a cursor, and from one in the future. With --crash-every, it subscribes again after each restart from the last event it had, and the stream must carry on with no gap or repeat.