Skip to content

Spaces: a repo whole, its blobs, and migration

What a repo host serves of a space repo beyond its records: the whole repo as a STAR archive, the blobs its records reference, and a repo imported from another host. Who reads them: the account itself, with a session, and a syncer, with a space credential, whose reads of every repo method are on this page. The space repo itself, its commits and its record methods are in The space repo; STAR, MACs and their headers in Spaces; credentials, and how a syncer gets one from the space host with authorizeSpaceSync {space, credentialKey}, in Spaces: credentials; what every method shares in The PDS: accounts and their records.

The methods, all com.atproto.space: getRepo, getBlob, listBlobs and importRepo. Their lexicons are in lexicons/com/atproto/space.

Each repo read, getRecord, listRecords, getLatestCommit and listRepoOps (The space repo), and getRepo, getBlob and listBlobs here, takes either:

  • a session, the account’s access token, from its password or an app password, which reads only the account’s own repo: another repo is 400 RepoNotFound, as The space repo says; or
  • a space credential, presented as Spaces: credentials (“Presenting a credential”) says, whose audience is the repo asked for. It reads any account’s repo in the credential’s space.

With a credential, a read checks, in this order, and answers the first that fails:

  1. its parameters against the lexicon, 400 InvalidRequest;
  2. the credential (verify_space_credential, Spaces: credentials, “Verifying a credential”), with the repo as its audience: each 401 its table names, a wrong audience 401 BadSpaceAudience;
  3. the credential’s space, which must be the space asked for: else 400 InvalidCredential;
  4. the repo: a DID no account here has is 400 RepoNotFound; an account taken down is 400 RepoTakendown, and one deactivated 400 RepoDeactivated (below);
  5. Accept-Signature, which a credential’s read of a method that answers a commit or a record must send, asking for a MAC of the method’s profile (The space repo, “MACs on request”): atproto-space-record for getRecord, atproto-space-commit for getLatestCommit, listRepoOps and getRepo. Without it, with one that asks for no MAC of that profile (an empty one, or one asking only for the other profile’s), or with one Spaces refuses: 400 InvalidSpaceMac. listRecords, listBlobs and getBlob take no MAC, and ignore the header, whatever it is;
  6. then the method, as with a session, the MAC as The space repo says.

So a repo never written answers a credential as it answers its account: getRecord RecordNotFound, listRecords {records: []}, getLatestCommit, listRepoOps and getRepo RepoNotFound, listBlobs {cids: []}, getBlob BlobNotFound. A credential is refused by the methods that take only a session, listSpaces and every write, as any scheme they do not take: 400 InvalidToken.

getRepo ?space&repo answers the repo whole: 200, Content-Type: application/vnd.dasl.star, the lexicon’s output encoding, whatever the request’s Accept, and as the body the repo’s STAR archive with the profile atproto-space-repo, as Spaces defines it:

  • its metadata the repo’s commit, {$type: "atproto-space-repo", ver: 1, repo, space, rev, data}, as getLatestCommit answers it, encoded as DRISL, so its keys in DRISL’s order;
  • then each record, its key the bytes of {collection}/{rkey} and its value the record’s DRISL bytes, those its CID names, in strict ascending order of the keys; a repo with no records, as after clearRecords, has no pairs.

So the archive of a commit and its records is one string of bytes, the one spec.mjs’s writer makes of them. Accept-Signature is taken as by getLatestCommit (The space repo, “MACs on request”), for the profile atproto-space-commit: with one that asks for a MAC, the answer has Atproto-Space-Commit of the archive’s commit, Signature-Input and Signature, and Access-Control-Expose-Headers naming the three; one Spaces refuses is 400 InvalidSpaceMac; every 200 has Vary naming Accept-Signature. A repo never written is 400 RepoNotFound, after the header is checked. It is a query: POST is 400 InvalidRequest.

A space record references blobs as a public record does: every blob ref in it is one, by the rules of Blobs (“Blob refs in records”), and a blob an account holds is one store for both. So, in a space write:

  • each ref is checked last, after everything The space repo says a write checks, the space (SpaceNotFound) and the records’ existence (RecordAlreadyExists, RecordNotFound) included, against the blobs the account held before the write: a blob it does not hold, another account’s too, is 400 BlobNotFound; one no record, public or in a space, referenced before must match the ref (400 InvalidMimeType, then 400 InvalidSize); one some record referenced is not checked;
  • a blob is held while any record of the account references it, public or in any of its space repos, each record counted once; a write that deletes or replaces the last record that references it, clearRecords included, and puts none in its place, stops it being held, and a ref to it is BlobNotFound until it is uploaded again.

The public methods count only public records: a blob that only space records reference is not served by com.atproto.sync.getBlob (400, as a blob no record references, Blobs) and not listed by com.atproto.sync.listBlobs, so a blob behind a space’s perimeter is read only through it.

getBlob ?space&repo&cid answers a blob the repo’s records in that space reference:

  • a cid that does not parse is 400 InvalidRequest, with the other parameters;
  • a CID no record of the repo in that space references, whatever else the account holds (an upload, a blob of a public record or of another space), or no repo, is 400 BlobNotFound;
  • a blob a record references whose bytes the account does not hold, as after importRepo (below), is 400 InvalidRequest.

Otherwise it answers 200 with the bytes, and the headers Blobs’s getBlob answers: Content-Type the stored mimeType (compared up to its first ;), Content-Length, X-Content-Type-Options: nosniff, Content-Disposition: attachment; filename="<cid>" and Content-Security-Policy: default-src 'none'; sandbox.

listBlobs ?space&repo&limit?&cursor? answers {cids, cursor?}: the CIDs of the blobs the repo’s records in that space reference, each once, in ascending order of the CID string, at most limit (1 to 1000, 500 by default), whether or not the account holds their bytes.

  • With cursor, only CIDs greater than it, comparing strings; it need not be a CID, and none is refused.
  • A since, or any other parameter the lexicon does not name, is ignored, whatever its value: every blob is listed.
  • cursor in the answer is the page’s last CID when the page is full (limit CIDs), and is left out otherwise. No repo answers {cids: []}.

importRepo takes a space repo’s STAR archive, as getRepo answers one, as the raw request body, and replaces the caller’s repo in the space it names with it. It checks, in this order, and answers the first error:

  1. The token: a session made with the account’s password. None is 401 AuthMissing; an app password’s session, a refresh token, a credential or a token the server did not give out is 400 InvalidToken.
  2. The Content-Type: its media type, up to the first ;, must be application/vnd.dasl.star, the lexicon’s input encoding, whatever parameters follow; none, or another, application/octet-stream included, is 400 InvalidRequest.
  3. The size: a body of more than PDS_BLOB_UPLOAD_LIMIT bytes (Limits), 5242880 (5 MiB) unless it is set, is 413 PayloadTooLarge, the name uploadBlob’s is; one of exactly the limit is taken.
  4. The archive, read as Spaces’s reader reads one, with no MAC and no space or repo expected: any fault is 400 InvalidRepoFile (no bytes, a bad magic, bytes after the end, a limit passed, metadata that is not an atproto-space-repo commit with ver 1, its repo not a DID, its space not a space-ref, its rev not a TID, a key that is not a record path, keys out of order or repeated, a value that is not a DRISL map, the records’ LtHash not data). Metadata fields beyond the commit’s are ignored.
  5. The commit’s repo, which must be the caller’s DID: else 400 RepoMismatch.
  6. The space, as a write’s (The space repo): one of the caller’s own must exist here, imported by importSpaceMeta or created (the policies part), else 400 SpaceNotFound. Another authority’s is not checked.
  7. Each record’s bytes, which must be its canonical DRISL encoding: else 400 InvalidRepoFile.

Then the caller’s repo in that space becomes the archive’s, all at once: exactly its records, as their bytes, the published collections too; an empty operation log; a new commit, its rev later than the repo’s last (a TID of the time, as a write’s), its data the archive’s. A repo not there before comes to be, as with a first write (listSpaces lists its space). It answers 200 with no body.

An imported record’s blob refs are taken without being checked: each blob is referenced, and listed by listBlobs, whether the account holds it or not. getBlob of one it does not hold is 400 InvalidRequest (above), until it is uploaded: uploadBlob of those bytes makes it held, as a record already references it, and getBlob serves it, with the mimeType of the upload. A blob the account held once and holds no longer, as after the last record that referenced it was deleted, is one it does not hold: the import’s reference to it does not hold it again, and the mimeType it was uploaded with before is forgotten.

An import is not a write the space host hears of: it calls no on_commit (The space repo), and the writer set keeps the rev it had. An account deactivated may import; one taken down may not (below).

For the account whose repo is read, deactivated (com.atproto.server.deactivateAccount) or taken down by the server’s operator:

ReadDeactivatedTaken down
with the account’s sessionanswered as for an active accountits allowTakendown session (createSession with allowTakendown: true): getRepo, listBlobs and listSpaces answered as for an active account; getBlob not said (below); every other read, every write and importRepo 400 InvalidToken
with a credential400 RepoDeactivated400 RepoTakendown

An account’s state is Account lifecycle’s, taken down before deactivated. The reference’s admin token (Basic) also reads these, and its sessions from before a takedown keep reading; neither is compared. Nor is getBlob with an allowTakendown session.

Note: the reference answered it 400 BlobNotFound in one run and served the blob in another.

As Durability says of the rest: an import the server answered, and the blobs space records reference, survive it being killed; after a restart, getRepo answers the same bytes, getBlob the same blobs and listBlobs the same CIDs.

As Commands: every method as an MCP tool and a CLI command makes them, group space: listBlobs answers JSON, so it is the tool space_list_blobs and the command pds space list-blobs. getRepo and getBlob answer bytes, so they have no tool, and their commands, pds space get-repo and pds space get-blob, write the bytes to standard output, as pds sync get-repo does. importRepo takes bytes, so it has no tool, and pds space import-repo sends standard input as the body, with Content-Type: application/vnd.dasl.star, the lexicon’s input encoding; it takes no flags. None of them sends Accept-Signature.

  • on_commit is not called by importRepo (above).
  • importSpaceMeta (the policies part) must come before importRepo of the authority’s own repo in a space it imports, as step 6 says.
  • Takedown and deactivation reach the repo reads through step 4 above and the table; the session side is the sessions’ (allowTakendown).

In the spaces run (Spaces), beside the space repo’s steps (The space repo), with the server and the alpha’s first PDS each the repo host of their side’s accounts, and the authorities’ spaces on the alpha’s second PDS, the peer, made with publicRead, so that the harness, as a syncer, gets credentials for them from the peer’s authorizeSpaceSync, with no delegation token:

  • Blobs. Accounts upload blobs (spec-pds blob bodies) and write space records referencing them, the uploads as answered and refs that may be wrong (another mimeType or size, a CID never uploaded, a blob no longer held); delete and replace them; and reference some from public records too. getBlob and listBlobs (paged, sometimes with a since, the side’s own revs or no TID, which must be ignored) are compared, and the harness holds each side to the blobs its own model says the account holds and each repo references; the public getBlob of a blob only space records reference must be 400 and the public listBlobs must leave it out.
  • getRepo of every repo, by session and by credential, with Accept-Signature drawn as the space repo’s reads draw it (The space repo, “How it is checked”). On each side the answer must be application/vnd.dasl.star, the archive must parse with spec.mjs’s reader, its MAC verify from the harness’s key share and its Atproto-Space-Commit name its commit, with the response headers The space repo holds, its commit be getLatestCommit’s and its records the model’s, and its bytes be exactly what spec.mjs’s writer makes of that commit and those records; a refused header, or a credential’s read asking for no MAC, must be 400 InvalidSpaceMac. Across the sides, the two archives must be the same but for the account’s DID, the rev, and record keys each side made: the same records, byte for byte.
  • Credential reads of every repo method, with credentials from the peer: the account’s own repo, another account’s on the same host, one never written, a DID no account here has, a wrong audience, a credential of another space; with a good Accept-Signature mostly, and otherwise none, or one of interop/spaces/mac.json’s cases, where the method answers a MAC each side must answer 400 InvalidSpaceMac unless a MAC is asked for, and listRecords must ignore the header and answer no MAC headers.
  • importRepo: each side’s own getRepo archive imported back, archives the harness writes with spec.mjs (records with blob refs to blobs held, not held, held no longer and never uploaded, published collections, an empty archive, into a space with no repo yet), sent with application/vnd.dasl.star, with or without parameters, and archives with one thing wrong for each step above, each side held to the error that step names: a metadata field of another format, another account’s repo (with a bad LtHash too, InvalidRepoFile first), a value not canonical DRISL into a space of the caller’s own never created (SpaceNotFound first); app password and refresh tokens; a JSON or application/octet-stream Content-Type, or none; bodies of 5242880 and 5242881 bytes. After each, the repo is read whole, and listBlobs, getBlob and an upload of a blob it references are compared.
  • A restart of the server: every repo’s getRepo, and the blobs, read again.
  • Takedown and deactivation: the reference alone, as above, until the server can be put in either state.

Not compared: a getBlob answer’s other headers, as Blobs says; the admin token; sessions from before a takedown; the blobs an import replaced (the reference forgets their refs without dropping the blobs); the writer set after an import (the notifications part’s run reads it); and importRepo with an OAuth token (OAuth: access tokens on every method, and what scopes allow, “What each method needs”), which no run calls.