Spaces: a repo whole, its blobs, and migration
What a repo host serves of a space repo beyond its records: the whole repo
as a STAR archive, the blobs its records reference, and a repo imported from
another host. Who reads them: the account itself, with a session, and a
syncer, with a space credential, whose reads of every repo method are on
this page. The space repo itself, its commits and its record methods are in
The space repo; STAR, MACs and their headers in
Spaces; credentials, and how a syncer gets one from the
space host with authorizeSpaceSync {space, credentialKey}, in
Spaces: credentials; what every method shares
in The PDS: accounts and their records.
The methods, all com.atproto.space: getRepo, getBlob, listBlobs
and importRepo. Their lexicons are in lexicons/com/atproto/space.
Who may read
Section titled “Who may read”Each repo read, getRecord, listRecords, getLatestCommit and
listRepoOps (The space repo), and getRepo, getBlob and listBlobs
here, takes either:
- a session, the account’s access token, from its password or an app
password, which reads only the account’s own repo: another
repois400 RepoNotFound, as The space repo says; or - a space credential, presented as Spaces: credentials (“Presenting
a credential”) says, whose audience is the
repoasked for. It reads any account’s repo in the credential’s space.
With a credential, a read checks, in this order, and answers the first that fails:
- its parameters against the lexicon,
400 InvalidRequest; - the credential (
verify_space_credential, Spaces: credentials, “Verifying a credential”), with therepoas its audience: each401its table names, a wrong audience401 BadSpaceAudience; - the credential’s space, which must be the
spaceasked for: else400 InvalidCredential; - the
repo: a DID no account here has is400 RepoNotFound; an account taken down is400 RepoTakendown, and one deactivated400 RepoDeactivated(below); Accept-Signature, which a credential’s read of a method that answers a commit or a record must send, asking for a MAC of the method’s profile (The space repo, “MACs on request”):atproto-space-recordforgetRecord,atproto-space-commitforgetLatestCommit,listRepoOpsandgetRepo. Without it, with one that asks for no MAC of that profile (an empty one, or one asking only for the other profile’s), or with one Spaces refuses:400 InvalidSpaceMac.listRecords,listBlobsandgetBlobtake no MAC, and ignore the header, whatever it is;- then the method, as with a session, the MAC as The space repo says.
So a repo never written answers a credential as it answers its account:
getRecord RecordNotFound, listRecords {records: []},
getLatestCommit, listRepoOps and getRepo RepoNotFound, listBlobs
{cids: []}, getBlob BlobNotFound. A credential is refused by the
methods that take only a session, listSpaces and every write, as any
scheme they do not take: 400 InvalidToken.
getRepo
Section titled “getRepo”getRepo ?space&repo answers the repo whole: 200, Content-Type: application/vnd.dasl.star, the lexicon’s output encoding, whatever the
request’s Accept, and as the body the repo’s STAR archive with the
profile atproto-space-repo, as Spaces defines it:
- its metadata the repo’s commit,
{$type: "atproto-space-repo", ver: 1, repo, space, rev, data}, asgetLatestCommitanswers it, encoded as DRISL, so its keys in DRISL’s order; - then each record, its key the bytes of
{collection}/{rkey}and its value the record’s DRISL bytes, those its CID names, in strict ascending order of the keys; a repo with no records, as afterclearRecords, has no pairs.
So the archive of a commit and its records is one string of bytes, the one
spec.mjs’s writer makes of them. Accept-Signature is taken as by
getLatestCommit (The space repo, “MACs on request”), for the profile
atproto-space-commit: with one that asks for a MAC, the answer has
Atproto-Space-Commit of the archive’s commit, Signature-Input and
Signature, and Access-Control-Expose-Headers naming the three; one
Spaces refuses is 400 InvalidSpaceMac; every 200 has Vary naming
Accept-Signature. A repo never written is 400 RepoNotFound, after the
header is checked. It is a query: POST is 400 InvalidRequest.
Blobs in space records
Section titled “Blobs in space records”A space record references blobs as a public record does: every blob ref in it is one, by the rules of Blobs (“Blob refs in records”), and a blob an account holds is one store for both. So, in a space write:
- each ref is checked last, after everything The space repo says a write
checks, the space (
SpaceNotFound) and the records’ existence (RecordAlreadyExists,RecordNotFound) included, against the blobs the account held before the write: a blob it does not hold, another account’s too, is400 BlobNotFound; one no record, public or in a space, referenced before must match the ref (400 InvalidMimeType, then400 InvalidSize); one some record referenced is not checked; - a blob is held while any record of the account references it, public or
in any of its space repos, each record counted once; a write that
deletes or replaces the last record that references it,
clearRecordsincluded, and puts none in its place, stops it being held, and a ref to it isBlobNotFounduntil it is uploaded again.
The public methods count only public records: a blob that only space
records reference is not served by com.atproto.sync.getBlob (400, as a
blob no record references, Blobs) and not listed by
com.atproto.sync.listBlobs, so a blob behind a space’s perimeter is read
only through it.
getBlob
Section titled “getBlob”getBlob ?space&repo&cid answers a blob the repo’s records in that space
reference:
- a
cidthat does not parse is400 InvalidRequest, with the other parameters; - a CID no record of the repo in that space references, whatever else the
account holds (an upload, a blob of a public record or of another
space), or no repo, is
400 BlobNotFound; - a blob a record references whose bytes the account does not hold, as
after
importRepo(below), is400 InvalidRequest.
Otherwise it answers 200 with the bytes, and the headers Blobs’s
getBlob answers: Content-Type the stored mimeType (compared up to its
first ;), Content-Length, X-Content-Type-Options: nosniff,
Content-Disposition: attachment; filename="<cid>" and
Content-Security-Policy: default-src 'none'; sandbox.
listBlobs
Section titled “listBlobs”listBlobs ?space&repo&limit?&cursor? answers {cids, cursor?}: the CIDs
of the blobs the repo’s records in that space reference, each once, in
ascending order of the CID string, at most limit (1 to 1000, 500 by
default), whether or not the account holds their bytes.
- With
cursor, only CIDs greater than it, comparing strings; it need not be a CID, and none is refused. - A
since, or any other parameter the lexicon does not name, is ignored, whatever its value: every blob is listed. cursorin the answer is the page’s last CID when the page is full (limitCIDs), and is left out otherwise. No repo answers{cids: []}.
importRepo
Section titled “importRepo”importRepo takes a space repo’s STAR archive, as getRepo answers one,
as the raw request body, and replaces the caller’s repo in the space it
names with it. It checks, in this order, and answers the first error:
- The token: a session made with the account’s password. None is
401 AuthMissing; an app password’s session, a refresh token, a credential or a token the server did not give out is400 InvalidToken. - The
Content-Type: its media type, up to the first;, must beapplication/vnd.dasl.star, the lexicon’s input encoding, whatever parameters follow; none, or another,application/octet-streamincluded, is400 InvalidRequest. - The size: a body of more than
PDS_BLOB_UPLOAD_LIMITbytes (Limits), 5242880 (5 MiB) unless it is set, is413 PayloadTooLarge, the nameuploadBlob’s is; one of exactly the limit is taken. - The archive, read as Spaces’s reader reads one, with no MAC and no
space or repo expected: any fault is
400 InvalidRepoFile(no bytes, a bad magic, bytes after the end, a limit passed, metadata that is not anatproto-space-repocommit withver1, itsreponot a DID, itsspacenot aspace-ref, itsrevnot a TID, a key that is not a record path, keys out of order or repeated, a value that is not a DRISL map, the records’ LtHash notdata). Metadata fields beyond the commit’s are ignored. - The commit’s
repo, which must be the caller’s DID: else400 RepoMismatch. - The space, as a write’s (The space repo): one of the caller’s own must
exist here, imported by
importSpaceMetaor created (the policies part), else400 SpaceNotFound. Another authority’s is not checked. - Each record’s bytes, which must be its canonical DRISL encoding: else
400 InvalidRepoFile.
Then the caller’s repo in that space becomes the archive’s, all at once:
exactly its records, as their bytes, the published collections too; an
empty operation log; a new commit, its rev later than the repo’s last (a
TID of the time, as a write’s), its data the archive’s. A repo not there
before comes to be, as with a first write (listSpaces lists its space).
It answers 200 with no body.
An imported record’s blob refs are taken without being checked: each
blob is referenced, and listed by listBlobs, whether the account holds it
or not. getBlob of one it does not hold is 400 InvalidRequest (above),
until it is uploaded: uploadBlob of those bytes makes it held, as a
record already references it, and getBlob serves it, with the
mimeType of the upload. A blob the account held once and holds no
longer, as after the last record that referenced it was deleted, is one it
does not hold: the import’s reference to it does not hold it again, and
the mimeType it was uploaded with before is forgotten.
An import is not a write the space host hears of: it calls no on_commit
(The space repo), and the writer set keeps the rev it had. An account
deactivated may import; one taken down may not (below).
Takedown and deactivation
Section titled “Takedown and deactivation”For the account whose repo is read, deactivated
(com.atproto.server.deactivateAccount) or taken down by the server’s
operator:
| Read | Deactivated | Taken down |
|---|---|---|
| with the account’s session | answered as for an active account | its allowTakendown session (createSession with allowTakendown: true): getRepo, listBlobs and listSpaces answered as for an active account; getBlob not said (below); every other read, every write and importRepo 400 InvalidToken |
| with a credential | 400 RepoDeactivated | 400 RepoTakendown |
An account’s state is Account lifecycle’s, taken down before deactivated.
The reference’s admin token (Basic)
also reads these, and its sessions from before a takedown keep reading;
neither is compared. Nor is getBlob with an allowTakendown session.
Note: the reference answered it 400 BlobNotFound in one run and served
the blob in another.
Durability
Section titled “Durability”As Durability says of the rest: an import the server answered, and
the blobs space records reference, survive it being killed; after a
restart, getRepo answers the same bytes, getBlob the same blobs and
listBlobs the same CIDs.
Commands
Section titled “Commands”As Commands: every method as an MCP tool and a CLI command makes them, group space: listBlobs answers JSON, so it
is the tool space_list_blobs and the command pds space list-blobs.
getRepo and getBlob answer bytes, so they have no tool, and their
commands, pds space get-repo and pds space get-blob, write the bytes
to standard output, as pds sync get-repo does. importRepo takes bytes,
so it has no tool, and pds space import-repo sends standard input as the
body, with Content-Type: application/vnd.dasl.star, the lexicon’s input
encoding; it takes no flags. None of them sends Accept-Signature.
For other parts
Section titled “For other parts”on_commitis not called byimportRepo(above).importSpaceMeta(the policies part) must come beforeimportRepoof the authority’s own repo in a space it imports, as step 6 says.- Takedown and deactivation reach the repo reads through step 4 above
and the table; the session side is the sessions’ (
allowTakendown).
How it is checked
Section titled “How it is checked”In the spaces run (Spaces), beside the space repo’s steps
(The space repo), with the server and the alpha’s first PDS each the repo
host of their side’s accounts, and the authorities’ spaces on the alpha’s
second PDS, the peer, made with publicRead, so that the harness, as a
syncer, gets credentials for them from the peer’s authorizeSpaceSync,
with no delegation token:
- Blobs. Accounts upload blobs (
spec-pds blobbodies) and write space records referencing them, the uploads as answered and refs that may be wrong (anothermimeTypeorsize, a CID never uploaded, a blob no longer held); delete and replace them; and reference some from public records too.getBlobandlistBlobs(paged, sometimes with asince, the side’s own revs or no TID, which must be ignored) are compared, and the harness holds each side to the blobs its own model says the account holds and each repo references; the publicgetBlobof a blob only space records reference must be400and the publiclistBlobsmust leave it out. - getRepo of every repo, by session and by credential, with
Accept-Signaturedrawn as the space repo’s reads draw it (The space repo, “How it is checked”). On each side the answer must beapplication/vnd.dasl.star, the archive must parse withspec.mjs’s reader, its MAC verify from the harness’s key share and itsAtproto-Space-Commitname its commit, with the response headers The space repo holds, its commit begetLatestCommit’s and its records the model’s, and its bytes be exactly whatspec.mjs’s writer makes of that commit and those records; a refused header, or a credential’s read asking for no MAC, must be400 InvalidSpaceMac. Across the sides, the two archives must be the same but for the account’s DID, therev, and record keys each side made: the same records, byte for byte. - Credential reads of every repo method, with credentials from the
peer: the account’s own repo, another account’s on the same host, one
never written, a DID no account here has, a wrong audience, a credential
of another space; with a good
Accept-Signaturemostly, and otherwise none, or one ofinterop/spaces/mac.json’s cases, where the method answers a MAC each side must answer400 InvalidSpaceMacunless a MAC is asked for, andlistRecordsmust ignore the header and answer no MAC headers. - importRepo: each side’s own
getRepoarchive imported back, archives the harness writes withspec.mjs(records with blob refs to blobs held, not held, held no longer and never uploaded, published collections, an empty archive, into a space with no repo yet), sent withapplication/vnd.dasl.star, with or without parameters, and archives with one thing wrong for each step above, each side held to the error that step names: a metadata field of another format, another account’srepo(with a bad LtHash too,InvalidRepoFilefirst), a value not canonical DRISL into a space of the caller’s own never created (SpaceNotFoundfirst); app password and refresh tokens; a JSON orapplication/octet-streamContent-Type, or none; bodies of 5242880 and 5242881 bytes. After each, the repo is read whole, andlistBlobs,getBloband an upload of a blob it references are compared. - A restart of the server: every repo’s
getRepo, and the blobs, read again. - Takedown and deactivation: the reference alone, as above, until the server can be put in either state.
Not compared: a getBlob answer’s other headers, as Blobs says; the
admin token; sessions from before a takedown; the blobs an import replaced
(the reference forgets their refs without dropping the blobs); the writer
set after an import (the notifications part’s run reads it); and
importRepo with an OAuth token (OAuth: access tokens on every method, and what scopes allow, “What each method
needs”), which no run calls.