Spaces
atproto spaces are permissioned data: records with an access perimeter, kept beside the public repository rather than in it, and read only with a credential the space’s authority issues. This page is where spaces start: who takes part, the keys, what this server is in each part, and the building blocks every later part rests on, each specified exactly enough to build from and held to test vectors. What every method shares, and how the server is checked, is in The PDS: accounts and their records.
Spaces are an alpha. The protocol is spaces/proposal.md, the atproto
proposal 0016 as vendored (spaces/README.md says which revision, and the
other sources and their licences). Where this page and the proposal say
the same thing, this page is the one to build from; where this page is
silent, the proposal is. The reference for spaces is the pinned spaces
alpha of the reference PDS, which the spaces run starts (below), not the
latest reference the other pages compare with.
Addressing
Section titled “Addressing”A space is the triple of its authority, a DID; its type, an NSID;
and its key (skey), a string with the syntax of a record key
(atproto's Record Key spec) and at most 512 bytes long. It is written as a
URI, and a record in it as a longer one:
at://{authority}/space/{type}/{skey}at://{authority}/space/{type}/{skey}/{author}/{collection}/{rkey}space is a literal segment, which no collection can be, as an NSID always
has dots. A string in the format space-ref is a space’s URI and no
more: exactly the first form, with each part valid as said.
Who takes part
Section titled “Who takes part”| Role | Description | This server |
|---|---|---|
| author | an account writing records into a space, each into its own space repo: one per account and space | each of its accounts |
| repo host | holds an author’s space repos, serves their writes and reads, and tells the space host of each write | the repo host of its accounts’ space repos |
| space authority | the DID at the root of a space: its #atproto key signs the space’s credentials, and it decides who may read the space and whose writes it tracks | an account of its own, for the spaces that account creates |
| space host | the authority’s PDS (its #atproto_pds): authorizes syncers, issuing their credentials, keeps the writer set, sequences the space’s updates and notifies the syncers registered for them | the space host of every space whose authority is one of its accounts |
| user’s PDS | mints the delegation token an app trades for a credential | the PDS of its accounts |
| syncer | an app keeping a copy of a space: pulls repos from their hosts, is told of the space’s updates by the space host at an endpoint of its own | none: the harness plays it |
The parts built on the building blocks below each have a page of their own:
- the space repo: the
com.atproto.spacerecord writes and reads, its commit and operation log (The space repo); - credentials: delegation tokens, space credentials, revocation, in Spaces: credentials, which verifies credentials for the methods that take them;
- policies and the space lifecycle: creating, updating and deleting spaces, and member lists: Spaces: policies and the space lifecycle;
- full-state sync: a repo as a STAR archive (
getRepo), blobs, and importing a repo: Spaces: a repo whole, its blobs, and migration; - write notifications, the writer set, syncers’ registrations and the space updates sent them: Spaces: write notifications, the writer set and syncers.
Their lexicons join lexicons/ when they are served; spaces/lexicons/
has those the server calls on others and does not serve.
com.atproto.space.defs (the commit, and the updates a writer set holds)
is in lexicons/, with the space repo’s methods.
Each account has two keys, both kept by the server for the account alone:
#atproto, the signing key Accounts gives it. For spaces, it is also the key that signs the delegation tokens the account’s PDS mints, the credentials of the spaces the account is the authority of, and the service auth it sends as their authority (later parts).#atproto_kem, a P-256 key of its own, below. It signs nothing: a syncer and the repo host agree a shared secret with it, under which the repo host authenticates what it serves with MACs.
The KEM key
Section titled “The KEM key”createAccount makes the account a new P-256 keypair, its KEM key,
with the signing key (Accounts), and keeps the private key as it keeps
the signing key: it survives restarts (Durability), and no other
account’s data holds it. Its public key, as a did:key (below), is:
verificationMethods.atproto_kemin the DID’s genesis operation, besideverificationMethods.atproto;verificationMethods.atproto_kemin whatgetRecommendedDidCredentialsanswers (Identity);- in the DID document the directory serves, as the directory renders a
verification method named
atproto_kem. The proposal’s form is aMultikeywith id<did>#atproto_kem. The harness’s directory renders it in the legacy form atproto's DID spec describes: id#atproto_kem, typeEcdsaSecp256r1VerificationKey2019, the uncompressed point, andhttps://w3id.org/security/suites/ecdsa-2019/v1added to@context. The server serves what the directory serves, as it does for the signing key.
The KEM key does not change: an update operation the server posts, as for a handle change, keeps it, as it keeps every field but the one it changes.
describeServer
Section titled “describeServer”describeServer answers spacesAlpha: true besides what Accounts
and Blobs list: the reference’s flag that it serves the spaces alpha.
Note: describeServer’s lexicon does not name it; the reference answers
it all the same, and calls it temporary.
The building blocks
Section titled “The building blocks”Each is held to a file of test vectors in interop/spaces/ (below), and
STAR to the published fixtures in interop/star/ as well. Hex in a vector
is lower case; a vector’s comment says what the case is about.
P-256 did:keys
Section titled “P-256 did:keys”A P-256 public key as a string is a did:key, as
atproto's Cryptography spec defines it: did:key:z, then base58btc of the
bytes 0x80 0x24 (the multicodec p256-pub) followed by the 33-byte
compressed point (0x02 or 0x03, then x). Every P-256 did:key starts
did:key:zDn. A key share or credential key that is not a did:key, does
not parse, is of another curve (such as secp256k1’s 0xE7 0x01), is not 33
bytes of compressed point, or names an x with no point on P-256, is no
P-256 key.
The uncompressed form of a point is 65 bytes: 0x04, x, y, each 32
bytes big-endian.
The shared secret
Section titled “The shared secret”A syncer that wants a MAC sends a key share: a fresh P-256 public key,
as a did:key. The repo host makes the shared secret from it and the
author’s KEM private key d, as HPKE’s secret export (RFC 9180 §5.3,
spaces/rfc9180.txt) in base mode, with the ciphersuite
DHKEM(P-256, HKDF-SHA256), HKDF-SHA256 and the Export-only AEAD, the key
share as enc, info the ASCII bytes atproto-mac, an empty exporter
context and a length of 32. Spelled out, ‖ joining bytes, quoted parts
ASCII, and u16(n) the two bytes of n big-endian:
Pis the key share’s point (an error if it is no P-256 key, as above).dhis the 32-byte big-endianxof the pointd·P(ECDH).encisPuncompressed, andauthoris the author’s KEM public key uncompressed, 65 bytes each.Extract(salt, ikm)is HMAC-SHA256 ofikmkeyed withsalt(an emptysaltan empty key), andExpand(prk, info)is HMAC-SHA256 ofinfo ‖ 0x01keyed withprk: HKDF’s extract, and its expand to 32 bytes.- With
kemthe bytes"KEM" ‖ u16(0x0010)andsuitethe bytes"HPKE" ‖ u16(0x0010) ‖ u16(0x0001) ‖ u16(0xFFFF),LabeledExtract(id, salt, label, ikm)isExtract(salt, "HPKE-v1" ‖ id ‖ label ‖ ikm), andLabeledExpand(id, prk, label, info)isExpand(prk, u16(32) ‖ "HPKE-v1" ‖ id ‖ label ‖ info). kem_secretisLabeledExpand(kem, LabeledExtract(kem, "", "eae_prk", dh), "shared_secret", enc ‖ author): DHKEM’s shared secret (RFC 9180 §4.1).contextis0x00 ‖ LabeledExtract(suite, "", "psk_id_hash", "") ‖ LabeledExtract(suite, "", "info_hash", "atproto-mac"), andexporter_secretisLabeledExpand(suite, LabeledExtract(suite, kem_secret, "secret", ""), "exp", context): the key schedule, with no pre-shared key.- The shared secret is
LabeledExpand(suite, exporter_secret, "sec", "").
The syncer makes the same secret from its private key and the author’s
public one. interop/spaces/kem.json: each valid case gives dh,
kem_secret and shared_secret for its author key, key share and info;
the first is RFC 9180’s test vector A.3.1, whose kem_secret is the
RFC’s shared_secret, and the second is HPKE’s own test vector for this
ciphersuite, with its info. Each invalid key share must be refused.
The set hash, records and commits
Section titled “The set hash, records and commits”A space repo’s records are summed by LtHash:
- The state is 2048 bytes, read as 1024 unsigned 16-bit lanes, little endian; it starts all zeros.
- An element is a string. To add it, expand its UTF-8 bytes to 2048 bytes with BLAKE3 in its extendable-output mode (the plain hash, unkeyed, read to 2048 bytes), read those as 1024 little-endian lanes, and add each to the state’s lane of the same index, modulo 65536. To remove it, subtract instead, modulo 65536.
- The digest is SHA-256 of the 2048 bytes of the state.
So the state depends only on which elements are in it, not the order they
came in, and the empty digest is
e5a00aa9991ac8a5ee3109844d84a55583bd20572ad3ffcd42792f3c36b183ad.
interop/spaces/lthash.json gives digests for elements added and removed,
and the whole state after adding a.
A record in a space repo is the same DRISL (DAG-CBOR) value a public
record is (The repository's records), with the same CID: version 1, codec 0x71, a
SHA-256 multihash of its bytes, in base32. Its element is
{collection}/{rkey}/{cid}, the CID as a string. A space repo’s
commit (com.atproto.space.defs#commit) is {ver: 1, repo, space, rev, data}: repo the author’s DID, space the space’s URI, rev a TID,
and data the LtHash digest of the elements of every record the repo
holds, 32 bytes. It has no signature.
HTTP Message Signatures
Section titled “HTTP Message Signatures”Both a request made with a space credential and a repo host’s MAC are
HTTP Message Signatures, as RFC 9421 (spaces/rfc9421.txt) defines them,
with the choices below. What they share:
Signature-Input,SignatureandAccept-Signatureare RFC 9651 Dictionaries. A signature’s label is its member’s name, and carries no meaning: a signature is found by itstagparameter. The members taggedTare those whose value is an Inner List with a parametertagthat is the StringT; any other member (another tag, no tag, atagthat is a Token, a value that is no Inner List) is passed over. A member, or a parameter, given more than once counts as its last, as RFC 9651 reads a Dictionary.- The signature base is one line per covered component, in the Inner
List’s order, then the line
"@signature-params":and the Inner List with its parameters, serialized as RFC 9651 §4.1.1.1 serializes one, so;sf=?1is written;sf; the lines are joined by a line feed, with none after the last, in UTF-8. A component’s line is the component as serialized there,:, and its value. alg, if a signature has it, must be the String the signature’s kind names below;keyid, if it has it, must be a String, the key it is checked against.
A credential’s signature. A request made with a space credential
carries it in Authorization, and signs it with the key the credential is
bound to (its cnf.kid, a P-256 did:key), as in:
Authorization: Atproto-Space-Credential <credential>Atproto-Space-Audience: <a DID>Signature-Input: sig=("authorization" "atproto-space-audience");tag="atproto-space-credential"Signature: sig=:<64 bytes, base64>:To verify one against the credential’s key:
-
Signature-Inputmust have exactly one member taggedatproto-space-credential, andSignaturea member of the same label. InSignature-Inputit must be an Inner List of exactly the Strings"authorization"and"atproto-space-audience", in that order, with no parameters on them.alg, if given, must beecdsa-p256-sha256, andkeyid, if given, the credential’s key; other parameters are allowed (they are signed, in the base). InSignaturethe member must be a Byte Sequence with no parameters, of 64 bytes; base64 padding is optional. -
The base’s two component lines are
"authorization": <the Authorization value>"atproto-space-audience": <the Atproto-Space-Audience value>each value as received, less leading and trailing whitespace:
Atproto-Space-Credentialkeeps the case it was sent in. -
The 64 bytes are
r ‖ s, each 32 bytes big-endian. The signature must verify as ECDSA over P-256 with SHA-256 of the base, against the credential’s key, which must be a P-256did:key. Both low-S and high-S signatures verify; a DER signature is refused.
interop/spaces/http-signature.json: requests’ headers, each valid or
not against key_id, with the exact signature_base of each valid one.
These hold the signature alone; the credential itself, and that the
audience is the one the request needs, are the credentials’ part (later).
A repo host authenticates a commit or a record to a syncer with a MAC: an
HTTP Message Signature with the algorithm hmac-sha256 (RFC 9421 §3.3.3),
keyed with their shared secret, over a response header that describes what
it answers. There are two profiles, each a tag and the header it
covers:
| Profile | For | Covers |
|---|---|---|
atproto-space-commit | a commit | Atproto-Space-Commit |
atproto-space-record | a record | Atproto-Space-Record |
Asking. A syncer asks for a MAC with the request header
Accept-Signature (RFC 9421 §5.1), as in:
Accept-Signature: mac=("atproto-space-commit";sf);keyid="did:key:zDn...";tag="atproto-space-commit"The repo host reads it for the one profile the method answers:
- No header, or one with no member tagged with the profile, asks for no MAC; members tagged otherwise are passed over.
- A header that is not a Dictionary is refused, and so is one with more than one member tagged with the profile.
- That member’s Inner List must be exactly one item, the String naming the
profile’s header in lower case (
"atproto-space-commit"or"atproto-space-record"), with exactly the one parametersf, the Boolean true. Its parameters must havekeyid, a String, the key share, and may havealg, which must be the Stringhmac-sha256; any other parameter is refused,createdandnonceamong them. - The key share must be a P-256 key (above), or the request is refused.
Answering. The response has three headers more, and Vary naming
Accept-Signature:
Atproto-Space-Commit: ver=1, repo="did:plc:...", space="at://...", rev="3mwcj3b6uak2f", data=:<the 32 bytes, base64 with padding>:Signature-Input: mac=("atproto-space-commit";sf);keyid="did:key:zDn...";tag="atproto-space-commit"Signature: mac=:<the 32 bytes of the MAC, base64 with padding>:-
The covered header is a Dictionary, serialized as RFC 9651 serializes one: for a commit, its fields in the order
ver(an Integer),repo,spaceandrev(Strings) anddata(a Byte Sequence of its 32 bytes); for a record,uri, its full space URI, andcid, its CID as a string, both Strings, as inuri="at://...", cid="bafyrei...". -
Signature-Input’s one member has the request’s label, and its value is the request’s Inner List, serialized as above. -
Signature’s one member has the same label, and its value is the MAC as a Byte Sequence: HMAC-SHA256, keyed with the shared secret of the key share and the author’s KEM key, of the signature base, whose one component line is the profile’s, as in"atproto-space-commit";sf: ver=1, repo="did:plc:...", space="at://...", rev="3mwcj3b6uak2f", data=:...:"@signature-params": ("atproto-space-commit";sf);keyid="did:key:zDn...";tag="atproto-space-commit"
Checking, as a syncer does: Signature-Input has exactly one member
tagged with the profile, an Inner List of exactly the one item above, with
keyid, if given, the key share asked with and alg, if given,
hmac-sha256 (other parameters are allowed: they are in the base);
Signature has a member of its label, a Byte Sequence of 32 bytes with no
parameters; the covered header, read as a Dictionary and serialized again
(the sf flag), gives the base; the MAC matches it; and the header names
what the answer holds: exactly the profile’s fields, none with parameters,
each of its type and equal to the answer’s.
Which methods take Accept-Signature, and the error they answer when it is
refused, are the space repo’s (a later part).
interop/spaces/mac.json: for each commit and record, the shared secret,
the request, the covered header, the signature base, the mac and the
response headers; and accept_signature, request headers each with the
profile it is read for and what it yields: the label, key share and
serialized Inner List, null where no MAC is asked, or "refused".
A space repo travels as a STAR archive, as spaces/star.md defines STAR:
the magic bytes 0x2A 0x6C 0x01 0x00; the metadata, a DRISL map with a
String $type, after its length; then each pair as 0x01, the key’s
length, the key, the value’s length and the value; then 0x00, and
nothing after. Lengths are unsigned varints, minimally encoded, at most 9
bytes. Any fault refuses the whole archive. interop/star/ has STAR’s
published fixtures: each valid one must parse to its metadata and pairs,
and each invalid one must be refused.
The profile atproto-space-repo is a space repo:
- The metadata is the repo’s commit, with
$type:{$type: "atproto-space-repo", ver: 1, repo, space, rev, data},dataas DRISL bytes. A writer writes these six fields; a reader ignores any others STAR allows. - A pair is a record: its key the UTF-8 bytes of
{collection}/{rkey}, its value the record’s DRISL bytes, those its CID names. Pairs are in strict ascending order of the key bytes, socom.example.x.y/1comes beforecom.example.x/10, as.is0x2Eand/is0x2F; an empty repo has none. So the archive of a commit and its records is one string of bytes. - Limits: the metadata is at most 65536 bytes, a key 4096, a value 1048576.
A reader, given the space and repo it asked for, the key share it asked with, the shared secret and the response’s headers, refuses the whole archive on any of these, and otherwise takes it as the repo:
- a fault of STAR’s, a limit passed, or metadata that is not canonical DRISL;
- a
$typeother thanatproto-space-repo, averother than1, or a commit field missing or of another type; - a
spaceorrepoother than those asked for; - a commit MAC that does not check (above) against the response’s
headers, or an
Atproto-Space-Commitheader that names another commit than the metadata; - a key that is not a record path (an NSID,
/, a record key), or one not after the key before it, duplicates included; - an LtHash digest of the records’ elements, each CID computed from the
value’s bytes, other than
data.
interop/spaces/repo-star.json: valid archives, each exactly the archive
of its commit and records, to serialize and to read; one a reader must
take though no writer here makes it; and archives to refuse, with their
space, repo, key share, secret and response headers.
Test vectors
Section titled “Test vectors”| File | Holds |
|---|---|
interop/spaces/kem.json | the shared secret |
interop/spaces/lthash.json | LtHash |
interop/spaces/mac.json | commit and record MACs, and Accept-Signature read |
interop/spaces/http-signature.json | a credential’s HTTP Message Signature |
interop/spaces/repo-star.json | space repos as STAR archives |
interop/star/*.json | STAR’s own fixtures, published with STAR |
Each file says in its description how to read its cases. The server’s
own tests (cargo test) hold each building block to every case of its
file. The spaces run holds every case to the reference’s own library and
to the harness’s code written from this page (below), so each vector is one
the reference and this page agree on; the one case the reference’s library
cannot take, kem.json’s second, whose info is not atproto-mac, is held
to the harness’s code alone.
How it is checked
Section titled “How it is checked”harness/spaces/compare.mjs is the spaces run. It starts the pinned
spaces alpha (harness/spaces/package.json; pin-alpha.sh pins another):
its PLC directory and its PDS, the reference here. It starts the server
beside it on that directory, in the environment The PDS: accounts and their records lists, with the
alpha’s recovery key as PDS_RECOVERY_DID_KEY, and as
PDS_LEXICON_AUTHORITY_DID the DID of the run’s lexicon authority, a
PDS of its own on that directory publishing space type declarations,
which the alpha resolves from too (OAuth: access tokens on every method, and what scopes allow, “How it is
checked”):
cd harness/spaces && npm ci # oncenode compare.mjs --seed 1 [--accounts 3] -- <command that starts the server>First it checks every vector, as above, and stops at the first that either side disagrees with. Then, on both sides, with each account’s DID named alike as The PDS: accounts and their records says:
- createAccount of a few accounts: the answers are compared, with keys,
tokens and URLs taken out. Each genesis operation in the directory must
be what Accounts says, with
verificationMethods.atproto_kema P-256did:keyof a point on the curve, other than the signing key; the answer’sdidDocmust be the directory’s document; and that document must have the#atproto_kemverification method, its key the operation’s. - getRecommendedDidCredentials of each: compared with keys and URLs
taken out; on each side
verificationMethods.atproto_kemmust be the genesis operation’s. - describeServer:
did,availableUserDomains,inviteCodeRequiredandspacesAlphacompared;spacesAlphamust betrueon each side. - updateHandle of most of them: the operation it posts must keep every
verificationMethodsentry, andgetRecommendedDidCredentialsstill name the same KEM key. - A restart: the server is killed and started again on its data, and
every account’s
getRecommendedDidCredentialsmust name the KEM key it had, with no new operation in the directory.
The reference is held to each check too, which checks the check. The other
runs, against the latest reference, hold what the KEM key changes to
this page instead (The PDS: accounts and their records).
That the server makes the shared secret, MACs, signatures and archives the vectors hold, over HTTP, against a syncer, each later part checks through its own methods, in this run.