Versions API

Versions are the core of Underlay. Each version is an immutable snapshot of a collection: schema + records + file references. Pushing a new version uses the negotiate protocol, a three-step flow similar to git's pack negotiation.


Push Protocol (Negotiate → Records → Commit)

All pushes use the negotiate protocol. You send a manifest of record hashes; the server tells you which ones it needs; you send only those records; then you commit. For collections where most records are unchanged between versions, only a few records are transferred.

Step 1: POST /api/collections/:owner/:slug/versions/negotiate

Auth: write scope

Start a negotiate session. Send your full manifest of record hashes plus schemas. The server checks which record and file hashes it already has and returns what it still needs.

Request

{
  "base_version": "v1.0.0",
  "schemas": {
    "Publication": {
      "type": "object",
      "properties": {
        "title": {"type": "string"}
      }
    }
  },
  "manifest": [
    {"id": "pub-001", "type": "Publication", "hash": "abc123..."},
    {"id": "pub-002", "type": "Publication", "hash": "def456..."},
    {"id": "pub-003", "type": "Publication", "hash": "789abc..."}
  ],
  "files": ["7a8b9c..."],
  "message": "Add new publications",
  "metadata": {
    "description": "PubPub archive"
  }
}

Fields

base_versionRequired. The semver this push is based on (e.g. "v1.0.0"). Use null for the first version. If the current version doesn't match, returns 409 Conflict.
schemasRequired. Per-type JSON Schema map (e.g. {"TypeName": {schema}}).
manifestRequired. Array of {"id", "type", "hash"} objects. Each hash is the SHA-256 of the canonical JSON {"id":...,"type":...,"data":...}.
filesArray of file hashes (SHA-256 hex strings) referenced by records.
messageHuman-readable commit message.
metadataOptional object with version metadata (description, readme, license, etc.). Merged with the previous version's metadata.
strip_unknown_fieldsIf true, the server strips fields not defined in the schema instead of rejecting the push.

Response 200

{
  "session_id": "uuid",
  "needed_records": ["def456...", "789abc..."],
  "needed_files": [],
  "total_records": 3,
  "total_files": 1,
  "already_have_records": 1,
  "already_have_files": 1
}

Step 2: POST .../negotiate/:sessionId/records

Auth: write scope

Send needed records as a JSONL body (Content-Type: application/x-ndjson). Each line is one JSON record. Only send records whose hashes appear in needed_records from the negotiate response.

Call this endpoint multiple times to send records in batches (up to 10,000 per request). The server tracks which records have been received. If needed_records was empty, skip this step and go directly to commit.

Request

{"id":"pub-002","type":"Publication","data":{"title":"New Paper"}}
{"id":"pub-003","type":"Publication","data":{"title":"Another Paper"}}

Response 200

{
  "received": 2,
  "remaining": 0,
  "total_needed": 2
}

When remaining reaches 0, all needed records have been received and you can commit.

Step 3: POST .../negotiate/:sessionId/commit

Auth: write scope

Finalize the push. The server validates all records against schemas, computes version hashes, and creates the new immutable version. No request body is needed.

Response 201

{
  "semver": "v1.1.0",
  "hash": "a1b2c3d4...",
  "recordCount": 3,
  "fileCount": 1
}

Schema privacy

You can add "private": true at two levels in the schema:

  • Type-level: Add "private": true to a type definition to hide all records of that type from public readers.
  • Field-level: Add "private": true to a field definition to strip that field from records returned to public readers.
"schemas": {
  "Article": {
    "type": "object",
    "properties": {
      "title": {"type": "string"},
      "body": {"type": "string"},
      "internalScore": {"type": "number", "private": true}
    }
  },
  "InternalNote": {
    "type": "object",
    "private": true,
    "properties": {
      "note": {"type": "string"}
    }
  }
}

Session management

GET .../negotiate/:sessionIdCheck session status. Returns remaining needed records and files.
DELETE .../negotiate/:sessionIdCancel a session. Returns 204.

Errors

400Unexpected record hash. A submitted record doesn't match any needed hash, or the batch is empty/malformed.
404Session expired or not found. Sessions expire after 10 minutes.
409Version conflict. Someone pushed since your base_version. Re-negotiate.
422Schema validation failed, missing files, or records contain extra fields not defined in the schema.

GET /api/collections/:owner/:slug/versions

No auth for public collections

List versions, newest first.

Query parameters

limitMax results (default 50, max 100)
offsetPagination offset

Response 200

[
  {
    "semver": "v1.1.0",
    "hash": "a1b2c3d4...",
    "message": "Add new publications",
    "appId": "pubpub-sync",
    "actorId": "user-42",
    "recordCount": 150,
    "fileCount": 12,
    "totalBytes": 52428800,
    "createdAt": "2026-04-01T00:00:00.000Z"
  }
]

GET /api/collections/:owner/:slug/versions/latest

No auth for public collections

Get the most recent version. Returns the full version object.


GET /api/collections/:owner/:slug/versions/:n

No auth for public collections

Get a specific version by semver (e.g. v1.1.0). Returns the full version object including schemas.


GET /api/collections/:owner/:slug/versions/:n/records

No auth for public collections

Get records for a specific version. Supports cursor-based pagination for efficient traversal of large collections.

Query parameters

typeFilter by record type
limitMax results (default 100, max 1000)
afterCursor: return records with IDs after this value (preferred for large sets)
offsetLegacy offset-based pagination (still supported)

Response 200

{
  "records": [
    {
      "id": "pub-001",
      "type": "Publication",
      "data": {
        "title": "Example Paper",
        "doi": "10.1234/example"
      }
    }
  ],
  "pagination": {
    "limit": 100,
    "hasMore": true,
    "nextCursor": "pub-002",
    "total": 150
  }
}

Use pagination.nextCursor as the after parameter in the next request. When hasMore is false, you've reached the end.


GET /api/collections/:owner/:slug/versions/:n/manifest

No auth for public collections

Get the manifest: a lightweight summary of what's in a version without the full record data.

Response 200

{
  "semver": "v1.1.0",
  "hash": "a1b2c3d4...",
  "schemas": {"Publication": "sha256:abc123..."},
  "records": [
    {"id": "pub-001", "type": "Publication", "hash": "sha256:def456..."},
    {"id": "pub-002", "type": "Publication", "hash": "sha256:789abc..."}
  ],
  "files": ["sha256:a1b2c3...", "sha256:d4e5f6..."]
}

GET /api/collections/:owner/:slug/versions/:n/diff

No auth for public collections

Diff two versions. By default compares version :n against the previous version.

Query parameters

fromSemver to diff from (e.g. v1.0.0). Default: previous version.

Response 200

{
  "from": "v1.0.0",
  "to": "v1.1.0",
  "added": [
    {"id": "pub-003", "type": "Publication", "data": {...}}
  ],
  "updated": [
    {"id": "pub-001", "type": "Publication", "data": {...}}
  ],
  "removed": ["pub-old"]
}