Binary DB v0: Snapshot and Content Records

This chapter defines immutable content identity and its physical storage relationships. Ordered parent edges and normalized Tree ranges are authority, while explicitly named cache families remain rebuildable or optional as stated below.

Content Snapshot Records#

CONTENT_SNAPSHOT_RECORD_SIZE = 88

ContentSnapshotRecord — snapshot.bin:
u8  snapshot_meta
u8  history_flags
u16 payload_len
u64 payload_offset
u64 snapshot_hash48
u32 parent_snapshot_index_plus1
u32 root_tree_pack_index_plus1
u32 root_entry_ordinal
u32 line_index_plus1
u8  manifest_hash[32]
u32 file_count
u64 total_bytes
u64 created_at_s

The canonical public Snapshot ID is SNP- followed by twelve uppercase hex characters. snapshot_hash48 stores that suffix in its low 48 bits; its high 16 bits must be zero. Non-standard Snapshot IDs are not representable in this layout and must be rejected.

history_flags is fixed Snapshot schema, not payload. Bit 0 marks an imported remote-head history boundary: the source Snapshot had at least one parent, but the local import intentionally did not materialize ancestry before that head. Such a Snapshot is not a true root. Bits 1 through 7 are reserved and must be zero.

Ordered Snapshot Parent Records#

SNAPSHOT_PARENT_EDGE_RECORD_SIZE = 12

SnapshotParentEdgeRecord — snapshot_parent_edge.bin:
u32 child_snapshot_index
u32 parent_snapshot_index
u16 parent_ordinal
u16 flags

When ContentSnapshotRecord.snapshot_meta bit 4 is set, the ordered rows in snapshot_parent_edge.bin are the sole parent authority for that Snapshot. The fixed parent_snapshot_index_plus1 field is then only a verified cache of ordinal zero: it is zero for a true root or an imported remote-head history boundary and otherwise must equal the ordinal-zero edge's parent_snapshot_index + 1. Snapshot payload bytes never carry parent indexes.

An admitted parent set has at most 1,024 rows. A non-empty set starts at ordinal zero and uses contiguous ordinals without duplicates. Ordinal zero is the primary parent used by explicit first-parent traversal. A child may not reference itself, a missing Snapshot, a tombstoned Snapshot, or form a cycle. Every flags value is zero in v0. Parent order is immutable and participates in the Snapshot manifest identity.

A Snapshot with history_flags bit 0 set must also have parent_edges_authority set, parent_snapshot_index_plus1 = 0, and no parent edge rows. Parent and first-parent traversal stops at that Snapshot. Export, reachability, and ancestry proofs must preserve the boundary distinction and must not report the locally truncated graph as complete history. Setting the flag on a source root, on a Snapshot with any local parent, or without evidence that the source had a parent is corruption and fails closed.

A record with bit 4 clear uses the original fixed pointer as its complete zero-or-one-parent authority and has no parent-edge rows. This state exists only to admit a predecessor layout-1 generation as offline conversion input. New writes and newly activated generations use edge authority for every live Snapshot, including roots. Mixed authority for one Snapshot, a payload parent extension, non-contiguous ordinals, pointer/edge disagreement, or an edge for a bit-4-clear child is corruption and fails closed.

Snapshot authority is a DAG forest, not a single-root tree. An authority with zero live Snapshots is valid. Every live non-boundary Snapshot must reach one proven true root, but distinct components may reach distinct true roots. A predecessor bit-4-clear record whose complete fixed pointer is zero proves a true root; conversion writes edge authority with no parent rows and keeps history_flags clear. Neither record position, Line membership, component size, nor current-head status permits merging components or converting a true root into a remote-head history boundary.

Task Snapshot Link Records#

TASK_SNAPSHOT_INDEX_RECORD_SIZE = 8

TaskSnapshotIndexRecord — task_snapshot_index.bin:
u32 latest_snapshot_link_index_plus1
u16 snapshot_count
u8  next_snapshot_ordinal
u8  reserved0

CHANGE_SNAPSHOT_INDEX_RECORD_SIZE = 8

ChangeSnapshotIndexRecord — change_snapshot_index.bin:
u32 latest_snapshot_link_index_plus1
u16 snapshot_count
u16 reserved0

AUTHORITATIVE_SNAPSHOT_LINK_RECORD_SIZE = 40

LocalTaskSnapshotLinkRecord / ServerTaskSnapshotLinkRecord — snapshot_link.bin:
u8  link_meta
u8  snapshot_ordinal
u16 payload_len
u64 payload_offset
u32 task_index
u32 change_index_plus1
u32 content_snapshot_index
u32 previous_task_snapshot_link_index_plus1
u32 previous_change_snapshot_link_index_plus1
u64 created_at_s

REMOTE_MIRROR_SNAPSHOT_LINK_RECORD_SIZE = 52

RemoteMirrorTaskSnapshotLinkRecord — snapshot_link.bin:
u8  link_meta
u8  remote_meta
u16 reserved1
u8  snapshot_ordinal
u8  reserved0
u16 payload_len
u64 payload_offset
u32 task_index
u32 change_index_plus1
u32 content_snapshot_index
u32 previous_task_snapshot_link_index_plus1
u32 previous_change_snapshot_link_index_plus1
u64 created_at_s
u64 fetched_at_s

snapshot_ordinal uses the full u8 range. Values 0..255 render as S-01..S-256. A writer must reject the next link after ordinal 255 and must not wrap, clamp, reuse, or invent an extended Snapshot handle.

Tree Records#

TREE_PACK_RECORD_SIZE = 32

TreePackRecord — tree_pack.bin:
u8  pack_meta
u8  pack_format_kind
u16 pack_hash_hi16
u32 pack_hash_lo32
u32 first_tree_index
u32 tree_count
u64 total_bytes
u64 created_at_s

TREE_RECORD_SIZE = 20

TreeRecord — tree.bin:
u8  tree_meta
u8  reserved0
u32 pack_entry_ordinal
u32 entry_count
u8  tree_hash80[10]

TREE_ENTRY_RANGE_RECORD_SIZE = 4

TreeEntryRangeRecord — tree_entry_range.bin:
u32 first_entry_index

TREE_ENTRY_RECORD_SIZE = 16

TreeEntryRecord — tree_entry.bin:
u8  entry_meta
u8  name_len
u16 mode_bits
u64 name_offset
u32 target_index

Tree entry names are individual normalized UTF-8 path segments, not full paths, and are limited to 255 bytes. Blob entries target blob.bin; Tree entries target tree.bin.

TreeRecord.pack_entry_ordinal is the exact physical entry ordinal inside the Tree pack archive that supplies this Tree's raw payload. For a Tree Pack with sparse_physical_ordinals clear, every Tree in its logical range satisfies pack_entry_ordinal = tree_index - first_tree_index and the ordinal is less than tree_count. When the bit is set, the ordinal is instead the exact sparse physical archive locator and need not equal the logical offset or be less than the logical tree_count; archive bounds, decoded Tree ID, and decoded entry count are still verified.

TreeEntryRangeRecord is fixed schema, not payload. Its record ordinal is the tree_index, and at every stable activation boundary tree_entry_range.bin has exactly as many records as tree.bin. Together with the owning TreeRecord.entry_count, first_entry_index names that Tree's authoritative contiguous normalized rows in tree_entry.bin. For an empty Tree, the canonical first_entry_index is the current tree_entry.bin record count. The range must remain within tree_entry.bin; new writes append normalized entry rows in Tree order and never overlap a committed Tree's range.

The normalized range is authoritative for metadata traversal. The physical pack locator is authoritative for locating and verifying the archived raw Tree payload. Names, modes, target kinds, targets, Tree ID, and entry_count must agree across the normalized range and decoded pack entry; disagreement is corruption and fails closed.

Tree creation appends and fsyncs its normalized entry rows and range dependency before appending the Tree commit record. Recovery ignores or truncates orphan entry/range dependencies, rejects a missing range for a committed Tree, and validates every committed range and physical pack locator.

A predecessor layout-1 generation without tree_entry_range.bin is offline conversion input only. Conversion preserves the existing pack_entry_ordinal, reconstructs and verifies every normalized range from the pack payload, and materializes the complete side file before activation.

Blob And Object-Pack Records#

OBJECT_PACK_RECORD_SIZE = 32

ObjectPackRecord — object_pack.bin:
u8  pack_meta
u8  pack_format_kind
u16 pack_hash_hi16
u32 pack_hash_lo32
u32 first_member_index
u32 member_count
u64 total_bytes
u64 created_at_s

OBJECT_PACK_MEMBER_RECORD_SIZE = 16

ObjectPackMemberRecord — object_pack_member.bin:
u8  member_meta
u8  delta_chain_depth
u16 reserved0
u32 pack_index
u32 blob_index
u32 base_blob_index_plus1

BLOB_RECORD_SIZE = 64

BlobRecord — blob.bin:
u8  blob_meta
u8  hash_kind
u16 reserved0
u64 size_bytes
u32 pack_member_index_plus1
u64 created_at_s
u64 pruned_at_s
u8  sha256[32]

Blob SHA-256 is stored as 32 raw bytes. Public BLB-* identity renders from the first 80 bits. Object and Tree pack records store their 48-bit public-ID suffix as (pack_hash_hi16, pack_hash_lo32).

For admitted legacy conversion, multiple source Blob rows with the same full 32-byte SHA-256 collapse to one target Blob only when blob_meta, normalized hash_kind, reserved0, size_bytes, and pruned_at_s agree. Distinct pack-member pointers are physical copies of the same content and every member, base-Blob, Tree-entry, and rebuilt-index reference is densely remapped. The target created_at_s is the minimum valid non-zero source creation time. A zero time, metadata disagreement, size disagreement, prune disagreement, hash prefix collision with different complete SHA-256 bytes, or byte/hash mismatch fails closed. This normalization adds no Blob alias row or mapping file.

Optional Manifest And File Cache#

These records are disposable local acceleration and are never authoritative:

SNAPSHOT_FILE_INDEX_RECORD_SIZE = 4

SnapshotFileIndexRecord — snapshot_file_index.bin:
u32 first_file_index_plus1

SNAPSHOT_FILE_RECORD_SIZE = 24

SnapshotFileRecord — snapshot_file.bin:
u64 path_hash64
u64 path_offset
u32 tree_entry_index
u16 path_len
u16 reserved0

The authoritative path remains snapshot.bin -> tree_pack.bin/tree.bin/ tree_entry.bin -> blob.bin. Readers must remain correct when every optional manifest/file cache is absent.