Advanced

Systems, Composition, and Evolution

Audience: system publishers and tool authors. Collection authors can skip this page until they want to share or compose systems.

Authoritative for:

the system fields of typedmark.md: release version, scaffold, and publishing metadata
the marketplace catalog, system composition, and canonical serialization
<metadata_directory>/history.md, system versioning, and the migration and update flow

Systems#

A system is a reusable, versioned, publishable collection model. It is the unit people share and instantiate: a personal-knowledge system, a dev-team AI-context system, a marketing-agency system, a PARA system, and so on. Where the core specification defines how to define note types, a system defines which note types exist and how a collection of that kind is configured and scaffolded.

A system is therefore the domain layer of TypedMark. It carries domain note types, house conventions, templates, and starter content on top of the domain-agnostic core.

There is no separate system manifest. Every collection already declares its identity (name, an optional label, and description) and structural fields in typedmark.md; a collection is a system when it additionally declares the system fields defined on this page, which describe how it is versioned, published, and scaffolded. A private working collection omits them; a publishable system declares them. Publishing a system is therefore nothing more than populating those fields and sharing the folder that contains typedmark.md and the metadata directory.

Rules:

SCE-1 The core specification defines the file layout, schema shapes, field semantics, validation model, composition semantics, and conformance rules.
SCE-2 A system applies that model to a concrete domain by shipping note-type schemas, property sets, templates, scaffold content, and validation defaults.
SCE-3 A system MAY establish house conventions such as preferred note types, folder conventions, and validation strictness through the artifacts it ships.
SCE-4 A system MUST NOT relax a core MUST requirement.
SCE-5 System-specific conformance is evaluated in addition to, not instead of, the core conformance rules in Conformance and Roadmap.
SCE-6 A system MAY be composed from other systems, as defined under System Composition below.

System Fields#

The system fields are the part of typedmark.md that makes a collection a publishable, versioned system. They live alongside the structural and identity fields defined in Collection Model. Every collection already declares name, description, and an optional label; a system additionally declares version and scaffold, and MAY add discovery metadata.

Example typedmark.md for a publishable system, showing the system fields together with the identity and structural fields they accompany:

specification_version: 0.0.1
name: "@example/knowledge-system"
label: Example Knowledge System
description: Reusable knowledge note system.
version: 0.2.0
keywords:
  - notes
  - reference

metadata_directory: .typedmark
exclude_paths:
  - .git/**
validation_defaults:
  path: error
default_property_sets:
  - base

audiences:
  - individual
  - team
publisher:
  name: Example Publisher
license: MIT
scaffold:
  folders:
    - Domains
    - Topics
    - Sources
  notes:
    - path: "Home.md"
      note_type: home
      from_template: "home.md"
      values:
        note_type: home
    - path: "Glossary.md"
      note_type: glossary
      from_template: "glossary.md"
      values:
        note_type: glossary

System fields:

version
scaffold
audiences, publisher, and license, which are optional discovery metadata

The name, label, description, and keywords fields are defined in Collection Model. A marketplace reuses name as the system's identity, label as its display name, description as its summary, and keywords for search.

Rules:

SCE-7 A collection is a system definition when its typedmark.md declares version and scaffold. Conformance requirements for a valid system definition are defined in Conformance and Roadmap.
SCE-8 version is the single marker that a collection is a publishable, versioned system; a collection without version is a working collection, not a system.
SCE-9 name, the collection identity defined in Collection Model, is also the distribution identity a marketplace and composition.sources resolve against. A collection has exactly one identity.
SCE-10 version identifies a publishable system release, MUST be a Semantic Versioning 2.0.0 string, and MUST follow the system versioning semantics defined under System Versioning below.
SCE-11 A collection that declares version MUST also declare scaffold.
SCE-12 scaffold SHOULD be present, even if empty, on a system definition.
SCE-13 scaffold.folders lists folders an importer SHOULD create when instantiating a collection from the system.
SCE-14 scaffold.notes lists note files an importer SHOULD create when instantiating a collection from the system.
SCE-15 Each scaffold note entry MUST define path, note_type, and from_template.
SCE-16 scaffold.notes[].from_template follows the same resolution rule as a schema's template.file defined in Note Type Schemas: it is a relative path resolved against <metadata_directory>/templates/, MUST NOT restate the metadata directory or the templates/ folder, and MUST end in .md.
SCE-17 scaffold.notes[].note_type MUST resolve to exactly one concrete note type.
SCE-18 scaffold.notes[].values MAY provide initial frontmatter values that are merged into the instantiated template.
SCE-19 Values supplied in scaffold.notes[].values override template placeholder values for that instantiated note only.
SCE-20 scaffold.notes[].path and scaffold.folders entries MUST be collection-relative, MUST use forward slashes, and MUST NOT escape the collection root with absolute paths or .. segments.
SCE-21 The system fields govern packaging, publishing, composition, and import only; they MUST NOT restate or override structural rules defined by the structural fields, note-type schemas, or property sets.

Publishing and Catalog#

Rules:

SCE-22 The canonical published form of a system is the folder that contains typedmark.md and the metadata directory selected by typedmark.md, including its governed internal layout.
SCE-23 A system MAY be shared as a directory tree, a Git repository, or an archive file, provided relative paths are preserved.
SCE-24 A marketplace entry SHOULD be keyed by name and version.
SCE-25 name, label, description, keywords, audiences, publisher, and license are the primary discovery fields for catalogs and application marketplaces.
SCE-26 Applications MAY present curated systems for different audiences such as individuals, teams, and organizations.
SCE-27 Marketplace or catalog implementations SHOULD be able to index a system from its typedmark.md alone.

Marketplace Catalog#

A marketplace publishes the systems it knows in a machine-readable catalog, so tools and websites can browse systems, download them, and compose them into collections.

Example marketplace.json:

{
  "specification_version": "0.0.1",
  "label": "TypedMark Systems Marketplace",
  "description": "Community catalog of reusable TypedMark systems.",
  "systems": [
    {
      "name": "@acme/para-system",
      "version": "1.2.0",
      "label": "PARA System",
      "description": "PARA-style organization system.",
      "keywords": ["para", "organization"],
      "source": {
        "path": "systems/para-system"
      }
    },
    {
      "name": "dev-team-ai-context",
      "version": "0.3.0",
      "description": "AI context system for development teams.",
      "source": {
        "repository": "https://github.com/example/dev-team-ai-context",
        "ref": "v0.3.0"
      }
    }
  ]
}

Rules:

SCE-28 A marketplace catalog is a JSON document named marketplace.json at the root of the marketplace repository.
SCE-29 The catalog is machine-facing distribution metadata, not collection content; it is plain JSON rather than a Markdown artifact with frontmatter.
SCE-30 The catalog MUST contain specification_version and systems, and MAY contain label and description describing the marketplace itself.
SCE-31 systems MUST be a list of catalog entries; each entry describes exactly one system release.
SCE-32 Each catalog entry MUST declare name, version, and source.
SCE-33 An entry's name follows the collection identity rules defined in Collection Model, and its version MUST be a Semantic Versioning 2.0.0 string.
SCE-34 The pair of name and version MUST be unique within one catalog.
SCE-35 An entry MAY restate the system's discovery metadata — label, description, keywords, audiences, publisher, and license; when present, these SHOULD equal the values in the system's typedmark.md, which remains authoritative.
SCE-36 source declares where the system lives and MUST take exactly one of two forms:
- the path form: path names the folder containing the system's typedmark.md, relative to the catalog's own repository root
- the repository form: repository is an absolute URL of the repository hosting the system, with an optional path to the system folder inside that repository, defaulting to the repository root, and an optional ref naming a tag, branch, or commit
SCE-37 When ref is omitted, the hosting repository's default state is used; publishers SHOULD pin a tag or commit so resolution stays reproducible.
SCE-38 A source MUST lead to a folder containing a typedmark.md whose name and version equal the entry's name and version.
SCE-39 Composition source resolution MAY use one or more marketplace catalogs to resolve composition.sources entries; a source that resolves through a catalog to a mismatching system is an invalid_composition failure.
SCE-40 A marketplace SHOULD validate every listed system as a valid system definition before listing it.
SCE-41 The machine-readable JSON Schema for the catalog is published in the specification repository under schema/json-schema/.

Importing and Instantiating a System#

Instantiating a system creates a working collection from it. The collection records which systems it came from in typedmark.md composition, defined in Collection Model.

Rules:

SCE-42 An importer MUST validate typedmark.md, any property set files under <metadata_directory>/property-sets/, the note-type schema files, and <metadata_directory>/history.md when present, before creating a collection from the system.
SCE-43 An importer MUST preserve typedmark.md and the configured metadata directory structure and file contents unless the user explicitly requests a transformation.
SCE-44 An importer SHOULD support a full scaffolded import mode that creates the declared folders and notes.
SCE-45 An importer MAY additionally support a metadata-only import mode that installs typedmark.md and the configured metadata directory without materializing scaffold notes.
SCE-46 When instantiating a note from a template, the importer MUST emit frontmatter that conforms to Managed Notes and Properties.
SCE-47 An importer that instantiates a collection from one or more systems MUST record those systems and their resolved versions in typedmark.md composition, so the result can be reproduced and later updated.
SCE-48 A full scaffolded import yields a usable collection, even if many instantiated notes still contain placeholder or null values.
SCE-49 A metadata-only import yields an installed structural model and does not by itself require the target collection root to contain any instantiated notes.

System Composition#

A collection or a system MAY be built by composing other systems. Composition is how a marketplace and a CLI let a user stack several systems together — for example combining a PARA system, a personal-knowledge system, and a dev-team AI-context system into one working collection, or publishing that combination as a new system.

Composition is a build-time operation. A composing tool loads the source systems at their declared versions, merges them deterministically, and materializes one self-contained collection in which every schema, property set, and template is physically present in the metadata directory. The composed collection then conforms exactly like any hand-authored collection, and remains understandable from typedmark.md and the metadata directory alone.

The ordered list of source systems and their resolved versions is the collection's composition lineage, recorded in typedmark.md composition, defined in Collection Model. The lineage is both provenance and the reproducible recipe: re-running composition over the same sources at the same versions reconstructs the same result.

Rules:

SCE-50 Composition resolves each source identity and version to exactly one system whose name and version match.
SCE-51 A composing tool MUST materialize the merged note-type schemas, property sets, and templates into the target metadata directory so the composed collection is self-contained.
SCE-52 A composing tool MUST record each source name and resolved version in typedmark.md composition.sources, in composition order.
SCE-53 A composing tool MUST NOT require network access, the source systems, or the composition tool itself to evaluate the conformance of an already-composed collection.
SCE-54 Composition MUST be deterministic, as defined under Composition Merge Semantics below.

Composition Merge Semantics#

The inputs to composition are the ordered source systems from composition.sources plus the local artifacts authored directly in the target collection. The merge is a deterministic ordered fold.

Rules:

SCE-55 Sources are merged in composition.sources order; the local collection's own artifacts are applied last.
SCE-56 Note-type schemas merge by note_type; property sets merge by property_set; templates merge by template path; scaffold.folders merge by set union; scaffold.notes merge by path.
SCE-57 When two inputs contribute the same keyed artifact, the later input in the merge order replaces the earlier one completely, and the local collection's artifact overrides every source.
SCE-58 typedmark.md metadata_directory MUST be identical across all sources and the target; a mismatch is a composition error.
SCE-59 typedmark.md default_property_sets merge by concatenation in merge order with duplicate identifiers removed, keeping the first occurrence.
SCE-60 A source's default_property_sets are scoped to that source: they apply only to the note types whose winning schema was contributed by that source, so one source's defaults MUST NOT silently change the effective schemas of another source's note types.
SCE-61 A composing tool MUST materialize that scoping using the existing composition constructs: for each concrete note type in the composed result, every merged default property set that is named neither in the defaults of the source that contributed the winning schema for that note type nor in the target collection's own default_property_sets MUST be added to that note type's exclude_property_sets, preserving any exclusions the schema already declares.
SCE-62 The target collection's own default_property_sets apply to every concrete note type in the composed result, including note types contributed by sources, unless a note type excludes them explicitly.
SCE-63 A composing tool MUST report each scoping exclusion it materializes, identifying the note type, the excluded property set, and the sources involved.
SCE-64 Relationship target note types referenced by a source's property sets and note-type schemas are resolved against the composed result, not against the source in isolation.
SCE-65 typedmark.md note_type_mappings merge by concatenation in merge order; because mapping rules are evaluated in order, earlier sources' rules are evaluated before later sources' rules unless the target overrides them.
SCE-66 typedmark.md validation_defaults, exclude_paths, and vocabularies merge by key, with later inputs overriding earlier inputs per key, and the target overriding all.
SCE-67 The composing collection's own name, version, and other system fields are authored on the result; they are never inherited from a source.
SCE-68 A composing tool MUST report every collision it resolves, identifying the artifact, the contributing sources, and the winner.
SCE-69 history.md from each source MAY be retained for update reasoning, as defined under Migration and Updates; composition itself does not require merging source histories into a single log.

Reproducibility and Canonical Form#

Rules:

SCE-70 Two conforming composing tools given identical composition.sources at identical versions MUST produce the same composed schemas, property sets, and templates under canonical comparison.
SCE-71 Canonical comparison uses the canonical field materialization rules in Managed Notes and Properties for frontmatter, and the deterministic merge order defined above for every ordered construct, including effective field order, property_sets order, and composition.sources order.
SCE-72 A composing tool that claims reproducible or hash-stable output MUST serialize composed artifacts in the canonical serialization defined below; two artifacts that are equal under canonical comparison then have byte-identical canonical serializations.
SCE-73 The local-only contribution of a composed collection is, by definition, the difference between its current materialized state and the state obtained by recomposing its composition.sources; a tool MUST be able to recover it by recomposition rather than by reading per-artifact origin tags.

Canonical Serialization#

The canonical serialization of a governed artifact's frontmatter is the byte sequence produced by the rules below. It exists so that composition and migration produce reproducible, hash-stable output. A hand-authored governed artifact need not be in canonical form to be valid; canonical serialization is the form a tool produces when it materializes composed artifacts or claims reproducible output.

A canonically serialized governed artifact file consists of one line containing ---, the canonical YAML serialization of its frontmatter, one line containing ---, and then the artifact body unchanged. The body is not part of canonical comparison: when composition replaces a keyed artifact, the winning input's body is carried unchanged, and two artifacts are canonically equal when their frontmatter is.

Encoding and layout:

SCE-74 The output MUST be UTF-8 without a byte-order mark.
SCE-75 Line endings MUST be a single LINE FEED (U+000A); carriage returns MUST NOT appear.
SCE-76 The output MUST end with exactly one LINE FEED, and no line may contain trailing whitespace.
SCE-77 Mappings and sequences MUST use block style; flow style, anchors, aliases, explicit tags, comments, and directives MUST NOT appear.
SCE-78 Indentation MUST be two spaces per nesting level.

Key and element order:

SCE-79 Order-significant mappings preserve their defined order; every other mapping serializes its keys sorted ascending by Unicode code point.
SCE-80 The frontmatter mapping and every object.fields mapping are order-significant and MUST preserve the effective field order defined by the merge rules.
SCE-81 property_sets, default_property_sets, composition.sources, history, and every changes list are sequences and MUST preserve their defined order.
SCE-82 Every other mapping, including a field definition's property keys and the storage, relationships, headings, and typedmark.md top-level mappings, MUST serialize its keys in ascending Unicode code-point order.

Scalars:

SCE-83 Null MUST serialize as null.
SCE-84 Booleans MUST serialize as true or false.
SCE-85 Integers MUST serialize in base ten, with no leading zeros, no leading +, and a leading - only for negative values.
SCE-86 Numbers MUST serialize as the shortest base-ten decimal that round-trips to the stored value, using . for any fractional part and a lowercase e for any exponent; negative zero MUST serialize as 0.
SCE-87 Strings MUST serialize in plain style when the value is plain-safe, and otherwise in double-quoted style with \ escapes.
SCE-88 A string is plain-safe when it is non-empty, does not begin with a YAML indicator character, contains no control characters or characters that require escaping, and does not match the canonical serialization of a null, boolean, integer, or number.
SCE-89 Each materialized field value MUST also satisfy the Canonical Field Materialization rules in Managed Notes and Properties.

System Versioning#

A system's version communicates the intent of a release to humans and catalogs. It does not, by itself, tell a tool what a given upgrade will do to a collection.

Change classes:

patch: non-structural changes only, such as edits to label, description, icon, or guidance.
minor: additive, backward-compatible structural changes, such as a new note type, a new property set, or a new optional field.
major: breaking structural changes, such as removing or renaming a note type or field, changing a field type, tightening a constraint, or changing storage rules.

Rules:

SCE-90 version MUST be a Semantic Versioning 2.0.0 string.
SCE-91 A system release SHOULD select its version change class according to the most impactful change it introduces, using the change classes above.
SCE-92 The version number conveys intent only. A tool MUST NOT infer the concrete migration impact of an upgrade from the version number alone.
SCE-93 A tool that evaluates an upgrade MUST compute the actual impact from the new system's typedmark.md, its governed artifacts, and its history.md, as defined under Migration and Updates, and MUST treat the version number as advisory.

Change History#

<metadata_directory>/history.md is an append-only, event-sourced log of the structural changes a system has undergone. Replaying its events in order reconstructs how the current schema state came to be. It is the authoritative record of what changed between versions, including changes such as field renames that cannot be inferred from a structural comparison alone.

History example:

specification_version: 0.0.1
history:
  - version: 0.1.0
    changes:
      - op: add_note_type
        note_type: topic
      - op: add_field
        note_type: topic
        field: title
      - op: add_field
        note_type: topic
        field: summary
  - version: 0.2.0
    changes:
      - op: rename_field
        note_type: topic
        from: summary
        to: overview
      - op: add_field
        note_type: topic
        field: status

Defined change operations:

add_note_type with note_type
remove_note_type with note_type
rename_note_type with from and to
add_field with note_type or property_set, and field
remove_field with note_type or property_set, and field
rename_field with note_type or property_set, from, and to
retype_field with note_type or property_set, field, from_type, and to_type
change_field with note_type or property_set, and field, for constraint or metadata changes that are neither a rename nor a retype
change_storage with note_type, for changes to the effective storage block
change_template with note_type, for changes to the effective template reference or to the canonical template content
change_headings with note_type, for changes to the effective headings block
change_relationships with note_type, for changes to the effective relationships block
change_note_type with note_type, for note-type-level changes that no more specific operation covers, such as changes to kind, extends, property_sets, exclude_property_sets, frontmatter_remove, or guidance
change_collection, for changes to the structural fields of typedmark.md, such as note_type_mappings, default_property_sets, exclude_paths, or validation_defaults
add_property_set with property_set
remove_property_set with property_set
rename_property_set with from and to

Rules:

SCE-94 <metadata_directory>/history.md MAY be omitted by a system that publishes no change history.
SCE-95 If present, history.md MUST physically contain specification_version and history.
SCE-96 history MUST be an ordered list of release entries.
SCE-97 Each release entry MUST declare version and changes.
SCE-98 Each release entry's version MUST be a Semantic Versioning 2.0.0 string.
SCE-99 Release entries MUST appear in ascending version order, and each version MUST be unique within history.
SCE-100 The last release entry's version MUST equal the collection's version in typedmark.md when the collection declares one.
SCE-101 Each entry in changes MUST declare op using one of the defined change operations.
SCE-102 A field operation MUST declare exactly one of note_type or property_set, naming the artifact that physically defines the field: note_type when the field is defined in a note-type schema, property_set when it is defined in a property set.
SCE-103 A field operand MAY be a dotted path to address a nested field inside an object.fields mapping.
SCE-104 A change_* operation records that the named block or artifact changed; it does not restate the new value, which lives in the governed artifacts themselves.
SCE-105 Every structural difference between two consecutive releases MUST be recorded by at least one change operation; a structural change that no operation records is a divergence between history.md and the current schemas.
SCE-106 Replaying history from the first entry to the last, applying each changes list in order, MUST reconstruct the system's current inventory of note types, property sets, and fields, and MUST account for every structural change between consecutive releases; the governed artifacts remain authoritative for the concrete content of each block.
SCE-107 A validator MAY check this reconstruction invariant and report a divergence between history.md and the current schemas.
SCE-108 When cutting a new release, a tool SHOULD generate candidate changes by comparing the previous version's state to the current state, and the author MUST confirm or correct any change that a structural comparison cannot classify unambiguously, in particular distinguishing a rename_field from a paired remove_field and add_field.

Migration and Updates#

Updating a collection to newer versions of one or more of its source systems re-resolves the lineage, recomposes the collection, and migrates existing notes to match the new effective schemas.

Because a structural comparison cannot distinguish a rename from a removal paired with an addition, migration does not trust the version number and does not rely on a structural diff alone. It computes the concrete impact on the specific target collection by examining the new system together with the change history that produced it.

The update flow:

SCE-109 Read the lineage from typedmark.md composition.sources.
SCE-110 Resolve the new target version for each source system being updated.
SCE-111 Recompose the collection deterministically at the new versions to obtain the new effective schemas.
SCE-112 Compute the structural impact on the specific target by comparing the collection's current effective schemas to the recomposed schemas.
SCE-113 Reconcile that impact with the changes recorded in each updated source's history.md between the collection's current source version and the target source version, so that renames and retypes are classified correctly rather than treated as drops and adds.
SCE-114 Produce an ordered migration plan of managed-note operations from the reconciled change set.
SCE-115 Apply the migration plan to the collection's managed notes, then update composition.sources to the new versions.

Rules:

SCE-116 A tool MUST recompute migration impact against the actual target collection, because local overrides recorded on top of the lineage MAY change which source changes are relevant.
SCE-117 A tool MUST classify field renames and retypes using the source history.md change operations rather than inferring them from a structural comparison.
SCE-118 The managed-note effect of each change operation is defined in Migration Effects.
SCE-119 After a migration completes, the collection MUST conform to the recomposed effective schemas, and typedmark.md composition.sources MUST record the new resolved versions.
SCE-120 A tool MUST NOT silently discard managed-note data; a migration step that cannot preserve data, such as an unclassifiable retype_field, MUST be reported for explicit resolution rather than applied destructively.