Stage ordering and lowering - Flow and algorithm

Platform spec article

Stage ordering and lowering - Flow and algorithm

Back to Stage ordering and lowering ◎ Open feature in platform map

Spec standingStandard

Owner

Piotr Mikstacki pmikstacki@cybernomad.it

Submitter

Piotr Mikstacki pmikstacki@cybernomad.it

Current document

ADRs (5)

1. 0001-feature-hub-authority Feature hub authority Accepted

   This feature hub owns normative MUST/SHOULD contract text for Stage ordering and lowering.

   Context

   Sibling articles under this feature previously restated requirements in inconsistent forms.

   Decision

   This feature hub owns normative MUST/SHOULD contract text. Sibling articles must not redefine hub requirements and should link here for authority.

   Consequences

   Contract changes start on the hub or in linked ADRs, then propagate to articles and implementation anchors.

   Verification anchors

   • site/website/src/content/docs/platform-spec/compiler/build-pipeline/stage-ordering/index.mdx
   • article bundle under the same feature directory.

   Open full ADR page

2. 0002-spec-over-implementation-notes Specification over implementation notes Accepted

   Platform-spec text supersedes informal crate comments for Stage ordering and lowering.

   Context

   Implementation crates accumulated informal notes that diverged from published contracts.

   Decision

   Normative platform-spec prose and ADRs under this feature supersede informal comments in implementation crates until explicitly migrated into spec text.

   Consequences

   Engineers file spec/ADR updates when behavior changes; crate comments are non-authoritative for conformance arguments.

   Verification anchors

   • compiler/crates/beskid_analysis/

   Open full ADR page

3. 0003-canonical-phase-dag Canonical parse-to-lowering phase DAG Accepted

   Pipeline ordering was fragmented across crates before `beskid_pipeline` phase ids.

   Context

   Reference compiler must emit stable beskid_pipeline phase events from parse through lower.ready without reordering semantic gates relative to mod boundaries.

   Decision

   Phase literals are defined in compiler/crates/beskid_pipeline/src/phases.rs and asserted by conformance tests; CLI/LSP observers rely on the same ids.

   Consequences

   Reordering phases requires an ADR and registry updates.

   Verification anchors

   • compiler/crates/beskid_pipeline/
   • compiler/crates/beskid_analysis/src/services/.

   Open full ADR page

4. 0004-mod-rewrite-before-semantic-gate Mod rewrite before semantic gate Accepted

   MOD_REWRITE must complete before SEMANTIC and LOWER in every host path.

   Context

   Analyze-only paths ran staged semantic rules and type-check gates on pre-rewrite AST, then discarded mod analyze rewrite output. Execute paths applied rewrite first—producing different diagnostics and HIR for the same source.

   Decision

   In all hosts (CLI gate, LSP, prepare_compilation, codegen front-end):

   1. Mod collect → generate → rewrite (MOD_REWRITE / run_analyze_rewrite) must finish before SEMANTIC staged rules.
   2. Semantic snapshot and typed HIR lowering must observe the post-rewrite entry program.
   3. Running rewrite after semantic/typecheck, or discarding rewrite in analyze-only mode, is forbidden.

   Phase ids in beskid_pipeline::phases must reflect this ordering for observers and conformance tests.

   Consequences

   flow-and-algorithm mermaid and phase DAG tests align analyze with run/build. Mod host integration changes require ADR updates if reordering is proposed.

   Verification anchors

   • compiler/crates/beskid_pipeline/src/phases.rs
   • compiler/crates/beskid_analysis/src/mod_host/
   • compiler/crates/beskid_analysis/src/services/analyze.rs
   • compiler/crates/beskid_analysis/src/services/front_end.rs

   Open full ADR page

5. 0005-salsa-database-incremental-memoization Salsa database owns incremental analysis memoization Accepted

   BeskidDatabase in beskid_queries is the host for incremental compilation memoization across CLI, LSP, and tests.

   Context

   Within-run deduplication and ad-hoc JSON unit caches did not provide dependency-tracked invalidation. LSP and CLI paths recomputed assembly and typing independently; session caches could return stale assemblies after edits.

   Decision

   1. beskid_queries::BeskidDatabase is the single incremental host for analysis memoization (Salsa 0.26 inputs + tracked queries + on-disk persistence under {project}/obj/beskid/cache/salsa/).
   2. CLI, LSP, and tests route prepare/assembly through beskid_queries entry points—CLI via process-scoped session in beskid_queries::session, LSP via workspace State.compilation_db, tests via local or session handles—no second analysis spine.
   3. Unit materialization during program assembly uses Salsa-backed unit queries (parse_and_expand_unit_tracked, unit_hir_tracked, unit_imports); legacy obj/beskid/cache/units/ JSON records are retired in favor of obj/beskid/cache/salsa/units/{content_fp}/ (see unit artifact cache schema).
   4. Pipeline observers emit salsa.query_hit, salsa.query_miss, salsa.revision_bump, salsa.artifact_disk_hit, and salsa.artifact_disk_miss counters for verification.

   Consequences

   • File text edits invalidate only affected units and dependents via import edges.
   • LSP didChange sets file_text inputs; diagnostics and IDE snapshots share the same database.
   • Mod-host invalidation keys (syntax_generation_id, manifest_generation_id, capability_set_id) map to Salsa inputs in beskid_queries::modhost.

   Status

   Accepted — implemented in compiler/crates/beskid_queries/ and compiler/crates/beskid_artifacts/. Legacy units/ JSON cache is retired; disk persistence uses content-addressed salsa/units/ snapshots.

   Open full ADR page

Articles

• Stage ordering and lowering - Contracts and edge cases Required ordering guarantees and behavior under parse or semantic failures. article Standard
• Stage ordering and lowering - Design model Pipeline stage model and artifact boundaries for the reference compiler. article Standard
• Stage ordering and lowering - Examples Newcomer-oriented examples of pipeline execution order and failure points. article Standard
• Stage ordering and lowering - FAQ and troubleshooting Troubleshooting guide for stage ordering assumptions in build and run commands. article Standard
• Stage ordering and lowering - Flow and algorithm Ordered lowering algorithm used by build and run command paths. article Standard
• Unit artifact cache schema On-disk and in-process unit snapshot layout under obj/beskid/cache/salsa/units/. article Standard Reviewed Fri Jun 05
• Stage ordering and lowering - Verification and traceability Source and test anchors that verify lowering order and backend artifact handoff. article Standard

History

0 revisions (git unavailable at build; counts may be empty)

Open full history on GitHub

No commits recorded for this path.

Completeness

OKfeature · 0 SpecSection(s) · 1 H2 heading(s)

Full tree: run pnpm verify:platform-spec-layout (writes src/generated/platform-spec-layout-report.json).

Related spec docs

• Feature
  Stage ordering and lowering

  Parent feature hub

• Feature
  Diagnostics parity (CLI and LSP)

  Diagnostic gates during lowering

• Area
  Compiler Mods

  Mod host flow and typed merge rounds

End-to-end lowering spine

flowchart TB
  r[1 Resolve CompilePlan]
  a[2 program.assemble]
  p[3 Parse units]
  m{Mods in graph?}
  mh[4 Mod collect generate reparse]
  s[5 Semantic rules]
  e{Error diagnostics?}
  stop[Abort]
  h[7 HIR resolve type-check]
  l[8 lower to CodegenArtifact]
  r --> a --> p --> m
  m -->|yes| mh --> s
  m -->|no| s
  s --> e
  e -->|yes| stop
  e -->|no| h --> l

Ordered steps

1. Resolve source path/text through services (manifest, workspace graph, CompilePlan, optional materialized workspace).
2. Assemble program — Build ProgramAssembly from effective roots (program.assemble); discover and parse compilation units per Program assembly.
3. Parse entry (and indexed units) and produce syntax diagnostics.
4. Mod host (when transitive Mod dependencies exist) — Load AOT artifacts, discover contracts, run collect → generate → analyze → rewrite under Mod host bridge policy, merge typed AST contributions, and re-parse as required (bounded by maxGeneratorRounds). Skip when no mod dependencies apply.
5. Run builtin staged semantic rules when diagnostics are enabled (same diagnostic channel as mod diagnostics).
6. Abort on error diagnostics.
7. Transform to HIR, resolve symbols against ModuleIndex, and type-check the entry unit.
8. Lower typed program to backend artifact for JIT/AOT.

The canonical lowering entrypoint remains beskid_codegen::services::lower_source (and _with_pipeline); mod orchestration must live in beskid_analysis services (or a dedicated host module re-exported there) so CLI, LSP, and codegen share one scheduling spine.