Qualityfolio: Test Management as Code & Traceability Mesh

Consequential GRC Engineering

Turn requirements and software tests into executable Markdown and queryable SQLite verification evidence. Replace disjointed web ALMs with version-controlled, Git-native assurance that a reviewer can follow from requirement to result.

Test Management as Code Markdown & YAML Frontmatter SQLite RSSD IEC 62304 / ISO 13485 Traceability

The Customer Problem: Four Regulated Testing and V&V Failures

The disjointed web ALM silo

Engineers write tests in Git while QA logs results in a separate ALM tool such as Jira with Xray or a legacy test manager. The two records drift apart, and nobody can say which one is true.

The manual traceability matrix scramble

The requirements-to-test matrix is assembled by hand in a fragile spreadsheet in the weeks before an audit or notified body review, then goes stale the day after.

Unverifiable execution proof

A result is recorded as a bare pass or fail, with no environment log, run record or raw output attached that would let a reviewer check it.

Brittle test-run reconstruction

Historical test cycles cannot be re-opened or inspected because the testing platform was upgraded, migrated or its licence lapsed.

The Consequential Engineering Solution: Test Management as Code

A requirements document states what a system should do. In Qualityfolio, test artifacts play that role for quality: they declare what must be verified. Evidence records what actually happened when the test ran. Both live in the same Markdown file, under version control, so intent and result can always be compared.

  • Tests are Git artifacts. Test plans, suites, cases and execution evidence are files reviewed in the same pull request as the code they verify, with the full change history Git keeps.
  • Evidence is data, not a screenshot in a ticket. Every cycle compiles into a SQLite database that any SQL tool can open, and the traceability matrix is regenerated from source instead of rebuilt by hand.
  • Truth gaps become visible. A test case with no evidence is an unverified expectation, not a pass. Evidence with no test case is execution without declared intent. Both show up in the data.
Point-in-time test records compared with SQL-native evidence
Question a reviewer asks Point-in-time ALM record Qualityfolio evidence
Which requirement does this test verify?A manual link that breaks when either side is edited.A requirementID in the test case's META block, versioned with the test.
What happened in cycle 1.1?Whatever the tool still shows after upgrades and exports.An evidence block per cycle with status, date, assignee and attached result files.
Is every requirement covered?A spreadsheet assembled before the audit.A query against the SQLite database, run again whenever the files change.
Can we see last year's run?Only if the licence and data export survived.Check out the Git revision and rebuild the database.

What Netspective Offers: Engineering Responsibility

Qualityfolio is the tooling. The engagement is the engineering work that makes it hold up under review for medical device software built to IEC 62304 and ISO 13485:2016.

Spry orchestration runbooks

We build and maintain the runbooks (qualityfolio.md and the SQL ETL) that turn your test artifacts and evidence into the SQLite database and dashboard, and keep them working as your repository grows.

Requirement-to-evidence schemas

We choose the hierarchy model, define the META keys and identifier scheme, and map your requirement sources so every software requirement has a place to be traced.

Qualified human verification review

Engineers who build regulated software read the traceability matrix and the evidence behind it, and flag gaps before your auditor does. Sign-off stays with your quality organization.

The Qualityfolio Engine: Architecture and Hierarchy Model

Qualityfolio installs locally on a workstation or server; it is not a web service. Everything it shows is derived from files under version control.

Runtime and tools
Deno 2.x or later and Homebrew, then three tools: Spry parses the Markdown and runs the runbooks (spry rb run, spry sp spc); surveilr, a local binary, ingests the evidence and applies the SQL ETL; SQLPage serves the dashboards.
Hierarchy by heading depth
A doc-classify block in the YAML frontmatter maps each heading level to a role: project, strategy, plan, suite, case or evidence. Spry validates the structure before anything is written.
Dashboards from SQLite
The generated resource-surveillance.sqlite.db is the system of record. SQLPage renders test metrics, cycle-wise execution status, case drill-downs and a requirement traceability matrix (RTM) straight from it.
  1. Markdown artifactsTest cases and evidence, in Git
  2. SpryParses and classifies the headings
  3. surveilrIngests evidence, runs the SQL ETL
  4. SQLite RSSDOne file, the system of record
  5. SQLPageMetrics, cycles and the RTM
The documented pipeline. The design-time preview (spry axiom web-ui) validates structure; it does not execute tests.

Four hierarchy models

Pick the model by how much structure the work needs. Heading levels are never skipped, and evidence always sits at the lowest level.

Heading depth and role in each Qualityfolio model
Model H1 H2 H3 H4 H5 H6
SmallProjectCaseEvidence–––
MediumProjectSuiteCaseEvidence––
LargeProjectPlanSuiteCaseEvidence–
ComplexProjectStrategyPlanSuiteCaseEvidence

Requirements are added by overriding the strategy or plan level, so one requirement can govern many test cases. Issues are added at the evidence level, next to the failure that raised them. The Large and Complex models suit compliance work.

Metadata a test carries

Identifiers and META keys in a Qualityfolio test artifact
Key Where What it records
@idUnder any headingA stable identifier (for example TC-MED-001) that links a case to its evidence.
requirementIDCase METAThe requirement the case verifies.
Priority, Tags, Scenario TypeCase METARisk ordering, grouping, and the kind of path under test.
cycle, cycle-dateEvidence METAWhich test cycle the evidence belongs to, and when.
assignee, statusEvidence METAWho ran it and the outcome (passed, failed, blocked).
Qualityfolio QA progress and performance dashboard showing test case, pass, fail and defect counts, automation and manual coverage, and execution status charts.
The SQLPage dashboard from the public Qualityfolio demo (demo data).
Qualityfolio test run summary for a failed test case, listing each step with its status, start and end time, and the error recorded on the failing step.
Step-level execution evidence for one test case, from the public demo (demo data).

Ingestion & Workflow Explorer

Four steps from a Markdown test case to a traceability question an auditor can re-ask. Run the commands from the qualityfolio/ folder of your repository.

Panel 1: Markdown test specification and metadata

A test case in the Medium model. Its @id links the case to its evidence, and its requirementID links it to the requirement it verifies.

---
doc-classify:
  - select: heading[depth="1"]
    role: project
  - select: heading[depth="2"]
    role: suite
  - select: heading[depth="3"]
    role: case
  - select: heading[depth="4"]
    role: evidence
---

# Infusion Dose Calculator (SaMD)

@id samd-dose-calculator

Software system testing for an IEC 62304 Class B software item.

## Dose Alert Verification

@id dose-alert-suite

Verifies the over-limit dose alert against its software requirement.

### Verify an over-limit dose entry raises a blocking alert

@id TC-MED-001

```yaml META
  requirementID: REQ-SAMD-010
  Priority: High
  Tags: [IEC 62304, Software System Testing]
  Scenario Type: Unhappy Path
```

**Steps**

- [x] Enter a dose above the configured maximum for the selected drug.
- [x] Attempt to confirm the order.

**Expected Results**

- [x] A blocking alert names the limit that was exceeded.
- [x] The order cannot be confirmed until the dose is corrected.

#### Evidence

@id TC-MED-001

```yaml META
  cycle: 1.2
  cycle-date: 04-11-2026
  assignee: QA engineer
  status: passed
```

**Attachment**

- [Results JSON](./evidence/TC-MED-001/1.2/result.auto.json)
- [Run MD](./evidence/TC-MED-001/1.2/run.auto.md)

Synthetic illustration Follows the public Qualityfolio Medium model (Project → Suite → Case → Evidence) and the META keys in the example project files on docs.opsfolio.com. The device, IDs, steps and assignee are illustrative.

Panel 2: Spry ingestion runbook

One runbook parses the test artifacts, ingests the evidence files and applies the SQL ETL.

# Parse test-artifacts/, ingest evidence/, apply qualityfolio-json-etl.sql
spry rb run qualityfolio.md

# Inspect what was captured
surveilr shell -d resource-surveillance.sqlite.db --output table \
  --cmd "SELECT nature, COUNT(*) AS files FROM uniform_resource GROUP BY nature;"

Writes: resource-surveillance.sqlite.db. The runbook command is from the Qualityfolio architecture overview; the inspection query uses the surveilr CLI's documented uniform_resource table.

Panel 3: Dashboard generation and local serving

Build the SQLPage configuration from qualityfolio.md, then serve the dashboard to this machine only.

spry sp spc --fs dev-src.auto --destroy-first --conf sqlpage/sqlpage.json --md qualityfolio.md

# Bind to localhost; the dashboard is then at http://localhost:9227/
surveilr web-ui -d ./resource-surveillance.sqlite.db --port 9227 --host localhost

Writes: sqlpage/sqlpage.json. Commands from the Qualityfolio quick start, except the host: the published sample binds 0.0.0.0, which exposes the dashboard to the network, so this one binds localhost.

Panel 4: Requirement traceability SQL query

Lists every requirement with its test cases, each cycle's status and the evidence file behind it. A requirement with no test case shows as NOT COVERED.

-- Synthetic illustration: a requirement traceability matrix from the RSSD.
-- uniform_resource is surveilr's table (it already exists in a real RSSD).
-- test_case and requirement stand in for the links Qualityfolio's ETL reads
-- from each test case's YAML META block (requirementID).
CREATE TABLE IF NOT EXISTS uniform_resource (uri TEXT, nature TEXT, content TEXT);
CREATE TABLE IF NOT EXISTS requirement (requirement_id TEXT PRIMARY KEY, title TEXT);
CREATE TABLE IF NOT EXISTS test_case (
  test_case_id   TEXT PRIMARY KEY,
  requirement_id TEXT NOT NULL REFERENCES requirement (requirement_id)
);

WITH evidence AS (
  -- .../evidence/TC-MED-001/1.1/result.auto.json -> TC-MED-001/1.1/result.auto.json
  SELECT uri,
         substr(uri, instr(uri, '/evidence/') + 10) AS rel,
         json_extract(content, '$.status') AS status
  FROM uniform_resource
  WHERE uri LIKE '%/evidence/%/result.auto.json' AND json_valid(content)
), runs AS (
  SELECT substr(rel, 1, instr(rel, '/') - 1) AS test_case_id,
         substr(rel, instr(rel, '/') + 1,
                instr(substr(rel, instr(rel, '/') + 1), '/') - 1) AS cycle,
         status, uri
  FROM evidence
)
SELECT r.requirement_id,
       COALESCE(t.test_case_id, 'NOT COVERED') AS test_case_id,
       COALESCE(x.cycle, '-')                  AS cycle,
       COALESCE(x.status, 'no evidence')       AS status,
       x.uri                                   AS evidence_path
FROM requirement r
LEFT JOIN test_case t ON t.requirement_id = r.requirement_id
LEFT JOIN runs x      ON x.test_case_id   = t.test_case_id
ORDER BY r.requirement_id, t.test_case_id, x.cycle;

Synthetic illustration requirement and test_case are illustrative, not Qualityfolio's ETL schema, and the query assumes each result file has a top-level status. It runs as written in SQLite 3.

Verified Proof Ledger (Engineering Ground Truth)

Proof ledger: each claim, its status and what it rests on
Claim Status What it rests on
Markdown test artifacts ingested and classified by Spry (doc-classify)Built and testedThe public opsfolio/qualityfolio repository, architecture overview and quick start.
SQLite RSSD generated from test artifacts and evidence (spry rb run)Built and testedDocumented end-to-end execution flow: resource-surveillance.sqlite.db.
SQLPage dashboard with a requirement traceability matrix (RTM)Built and testedQuick start dashboard contents; public demo screenshots.
Requirement-to-test linking through @id and requirementID metadataBuilt and testedTest artifact models and example project files.
Cycle-wise manual evidence capture (cycle, cycle-date, assignee, status)Built and testedEvidence META blocks and the role-based test management workflow.
Automated execution capture from a CI runnerDesignedListed as a next step in the docs; scoped per engagement, not shown in the public example.
Automated test generation from requirementsDesignedDescribed on qualityfolio.dev; not part of the documented workflow.
Built-in 21 CFR Part 11 cryptographic electronic signaturesNot establishedEvidence is manual checkbox entries in local SQLite. Part 11 controls belong to the validated process.
Push-button regulatory clearanceNot establishedClearance is a regulator's decision on the sponsor's submission.
SaaS multi-tenant cloud test managementNot establishedQualityfolio installs locally; the docs state it is not web-based.

Real-World Regulated Scenarios

Where the evidence goes. Qualityfolio supports Part 11-relevant record keeping when configured within a validated process; your quality organization and your regulator decide whether a record is sufficient.

IEC 62304 software verification

For Class B or C medical device software: system and software requirements linked by requirementID to unit, integration and system tests, with each cycle's result files kept as evidence.

ISO 13485:2016 §7.3 design verification records

Verified test runs compiled from the repository into the design and development file, without reconciling spreadsheets by hand before each design review.

A Git-native release quality gate

A CI step queries the database and blocks a release unless every defined requirement maps to a passed test record. Wiring the gate into your pipeline is engagement work, not a built-in feature.

Service Boundaries and a Low-Risk First Step

The 2-week V&V traceability pilot

Week 1: structure

Move your existing Word or Excel test matrices into structured Markdown, set up the Git repository and write the Spry runbooks.

Week 2: evidence

Compile the baseline SQLite evidence database, run the SQLPage traceability dashboard locally, and walk the requirement coverage gaps with your engineering leads.

Boundaries

  • Qualityfolio is tooling for test management and evidence structuring. It does not replace regulatory counsel, notified body audits or your verification sign-off.
  • It has no built-in electronic signatures. Evidence is recorded as manual checkbox entries in a local SQLite database.
  • No outcome is promised: clearance, certification and audit results are decided by regulators, notified bodies and auditors.
Where your tests live today

Please don't attach test records, requirement documents or device files to the first message; we agree how they move during scoping.

Just start a conversation

Engagement detail

What does the Regulated Software Quality & V&V engagement deliver?

It closes the gap between your verification evidence and what a notified body expects under IEC 62304. The deliverables are a documented safety classification, a traceability matrix tracing 100% of software requirements to a test, and a V&V plan.

A scoped engagement to close the gap between your current verification evidence and what a notified body or auditor actually expects: safety classification, requirements-to-test traceability, and a V&V plan your quality system can stand behind — delivered by engineers who build and operate regulated software themselves, not just advise on it.

What this engagement produces

The IEC 62304 gap assessment and its exported remediation checklist are the starting point, not the deliverable. Under IEC 62304, this service closes the specific gaps that checklist surfaces for MedTech software: a documented safety classification, a working requirements-to-test traceability matrix, and a V&V plan scoped to your software's actual safety class — built by engineers who build and operate their own regulated software products, and have walked through FDA submissions, FedRAMP authorizations, and HIPAA audits directly.

Risk-Classified
Safety class (A/B/C) drives required rigor per IEC 62304 §4.3.
V&V Decoupled
Verification (built right) evidenced separately from Validation (built the right thing).
Living Traceability
Machine-regenerable links from requirements to test executions.
Audit-Defensible
Evidence structured for inclusion in your FDA submission or Notified Body technical file.
Regulated V-model and evidence architecture Four-stage flow: User Needs and Intended Use, then Software Requirements Specification, then Unit and Integration Verification, then Summative Validation and DHF Compilation, connected by directional arrows, with a dashed perimeter marking the IEC 62304 / FDA QMSR boundary around the requirements and verification stages. IEC 62304 / FDA QMSR BOUNDARY User Needs & Intended Use Use Environment / Stakeholders ISO 14971 Hazard Context Software Requirements Spec IEC 62304 §5.2 Risk-Classified Requirements Unit & Integration Verification Static Analysis / Test Harnesses §5.5–§5.6 Evidence Summative Validation & DHF Compilation IEC 62366 Validation Audit-Ready Evidence Packet
V&V-SPEC 01: Regulated V-Model & Evidence Architecture — Requirements → Traceability → Validated State

Six capability areas

IEC 62304 Software Classification & Gap Remediation

Safety class A/B/C determination grounded in an ISO 14971 risk analysis, with a remediation roadmap for the specific gaps it surfaces.

  • Safety Class A/B/C Determination
  • Legacy SOUP Evaluation
  • Remediation Roadmap

Requirements-to-Test Traceability Matrix

Multi-directional tracing from requirement to design element to test to risk control, regenerated from source control rather than hand-maintained.

  • Multi-Directional Requirement Tracing
  • Design Control Linkage
  • Automated CI-Generated Trace Records

Verification Protocols & Automated Test Harnesses

Unit and integration verification evidence, backed by static code analysis records a reviewer can check against the specification directly.

  • Unit Verification Protocols
  • Integration Boundary Testing
  • Static Code Analysis Evidence

Clinical Validation & Human Factors Evidence

User-needs acceptance criteria and IEC 62366 usability testing protocols, with support through a summative validation study.

  • User Needs Acceptance Criteria
  • IEC 62366 Usability Testing Protocols
  • Summative Study Support

ISO 14971 Risk Management Integration

Risk control verification linked to the specific hazard each one closes, with residual risk reported against the same file the design team uses.

  • Hazard Mitigation Verification
  • Software Risk Control Option Analysis
  • Residual Risk Reports

Audit & Premarket Submission Support

FDA 510(k)/PMA software documentation and Notified Body technical files, plus direct engineering support through the audit itself.

  • FDA 510(k)/PMA Software Documentation
  • Notified Body Technical Files
  • Audit Defense Support
SOC 2 TYPE 1 & 2 ATTESTED IEC 62304 ISO 14971 FDA QMSR

Ready to Build Your Verification Evidence?

Discuss a specific V&V gap, traceability chain, or safety classification.

Engineering services description. Not a quote or a statement of work — scope and pricing are established during the initial consultation referenced from the gap-assessment tool.

Sign in or sign up

Enter your email to receive a temporary sign-in link.

By continuing, you agree to our Terms of Service and Privacy Policy.