unittest Framework

Scope

The unittest area validates the CT framework and provides extension points for product-level Python, C/C++, firmware, and shared tests. Tests may use unittest.TestCase, but VS Code and the project command run them through the pytest adapter so that the repository result hook is applied.

Architecture

flowchart TD
    A[VS Extension Testing or pytest command] --> B[pytest adapter]
    B --> C[test_envs/tests/unittest]
    C --> D[Function-level reports]
    D --> E[unittest conftest hooks]
    E --> F[Execution Result JSON]
    E --> G[Execution Result log]
    F --> H[Unittest Markdown reporter]
    G --> H
    H --> I[MkDocs execution page and index]

The unittest branch does not call Ollama, create a Local LLM log, add test_analysis, or evaluate Codex escalation.

Repository Structure

test_envs/tests/unittest/
├── conftest.py
├── ct_framework/
│   └── python/
│       ├── test_configuration.py
│       ├── test_environment_setup.py
│       ├── test_extension_runner.py
│       ├── test_fixture_contract.py
│       ├── test_issue_parser.py
│       ├── test_latest_result.py
│       ├── test_local_llm.py
│       ├── test_log_parser.py
│       ├── test_markdown_reporter.py
│       ├── test_mock_digilent.py
│       ├── test_mock_network.py
│       ├── test_mock_uart.py
│       ├── test_mock_usb.py
│       ├── test_pandoc_reporter.py
│       ├── test_reporting.py
│       ├── test_repository_structure.py
│       ├── test_result_normalizer.py
│       └── test_vscode_local_llm_contract.py
├── python/
├── c_cpp/
├── firmware/
└── common/

Execution

Project command

.\.venv\Scripts\python.exe -m pytest -p no:cacheprovider test_envs/tests/unittest

VS Code Testing

Running python -m unittest bypasses the pytest hooks and therefore does not generate this framework's normalized result or log.

GitHub Actions

continuous-test.yml selects the Unittest path when the request comes from unittest_request.yml or the manual Test Type is Unittest:

unittest_request.yml Issue
        ↓
`test_envs.tool_github.issue_parser` selects Unittest scope and runner
        ↓
pytest test_envs/tests/unittest
        ↓
unittest conftest session result
        ↓
pipeline processes the explicit result
        ↓
Markdown + Issue comment + uploaded evidence

The unified workflow uses the unittest pytest hook as the single result-capture path. It does not run Local LLM analysis for the resulting category: unit record.

Result Capture Lifecycle

Source: test_envs/tests/unittest/conftest.py

Function capture

The pytest_runtest_logreport hook accepts only node IDs under test_envs/tests/unittest/. Results are stored by full pytest node ID and written in sorted order when the session finishes.

A teardown error replaces the earlier function result and adds teardown duration to the previously captured duration.

Session result

If no unittest function report is captured, the session hook does not create result files. A session containing only skipped functions currently has overall status PASS, while each function retains SKIP.

Result Files

Unlike pytest CT, unittest results are flat because they are keyed only by Execution ID.

test_reports/results/unittest/
├── <execution-id>_result.json
└── <execution-id>_result.log

Result JSON schema

execution
├── execution_id
├── timestamp
├── status
├── duration
├── environment
├── runner
├── commit
├── branch
└── logs
summary
├── total
├── passed
├── failed
├── errors
└── skipped
test_functions[]
├── path
├── function
├── pass
├── status
├── duration
└── failure

Result log schema

[execution]
execution_id=...
timestamp=...
status=...
total=...
passed=...
failed=...
errors=...
skipped=...

[test_functions]
PASS | <function> | duration=<seconds>s

[failed_functions]
<function> | FAIL or ERROR
<failure detail>

The [failed_functions] section is emitted only when at least one function has FAIL or ERROR status.

Markdown and MkDocs

Generate every pending execution report and publish the MkDocs pages with:

.\.venv\Scripts\python.exe -m test_envs.tools.test_result --pending --docs

test_reports/results/unittest/<execution-id>_result.json
                         ↓
test_reports/markdown/unittest/<execution-id>_result.md
                         ↓
docs/tests/unittest/<execution-id>.md
                         ↓
docs/tests/unittest/index.md

Markdown sections

# unittest Result
└── Test Summary
    ├── Execution summary
    ├── Test Functions
    │   ├── PATH directory mapping
    │   └── Function result table
    ├── Failed Functions
    ├── Test Source
    └── Logs

The function table's Pass column is a simplified Boolean display: PASS when test_functions[].pass is true and FAIL otherwise. Use the JSON status field for the authoritative distinction between FAIL, ERROR, and SKIP; skipped functions are excluded from the Markdown Failed Functions table.

Index columns

Pytest CT Comparison

Related Documents

Report

Unittest and Pytest connect to the same Markdown-first Report pipeline. Unittest bypasses Local LLM analysis and escalation, then uses the shared Markdown, MkDocs, and Pandoc reporters.

Unittest result JSON + result log
                ↓
test_envs.tools.test_result
                ↓
Local LLM and escalation: not used
                ↓
test_envs.tools.mkdocs_reporter
                ↓
Canonical Markdown
       ┌────────┴────────┐
       ↓                 ↓
MkDocs Results       Pandoc Report

Generate pending Markdown and publish MkDocs result pages:

.\.venv\Scripts\python.exe -m test_envs.tools.test_result --pending --docs

Convert the latest Markdown to HTML:

.\.venv\Scripts\python.exe -m test_envs.tools.pandoc_reporter --latest --format html

Convert the latest Markdown to DOCX:

.\.venv\Scripts\python.exe -m test_envs.tools.pandoc_reporter --latest --format docx

The same commands are available from VS Code through Run and Debug-Report and Tasks-Report. Unittest result pages are published under Unittest Results.