Pytest Operation

Pytest executes Fixture-based CT Test Cases in mock or hil mode. After execution, the Report operation uses a Local LLM to analyze the result and generate Markdown.

Framework structure and Fixture contracts are documented in Pytest CT Framework.

Operation Summary

Execution Sequence

Collect CT Test Cases
        ↓
Validate TEST ID, Fixture ID, marker, and Fixture metadata
        ↓
Apply optional --test-id selection
        ↓
Resolve mock or hil mode
        ↓
Connect Fixture interfaces and equipment
        ↓
Execute Test Case and record metrics/statistics
        ↓
Disconnect Fixture resources
        ↓
Write Result JSON and Test Log

marker is a mode-selection instruction, not a final execution mode. It resolves to the mock or hil value declared by @pytest.mark.ct.

Run Pytest

Run every configured test:

.\.venv\Scripts\python.exe -m pytest

Run all CT Test Cases with their marker modes:

.\.venv\Scripts\python.exe -m pytest test_envs/tests/pytest/test_cases --fixture-mode=marker

Force Mock mode:

.\.venv\Scripts\python.exe -m pytest test_envs/tests/pytest/test_cases --fixture-mode=mock

Run one TEST ID:

.\.venv\Scripts\python.exe -m pytest test_envs/tests/pytest/test_cases --test-id CT-UART-001 --fixture-mode=mock

The current executable CT Test Cases use Mock implementations. Selecting HIL fails explicitly where a physical Fixture implementation has not been completed.

Test Coverage

Run CT Test Cases and print missing Python lines:

.\.venv\Scripts\python.exe -m pytest test_envs/tests/pytest/test_cases --fixture-mode=mock --cov=test_envs --cov-report=term-missing

pytest-cov stores coverage data in .coverage. Add --cov-report=html when an HTML report under htmlcov/ is needed.

In VS Code Testing, select a Pytest node and use Run Test with Coverage. Coverage measures executed Python code; it does not measure physical HIL signal or protocol coverage.

Result and Report

Pytest execution and Report generation are separate operations.

Pytest execution
      ↓
Result JSON + Test Log
      ↓
test_envs.tools.test_result
      ↓
Local LLM analysis
      ↓
Conditional escalation decision
      ↓
Canonical Markdown
      ├── MkDocs Results
      └── Pandoc Report

Generate every pending Markdown report and publish MkDocs result pages:

.\.venv\Scripts\python.exe -m test_envs.tools.test_result --pending --docs

If Ollama is unavailable or returns invalid analysis, the reporter records the attempts and uses deterministic fallback analysis so Report generation can continue.

See Pytest Results for published results.