Multimodal Prompt Test Harness Spec

The Prompt

You are a QA architect. Design a test harness specification for multimodal prompt workflows.

PRODUCT GOAL:
{{product_goal}}

PROMPT SUITE:
{{prompt_suite}}

MEDIA TYPES:
{{media_types}}

QUALITY THRESHOLDS:
{{quality_thresholds}}

FAILURE CONDITIONS:
{{failure_conditions}}

CI CONSTRAINTS:
{{ci_constraints}}

Return:
1. Test harness architecture (inputs, execution, evaluation, reporting).
2. Test case matrix covering happy path, edge cases, and regressions.
3. Evaluation rubric for generated media outputs.
4. Failure triage protocol and severity levels.
5. Fallback test strategy for environments without native media generation.

Rules:
- Keep framework provider-agnostic.
- Distinguish deterministic checks vs human-review checks.
- Include guidance for reproducibility and prompt versioning.

When to Use

Use this when product teams need repeatable quality checks for media-oriented prompts before shipping changes to users.

Variables

• product_goal: What user-facing workflow this harness protects.
• prompt_suite: Set of prompts under test.
• media_types: Which output modalities are in scope.
• quality_thresholds: Acceptance criteria for pass/fail.
• failure_conditions: Known critical breakpoints.
• ci_constraints: Runtime, cost, environment, and tooling limits.

Tips & Variations

• Ask for a “daily smoke” and “weekly deep” test split for cost control.
• Add golden-sample checks for stable reference comparisons.
• Include manual-review escalation rules for non-deterministic failures.
• If media generation is unavailable in CI, run metadata and prompt-structure checks plus scheduled human audits.

Example Output

A strong output includes a test harness blueprint, prioritized case matrix, reproducibility rules, and a triage model that engineering and QA can operate jointly.