IMI Campaign Evaluation Framework

This skill teaches the AI to evaluate campaigns and creative concepts the way IMI's senior

strategists do: with normative rigour, clear go/no-go frameworks, and recommendations that

tell clients what to DO, not just what the numbers say.

The Campaign Evaluation Lifecycle

IMI's campaign evaluation spans two pillars:

CONFIRM (Pre-Market):

• Concept testing — Should we proceed with this idea?
• Message testing — Which message resonates most?
• Creative diagnostics — What's working and what's not in the execution?

OPTIMIZE (Post-Market):

• Brand lift measurement — Did the campaign change perceptions?
• In-market ROI — Was the spend justified?
• Learning extraction — What should we do differently next time?

Part 1: Concept Testing

The IMI Concept Test Battery

IMI's standard concept test measures these dimensions:

T2B Scoring

Top-2-Box (T2B) is IMI's standard metric for concept evaluation. It counts the percentage

of respondents who selected the top two options on the scale.

SELECT
    concept_id,
    metric_name,
    SUM(CASE WHEN response_value IN (4, 5) THEN resp_weight ELSE 0 END)
        / SUM(resp_weight) * 100 AS t2b_pct,
    SUM(resp_weight) AS weighted_n,
    COUNT(DISTINCT respondent_id) AS unweighted_n
FROM concept_test_responses ctr
JOIN respondents r ON ctr.respondent_id = r.respondent_id
WHERE study_id = :study_id
    AND wave_id = :wave_id
GROUP BY concept_id, metric_name;

Critical: Always use WEIGHTED calculations. Unweighted T2B will be wrong if the sample

is not naturally representative.

The IMI Norm Database

IMI maintains a normative database built from thousands of concept tests across categories.

Norms are the benchmark against which every concept score is evaluated.

Norm comparison interpretation:

Norm categories matter: A beer concept should be compared to beer norms, not FMCG norms.

Always use the most specific norm available.

Go/No-Go Decision Framework

IMI uses a structured decision matrix, not a single score:

STRONG GO: Purchase Intent above norm AND Overall Appeal above norm
           → Proceed to production/launch

CONDITIONAL GO: Purchase Intent above norm BUT Appeal at/below norm
           → Concept has commercial potential but needs creative enhancement
           OR: Appeal above norm BUT Purchase Intent at/below norm
           → Strong creative but value proposition needs strengthening

REVISE: Both metrics within ±4pts of norm, with at least one diagnostic
        metric (uniqueness, relevance, believability) significantly above norm
        → There is a seed worth developing — diagnose and optimise

REJECT: Purchase Intent below norm AND at least one other primary metric below norm
        → Do not proceed. Fundamental issue with the proposition.

The hierarchy of metrics:

• Purchase Intent — Most predictive of in-market behaviour
• Overall Appeal — Best measure of creative resonance
• Uniqueness — Differentiator; high uniqueness + low PI = creative but not commercial
• Relevance — Table stakes; below-norm relevance is a fatal flaw
• Believability — Below-norm believability kills conversion
• Brand Fit — Below-norm fit means the concept is good but wrong for THIS brand

Worked Example: Three-Concept Evaluation

Concept A: Appeal 67% [norm 58%], PI 52% [norm 57%], Uniqueness 74% [norm 51%]
Concept B: Appeal 59% [norm 58%], PI 61% [norm 57%], Uniqueness 56% [norm 51%]
Concept C: Appeal 51% [norm 58%], PI 49% [norm 57%], Uniqueness 62% [norm 51%]

Analysis:
- Concept A: CONDITIONAL GO — Exceptional appeal (+9) and uniqueness (+23) but PI
  is 5pts below norm. The creative idea is powerful but the value proposition or
  call-to-action is not converting interest to intent. Revise the commercial hook
  and retest.
- Concept B: STRONG GO — Both primary metrics above norm. Not spectacular on any
  single dimension but commercially solid. Proceed to production.
- Concept C: REJECT — Appeal 7pts below norm, PI 8pts below norm. Uniqueness above
  norm is not enough to compensate. Do not proceed.

Recommendation: Launch Concept B. Revise Concept A's value proposition (retain
the creative idea, strengthen the reason-to-buy) and retest. Drop Concept C.

Part 2: Creative Diagnostics

When a concept scores poorly, the client needs to know WHY and WHAT TO FIX.

Diagnostic Framework

Step 1: Identify the failing metric (which dimension is below norm?)
Step 2: Cross-tabulate with open-ended responses (what are consumers saying?)
Step 3: Segment the response (who likes it? who doesn't?)
Step 4: Identify the fix (is it message, visual, tone, CTA, or fundamental proposition?)
Step 5: Recommend specific changes (not "make it better" — "change the headline from X to Y")

Diagnostic SQL: Open-End Analysis by T2B Group

SELECT
    CASE WHEN ctr.response_value IN (4, 5) THEN 'T2B' ELSE 'B3B' END AS response_group,
    oe.theme_code,
    oe.theme_label,
    COUNT(DISTINCT oe.respondent_id) AS mention_count,
    ROUND(100.0 * COUNT(DISTINCT oe.respondent_id) /
        COUNT(DISTINCT ctr.respondent_id), 1) AS mention_pct
FROM concept_test_responses ctr
JOIN open_ends oe ON ctr.respondent_id = oe.respondent_id
    AND ctr.concept_id = oe.concept_id
WHERE ctr.study_id = :study_id
    AND ctr.metric_name = 'OVERALL_APPEAL'
GROUP BY response_group, oe.theme_code, oe.theme_label
ORDER BY response_group, mention_pct DESC;

This reveals what the HIGH scorers liked (to preserve) vs. what the LOW scorers disliked (to fix).

Part 3: In-Market Campaign Measurement

Brand Lift Methodology

IMI measures brand lift using a control vs. exposed design:

Brand Lift = Metric (Exposed group) - Metric (Control group)

Key metrics for brand lift:

• Aided awareness
• Ad recall (prompted and unprompted)
• Message association
• Brand consideration
• Purchase intent
• Brand favourability

Significance Testing for Brand Lift

import numpy as np
from scipy.stats import norm as normal_dist

def brand_lift_significance(exposed_pct, control_pct, exposed_n, control_n,
                            confidence=0.95):
    """
    Test whether the brand lift (exposed - control) is statistically significant.
    Returns: lift, z_score, p_value, is_significant
    """
    lift = exposed_pct - control_pct
    se = np.sqrt(
        (exposed_pct * (100 - exposed_pct)) / exposed_n +
        (control_pct * (100 - control_pct)) / control_n
    )
    z = lift / se if se > 0 else 0
    p = 2 * (1 - normal_dist.cdf(abs(z)))
    z_crit = normal_dist.ppf(1 - (1 - confidence) / 2)

    return {
        'lift': round(lift, 1),
        'z_score': round(z, 2),
        'p_value': round(p, 4),
        'is_significant': abs(z) > z_crit,
        'confidence': confidence
    }

ROI Framework

Campaign ROI = (Incremental Revenue Attributable to Campaign - Campaign Cost) / Campaign Cost

Where:
  Incremental Revenue = Brand Lift in Purchase Intent × Conversion Rate × Average Transaction Value × Audience Size

IMI's ROI interpretation:

• ROI > 3:1 = Strong return — campaign is working efficiently
• ROI 1.5:1 - 3:1 = Acceptable return — monitor and optimise
• ROI 1:1 - 1.5:1 = Marginal — campaign is barely paying for itself
• ROI < 1:1 = Negative — campaign cost more than it generated

Output Templates

Template: Concept Test Recommendation

## Concept Test Results: [Study Name]
**Client:** [name] | **Category:** [name] | **Base per concept:** n=[n]
**Fieldwork:** [dates] | **Norms:** [category] category norms (n=[norm base])

### Summary Scorecard
| Metric | Concept A | Concept B | Concept C | Category Norm |
|---|---|---|---|---|
| Overall Appeal T2B | X% | X% | X% | X% |
| Purchase Intent T2B | X% | X% | X% | X% |
| Uniqueness T2B | X% | X% | X% | X% |
| Relevance T2B | X% | X% | X% | X% |

### Recommendations
**Concept A:** [GO / CONDITIONAL GO / REVISE / REJECT] — [rationale]
**Concept B:** [GO / CONDITIONAL GO / REVISE / REJECT] — [rationale]
**Concept C:** [GO / CONDITIONAL GO / REVISE / REJECT] — [rationale]

### Essential Insight
[One sentence: the strategic learning from this test]

### Recommended Next Step
[Specific IMI capability: optimise creative, proceed to production, commission Optimize tracking]

Common Pitfalls

• Picking the highest number. The concept with the highest appeal is not always the

best — purchase intent is more predictive of in-market success.

• Ignoring norm specificity. Comparing a luxury concept to mass-market norms inflates

every score. Always use the most category-specific norm.

• Reporting significance without base sizes. A 3-point lift on n=50 is not significant.

Always report base sizes and confidence levels.

• Confusing appeal with conversion. A concept can be loved (high appeal) but not bought

(low PI). These require different fixes.

• Presenting data without a recommendation. The client hired IMI for a decision, not

a dashboard. Every concept test must end with a clear go/no-go.

Cross-Skill References

• For audience profiling before concept testing → imi-pulse-intelligence
• For segmentation of concept test respondents → imi-segmentation-engine
• For SQL queries over concept test databases → imi-rag-sql-intelligence
• For writing up concept test results → imi-client-deliverable
• For Say/Do gap when PI is high but conversion is low → imi-say-do-gap

*Built for IMI International's Local AI — grounded in thousands of concept tests and

IMI's proprietary normative database.*

*Purpose: Insight. Method: Rigour. Outcome: Profit.*

AXE MCP Server Integration

Every skill in the AXE Skills Hub runs with access to the AXE MCP Server — giving it the full fleet intelligence toolkit automatically. No setup required; tools are available in any AXE-powered session.

Core Tools Available

Quick Start

# In any AXE session, tools are pre-loaded. Example chaining:

# 1. Search for context
results = qdrant_search("user query here", collection="axe_persistent_memory")

# 2. Fetch live data if needed
content = web_fetch("https://docs.example.com/api")

# 3. Write result to memory for next session
write_memory("shared/last_result.md", output)

# 4. Log quality output to Edge training pipeline
hydra_add(prompt=user_query, response=output, score=0.9, source="skill-name")

Edge Training Integration

High-quality skill outputs are automatically eligible for Edge model training via hydra_add. When a response scores ≥0.85 in evals, pipe it to the Hydra pipeline to compound Edge's knowledge. This is how skills make Edge smarter over time.

# After generating a high-quality response:
hydra_add(
    prompt=user_input,
    response=final_output,
    score=0.9,          # eval score
    source="skill-name" # tracks provenance
)