---
name: pumpgtm-test-designer
description: Design a bounded GTM test for PumpGTM or a team using it, covering buyer selection, conversation starters, offers, and follow-up. Use when a growth decision needs evidence before a wider rollout.
---

# PumpGTM test designer

Turn a disputed growth assumption into a decision the team can make from observable results. This skill prepares the test; running campaigns or contacting buyers requires the user's instruction.

## Intake and assumption selection

Before proposing a test, extract the following from the brief. Distinguish an unavailable input from one that is irrelevant to the chosen method.

| Input | Why it changes the design |
|---|---|
| Product and offer being tested | Prevents applying PumpGTM's audience to a customer's offer |
| Current workflow and proposed change | Establishes the comparison and what can be attributed |
| Audience eligibility and exclusions | Defines who can produce meaningful evidence |
| Channel and acquisition route | Reveals delivery, selection, and exposure differences |
| Existing outcome counts and dates | Establishes whether a usable baseline exists |
| Eligible weekly volume | Tests whether a quantitative design is feasible |
| Time and operating budget | Limits the number and complexity of tests |
| Decision owner and next action | Makes the result operationally useful |

If the user presents several ideas, create an assumption map before selecting the first test:

| Assumption | What would fail if false | Existing evidence | Cheapest discriminating observation | Priority reason |
|---|---|---|---|---|

Prioritize assumptions that are both weakly supported and capable of invalidating the proposed investment. Do not produce arbitrary numerical priority scores. Explain why the chosen assumption should be tested before the others. If an assumption depends on another, establish the prerequisite first: message optimization is premature when the audience cannot use the offer.

## Locate the uncertainty

Start with the decision being delayed: which buyers to pursue, which problem to lead with, what to offer, or when to follow up. Use the supplied campaign notes and results before asking for more context. If a missing input prevents a valid design, ask one focused question. Otherwise label the assumption and continue.

Separate these failure modes before choosing a method:

| Uncertainty | Useful evidence | Smallest practical check |
|---|---|---|
| An account has the problem | Recent examples of the problem and its consequences | Review customer notes or hold problem interviews |
| A signal identifies likely buyers | Verified fit among accounts selected by that signal | Manually audit a bounded account sample |
| An opener earns a useful response | Replies that acknowledge the problem or request a next step | Compare messages across comparable account groups |
| The offer warrants commitment | A completed action with meaningful effort | Invite eligible buyers to a clearly described pilot |
| A workflow removes friction | Successful completion of the intended task | Observe a prototype with representative users |

Choose the check that addresses the uncertainty. Interviews can explain a problem, but cannot establish a campaign conversion lift.

## Method-specific design requirements

### Problem interviews

Recruit people who recently encountered the relevant situation. Ask about the last occurrence, the action they took, the consequence, the workaround, and who became involved. Separate firsthand accounts from general opinions. Avoid leading with a product pitch or treating willingness to take a free call as buying intent.

Prepare an interview guide and a coding sheet for observed behavior, reported cost, current alternative, and conflicting evidence. Describe recurring themes within this sample; do not extrapolate a prevalence percentage to the entire market.

### Account-selection audit

Take a bounded sample from the actual selection process, not a handpicked set of obvious successes. Define what reviewers can verify, what requires a conversation, and how to record an ambiguous account. Include a comparison sample when the decision concerns the value of a new selection signal. Public signals establish observable context; private pain remains unconfirmed.

### Message or offer comparison

Keep eligibility, channel, sender conditions, observation window, and follow-up treatment comparable. Record assignment before exposure. Group related contacts under the same account. Define how late replies, bounces, missing exposures, and repeat contacts affect denominators before observing results.

If delivery changes between groups, show that difference alongside the business outcome. A message cannot be credited for improved interest when a different list or sender explains the result. Do not infer which element caused a package-level improvement.

### Prototype or assisted workflow

Specify the task, starting state, success event, assistance allowed, and failure conditions. Observe completion, time, errors, and help required. If the process is manually operated behind the scenes, explain that limitation to participants where relevant and to the team interpreting results. Manual success does not prove scalable automation.

### Technical feasibility check

Write the specific technical question, representative inputs, required output, operating constraints, and acceptance criteria. Keep a feasibility result separate from demand validation. A working prototype proves that a narrow task can run under the tested conditions; it does not establish customer adoption or production reliability.

## Define what the result will mean

Write a test card with:

- **Decision:** the action this evidence will unlock.
- **Prediction:** the audience, proposed change, expected behavior, and reason.
- **Eligibility:** inclusion criteria, exclusions, and the unit being compared.
- **Comparison:** current approach versus proposed approach, or an explicit discovery-only study.
- **Primary outcome:** an event, denominator, observation window, and counting rule.
- **Limits:** time, spend, exposure, and conditions for stopping because of harm.
- **Interpretation:** evidence needed to adopt, reject, or investigate further.

For outreach, define a qualified reply before counting it. Exclude automated responses and duplicates; distinguish interest from a refusal or an unsubscribe. Track negative responses separately so a higher total reply rate cannot disguise a worse buyer experience.

Keep accounts together when several contacts share a buying decision. Randomize comparable accounts when feasible and avoid exposing the same account to both variants. A package test may change several elements, but its result supports only the package, not any single component.

## Metric definitions and arithmetic

For each metric, define the event, eligible unit, denominator, counting period, deduplication rule, and source. For example, account-level qualified reply rate can be defined as unique assigned accounts producing at least one qualified reply divided by all eligible accounts assigned to that group during the window. Report delivered-contact rates separately if useful. Never switch from assigned accounts to delivered contacts after seeing which presentation looks better.

For two observed proportions, distinguish:

- Absolute change: variant rate minus control rate, expressed in percentage points.
- Relative change: absolute change divided by the control rate, expressed as a percentage. It is undefined when the control rate is zero.
- Practical value: the expected benefit at the team's actual volume, balanced against delivery cost and adverse outcomes.

Name the method used for uncertainty estimates. Sparse events, dependent contacts, and clustered assignment can invalidate simple assumptions. Do not use a statistical label just because two percentages differ. Statistical support, commercial usefulness, and operational feasibility are separate judgments.

## Match the claim to the volume

For a conversion comparison, request the baseline, smallest worthwhile difference, available eligible volume, and follow-up lag. Calculate statistical requirements with an available, appropriate method and state its assumptions. If calculation or inputs are unavailable, leave the required sample unresolved. Never invent a confidence level or call a convenient sample sufficient.

When volume is too low, use a learning study and describe its limits. A few encouraging conversations can justify another test; they do not prove an uplift. Set the observation window and analysis rule before launch. Stop early for a defined adverse outcome, and disclose that interruption when interpreting results.

## Analysis and closeout

Before reading outcomes, check assignment, exposure, duplicates, missing data, concurrent changes, and whether each group received its full observation window. An interrupted or contaminated test may be inconclusive even if the raw numbers look favorable.

Report the original decision rule, raw counts, calculated rates, uncertainty if validly calculated, adverse outcomes, and deviations. Give one of four conclusions: supported for the tested conditions, not supported at the tested effect, inconclusive, or invalidated by execution problems. Do not interpret an inconclusive test as evidence of equivalence.

Distinguish a next experiment from a recommendation to roll out. State which audience and conditions the result covers and what remains untested.

## Return a usable handoff

Deliver the test card, the exact material needed to conduct it, and a results table with columns for group, eligible accounts, exposures, qualified outcomes, adverse outcomes, and unresolved observations. Close with the decision rule and the next action for each possible result. Keep unrun tests in future tense and leave actual results blank.

## Templates and worked cases

Read [the experiment workbook](references/experiment-workbook.md) when producing a full plan or interpreting supplied results. It contains the copyable plan, interview prompts, an illustrative PumpGTM test, calculation examples, and exception handling. Use only the sections relevant to the user's request.

## Completion check

The final plan must identify a decision, a falsifiable assumption, an observable outcome, a valid comparison or discovery method, a feasible operating limit, and a rule for interpreting the result. Any missing baseline, uncalculated sample requirement, or unverified product capability must be visible. A finished planning document must not imply that the test has already run.