Tasks / Design
Activation & onboarding review
Can the model find the friction that matters most and prioritise the fixes?
Measures the modelTask v1.0 · Release · 2 casesDifficulty
The PM job
Reviewing a signup and onboarding flow that is losing users.
Why it matters
Anyone can list fifty UX nits. The job is finding the two that explain the drop-off, backed by the funnel data supplied.
What good looks like
- Ties each issue to the funnel data
- Prioritises by likely impact
- Distinguishes activation from mere completion
Deliberately not measured
- Accessibility audit completeness
- Visual redesign
Capability tested
Consequential critique
The failure we’re looking for
A generic UX checklist
Grading
Jev + blind human review
Results
Every evaluated configuration on this task, all cases and repeats.
| # | Model · Harness | Task score | Jev | Martin’s | Runs | Critical failures | Cost / run | Latency |
|---|
Case viewer
Read the brief, then compare up to three outputs side by side.
The brief
Review this onboarding flow and funnel. Where are we losing people and what should we fix first?
FunnelSignup 100% → Connect data source 41% → First chart 33% → Invite teammate 9%. Week-2 retention: 62% for users who reach first chart, 11% otherwise.
FlowEleven screenshots of the onboarding flow.
What a strong answer does
The data-source connection step is the activation bottleneck; prioritise a sample-data path to first chart.
Case
v1.0 · anonymised real · B2B SaaS, data