A/B testing campaign messages

Campaign steps can carry multiple message variants. Crispy splits new enrollments across the arms and tracks results per variant.

Managing variants

Use 'manage_variants' with a campaign_id, an action, and a step_index (the 0-based index of the step you are testing):

• 'list' returns the variants on that step
• 'add' adds an arm: pass a short label (e.g. 'B') and a message_template. Templates support {first_name}, {company}, and the other variables. Add a subject for InMail steps.
• 'remove' retires an arm by label. New enrollments stop receiving it, and leads already assigned to it fall back to the step's base template. Historical assignments are preserved.

When you add arms, sending limits are split evenly across them.

Sequence vs follow-up steps

Campaigns have two step arrays: 'sequence' (pre-send) and 'follow_up' (post-acceptance messages). manage_variants targets 'sequence' by default. If your variants live on follow-up messages, which is the common case after applying a sequence template, pass target 'follow_up'. If you list variants and get an empty result, Crispy probes the other array and hints where your variants actually are instead of returning a silent empty list.

Reading results

'get_variant_stats' returns per-step, per-arm counts: sent, accepted, replied, positive_replied, and positive_reply_rate. A reply counts as positive when the contact was tagged auto:interested or auto:wants_meeting. Results are empty until leads are actually enrolled, so do not be surprised by zeros right after launch.

Practical advice: let two arms reach meaningful volume before adding a third. Two arms with real volume beat five arms with noise.