A/B Testing a Payment MethodLimited pilot
What is A/B testing?
A/B testing allows users to test payment methods on a subset of traffic before turning them on for their entire buyer population. Users can select their desired payment method and percentage of traffic to test, run an experiment, and see the resulting impact on conversion rate, AOV, and shift in volume from other payment methods. After running an experiment, users can decide to roll the payment method out to more buyers or roll it back to none.
Limited Pilot
Currently A/B testing is available in a limited pilot. For early access, contact us.
Why to consider A/B testing
Users often want to first try out payment methods on a subset of their buyer population. They want to use this as a test to understand the impact local payments methods (LPMs) have on their business. They want to know if adding a payment method positively or negatively impacts their conversion rate, average order value, and fees. They also want to understand the impact the payment has on disputes, refunds, and support requests.
Running these types of tests and getting empirical data from experiments can be challenging and time consuming for many users.
We’ve simplified how you can gather this data through experimentation so you can try out different payment methods and understand what effect they have.
Are you interested?
If you’re interested in joining in on the limited pilot, please contact us.
Requirements
Integration Type
We currently offer A/B testing support only for the Stripe Payment Element.
Dynamic payment methods
Next, if you haven’t already, you’ll need to upgrade to dynamic payment methods. Dynamic payment methods allows you to add payment methods—after you turn it on, you can add additional payment methods from the Stripe Dashboard, no code changes required. See the instructions for Connect platforms or the instructions for direct users.
- To make sure you have a consistent control group during your A/B test, before you turn on dynamic payment methods you’ll need to turn off a few payment methods that are on by default if you’re not already using them: Bancontact, EPS, iDEAL, giropay, and PayNow1. You can do this through the Payment Methods Setting page in the Stripe Dashboard.
To see how your payment methods appear to customers, enter a transaction ID or set an order amount and currency in the Dashboard.
If testing Buy Now Pay Later (BNPL) payment methodsRecommended
Install the Payment Method Messaging Element. The Payment Method Messaging Element is an embeddable UI component that helps your customers know which payment options are available at checkout directly from your product, cart, or payment pages. We recommend including the Messaging Element when A/B testing BNPLs2 because BNPLs improve conversion. Businesses that offer BNPL on Stripe have seen a 25% increase in sales3, but that increase often doesn’t materialize if buyers don’t know that the BNPL will be available at checkout.
- Instructions on installing the Payment Method Messaging Element are available here.
1 These payment methods are automatically turned on for eligible buyers based on their geography (for example, PayNow for Singaporean buyers).
2 E.g. Affirm, Afterpay, Klarna.
3 Average results from a Stripe study of businesses who have $100K+ in weekly PIV and who adopt a BNPL after going live on Stripe.
Are you interested?
If you’re interested in joining in on the limited pilot, please contact us.