- Download the base workflow
- The Workflow
- Select Template / Delivery and send proof
- Split test settings
- Create split test
- Segment recipients
- Split test “Prepare”
- Wait for split send time
- Split test “Send”
- Wait X hours before sending final email
- Email split test results
- Send final delivery with winning subject line
- Wait 5 days for final results
- Email final results
The Adobe split test workflow is provided to customers as a starting point for maximising the effectiveness of the awesome Phrasee platform. This starting point will allow you to apply Phrasee recommended split testing methodology to any campaigns you’re planning to use Phrasee optimized language on.
It is a base to incorporate into your existing workflows and should be edited and changed to suit your needs (just make sure that the same methodology is still applied!). This workflow is provided for you as a working example. You’ll still need to test the workflow’s implementation in your own Adobe account, as we can’t cater for the many differing configurations in Adobe.
This document explains all the components of the Adobe split test workflow. It will give anyone using or configuring the workflow an understanding of the concepts used.
Download the base workflow
This is the base workflow and is provided as is for you to use or change to suit your needs.
Phrasee Split Test Workflow
Existing Adobe workflows
We understand that many of our awesome clients already have workflows in place that are used to select audiences and send emails. These workflows normally terminate with one email delivery to a large (300 000+) audience. The idea of this workflow is to incorporate Phrasee into those existing workflows. This can be accomplished by replacing the final email delivery activity with the Phrasee split test activities, from “Select Template / Delivery and send proof” to “Send final results”, to use one example
This split test workflow essentially replaces the final email delivery in any workflow that you want to run a Phrasee subject line optimization on.
Select Template / Delivery and send proof
You’ll select (or create) an email that can be selected from the “Select Template / Delivery and send proof”. This delivery should have all the setting that are needed for this campaign/send, Including proof audience and settings like Delivery.code and pressure settings.
This is important, as the workflow “Duplicates” this delivery, which means all the settings (apart from the label and subject line) for all the deliveries created by the workflow will be the same as the selected delivery. When the workflow is started this activity will also send the proof email to the recipients selected in the proof audience.
Split test settings
This is where you’ll input the subject lines generated by Phrasee for each campaign.
Dates and times
Adding your send dates and time is one of the MAJOR driving factors in Phrasee creating this workflow.
This was done specifically to ensure delivery of your split tests in good time.
All the data processing during audience selection and the preparing of the email in the workflow can be done before the split test delivery send begins. This allows the split test enough time to accumulate statistically significant data – particularly unique opens – in order to select your winning (best performing) subject line. That’s the subject line that’ll be sent to the majority of your audience in the final send, so it’s important to make sure it’s as optimized as possible.
For the purposes of this workflow, the relevant fields are:
FinalSendDate – the date your campaign will be sent
FinalSendTime – the time of day your campaign will be sent
FinalSendWaitTime – the number of hours your split test will run (typically at least 4 hours)
The “finalSendDate”, “finalSendTime” and “FinalSendWaitTime” fields all work in conjunction with the “Wait” activities set before your split test and final send deliveries.
What the workflow code does is calculate the split test send time by subtracting “FinalSendWaitTime” from the “finalSendTime” eg. Setting the “finalSendTime” = “14:00:00” and “FinalSendWaitTime” = 4 would translate into (14:00:00 – 04:00:00 = 10:00:00). This would mean the split tests should start to send at 10 am.
The workflow then calculates the time difference between the current time and the split test and final send time and sets the relevant “Wait” activities to these times. These calculations take the date into account. If the user schedules the day before at 14:00:00, the split test “Wait” activity would be set to wait 20 hours and the final send “Wait” activity would be set to wait 24 hours (to allow for 4 hours of split testing).
Confused yet? The above is a bit complicated, and may require a second reading, but it’ll all make sense, we promise.
What this all means is that you can schedule a delivery and then start your workflow long before your deliveries are meant to be sent, giving the workflow time to process your audience selection and prepare your emails prior to the send time you’ve selected before sending your campaign at the scheduled time. This should significantly improve delivery times of your Phrasee-optimized email campaigns!
Create split test
Duplicating the selected email 10 times and only changing the “Label” and “Subject Line” of the original. The Labels are changed to the name as specified in the configuration document, and the subject lines are used from the “Split test settings” you’ve set. This creates versions <label>_Split_1 to <label>_Split_10 for the split tests and <label>_Split_10 for the final send.
The wait time calculations described above are also contained in this code.
This is a standard Adobe activity. The default settings in the workflow create 10 random split test audiences, each made of of 3% of your recipients. This means the 10 split tests will be sent to a grand total of 30% of your total audience. The remaining 70% of the audience is channelled to the final send, where they will soon be treated to your winning, Phrasee-optimized email subject line!
Split test “Prepare”
For each split test, this activity prepares your emails to be sent to each of your random split test samples. The processes you’ll need here are documented in Adobe’s documentation.
Wait for split send time
This activity pauses the workflow for the calculated amount of time for the split test delivery deployment as described above.
Split test “Send”
This is the best part. It sends all the prepared split test emails
“Wait X hours before sending final email”
This activity ensures that your final send is delayed the calculated amount of time for the final send delivery deployment as described above
Email split test results
Send final delivery with winning subject line
ends your final delivery with your winning, Phrasee-optimized subject line to the 70% of your audience who weren’t involved in the split test from the above code. This is set to “Prepare and start” as this delivery should start sending as soon as possible
Wait 5 days for final results
Standard Adobe wait activity set to wait 5 days.
Email final results
Retrieves the metrics for all the deliveries created by the workflow including the final send metrics and sends an email with these to the specified user.