Email Engagement Campaign
December 2024
I completed this project as the Product Designer on an automated communications team for a mid-sized e-commerce company. I partnered with a cross-functional team consisting of a product manager, strategy analyst, front-end developer, and back-end developer.
Background
Customers who canceled their test drive appointment did not receive any follow-up communication outside of this confirmation email (Fig. 1) This means we were potentially losing a significant customer population by not re-engaging them.
More importantly, we were not helping those customers find a car they felt confident about.
​
The goal was to learn what customers needed in this moment to help them take the next step in their car buying journey.
Summary of Insights
Fig. 1 - Appointment Cancelation Email
-
Customers LOVE recommendations, especially for cars that are similar to the one they canceled their appointment for
-
Customers have an idea of what they’re looking for and are motivated to buy, but aren’t sure what to move forward with
-
Customers appreciate helpful content, particularly on our return policy
Methods
-
Fullstory sessions
-
Customer interviews
-
Engagement metrics
ToolsÂ
-
Figma + Internal design system
-
Fullstory
-
Salesforce Marketing Cloud
Strategy
I knew from previous discovery that over 50% of customers canceled their appointment because they changed their mind about the car. But I didn’t know what they needed to feel confident in their next choice. So, I partnered with our strategy analyst to build a 5-variant email test to learn what sort of content resonated most with customers in this moment.
​
I could have started with qualitative discovery by interviewing abandoned customers, but quantitative email testing gave us insight into a much bigger population and allowed us to observe what customers actually did in this moment. This testing-first approach also offered:
-
A quick way to learn: We only needed to weeks of engagement data to learn what resonated and what didn’t
-
Insight into a larger journey: The read from this test would inform our next test, as well as additional emails we would add as part of a larger re-engagement campaign.​​
Constraints
-
Limited resources. Our front-end-developer spanned multiple teams and did not have the bandwidth to build 5 new emails, so we repurposed a set of emails that had already been built.
-
Timing. My PM was facing pressure to move quickly. Despite him wanting to launch this test as soon as possible, I advocated for us to clearly articulate our riskiest assumption and how we intended to validate our hypothesis. I ensured we had a clear learning agenda and test plan so the results would inform our next steps (Fig. 2).
Fig. 2- Experiment Design Canvas
Process
I was responsible for the content and visual design of each variant, including a single subject line and preview header that would work across all 5 emails so as to not skew the read.
​ I was also responsible for:
-
Validating the journey with our back-end developer before it was built in Salesforce Marketing Cloud, including when the emails would send and which customer segments would receive them
-
Pulling UTM links through GitHub to speed up the front-end development process and ensure we could trace customers’ behavior back to the corresponding email variant (Fig. 3)
-
Conducting visual QA on each email to ensure the designs were performing as expected in production
-
Monitoring engagement metrics in Salesforce Marketing Cloud and customer behavior through session replays on Fullstory
Fig. 3 - One step of the UTM link management process
Findings
I facilitated Fullstory watch parties with my team where we primarily observed customers:
-
Scrolling through pages of recommended cars
-
Editing their search criteria and favorited cars
-
Repeatedly canceling their orders and reserving a different vehicle
Engagement metrics* from Salesforce Marketing Cloud also showed:
-
That customers were engaging with the recommendation emails and clicking on more than one car tile in those emails
*Total clicks, unique clicks, click rate, click to open rate, open rate, unsubscribes, and unsubscribe rate.
Key Insights
-
Recommendations are the most valuable type of content for customers who cancel a test drive, but customers can get stuck in an endless scrolling pattern
-
Customers have an idea of what they’re looking for and are motivated to progress, but seem to doubt their choices
-
Customers have already made steps toward financing, so financing options are neither helpful nor relevant
-
Customers are interested in helpful content, and engage the most with our FAQ and 10-day return policy
Impact
Qualitative and quantitative data indicated that customers engaged most with the recommendation email for similar cars, but customers who opened the return policy email spent a lot of time in the FAQ section of our website.
So we moved forward with the recommendations email for similar cars, but I added a component that highlighted our return policy and linked customers to our FAQ page (Fig. 5).
This way, we could guide customers’ next steps while building confidence in the car they chose.
Next Steps
After the demand test, our options were either to move straight into an evidence test that measured progression over 6 weeks, or take 3 weeks to see if a different subject line would drive higher engagement rates.
​
I convinced my product manager that testing subject lines now would give us a better experience later, and we saw a 2.5% increase in engagement with another subject line.
​
The evidence test is currently in production. Early testing shows good but not statistically significant results yet.