Retail Rocket Growth Hacking Team At Work: Tehnosila Case Study

Almost two years ago, we at Retail Rocket were working with trigger e-mails for e-commerce. During this time, we brought a lot of e-shops which employed various means, from a primitive “abandoned shopping cart” to advanced automation projects for e-mail marketing.

And at some point we realized that it is not enough to just run automatic e-mail campaigns, because here, as with any other marketing tool, there is a huge scope for optimization. Just think, have you ever tried to run a simple test? For example, check when it is more effective to send “abandoned shopping cart” e-mail — after 30 minutes or an hour? What is the title? Text? Call to action? And how many more ideas for the test came in your head right now? Lots!

That is why growth hackers joined our team — men with knowledge at the intersection of marketing, mathematics, programming, and project management. They are responsible for the continuous work to improve the efficiency of trigger e-mails for our customers.

Now let’s talk about how we do it. A standard A/B test was employed: all recipients of an e-mail campaign are randomly divided into two segments in real time. The “A” segment is sent to the original version of the e-mail, and the “B” segment is sent an e-mail with a change, which should increase the effectiveness of the campaign. By achieving statistical significance of the experiment in a number of KPIs (Open Rate, CTR, conversion, unsubscribe rate, revenue) we select a winner e-mail, which in the future will be the base for the new hypotheses tests.

Today we want to share with you how the growth hacker work is built. As an example, we will take of one of our clients — an e-shop of home appliances and electronics.

About the company “TEHNOSILA”
For more than 20 years “TEHNOSILA” has been one of the leaders of the Russian market for home appliances and electronics. About one and a half million customers a month visit almost 100 “Tehnosila” stores in 59 cities of the Russian Federation, and more than
5 million users visit its website.

Case 1: Adding a personal horizontal menu

Description
In the trigger “abandoned category view”, the e-mail user gets popular products from the categories in which the user showed interest (where they have opened a product page and but have not bought anything during the visit).

Hypothesis
If we mention in the e-mail product categories with which a user has interacted during his visit, in the horizontal menu in the header of the e-mail, that will allow us to make the e-mail content more relevant to the interests of the user, and that will lead to an increase in the main KPI: CTR (click through rate) and CTOR (click to open rate).

The appearance of e-mails:

Basic e-mail

 

E-mail with product categories menu

Result

More than 4,000 people were involved in the testing of this hypothesis. The test produced the following results:

	CTR	CTOR
Growth	+21.8%	+18.9%

Thanks to personalized product menus, the CTR increased by 21.8% at constant conversion, which led to a noticeable increase in sales. In addition, the CTOR rate increased to 18.9%. The CTOR growth indicates that recipients began to make more transitions from the same e-mail as the user repeatedly returns to it, and that indicates the rise in the value, or benefits of e-mail. The statistical accuracy of the results is 95.5% and 97.7%, respectively.

Case 2: Call-to-action in the product cards

Description
In an alternate version of the “abandoned product page”, trigger e-mails were involved; the visitor gets an e-mail with the products, which he viewed during a visit to the site, in case he did not add anything to the cart and has not made any purchases during the visit. We changed the product cards design.
This type of trigger e-mail allows you to save almost everything that was interesting to the visitor, so that the visitor does not have to look for the products he was interested in again, in case he needs one of them.

Hypothesis

• New placement of the text link (above the picture) with the name of the product is a great call to action; the underlining of the link clearly shows the possibility of visiting the product page at the website.
• Displaying a call to action as a button such as “Buy” in official e-shop style helps to use the user experience gained during his visit to the website.
• Adding a call to action link to “Product details” helps to involve users who do not have enough interest in your product.

The appearance of product cards in different segments of the e-mail:

Basic e-mail	E-mail with new product card

Result

The test included more than 18,000 people and we got the following results:

	CTR	CTOR	Conversion Rate
Growth	+15.6%	+27.5%	+57.4%

Implementation of an alternative product card design increased KPIs like CTR, CTOR, and the conversion rate by 15.6%, 27.5%, and 57.4%, respectively. Statistical significance of the test results is 99.8%, 99.9%, and 97%, respectively.

Case 3: The presence of discounts and motivation title

Description
The Retail Rocket platform is able to predict the next most likely customer purchase (what the person will need and when the person will need it) and to determine the right moment to offer such products. For the “predictive” e-mail, we use a special script, which is called the “Next Best Offer”.

In “predictive” e-mails, we tested various motivating titles, and a discount coupon in a separate segment of the e-mails.

Hypotheses
The presence of an exclusive discount that stimulates repeat purchases in the header of an e-mail should have a positive impact on key KPIs.

The title of the e-mail in the control group A:	The title of the e-mail in the control group B:
“Just look at this!”	“Special discount for you on your next purchase”

The availability of accentuated information about the possibility of a discount on the next purchase, located in the triggering text and title of the e-mail with product recommendations, also should have a positive impact on performance KPIs.

The triggering text in group A template:	The coupon and triggering text in group B template:

Recommendations block title in group A template:	Recommendations block title in group B template:

Result

More than 54,000 people took part in the test, with the following results:

	Unsubscribe rate	Open Rate	Conversion Rate
Result	-48%	+20.2%	+428%

The complex progression of changes led to the following results:

• The unsubscribe rate (the number of individuals who unsubscribe from your list in response to each mailing sent) decreased by 48%.
• The open Rate rose by 20.2%.
• The Growth Conversion Rate was 428%.

The statistical accuracy of the test results was 99.9%, 100%, and 99.3%, respectively.

Despite the discount, which, of course, is a significant motivation for the purchase, and takes over part of the net profit, KPI ARPU (average revenue per user, which takes into account the net profit) retains the allowable parameters for the channel.

Case 4: Product reviews in e-mail

Description
Using customer feedback leads to an effect of social proof, which stimulates the user to commit the conversional action.
The alternative version of “abandoned shopping cart e-mails” has been tested.
The user receives an “abandoned shopping cart” reminder whenever he adds a product to the cart, but for some reason does not finish his purchase.

Hypothesis
Availability of product reviews should have a positive impact on all the key KPI of the e-mail.

Basic e-mail

E-mail with product reviews

Result

The test included more than 3,000 customers, and the results were:

	CTR	CTOR	Conversion Rate
Growth	+10.6%	+36.1%	+12.5%

According to test results, the introduction of the block “product review” in the “abandoned shopping cart” increased the CTR, CTOR, and CR 10.6%, 36.1%, and 12.5%, respectively.

The statistical accuracy of the test results for the CTR and CTOR were 91.4% and 100%, respectively. Conversion Rate KPI currently has not received enough data to achieve statistical reliability, but preliminary results allow us to talk about the unconditional positive dynamics.

Case 5: Intriguing calls to action

Description
To test the effectiveness of the introduction of an intriguing call to action in the e-mails, a script trigger for a “post-transactional e-mail” was involved. Once a user completed a purchase, we sent him an offer with related products based on his previous purchases.

Hypothesis
Alternative Call-to-Action: “Check the Price” will add to user experience some elements of gamification and in the end will lead to an increase in KPIs: specifically, the CTR and CTOR.

Basic e-mail	E-mail with “game character”

Result

More than 12,000 people took the test and it produced the following results:

	CTR	CTOR	Conversion Rate
Growth	+54.7%	+63.7%	+20%

According to the test results, the introduction of an alternative Call-to-Action element increases the CTR, CTOR, and CR 54.7%, 63.7%, and 20% respectively.

The statistical accuracy of the test results for the CTR and CTOR is 100%. The conversion rate currently has not received sufficient data for statistical significance, but preliminary results suggest the unconditional positive dynamics.

Summary

Obviously, the trigger e-mail optimization is a point of additional sales growth; it allows a significant increase in the impact of the trigger e-mail campaigns. Retail Rocket provides such services to all enterprise-customers at no additional charge by its internal team of growth hackers.

Comments from “Tehnosila”

Today, everyone understands that the “abandoned shopping cart” trigger works well. However, to have just a simple trigger e-mail campaign is not enough; without fully automated e-mail communications, marketing today is not effective. Not many players are seriously engaged in optimizing the efficiency of e-mail campaigns — simply they do not have time and resources for such work.

In addition to the introduction of an advanced trigger e-mail platform, Retail Rocket colleagues have done, and continue to do excellent work in the direction of growth hacking for our e-mail campaigns; we are pleased with the result and look forward to even more, similar results in the future.

Maxim Sheluhanov, Head of e-commerce “Tehnosila”