At FARFETCH, one of our values is "amaze customers” and we are always looking for ways to be closer to our customers, whether it’s through social media or email marketing. Every day, we work on making our email marketing process more agile and cost-effective for the responsible teams, allowing them to dedicate their time to other tasks. In this article, I will provide an overview of the solutions we have built to automate email lists for our email campaigns that need to be sent out on an hourly basis using BI Data. 

Our first case was to identify which users had failures during the purchase process, either from incorrectly entered information or because something went wrong with the card operator.

The marketing team uses the Oracle platform called Responsys. With the knowledge of the API, it was possible to carry out the automation of all these processes, making them more agile and safe.

We have automated the creation of lists for sending emails within the Oracle Responsys platform. Through technology, we have developed our system that has allowed the process to be more agile, allowing us to follow all the processes that are executed until the data is ready to be sent.

Oracle Responsys makes data from disparate sources actionable by creating precisely targeted audiences and delivering highly personalized messages to those audiences in near real-time.

Among the various technologies available, we chose to develop with the following tools: Ruby/Rails/Sidekiq/Redis/Gush, as they have already reached a degree of maturity over the years and have proven efficient for our task automation process.

As a team, we were also motivated to challenge ourselves to learn new technologies that we had not worked with before. 

Profile List Extension - From the profile list (contains the primary information of the user) you can create additional fields for these users.

1. We create a set of tasks that are executed every hour. The first task was to create a task for the login process, in which we store the information returned by the Responsys API inside the Redis, all our keys have the lifetime of one day after their creation. The token has an expiration time every 2 hours, we implement the refresh token, from an existing token, send it to the endpoint and receive a new valid token, without the need of authorization through login/password, and update our existing key with the new token.

2. At each execution it was necessary to clear our entire profile list extension. Our first implementation was the Payment Failed list, containing the information of failures during the payment process of orders placed at FARFETCH Brazil. In it, we made the match in payment_failed and sent in batches the recipients that need to be deleted from the API of Responsys, therefore avoiding that the user stays in the list and receives the same email again.

3. At this moment we need to access the BI data, where the payment information is contained. Here we do the mapping of the database table to which we want to obtain the information. The data received is normalized and persisted in Redis.

4.  Before this list is filled, we need to map which fields we should create, having the schema ready, we send it to the Responsys API. If the list is not yet created, the job will execute the creation, otherwise, nothing happens.

5. This is one of the most important points because we need to know which is the unique identifier (RIID) of each user, which allows us to track the user inside the marketing tool. We need to process all the users that were inserted in the last hour. To do this, we match the data stored in the Redis and get all the necessary emails. Then we send this data to the Responsys, and with the data returned we update each record of our Redis with the respective RIID.

6. When we send a user who does not have RIID to the Responsys, an error is returned in the call. To avoid this, we make a treatment in the local registers, keeping only the registers that are with the RIID filled, the others are removed.

7. With all the records properly filled in Redis, we create our JSON based on the fields that were mapped for the creation of the list, this step consists of inserting all the fields in the Responsys. Those fields will be used for the creation of the email.

8. As a way to generate data, every day we group all the values that stopped being sold because of some error that happened in the transaction processing.

We need all of these tasks to be executed in a sequential way, with each step of the process following the previous one.  In order to achieve this, we use a tool called Gush that has an easy integration with Sidekiq, allowing us to write a simple workflow for scheduling and executing tasks.

We were able to easily incorporate into our system any information that is related to the creation of a mailing list, using as a basis the tasks described above. We also implemented another group of tasks that we named Checkout, responsible for sending emails to users who ended up forgetting something in the shopping cart. With this, we opened a range of email campaigns that can be automated, allowing greater agility and efficiency in the process.