Shopper Intent Prediction and Causal Inference: Understanding E-commerce Clickstream Data

Shopper Intent Prediction and Causal Inference: Understanding E-commerce Clickstream Data

Introduction:
In the world of e-commerce, understanding shopper intent is crucial for businesses to optimize their strategies and increase conversions. By analyzing clickstream data, researchers have developed methods to predict shopper behavior and make accurate predictions about purchase events. Additionally, the use of doubly robust estimation techniques in causal inference allows for a more reliable analysis of the impact of certain variables on shopper behavior. In this article, we will explore the findings of two separate studies and discuss the commonalities and insights they provide.

Predicting Shopper Intent with Clickstream Data:
One study titled "Shopper Intent Prediction from Clickstream E-commerce Data with Minimal Browsing Information" delves into the prediction of purchase events based on click sequences. The researchers propose two approaches: hand-crafted feature-based classification and deep learning-based classification. Both methods show promising results, even when considering extremely short observation windows. By analyzing the frequencies of specific click sequences and using k-gram statistics with visibility graph motifs, accurate classifications can be achieved. This highlights the potential for predicting shopper intent based on clickstream data alone.

Understanding Trajectories and Classes:
In the same study, the researchers categorize trajectories into two classes: conversion class (C) and non-conversion class (NC). By applying sessionization to the clickstream data, each trajectory is assigned to one of these classes. The analysis reveals that a significant majority of trajectories belong to the NC class, while a small percentage represents the C class. Interestingly, when examining the entropy rate of the click sequences, it becomes evident that higher order statistics are necessary for accurate predictions. The relative abundance of each symbol is not sufficient on its own, emphasizing the importance of considering the sequence as a whole.

Insights from Horizontal Visibility Graph Motifs:
The study also introduces the concept of horizontal visibility graph motifs (HVGm) to describe the structure and dynamics of time series data. These motifs offer a combinatorial perspective on analyzing clickstream data. By associating motifs with the probability of their appearance, researchers can compute the entropy of the HVGm profile. This measure provides insights into the representation of different motifs within the click sequences. However, it is essential to note that an overrepresentation of a specific motif may not necessarily be a reliable indicator of shopper intent.

Actionable Advice:

1. Incorporate higher order statistics: When analyzing clickstream data for shopper intent prediction, consider the frequencies of specific click sequences and calculate k-gram statistics. This will provide a more comprehensive understanding of shopper behavior and improve the accuracy of predictions.

2. Utilize personalized recommendations: Customers who end up making a purchase often have a prior idea of what they want. Take advantage of personalized recommendation systems to enhance the shopping experience and guide customers towards their desired products.

3. Implement a combination of classification algorithms: Experiment with different classification algorithms, such as logistic regression, random forest, support vector classifier, XGBoost, and neural networks. Each algorithm may have strengths and weaknesses, so incorporating a combination of models can lead to improved prediction accuracy.

Causal Inference and Doubly Robust Estimation:
In the study titled "Doubly Robust Estimation - Causal Inference for the Brave and True," the focus shifts towards causal inference in the context of participation in randomized opportunities. The researchers introduce the concept of doubly robust estimation, which combines propensity score and linear regression to mitigate the reliance on either method alone. By accounting for the selection bias in participation, doubly robust estimation allows for more accurate estimation of causal effects.

Understanding Participation Bias:
The study acknowledges that although the opportunity to participate may be random, the actual participation is not. The multiplication by the inverse of the propensity score selects only the treated individuals, while the residual of the treatment effect on the treated has a mean of zero. This highlights the importance of accounting for participation bias when analyzing the impact of variables on shopper behavior.

Insights from Doubly Robust Estimation:
Doubly robust estimation provides a robust approach to causal inference by incorporating both propensity score and linear regression. By combining these two methods, researchers can obtain more reliable estimates of causal effects, even in the presence of selection bias. This approach proves valuable in understanding the true impact of variables on shopper behavior and can guide businesses in optimizing their strategies.

Conclusion:
Predicting shopper intent based on clickstream data and understanding causal effects in e-commerce are vital for businesses aiming to optimize their strategies. By analyzing click sequences, applying higher order statistics, and utilizing classification algorithms, accurate predictions can be made. Additionally, by employing doubly robust estimation techniques, businesses can obtain reliable estimates of causal effects and make informed decisions. Incorporating these insights and actionable advice can lead to improved conversion rates and enhanced customer experiences in the world of e-commerce.