E-commerce Behaviour Analytics

01 / Context

The problem

The raw behaviour data was large enough that naive transformations would create unnecessary network traffic. The analysis also needed to identify where users dropped out across the conversion journey and prepare a scalable recommendation input.

02 / Approach

What I did

• Cleaned and aggregated approximately two million user-behaviour records with Spark RDD and DataFrame APIs.
• Built multi-dimensional aggregation and conversion-funnel analysis across five stages from browse to add-to-cart and checkout.
• Reduced shuffle overhead by roughly 30% using broadcast variables and deliberate partitioning.
• Constructed a user-item click matrix with CoordinateMatrix for collaborative-filtering recommendations.

03 / Outcome

What came out of it

• Located the principal drop-off points in the five-stage conversion funnel.
• Improved processing efficiency through broadcast-variable and partitioning decisions.
• Produced a scalable user-item representation suitable for recommendation modelling.

04 / Artifacts

Reports & code