Traditional data storage and analytics tools can no longer provide the agility and flexibility required to deliver relevant business insights. With the volume of data generated being large, separating storage and compute-enabled scaling of each component is required.

The question that arises here is how data analytics plays a pivotal role in helping out these companies. Most of the companies assess their territory by drawing insights from the following:

• Kind of products customers like, their preferences towards a certain kind of product, based on their location, time of the year, the changes in their likes and dislikes on the basis of the quality, price, value, lifestyle and class quotient of the product
• Why a product has higher sales, as compared to others, what products are complementary and which ones can be substituted on the basis of the customer response
• Data collected from the hundreds and thousands of vendors out there and this data analytics is then assimilated, in the growth model of the companies

If we take the case of Amazon, Prime Day is its major flagship event where it offers its customers with a variety of options and discount. Customers go into a buying frenzy in these two days. This surge traffic comes with its own set of problems. It is a critical case of infrastructural handling, and Amazon does this with the help of its sister company, Amazon Web Services (AWS). 

It was projected that Prime Day would make $5.8 billion in global sales.

AWS dominates 33% of the cloud infrastructure. On the fly, the cloud computing division leverages its global network of servers — ramping up capacity, as needed — to handle and route the massive spikes in traffic on this big day to the Amazon website.

Running the Amazon website and mobile app on AWS makes short-term, large scale global events like Prime Day technically feasible and economically viable. 

AWS enables customers to add the capacity required to power big events like Prime Day in an elastic and cost-effective manner. All of the undifferentiated heavy-lifting required to create an online event at this scale is now handled by AWS so that the Amazon retail team can focus on delivering the best possible experience for its customers.

What Do The Prime Day Numbers Say

In case of Amazon, the prime day has seen a great swell in numbers:

• Amazon Mobile Analytics events increased by 1,661% compared to the same day the previous week
• Amazon’s use of CloudWatch metrics increased 400% worldwide on Prime Day, compared to the same day the previous week
• DynamoDB served over 56 billion extra requests worldwide on Prime Day compared to the same day the previous week

How AWS Came To The Rescue

A combination of NoSQL and relational databases are used to deliver high availability and consistent performance at extreme scale during Prime Day:

Amazon DynamoDB supports multiple high-traffic sites and systems including Alexa, the Amazon.com sites, and all 442 Amazon fulfilment centres.In fact, during the 48 hours of Prime Day, these sources made 7.11 trillion calls to the DynamoDB API, peaking at 45.4 million requests per second.

Whereas Amazon Aurora, which supports the network of Amazon fulfilment centres processed 148 billion transactions, stored 609 terabytes and transferred 306 terabytes of data.

Prime Day 2019 also relied on a massive, diverse collection of EC2 instances. The internal scaling metric for these instances is known as a server equivalent; Prime Day started off with 372K server equivalents and scaled up to 426K at peak.

The EC2 instances made great use of a massive fleet of Elastic Block Store (EBS) volumes. The team added an additional 63 petabytes of storage ahead of Prime Day; the resulting fleet handled 2.1 trillion requests per day and transferred 185 petabytes of data per day

 On a typical Prime Day, Amazon deploys the following AWS services:

• The analytics end of the business is taken care by services like Amazon Redshift, Amazon Machine Learning.
• Amazon API Gateway, CloudSearch, Data Pipeline, Elastic Transcoder, SES, SNS, SQS, SWF form the crucial application Services 
• EC2, Auto Scaling, EBS, EMR, Lambda are deployed for computing.
• Database management is done with the help of DynamoDB, ElastiCache, Kinesis, Kinesis Firehose, RDS.
• Management Tools – CloudTrail, CloudWatch, Trusted Advisor.
• The Mobile Analytics service plays a key role as most of the orders are made via mobile devices.
• To keep the data secure, CloudHSM, IAM, KMS are used.
• The collected data is stored and delivered with the help of CloudFront, S3, Amazon Glacier

Data Analytics Takes Charge In India

India has become the battleground for one of the world’s largest ecommerce companies.  With the backing of Walmart and its own home-grown technology, Flipkart has been going toe-to-toe with Amazon India. Amazon on the other hand, which is a top player at the international scene, has been pumping money into Amazon India and there looks to be no stopping with an unprecedented swell in the profits. 

To make the most of data, companies like Walmart-owned Flipkart have gone as far as using it for something called the ‘selection insights’, which helps the seller understand consumer demand as well as allowing them to stock products which are extremely popular at a particular point in time. Every single seller now has a personalized, customized portal to help stock up relevant products.

They also make use of geo-coding of addresses using machine learning, artificial intelligence and data science to ensure faster and effective delivery even in remote parts of the country.

Whereas, Amazon’s homegrown services are not only powering top global firms but also are providing huge returns for the e-commerce giant as well. Amazon has excelled at developing technologies that are tailored to tackle specific tasks like data ingestion or security and then bring them all under one roof to give a flawless experience for the customer.

Every city has a certain type of customers which are categorised as Tier I, II and so on based on their spending history, metro or non-metro city etc., Information such as this plays a major role in categorising the products with respect to audience. 

Recommending a widely purchased product by the rich in the metros to a middle class customer in a non-metro city makes no sense. Achieving great accuracies in the way recommendation engines work depends on the quality of data gathered throughout the year running up to the coveted big billion or prime days. 

A lot of these e-commerce companies depend on a bunch of data science teams to achieve such staggering results.

That said, the e-commerce industry occupies a very tiny share of the total retail consumption in India. With a lot of market to tap into, it is very obvious that these online retail giants will use every technology at their disposal.  Moreover, with the rising popularity of algorithmic driven solutions, we can safely assume that machine learning combined with cloud computing will be at the epicenter of this 21st digital disruption.