AWS Elastic MapReduce

AWS Elastic MapReduce (EMR) Increased operational efficiency 40% and reduced 34% of cost

Business Need: 

Transportation Industry vertical has a wide range of usage pattern in which business has some peak hours and non-peak hours. Business requirement is to have a specific architecture where it can handle various types of Loads during the day and night. It should provide a facility to scale up and scale down based on the usage of the system.

Solution and Approach:

AWS MapReduce (EMR) provides the feasibility to scale up and scale down the resources based on the CPU/Memory/Requests on the cluster. Based on Minimum and Maximum Servers combination in the EMR configuration, Number of Servers can be directly scaled up or down between the Minimum and Maximum numbers provided during the installation.

With AWS EMR in place, Transportation industry vertical can handle the distinct Loads like Low or High resource consumption data loads. S3 is used as backend storage for EMR clusters, which provides the backup facility by default as it has versioning maintained.

AWS EMR is very fast in scaling up/down as it might not take more than 20 Mins. Resources like CPU/Memory/Compute Power will be used efficiently without the Manual Intervention, with the Architecture shown below.

Benefits:

Administration and Operational efforts reduced more than 40% as compared to the Traditional Warehouse Maintenance.

Cost Optimization has been achieved by adopting EMR, AWS Managed Service as resources will be scaled down during the Idle Time of the cluster. It led to reduce the overall cost of cluster to 34%.

AWS EMR is having MapReduce (Query Engine) as default, which gave 28% faster Analytics when compared to previous way of handling.

Posted by wissenadmin | 11 August 2022
APPROACH & SOLUTION: OwlDQ Web application that can connect to source and destination data stores & run spark-based jobs to compare & score the data. This tool helps business visualize…
21 LikesComments Off on RDS – Performance Improvement & Cost Reduction
Posted by wissenadmin | 11 August 2022
Transportation (Heterogeneous) Industry Vertical made their application availability 100% with 45% Increase in End-to-End Query performance Business Need:  Industries that depend on data extractions from Distinct source like Databases, Sensors,…
20 LikesComments Off on AWS Relational Database Service
Posted by wissenadmin | 10 February 2022
Business need HealthCare is one of the vertical for General Electric. Healthcare system is to bring different type of source data from Hospital equipment which includes sensor data, PHI data,…
30 LikesComments Off on Serverless analytics on AWS