Badges
Certifications
Work Experience
Data Engineer
TTEC Digital Analytics India LLP• January 2020 - Present
Reduced the coding time by 30%, by creating a python library and a code & QC frameworks; thus avoiding repetitive codes and logics. | Automated Data Piplines using Auzre Data Factory and Azure Databricks. | Analyzed & Transformed Adobe CDF and Google DCM batch files using PySpark on Azure Databricks, Delta Tables and Blob Store and provided business insights on the Campaign performance by visualizing the cubes on PowerBI Dashboards.
Big Data Engineer
Open Insights Solutions LLP• November 2018 - January 2020
Created batch streaming pipleines using NiFi, Kafka, Scala and Spark on EC2 Cluster as part of POCs. | Designed and created Design Documents, Data Models (Business Semantic Layer (BSL) & Integrated Data Layer (IDL) for Telecom and Banking domains and created synthetic/hybrid data for ingestion & transformation using Spark, Scala, JSON. | Ingested data from structured (csv, RDBMS) and semi-structured (JSON files) sources, transformed the data using Hive and generated PowerBI Dashboards
Software Engineer
Tech Mahindra • August 2016 - November 2018
Designed systems to transform data from different sources in Hadoop Hive on TEZ, MR Engines by using HQLs and Hive UDFs as per telecom business requirements thereby creating various summaries, reports and visualization patterns in Tableau for better business insights. | Worked on various file compression products like Bzip, Snappy parquets to achieve optimization. | Created a near real time model which streams in data in the form of XMLs and parses using Pig UDFs, Hive, Shell Scripting, cURL and Oozie. | Created a complete datalake in Hadoop using various Hadoop components and data warehousing techniques thus making a common data pool from stacks like Single View, Voucher Management Systems, Campaign Management Systems and Siebel CRM systems
Education
Liverpool John Moores University
Data Science, MS• May 2021 - Present
IIIT, Bangalore (International Institute of Information Technology)
Data Science, Post Graduate Diploma• March 2020 - April 2021
Specialization in Data Engineering.
St Claret College
Computer Science, BCA• June 2013 - May 2016
Links
Skills
jobinmadayil has not updated skills details yet.