Badges
Certifications
Work Experience
Software Engineer
247lytics• January 2023 - Present
● Specialized in designing and developing end-to-end data pipeline solutions using technologies like Fivetran, Databricks, dbt, and GCP. ● Analyze use cases and client needs to identify suitable data pipeline solutions considering factors like data volume, processing time, and cost. ● Develop performance testing frameworks to evaluate solution effectiveness and optimize for efficiency and cost. ● Collaborate with technical teams to develop, test, and implement data pipeline ensuring Timely delivery, High quality, Smooth integration with existing systems. ● Employed Apache Airflow for orchestrating complex data pipelines and ensuring dependencies are met. ● Used Fivetran for seamlessly extracting and transforming data from various sources like databases and APIs, reducing development and maintenance time ● GCP Dataproc for scalable and efficient data processing using Hive and Spark, allowing for complex transformations and handling large data volumes ● Implemented data quality validation techniques throughout the pipeline, leveraging both Snowflake Data Quality and custom checks. ● Using AWS S3 for cost-effective and scalable data storage. ● Leveraged Apache Kafka for ingesting and processing real-time data streams, enabling near real-time analytics and insights ● Automated data ingestion to Snowflake for centralized data storage and analytics in a scalable and cloud-based environment ● Designed and implemented complex data pipeline automation using dbt with Snowflake, handling large volumes of data from diverse sources. ● Created interactive and insightful dashboards with Tableau to effectively communicate data insights to stakeholders. ● Trained and deployed ML model with GCP's Vertex AI using the processed data in snowflake. ● Proficient in advanced SQL, including analytical functions, materialized views, and stored procedures, ensuring efficient data manipulation and retrieval. ● Experienced in building and optimizing data pipelines by designing complex data workflows, utilizing Snowflake's data loading features (Stages, Streams, Tasks, and Snowpipe) for efficient data ingestion and management. ● Implemented data quality validation throughout pipelines using both Snowflake Data Quality and custom checks, ensuring data accuracy and integrity. ● Leveraged dbt models as Software-Defined Assets (SDAs) within Dagster for centralized orchestration and dependency management of data pipelines. ● Trained machine learning model with XGBoost regression and automated the model training on scheduled basis.
Solution engineer
Nokia• August 2020 - August 2022
● I was part of the 5G Telecom software deployment team (CBIS, CBAM, NCS, NCOM) for Bouygues Telecom in France. ● I participated in configuring hardware prerequisites, including racking and cabling HPE ProLiant DL360/DL380 servers and managing them through HP iLO, verifying and updating server firmware. ● I was involved in the installation and configuration of OpenStack 18/19/20 with CBIS (18.5/19/19A/20) , utilizing NetApp storage integrated through Cinder driver for encrypted high IOPS storage. ● I handled the complete installation, custom configurations, and scheduled health checks for the system. ● I conducted sanity tests, integrated NetApp storage, and performed scaling operations as needed. ● Additionally, I managed controller and compute node replacements, backups, restores, NetApp upgrades, and downgrades. ● I collaborated with clients to complete acceptance testing procedures (ATPs). ● I developed and implemented automated scripts to monitor the daily health of the entire infrastructure. ● I created a procedure for the creation and updating of TLS certificates within web services. ● I actively took part in VNF onboarding, performance checks, and connectivity testing. ● I meticulously documented procedures, conducted regular maintenance, and ensured optimal system health through proactive management of Jira tickets. ● I fostered client engagement by automating integration testing procedures using Ansible and shell scripting. This significantly reduced manual testing time during each sprint and enhanced overall product quality. ● Similar support to NCS deployment and provided troubleshooting for Docker and Kubernetes-related issues. ● Led the efficient installation of CBIS and NCS deployments at multiple sites, consistently completing each within a three-month timeframe, ultimately enabling service delivery to millions of users. ● Leveraged my expertise in Ansible, shell scripting, Docker, Kubernetes, Linux, and Python to automate tasks, manage infrastructure, and troubleshoot issues. ● Utilized Jira for efficient project management and maintained comprehensive documentation for seamless knowledge transfer.
Data engineer
Exusia• June 2019 - May 2020
● As part of the data engineer team, I contributed to designing and developing a data warehouse solution using Azure Data Factory (ADF) and Azure Synapse Analytics for a supermarket chain in Dubai. ● This involved tasks like configuring data pipelines, transforming data using Spark jobs in Databricks, and implementing security measures to ensure data integrity. ● The data warehouse was designed to consolidate and integrate data from various sources (Oracle, MySQL, CosmosDB) for analytical purposes such as reporting, forecasting, and trend analysis. ● I played a key role in optimizing data pipelines, resulting in significant performance improvements. ● Implemented data validation rules within Azure Data Factory (ADF) to enforce data format, content, and consistency checks at the point of data ingestion. ● Used data profiling tools with Azure Data Catalog to identify data characteristics, anomalies, and potential data quality issues. ● Developed techniques like completeness checks, data type validation, and outlier detection to identify and clean inconsistent or invalid data. ● Employed Azure Data Factory Lineage to track the flow of data through the data pipelines, facilitating the identification of potential bottlenecks or data quality issues at any point in the pipeline. ● Leveraging my analytical skills, I also actively contributed to the analytics team and utilized my expertise in complex SQL queries to extract insights from the data warehouse. ● These insights were then translated into visually compelling dashboards built using Tableau, enabling stakeholders to identify trends and make informed data-driven decisions regarding the supermarket chain's operations. ● Achieved a significant development time reduction of nearly 60% compared to traditional approaches, by utilizing the pre-built connectors and functionalities offered by ADF and Azure Synapse Analytics. ● Implementing efficient data transformation methods within Spark jobs, Focusing on modular design and code reusability. ● By leveraging insights from the data warehouse, the supermarket chain was able to optimize operations, resulting in enhanced decision-making and improved business performance.
Education
Clark University
Computer Science, MS• August 2022 - December 2023
NIT, Agartala (National Institute Of Technology)
Computer Science, B.Tech• June 2016 - May 2020