Gopal G.

Gopal G.

Data Engineer

Top performer 9/10
India
Trusted member since 2024
8 years of experience

His proudest achievements include creating and optimizing ETL/ELT pipelines across multi-cloud environments. Gopal's Google Cloud, AWS, Microsoft Azure, and Snowflake certifications highlight his commitment to continuous learning and professional excellence.

He holds a Master's degree in Computer Engineering.

Main expertise

DatabricksDatabricks1 year
Fact Data ModelingFact Data Modeling8 years
ETLETL8 years
Unix shell7 years
15+

Experience4

Nissan Motor Corporation

Data Engineer

Nissan Motor Corporation
Automotive
Nov 2023 · 2y 11m
  • Designed and implemented efficient and scalable data pipelines on Google Cloud Platform (GCP) to collect, process, and transform raw data into usable formats for analysis and consumption;
  • Led and managed offshore teams to successfully implement various data engineering tasks, ensuring alignment with project goals and maintaining high-quality standards through regular communication, clear documentation, and effective task delegation;
  • Oversaw governance and compliance of data stored in BigQuery, ensuring adherence to UK and EU GDPR regulations;
  • Conducted Data Privacy Impact Assessments (DPIA) for various projects at Nissan UK Limited and implemented necessary measures to mitigate or reduce risks;
  • Built and maintained data warehouses, data lakes, and data lake houses on GCP using services such as BigQuery, Google Cloud Storage (GCS), and Bigtable;
  • Integrated data from various sources into GCP using services like Cloud Storage, Cloud Pub/Sub, and Cloud SQL;
  • Implemented proper data governance and security measures using GCP Identity and Access Management (IAM) and Data Loss Prevention (DLP) to ensure compliance;
  • Built data pipelines using Google Dataflow to efficiently handle large volumes of data;
  • Implemented ETL/ELT processes to extract data from various sources and load it into data warehouses or data lakes;
  • Developed streaming pipelines for real-time data ingestion utilizing Kafka and Kafka Connect;
  • Implemented Python-based transformations and BigQuery procedures, orchestrating their execution seamlessly using Google Cloud Composer;
  • Engineered data transformations using Apache Beam, optimized for peak performance on Google DataProc clusters.
Google CloudGoogle Cloud
PandasPandas
AWS AthenaAWS Athena
BigQueryBigQuery
Data AnalyticsData Analytics
7+
Technovert

Lead Data Engineer

Technovert
Information Technology (IT) and Services
Nov 2020 - Jun 2023 · 2y 7m
  • Developed ETL processes using Python and SQL to transform raw data into usable formats and load them into BigQuery for analysis;
  • Built and architected multiple data pipelines, managed end-to-end ETL and ELT processes for data ingestion and transformation in GCP, and coordinated tasks among the team;
  • Designed and implemented data pipelines using GCP services such as Dataflow, Dataproc, and Pub/Sub;
  • Migrated Oracle DSR to BigQuery using Dataproc, Python, Airflow, and Looker;
  • Designed and developed a Python ingestion framework to load data from various source systems, including AR modules, inventory modules, files, and web services, into BigQuery;
  • Developed pipelines to load data from customer-placed manual files in Google Drive to GCS and subsequently to BigQuery using BigQuery stored procedures;
  • Participated in code reviews and contributed to the development of best practices for data engineering on GCP;
  • Implemented data security and access controls using GCP's Identity and Access Management (IAM) and Cloud Security Command Center.
OracleOracle
Azure Data FactoryAzure Data Factory
PandasPandas
ClouderaCloudera
BigQueryBigQuery
8+
Accenture

Data Engineer

Accenture
Information Technology (IT) and Services
Mar 2019 - Nov 2020 · 1y 8m
  • Designed and implemented Snowflake data warehouses, developing schemas, tables, and views optimized for performance and data accessibility;
  • Extracted data from Oracle databases, transformed it into CSV files, and loaded these files into a Snowflake data warehouse stage hosted on AWS S3, ensuring secure and efficient data transfer and storage;
  • Created and utilized virtual warehouses in Snowflake based on business requirements, effectively tracked credit usage to enhance business insights and resource allocation;
  • Designed and configured Snowpipe pipelines for seamless and near-real-time data loading, reducing manual intervention and enhancing data freshness;
  • Parsed XML data and organized it into structured Snowflake tables for efficient data storage and seamless data analysis;
  • Designed and implemented JSON data ingestion pipelines, leveraging Snowflake's capabilities to handle nested and complex JSON structures;
  • Designed and deployed Amazon Redshift clusters, optimizing schema design, distribution keys, and sort keys for optimal query performance;
  • Leveraged AWS Lambda functions and Step Functions to orchestrate ETL workflows, ensuring data accuracy and timely processing;
  • Created and maintained data visualizations and reports using Amazon QuickSight to facilitate data analysis and insights.
OracleOracle
Data AnalyticsData Analytics
TableauTableau
Fact Data ModelingFact Data Modeling
Tech Mahindra

BI Consultant, General Electric

Tech Mahindra
Information Technology (IT) and Services
Aug 2016 - Mar 2019 · 2y 7m
  • Designed and implemented Teradata packages to facilitate seamless data extraction, transformation, and loading (ETL) operations from diverse sources into data warehouses;
  • Developed interactive and dynamic reports using SSRS, providing stakeholders with timely and insightful data visualizations for informed decision-making;
  • Conducted rigorous data validation and quality checks to ensure the integrity and accuracy of processed data;
  • Optimized ETL performance by employing advanced techniques, resulting in a 25% reduction in processing time;
  • Developed the ingestion strategy for loading data from multiple source systems to the operational layer in the data warehouse using Python, SQL, and stored procedures;
  • Understood and developed design documents as deliverables for the project;
  • Implemented SCD Type 1 and Type 2 functionality and developed custom scripts in Teradata for integration and functionality development for different modules like Primavera P6 and Oracle Project module;
  • Managed and troubleshot issues as a DWH analyst to ensure the smooth flow of business operations;
  • Prepared unit test cases and performed end-to-end integration testing;
  • Actively participated in design discussions and reviewed solutions;
  • Participated in peer review discussions on development before moving to higher environments;
  • Loaded data from multiple files to a single target table using ODI variables;
  • Configured and developed ETL mappings to load data from XML and complex (unstructured/semi-structured) files;
  • Utilized Power BI to design and develop insightful visualizations and interactive dashboards, enabling data-driven decision-making for stakeholders and enhancing overall data engineering solutions.
OracleOracle
Data AnalyticsData Analytics
XMLXML
TableauTableau
Fact Data ModelingFact Data Modeling

Engineering excellence

All Data Engineers who have applied to Proxify are scored from 0 to 300 on engineering excellence, one of the five parameters we evaluate. This score reflects engineering excellence only, based on interviews, take-home assignments, live coding sessions, and/or on-the-job performance reviews. The curve shows how all evaluated Data Engineers are distributed across that range, where our acceptance threshold for this parameter sits, and where Gopal stands.

050100150200250300Engineering excellence scoreShare of engineersmedianmeanProxifyacceptancethreshold
Gopal
Score 219 · Top 3% of engineers

Certificates 3

Databricks, Inc.
Databricks Certified Data Engineer AssociateDatabricks, Inc.

Issued Jan 2025 - Expires Jan 2027
Credential ID 130232779

DatabricksDatabricks
Oracle
Autonomous Database Cloud 2021 Certified SpecialistOracle
Databricks, Inc.
Databricks Certified Data Engineer AssociateDatabricks, Inc.

Issued Jan 2025 - Expires Jan 2027
Credential ID 130232779

DatabricksDatabricks
Do you want to know more about Gopal’s certifications?Book a call

Education

University of West London
University of West London
Computer Software Engineering2022 - 2023
JUO
Jawaharlal university of Hyderabad
Electronics and Communications2012 - 2016

Stop browsing.
Get matched faster.