Data Engineer

• Etablerte grunnlag for en dataanalyseplattform ved hjelp av Airbyte, Google BigQuery, Apache Airflow, DBT og Looker;
• Konfigurerte Airbyte for strømlinjeformet datainntak.
• Anvendte Google BigQuery som det sentrale datalageret;
• Integrerte Apache Airflow for å automatisere og administrere dataarbeidsflyter;
• Anvendte DBT for å skape og vedlikeholde en strømlinjeformet datamodellerings- og transformasjonsprosess;
• Integrerte Looker for datavisualisering;
• Feilsøking som støttes og tekniske problemer.

Joined the data team at its inception, working on building the data platform from scratch. This included selecting the technology stack, setting the foundations, and applying best practices to deliver a platform that drives data-driven initiatives and supports business operations.

• Established Google BigQuery as the core data warehouse, optimizing data storage and retrieval.
• Designed and deployed an ELT pipelines using Airbyte on Kubernetes, facilitating seamless data extraction from sources like PostgreSQL and MongoDB, and leveraging CDC for efficient updates.
• Utilized Airflow for orchestrating jobs and dbt/Dataform for data transformations, ensuring smooth workflow automation.
• Automated CI/CD processes with GitHub Actions and Google Cloud Build, enhancing deployment efficiency.
• Implemented comprehensive data quality checks to ensure data integrity and governance.
• Leveraged Looker for business intelligence and reporting, delivering actionable insights to drive decision- making.

• Bidro til utviklingen av en maskinlæringsplattform og dataanalyseplattform ved bruk av Google Cloud Ecosystem, inkludert BigQuery, Apache Airflow, Pub/Sub og Google Analytics;
• Anvendte BigQuery for effektiv datalagring og analyse;
• Integrerte Apache Airflow for å automatisere og administrere dataarbeidsflyter;
• Implementerte Pub/Sub for pålitelige meldinger og datastrømming i sanntid;
• Innlemmet Google Analytics for å samle inn innsikt fra brukerinteraksjoner;
• Jobbet med å optimalisere plattformytelse og skalerbarhet.

• Bidro til utviklingen av en Big Data-plattform innenfor Cloudera-økosystemet, med Apache Spark, Hadoop, Apache Kafka , Apache Flume, Apache Hive og Impala;
• Designet robuste og skalerbare datainntaksløsninger og pipeliner;
• Formulerte og automatiserte forbehandling og inntak av databatcher og -strømmer.

• Led the development of an adaptable video processing and analysis platform, enabling seamless integration of numerous Artificial Intelligence and Computer Vision algorithms for efficient real-time video analysis;

• Spearheaded the design and upkeep of the platform's pipeline, leveraging significant big data tools including Apache Storm, gRPC, RabbitMQ, Redis, and Kubernetes;

• Contributed to the creation of a comprehensive Data warehouse aggregating data from diverse sources;

• Conducted data analysis by executing intricate SQL Ad-Hoc Queries in response to business requirements from multiple origins;

• Developed SQL procedures to automate ETL tasks and workflows;

• Managed the complete data warehouse lifecycle, ensuring its continued functionality and relevance;