Arthur J.

Data Engineer

Data Engineer with 6+ years of experience in Python, Apache Spark, Data Engineering, Apache Hive, and ETL.

Arthur is a passionate and result-focused Data engineer with ten years of experience developing solid and reliable data pipelines and BI dashboards and solving problems through data for international companies. He has mainly worked on Big data and Machine learning.

He has a strategic mindset focused on understanding context, testing hypotheses, drawing conclusions based on facts, and establishing a data-driven culture with team members. Analytical and well-organized with strong theoretical engineering and mathematical background, Arthur quickly learns new technologies.

He can play a vital role throughout the engagement's development/support life cycle to ensure quality solutions.

Main expertise

  • Python
    Python 8 years
  • Data Engineering 6 years
  • Apache Spark
    Apache Spark 6 years

Other skills

  • Git
    Git 6 years
  • Scrum
    Scrum 5 years
  • Java
    Java 5 years
Arthur

Arthur J.

Brazil

Get started

Selected experience

Employment

  • Python developer/Data Engineer

    US-based clinical research company - 1 year

    Arthur's team was developing a POC to gather medical data from multiple partners and transform it into a research standard to make it available to their clients. In this project, Arthur built some tools to transform and ingest data using Python and Rust. They used an SQL Server as the database and started to develop scripts to run in Azure.

  • Data Engineer

    Thoughtworks - 1 year 9 months

    • Data migration using Azure Data Factory. Data processing using Apache Spark at Databricks. Processing automation using Python.

    Technologies:

    • Technologies:
    • Azure Data Factory Azure Data Factory

  • I.T. Analyst/Data Engineer

    Grupo Pão de Açúcar - 1 year 5 months

    • Data ETL from Teradata DW using Sqoop on Hive, Impala, and Apache Kudu. Data processing with Apache Spark 2 in a Hadoop environment. Maintenance of legacy systems using Python, Shell Script (Bash), and Java.

  • Data Engineer

    Grupo Pão de Açúcar - 1 year 5 months

    • Data ETL from Teradata DW using Sqoop on Hive, Impala, and Apache Kudu. Data processing with Apache Spark 2 in a Hadoop environment. Maintenance of legacy systems using Python, Shell Script (Bash), and Java.

  • I.T. Analyst/Data Engineer

    Nextel (Stefanini IT Solutions contractor) - 3 months

    • Load data from PostgreSQL using Sqoop, Apache Spark 2, and Python 3. Versioning data on a “snapshot” table with Apache Spark 2.

  • Data Engineer

    Nextel (Stefanini IT Solutions contractor) - 3 months

    • Load data from PostgreSQL using Sqoop, Apache Spark 2, and Python 3. Versioning data on a “snapshot” table with Apache Spark 2.

  • Data Engineer

    Banco Santander (everis & BRQ contractor) - 9 months

    • ETL and data processing in the Hadoop environment using Apache Spark to fill business reports

  • I.T. Analyst/Data Engineer

    Semantix - 1 year 1 month

    • Data analysis using Hive and Impala (Cloudera distribution). Data processing in the Hadoop environment. Automation script development using Python and Shell script. Result of IoT engagements. Real-time batch processing using Apache Spark, Kafka, and Elasticsearch.

  • Data Engineer

    Semantix - 1 year 1 month

    • Data analysis using Hive and Impala (Cloudera distribution). Data processing in the Hadoop environment. automation scripts development using Python and Shell script. Result of IoT engagements. Real-time batch processing using Apache Spark, Kafka, and Elasticsearch.

  • Java Developer

    Stefanini IT Solutions - 6 months

    • Maintenance and development using Hibernate, Git, Maven, Tomcat 7, Oracle 10g, and JSP.

    Technologies:

    • Technologies:
    • Maven Maven
    • Hibernate Hibernate
    • Oracle Oracle
    • Tomcat Tomcat

  • IT Analyst

    CVC Viagens - 8 months

    • Project migration from SVN to Git. Processes Standardization to engagement versioning with automatization using Python and Jenkins. Creation and maintenance of automated tests with Selenium, Python, and Testlink. Automatic monitoring with Python, Selenium, and Zabbix

  • Java Developer

    MAPS Soluções e Serviços - 2 years 10 months

    • Development of mission-critical Java web systems to financial institutions, e.g. Caixa Econômica Federal, with JBoss, Wicket, Hibernate, JUnit, Selenium, continuous integration with Jenkins and Scrum as agile philosophy.

    Technologies:

    • Technologies:
    • Hibernate Hibernate

Education

  • Standalone courseAnalysis & Systems Development

    Faculdade de Tecnologia de São Paulo · 2022 - 2022

  • BSc.Computer and Information Sciences

    Universidade Federal do ABC · 2015 - 2019

Find your next developer within days, not months

In a short 25-minute call, we would like to:

  • Understand your development needs
  • Explain our process to match you with qualified, vetted developers from our network
  • You are presented the right candidates 2 days in average after we talk

Not sure where to start? Let’s have a chat

First developer starts within days. No aggressive sales pitch.