Big data developer
Project is a business intelligence type of solution dealing with data from different sources: logs, site visitors, CRM lists or purchase history data, etc. An EMR based solution is used to analyze and develop person-based marketing campaigns that can be deployed to any of the media platforms.
AWS: S3, EC2, Elastic Beanstalk, Aurora RDS, EMR
DB: MySQL (using Aurora DB from AWS)
Hadoop: Spark on top of AWS EMR
ETL: Apache Nifi
correction: AWS: S3, EC2, SQS, Aurora RDS, EMR
Experience as a Data Engineer
Knowledge of database design
Experience in Data warehousing, ETL, Data Quality, batch and stream processing.
Knowledge of Big Data stack - Hadoop, Hive, Spark, Kafka, etc.
Strong knowledge of SQL
Strong knowledge of object-oriented programming and design patterns
Knowledge of Scala, or proficient in a typed or functional language
Knowledge of NoSQL databases
Bachelor or Masters degree in Engineering/Computer Science/Math or related discipline
Experience with Google Cloud, AWS or Azure
Experience with Big Query
Experience with Kubernetes
Experience with Scrum
Experience with BDD
Knowledge of Machine Learning