AWS Marketing Team is hiring a Sr. Big Data Engineer to build a flexible and scalable data brokerage (ETL and Data Integration) service which is capable of (i) efficiently bringing data from multiple data sources, (ii) transform and process large data-sets in real time, and (iii) supports real-time data sync data between two heterogeneous data sources. This platform will be used by multiple teams within AWS Marketing like Central Business Intelligence team, Data Science team, Marketing Automation team for their data processing and integration needs. Role and Responsibilities · Build a long-term ETL Infrastructure roadmap, perform POC (proof of concepts), choose the right tools and technologies to be used. · Build a platform to run micro ETL batches to process data-sets in near real time. · Move existing ETL jobs from traditional data warehouse processing to the big data processing platform, ensure that the jobs are designed to scale. · Setup and maintain Data Lake for AWS Marketing team
· Hands-on experience working with data processing platforms like Hadoop, Hive, Spark, AWS Kinesis and AWS Lambda. · Experience with programming languages like Python or Java. · Deep understanding of database and analytical technologies in the industry including MPP databases, NoSQL storage, and Data Warehouse design. · Knowledge of SQL and Unix/Linux. · Bachelors degree in CS or related technical field. · 7+ years of relevant experience with ETL, data modeling, and business intelligence architectures.Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability /