Microsoft envisions a world where passionate innovators come to collaborate, envisioning what can be and taking their careers places they simply couldn't anywhere else. This is a world of more possibility, more innovation, more openness, and sky's-the-limit thinking - a cloud-enabled world. Our mission is to empower every person and every organization on the planet to achieve more. This mission is bold and at the core of what our customers and employees care deeply about. We have unique capability in harmonizing the needs of both individuals and organizations. We deeply care about taking our ideals and vision global and making a difference in lives and organizations in all corners of the planet. We are always learning. Insatiably curious. We lean into uncertainty, take risks, and learn quickly from our mistakes. We build on each other's ideas, because we are better together. We stand in awe of what humans dare to achieve and are motivated every day to empower others to do more and achieve
Act as technical lead of Open source technologies such as Spark, Scala and other Apache foundation open source platforms. Design and build a high performing data lake that can store multi-petabyte data in Azure cloud. Design and build or use available data ingestion pipelines to feed the raw data efficiently into the data lake. Build ETL/ELT processes to process raw ingested data ready for Machine Learning [ML] and analytics layer Create Data Quality process, and reporting systems to publish the quality of the data in the lake Create and maintain security roles and rule to administer the data lake
Basic Qualifications: Bachelor's Degree in Computer Science, Engineering 5+ years of cod ing, architecture, design, implementation of highly distributed applications (i.e. having an architectural sense for ensuring availability, reliability, etc.) Preferred Qualifications: 1 0 + years software development experience 2+ years' experience in managing small to medium size internal teams and vendor resources Deep technical experience in one or more of the following areas: Hadoop, Sqoop, Kafka, Spark and ser verless design concepts Consistent track record of designing and building large data lake solutions