The Commerce + Ecosystems organization is all about making it easier for our customers and partners to do business with Microsoft. A key aspect of that mission is delivering a solution to monetization and payment for all of Microsoft's offerings in Gaming, Experiences and Devices, and Cloud + AI. We need to ensure that all channels can sell and transact these offerings, provision/fulfill them, and transform the millions of daily transaction records into accurate bills, payments, and financials to run our business. We are looking for a Software Engineer II engineer, with passion for big data, to join us in building a scalable big data platform for processing Payouts Reporting and provide Advanced Insights on Parners Incentives. The pipelines in this platform process terabytes of data and billions of dollars of revenue; they also closely integrate with sophisticated monitoring, triangulation, data quality, advanced analytics and machine learning algorithms. As an engineer on our team, you
Work with upstream teams to understand their data and how that data can be most effectively modeled in our platform Design and develop end to end pipelines for data ingress and egress, including designing triangulation and validation controls Use technologies such as Spark, Elastic Search, Azure Event Hub. Engage with multiple disciplines, including engineering, program management, data science, finance, and business operations to deliver a comprehensive end-to-end solution Be responsible for live-site on-call in a Dev-Ops environment
Required Qualifications: • Bachelor's Degree in computer science or related field, or equivalent level of practical experience • 4+ years of big data-related software engineering experience • Experience with at least one of: Kafka, Hadoop, Spark, Azure Event Hub, or Azure Data Lake Preferred Qualifications: • Proficiency in C# and .NET Framework, and/or Java • Experience in working in a DevOps environment • 2+ years of software engineering experience building and shipping large-scale, high performance scalable systems and services