New job, posted less than a week ago!
Job Details
Posted date: Oct 06, 2026
Location: Kirkland, WA
Level: Senior
Description
Own the end-to-end technical strategy for BigQuery's storage formats. Determine the roadmap for our existing proprietary formats (Capacitor) versus adopting or contributing to emerging open standards (Vortex, Lance, BtrBlocks). Design storage layer innovations that unlock zero-copy column updates/additions, extremely efficient random access, and seamless integration with deep learning training pipelines. Partner with Dremel, BigLake, and GCS teams to ensure our execution engine seamlessly adopts next-generation formats without regressing SQL concurrency. Position Google as a thought leader in the open data lakehouse ecosystem, steering the direction of the formats that will define the next decade of data infrastructure. Lead technical and ROI evaluations of open-source formats. Determine engineering costs to integrate Arrow-native execution with BigQuery pipelines for wide tables and multimodal AI.BigQuery is at a pivotal moment in the evolution of cloud data warehouses. The industry is seeing a massive shift in file format momentum—with next-generation columnar formats like Vortex redefining core analytics performance, and multimodal formats like Lance pulling data gravity toward AI/ML training and vector search workloads.
We are seeking a visionary Tech Lead to own the goal, strategy, and architectural design for BigQuery’s file format. In this role, you will define how BigQuery adapts to these industry shifts, whether by evolving our native Capacitor format, adopting and extending open-source standards like Vortex, or bridging the gap to AI-native lakehouses.Google Cloud accelerates every organization’s ability to digitally transform its business and industry. We deliver enterprise-grade solutions that leverage Google’s cutting-edge technology, and tools that help developers build more sustainably. Customers in more than 200 countries and territories turn to Google Cloud as their trusted partner to enable growth and solve their most critical business problems.Individual pay is determined by factors including job-related skills, experience, and relevant education or training.
US: $174000 - $252000 (USD) + 15% bonus target + equity + benefits
Learn more about benefits at Google.
Qualifications
Minimum qualifications: Bachelor's degree in Computer Science or related technical field, or equivalent practical experience. 5 years of experience in software development, with experience in C++ or Go. 3 years of experience designing and optimizing distributed systems, storage architectures, or database internals (columnar storage formats, serialization, or query execution engines).Preferred qualifications: Experience with next-generation or specialized file formats like Vortex, Lance, Nimble, or FastLanes. Experience with vector search (ANN) and building storage layers optimized for retrieval-augmented generation (RAG) and multimodal data. Strong understanding of modern machine learning data pipelines, including how PyTorch/TensorFlow consume data and the I/O bottlenecks of AI training. History of successful contributions to major open-source data projects (e.g., Apache Arrow, Parquet, Iceberg).