Director, Big Data Infrastructure

Walmart Stores SAN BRUNO, CA

About the Job

Position Description


We’re looking for a leader that has extensive experience building big data applications using the Hadoop ecosystem and related technologies, both on traditional clusters and cloud platforms, to collaborate with our internal product development teams constructing scalable and performant big data applications using cloud-based infrastructure.

You’ll interact with the engineers, product managers and architects to provide scalable robust technical solutions.

Minimum Qualifications


• Collaborating closely with application team architects and engineers to identify technologies and platforms suitable for their big data processing requirements, and then assisting those teams with onboarding, development, deployment, and debugging on those platforms
• Effectively handle multiple initiatives and be accountable for delivering multiple releases on schedule with highest quality standards
• Investigating new big data tools and technologies for their potential application to common use cases; establishing best practices, developing design patterns, and working with platform engineers and product managers to specify and deliver new major technology features
• Technical Leadership: Drive high-level architectural decisions and designs based on business priorities and requirements. Lead the team in providing technical vision and guidance to a broad community of big data infrastructure users, such as software application engineers and data scientists, through research, investigation, collaboration, and hands-on debugging, often driven by specific use case requirements. Ensure that application big data solutions adhere to best practices and enterprise standards for scalability, availability, efficiency, data lifecycle management, information security, fault tolerance, and disaster recovery
• People Management: Coach and mentor first-level managers and tech leads. Identify, implement, and communicate best practices for engineering. Build a high-performance team

Additional Preferred Qualifications


• Hands-on experience with managing and optimizing big data processing systems using Hadoop Streaming, Hive or Spark; ODBC/JDBC connectivity such as HiveServer and Thrift; and streaming data management using Kafka
• Familiarity with the strengths, weaknesses, and idiosyncrasies of big data solutions of cluster-based platforms (e.g., Hortonworks, Cloudera, and MapR) as compared to as cloud resource providers (e.g., Google Cloud Platform, Microsoft Azure, and similar object storage and ephemeral compute paradigms)
• Experience building and nurturing high-performing big-data engineering and/or infrastructure teams

Company Summary


The Walmart eCommerce team is rapidly innovating to evolve and define the future state of shopping. As the world’s largest retailer, we are on a mission to help people save money and live better.  With the help of some of the brightest minds in technology, merchandising, marketing, supply chain, talent and more, we are reimagining the intersection of digital and physical shopping to help achieve that mission.

Position Summary


The Walmart Labs Big Data Platforms team is seeking a Director of Engineering for its Big Data infrastructure team. In this role, you’ll be responsible for managing the teams overseeing the design, development and operations of large-scale data systems operating at petabytes scale.

We’re looking for a leader that has extensive experience building big data applications using the Hadoop ecosystem and related technologies, both on traditional clusters and cloud platforms, to collaborate with our internal product development teams constructing scalable and performant big data applications using cloud-based infrastructure.

You’ll interact with the engineers, product managers and architects to provide scalable robust technical solutions.