Big Data Engineer - Opportunity for Working Remotely
Primary Location: Bellevue,Washington
San Francisco,California,USA; Palo Alto,California,USA;
We are a hard-working, diverse product organization building a SaaS cloud security platform to deliver the future of cloud security and governance. We live and breathe distributed systems, horizontally scaled microservice architectures, graph data models, security-focused data analytics and data science, cloud services, and high availability designs all with integrated instrumentation and logging. We are seeking capable problem solvers and critical thinkers with growth mindsets to join our team. While we have a high bar for commitment and performance, we also expect our work to be balanced, fun and fulfilling.
We're looking to add a distributed systems software engineer to focus on big data ingestion services and features. Not only will you add features to existing collection and ingestion systems, you will also help put major new components in place within our data collection pipeline. You will help make expert decisions on pivotal technical and design choices based on operational considerations, constraints of our current product and customer-facing product roadmap. Subsequently you must be able to design, code, deploy and operationalize microservices into a public cloud and Kubernetes-based SaaS. Your work will involve both horizontal scale and data ingestion and storage challenges. You will have opportunities both to learn from senior engineers and also take a technical leadership role, while working to promote and sustain a culture of engineering and operational excellence. This position presents a long-term opportunity to lead a team dedicated to this ongoing statement of work.
Duties and Responsibilities:
- Architect, design and implement new distributed systems for collecting and storing log data, scaling parallel ingestion and analysis, supporting machine learning models, etc.
- Enhance existing backend services and pipelines that collect, store, analyze and correlate public cloud inventory, log and event data
- Analyze fault-tolerance and high availability issues, performance and scale challenges; and discuss and propose ideas to solve them
- Pinpoint problems, instrument relevant components as needed, and ultimately implement solutions
- Learn, understand and apply trade-offs between consistency, durability and costs to build solutions which can meet the demands of rapidly growing services
- Deploy and ensure operational readiness of the services, define and meet commitments to our customers regarding availability and performance
- Lead and mentor junior developers working on these services
Requirements and Qualifications:
- 3-5+ years industry experience designing, building and supporting large scale systems in production
- Experience building distributed systems microservices to support Big Data and Data Lake applications
- Familiarity with technologies like Spark, Flink, Kafka used to build high scale data pipelines
- Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking and storage, performance and scale
- Strong CS fundamentals including data structures, algorithms, and distributed systems
- Systems programming skills including multi-threading, concurrency, etc. in a compiled language like Golang, C++ or Java
- Experience with cloud infrastructure - AWS, Azure or Google Cloud
- Track record of identifying and implementing creative solutions
- Ability to influence decisions with excellent verbal and written communication skills
Category : Engineering and Technology
Subcategory: Software Engineering
Experience: Manager and Professional
Full Time/ Part Time: Full Time
Posted Date: 2021-05-06
VMware Company Overview: At VMware, we believe that software has the power to unlock new opportunities for people and our planet. We look beyond the barriers of compromise to engineer new ways to make technologies work together seamlessly. Our cloud, mobility, and security software form a flexible, consistent digital foundation for securely delivering the apps, services and experiences that are transforming business innovation around the globe. At the core of what we do are our people who deeply value execution, passion, integrity, customers, and community. Shape what’s possible today at http://careers.vmware.com.
Equal Employment Opportunity Statement: VMware is an Equal Opportunity Employer and Prohibits Discrimination and Harassment of Any Kind: VMware is committed to the principle of equal employment opportunity for all employees and to providing employees with a work environment free of discrimination and harassment. All employment decisions at VMware are based on business needs, job requirements and individual qualifications, without regard to race, color, religion or belief, national, social or ethnic origin, sex (including pregnancy), age, physical, mental or sensory disability, HIV Status, sexual orientation, gender identity and/or expression, marital, civil union or domestic partnership status, past or present military service, family medical history or genetic information, family or parental status, or any other status protected by the laws or regulations in the locations where we operate. VMware will not tolerate discrimination or harassment based on any of these characteristics. VMware encourages applicants of all ages. Vmware will provide reasonable accommodation to employees who have protected disabilities consistent with local law. Job ID: R2104481