is seeking Senior CI / CD Software Engineers to accelerate Apache Spark and related data science frameworks.
Data scientists spend a considerable amount of time exploring data, iterating over machine learning (ML) experiments. Every hour of compute required to sort through datasets, extract features, fit ML algorithms, hinders the ability of data scientists to drive towards results.
Apache Spark is the most popular data processing engine in data centers for data science. It is used for interactive data science, from data preparation, to running ML experiments, and all the way to deployment of ML applications.
You will work with open source community to accelerate Apache Spark with GPU for data science. Nvidia believes that data science workflows can benefit tremendously from being accelerated, to enable data scientists to explore many more and larger datasets to drive towards their business goals, faster, and more reliably.
What you'll be doing :
Serve technical lead in defining, designing, developing and maintaining the DevOps tools, frameworks & platforms.
Implement, advocate and carry out CICD conventions and write tools to automate various steps involved in this process.
Develop and maintain Build, Deployment, and Continuous Integration infrastructure.
Enable development team by providing automated build and test solutions using Docker, Kubernetes / YARN and on-prem / CSPs.
Engaging open source communities, including Apache Spark and , to develop the CI / CD pipelines.
Work closely with Development and QA teams to help ensure end-to-end quality.
Full stack development opportunities depending on the candidate's capabilities.
What we need to see :
BS or MS in Computer Science, Computer Engineering, or closely related field
5+ years of working experience in software development
2+ years experience in CI / CD system
Good understanding of Quality Control and Test Automation in Agile-based Continuous Integration environments
Strong programming and debugging skills in Python / Java / Go / C# with extensive bash scripting experience.
Excellent knowledge with Gitlab / Github or other source version control systems.
Excellent knowledge on Docker, Jenkins
Excellent knowledge on build tools like MAVEN, SBT and CMake, unit testing and code-coverage tools
Excellent knowledge of various Linux systems like Ubuntu, CentOS.
Experience with configuration management
Excellent programming skills in Python, Shell script
Able to work successfully with multi-functional teams across organizational boundaries and geographies
Highly motivated with strong communication skills
Ways to stand out from the crowd :
Experience with Spark and Spark cluster(like Yarn, Kubernetes) management would be a plus
Excellent knowledge of open source project management would be a plus
We are an AA / EEO / Disabled employer and with highly competitive salaries and a comprehensive benefits package, NVIDIA is widely considered to be one of the technology world’s most desirable employers.
We have some of the most brilliant and talented people on the planet working for us. Are you creative and autonomous? Do you love a challenge?
If so, we want to hear from you.