Senior Software Development Manager - Machine Learning Services

Job ID: 1249145 | Services LLC


At AWS AI, we want to make it easy for our customers to deploy machine learning models in the cloud or at the edge. SageMaker provides a complete set of services to simplify the task of building and training a model. As one of the SageMaker services, Neo provides a deep learning compiler and runtime that is designed to accelerate any machine learning model on any hardware.

Neo uses open source Treelite and Apache TVM along with partner-provided compilers to optimizes machine learning models. After compilation, models perform at up to twice the speed of the original framework with no loss in accuracy. Upload a pre-trained model built with MXNet, TensorFlow, PyTorch, or XGBoost to your S3 bucket, choose your target hardware platform from Intel, NVIDIA, ARM, Ambarella, Qualcomm, NXP, or TI. With a single API call, SageMaker Neo optimizes the model, converts it into an executable module, and returns it to your S3 bucket. Then the free open source Neo runtime uses less than 100th of the space of the framework to run the model on the target hardware.

The SageMaker Neo team is growing rapidly to keep up with growth in customers and their requests. We are hiring well-rounded technologists with backgrounds in machine learning, compilers, systems, and AI accelerators. If have worked on HPC and performance tuning, you will enjoy working on the breadth of ML applications that we optimize. The work offers an extremely broad set of opportunities with exposure to multiple AI applications, ML frameworks, models, compilers, systems SW, and various AI hardware including ARM, Intel, AWS Inferentia, Nvidia, and the emerging edge AI ASICs.

Join the Amazon SageMaker Neo team to help AWS customers deploy machine learning models in the cloud and on edge devices at scale in production. Work on an open source industry-standard compiler and runtime for machine learning that is already deployed on millions of devices.

This position is a rare opportunity to join a fast-growing business and to shape open source technologies, AWS services, and the business based on them. A successful candidate will bring deep technical and software expertise, strong business acumen and judgment, the ability to define breakthrough innovations, and the desire to have an industry wide impact. To be successful in this role, you must have the aptitude to work within a fast moving, startup environment in a large company to rapidly deliver services that have a broad business impact.


- Ph. D. in Engineering, Computer Science or related technical field (or equivalent)
- 10+ years of professional research and development experience
- 5+ years of experience contributing to the architecture and design (architecture, design patterns, reliability and scaling) of new or current systems
- 10+ years of experience leading a software development teams through full development life-cycle, which must include 2+ years experience managing managers
- Programming expertise with at least one modern language such as Java, C++, or C# including object-oriented design
- Knowledge and/or experience of machine learning, compilers, and system architecture


- Proven expertise in compilers and optimization techniques; high performance, distributed systems including GPUs and other HW accelerators
- Proven experience with machine learning libraries such TensorFlow, MxNet and PyTorch and deep learning compilers such as XLA and TVM
- A passion for leading deeply technical research and development teams
- Strong verbal and written communication skills
- Experience attracting, hiring and maintaining top talent

Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation/ Age