Research Scientist

Job ID: 1601448 | Amazon Web Services, Inc.


To meet the growing demand for Amazon Web Services around the globe, we need exceptionally talented, bright, and driven people. We are looking for a dynamic, organized self-starter to join the AWS Certification Team focusing on holistic strategies and multiple audiences for our AWS Certification offerings. As part of this team, you will be responsible for maintaining the validity of our offerings and supporting initiatives to improve quality and efficiency and aligned to AWS technologies, solutions, and services.
We are seeking a Research Scientist with experience working with criterion-referenced assessment programs to support a large global AWS Certification program. In this role, you will support AWS Certification efforts to conduct job analyses, set passing scores, create construct maps, deploy methodologies for automatic item generation, support item and test analyses for active exams, and evaluate the adequacy of existing item banks for form development plans.
You will work closely with an existing team of senior psychometricians and certification exam program managers in development, publishing, delivery, security, product management and translation/localization to support ongoing analyses of exam data. To be successful in this position, you must be highly motivated, creative, and a self-starter who is able to think big, execute, yet stay focused on the details.

Key Responsibilities:
· Conduct/support Job Task Analysis (JTA) workshops and post-JTA survey analyses to define the blueprint and test specifications for new certifications or updates to existing certifications
· Conduct standard setting studies to set the passing score for an exam
· Support ongoing efforts to incorporate Automatic Item Generation and Assessment Engineering into the item development processes
· Support item analysis to evaluate quality and performance of active exam items
· Interpret and clearly communicate the results of analyses to stakeholders through written and oral reports
· Follow the accreditation standards set by ISO/IEC:2012 17024 and the National Council for Certifying Agencies (NCCA) as they relate to valid psychometric practices
· Contribute to the development and execution of the strategic goals regarding the AWS certification program.
· Consult with leadership, internal staff, external consultants, and industry leaders regarding advancement of current offerings

About Us
As cloud technologies continue to transform businesses, skilled individuals are in high demand. At AWS Training and Certification (T&C), we are passionate about revolutionizing the way people advance their cloud skills and careers. We equip diverse builders of today and tomorrow with the knowledge they need to leverage the power of the AWS Cloud. Join our dynamic, fast-growing team and help us empower our customers to build cloud skills.
Inclusive Team Culture
Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have twelve employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.
Work/Life Balance
Our team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.

Mentorship & Career Growth
Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded employee and enable them to take on more complex tasks in the future.

This position can be performed from any AWS US corporate office.

The pay range for this position in Colorado is $132,800 - $160,000/yr; however, base pay offered may vary depending on job-related knowledge, skills, and experience. A sign-on bonus and restricted stock units may be provided as part of the compensation package, in addition to a full range of medical, financial, and/or other benefits, dependent on the position offered. This information is provided per the Colorado Equal Pay Act. Base pay information is based on market location. Applicants should apply via Amazon's internal or external careers site.


· Doctorate in I/O Psychology, Cognitive Science, Educational Measurement or closely related field.
· 2+ years of applied experience designing and leading assessment programs
· Experience with Assessment Engineering principles and evidence-centered design (e.g., developing construct maps and cognitive task models)
· Direct experience working with Subject Matter Experts, vendors, and/or stakeholder groups
· Experience conducting job analysis and standard setting studies


· Experience with automatic item generation
· Experience with different exam designs and delivery (e.g. LOFT, CAT)
· Excellent operational research abilities
· Experience in R or Python programming
· Working knowledge of item calibration software (e.g. IRTPRO, Winsteps, FlexMIRT)
· Working knowledge of online survey software packages (e.g., SurveyMonkey, Qualtrics)
· Highly adaptable, creative, and thrives in a fast-paced work environment
· Strong organizational skills, time management, and program management skills
· Strong communication skills (oral and written)
· Ability to work well with diverse teams
· Ability to work independently with minimum supervision
· Experience designing and conducting custom validation studies

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Pursuant to the Los Angeles Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.