Senior Product Manager, Scoring & Assessment Systems

The National Conference of Bar Examiners (NCBE) is a nonprofit organization dedicated to advancing a competent, ethical, and diverse legal profession. We develop and deliver high-quality assessment products, services, and research that support state courts and licensing authorities in evaluating the readiness of aspiring lawyers.

NCBE is committed to fostering a collaborative, mission-driven environment with competitive salaries, comprehensive benefits, flexible scheduling, and professional development opportunities. NCBE's headquarters is based in Madison, Wisconsin, a dynamic community with exceptional amenities.

The Senior Product Manager, Scoring & Assessment Systems will own the product strategy, vision, and evolution of scoring across NCBE's exam products. This is a senior product leadership role with clear decision authority, responsible for defining how scoring works at NCBE: from human grading models to the responsible adoption of automated scoring. The role requires strong product judgment, industry perspective, and strong communication skills to evaluate tradeoffs and make recommendations that shape the future of high-stakes legal licensure assessment.

This Senior Product Manager will serve as thesingle accountable product leaderon scoring decisions, working across test development, psychometrics, operations, and technology. Whenprioritiesconflict or perspectives differ, this role will guide teams toward decisions grounded in best practices, data, and product outcomes.

This position is ideal for someone who has owned or significantly shapedend-to-endscoring in a large-scale assessment organization and can bring thatexpertiseto a complex, evolving product ecosystem.

Essential Duties and Responsibilities

Own Scoring Product Strategy & Decisions

• Define and drive NCBE's scoring strategy across multiple-choice and constructed-response item types.

• Own the product vision and roadmap for scoring, aligning it to organizational priorities.

• Make informed product decisions on scoring approaches, balancing validity, reliability, operational feasibility,and grader and examineeexperience.

• Serve as the decision-making authority when tradeoffs arise across stakeholder groups.

Lead Evolution of Scoring Models (Human & Automated)

• Evaluate and guide the adoption of automated scoring, including where and how it should (or should not) be used.

• Articulate the benefits, risks, and limitations of different scoring approaches, including hybrid human/AI models.

• Partner with psychometric andresearchteams to ensureappropriate validation, calibration, and evidence frameworks.

Bring Industry Best Practices to NCBE

• Applyindustryexperienceand best practicesfrom leading assessment organizations to inform scoring design, workflows, and policies.

• Ensure NCBE's approach reflects current best practices in large-scale, high-stakes testing.

• Introduce improvements to rubrics, training materials, and scoring processes (e.g., training papers, calibration methods).
• Manage staff in the day-to-day performance of their roles, including mentoring, setting performance goals, reviewing performance, training, professional development, recruitment, and onboarding. Foster a high-performing, accountable team environment aligned to NCBE's product strategy and organizational goals.

Translate Strategy into Product Requirements

• Define clear product requirements for scoring systems, workflows, and tools.

• Partner with technology teams to deliver scalable, reliable scoring platforms and integrations.

• Maintain a forward-looking roadmap for scoring capabilities.

Influence & Align Cross-Functional Stakeholders

• Work closely with test development, psychometrics, operations, and external partners to align on scoring approaches.

• Work with senior stakeholders to make decisions and align teams when priorities conflict.

• Communicate and defend product decisions with clarity and credibility.

• Build alignment across teams, especially in areas of ambiguity or disagreement.

Represent Scoring Internally and Externally

• Serve as a key point of contact for scoring-related questions, including with external partners (e.g.,jurisdictionssuch as JX).

• Presentstrategy, decisions,and outcomes toexecutive leadership, governanceboards and committees, and externalstakeholders.

• Clearly explain complex scoring concepts to both technical and non-technical audiences.

Qualifications

Required Experience

• 5-10+ years of experience in large-scale assessment, licensure, or certification testing.

• Direct experience owning or leadinganassessmentscoring product, capability, or majorcomponentend-to-end(not solely supporting, coordinating, or analyzing).

• Experience working at a recognized or high-quality assessment organization.

• Demonstrated ability to make, own,and defend product decisions in complex, cross-functional environments.

• Experience writing product requirements and working closely with technology teams.

Scoring & Assessment Expertise

• Strong understanding of:

• Constructed-response scoring and rubric design

• Rater training, calibration, and monitoring (including training papers)

• Inter-rater reliability and why it matters

• Core concepts of validity and scoring quality (no advanced psychometric analysisrequired)

• Practical knowledge of scoring operations at scale, including tradeoffs between quality, speed, and cost.

Automated Scoring Familiarity

• Working knowledge of automated or AI-assisted scoring approaches, including:

• Strengths and limitations of different models

• Validation and monitoring considerations

• Appropriate use cases in high-stakes environments

Product & Leadership Skills

• Strong product management mindset with the ability to translate strategy into execution.

• Proven ability to influence without authority and navigate competing perspectives.

• Excellent communication skills, including the ability to explain and justify decisionsto senior and external audiences.

• Able to communicate confidently in high-stakes discussions.

The Senior Product Manager may work remotely. NCBE will consider applicants from the following states: FL, IA, IN, MA, MD, ME, MI, MO, MN, NE, NJ, NM, NV, OK, PA, WA, and WI.

This position may include up to 10% travel for meetings with NCBE's Board of Trustees, committees and/or vendors, and/or for conferences, workshops, and/or outreach.

Mission

NCBE promotes fairness, integrity, and best practices in admission to the legal profession for the benefit and protection of the public. We serve admission authorities, courts, the legal education community, and candidates by providing high-quality

• assessment products, services, and research;
• character investigations; and
• informational and educational resources and programs.

EEO Statement
NCBE is proud to be an equal employment opportunity organization. We are committed to providing equal employment opportunity to all applicants and employees.

Please note that applicants may be contacted via email throughout the hiring process. We suggest that you add BambooHR (@bamboohr.com and @app.bamboohr.com) to your Approved/Safe Sender list so that email notifications are delivered to your inbox and not marked as spam.