The Manager of Assessment Reporting and Analysis oversees the data analysis and reporting systems for the Commonwealth's statewide assessments, including the Massachusetts Comprehensive Assessment System (MCAS); ensures on-time access to accurate data and reports; and provides data management and psychometric expertise to support the delivery and reporting of technically sound assessments.

Key Responsibilities

• Manage the timely delivery of clear and accurate large-scale assessment results through coordination among staff members, internal groups, and external contractors/consultants
• Perform and monitor statistical analyses required for data reporting and interpretation of results
• Oversee psychometric analyses and processes to support test development and outcomes (e.g., verification of psychometric properties of assessments, validation of equating)
• Develop and deliver presentations on assessment results and technical issues to a variety of audiences and stakeholders, including DESE leadership
• Provide expertise in planning for standard setting and standards validation activities, and ensure that best practices are used to set and maintain the achievement levels and scales used to report MCAS achievement
• Collaborate with the Associate Commissioner to plan and coordinate meetings of the Technical Advisory Committee (develop agendas, deliver presentations, perform follow-up activities)
• Conduct data analyses to ensure the validity, reliability, and fairness of the assessments
• Develop and maintain expertise and knowledge of Student Assessment Services data and analytical procedures, and oversee steps to document those procedures
• Provide leadership in the execution of research projects and initiatives in SAS
• Review and revise technical reports and other technical documentation to ensure they are clearly written and accurate
• Develop and implement quality control processes of deliverables, such as reports for schools and districts
• Collaborate with other divisions in the agency on related systems, such as accountability and data collection

Supervision and Management

• Supervise a team of data, reporting, and psychometric specialists and be responsible for employee goal setting, regular supervisory meetings, annual reviews, and growth planning
• Ensure that all work is performed with a focus on equity for all students
• As needed, lead or contribute to efforts to recruit, interview, hire, and induct new staff
• Serve as a core member of the leadership team within SAS, providing guidance, support, and recommendations, and contributing to major projects and initiatives
• Attend and coordinate key meetings and events and represent Associate Commissioner as needed
• Collaborate with staff throughout the agency to achieve shared goals
• Manage and oversee the work of contractors, and contribute to the development and review of RFRs, proposals, contracts, and budgets

Based on assignment, travel throughout the Commonwealth and to regional and/or national conferences may be required.

Preferred Qualifications

• Demonstrated experience in analysis and reporting of large-scale assessments, preferably in a leadership position
• Demonstrated ability to manage complex data deliveries
• Experience supervising, managing, and motivating staff
• Ability to explain measurement and statistical concepts to a variety of audiences, and ability to prepare effective visual presentations of complex data
• Experience providing technical assistance and training to school and district leaders
• Experience developing and implementing quality control and assurance measures
• Ability to perform analyses on large data sets using SPSS, R, and/or other appropriate statistical packages
• Knowledge of and experience in the application of a variety of psychometric models (e.g., classical test theory, factor analysis, item response theory, differential item functioning, dimensionality analysis, interrater reliability analysis)
• Familiarity with professional assessment standards (e.g., AERA/APA/NCME Standards)
• Demonstrated experience reviewing and revising technical documents
• Experience successfully managing contracts and projects, meeting strict deadlines, and adapting to changing circumstances
• Experience developing practical solutions to applied educational assessment problems
• Strong interpersonal skills and ability to collaborate effectively with internal and external stakeholders, organizations, and contractors
• Excellent written, oral, and presentation skills
• Proficiency with Microsoft Outlook, Word, Excel, and PowerPoint, Adobe Acrobat Pro, and video conferencing programs (Zoom/Teams)

Applicants must submit a cover letter along with their resume.

Questions regarding this position or the application process should be directed to Dawn.E.DeRoche@mass.gov.

Qualifications

MINIMUM ENTRANCE REQUIREMENTS:

Applicants must have at least (A) five (5) years of full-time or, equivalent part-time, professional, administrative, supervisory, or managerial experience in business administration, business management, public administration, public management, clinical administration or clinical management of which (B) at least one (1) year must have been in a project management, supervisory or managerial capacity or (C) any equivalent combination of the required experience and substitutions below.

Substitutions:

I. A certificate in a relevant or related field may be substituted for one (1) year of the required (A) experience.

II. A Bachelor's degree in a related field may be substituted for two (2) years of the required (A) experience.

III. A Graduate degree in a related field may be substituted for three (3) years of the required (A) experience.

IV. A Doctorate degree in a related field may be substituted for four (4) years of the required (A) experience.

Comprehensive Benefits

When you embark on a career with the Commonwealth, you are offered an outstanding suite of employee benefits that add to the overall value of your compensation package. We take pride in providing a work experience that supports you, your loved ones, and your future.

Want the specifics? Explore our Employee Benefits and Rewards!

An Equal Opportunity / Affirmative Action Employer. Females, minorities, veterans, and persons with disabilities are strongly encouraged to apply.

The Commonwealth is an Equal Opportunity Employer and does not discriminate on the basis of race, religion, color, sex, gender identity or expression, sexual orientation, age, disability, national origin, veteran status, or any other basis covered by appropriate law. Research suggests that qualified women, Black, Indigenous, and Persons of Color (BIPOC) may self-select out of opportunities if they don't meet 100% of the job requirements. We encourage individuals who believe they have the skills necessary to thrive to apply for this role.