Finding
Finding 3: NYSED’s MIS does not meet NRS requirements for reporting certain indicators of performance.
Discussion
WIOA identifies the primary indicators of performance required to be reported annually by all WIOA core programs. WIOA Common Performance Reporting identifies the reporting criteria for these indicators. Document review, discussion with NYSED staff, and a review of the State MIS, ASISTS, revealed data collection and reporting issues within the system regarding the credential attainment rate and MSG indicators of performance. A review of the methodology for the credential rate indicator revealed the following calculation errors:
- The denominator for the secondary credential component of the credential rate indicator includes all participants who, during the program year, enter the program without a secondary credential, enroll in a secondary level program, and exit the program. NYSED did not have the restriction of “entered the program without a secondary credential” as part of the cohort definition.
- NYSED includes only participants in the secondary credential cohort who began the program year in a secondary level program but does not account for students who move into a secondary level program during the program year.
- The credential rate indicator is reported based on participants who exit between January 1 and December 31. For PY 2021, this date range is January 1, 2020 to December 31, 2020. However, NYSED’s methodology reported the credential rate indicator based on a timeframe of exiting between July 1, 2020 to June 30, 2021.
- The denominator for postsecondary includes participants dually enrolled in an adult education program and a postsecondary level program who exit from both programs. This includes participants who enrolled in and exited from an Integrated Education and Training (IET) program. NYSED’s methodology for the postsecondary credential cohort includes participants who enrolled in IET or Integrated English Literacy and Civics Education (IELCE), but it does not require exiting from these programs. Additionally, not all IELCE participants are dually enrolled in a postsecondary level training program, so they may not all be eligible for the postsecondary credential cohort.
Each component must be addressed to accurately report State performance for the credential rate indicator.
Additionally, Measures and Methods for the National Reporting System for Adult Education indicates that only the most recent MSG should be reported for a participant on NRS Table 4. MSG includes five types of gain: achievement of at least one Educational Functioning Level (EFL), attainment of a secondary school diploma or its recognized equivalent, secondary or postsecondary transcripts or report cards, satisfactory or better progress report towards established milestones, or successful passage of an exam that is required for a particular occupation or progress in attaining technical or occupational skills. MSG data on local program reports reflected more gains on NRS Table 4 than the total participant count for the program. This indicated that the methodology did not account for the most recent MSG gain. There is a concern that this could also impact the State-level MSG data, so the correct methodology must be applied to ensure accurate reporting for MSG.
Relevant sections of law and regulation
Relevant sections of law and regulation: Sections 116(b) and 116(d) of WIOA; WIOA Common Performance Reporting, OMB Control Number 1205-0526; Measures and Methods for the National Reporting System for Adult Education, OMB Control Number 1830-0027