Narrative Analysis Tool
Instructions
The Narrative Report responses below can be further filtered by one or more states, as well as keywords.
For more information on Narrative Reports please see the technical assistance documents.
| State | Describe how the adult education program performed in the overall assessment of core programs based on the core indicators of performance. Discuss how the assessment was used to improve quality and effectiveness of the funded eligible providers and any plans to further increase performance in future reporting years. |
|---|---|
| Alabama | Alabama’s adult education follows a continuous improvement approach with data analysis as the foundational base. Regional Directors conduct routine performance meetings with each of their assigned programs. ACCS directors use this opportunity to highlight and discuss best practices relevant to program in each of their areas. The dashboard on AAESAP provides real-time information for enrollment, progress post-test rate, MSG, credential attainment, and career pathway completions. The ALAPCE conference provides multiple breakout sessions that cover such topics as rules surrounding MSGs and the reports to help monitor performance. Additional topics include how to understand and drill down into performance information on the dashboard and using reports for monitoring and program improvement. This conference is open to all grantees with instructors, program directors, and data entry specialist attending various sessions. Program year 2021-22 saw a 20% increase in the number of participants served over the previous PY 2020-21. There was also a 24% increase in the number of contact hours in PY 21-22. These indicators show signs of recovery to pre-pandemic enrollment. Additional recruitment plans are in place to increase enrollment and services in PY 2022-23 by increasing our student computer equipment loan program and increasing distance learning opportunities with remote assessment and access to a library of career centric curriculum. Of the 14,351 students enrolled, 12% were English Language Learners (ELLS). The ELLs attended an average of 62 hours. The Adult Basic and Adult Secondary (ABE and ASE) students attended an average of 57 hours. Retention for all students is an area that requires constant attention by the providers. Defining and assisting with barriers that inhibit attendance by the participant is an area that is addressed by the local programs and state staff through professional development and targeted technical assistance. Enrollment increases in serving incarcerated and institutionalized participants stemmed largely from corrections facilities once again allowing instructors to reconvene classes in PY 2021-22. Local providers can analyze information on the real time information dashboard of our management information system, AAESAP. The demographics provided in the NRS tables can be used to design curriculum and customize classes for age ranges and ethnicity. Dashboard information allows providers to self-analyze their program data to identify potential problems and proactively work to resolve the issues. The system provides a multitude of reports for targeting various aspects of programmatic performance. The educational functioning level completion percentage for PY 2021-2022 was greatest in the ABE Beginning Literacy and ABE Beginning Basic levels. For ESL students the strongest areas of level completion were Low and High Beginning ESL. Local program providers are using technology to create synchronous and asynchronous instruction for all levels of learners and employing interactive hybrid approaches. These flexible instructional methods are assisting other educational functioning levels of learning to advance and achieve a skill gain. The AAESAP system also allows students to self-enroll online for Adult Education services. Classes are offered on-site, virtually, or in a hybrid format. An auto alert notification allows a local program to know when a public inquiry or enrollment has been created. Alabama Adult Education Core Performance Measures for PY 2021-2022: Employment 2nd QTR after exit 49.30% Employment 4th QTR after exit 42.77% Attained HSE and enrolled in postsecondary within one year of exit 8.68% Attained HSE and employed within one year of exit 33.17% Attained a postsecondary credential while enrolled or within one year of exit 39.05% Attained any credential 39.32% Measurable Skill Gain 50.15% Alabama’s MSG percentage continued to see improvement with the extension of tracking gains specific to Workplace Literacy and Integrated Education and Training programs. Increased enrollment in postsecondary education has been tracked and data reflects… |
| Alaska | Participant Eligibility Requirements
Although still recovering from challenges associated with the pandemic, the data for the period of July 1, 2021 – June 30, 2022 has started to show increased numbers in almost all areas of performance, including Overall Enrollment, Educational Functioning Levels (EFL) gain, and Employment Gains. The AAE Measurable Skill Gains (MSG) decreased this program year.
Alaska is unique in geographic, cultural, and economic barriers which can hinder access to training, education, and employment. Each regional program serves vast areas with barriers unique to Alaska. Transportation, weather, and the digital divide are among the most prominent barriers to adult education. Some examples of Alaska’s regional barriers include local programs:
|
| American Samoa | EVALUATION OF THE AMERICAN SAMOA ADULT BASIC EDUCATION PROGRAM PER INDICATOR Indicator #1: Obtained a GED or Secondary School Diploma The Adult Education Literacy and Extended Learning Program (AELEL) have continuously offered educationally disadvantaged adults the opportunity to earn their high school credentials. In the Territory, as it is in some of the U.S. states, the GED diploma represents a second chance at secondary education and opening of new opportunities. Those in the general public who have not had the opportunity to complete their high school education are the primary targets of the program. Preparatory courses for the GED and HiSet exams are part of the AELEL course offered at the college every semester. In SY 2021-2022, 4% (4) achieved their goal of obtaining a HiSet diploma, this percentage was the result of student testing through the test approaches in the set of three. The Pandemic Covid-19 hardship has contributed to the decrease, for many students opted to join the workforce due to online processes that made things difficult for learning. Indicator #2: Enter Employment Many students who enter our Adult Basic Education and English Second Language programs indicated entering employment as one of their goals. It is also a fact that most of them if not all do not have a high school diploma to land a decent job. Having this information on hand encourages the program to advise the students to seek their diploma through the HiSET exam before making a job choice. Although this is our preference, students have shown to leave the program to take up a job when opportunity is available. This is an ongoing issue from the previous reporting year, as our local economic forecasting doesn’t look promising with one of the two canneries closed and the other is uncertain with its operation. The employment percentage for this year is 65%. Many of these individuals will return to complete their studies and obtain a high school equivalency when they are faced with a job promotion that requires a diploma or decided to go into the arm forces. Upon the opening up of employment since the Covid-19 limitations, more students have sought jobs in government and private agencies. Indicator #3: Retained Employment Majority of AELEL students are currently employed and we are continuously working with the Department of Human Social Service literacy program for newly hired and previous participants. We are currently moving forward with prospective literacy programs in partnership with the American Samoa Government to assist with new employee certification, current employee recertification, and workplace literacy. AELEL has experienced a slight decrease in this indicator from this year due to the Pandemic. Indicator #4: Enter Postsecondary Education or Training The only institution of higher learning in the Territory is the American Samoa Community College. This is also the major service provider for AELEL. We are able to work closely with the Office of the Registrar in transitioning our students to college. A few of our students left the island to further their education and/or training but when they do, it is difficult to track them. Due to the effects of the Covid-19 Pandemic, many of our students are seeking jobs instead of continuing their education. These jobs are mostly low wage jobs and are not reported in any data available. This reporting year we are seeing an increase in this indicator due to hard push by the community for students to stay in school and go further than a high school diploma/equivalency. In SY 2018-19 we had 86% and now in SY 2020-21 our performance dropped to 45%. |
| Arkansas | During the 2021-22 program year, 60% of adult education students in Arkansas achieved an MSG on Table 4, exceeding the negotiated indicator of 49%. Arkansas met or exceeded all 12-level benchmarks. CORE INDICATORS OF PERFORMANCE 2020-2021 Performance 2021-2022 Targets 2021-2022 Performance Difference of Targets to Performance ABE Level 1 53% 45% 63% +8 ABE Level 2 47% 44% 51% +7 ABE Level 3 50% 48% 57% +9 ABE Level 4 69% 56% 72% +16 ABE Level 5 73% 55% 77% +22 ABE Level 6 60% 53% 74% +21 ESL Level 1 55% 47% 52% +5 ESL Level 2 62% 53% 62% +9 ESL Level 3 56% 54% 68% +14 ESL Level 4 63% 53% 60% +7 ESL Level 5 59% 46% 61% +15 ESL Level 6 53% 31% 48% +17 AVERAGE 56% 49% 60% +11 Arkansas Adult Education also reported the following data for its follow-up measure indicators: CORE INDICATORS OF PERFORMANCE 2021-2022 Targets 2021-2022 Performance Difference of Targets to Performance Employment Second Quarter After Exit 45% 46% +1 Employment Fourth Quarter After Exit 47% 42% -5 Median Earnings Employment Second Quarter After Exit $4,200.00 $4,818.54 +$618.54 Credential Rate 49% 43% -6 For the program year 2021-2022, Arkansas Adult Education exceeded the Median Earnings Employment Second Quarter After Exit indicator while falling short in Fourth Quarter Employment Measure and the Credential Attainment rates. To complete the data collection for the Follow-up Measures, Arkansas data-matched with the Department of Higher Education and the Division of Workforce Services. Programs submitted manual outcome forms to the state office for validation for participants or follow-up measures that could not be data matched. Once a program year has ended, ADWS/AES generates an Effective & Efficient (E&E) report based on the benchmarks of core performance indicators for each local provider. For the 2021-22 fiscal E&E report, local providers must meet the state’s MSG benchmark of 49%. This year 34/37 met or exceeded the benchmark. ADWS/AES uses Quarterly Reports to help assess the effectiveness and ability to meet the targets and overall performance of the programs throughout the year. Data is reported on wide-ranging aspects for both qualitative and quantitative analysis. Programs that do not meet the determined percentage of the benchmarks are provided additional technical assistance to prepare a performance improvement plan. Plan implementation is monitored throughout the year. Meetings occur bi-weekly with technical assistance from a team of advisors where goal setting and crucial data analysis elements are identified and planned. Small, achievable goals for positively impacting performance are identified and set with a plan to achieve them bi-weekly. Technical assistance is provided to all programs, including those that meet or exceed the performance benchmarks to address deficiencies in any specific EFL. Under normal circumstances, programs that do not meet the E&E requirement for three consecutive years are reviewed for possible cessation of funding. Since the beginning of the COVID pandemic, programs were not penalized; however, intense technical assistance and monitoring are provided to promote performance improvement. Through E&E team assistance and targeted training, 34/37 programs met or exceeded the MSG goal of 49% at an overall state average of 60% MSG attainment. This was an improvement over 2021-22, in which 31/37 programs met the requirement. Distance Education training for adult education instructors was offered through the AALRC to meet the continuing need. The state distributed state-provided funds for each program to purchase Chromebooks, laptops, and hot spots to meet program student technology needs and for testing. The 2020-2021 state and national performance data and negotiated targets were shared with local providers at the Fall Administrators’ meeting, through advisory committee meetings, specialized training, and through the ADWS/AES committees that identified the professional development needs and requests to promote continued improvement in program performance. In the analysis of performance data, the Administrators’ Meetings also highlighted training pieces that addressed recruitment and retention strategies, distance education software usage guidance and strategies, addressing ESL needs, and incorporating IET programs into the services offered by each program to increase the credential attainment rate and help participants prepare for the workforce. In analyzing the data to address statewide training needs, training was held multiple times throughout the year, including regional, in-person, and webinar training. Professional development addressed the importance of the follow-up process. It guided the process of collecting and entering the data in the state’s data management system, best practices for developing data entry processes and procedures, monitoring data at the program and class levels, understanding federal tables, utilizing state reports, detailing participant progress by assessment and subject area, developing IETs, Quarterly Report reporting and analysis, and training for diversity and test anxiety. Professional development training will continue to be developed as ADWS/AES moves to improve and meet targets for performance in measurable skill gains and follow-up measures. |
| California | The WIOA, Title II: AEFLA program offered at 215 local agencies, including two coalitions, enrolled 311,879 learners. Of these, 185,371 (59.4 percent) qualified for NRS federal reporting. California continued to face unprecedented COVID-19 challenges throughout PY 2020–21 and the beginning of 2022. All AEFLA providers including adult schools, community colleges, CBOs, and COEs felt the effects of COVID-19 agency closures and the interruption of services. Enrollment in adult schools declined by 45 percent and in community colleges by 70 percent. IET/IELCE program enrollment declined by 70 percent because of training center closures, and some pathways were no longer practical because of COVID-19. As the COVID-19 conditions improved in PY 2021−22, enrollment in WIOA, Title II: AEFLA programs increased by 20 percent compared to PY 2020−21. ESL increased by 38 percent and ABE by 21 percent. Enrollment in ASE decreased slightly (13 percent). All AEFLA providers reported an increase in overall enrollment in PY 2021−22. Enrollment in IET/IELCE programs doubled, compared to 2020–21 as students started in-person instructions. During the peak COVID-19 period, many AEFLA providers successfully transitioned to distance learning and remote implementation of services. In PY 2020–21, 65 percent (88,749) of 134,492 learners and in in PY 2021–22 more than 36 percent (67,588) of 185,371 learners enrolled in distance learning. Adult learners who qualified for NRS federal reporting in PY 2021−22 reflect the diversity of the state. The largest ethnic groups of learners are Hispanic (64.5 percent), Asian (13.1 percent), White (12.7 percent), Black (6.4 percent), or two or more races (2.7 percent). Adult learners are more likely to be female (57.9 percent). Two hundred seventy-three students indicated non-binary as their gender. The percentage of Asians in the overall adult learners enrolled continues to be lower by three percent compared to pre-COVID-19 period. Adult learners between the ages of 25 and 44 (52.5 percent) comprise the largest age group. Close to 72 percent of learners enrolled are non-English-speaking. COVID-19 greatly affected California’s performance in PY 2020–21. California’s performance in PY 2021–22 improved significantly compared to PY 2020–21. In 2021−22, of the 185,371 learners who qualified for NRS federal reporting, 77,266 (41.7 percent) achieved at least one Measurable Skills Gain (MSG) compared to 34 percent in PY 2020–21. More students were able to take a pretest and a post-test in PY 2021–22 (106,393) compared PY 2020–21 (62,315). More than 22 percent of CASAS pre- and post-tests were administered remotely during PY 2020-21. More than 13,000 learners obtained a high school diploma or high school equivalency certificate. The 185,371 learners who qualified for NRS federal reporting averaged 134 hours of instruction similar to previous program years. The learners (106,393) who persisted in the program and took pre- and post-tests reported 175 hours of instruction. The CDE has data-sharing agreements with the California Community College Chancellor’s Office (CCCCO) and the Employment Development Department (EDD) to conduct a data match to provide postsecondary enrollment, secondary or postsecondary credential attainment, and employment and earnings outcomes. To supplement the employment outcome data match, local WIOA, Title II: AEFLA grantees conduct a universal survey of all students who have not volunteered a Social Security number (77 percent of enrollees). The CDE uses a standardized automated follow-up survey in the TOPSpro Enterprise management information system to collect follow-up outcomes. This process includes an automated survey notification feature sent via email and text messages sent by agencies to students who have exited the program. When the student completes the survey online, TOPSpro Enterprise automatically receives the results. Agencies are able to customize the survey invitation by adding agency logo and name, a survey greeting, and a thank-you note. Survey invitations and the survey itself are available in eight languages. All agencies conduct the Employment and Earnings Survey each quarter. Of those students who exited in PY 2020−21, and either had records that could be matched with the EDD or responded to the employment and earnings follow-up survey, 48 percent were employed in the second quarter after exit. The overall reported percentage of students who exited their program and achieved employment is 23 percent, as most could not be matched with the EDD file. For the students who did not volunteer a Social Security number, 33 percent responded to the survey. The response rate to surveys improved by five percent compared to the previous program year. Also, students who had a primary or secondary goal to get a job or retain a job or get a better job, 30 percent were employed in the second quarter after exit. During PY 2020−21, the CDE continued to provide training and guidance for local agencies to conduct remote instruction and testing. Local agencies continued to submit data to the CDE on a quarterly basis. The previously mentioned data submission structure allows quarterly analysis and early identification of incomplete or inaccurate data. The CDE has provided statewide AFELA network meetings to disseminate successful retention and recruitment strategies to more schools. In terms of action and technical support, the state has focused on student recruitment and retention. |
| Colorado | From April through May annually, monthly data close out webinars and procedural resources were provided to grantees. At the local level, grantees were required to conduct end-of-year data reporting no later than the second Friday in July. Although data entered in LACES by grantees was monitored monthly for errors, AEI staff ran a final diagnostic report of learner data after that second Friday in July. Any errors identified were sent to grantees for correction in LACES by the end of July to officially close out the data reporting. Once the data was finalized and frozen, grantees were required to complete an Annual Performance Report (APR) either in writing or collaboratively through a virtual meeting. A majority of grantees elected to complete the APR collaboratively and these meetings took place in July and August of 2022. The APR required local grantees to analyze enrollment, post-testing, and MSG outcomes. This analysis covered all program types, Educational Functioning Level (EFLs), classes, and included a prior year comparison to identify areas of strength and areas needing improvement. Grantees were encouraged to document in the report outliers and anomalies as well as any plans for further research and improvement. Grantees have access to mostly finalized Table 4 results in LACES upon the completion of their data close out in July annually. Due to current limitations with data matching partners, Table 4 postsecondary entrance MSGs and all Table 5 employment and postsecondary credential data are not available to grantees during data close out. Instead, grantees receive final aggregate program level Tables 4 and 5 after AEI submits data to the NRS. During annual Table 4 and 5 training offered in the fall and during bi-monthly Office Hours virtual meetings, grantees are encouraged to compare their Table 4 & 5 data to statewide totals and the performance of other AEFLA-funded programs. The AEI team continues to share current, year-to-date grantee MSG rates quarterly through Office Hours meetings to provide more opportunities for grantees to visualize where their performance ranks across the state. Each also receives an individual report each quarter to show how their data compares to statewide totals and averages on a series of progress measures and outcomes including enrollment, retention to 12 hours, average instructional hours, post-testing rates, and MSG rates by level. Participants increased from 2,973 in 20-21 to 3,715 in 21-22, with 10 out of 14 grantees increasing their participants in the 21-22 program year; and we anticipate continued gradual growth over the next 2 years as impacts of the last competition for funds and the pandemic lessen. Additionally, 11 of the 14 grantees exited fewer learners in 21-22 than compared with 20-21. The focus on retaining learners through graduation or completion was, and continues to be, a focus of statewide technical assistance and training. Improvements are needed in reducing the numbers of learners who exited specifically before making any Measurable Skill Gain as 10 of 14 grantees saw increases in this rate from the prior year. Statewide, the MSG rate increased just slightly from 44.2% in 20-21 to 44.4% in 21-22. Twelve of 14 grantees met or exceeded the 38% MSG target for the 21-22 program year. Asian Pacific Development Center, located in the Denver Metropolitan area, and Pikes Peak Library District, located in Colorado Springs, both fell below that target at 37.43% and 37.61%, respectively. Neither grantee has been subject to technical assistance as they met 90% of the target, a threshold used by AEI to determine if required technical assistance would be implemented. Asian Pacific Development Center makes up roughly 20% of the statewide participants so technical assistance in pulling MSG rates by instructional staff and regularly monitoring these rates was provided to the grantee in July 2022. Staff turnover at Asian Pacific Development Center has presented challenges for the grantee to implement this regular monitoring. Post-test EFL gains once again made up the majority of MSGs in 21-22 with over 88% of MSGs coming from post-testing. Post-testing increased statewide from 67% in 20-21 to 69% in 21-22. High school diploma and equivalency gains made up a little over 10% of all MSGs. Postsecondary entrance made up just 1% of MSGs and IET MSGs made up less than 1% of all MSGs in 21-22. Overall, we saw a decrease in MSG rates for ABE learners with decreases specifically at Levels 2-4 (levels 2-4 contained the majority of ABE learners in 21-22). We saw increases at Levels 1 and 5 but these groups make up far fewer of our ABE learners statewide. We have not yet identified causes for the decreases at Levels 2-4. For ESL, overall and at all levels except for ESL Level 5, we saw increases in MSG rates. Like ABE, we have not been able to identify potential causes for the decrease at ESL Level 5. Statewide, Colorado met our 24% employment in the 2nd quarter after exit target as well as the associated $5,176.00 median earnings in the 2nd quarter after exit target at 25.40% and $6,226.00, respectively. Both are increases from the prior year. Similarly, Colorado met our 12% credential attainment rate target with a rate of 22.72% in 21-22, also an increase from the prior year. Work continues with grantees to focus on increasing these exit-based outcomes through stronger relationships across WIOA partners, in-house career coaching services, the development of more and more diverse IETs and workforce training, and expanding the match criteria and increasing the frequency of data matches. For the 26% employment in the 4th quarter after exit target, we once again did not meet the target with just 19.82% of learners in the cohort being employed 12 months after exit. Of the cohort, a little over 42% of learners reported a Social Security Number (SSN). Currently, this is the only match criteria the Colorado Department of Labor and Employment is willing to match on which increases the challenge in meeting employment targets. Conversations are ongoing with the Data Trust convened by the Colorado Workforce Development Council on accessing employment and wage data quarterly and on using additional learner demographics to match with the Unemployment Insurance Compensation data set. AEI continues to work with grantees on processes for collecting acceptable forms of supplemental wage information, especially with learners who are not able or willing to report an SSN. Eighty-five percent of the cohort exited before July 2020 during the height of the pandemic and before the latest AEFLA competition. It seems that adults exiting due to the pandemic faced additional barriers in obtaining or retaining jobs a year later. Additionally, regardless of exit date in the cohort, 38% of the cohort were enrolled in programs that did not continue to receive AEFLA funding after June 2020. These grantees had little incentive to support learners in pursuing Table 5 outcomes once they were no longer receiving AEFLA funds. AEI has had preliminary conversations around how to best support grantees who were not able to meet or exceed the Table 5 targets. Current considerations include adding Table 5 targets to the state’s risk assessment and monitoring tools, implementing supplemental employment and wage data collection, and ongoing work with grantees on SSN data collection. |
| District of Columbia | In FY22, OSSE AFE awarded continuation funding to 12 sub-grantees to implement Integrated Education and Training service models introduced in the FY17 grant competition and continued in the FY21 grant competition. The models include providing adult education and literacy, workforce preparation, and training services for a specific occupation or occupational cluster to 1,000 District residents for education and career advancement. In total, 1,122 adult learners received services in OSSE AFE-funded programs in FY22. Of this number, 1,080 learners met the National Reporting System (NRS) guidelines of having a valid assessment and 12 or more instructional hours in the program year to be reportable to the US Department of Education per NRS Table 4. The remaining 42 adult learners engaged in one to 11 instructional hours per NRS Table 2A.
DC FY22 Participants by Gender (NRS Table 2)
In FY22, female learners represented 62 percent (n = 674) and male learners represented 38 percent (n = 406) of the total number of students served (n = 1,080).
DC FY22 Participants by Ethnicity (NRS Table 2)
At 66 percent (n = 712 of 1,080 students), black or African American participants comprised the single largest ethnic group of learners served; Hispanic or Latino students followed at 29 percent (n = 314 of 1,080 students). The percentage of American Indian or Alaskan, Asian, or white participants and persons of Two or More Races was 5 percent (n = 54 of 1,080 students).
DC FY22 Participants by Age
Consistent with past years, the largest single group of learners served were between ages 25 and 44 (49 percent/n = 538 of 1,080 students). The second largest group of learners served were between 19 and 24 years of age (26 percent/n = 283 of 1,080 students), followed by learners between the ages of 45 and 54 (10 percent/n = 112 of 1,080 students). The smallest groups of learners served were at opposite ends of the age continuum and included students ages 60 and older (4 percent/n = 48 of 1,080 students), 55-59 years of age (4 percent/n = 44 of 1,080 students), and 16 to 18 years of
age (7 percent/n = 55 of 1,080 students).
DC FY22 Participants by Program Type (NRS Table 3)
In FY22, of the total number of learners (n = 1,080) who met the NRS guidelines, students in Adult Basic Education (ABE)/Integrated Education and Training (IE&T) Programs comprised the single largest group by program type (56 percent/n = 607 of 1,080 students). The second and third largest groups by program type were Adult Secondary Education (ASE)/IE&T Programs (24 percent/n = 259 of 1,080 students), followed by Integrated English Literacy and Civics Education/IE&T Programs (12 percent/n=131 of 1,080 students). The fourth and smallest program type was English Language Acquisition/IE&T Programs (8 percent/n = 83 of 1,080 students).
DC FY22 Measurable Skills Gains by Entry Level (NRS Table 4)
Adult Basic Education (ABE) Participants by Entry Educational Functioning Levels
Of the total number of ABE participants, the most significant number of students entered at ABE Level 4 (33 percent/n = 306 of 917 students), followed by ABE Level 3 (24 percent/n = 223 of 917 students) level. The smallest number of participants entered at ABE Level 1 (.02 percent/n = 22 of 917 students).
English as a Second Language (ESL) Participants by Entry Educational Functioning Levels
Of the total number of ESL participants, the largest number of participants entered at ESL Level 5 at (29 percent/n = 47 of 163 students), followed by the second largest number of participants who entered ESL Level 6 (27 percent/n = 44 of 163 students). The smallest number of ESL participants entered in ESL Level 2 (.06 percent/n = 9 of 163 students).
Measurable Skills Gains by Entry Level
For FY22, OSSE AFE negotiated a measurable skill gains performance target of 47 percent for all ABE and ESL Educational Functioning Levels (EFLs). This target represents the proposed percentage of adult learners making a measurable skill gain (e.g., achieving an educational functioning level gain, acquiring a secondary school diploma or its equivalent, exiting a program below the Postsecondary level and enrolling in postsecondary education and training during the program year, attaining a postsecondary or secondary transcript, making progress towards milestones, or passing a technical/occupational skills exam). OSSE AFE exceeded its target of 47 percent, with 58.46 percent of students enrolled in an OSSE AFE-funded program achieving a gain. This reflects a 2.83-point increase in the percentage of students with measurable skill gains in FY21 – (55.63 percent).
The chart below reflects the percentage of adult learners who made measurable skills gains. Students at ABE Level 5 had the highest rate of measurable gains at 58.52 percent, followed by students at ABE Level 4 (58.06 percent) and ABE Level 3 (57.96 percent). Students at ESL Level 3 had the highest percentage of measurable gains at 93.33 percent, followed by students at ESL Level 2 (88.89 percent) and ESL Level 5 (70.83 percent).
DC FY22 Core Outcome Follow-up Achievement (NRS Table 5)
For FY22, OSSE AFE negotiated targets for the core follow-up outcome measures on NRS Table 5 as follows:
Employment (2nd quarter after exit) – OSSE AFE’s federally negotiated target for this metric during this reporting period was 21 percent. OSSE exceeded this target in FY22. The state’s performance was 38.36 percent which reflects an increase of 23.24 percentage points in FY22 compared to 15.12 percent in FY21.
Employment (4th quarter after exit) – OSSE AFE’s federally negotiated target for this metric during this reporting period was 23.0 percent. OSSE exceeded this target in FY22. The state’s performance was 26.65 percent, which reflects an increase of 14.63 percentage points in FY22 compared to 12.02 percent in FY21.
Median Earnings (2nd quarter after exit) – OSSE AFE’s federally negotiated target for this metric during this reporting period was $8,000. OSSE achieved 98 percent of this target in FY22. The state’s performance was $7,800, which reflects an increase of $910 in FY22 compared to $6,890 in FY21.
All employment and wage data are collected through follow-up surveys with program exiters. Where possible, data match with DOES’s Unemployment Insurance wage data and the State Wage Interchange System (SWIS).
However, matching against the DOES UI wage data and SWIS wage data requires a social security number which we do not require for enrollment in AFE programming and for which only a fraction of our learners voluntarily provide. We know that this leads to an under-representation of our learners’ employment and wage data.
Credential Attainment (Unduplicated) – OSSE AFE’s federally negotiated target for this metric during this reporting period was 48 percent. OSSE AFE achieved 69 percent of this target. The state’s performance was 33.08 percent which reflects a decrease of 11.79 percentage points compared to 44.87 percent in FY21. Because the credential attainment rate is a lagging post-exit measure based on survey responses, the FY22 outcome data for credential attainment rate focusses on students who exited the program during Jan. 1 2020 through Dec. 31 2020 which was the first year of the pandemic when much of the city had shut down. This had a direct impact on a student’s ability to test or get their practicum hours associated with their certifications. Therefore, the drop in FY21-22 for credential attainment was significantly impacted by the effects of the pandemic.
Attainment of a Secondary School Diploma/Recognized Equivalent
From July 1, 2021, to June 30, 2022:
|
| Georgia | During FY22, GOAE continued its target negotiation process for local programs to set enrollment and Measurable Skills Gain (MSG) targets for the year. The purpose of the target negotiation process is to allow providers to negotiate reasonable, but ambitious, targets while still ensuring GOAE fulfills its roles and responsibilities as a pass-through entity of WIOA funds. GOAE provided local programs with data, a negotiations workbook, and factors to consider in a process that mimics the federal target negotiation process. GOAE then monitored each program’s progress towards their targets throughout the year and provided support accordingly. GOAE’s data system, GALIS, produces state- and local-level reports to support regular data analysis by GOAE and local programs. For example, GOAE and local programs used Table 4 reports disaggregated by instructor, class, and student to monitor performance regularly. GOAE also created a weekly internal report to monitor each program’s progress towards its FY22 targets throughout the year. All programs received a mid-year data report from GOAE in January 2022 to show their progress, and programs falling behind their targets were provided targeted technical assistance. GOAE also implemented quarterly local program data quality reports that included benchmarks related to performance to help programs monitor their data throughout the year, such as post-test rates. Georgia’s overall MSG percentage for FY22 was 46.01%, which did not meet the 90% threshold of the negotiated 53.8% target for FY22. Programs continued to face challenges posed by the COVID-19 pandemic during FY22 such as staffing shortages and student enrollment. Nevertheless, overall enrollment increased by 30% compared to the prior year and although MSG performance remained lower than pre-pandemic rates, the overall MSG percentage was six percentage points higher than the prior year. In FY22, GOAE began reporting MSGs for Integrated Education & Training (IET) and Workplace Literacy students. Additionally, programs improved retention and overall post-test rates, both of which contributed to improved MSG performance. When analyzing students who did not earn gains, the majority of students were those who exited prior to earning an MSG. Upon analysis, the percentage of students earning educational functioning level gains continues to be 13 percentage points lower than pre-pandemic percentages, indicating challenges with retention and student post-test success. GOAE plans to provide additional support for retention and instruction in FY23 including targeted professional development, access to instructional resources for staff and students, and implementing a customer relationship management tool to better engage students. GOAE saw similar trends in enrollment and MSG for ABE and ESL students. Both ABE and ESL student enrollment increased by 30% in FY22 compared to the prior year, indicating a rebound in enrollment from the COVID-19 pandemic. However, overall enrollment remains 35% lower than pre-pandemic FY19 enrollment. GOAE will continue to provide marketing and recruitment support to programs in FY23 to continue to increase enrollment. ABE MSG performance increased by seven percentage points to 45% in FY22, and ESL MSG performance increased by two percentage points to 49%. The statewide post-test rate improved by five percentage points to 52% in FY22, and this increase was reflected for both ABE and ESL students. However, the post-test rate and MSG percentage for post-tested ABE students remains lower than ESL students. 49% of ABE students are post-tested, and the MSG performance for those students is 51%; for ESL students, the post-test rate is 61% and the MSG performance for those students is 77%. The combination of increased rigor of the TABE 11/12 and retention struggles continue to create challenges for achieving MSGs for ABE students. GOAE plans to provide technical assistance and access to instructional resources to assist programs in this area. For the other indicators of performance, GOAE saw a five-percentage point increase in second quarter employment compared to FY21 and a two-percentage point increase in fourth quarter employment. GOAE saw a slight decrease of one percentage point in credential attainment, largely due to a change in business rules to reflect clarifications provided during an NRS webinar. GOAE also saw a decline in the overall raw number of participants included in the follow-up indicator calculations which is a reflection of the drop in enrollment from the pandemic and the time lag for cohort inclusion. In FY22, GOAE established specific strategic goals focused on employment and postsecondary transition outcomes to ensure our programs are also monitoring data in these areas. For instance, GOAE partnered with the technical college system to launch a “Keep GOing” marketing effort in FY22 that provided programs with templates and resources to support transitioning adult education students to postsecondary. Programs continued to offer distance education and virtual services through the Blackboard learning management system in FY22, but also re-opened many in-person services during the year. In FY22, 39% of participants were classified as distance education and 42% of contact hours were proxy contact hours, compared to 62% distance education students and 64% proxy contact hours in FY21. Given that FY22 was the second year GOAE implemented negotiated targets with local programs, GOAE used progress towards FY22 targets to identify programs for additional technical assistance throughout the year and placed three programs on Strategic Improvement Plans in FY23 based on FY22 performance. GOAE also used FY22 performance and its strategic initiatives to revise the target negotiation process for FY23 to include targets for IET credential attainment and HSE attainment. GOAE will continue to monitor progress towards these targets through regular data analysis, mid-year reports, and accountability conversations. Based on program performance, GOAE will implement technical assistance, Strategic Improvement Plans, sanctions, and incentives. |
| Guam | During this reporting period, there were 192 adult learners with at least 12 instructional hours, with 61.45% achieving at least 1 EFL. Twenty-three students (12%) attained a high school diploma or equivalent. Most of the students were female (58%), and the ethnicity of the participants was primarily Pacific Islanders (60%). The percentages of adult learners’ functioning levels were 60.64% (ABE and ASE) and 64.86% in English-as-a-Secondary Language (ESL). The 2% decrease in participants compared to the previous program year may be due to the continued rise in positive cases of COVID-19. Some students cared for elderly parents or other family members who were immunocompromised. Additionally, transportation continued to be a challenge for some students. Hence, the decrease in enrollment. Table A indicates students’ profiles extracted from NRS Tables 1 through 3. Table A - Summary of Program Participants Program Year Participants Male Female Number who achieved at least one educational functioning level gain Number who attained AHS or HSE diploma Separated before achieving a measurable skill gain Remaining in program without Measurable Skills gain Native Hawaiian or Other Pacific Islander All other 2019-2020 224 77 147 94 29 84 17 147 77 2020-2021 196 63 133 110 25 31 30 154 42 2021-2022 192 88 111 95 23 47 27 116 76 Once again, Guam exceeded its measurable skills gain (MSG) target (50%) for this program year. Surpassing the target may be attributed to having a dedicated counselor for the program, which is critical for retention. Additionally, the local program’s commitment to addressing student needs and engagement with students ensures student success, especially during the pandemic. Table B - Summary of Measurable Skills Gain by Entry Level Description PY2019-2020 PY2020-2021 PY2021-2022 ABE Level 1 (Beginning Literacy) 50.00% 100.00% 0.00% ABE Level 2 (Beginning Basic) 75% 80.95% 52.38% ABE Level 3 (Intermediate Low) 46.51% 68.97% 58.97% ABE Level 4 (Intermediate High) 53.93% 68.18% 65% ABE Level 5 (ASE Low) 29.63% 47.06% 62.50% ABE Level 6 (ASE High) 66.67% 100.00% 33.33% ABE Total 50.82% 68.45% 60.64% ESL Level 1 (Beginning Literacy) 0.00% 0.00% 0.00% ESL Level 2 (Beginning Low) 0.00% 0.00% 0.00% ESL Level 3 (Beginning High) 50% 0.00% 0.00% ESL Level 4 (Intermediate Low) 80% 100.00% 55.55% ESL Level 5 (Intermediate High) 84.21% 66.67% 84.61% ESL Level 6 (Advanced) 66.676% 66.67% 50% ESL Total 75.61% 77.78% 64.86% Grand Total 55.36% 68.88% 61.45% For PY 2021 - 2022, the performance indicators for Employment Rate Q2, Q4, Median Earnings, and the Measurable Skill Gains exceeded the statewide target rate. Guam did not meet the Credential Rate. Although the exit surveys were automated for follow-up surveys in the TOPSpro Enterprise this program year and conducting personal callouts, contacting participants who exited the program was challenging. The local program planned on establishing a system to follow up with participants who exited during the reporting period by obtaining next of kin contact information, such as phone numbers, email addresses, and mailing addresses, that may improve or increase the performance indicators in the Credential Rate. Program Year Participants Served: 7/1/2021 – 6/30/22 Exited: 7/1/2021-6/30/2022 Total Statewide Target Total Statewide Actual 204 150 Employment Rate (Q2): 7/1/2020 – 6/30/2021 Number Rate Total Statewide Target 64 16% Total Statewide Actual 27 20.30% Employment Rate (Q4): 1/1/2020 – 12/31/2020 Number Rate Total Statewide Target 64 16% Total Statewide Actual 36 19.14% Median Earnings: 7/1/2020-6/30/2021 Earnings Total Statewide Target $1,000 $3,180 Credential Rate: PY 1/1/2020 – 12/31/2020 Number Rate Total Statewide Target 0 26% Total Statewide Actual 0 0.00% Measurable Skills Gains: PY 7/1/2021 – 2022 Number Rate Total Statewide Target 138 44% Total Statewide Actual 118 57.84% |
| Hawaii | The state met its PY 2021 - 2022 targets for median earnings and the credential rate. The state did not meet its targets for employment rate in Q2 and Q4 and measurable skills gain (MSG). Although it did not meet the MSG target, it reduced the gap between the target and actual from PY 2020 - 2021, -5.62, and PY 2021 - 2022, -3.03.
The statewide performance report indicators for Employment Rate Q2, Employment Rate Q4, Median Earnings, and Credential Rate outcomes and targets are listed in the Hawaii PY 2021 - 2022 table.
Hawaii PY 2021 – 2022
Total Participants Served
Cohort Period:
7/01/2021 - 6/30/2022
Total Participants Exited
Cohort Period:
4/01/2021 - 03/31/2022
Employment Rate (Q2)
Cohort Period:
07/01/2020 - 06/30/2021
Number
Rate
Total Statewide
Target
1,086
28.60%
Total Statewide
Actual
3,370
2,018
302
14.09%
Employment Rate (Q4)
Cohort Period:
1/01/2020 - 12/31/2020
Median Earnings
Cohort Period:
7/01/2020 - 06/30/2021
Number
Rate
Earnings
Total Statewide
Target
450
11.30%
$5,191.00
Total Statewide
Actual
243
9.72%
$5,232.50
Credential Rate
Cohort Period:
1/01/2020 - 12/31/2020
Measurable Skill Gains
Cohort Period:
7/01/2021 - 06/30/2022
Number
Rate
Number
Rate
Total Statewide
Target
38
4.70%
1,892
40.00%
Total Statewide
Actual
31
5.22%
1,246
36.97%
Employment Rate Q2 and Q4
The Employment Rate Q2 and Q4 declined in PY 2021 - 2022. A data matching agreement is being developed with the Hawaii Department of Labor Unemployment Insurance Division to provide better data collection for Q2 and Q4 indicators and to inform program improvement.
Employment Rate
Q2
2019 – 2020
2020 – 2021
2021 – 2022
Statewide Target
0%
28.60%
28.60%
Statewide Actual
16.37%
16.30%
14.09%
Difference
Between Target
and Actual
+16.37%
-12.30%
-14.51%
Employment Rate
Q4
2019 – 2020
2020 – 2021
2021 – 2022
Statewide Target
0%
11.30%
11.30%
Statewide Actual
16.77%
16.19%
9.72%
Difference
Between Target
and Actual
+16.77%
+4.89%
-1.58%
Median Earnings
In PY 2021 - 2022, the median earnings were significantly lower compared with PY 2020 - 2021 but still met the target.
Median Earnings
2019 – 2020
2020 – 2021
2021 – 2022
Statewide Target
0
$5,090.00
$5,191.00
Statewide Actual
$7,200.00
$7,930.50
$5,232.50
Difference
Between Target
and Actual
+$7,200
+$2,840.50
+41.50
Measurable Skills Gain
Although the performance target for MSG was not met in PY 2021 - 2022, the difference between the statewide target and the actual percentage improved from PY 2020 - 2021. The gap between the target and the actual decreased from -5.62% to -3.03%.
MSG
2019 – 2020
2020 – 2021
2021 – 2022
Statewide Target
38.00%
40.00%
40.00%
Statewide Actual
29.89%
34.38%
36.97%
Difference
Between Target
and Actual
-8.11%
-5.62%
-3.03%
Credential Rate
The PY 2021 - 2022 credential rate decreased by 1.02% from PY 2020 - 2021 but still met the performance target.
Credential Rate
2019 – 2020
2020 – 2021
2021 – 2022
Statewide Target
0%
4.70%
4.70%
Statewide Actual
3.54%
6.24%
5.22%
Difference
Between Target
and Actual
+3.54%
+1.54%
+.52%
Participants
In PY 2021 - 2022, participants increased by 17% from PY 2020 - 2021. Although the participant count is still below pre-pandemic numbers, it significantly increased from PY 2020 - 2021.
Participants
2019 – 2020
2020 - 2021
2021 – 2022
Served
3,830
2,885
3,370
Exited
3,394
1,636
2,018
Difference
Between Served
and Exited
-436
-1,249
-1,352
Looking toward PY 2022 – 2023, the state expects performance to change by implementing the following actions:
|
| Idaho | Process: The state ran monthly reports to analyze data related to core performance targets. The TRC1 and the state director reviewed these reports for any anomalies or concerns. Particular attention was given to enrollment numbers, retention rates, post-testing rates, and MSG level gains. Any red flags were documented and a follow-up phone call to the local director took place. Collaboratively, the local director and state director would discuss potential data input errors or identify strategies for resolving the concern. These monthly reports were used as additional information to compare against the quarterly desk audits. At the local level, data specialists or data staff review their data daily – particularly to check for new student enrollment data and accurate scores. Accurate class placement is reviewed daily. Any data concerns are taken to the local director and may or may not include instructor input. If any data continues to not accurately reflect a program, a technical assistance ticket is submitted to LiteracyPro along with notification to the state that a concern has been elevated to LiteracyPro. Results: PY21 Employ Q2 Employ Q4 Median Earnings MSGs CR Target 60.00% 45.00% $4600.00 43.00% 40.00% Actual 31.90% 40.57% $5446.07 32.49% 28.84% PY20 Actual 32.03% 34.94% $5853.12 21.59% 40% Analysis: Comparing data to the prior program year, Idaho made some positive data increases in some performance areas while dropping in other areas. Noteworthy accomplishments included Q4 employment 5.63% increase and MSGs 10.9% increase. This trend is positive given the new negotiable targets for this current fiscal year. The median earnings target was met. Two performance areas improved while one other was met. While some performance targets were not met, Idaho witnessed improvements from the prior program year. Given this was a rebuilding year from the pandemic, these increases demonstrate that programs used a variety of learning platforms which created access as witnessed by the increase in enrollment. Idaho has several industries that aggressively hired without a GED or high school diploma requirement and provided on-the-job training. Construction, mechanic shops, food service, and healthcare facilities are examples. The unemployment rate in Idaho was and continues to be very low, and programs saw participants leave for $15 - $ 17-hour wages. It is critically important that AE continues its work with the Idaho workforce system and prepare students for sustainable wages and additional educational opportunities. Action: The overall assessment of programs to improve quality and effectiveness was built into the bi-weekly AEFLA director meetings and the quarterly data desk audits. Reviews of the data and corresponding trends at both the program and state levels allow for more timely intervention and correction of issues. This approach contributed to the increases in some of the areas of performance. This will continue in future reporting. Additionally, data teams were created at the local programs to review data on a daily and weekly basis. If students needed to be placed higher or lower in a given class, the intervention was immediate. Likewise, if a student needed additional academic intervention, this too was immediately provided. Additional supports were added to programs such as group and individual academic tutoring and career plans. These interventions will continue in the future to increase retention and completion rates. |
| Illinois | Process: Illinois completed ongoing data analysis through desk-top monitoring from program support staff, data staff, and the Senior Director for Adult Education to monitor data quality. The required annual submission of a Data Quality Checklist reinforced the requirement of data integrity and identified any programmatic misunderstandings of adult education data, allowing for immediate and ongoing support. Additionally, the State MIS system prevents student files with errors to be included in the final report and the Error Status report in the MIS identifies errors that are corrected throughout the fiscal year.
Once the end of year reporting was completed, the ICCB Research and Analytics division compare current year data with prior year data to further test for anomalies and address areas of concern. There were no anomalies in the data for PY21.
Results: Illinois Adult Education providers’ performance were measured for outcomes by the following NRS Core Indicators of Performance:
|
| Indiana | Overall, Indiana continued to perform at high levels with respect to measurable skill gains, high school equivalencies, and credential attainment. Enrollment was up nearly 11 percent from the previous year, a strong emphasis on incorporating hybrid and virtual instruction, coupled with in-person classes was highlighted in professional development activities. In an assessment of core programs based on core indicators of performance, targeted instruction and a focused assessment program were key to achieving high measurable skill gain rates for ABE and ELL. These percentages exceeded core indicators of performance.
The pandemic hit ELL enrollment the hardest, and while recognizing challenges faced by ELL students with fewer in-person options, professional development activities focused on ensuring all students achieved measurable skill gains. Local providers modeled promising practices that were highlighted in monthly regional and professional development facilitator meetings and statewide webinars.
The measurable skill gain percentages for ABE and ELL were near the highest the state has ever achieved based on a concerted effort to ensure that learners were afforded the best instructional strategies and support services in uncertain times. While measurable skill gains were down approximately one percentage point from PY 2020, the overall performance outperformed national averages.
An analysis of enrollment trends was conducted in PY 2021 as numbers decreased from the previous year and as the state office began efforts to rebuild participation levels. However, the statewide online- only programs for ABE and ELL flourished and continued to grow during this period though concerns about retention and positive outcomes of students escalated as numbers climbed.
Building enrollments was a focus in PY 2021 ensuring quality and effectiveness of service. An analysis of target populations and “most in need” were an emphasis as well.
The spotlight on enrollments paid off as the pandemic eased and more ELL students returned to in-person classes. Nearly 2,000 more ELL students, a 54 percent increase, were enrolled in PY 2021 than the previous program year while ABE enrollments remained steady. Though ELL fueled the increase in statewide enrollment, measurable skill gains for these students dropped slightly.
PY 2021 Performance Outcomes
19,430
|
| Iowa | Performance Data Analysis
In PY 2021-22, a typical adult education and literacy student served was female (52.4%), employed (50%), a minority (67.8%), averaging 25-44 years of age (51.0%), with the highest year of school completed 9-12th grade (69.8%).
Iowa had a post test rate of 60% this program year with nine of twelve educational functional levels meeting the target and an overall measurable skill gain rate of 49%. We also had an increase in high school equivalency completers with a total of 1,549. Our participants' median program hours is 63 with those making a gain having 94 hours. Both fall within the recommended hours from CASAS, our assessment vendor.
Covid-19 Impact on Performance
Enrollment increased this program year with 9,328 participants and has yet to reach pre-pandemic levels of 11,039.
Distance education participants declined from 30.5% of participants last program year to only 12.8% of participants this program year. That said, it is higher than pre-pandemic levels. Also, the measurable skill gain rate was higher at 51.4% compared to our statewide average of 48.8% this program year and to the 40.0% rate last program year.
Iowa’s Adult Education and Literacy Enrollment
Overall enrollment has decreased 4.6 percent over the past five years with ABE enrollment decreasing 4.8 percent and ESL decreasing 4.3 percent. Participants increased 26.4 percent in PY 2022 with 9,328 participants compared to PY 2020-21 with 7,501 participants. Of the total number of participants that met the NRS guidelines, students in ABE, including ABE Levels 1-6, comprised the largest group by program type with 57.1 percent of the total learners served. Students enrolled in ESL, including ESL Levels 1-6, increased to 42.9 percent from 34.1 percent enrolled. This is a return to pre-pandemic levels in PY 2019-2020 with ABE at 56.1 percent and ESL at 43.9 percent.
Participant Enrollment
2018
2019
2020
2021
2022
Adult Basic Education
6,486
6,507
5,319
4,942
5,328
English as a Second Language
4,761
4,532
4,159
2,559
4,000
Total
11,247
11,039
9,478
7,501
9,328
Retention
Student retention is critical to the process of assessing progress. Persistence and sufficient hours increased slightly with 74.5 percent of the 12,529 individuals seeking services persisted for a minimum of 12 or more hours of instruction and a pre-test. Average hours of instruction for the 9,328 enrollees federally reported increased again this year to 74 hours which is up from 67 last program year. This remains less compared to the PY 2018-2019 where the average was 124 hours. ESL averaged the highest with 90 hours while ABE also increased to 62. Using ABE Level 1-5 and ESL levels 1-6, the posttest rate was 60%.
Of the 5,198 post tested, 70.0 percent achieved a measurable skill gain, up from 65.0 percent in program year 2020-21. Of the 663 participants at ABE Level 6, 70.5 percent achieved a measurable skill gain.
Measurable Skill Gains
Of the 9,328 participants, Iowa had 9,492 periods of participation. Of the 9,492 periods of participation, 48.8 percent (4,636) made a measurable skill gain. This is a 10.0 percent increase over PY 2020-2021 and exceeded our overall target of 47 percent.
Measurable Skill Gains
Functioning Level
PY18 MSG
PY19 MSG
PY20 MSG
PY21 MSG
PY22 MSG
PY22 Goal
ABE Beginning Literacy
43%
36%
26%
36%
39%
38%
ABE Beginning Basic
44%
35%
31%
34%
46%
45%
ABE Intermediate Low
44%
36%
32%
37%
42%
43%
ABE Intermediate High
42%
42%
39%
43%
51%
45%
ASE Low
60%
60%
48%
49%
55%
60%
ASE High
69%
61%
69%
70%
68%
ESL Beginning Literacy
54%
54%
36%
53%
46%
45%
ESL Low Beginning
52%
50%
39%
50%
53%
51%
ESL High Beginning
54%
47%
36%
47%
45%
51%
ESL Low Intermediate
47%
41%
33%
46%
46%
46%
ESL High Intermediate
49%
48%
36%
46%
49%
48%
ESL Advanced
33%
30%
26%
31%
35%
30%
Primary Indicators of Performance
Iowa utilizes a data match process to determine if performance targets have been met for the follow-up core measures. Employment the 2nd quarter after exit was down slightly to 52.3 percent from 53.0 percent the previous program year. Median quarterly wage earnings for 2nd quarter after exit were also down to $6,069 compared to last reporting year at $6,681. IELCE participants continue to have the highest quarterly earnings with $12,773 while IET participants have a high rate of employment at 73.7 percent. Distance learners were employed at 52.3 percent earning a quarterly wage of $5,156. Overall employment in the 4th quarter after exit is 54.5 percent.
Improvements
Quarterly reports have been implemented as an additional check. Our focus on achieving measurable skill gains with managed enrollment and instruction target to EFL proved successful. We exceeded our target of 47 percent measurable skill gains with 48.8 percent.
Future Directions in PY 2022- 2023
|
| Kansas | Despite the lingering effects of the pandemic, which particularly impacted coordination of services with corrections and employer partners, Kansas increased performance or remained steady in multiple core indicators. Employment (Second Quarter After Exit) saw an increase from 58% in PY2020 to 60.4% in PY2021, while Employment (Fourth Quarter After Exit), at 52.9% in PY2021, was within half a percent of the 53.3% from PY2020. The increase in participants finding and retaining employment, combined with rising wages in most areas of Kansas and the increased commitment of local programs to complete follow-up surveys among exited participants, resulted in a significant increase in Median Earnings, from $5510.41 in PY2020 to $5874.08 in PY2021. For the second year in a row, the Credential Attainment core indicator was the hardest hit, decreasing from 35.5% in PY2020 to 33.3% in PY2021. The majority of exiting participants indicated employment goals were higher priority at the time than educational goals. This decrease was also partially fueled by two-year institutions in Kansas, with which Adult Education programs work closely, struggling to staff sufficient in-demand courses, along with the aforementioned challenges in collaborating with workforce partners that would support participants transitioning into and continuing in postsecondary education. Measurable Skill Gains (MSGs) remained virtually the same, with 58.8% in PY2020 and 58.6% in PY2021. With Kansas Adult Education also struggling to fill staff positions, the posttesting rate dipped very slightly (from 70.5% to 70.2%), with a correlating slight decrease in total MSGs. The overall GED® pass rate remained high at 83% in PY2021, above the national average of 76%. To help address the teacher shortage in the state, Kansas launched a pilot program in PY2021 called Cross-TREK (Cross-Teaching Remote Education in Kansas) that brings together students from multiple locations from different areas of the state with an instructor at another site. Synchronous virtual classes give these learners access to high-quality, standards-aligned instruction, while supplemental assignments are completed online on the student’s own time. The pilot served ESL students from two programs, 66.7% of whom made an educational functioning level gain of one or more levels. In PY2022, Cross-TREK added a Math class for students working on diploma completion and is continuing to offer an ESL class, which concluded in December 2022 with 82% of participants achieving EFL gains. Total enrollment in PY2021 increased by 20.6% over PY2020, from 4324 to 5213, with an accompanying increase of 20.3% in high-school-equivalency diplomas earned. English Language Learners (ELLs) contributed to the greatest growth in enrollment, up 47.9%. Adult Secondary Education (ASE) participants decreased by 12.9%, while Adult Basic Education (ABE) participants saw a modest 6.4% increase. Adult Education programs continue to reach participants with barriers, populations that need more services and support to achieve outcomes. In addition to the quarterly data analysis completed by each local program and the state, in PY2021, Kansas introduced an extra measure of performance: a quarterly evaluation of each program’s progress toward goals, which encouraged celebration of incremental successes and increased efforts to areas of greatest need. At the end of the program year, the state published “Program Portraits,” a presentation of two years of data for each program and for Kansas as a whole, available to all Adult Education providers to facilitate cross-program discussion, with program leaders sharing strategies for success and seeking advice for underperforming areas. As determined by needs assessments, outreach and recruitment strategies were a major focus of professional development in PY2021, with a statewide initiative to launch standard marketing tools begun in spring 2022. The Kansas Adult Education Association (KAEA) is the lead on the project to create and disseminate these marketing tools for all programs to use. Another major concern in Kansas is the statewide shortage of Adult Education instructors. KBOR considered methods of expanding the talent pool, such as moderating state degree requirements while developing strategies to ensure high-quality instruction is maintained, with new guidelines being implemented in PY2022. The state believes increased communication, clarity, and focus on core indicators among local programs will help drive growth in performance. Each program must annually submit a Continuation Funding Application (CFA), which is reviewed by state staff and used, along with the risk assessment, as grounds for continuing funding in the next year of the grant cycle or instituting interventions or remedies for potential noncompliance. In PY2021, KBOR met individually with each program to discuss the CFA, gain greater clarification regarding the program’s goals for the next year, and set times for interim progress checks throughout PY2022. When negotiating PY2022 performance targets with local programs, the state included the sponsoring institution’s leadership to ensure awareness and support of Adult Education goals. For PY2022, the state has added another data analysis tool: intense and detailed reviews of data between state and local staff, covering 25% of programs each quarter. These discussions provide opportunity for adjustments and more strategic deployment of resources. The state continues to provide or facilitate professional development and technical assistance as needed, including more in-depth explanations of the core indicators and training for providing short-term focused secondary diploma completion classes, among other professional development. |
| Kentucky | Process The Office of Adult Education utilized two primary methods to analyze ongoing data trends for each local provider by service area as it relates to in program year performance indications (MSG and State-specific performance indicators: enrollment and GED Attainment). Weekly and monthly reports illustrated overall statewide performance trends in enrollment, GED attainment, and MSG rate, with comparison data from previous years. State Performance Rankings were released monthly to capture progress across the network. In addition to quarterly desktop audits, in which a sample of data entry was monitored and verified for ⅓ of fiscal agents, OAE periodically ran reports such as “Level 99”, “Participants by Age”, and others as deemed necessary. OAE identified outliers in the data set to resolve anomalies. In addition, the OAE is in the process of addressing the challenges associated with the remaining US ED performance indicators (Employment-2nd/4th Quarter after exit; Median Earnings-2nd Quarter after exit; and Credential Attainment). Success has been difficult to achieve, however, courses of action and processes are being analyzed to set the conditions for greater success in the future. Notwithstanding the effects of the pandemic, Kentucky saw a significant decrease in the employment after exist rate, the median earnings, and the credential rate. The Way Ahead is to enhance the technical assistance and monitoring process as a component of the newly established Monitoring and Compliance Attestation Tool to focus on the full spectrum of US ED performance indicators and student outcomes as opposed to the previous focus on GED Attainment exclusively. The OAE has significantly modified the approach to training and engagements with the local provider network/sub-recipients to ensure that Sections 202 and 203 of WIOA, 2014 are clearly understand in support of the service delivery across the full spectrum of the definition of adult education and family literacy. Results OAE established state performance targets for each Fiscal Agent utilizing ACS data of individuals with less than a high school diploma. The negotiated MSG rate was 51.4%. KYAE Local Providers finished Program Year 2021 with an MSG rate of 46.67%. Fourteen of the twenty-seven Local Providers met the negotiated MSG rate. Seven programs achieved 50% of their enrollment goal, and eleven Local Providers reached 50% or higher of their GED goal. Multiple variables to include the impact of the pandemic contributed to the decreased performance across the state. OAE provided targeted support to the larger providers to reverse this trend. Below is a table representing data from the largest local provider. Performance Year 2021 concluded with Kentucky failing to achieve the negotiated rates for employment, median earnings, and credential attainment. Employment-2nd Quarter was 42.96%; Employment-4th Quarter was 29.74%; Median Earnings-2nd Quarter was $3,870.60; and Credential Attainment was 20.83%. Analysis Statewide performance increased in PY21 due to a rebound from the COVID-19 pandemic, a return to services in state and local correctional facilities and increases from Integrated Education & Training and Workplace Literacy programming. At both state and local levels, KYAE experienced a significant turnover and a need for intensive staff development with the loss of veteran educators and the hiring of 75 new instructors across the state in the program year. The impact of the pandemic was significant in Kentucky consistent with challenges experienced across the nation, however, the pandemic does not address the totality of the issues in Kentucky as it relates to service delivery and performance. A new leadership philosophy has been adopted in Kentucky that focuses on US ED requirements and success across all five performance indicators vice a mindset of exclusively emphasizing GED Attainment. Despite the challenges presented by COVID and the onboarding of new staff, the opportunity for Online Proctored GED (OP GED) testing impacted opportunities and brought some successes. According to GED Analytics, 16.8% of all tests were online proctored, demonstrating an increase of 13.68% from the previous program year. To set the conditions for greater levels of service delivery and performance, the OAE negotiated with US DOE, OCTAE to amend the following performance goals for Program Year 2022 to the following levels: ADULT EDUCATION AND LITERACY PROGRAM PERFORMANCE INDICATORS (KENTUCKY) Performance Negotiations Table: PERFORMANCE INDICATORS PY 2022/FY 2023 Expected Level PY 2022/ FY 2023 Negotiated Level Employment Rate (Q2) 45.00% 39.2% Employment Rate (Q4) 44% 41.2% Median Earnings $4,125.00 $4,000.00 Credential Attainment Rate 48.00% 38.6% Measurable Skill Gains (MSG) 51.4% 45.2% PY 2022/FY 2023 HSE Attainment Goal and Total Enrollment of Participants: HSE Attainment Goal 3,600 Total Enrollment of Participants 20,000 Action OAE’s monitoring and evaluation of these goals consist of: Quarterly Performance Check-ins, Desktop Audits, and On-Site Compliance Visits. KYAE has implemented additional accountability processes by requiring the LPN to adhere to KYAE operational calendars and contingency plans. In alignment with 2 CFR 200.332 (e)(1) KYAE addressed local program performance issues immediately by focusing on the bottom ⅓ of the state rankings to provide Technical Assistance. This consisted of making site visits to those providers to identify areas of improvement and provide technical assistance. Desk Audits were conducted, and Quarterly Performance Check-Ins were completed. Providers also had the opportunity to request TA and virtual meetings were conducted with program directors to provide feedback to increase performance. |
| Louisiana | In terms of the overall assessment of core programs based on the core indicators of performance, WorkReady U served 20,763 (Table 2A + Table 4) undereducated and underemployed citizens in FY 2021-22 with 8,975 students participating in basic skill instruction (NRS Tables 4). Data reveals that 43% of those who participated in academic services during the FY 2021-2022 program year made a measurable skill gain (MSG) (NRS Table 4). While we continued to show reduced enrollment compared to pre-pandemic levels, we are pleased to know that enrollment ticked up in FY 21-22 and the percentage of students making a measured skill gain increased by nearly 3%. Effective instruction at the local level is evident by the fact that 60% of students post-tested (NRS Table 4B) demonstrated an EFL gain. Despite the incredible challenges to intake, assessment, and instruction posed by natural disasters and the COVID-19 pandemic, a review of the 2020-2021 adult education performance data indicates some growth areas. The percentage of students with a measurable skill gain (MSG) increased by 3% and the number of students post-tested increased by 17%. Additionally, the number of adult learners who earned an industry-based credential increased by 12% over the previous fiscal year. This shows that our providers, while significantly challenged by the obstacles of the pandemic and natural disasters, have remained committed to innovating and building educational opportunities for Louisiana’s adult learners. Louisiana was also committed to ensuring that all individuals who were undereducated and underemployed had an opportunity to receive basic skill remediation and an opportunity toward self-sufficiency as evidenced by Table 3 which shows that approximately 99% of the students served were Adult Basic Education or English Language learners. This long-term commitment to Louisiana’s most at-risk citizens is evidence that WRU is dedicated to building a stronger Louisiana by providing comprehensive adult educational services. To improve program quality and effectiveness, WRU encouraged data sharing and discussion with faculty and staff. When daily, weekly, and monthly data reports are analyzed, specific targets and goals for improvement are uncovered. Data is used to determine necessary instructional shifts as well. These data-driven shifts enable teachers to better prepare their students. Utilizing data from the Student Information System (SIS), it is our goal to see areas where we need improvement and those in which we excel. Plans to further increase performance revolve around a set of quality indicators developed jointly with providers, state staff, and stakeholders. The indicators are aligned with federal and state considerations for funding programs, and performance on the indicators will eventually become part of the funding formula. Elements of the quality indicators framework include the proportion of low-skilled students served, the number of older adult students served, implementation of digital literacy instruction and assessment in programming, demonstration of workforce preparation and occupational skills training for learners, and an analysis of program effectiveness and efficiency in the use of funds. |
| Maryland | All WIOA programs are measured on several core indicators of performance. The information that follows demonstrates the performance of Maryland’s Title II programs on these core indicators. Employment Rate/Median Earnings Maryland’s Title II employment rate 2nd quarter after exit for all Periods of Participation was 33.80% in PY21. The Title II employment rate 4th quarter after exit for all Periods of Participation was 35.99% in PY21. Median earnings 2nd quarter after exit for all Periods of Participation was $5,525.00. Measurable Skill Gains Considerable changes in enrollment trends, course delivery types, and remote proctoring continued in PY21. While MSG attainment is not yet as high as Maryland would like, PY21 saw considerable growth in this area due to increasing enrollment and the return to in-person assessment. After showing a noticeable decline in enrollment from PY19, enrollment from PY20 to PY21 showed double-digit percentage increases. ABE enrollment increased year-over-year by 20.50%, ESL by 24.18%, and overall enrollment by 27.84%. The ESL enrollment increase was especially heartening, as it saw a significant decrease during the pandemic. Despite the impressive one-year gains, enrollment is still below pre-pandemic levels, with a decrease of 24.46% in overall enrollment when compared to PY19 levels (Overall PY19 enrollment was 21,473). Another bright spot in enrollment trends is the average attendance rate. Average attendance hours per participant saw an increase this year at 91.99 hours on average, an increase of 3.33 hours per participant over PY20. In PY21, 5,808 out of 16,985 Periods of Participation saw a Measurable Skill Gain, resulting in a 34.19% MSG rate. This was below the negotiated level of performance of 43%, but it was an increase of 13.27 percentage points over PY20 performance. While enrollment is still lower than the pre-pandemic level in PY19, this MSG rate does exceed the 28.64% MSG rate set in PY19. For high school diploma attainment, completion levels were on par with pre-pandemic years. A total of 417 participants earned a secondary school diploma in PY21, compared to only 230 in PY20 and 426 in PY19. Of the 417 completers in PY21, 78 were with the National External Diploma Program and the remaining 339 were through the GED Tests Credential Attainment The credential attainment rate for all Periods of Participation was 26.98% in PY21. This lags the negotiated target of 37%. However, this result is considerably higher than in previous years. Maryland credits increased data training on the importance of this metric and how to properly capture and report it in our data system in making great improvements in this level. With the renewal of Maryland’s license with LiteracyPro’s LACES system for a new five-year contract, training on utilizing the data system to its full potential to benefit the reporting of all metrics will continue in earnest. |
| Michigan | The adult education programs in Michigan showed signs of rebounding from the pandemic in PY 2021. Both enrollment (1+ hour of instruction) and participant counts increased by just over 20%, but are still about three thousand below pre-pandemic levels. The number of ESL participants increased by 41%, while ABE/ASE participants increased by 13% over the year. The participant barriers data collection has improved significantly in recent years since this section was updated on the Adult Learning Plan (ALP) in PY 2020. The participant barriers section now includes questions for participants that define each barrier. Adult education providers have shared that these questions have been very helpful for both staff and participants. Since PY 2019, the number of participant barriers reported has more than doubled. In PY 2021, the most prevalent barriers for adult education participants were Low Income (47%), Long-Term Unemployed (28%), Individuals with Disabilities (20%), Ex-Offenders (17%), and Single Parent (13%). The decline in correctional education programming continues to impact statewide enrollment and performance. While the number of participants in correctional education programs increased over the year from 3,384 to 4,685, this is still significantly below the pre-pandemic level of 7,002. The Michigan Department of Corrections (MDOC) enrollment increased by 26.5%, but the MSG rate continues to be well below the statewide average at 27%. As such, the statewide MSG rate continues to be below pre-pandemic levels. Adult education providers in Michigan had outperformed the statewide MSG targets in PY 2017 and PY 2018, with more than 50% of participants reporting a measurable skill gain, but that percentage dropped drastically to 35.95% in PY 2019. For the second straight year, the MSG rate has increased slightly and was 38.22% in PY 2021. Over the year, there was a significant increase in both educational gains from pre/post-testing and high school equivalency attainments at 26% and 39%, respectively. Adult education participants in Michigan also had success obtaining employment after exit from the program. Michigan met or exceeded the target for Employment 2nd Quarter After Exit, Employment 4th Quarter After Exit, and Median Earnings. The statewide average for Credential Attainment fell shy of the target with 31% achieving that outcome measure. The percentage of participants that earned a diploma or its recognized equivalent and transitioned to postsecondary within a year was only 4%, while the percentage that attained a diploma or equivalent and are employed within one year was just over 30%. With the launch of the Futures for Frontliners and Reconnect scholarship programs in Michigan, we hope to see the percentage that continue onto postsecondary education increase. LEO-WD is also working to implement data match with community college records to improve the collection and reporting of this data. The number of distance learners declined by 21% over the year, as most providers were able to offer instruction in-person. Similarly, the MSG rate for distance learners declined over the year and fell below the statewide average for all learners. This aligns with what the state office heard anecdotally from many directors and teachers – that their distance learning participants were not making the gains that their participants attending in-person were making. Providers noticed that distance learners were not putting in the hours needed to make gains. While most providers had intended to continue offering virtual learning options based on lessons learned during the pandemic, the latest trends were forcing more providers to put more emphasis on face-to-face instruction. To support the evaluation of these trends, MAERS now allows users to run performance reports based on a range for the percentage of distance learning hours of total attendance hours. For example, the Office of Adult Education recently did an analysis of enrollments and performance of distance learning hours in increments of 25%. The highest number of learners had between 76-100% of attendance hours from distance learning, while participants with between 1-25% of distance learning attendance hours had the highest MSG rate. Participants with between 50% or less distance learning attendance hours had a higher MSG rate than the statewide average, but those with more than 50% reported gains below the statewide average. The Office of Adult Education shared this data with local providers and encouraged them to do a similar analysis to guide program improvement going forward. |
| Minnesota | Minnesota Adult Education Performance Results.
WIOA Indicator
FY 2019–2020
FY 2020–2021
FY 2021–2022
Measurable skill gain (MSG)
36.89%
23.01%
30.58%
Employment at second quarter after exit
37.74%
34.25%
36.16%
Employment at fourth quarter after exit
36.46%
35.26%
34.06%
Median quarterly earnings at second quarter after exit
$6,070
$6,188
$6,889
Credential Attainment
13.41%
19.03%
17.13%
Key data points for the Minnesota Adult Education system include:
|
| Mississippi | Mississippi served 8,546 (NRS Table 2A + Table 4) undereducated and underemployed citizens in FY 2021-2022 with 5,915 students participating in basic skill instruction (NRS Tables 4). Mississippi achieved a Performance Rate (MSG) of 56.98% for program year 2021-22, exceeding the required MSG target of 50% (NRS Table 4). Effective instruction at the local level is evident by the fact that 70.23% of students who post-tested (NRS Table 4B) demonstrated an EFL gain. Mississippi OAE works with the National Strategic Planning and Analysis Research Center (NSPARC) to data match for the primary indicators of performance on Table 5. Results for PY 2021-2022 participants show: 59.94% were employed second quarter after exit; 57.85% were employed fourth quarter after exit; the median earnings second quarter after exit is $3,736.00; 36.36% attained a secondary school diploma/equivalent and enrolled in postsecondary education or training within one year of exit; 34.29% attained a secondary school diploma/equivalent and were employed within one year of exit; 38.78% attained a postsecondary credential while enrolled or within one year of exit; and 42.66% attained any credential (deduplicated). Mississippi has 82 counties with 19 local providers, accounting for over 300 classes, providing services to students in each county. Adult education did experience a decline in local staff in 2021-2022 with an overall total of 285. Mississippi still experienced the effects of COVID-19 in the 2021-22 program year with programs navigating class site closures, limited seating, student’s lack of reliable technology, while at the same time ensuring adult learners and adult educators are acclimated to the virtual environment, specifically with coursework and assessments. Furthermore, students overcoming barriers such as transportation issues, participant attendance decreases due to inconsistent schedules with their children’s needs, or simply still having a fear of exposure, brought about decreased participation. Many programs have yet to recover in student attendance compared to pre-pandemic enrollment. To continue the trend of performance achievement, local programs were provided target percentages for each educational functioning level (EFL) and analyzed findings on a monthly basis through the utilization of the Desktop Monitoring Tool. This allowed programs to identify specific areas in which to focus in a timely manner. Technical assistance has become strategically focused on ways to earn an MSG, components of the NRS data tables, and ways to improve performance. We encourage the use of the OAE-developed Instructor Class Monitoring Tool (https://www.mccb.edu/sites/mccb/files/Adult%20Education/Local%20Program%20Resources/Instructor_Class_Monitoring_Tool.pdf) which allows programs to disaggregate data at the classroom, instructor, and student level in order to compare their performance to the benchmarks and identify which staff members need additional support. Important state data which is not reflected in the NRS data is the number of work ready certifications earned through adult education. Two of the state recognized certifications in Mississippi are the National Career Readiness Certification (NCRC) and the Smart Start Certificate and/or Credential which aligns with our State WIOA Plan. The number of National Career Readiness Certifications (NCRC) for FY 21-22, based on the ACT Work Keys assessments, was 2,801. Also, the state awarded over 2,120 Smart Start Credentials to adult education students supporting the State’s mission of producing individuals with the necessary skills needed to be successful in the workplace as well as life. For FY 2021-2022, students enrolled in Smart Start, specifically those below the 8th grade level, out performed classroom peers in their educational and career goals with 61.02 percent earning a MSG, as opposed to those who were not enrolled in the course with 52.32 percent MSGs. MS OAE continues to develop reports, trainings, and resources which support local programs’ abilities to analyze performance in real-time and use data to drive instruction, retention, and transition. |
| Nebraska | In Program Year 2021-2022, Nebraska continued to shine through the latter part of the pandemic. Despite ongoing challenges early on with student attendance and the availability of in-person classes, Nebraska Adult Education achieved an MSG performance of 46.32%, which once again represented the highest MSG performance ever achieved by Nebraska Adult Education. Title II was also able to exceed the negotiated performance benchmark for measurable skills gains, which is an ever-present goal. Nebraska Adult Education performed at a high level on all exit-based indicators, making fourth quartile performance in all four measures. Statewide job availability and a low unemployment rate continued during Program Year 2021-2022, which again aided positive outcomes for adult learners. Industries common in the state tend to continue operations, even during pandemic circumstances. With industries such as transportation and logistics, advanced manufacturing, meat processing and agriculture, job availability continued to be a positive. Additional industries like retail and entry-level tech allow for additional job availability. To support the goal of placing adults in long-term, sustainable employment and postsecondary education, Nebraska Adult Education continued to prioritize student learning through a robust goal setting and career pathways process, ensuring student’s needs were being met academically, while providing information and data to support positive placement post-exit. As students identified postsecondary education as their goal, it was important to understand that post-exit employment outcomes were delayed for many adult learners achieving HSE goals. These adult learners tended to obtain lower paying jobs while focusing their time on achieving additional educational credentials, with the overall goal of securing higher paying employment after earning their HSE credential. Nebraska’s providers continued to increase post-exit performance through improved surveying of exited participants. Since Nebraska has a high percentage of students without social security numbers, surveys were necessary to show the positive outcomes of students enrolled in Title II programs. As previously stated, Nebraska Adult Education continued to prioritize the individual learning goals of adults entering programs across the State. Assessment was an important and vital tool used to measure the progress of adults in the academic classroom. With the ability to delivery both instruction and assessment in-person and virtually, students were able to succeed at a higher level. With the continued negativity towards the use of assessments in adult education by employers and non-adult education critics, it was more important than ever to prioritize language learning and high school equivalency credential preparation over short-term employment placements to demonstrate the benefits of academic assessment in adult education and better support the students seeking long-term, higher paying employment. When looking at the goal of earning a HSE credential by most adult learners, it is vital to the economy and the future of the state that educated and literate adults are transitioned into the workforce. Short-term workforce agendas with the goal of placing adults in the workforce was a short-term solution. The benefits of integrated education and training although certainly justified and successful for certain demographics, intensive support through individualized career pathways and educationally focused instruction that includes the goals of earn a high school equivalency credential and matriculation into community college was most necessary to achieving the end-goal of long-term sustainable employment for most adults and positive exit-based outcomes for adult education. |
| Nevada | The Nevada Business Accountability Specialist for Adult Education tracked enrollment data and MSG performance data monthly during PY21, looking at individual program data. Local programs were informed at bi-monthly shop talk calls and programs with identified data anomalies were contacted individually to resolve errors. The enrollment for PY21 was affected by the pandemic but recovered somewhat from the first year of the pandemic. Programs have continued to increase the number of students enrolled in virtual learning, although courses were offered in-person during the 2021-2022 program year. However, Measurable Skill Gains were up with the approved target met with an overall NRS Table 4 outcome of 51.64% and a target of 43%. There were 112 participants in distance education in PY19, 808 in PY20, and 955 in PY21. The significant difference can be attributed to the fact programs continued to increase the distance education available, as well as, increasing options for hybrid, hyflex learning. Many students chose to continue distance learning even with in-person classes resuming. We saw a continued improvement with the overall MSG reaching a percentage point higher than the previous year. This improvement can largely be attributed to the fact that those post tested achieved an MSG at a rate of 72% during PY21 and 70% the previous year and the post-test rate stayed at approximately 69% with the state target at 70%. Performance outcomes were up in most educational functioning levels. Gains were down slightly in ABE levels 2, 3, and 6 and ESL levels 5 and 6, with gains in all other levels increasing. The Warning, Probation, and/or Termination Status system put in place has led to significant improvement among the programs that had been placed under Warning Status. One of the programs previously under Warning status was the highest performer for PY21. The number of participants achieving high school equivalency through the GED, or HiSET assessments continues to decrease from previous years, at 129 in PY19, 122 in PY20, and 120 in PY21. After a significant increase in the number of individuals served in IET programs from 118 in PY19 to 174 in PY20, the number served in IET or IELCE went down to 130 in PY21. Programs will continue to receive additional technical assistance in developing and expanding integrated education and training programs. There was a significant increase in the outcomes on Table 5, with employed second quarter after exit going from 25.66% to 46.1%. This increase can be attributed to being able to use the SWIS data for the first time. The fourth quarter after exit employment went from 25.7% to 38.8%. The median earnings went from $6,120 up to $8,320. This is likely due to the reduction in lower paying jobs reported following the pandemic. |
| New Jersey | The lingering effects of the pandemic continue to be felt throughout New Jersey’s adult education programs during PY 2021. Enrollment, testing, retention, and performance were impacted by the pandemic and continue to pose challenges. The digital divide exposed challenges for participants in many ways. The obstacles were overwhelming, and students and staff encountered personal, medical, educational, and employment challenges. Fortunately, in PY 2021, for the first time in a great number of years, NJ was able to provide Title II providers with state funds for loaner device programs, technology to enhance hybrid and distance learning, and the addition of technology staff positions to programs.
At the start of the pandemic massive professional development plans were put together. Instructors became fluent in Google Classroom and Schoology among other resources including Kami, Nearpod, Padlet, etc. and online software tools such as Aztec and Burlington English became the norm. Agency staff had to support learners with increased flexibility and put great effort into engaging students. Initially, distance learning and remote instruction were often the only option. Agencies utilized instructional options for participants via computers, tablets, cell phones, and worksheet packets. Videoconferencing tools such as Google Meet, Zoom, etc. were the new way to provide engaging and interactive instruction. Though the option for in-person instruction has increased greatly this year there are still many programs offering hybrid instruction or remote options as this worked very well for some students and at times is the preferred option depending on a student's other responsibilities outside of their education.
The overall number of participants enrolled from PY 2020 to PY 2021 remained very close. In PY 2020, 11,709 participants were enrolled. In PY 2021, 11,714 participants were enrolled. There was a decrease in the number of distance learners as more providers moved back to in-person instruction options. The number of distance learners in New Jersey decreased from 9,420 students in PY 2020 with a total of 962,252 attendance hours to 3,579 students in PY 2021 with a total of 301,799 hours. NJ had 5,841 less distance learners and 660,452 less attendance hours in PY 2021 than in PY 2020. In PY 2021, 30% of the participants were distance learners compared with 80% in PY 2020, 12% in PY 2019 and 4% in PY 2018.
Although agencies were more familiar with remote instruction than when the pandemic hit there were still some challenges to overcome and planning and coordination to be considered; especially for programs offering hybrid instruction or both in-person and remote/hybrid options. Performance was still affected amid the uncertainty of what the program year would look like, and the difficulties participants continued to face. Providers who lost staff during the pandemic often struggled to replace positions. Some providers faced challenges because of Hurricane Ida and had to re-locate and deal with other planning difficulties due to extensive flooding. Some of the relocations were not as easily accessible by public transportation, causing a hardship for students who don’t drive.
The NJ Department of Corrections serves between 1,000 and 2,000 of NJ’s adult basic education students each year and in PY 2021 there were several changes made throughout NJ’s correctional system that affected performance. Southern State Correctional facility and Mountainview Correctional facility closed between June and December. Southern State Correctional facility housed 800 inmates, 400 of whom were moved to Bay Side Correctional facility and would not be able to be post tested. In addition, 4,500 inmates were released a year early because of COVID without post testing. Prison laws will continue to allow the release of inmates periodically. NJ DOC staff indicated that they foresee a shift in their Measurable Skill Gains (MSGs) performance compared with pre-COVID performance but are working diligently to develop new ways to ensure increased performance.
During the pandemic NJ allowed providers to use provisional assessments but that allowance was removed for PY 2021. All agencies are pre and post testing with NRS approved assessments. In PY 2021, NJ met its overall MSG target of 54%; 6 of 12 Educational Functioning Level (EFL) MSG targets were met. The targets that were not met were all ABE/ASE levels. NJ exceeded reaching its ESL targets by between 1% and 10% and missed meeting its ABE targets by between 5% and 14%.
NJ is very proud to report that despite continued challenges, that our providers have met NJ’s overall MSG target through so much dedication, persistence, and hard work and even though NJ did not meet individual PY 2021 ABE/ASE targets there was significant improvement from last year. NJ didn’t meet any of the ABE/ASE targets last year either but the percentage ranges that the targets were missed by last year, in PY 2020, ranged from 13% to 24%, whereas the range for this year, PY 2021, was 5% to 14%.
From PY 2020 to PY 2021 NJ demonstrated substantial increases in almost all of the ABE/ASE and ESL EFLs. The increases were greatest for ABE/ASE students. ABE Level 1 increased from 28% to 42%; ABE Level 2 increased from 31% to 39%; ABE Level 3 increased from 29% to 39%; ABE Level 4 increased from 34% to 43%; ABE Level 5 increased from 33% to 45% and ABE Level 6 increased from 31% to 42%. From PY 2020 to PY 2021 the ABE/ASE level increases ranged from 8% to 14%. These increases resulted in an average gain of 11% for ABE instruction which is up from a 4% increase from PY 2020.
For PY 2021, ESL increases were obtained in Levels 1, 2, 4, and 5 and ranged from 2% to 9%. Although there no increases in ESL Levels 3 and 6, the decrease was just 2%. Despite the difficulties and uncertainties that remained during PY 2021, there was an 8% increase in the number of participants post tested compared to the previous year, which is very positive in light of some of the NJDOL and Hurricane Ida issues that arose.
Prior to the pandemic in PY 2018, New Jersey’s adult education agencies post tested 70% of their students. In PY 2019 they were only able to posttest 54% of their students and in PY 2020 they were able to posttest 58% of their students. This year PY 2021, 66% of NJ’s participants were post tested meeting the required posttest rate of 60%.
A comparison of New Jersey’s performance targets for PY 2019, 2020, and 2021 and actual performance toward meeting the targets is provided in Chart 1 below. There is a clear trend of improved performance percentages as NJ made its way through the pandemic and it is expected that NJ’s performance will continue to increase.
Chart 1: New Jersey Adult Education Performance for the last three years
Performance Objectives
PY19
Target
PY19
Performance
PY20
Target
PY20
Performance
PY21
Target
PY21
Performance
ABE Level 1
51%
20%
51%
28%
52%
42%
ABE Level 2
49%
26%
50%
31%
51%
39%
ABE Level 3
46%
29%
50%
29%
51%
39%
ABE Level 4
43%
32%
47%
34%
48%
43%
ABE Level 5
56%
29%
57%
33%
59%
45%
ABE Level 6
45%
26%
52%
31%
53%
42%
ESL Level 1
57%
54%
57%
55%
58%
64%
ESL Level 2
56%
57%
56%
63%
57%
65%
ESL Level 3
59%
52%
57%
63%
58%
61%
ESL Level 4
57%
50%
57%
54%
58%
59%
ESL Level 5
45%
44%
45%
53%
46%
56%
ESL Level 6
38%
33%
40%
46%
41%
44%
Overall MSG
baseline
59%
53%
47%
54%
54%
Employed Q2…
baseline
44%
45%
38%
46%
36%
Employed Q4…
baseline
46%
43%
39%
44%
39%
Med Earn Q2…
baseline
$5,930
$4,783
$6,370
$4,833
$6,747
Attained SSD & PSE/Training…
baseline
3%
2%
4%
3%
3%
Attained SSD & Employed…
baseline
22%
18%
25%
19%
14%
Attained PS Credential…
baseline
40%
58%
41%
59%
25%
Attained any Credential…
baseline
31%
27%
34%
28%
19%
This chart shows data over the past three years on primary indicators of performance and; MSGs in the areas of: Adult Basic Education (ABE) and English Language Acquisition (ELA). Chart 1 shows improvement in almost all EFLs, median earnings and overall MSGs, which went from 47% in PY 2020 to 54% in PY 2021.
For years prior to the COVID-19 pandemic, New Jersey demonstrated consistent and significant improvement in all of the Measurable Skill Gains (MSG)/Educational Functioning Level (EFL) gain areas. For PY 2017 and PY 2018 New Jersey met all of its performance targets and was on track to meet their performance goals for PY 2019 when COVID-19 shocked all of New Jersey but especially our adult education programs. There was great improvement from PY 2019 to PY 2020 especially in regard to English Language Learners (ELLs) but there is even greater improvement from PY 2020 to PY 2021 in regard to MSGs in virtually all EFLs.
During PY 2021, educational services and instruction to adults were provided through a variety of program offerings including: ABE/ASE, ELA, volunteer literacy programs, adult high school programs, High School Equivalency (HSE) test preparation, and programs at state and local correctional facilities. Despite the pandemic, agencies continued providing adult education services consistently through the program year. The provision of supportive services and information outside of the direct realm of adult education became increasingly necessary and agency staff did an amazing job of supporting and helping participants in many ways.
The ABE/ELA and Integrated English Literacy and Civics Education (IELCE) program services were delivered through 18 grantees with multiple delivery sites throughout the counties coordinated by either a lead agency heading a consortium or a single entity. Agencies included community/technical colleges or public and private non-profit agencies and New Jersey State correctional institutions. Together, 76 agencies participated in delivering services to 11,714 participants in PY 2021.
Chart 2 provides a breakdown of participants by EFL for PY 2020 and PY 2021 demonstrating the enrollment difference by levels between the years. In PY 2021, New Jersey’s WIOA Title II agencies served 11,714 adult participants. In PY 2020 they served 11,709 adult participants. The populations served for PY 2021 included 4,110 ABE participants (3,439 ABE Levels 1 - 4 participants and 671 ABE Levels 5 and 6) and 7,604 ELA participants.
Chart 2: New Jersey Adult Education Enrolled Participants
Educational Functioning Level
PY 2020
Total Enrollment
PY 2021
Total Enrollment
Enrollment Difference between
PY20 and PY21
ABE Level 1
581
476
105 less students
ABE Level 2
713
657
56 less students
ABE Level 3
1,121
1,093
28 less students
ABE Level 4
1,276
1,213
63 less students
ABE Level 5
498
477
21 less students
ABE Level 6
258
194
64 less students
Total ABE
4,447
4,110
337 less students
ESL Level 1
2,124
2,484
360 more students
ESL Level 2
1,662
1,797
135 more students
ESL Level 3
1,116
1,161
45 more students
ESL Level 4
1,025
960
65 less students
ESL Level 5
921
819
102 less students
ESL Level 6
414
383
31 less students
Total ESL
7,262
7,604
342 more students
Total ABE and ESL
11,709
11,714
5 more students
New Jersey’s ELA programs are offered at different venues from literacy-based volunteer programs to adult education programs. The ELA population represented approximately 65% of all the students being served in the WIOA Title II program. This specific population may have limited reading or writing skills in any language and may function minimally or not at all in English. For the volunteer-based programs, most services are provided through tutor-learner matches that deliver one-on-one or small-group instructional services. Most of the ELA students served (33%) were assessed at the ESL Level 1 EFL. This information is reflected in chart 3. This population may only understand isolated words or phrases, may have no or minimal reading or writing skills in any language, and functions minimally or not at all in English.
Chart 3: New Jersey Adult Basic Education English Language Learners
Educational Functioning Level
Total Number of ELLs PY 2021
Total Percentage of ELLs PY 2021
ESL Level 1
2,484
33%
ESL Level 2
1,797
24%
ESL Level 3
1,161
15%
ESL Level 4
960
13%
ESL Level 5
819
11%
ESL Level 6
383
5%
Total
7,604
100%
From PY 2020 to PY 2021 our programs demonstrated an increase in the overall ESL MSG from 57% to 61%. There were increases in educational gains in ESL EFLs 1, 2, and 5. There was a 9% increase at Level 1, a 2% increase at Level 2 and a 3% increase at Level 5. Of the 11,714 students served in the WIOA Title II program for PY 2021, 7,604 of them were English Language Learners (ELLs). New Jersey met all the ESL EFL gain targets for PY 2021. For PY 2020, NJ only met 4 of the ESL EFL targets.
New Jersey embraces its cultural diversity. New Jersey’s diverse makeup is consistent with its status as a gateway for immigrants. Agencies were required to integrate an English language/civics education component for their Section 243 funded IELCE programs. They are required to design programs to prepare ELLs for employment in in-demand industries and occupations that lead to economic self-sufficiency and integrate with the workforce development system. The IELCE programs support the immigrant community and other limited English proficient populations.
During PY 2021, 2,070 participants were served through the IELCE program and almost 200 more students passed their citizenship exam compared with PY 2020. Two hundred and eighty-three of our ELLs passes their citizenship exam between July 1, 2021 and June 30, 2022, compared with just eighty-five students in PY 2020. The IELCE program emphasizes contextualized instruction on the rights and responsibilities of citizenship, naturalization procedures, civic participation, and U.S. history and government, in accordance with the National Standards for Civics and Government. Types of activities supported by these funds included:
|
| New Mexico | Enrollment and its dynamic serve as a good indicator of overall health of the program. New Mexico’s NRS enrollment trends for the past four years are highlighted in the table below. Program Year Beginning Literacy Adult Education Adult Secondary Education ESL Total Enrollment PY 18/19 2632 3813 361 4,112 10,918 PY 19/20 2275 3462 230 3,527 9,494 PY 20/21 1367 2264 255 1,424 5,310 PY 21/22 1410 2375 259 2,010 6,054 The enrollment in PY 21/22 started showing signs of post-pandemic recovery. While the enrollment fell over 44% in PY 20/21 from PY 19/20, it increased by over 14% in PY 21/22. The largest increase was in ESL enrollment that showed growth of over 41%. While numbers have not reached the pre-pandemic levels just yet, a gradual recovery in numbers of participants is expected. Different local programs showed different performance in enrollment numbers, with a spread between 37% decrease to 160% increase in enrollment. New Mexico is hoping to use face-to-face instruction alongside the online learning experience gained through the pandemic to increase the enrollment by more than 14% in PY 22/23 and get closer to pre-pandemic levels. Another important performance indicator to examine is measurable skill gains. Four years of the aggregate data is presented in the following table. Measurable Skill Gains Program Year ABE Level 1 ABE Level 2 ABE Level 3 ABE Level 4 ABE Level 5 ABE Level 6 ESL Total Enrollment PY 18/19 37.1% 37.4% 33.3% 33.9% 42.2% 37.8% 36.4% 35.7% PY 19/20 37.9% 32.2% 28.9% 31.7% 30.2% 16.7% 29.4% 30.3% PY 20/21 40.3% 31.8% 28.3% 33.3% 27.1% 29.4% 31.1% 31% PY 21/22 37.5% 36.4% 37.1% 42.8% 31.9% 40.3% 33.9% 36.5% Overall % MSGs have improved significantly in PY21/22 - by more than 5% - and have now surpassed pre-pandemic levels. The same trend can be seen in almost all MSGs levels. The largest increase is seen in ABE Level 6. While the MSGs were either decreasing or staying steady in PY 18/19 – PY 20/21, this is a first increase seen in recent years. It can be attributed to program recovery from the pandemic as well as improved data procedures. Post-exit indicators also serve an important role in overall assessment of program performance and are presented in the next table. Indicator of Performance PY 20/21 PY 21/22 Target for PY 21/22 Difference of Targets to PY 21/22 Performance Employment 2d Quarter after Exit 22.0% 37.2% 25.0% +12.2% Employment 4th Quarter after Exit 10.6% 23.5% 35.3% -11.8% Median Earnings Employment 2d quarter after Exit 3,730.30 4,219.96 3,220.00 +999.96 Credential rate 26.86% 22.8% 37.1% -14.3% Most of the core performance indicators have increased in PY 21/22 compared to PY 20/21. The only decrease is seen in Credential attainment rate. This decrease can be attributed to the remaining effects of the pandemic since this indicator lags a year behind the reporting year. The decrease can also be attributed to difficulties with our post-secondary match. Employment 4th Quarter after-exit and Credential rate are still below the negotiated targets. Improvements in gathering after-exit information are planned for next year. Initiatives to encourage credential attainment rate are also in process during PY 22/23. |
| North Carolina | For PY 2021-22, the COVID pandemic continued to have an adverse impact on North Carolina's MSG rates and negatively impacted the enrollment numbers of participants throughout the state. Pre-COVID, the State of North Carolina’s participant enrollment was 41,261 (PY 2019-20). During PY 2021-22, participant enrollment was 37,742. However, this is a significant increase over PY 2020-21 where the state had only 31,309 participants. Participant enrollment increased 21% for PY 2021-22 which is most encouraging for our state, providers, and students. Pre-COVID, the State of North Carolina saw continuous improvement with measurable skill gains (MSGs): PY 2016-17: 35.48%; PY 2017-18: 39.89%; PY 2018-19: 44.88%. During the pandemic the state’s MSG rates dropped significantly: PY 2019-20: 33.9% and PY 2020-21: 36.8%. North Carolina’s MSG target for PY 2021-22 was 43.6% and the state reached a final MSG of 43% which exceeds PY 2021-22 and builds confidence that the state will exceed the current PY 2022-23 MSG target of 43.6% by June 30, 2023. ABE/ASE Our aggregate Adult Basic Education (ABE) and Adult Secondary Education (ASE) levels overall fell below the 43.6% target for PY 2021-22 with a performance of 38% and 42% respectively. All ABE levels except for ABE Level 6 (50.3%) fell below their targets of 43.6%. Most MSGs were achieved by pre and post-tests, followed by high school equivalency diplomas and post-secondary enrollment. ESL Our aggregate English as a Second Language (ESL) levels well exceeded the state’s target of 43.6% for PY 2021-22 with a performance of 49.5%. ESL levels 1 through 5 surpassed the 43.6% target rate (51.7%, 57.9%, 55%, 48.2%, 47.8% respectively). Our only shortfall occurred at ESL level 6 with a performance rate of 38.8%. Most MSGs were achieved by pre and post-test EFL gains, followed by enrollment into post-secondary education after exit. Combined Program Performance Factoring in periods of participation (POP) for PY 2021-22, our overall MSGs were 43% which was a 5% point increase over the prior program year’s 38% performance but fell just shy of our 43.6% target. The state’s participant population for ABE/ASE was approximately 61% and ESL accounted for 39% of our participants. The number of participants in all ABE/ASE levels increased from the prior program year by 1,716 participants. Across all ESL levels, participants increased by 4,717 over PY 2020-21. Over the past several program years, North Carolina has worked to customize and enhance our student information system database to include views of all NRS tables and to make data more easily accessible for programs. Part of this programming included the creation of additional tools and reports to aid providers in proactively monitoring and understanding their data and performance. The state provides monthly feedback, routine training sessions and open office hours throughout the year with a focus on data quality and performance improvement. The state has developed Power BI data dashboards, which allow providers to pull data to create specific data reports for programs to analyze to help identify potential problems and or to resolve problem. The Power BI data dashboards were developed to extend the functionality of the North Carolina Community College System’s (NCCCS) Performance Dashboards presented through Tableau. Within the Power BI dashboards providers can use the detailed student-level views of data that are submitted monthly to the NCCCS Data Warehouse to monitor their programs performance and outcome measures. Moreover, the State Office utilizes their statewide view of the Power BI dashboards to monitor (monthly) state performance, as well as monitoring individual providers for potential technical assistance needs throughout the program year. We are currently in the implementation phase for the new Adult Education and Literacy state MIS, ADVANSYS by Benchmark. For PY 2021-22 the state’s 58 community colleges were in the beginning stages of transitioning to the new MIS system. (The state’s 10 community-based organization, Title II providers successfully transitioned to ADVANSYS in December of 2020. The new MIS system will incorporate NRS measures using common definitions and categories. NRS tables will calculate accurately to include error checks and prevent double counting with the new state MIS. For PY 2021-22, the State Office provided biweekly ADVANSYS sprints through the Agile method of project development and management. Weekly cohort meetings for colleges first identified to convert to ADVANSYS were held for the second half of the program year and will continue through PY 2022-23 as more cohorts are onboarded. Moreover, the State Office ADVANSYS team, comprised of state staff representatives from Adult Education & Literacy, Business Systems Analysts, Research and Performance Management and Information Technology meet daily with the vendor to ensure accuracy in reporting and data management as the product is customized and finalized to meet the needs to North Carolina. During PY 2021-22, to address areas of weak performance, state staff members conducted webinars and training sessions focused on data, performance, and instruction. Specific technical assistance was offered to individual programs who were not meeting data reporting deadlines and or performance outcomes. The State Office also continued the services of the Center of Excellence for Adult Education (CEAEP) to provide professional development and technical assistance projects to providers on improving their performance. Some of the professional development and technical assistance provided focused on performance and accountability, post-exit indicators, conquering their data challenges, and tools and strategies to increase their overall MSG rates. During the 2021-22 program year, state staff provided monthly virtual statewide meetings with providers to provide important guidance/policy updates and resources for programs regarding the development of data performance dashboards and Power BI data views. |
| Northern Mariana Islands | Despite enrollment decreasing yet again this program year 2021-2022, we were still able to Improve the percentage of Periods of Participation (PoP) with Measurable Skills Gains (MSG) from 52.73% in PY 2020-2021 to 61.18% this year. We worked hard in preparing those who were getting ready to achieve their goal to graduate. Our office graduated 30 students. Face-to-face courses were converted to online instruction. It was a great challenge for all of us who were not used to this delivery mode. We opened up all lines of communication with our students. They were able to contact us through personal phone numbers, emails, social media messages, and WhatsApp. They could contact most of us anytime during the week, including evenings and weekends. Some students who were not able to do online courses were treated as independent study students and did assignments or projects through emails or through packets. We tried to accommodate students in different ways. We weren’t able to increase our enrollment, but we also didn’t lose too many of our students. Students were able to continue to come on campus to take their assessments. Individual student profile reports were given to instructors to incorporate into their online lesson plans to ensure weak skills, as well as strengths, were addressed and reinforced. Northern Marianas College’s General Education Learning Outcomes (GELOs), CASAS competencies, Adult Education CCRS, and Student Learning Outcomes (SLO) were all mapped and laid out in the program course syllabi. Students entered the online class knowing what they were going to learn as well as the expectations from the instructor. Our intake process continues to be improved to meet student needs. Educational advising sessions were scheduled often to track progress towards goals and individual concerns were also addressed during these sessions. We’ve found that this particular practice has had a positive effect and student retention improved overall with them posting enough hours to posttest and continuing until they fulfill their goals. We also created a separate email for potential students to send inquiries. This helped us track the different types of requests that came in. The office continues to communicate with the institution and a research associate from REL (Regional Educational Laboratory) for the pacific region at McREL International (a non-profit, non-partisan education research and development organization) to assist in the development of an IET program that will work and be sustainable as well as meet the needs for our community. We plan to continue working on developing an IET program and hopefully launch it in the near future. |
| Ohio | Highlights from the Achievements of the Core Indicators of Performance
|
| Oklahoma | Program Year 2021 - 2022
|
| Oregon |
Educational Functioning Level
2020-2021 Actual Performance
2021-2022 Target
2021-2022 Actual Performance
Change from 2020-2021 Actual Performance
ABE Level 1
10%
51.5%
24.8%
+14.8%
ABE Level 2
30%
54.5%
41.1%
+11.1%
ABE Level 3
20%
48.5%
39.0%
+19.0%
ABE Level 4
19%
44.0%
32.0%
+13.0%
ABE Level 5
22%
55.0%
38.7%
+16.7%
ASE Level 6
33%
45.0%
48.5%*
+15.5%
ABE/ASE Total
22%
36.7%
+14.7%
ESL Level 1
2%
47.5%
13.6%
+11.6%
ESL Level 2
3%
54.5%
17.6%
+14.6%
ESL Level 3
5%
53.5%
27.3%
+22.3%
ESL Level 4
7%
49.5%
26.0%
+19.0%
ESL Level 5
3%
43.5%
24.9%
+21.9%
ESL Level 6
3%
23.5%
13.6%
+10.6%
ESL Total
4%
24.5%
+20.5%
*Met or exceeded federal target
Overall, performance in ABE/ASE measurable skill gains increased by 14.7% and performance in all six ABE levels was significantly greater in 2021-2022 than in 2020-21. Additionally, Oregon exceeded the MSG federal target in ABE Level 6 by 3.5%. The next highest performing EFL was ABE Level 2 (federal target missed by 13.4%) and analysis indicates that only 55 additional students achieving a measurable skill gain in ABE Level 2 were needed to meet the federal target. Similar analysis for ABE Level 3 (federal target missed by 9.5%) indicates that 93 additional students achieving MSG were needed to meet federal targets.
Performance in ESL measurable skill gains increased overall by 20.5% in 2021-22. While federal targets were not met, performance in all six ESL levels was significantly greater in 2021-22 than in 2020-21, with the greatest increase occurring in ESL Level 3 with an increase of 22.3% year over year. Analysis indicates that while the federal target was missed by 26.2%, an additional 196 participants achieving a measurable skill gain were needed to meet the target. Similar analysis for ESL Level 6 (federal target missed by 9.9%) indicates that 96 additional participants achieving MSG were needed to meet federal targets.
In response to local program director feedback (as highlighted in the 2020-21 Federal Narrative Report), Oregon contracted with a local program to establish a statewide remote proctoring service. Available to any local ABS program in Oregon, the statewide remote proctoring service addressed barriers that local programs had with pre- and post-testing, including not having the staffing capacity to conduct widespread remote testing. As a result, pre- and post-testing pairs and assessment rates have increased. Due to the success of the initial year of statewide remote proctoring, the contract was renewed and online proctoring continues for PY2022-23. The contracted program anticipates doing additional analysis of this practice, to determine whether continuation would be in the best interest of the programs and participants.
During the 2021-22 program year, the State ABS Team intentionally emphasized technical support with more frequent, consistent communication with directors and interaction with programs. The State ABS Team focused on how best to support local programs as the pandemic and its effects continued. Some activities included but were not limited to:
|
| Puerto Rico | The AEP performed an analysis of the performance data reported this year. To perform that analysis the process conducted at the State and local levels was based on the data collection done by the service providers and uploaded to the Adult Information System (AIS) at AEP central level. A detailed data review and validation for each data indicator was performed by the central office. The specific anomalies identified were checked for corrections. The errors, when identified were resolved by telephone calls and discussion of the raw data indicators with the education centers directors, teachers and counselors at each center.
The AEP completed 10 visits to education centers to provide technical assistance on compliance with data collection and data quality management for NRS purposes.
The outcome reached
The State performed adequately regarding its negotiated levels of performance and outcomes for each indicator for PY2021-2022. The targets and outcomes are the following: (Table 4 NRS).
Performance indicator
PY 2021-2022 negotiated level (%)
PY 2021-2022 outcome level (%)
Percent achieving Measurable skills gains Average ABE and ESL
54.5
57.09%
COVID pandemic and post-hurricanes federal incentives impact during PY 2021-2022
The COVID pandemic effects on participants recruiting and attendance have been fading out, but during 2021-2022 program year the difficulties in participants’ recruitment continued as result of the federal economic stimulus program providing cash incentives to individuals that resulted in a lack of incentive for work in the whole economic system. In addition, Puerto Rico continues the recovery from the sequence of hurricanes that severely impacted the economic and social collective life during the PY 2021-2022. This situation provoked a challenging situation for enrollment of participants during the PY.
Adult Education Participants’ profile for PY-2021-2022
The Adult Education Program served, during 2021-2022 school year, an active participants enrollment of 3,946. The previous year 2020-2021 the enrollment was higher with 5,801 participants. In conclusion, the PY 2021-2022 has experience a reduction of 1,855 participants served in comparison with the previous year. The decrease is explained by the two-fold factors explained in previous section: the impact of hurricanes and lack of interest in workforce training due the economic cash incentives provided to low income families by the federal government.
The distribution by gender in participants served was as follows: 1,565 males, equivalent to 39.7% of total and 2,381 females, equivalent to 60.3%. The males’ relative participation decreased by 6.5% in relation to previous program year when it was 46.2% of total, while female participants increased by an equivalent percentage.
Participants by services was as follows:
|
| South Carolina | Assessment of Core Programs
The OAE formally and informally assesses each local program throughout the year via Desktop Monitoring, subgrant Interim and Final Reports, Compliance Monitoring, and a variety of informal data reviews via the state’s data management system, LACES.
The Desktop Monitoring Tool (DMT) is a quarterly report that collects data from NRS Tables, as well as secondary credentials and Career Readiness Certificates (CRC). Each local program completes and submits a quarterly DMT. At the end of each year, a program overview report is completed at the state level using the DMT data. This report provides a snapshot of each local program and assists the state with developing goals for the next year by identifying promising practices and professional development and technical assistance needs for the coming year.
All programs receiving federal subgrant(s)—Adult Education, Family Literacy, Corrections Education, and Integrated English Literacy and Civics Education (IEL/CE) grants—completed an Interim Report and a year-end Final Annual Report, designed specifically for each grant. Data from these reports guide specific technical assistance delivery for programs receiving this grant.
In addition to the DMT and subgrant reports, the OAE conducted data reviews in order to identify areas of strength and weakness and to measure progress toward meeting the 2021-2022 OAE Goals.
Outcomes of the 2021-2022 Strategies for Increasing Performance
Based on the assessments of core programs, the SCDE OAE sets goals to determine the focus for each program year in order to meet the performance targets for measurable skill gains and core follow-up outcome indicators.
The 2021-2022 OAE Goals were to:
|
| South Dakota | In a post-vaccine milieu, it yet seems most reasonable to contrast last year with PY2020. While South Dakota’s WIOA Title II program served 1,365 participants in PY2020, PY2021 saw 1,635 participants [for a noteworthy 19.8% increase in participation]. The Adult Education Program evidenced 34.49% Measurable Skill Gains in PY2020, while PY2021 finished at 36.44% MSG. Similarly, South Dakota’s number of reported PY2021 GED credentials trended upward, albeit at a more modest rate. Performance Outcomes (PY2017 – PY2021) Program Year ABE MSG ESL MSG Total MSG Q2 Employment Q4 Employment Median Earnings Credential Rate 2017 41.38% 37.99% 40.25% 51.31% 46.17% $5,721.14 21.70% 2018 45.56% 30.72% 40.80% 58.70% 56.20% $5,575.85 40.52% 2019 25.82% 24.95% 25.47% 55.85% 58.80% $6,151.15 55.20% 2020 34.37% 34.89% 34.49% 50.20% 49.95% $7,220.47 43.42% 2021 37.93% 31.98% 36.44% 56.22% 50.20% $6,405.47 30.50% WIOA Title II’s High School Equivalency Attainment Program Year Number of GED Credentials Conferred 2017 281 2018 365 2019 126 2020 216 2021 259 Due to South Dakota’s pandemic-related recovery, unemployment rates are some of the lowest in the nation. And like everywhere, businesses and employers are clamoring for entry-level, mid-level, and skilled workers. Wages are seeming to increase as industries vie for qualified candidates. And still, these workforce-related factors and our clients’ personal exigencies affect WIOA Title II’s abilities to engage students long enough to post-test, confer GED Credentials, and make significant investments in English language instruction. South Dakota’s national rankings in the Employment Outcomes (i.e., Q2 Rate, Q4 Rate, & Median Earnings) highlight well the socio-economic realities [as best as such data-collection protocol can afford]. If someone needs or wants a job in South Dakota, the opportunities most certainly exist. The PREP initiative [noted throughout South Dakota’s PY2021 Narrative Report] is meant to support our co-enrolled learners so they can address some of the pressing needs of immediacy (e.g., food security, housing, transportation, childcare, etc.) while simultaneously concentrating on their studies in a scaffolded, tandem case-management model. The agency continues to explore creative ways to not only support our co-enrollments to the fullest extent of the statute, but also continues to collect data to validate efforts, identify promising practices, highlight opportunities for technical assistance, and improve partnerships [for the benefit of our participants and our WIOA Joint PIRL measures]. Therefore, beyond the marked improvement in participation-figures over the prior Program Year, the South Dakota AEFLA Program is also rather proud of more than doubling its reported Percent Enrolled in More Than One Core Program on the PY2021 Statewide Performance Report. And even though this figure is still likely a bit underreported, we are making demonstrable progress in effectively tracking these common participants. Perhaps most importantly, we are strengthening the communication, coordination, and collaboration that underpin these efforts across the One-Stop System! |
| Tennessee | In PY21, TDLWD’s systems and analytics specialist focused on three main goals: (1) oversight of the management information system (“Jobs4TN”), including troubleshooting issues with the system and monitoring for anomalies and errors from local data entry; (2) maintaining performance dashboards to help AE stakeholders quickly and effectively assess local program performance; and (3) establishing and measuring KPIs that directly impact student performance—the idea of “measuring what matters”. We plan to refine these dashboards and provide additional training on how to use them effectively in the next year.
TDLWD staff also provided continuous technical assistance on the data entry and evaluation process to ensure that local staff were capable of doing accurate data entry. We also included robust data assessment as part of our monitoring process. This included thorough investigations into specific local providers’ data, and when errors were found, we brought this to the locals’ attention and included the requirement for them to improve data practices as part of their corrective action plan.
The Tennessee adult education program achieved the following performance outcomes in PY21:
|
| Texas | Through TWC’s state-developed data system, we are able to see data in real-time and utilize information to target technical assistance or professional development needs of the state provider system. The TWC AEL web-based data system, Texas Educating Adults Management System (TEAMS), allows easy access for end users to input data daily with a state policy that dictates a monthly data entry requirement. Monthly data validation procedures are built into the data system providing state staff with real-time reports if local data entry is delayed. Once data is validated, it is locked for further input by local users. This allows the TWC’s AEL team to pull data reports each month to review the number of participants being reported, Measurable Skill Gains (MSGs), and posttest eligible students among other performance-related reports and quality checks. State staff analyze the reports to identify:
• significant increases or decreases in enrollment since last month;
• participants who are eligible for posttest who do not have another MSG in the system;
• time between tests in hours and days;
• IET students who have not earned an MSG or a credential; and
• concerns with retention for students who stopped attending and are in jeopardy of exiting the program early.
System reports are more useful than NRS tables for programmatic discussions with providers. System-generated reports provide disaggregated data down to the instructor and participant level. TEAMS also has built-in controls to cross check for anomalies with automated data checks in place that support accurate data reporting, such as but not limited to:
|
| Virgin Islands | For the reporting period of July 1, 2020 through June 30, 2021 in the USVI Adult Education programs showed significant changes from preceding years as a result of impact from COVID-19. Although learning continued to be online because of the pandemic, a hybrid model was implemented to accommodate adult learners who were not adept to technology, struggled with following simple directions and became easily frustrated with learning basic commands. The pandemic has emphasized the need for diverse and structured ways for Adult Education programs to provide learning opportunities. However, there exists genuine concern that students may become even further marginalized because they do not have a smartphone, tablet, laptop, internet access, or the necessary digital literacy to function properly in a remote learning environment. A pivotal example of Measurable Skills Gain (MSG) over the most recent two Program Years highlights the difficulty VIDE’s AE program had in meeting program mandates due to COVID-19’s adverse effect on areas of performance (e.g. Data, enrollment, MSG, educational functioning levels (EFL) gain, and employment gains). Examination of measurable skill gains comparisons of program data reported for 2020-2021 in comparison with 2021-2022 showed a decline from 22 percent to 25 percent. This was also due to the instability of many power outages territory wide as the territory re-built the electrical infrastructures post the natural disasters. This also was a significant barrier that served as a contributing factor that affected the online delivery services to our adult learners. Overall, as a territory, there was a 20 percent decline in NRS participants. ESL enrollment increased by almost 22 percent and ABE enrollment increased by nearly 19 percent. There was an overall increase in students age 19-24 (22.5 percent) and students age 25-44 (23 percent). Recruitment and retention will be essential components of professional development moving forward. Additionally, concentrated monthly monitoring of the MIS per program by the State to systematically track local program’s data entry and provide one on one technical assistance for programs that have been deficient will be an area of targeted focus. |
| Washington | Measurable Skill Gains
Washington State achieved a 27.91% (28% when rounded) overall Measurable Skill Gain (MSG) rate for the program year 2021-22, significantly lower than its MSG target of 43.8% (rounded to 44%). This is due to statewide governor-ordered suspended operations caused by the COVID-19 pandemic and our providers’ inability to CASAS test. According to Table 4B, only 3,967 participants were able to post-test in PY2021-22 out of a possible 26,251 participants in all periods of participation. This is a minuscule amount compared to the number that have been able to post-test before the pandemic. It is nearly four times the amount as last year in which our state was also ordered to suspend operations. Gains achieved using the HS Credit Option, IET MSGs, and the EFL gain for a program exit and transition to postsecondary EFL during the program year, contributed to the higher levels of gains showing in ABE 4 (39%), ABE 5 (40%), and ABE 6 (65%) [see Measurable Skill Gains: All Colleges/Organizations chart below].Post-exit Outcome Measures Washington State exceeded all targets for post-exit outcome measures for PY2021-22, but it was noted in our federal program review that our Credential Rate was not calculated correctly. Table 5 shows that although performance levels were similar to last year, the cohorts are lower than the previous year by 9-38%. Our employment results are likely skewed since Washington relies solely on social security numbers (SSN) matching with the SWIS unemployment insurance dataset. English language learners made up about 44% of our program and reported SSNs at a much lower rate (38.8%) than students in our ABE programs (78.7%) according to internal research. It’s impossible to know the true rate of SSN collection among the ELL population since our state’s college data system collects SSNs and ITINs (tax ID numbers) in the same field. The strongest component of the overall credential rate is still the Attained a Postsecondary Credential while enrolled or within one year of exit measure. Strong performance is supported by our state’s special program Integrated Basic Education and Skills Training Program (I-BEST). This program allows students to work on college-level studies right away, while receiving basic skills instruction, and leads to certificates and degrees. Many of these programs were suspended during the pandemic leading to a drop in MSG as well as our Credential Rate overall. |
| Wisconsin | The Wisconsin AEFLA program is a national leader with strong educational and employment outcomes across the core indicators of performance. As demonstrated in Table 1, the Wisconsin Adult Education program exceeded the nation across the federal WIOA primary indicators of performance. Table 1. WIOA Primary Indicators of Performance Outcome WI (PY 2021-22) Nation (PY 2020-21)[1] Difference Measurable Skill Gain Rate 53.5% 34.7% +18.8% 2nd Quarter Employment after Exit 53.0% 29.3% +23.7% 4th Quarter Employment after Exit 43.7% 28.6% +15.1% Median Earnings 2nd Quarter after Exit $ 6,111 $ 4,899 +$1,212 Credential Attainment Rate 62.1% 25.9% +36.2% Strengthening Adult Education Recruitment Chart 1. Wisconsin AEFLA Participation: Participants & Reportable Individuals (PY 2017-18 to 2021-22) While the Wisconsin Adult Education program has reported strong outcomes, actual participation has plateaued with the exception of PY 2020-21 due to the COVID-19 pandemic (reference Chart 1). Stagnant participation in the Adult Education program is significant given over 354,000 Wisconsin adults are without a high school credential and over 142,000 Wisconsin adults do not speak English very well or at all (US Census American Community Survey, 2021). These data points mirror national research suggesting that only 10% of adults who could benefit from adult education skills support actually participate in the Adult Education system (Patterson, 2018). In response to last year’s Narrative Report and provided feedback during monitoring activities, the WTCS Office coordinated a series of professional development opportunities to strengthen student recruitment efforts. During the October 2021 Common Ground Conference, the WTCS Office delivered a session titled Strategies for Adult Education Recruitment. During the session, participants were provided a step-by-step overview of how to retrieve US Census Data and use the data to inform targeted recruitment. In addition, empirical evidence from existing Adult Education recruitment research (e.g., Patterson & Song, 2018; Smith, 2019; Smith, 2020) was reviewed to build awareness of potential student recruitment strategies. The WTCS Office plans to continue delivering professional development during the 2022-23 program year focused on equity in recruitment practices and how these efforts, coupled with strong educational and employment outcomes, can enhance the economic mobility of Title II learners. The WTCS Adult Education team participated in the NRS 2022 Regional Workshop titled The Journey to Increased Enrollment: Recruitment and Service Strategies for Adult Education. The WTCS Office staff who participated in the training plan to contextualize the content for Wisconsin and scaffold the content over three professional development opportunities during the 2022-23 program year. Fully Realizing the Employment Outcomes of Title II Participants During the 2021-22 program year, the WTCS Office coordinated a technical assistance session to build the fields awareness of the three-employment based, WIOA primary indicators of performance. The indicators included 2nd quarter employment after exit, median earnings 2nd quarter after exit, and 4th quarter employment after exit. Each provider reviewed trend data across the three indicators for their program and engaged in reflection related to what might be contributing to their program’s outcomes. In addition, the WTCS Office shared an overview of each indicator’s calculation methods. Feedback from Wisconsin’s Title II providers revealed three important session findings. First, participants engaged in Integrated Education and Training had stronger employment outcomes compared with their peers not engaged in Integrated Education and Training. This finding is logical and supportive of the occupational alignment of WTCS Integrated Education and Training offerings. It also adds fuel to Wisconsin Title II’s existing Integrated Education and Training expansion efforts. Second, providers believed there was an opportunity to better partner with WIOA Title I to strengthen pathways to employment. This finding triangulates with small levels of Title II co-enrollment across WIOA Title partners; only 4 percent during PY 2021-22. Third, providers shared challenges with data quality that may be impacting employment indicator results. These included challenges in collecting Social Security Number for employment data matching purposes, which are especially challenging among ELL learners, and the limited nature of Wisconsin leveraging the state’s wage-record system for employment data. In response to employment indicator data quality feedback, the Wisconsin Title II program is planning a series of activities for PY 2022-23. The WTCS Office is planning to explore additional ways to conduct data matches that extend beyond solely matching on Social Security Number. This might include matching on name and date of birth. In addition, the WTCS Office plans to coordinate supplementary technical assistance during PY 2022-23 on strategies for collecting data at intake, including Social Security Number. Finally, the WTCS Office is exploring how it can best leverage the SWIS to complement state employment data with surrounding state data. Together, these data quality activities will help Wisconsin Title II fully realize the employment outcomes among its participants. [1] National PY 2021-22 data unavailable at the time of writing Wisconsin’s PY 2021-22 Narrative Report |
| Wyoming | Process:
The State takes a multi-tiered approach to analyzing and monitoring data. Because data is the key to success, the State monitors and reviews data through multiple sources. This year the State implemented a ‘Roadmap to Data Success’ system which highlighted six key aspects to data analysis:
|