Sorry, you need to enable JavaScript to visit this website.
Skip to main content
U.S. flag

An official website of the United States government

Narrative Analysis Tool


The Narrative Report responses below can be further filtered by one or more states, as well as keywords.

For more information on Narrative Reports please see the technical assistance documents.

Print Download Word document

    Narrative Selection Switch - (Click box below for list)
State Describe how the adult education program performed in the overall assessment of core programs based on the core indicators of performance. Discuss how the assessment was used to improve quality and effectiveness of the funded eligible providers and any plans to further increase performance in future reporting years.
Alabama Alabama’s adult education follows a continuous improvement approach with data analysis as the foundational base. Regional Directors conduct routine performance meetings with each of their assigned programs. ACCS directors use this opportunity to highlight and discuss best practices relevant to program in each of their areas. The dashboard on AAESAP provides real-time information for enrollment, progress post-test rate, MSG, credential attainment, and career pathway completions. The ALAPCE conference provides multiple breakout sessions that cover such topics as rules surrounding MSGs and the reports to help monitor performance.  Additional topics include how to understand and drill down into performance information on the dashboard and using reports for monitoring and program improvement. This conference is open to all grantees with instructors, program directors, and data entry specialist attending various sessions. Program year 2021-22 saw a 20% increase in the number of participants served over the previous PY 2020-21. There was also a 24% increase in the number of contact hours in PY 21-22. These indicators show signs of recovery to pre-pandemic enrollment. Additional recruitment plans are in place to increase enrollment and services in PY 2022-23 by increasing our student computer equipment loan program and increasing distance learning opportunities with remote assessment and access to a library of career centric curriculum. Of the 14,351 students enrolled, 12% were English Language Learners (ELLS).  The ELLs attended an average of 62 hours. The Adult Basic and Adult Secondary (ABE and ASE) students attended an average of 57 hours.  Retention for all students is an area that requires constant attention by the providers. Defining and assisting with barriers that inhibit attendance by the participant is an area that is addressed by the local programs and state staff through professional development and targeted technical assistance. Enrollment increases in serving incarcerated and institutionalized participants stemmed largely from corrections facilities once again allowing instructors to reconvene classes in PY 2021-22. Local providers can analyze information on the real time information dashboard of our management information system, AAESAP.  The demographics provided in the NRS tables can be used to design curriculum and customize classes for age ranges and ethnicity. Dashboard information allows providers to self-analyze their program data to identify potential problems and proactively work to resolve the issues. The system provides a multitude of reports for targeting various aspects of programmatic performance. The educational functioning level completion percentage for PY 2021-2022 was greatest in the ABE Beginning Literacy and ABE Beginning Basic levels. For ESL students the strongest areas of level completion were Low and High Beginning ESL. Local program providers are using technology to create synchronous and asynchronous instruction for all levels of learners and employing interactive hybrid approaches. These flexible instructional methods are assisting other educational functioning levels of learning to advance and achieve a skill gain.  The AAESAP system also allows students to self-enroll online for Adult Education services.  Classes are offered on-site, virtually, or in a hybrid format.  An auto alert notification allows a local program to know when a public inquiry or enrollment has been created.  Alabama Adult Education Core Performance Measures for PY 2021-2022: Employment 2nd QTR after exit 49.30% Employment 4th QTR after exit 42.77% Attained HSE and enrolled in postsecondary within one year of exit   8.68% Attained HSE and employed within one year of exit   33.17% Attained a postsecondary credential while enrolled or within one year of exit   39.05% Attained any credential 39.32% Measurable Skill Gain 50.15%   Alabama’s MSG percentage continued to see improvement with the extension of tracking gains specific to Workplace Literacy and Integrated Education and Training programs. Increased enrollment in postsecondary education has been tracked and data reflects…
Alaska Participant Eligibility Requirements Although still recovering from challenges associated with the pandemic, the data for the period of July 1, 2021 – June 30, 2022 has started to show increased numbers in almost all areas of performance, including Overall Enrollment, Educational Functioning Levels (EFL) gain, and Employment Gains. The AAE Measurable Skill Gains (MSG) decreased this program year. Alaska is unique in geographic, cultural, and economic barriers which can hinder access to training, education, and employment. Each regional program serves vast areas with barriers unique to Alaska. Transportation, weather, and the digital divide are among the most prominent barriers to adult education. Some examples of Alaska’s regional barriers include local programs:
  • Serving vast numbers of communities with regions the size of most states in the Continental US.
  • Without a road and must be accessed by boat, ferry, bush plane, snowmobile, dog sled, or distance delivery. 
  • Serving communities with little or no internet access.
Alaska Adult Education contends with challenges that have a profound impact on both urban and rural students’ ability to access education, as well as retention rates. Some challenges include:
  • The digital divide is of great concern to AAE programs. Reaching students through digital means, including the internet, continues to be the largest barrier to instituting distance education and working with rural students. The most prominent barriers continue to be slow internet speed; expensive, unreliable, and unaffordable 3G networks; lack of internet access and technology; and students struggling to pivot to remote learning.
  • GED testing for students in remote villages involved taking a flight to the nearest local program, arranging room, board and study time, and taking the exam. Remote testing provided some options for village students with adequate internet access.
  • USPS mail service to remote villages which relies on private air carriers and contractors continues to be a slow process. This can be frustrating to students who must wait for materials. Programs also struggled to mail information out to students and/or get homework back.
Local and state program successes during PY 2021 include:
  • Local programs continue to incorporate online teaching which increased student access education where internet access allowed.  This included increased attendance via Zoom by students who had been unable to commute or were limited by care giving needs in the past.
  • NRS level classes provided teachers a means for scaffolded learning including one-to-one tutoring, visual icons, and oral and written information using online methods.
  • Local programs continued to use social media to inform students of closures, hours, graduations, success stories, and other important program information.
Overall enrollment numbers increased in PY 2021. Alaska reported 933 full time students, a 12 percent increase from the previous year’s totals. As indicated on Table 4: Measurable Skill Gains (MSG) by Entry Level, showed 14.7 percent of total number of period of participations (POP), which was a decrease from PY 2020.  The MSG for English language learners (ELLs) decreased, while the adult basic education (ABE) levels remained steady. The ABE MSG rate was 20.28 percent, and the ELL MSG was 6.80 percent. There was a significant drop in educational functional level (EFL) gains due to post testing. The AAE Office has addressed these concerns with programs and through professional development. PY 2021 Core Follow-up Outcome Achievement measure remained high for employment outcomes and decreased for both Credential Attainment and MSG, as noted above. The AAE State Director continued to collaborate with partner agencies to maintain policies and procedures for collecting common data elements for future reporting. Data in Table 1.1 below shows Alaska is meeting three of the five performance indicators. Further analysis from NRS Table 5, demonstrates that Alaska is on track with employment goals to meet the negotiated levels for PY 2022. Table 1. 1 PY 2021 Indicator of Performance   PY 2021 Negotiated Levels PY 2021 Indicators of Performance + or - Achievement Employment 2nd Quarter After Exit 30.00% 45.08% +15.08% Employment 4th Quarter After Exit 30.00% 40.22% +10.22% Median Earnings 2nd Quarter After Exit $4,200.00 $4,757.72 +$557.72 Credential Attainment 30.00% 21.88% -8.12% MSG 30.00% 14.70% -15.30% Table 1.2 below shows full time student counts over the last three years which are used with programs to prompt discussions about recruitment needs, training delivery methods, and retention. Table 1. 2 Full Time Student Data State Fiscal Year: Full-Time Students PY 19 (SFY 20) 1,080 PY 20 (SFY 21) 823 PY 21 (SFY 22) 933 Most AAE students enter programs due to a desire to complete high school or improve their English skills. Based on trend data from Table 1.2 above, the need for recruitment and retention are essential components of professional development and brainstorming efforts. Programs continue to use creative solutions, such as social media campaigns, for outreach and collaboration with partner agencies. As indicated in Table 1.3 below, enrollment in ABE skills classes dropped slightly, with a 3 percent drop in attendance. The secondary education student numbers remained consistent. Alaska saw a 33 percent increase in ELL students from the previous program year. This is consistent with the overall increase in refugees. Table 1. 3 Adult Education and English Language Learner Trends State Fiscal Year Adult Ed Secondary Ed ELL PY 19 (SFY 20) 605 23 452 PY 20 (SFY 21) 535 26 262 PY 21 (SFY 22) 519 27 387 PY 2021 was the first year of a four-year grant competition for service delivery. There were two new regional grants, but no new eligible providers. The PY 2021 grant competition was release in March 2021 and new grants were awarded in July 2021. Due to programs not scoring high enough on the grant matrix, 3 grants were awarded with a late start (October 2021) and one region is now observed by a well-established program.
American Samoa EVALUATION OF THE AMERICAN SAMOA ADULT BASIC EDUCATION PROGRAM PER INDICATOR Indicator #1: Obtained a GED or Secondary School Diploma The Adult Education Literacy and Extended Learning Program (AELEL) have continuously offered educationally disadvantaged adults the opportunity to earn their high school credentials. In the Territory, as it is in some of the U.S. states, the GED diploma represents a second chance at secondary education and opening of new opportunities. Those in the general public who have not had the opportunity to complete their high school education are the primary targets of the program. Preparatory courses for the GED and HiSet exams are part of the AELEL course offered at the college every semester. In SY 2021-2022, 4% (4) achieved their goal of obtaining a HiSet diploma, this percentage was the result of student testing through the test approaches in the set of three. The Pandemic Covid-19 hardship has contributed to the decrease, for many students opted to join the workforce due to online processes that made things difficult for learning. Indicator #2: Enter Employment Many students who enter our Adult Basic Education and English Second Language programs indicated entering employment as one of their goals. It is also a fact that most of them if not all do not have a high school diploma to land a decent job. Having this information on hand encourages the program to advise the students to seek their diploma through the HiSET exam before making a job choice. Although this is our preference, students have shown to leave the program to take up a job when opportunity is available. This is an ongoing issue from the previous reporting year, as our local economic forecasting doesn’t look promising with one of the two canneries closed and the other is uncertain with its operation. The employment percentage for this year is 65%. Many of these individuals will return to complete their studies and obtain a high school equivalency when they are faced with a job promotion that requires a diploma or decided to go into the arm forces. Upon the opening up of employment since the Covid-19 limitations, more students have sought jobs in government and private agencies. Indicator #3: Retained Employment Majority of AELEL students are currently employed and we are continuously working with the Department of Human Social Service literacy program for newly hired and previous participants. We are currently moving forward with prospective literacy programs in partnership with the American Samoa Government to assist with new employee certification, current employee recertification, and workplace literacy. AELEL has experienced a slight decrease in this indicator from this year due to the Pandemic. Indicator #4: Enter Postsecondary Education or Training The only institution of higher learning in the Territory is the American Samoa Community College. This is also the major service provider for AELEL. We are able to work closely with the Office of the Registrar in transitioning our students to college. A few of our students left the island to further their education and/or training but when they do, it is difficult to track them. Due to the effects of the Covid-19 Pandemic, many of our students are seeking jobs instead of continuing their education. These jobs are mostly low wage jobs and are not reported in any data available. This reporting year we are seeing an increase in this indicator due to hard push by the community for students to stay in school and go further than a high school diploma/equivalency. In SY 2018-19 we had 86% and now in SY 2020-21 our performance dropped to 45%.
Arkansas During the 2021-22 program year, 60% of adult education students in Arkansas achieved an MSG on Table 4, exceeding the negotiated indicator of 49%. Arkansas met or exceeded all 12-level benchmarks. CORE INDICATORS OF PERFORMANCE 2020-2021 Performance 2021-2022 Targets 2021-2022 Performance Difference of Targets to Performance ABE Level 1 53% 45% 63% +8 ABE Level 2 47% 44% 51% +7 ABE Level 3 50% 48% 57% +9 ABE Level 4 69% 56% 72% +16 ABE Level 5 73% 55% 77% +22 ABE Level 6 60% 53% 74% +21 ESL Level 1 55% 47% 52% +5 ESL Level 2 62% 53% 62% +9 ESL Level 3 56% 54% 68% +14 ESL Level 4 63% 53% 60% +7 ESL Level 5 59% 46% 61% +15 ESL Level 6 53% 31% 48% +17 AVERAGE 56% 49% 60% +11   Arkansas Adult Education also reported the following data for its follow-up measure indicators: CORE INDICATORS OF PERFORMANCE 2021-2022 Targets 2021-2022 Performance Difference of Targets to Performance Employment Second Quarter After Exit 45% 46% +1 Employment Fourth Quarter After Exit 47% 42% -5 Median Earnings Employment Second Quarter After Exit $4,200.00 $4,818.54 +$618.54 Credential Rate 49% 43% -6   For the program year 2021-2022, Arkansas Adult Education exceeded the Median Earnings Employment Second Quarter After Exit indicator while falling short in Fourth Quarter Employment Measure and the Credential Attainment rates. To complete the data collection for the Follow-up Measures, Arkansas data-matched with the Department of Higher Education and the Division of Workforce Services.  Programs submitted manual outcome forms to the state office for validation for participants or follow-up measures that could not be data matched. Once a program year has ended, ADWS/AES generates an Effective & Efficient (E&E) report based on the benchmarks of core performance indicators for each local provider. For the 2021-22 fiscal E&E report, local providers must meet the state’s MSG benchmark of 49%.  This year 34/37 met or exceeded the benchmark. ADWS/AES uses Quarterly Reports to help assess the effectiveness and ability to meet the targets and overall performance of the programs throughout the year. Data is reported on wide-ranging aspects for both qualitative and quantitative analysis.  Programs that do not meet the determined percentage of the benchmarks are provided additional technical assistance to prepare a performance improvement plan. Plan implementation is monitored throughout the year. Meetings occur bi-weekly with technical assistance from a team of advisors where goal setting and crucial data analysis elements are identified and planned. Small, achievable goals for positively impacting performance are identified and set with a plan to achieve them bi-weekly. Technical assistance is provided to all programs, including those that meet or exceed the performance benchmarks to address deficiencies in any specific EFL.  Under normal circumstances, programs that do not meet the E&E requirement for three consecutive years are reviewed for possible cessation of funding. Since the beginning of the COVID pandemic, programs were not penalized; however, intense technical assistance and monitoring are provided to promote performance improvement. Through E&E team assistance and targeted training, 34/37 programs met or exceeded the MSG goal of 49% at an overall state average of 60% MSG attainment. This was an improvement over 2021-22, in which 31/37 programs met the requirement. Distance Education training for adult education instructors was offered through the AALRC to meet the continuing need. The state distributed state-provided funds for each program to purchase Chromebooks, laptops, and hot spots to meet program student technology needs and for testing. The 2020-2021 state and national performance data and negotiated targets were shared with local providers at the Fall Administrators’ meeting, through advisory committee meetings, specialized training, and through the ADWS/AES committees that identified the professional development needs and requests to promote continued improvement in program performance. In the analysis of performance data, the Administrators’ Meetings also highlighted training pieces that addressed recruitment and retention strategies, distance education software usage guidance and strategies, addressing ESL needs, and incorporating IET programs into the services offered by each program to increase the credential attainment rate and help participants prepare for the workforce. In analyzing the data to address statewide training needs, training was held multiple times throughout the year, including regional, in-person, and webinar training. Professional development addressed the importance of the follow-up process. It guided the process of collecting and entering the data in the state’s data management system, best practices for developing data entry processes and procedures, monitoring data at the program and class levels, understanding federal tables, utilizing state reports, detailing participant progress by assessment and subject area, developing IETs, Quarterly Report reporting and analysis, and training for diversity and test anxiety. Professional development training will continue to be developed as ADWS/AES moves to improve and meet targets for performance in measurable skill gains and follow-up measures. 
California The WIOA, Title II: AEFLA program offered at 215 local agencies, including two coalitions, enrolled 311,879 learners. Of these, 185,371 (59.4 percent) qualified for NRS federal reporting. California continued to face unprecedented COVID-19 challenges throughout PY 2020–21 and the beginning of 2022. All AEFLA providers including adult schools, community colleges, CBOs, and COEs felt the effects of COVID-19 agency closures and the interruption of services. Enrollment in adult schools declined by 45 percent and in community colleges by 70 percent. IET/IELCE program enrollment declined by 70 percent because of training center closures, and some pathways were no longer practical because of COVID-19. As the COVID-19 conditions improved in PY 2021−22, enrollment in WIOA, Title II: AEFLA programs increased by 20 percent compared to PY 2020−21. ESL increased by 38 percent and ABE by 21 percent. Enrollment in ASE decreased slightly (13 percent). All AEFLA providers reported an increase in overall enrollment in PY 2021−22. Enrollment in IET/IELCE programs doubled, compared to 2020–21 as students started in-person instructions. During the peak COVID-19 period, many AEFLA providers successfully transitioned to distance learning and remote implementation of services. In PY 2020–21, 65 percent (88,749) of 134,492 learners and in in PY 2021–22 more than 36 percent (67,588) of 185,371 learners enrolled in distance learning. Adult learners who qualified for NRS federal reporting in PY 2021−22 reflect the diversity of the state. The largest ethnic groups of learners are Hispanic (64.5 percent), Asian (13.1 percent), White (12.7 percent), Black (6.4 percent), or two or more races (2.7 percent). Adult learners are more likely to be female (57.9 percent). Two hundred seventy-three students indicated non-binary as their gender. The percentage of Asians in the overall adult learners enrolled continues to be lower by three percent compared to pre-COVID-19 period. Adult learners between the ages of 25 and 44 (52.5 percent) comprise the largest age group. Close to 72 percent of learners enrolled are non-English-speaking. COVID-19 greatly affected California’s performance in PY 2020–21. California’s performance in PY 2021–22 improved significantly compared to PY 2020–21. In 2021−22, of the 185,371 learners who qualified for NRS federal reporting, 77,266 (41.7 percent) achieved at least one Measurable Skills Gain (MSG) compared to 34 percent in PY 2020–21. More students were able to take a pretest and a post-test in PY 2021–22 (106,393) compared PY 2020–21 (62,315). More than 22 percent of CASAS pre- and post-tests were administered remotely during PY 2020-21. More than 13,000 learners obtained a high school diploma or high school equivalency certificate. The 185,371 learners who qualified for NRS federal reporting averaged 134 hours of instruction similar to previous program years. The learners (106,393) who persisted in the program and took pre- and post-tests reported 175 hours of instruction. The CDE has data-sharing agreements with the California Community College Chancellor’s Office (CCCCO) and the Employment Development Department (EDD) to conduct a data match to provide postsecondary enrollment, secondary or postsecondary credential attainment, and employment and earnings outcomes. To supplement the employment outcome data match, local WIOA, Title II: AEFLA grantees conduct a universal survey of all students who have not volunteered a Social Security number (77 percent of enrollees). The CDE uses a standardized automated follow-up survey in the TOPSpro Enterprise management information system to collect follow-up outcomes. This process includes an automated survey notification feature sent via email and text messages sent by agencies to students who have exited the program. When the student completes the survey online, TOPSpro Enterprise automatically receives the results. Agencies are able to customize the survey invitation by adding agency logo and name, a survey greeting, and a thank-you note. Survey invitations and the survey itself are available in eight languages. All agencies conduct the Employment and Earnings Survey each quarter. Of those students who exited in PY 2020−21, and either had records that could be matched with the EDD or responded to the employment and earnings follow-up survey, 48 percent were employed in the second quarter after exit. The overall reported percentage of students who exited their program and achieved employment is 23 percent, as most could not be matched with the EDD file. For the students who did not volunteer a Social Security number, 33 percent responded to the survey. The response rate to surveys improved by five percent compared to the previous program year. Also, students who had a primary or secondary goal to get a job or retain a job or get a better job, 30 percent were employed in the second quarter after exit. During PY 2020−21, the CDE continued to provide training and guidance for local agencies to conduct remote instruction and testing. Local agencies continued to submit data to the CDE on a quarterly basis. The previously mentioned data submission structure allows quarterly analysis and early identification of incomplete or inaccurate data. The CDE has provided statewide AFELA network meetings to disseminate successful retention and recruitment strategies to more schools. In terms of action and technical support, the state has focused on student recruitment and retention.  
Colorado From April through May annually, monthly data close out webinars and procedural resources were provided to grantees. At the local level, grantees were required to conduct end-of-year data reporting no later than the second Friday in July. Although data entered in LACES by grantees was monitored monthly for errors, AEI staff ran a final diagnostic report of learner data after that second Friday in July. Any errors identified were sent to grantees for correction in LACES by the end of July to officially close out the data reporting. Once the data was finalized and frozen, grantees were required to complete an Annual Performance Report (APR) either in writing or collaboratively through a virtual meeting. A majority of grantees elected to complete the APR collaboratively and these meetings took place in July and August of 2022. The APR required local grantees to analyze enrollment, post-testing, and MSG outcomes. This analysis covered all program types, Educational Functioning Level (EFLs), classes, and included a prior year comparison to identify areas of strength and areas needing improvement. Grantees were encouraged to document in the report outliers and anomalies as well as any plans for further research and improvement. Grantees have access to mostly finalized Table 4 results in LACES upon the completion of their data close out in July annually. Due to current limitations with data matching partners, Table 4 postsecondary entrance MSGs and all Table 5 employment and postsecondary credential data are not available to grantees during data close out. Instead, grantees receive final aggregate program level Tables 4 and 5 after AEI submits data to the NRS. During annual Table 4 and 5 training offered in the fall and during bi-monthly Office Hours virtual meetings, grantees are encouraged to compare their Table 4 & 5 data to statewide totals and the performance of other AEFLA-funded programs.  The AEI team continues to share current, year-to-date grantee MSG rates quarterly through Office Hours meetings to provide more opportunities for grantees to visualize where their performance ranks across the state. Each also receives an individual report each quarter to show how their data compares to statewide totals and averages on a series of progress measures and outcomes including enrollment, retention to 12 hours, average instructional hours, post-testing rates, and MSG rates by level. Participants increased from 2,973 in 20-21 to 3,715 in 21-22, with 10 out of 14 grantees increasing their participants in the 21-22 program year; and we anticipate continued gradual growth over the next 2 years as impacts of the last competition for funds and the pandemic lessen. Additionally, 11 of the 14 grantees exited fewer learners in 21-22 than compared with 20-21. The focus on retaining learners through graduation or completion was, and continues to be, a focus of statewide technical assistance and training. Improvements are needed in reducing the numbers of learners who exited specifically before making any Measurable Skill Gain as 10 of 14 grantees saw increases in this rate from the prior year.  Statewide, the MSG rate increased just slightly from 44.2% in 20-21 to 44.4% in 21-22. Twelve of 14 grantees met or exceeded the 38% MSG target for the 21-22 program year. Asian Pacific Development Center, located in the Denver Metropolitan area, and Pikes Peak Library District, located in Colorado Springs, both fell below that target at 37.43% and 37.61%, respectively. Neither grantee has been subject to technical assistance as they met 90% of the target, a threshold used by AEI to determine if required technical assistance would be implemented. Asian Pacific Development Center makes up roughly 20% of the statewide participants so technical assistance in pulling MSG rates by instructional staff and regularly monitoring these rates was provided to the grantee in July 2022. Staff turnover at Asian Pacific Development Center has presented challenges for the grantee to implement this regular monitoring.  Post-test EFL gains once again made up the majority of MSGs in 21-22 with over 88% of MSGs coming from post-testing. Post-testing increased statewide from 67% in 20-21 to 69% in 21-22. High school diploma and equivalency gains made up a little over 10% of all MSGs. Postsecondary entrance made up just 1% of MSGs and IET MSGs made up less than 1% of all MSGs in 21-22. Overall, we saw a decrease in MSG rates for ABE learners with decreases specifically at Levels 2-4 (levels 2-4 contained the majority of ABE learners in 21-22). We saw increases at Levels 1 and 5 but these groups make up far fewer of our ABE learners statewide. We have not yet identified causes for the decreases at Levels 2-4. For ESL, overall and at all levels except for ESL Level 5, we saw increases in MSG rates. Like ABE, we have not been able to identify potential causes for the decrease at ESL Level 5. Statewide, Colorado met our 24% employment in the 2nd quarter after exit target as well as the associated $5,176.00 median earnings in the 2nd quarter after exit target at 25.40% and $6,226.00, respectively. Both are increases from the prior year. Similarly, Colorado met our 12% credential attainment rate target with a rate of 22.72% in 21-22, also an increase from the prior year. Work continues with grantees to focus on increasing these exit-based outcomes through stronger relationships across WIOA partners, in-house career coaching services, the development of more and more diverse IETs and workforce training, and expanding the match criteria and increasing the frequency of data matches.  For the 26% employment in the 4th quarter after exit target, we once again did not meet the target with just 19.82% of learners in the cohort being employed 12 months after exit. Of the cohort, a little over 42% of learners reported a Social Security Number (SSN). Currently, this is the only match criteria the Colorado Department of Labor and Employment is willing to match on which increases the challenge in meeting employment targets.  Conversations are ongoing with the Data Trust convened by the Colorado Workforce Development Council on accessing employment and wage data quarterly and on using additional learner demographics to match with the Unemployment Insurance Compensation data set. AEI continues to work with grantees on processes for collecting acceptable forms of supplemental wage information, especially with learners who are not able or willing to report an SSN.  Eighty-five percent of the cohort exited before July 2020 during the height of the pandemic and before the latest AEFLA competition. It seems that adults exiting due to the pandemic faced additional barriers in obtaining or retaining jobs a year later. Additionally, regardless of exit date in the cohort, 38% of the cohort were enrolled in programs that did not continue to receive AEFLA funding after June 2020. These grantees had little incentive to support learners in pursuing Table 5 outcomes once they were no longer receiving AEFLA funds.  AEI has had preliminary conversations around how to best support grantees who were not able to meet or exceed the Table 5 targets. Current considerations include adding Table 5 targets to the state’s risk assessment and monitoring tools, implementing supplemental employment and wage data collection, and ongoing work with grantees on SSN data collection.  
District of Columbia In FY22, OSSE AFE awarded continuation funding to 12 sub-grantees to implement Integrated Education and Training service models introduced in the FY17 grant competition and continued in the FY21 grant competition. The models include providing adult education and literacy, workforce preparation, and training services for a specific occupation or occupational cluster to 1,000 District residents for education and career advancement. In total, 1,122 adult learners received services in OSSE AFE-funded programs in FY22. Of this number, 1,080 learners met the National Reporting System (NRS) guidelines of having a valid assessment and 12 or more instructional hours in the program year to be reportable to the US Department of Education per NRS Table 4. The remaining 42 adult learners engaged in one to 11 instructional hours per NRS Table 2A. DC FY22 Participants by Gender (NRS Table 2) In FY22, female learners represented 62 percent (n = 674) and male learners represented 38 percent (n = 406) of the total number of students served (n = 1,080). DC FY22 Participants by Ethnicity (NRS Table 2) At 66 percent (n = 712 of 1,080 students), black or African American participants comprised the single largest ethnic group of learners served; Hispanic or Latino students followed at 29 percent (n = 314 of 1,080 students). The percentage of American Indian or Alaskan, Asian, or white participants and persons of Two or More Races was 5 percent (n = 54 of 1,080 students). DC FY22 Participants by Age Consistent with past years, the largest single group of learners served were between ages 25 and 44 (49 percent/n = 538 of 1,080 students). The second largest group of learners served were between 19 and 24 years of age (26 percent/n = 283 of 1,080 students), followed by learners between the ages of 45 and 54 (10 percent/n = 112 of 1,080 students). The smallest groups of learners served were at opposite ends of the age continuum and included students ages 60 and older (4 percent/n = 48 of 1,080 students), 55-59 years of age (4 percent/n = 44 of 1,080 students), and 16 to 18 years of age (7 percent/n = 55 of 1,080 students). DC FY22 Participants by Program Type (NRS Table 3) In FY22, of the total number of learners (n = 1,080) who met the NRS guidelines, students in Adult Basic Education (ABE)/Integrated Education and Training (IE&T) Programs comprised the single largest group by program type (56 percent/n = 607 of 1,080 students). The second and third largest groups by program type were Adult Secondary Education (ASE)/IE&T Programs (24 percent/n = 259 of 1,080 students), followed by Integrated English Literacy and Civics Education/IE&T Programs (12 percent/n=131 of 1,080 students). The fourth and smallest program type was English Language Acquisition/IE&T Programs (8 percent/n = 83 of 1,080 students). DC FY22 Measurable Skills Gains by Entry Level (NRS Table 4) Adult Basic Education (ABE) Participants by Entry Educational Functioning Levels Of the total number of ABE participants, the most significant number of students entered at ABE Level 4 (33 percent/n = 306 of 917 students), followed by ABE Level 3 (24 percent/n = 223 of 917 students) level. The smallest number of participants entered at ABE Level 1 (.02 percent/n = 22 of 917 students). English as a Second Language (ESL) Participants by Entry Educational Functioning Levels Of the total number of ESL participants, the largest number of participants entered at ESL Level 5 at (29 percent/n = 47 of 163 students), followed by the second largest number of participants who entered ESL Level 6 (27 percent/n = 44 of 163 students). The smallest number of ESL participants entered in ESL Level 2 (.06 percent/n = 9 of 163 students). Measurable Skills Gains by Entry Level For FY22, OSSE AFE negotiated a measurable skill gains performance target of 47 percent for all ABE and ESL Educational Functioning Levels (EFLs). This target represents the proposed percentage of adult learners making a measurable skill gain (e.g., achieving an educational functioning level gain, acquiring a secondary school diploma or its equivalent, exiting a program below the Postsecondary level and enrolling in postsecondary education and training during the program year, attaining a postsecondary or secondary transcript, making progress towards milestones, or passing a technical/occupational skills exam). OSSE AFE exceeded its target of 47 percent, with 58.46 percent of students enrolled in an OSSE AFE-funded program achieving a gain. This reflects a 2.83-point increase in the percentage of students with measurable skill gains in FY21 – (55.63 percent). The chart below reflects the percentage of adult learners who made measurable skills gains. Students at ABE Level 5 had the highest rate of measurable gains at 58.52 percent, followed by students at ABE Level 4 (58.06 percent) and ABE Level 3 (57.96 percent). Students at ESL Level 3 had the highest percentage of measurable gains at 93.33 percent, followed by students at ESL Level 2 (88.89 percent) and ESL Level 5 (70.83 percent). DC FY22 Core Outcome Follow-up Achievement (NRS Table 5) For FY22, OSSE AFE negotiated targets for the core follow-up outcome measures on NRS Table 5 as follows: Employment (2nd quarter after exit) – OSSE AFE’s federally negotiated target for this metric during this reporting period was 21 percent. OSSE exceeded this target in FY22.  The state’s performance was 38.36 percent which reflects an increase of 23.24 percentage points in FY22 compared to 15.12 percent in FY21. Employment (4th quarter after exit) – OSSE AFE’s federally negotiated target for this metric during this reporting period was 23.0 percent. OSSE exceeded this target in FY22.  The state’s performance was 26.65 percent, which reflects an increase of 14.63 percentage points in FY22 compared to 12.02 percent in FY21. Median Earnings (2nd quarter after exit) – OSSE AFE’s federally negotiated target for this metric during this reporting period was $8,000. OSSE achieved 98 percent of this target in FY22.  The state’s performance was $7,800, which reflects an increase of $910 in FY22 compared to $6,890 in FY21. All employment and wage data are collected through follow-up surveys with program exiters. Where possible, data match with DOES’s Unemployment Insurance wage data and the State Wage Interchange System (SWIS). However, matching against the DOES UI wage data and SWIS wage data requires a social security number which we do not require for enrollment in AFE programming and for which only a fraction of our learners voluntarily provide. We know that this leads to an under-representation of our learners’ employment and wage data. Credential Attainment (Unduplicated) – OSSE AFE’s federally negotiated target for this metric during this reporting period was 48 percent. OSSE AFE achieved 69 percent of this target.  The state’s performance was 33.08 percent which reflects a decrease of 11.79 percentage points compared to 44.87 percent in FY21. Because the credential attainment rate is a lagging post-exit measure based on survey responses, the FY22 outcome data for credential attainment rate focusses on students who exited the program during Jan. 1 2020 through Dec. 31 2020 which was the first year of the pandemic when much of the city had shut down. This had a direct impact on a student’s ability to test or get their practicum hours associated with their certifications. Therefore, the drop in FY21-22 for credential attainment was significantly impacted by the effects of the pandemic. Attainment of a Secondary School Diploma/Recognized Equivalent From July 1, 2021, to June 30, 2022:
  • 169 District residents passed all four components of the GED Exam and earned a DC State Diploma.
  • 37 DC residents completed the National External Diploma Program and earned a DC Public School, DC Public Charter School, or DC State Diploma
Georgia During FY22, GOAE continued its target negotiation process for local programs to set enrollment and Measurable Skills Gain (MSG) targets for the year. The purpose of the target negotiation process is to allow providers to negotiate reasonable, but ambitious, targets while still ensuring GOAE fulfills its roles and responsibilities as a pass-through entity of WIOA funds. GOAE provided local programs with data, a negotiations workbook, and factors to consider in a process that mimics the federal target negotiation process. GOAE then monitored each program’s progress towards their targets throughout the year and provided support accordingly. GOAE’s data system, GALIS, produces state- and local-level reports to support regular data analysis by GOAE and local programs. For example, GOAE and local programs used Table 4 reports disaggregated by instructor, class, and student to monitor performance regularly. GOAE also created a weekly internal report to monitor each program’s progress towards its FY22 targets throughout the year. All programs received a mid-year data report from GOAE in January 2022 to show their progress, and programs falling behind their targets were provided targeted technical assistance. GOAE also implemented quarterly local program data quality reports that included benchmarks related to performance to help programs monitor their data throughout the year, such as post-test rates. Georgia’s overall MSG percentage for FY22 was 46.01%, which did not meet the 90% threshold of the negotiated 53.8% target for FY22. Programs continued to face challenges posed by the COVID-19 pandemic during FY22 such as staffing shortages and student enrollment. Nevertheless, overall enrollment increased by 30% compared to the prior year and although MSG performance remained lower than pre-pandemic rates, the overall MSG percentage was six percentage points higher than the prior year. In FY22, GOAE began reporting MSGs for Integrated Education & Training (IET) and Workplace Literacy students.  Additionally, programs improved retention and overall post-test rates, both of which contributed to improved MSG performance. When analyzing students who did not earn gains, the majority of students were those who exited prior to earning an MSG.  Upon analysis, the percentage of students earning educational functioning level gains continues to be 13 percentage points lower than pre-pandemic percentages, indicating challenges with retention and student post-test success. GOAE plans to provide additional support for retention and instruction in FY23 including targeted professional development, access to instructional resources for staff and students, and implementing a customer relationship management tool to better engage students. GOAE saw similar trends in enrollment and MSG for ABE and ESL students. Both ABE and ESL student enrollment increased by 30% in FY22 compared to the prior year, indicating a rebound in enrollment from the COVID-19 pandemic. However, overall enrollment remains 35% lower than pre-pandemic FY19 enrollment. GOAE will continue to provide marketing and recruitment support to programs in FY23 to continue to increase enrollment. ABE MSG performance increased by seven percentage points to 45% in FY22, and ESL MSG performance increased by two percentage points to 49%. The statewide post-test rate improved by five percentage points to 52% in FY22, and this increase was reflected for both ABE and ESL students. However, the post-test rate and MSG percentage for post-tested ABE students remains lower than ESL students. 49% of ABE students are post-tested, and the MSG performance for those students is 51%; for ESL students, the post-test rate is 61% and the MSG performance for those students is 77%. The combination of increased rigor of the TABE 11/12 and retention struggles continue to create challenges for achieving MSGs for ABE students. GOAE plans to provide technical assistance and access to instructional resources to assist programs in this area. For the other indicators of performance, GOAE saw a five-percentage point increase in second quarter employment compared to FY21 and a two-percentage point increase in fourth quarter employment. GOAE saw a slight decrease of one percentage point in credential attainment, largely due to a change in business rules to reflect clarifications provided during an NRS webinar. GOAE also saw a decline in the overall raw number of participants included in the follow-up indicator calculations which is a reflection of the drop in enrollment from the pandemic and the time lag for cohort inclusion. In FY22, GOAE established specific strategic goals focused on employment and postsecondary transition outcomes to ensure our programs are also monitoring data in these areas. For instance, GOAE partnered with the technical college system to launch a “Keep GOing” marketing effort in FY22 that provided programs with templates and resources to support transitioning adult education students to postsecondary. Programs continued to offer distance education and virtual services through the Blackboard learning management system in FY22, but also re-opened many in-person services during the year. In FY22, 39% of participants were classified as distance education and 42% of contact hours were proxy contact hours, compared to 62% distance education students and 64% proxy contact hours in FY21. Given that FY22 was the second year GOAE implemented negotiated targets with local programs, GOAE used progress towards FY22 targets to identify programs for additional technical assistance throughout the year and placed three programs on Strategic Improvement Plans in FY23 based on FY22 performance. GOAE also used FY22 performance and its strategic initiatives to revise the target negotiation process for FY23 to include targets for IET credential attainment and HSE attainment. GOAE will continue to monitor progress towards these targets through regular data analysis, mid-year reports, and accountability conversations. Based on program performance, GOAE will implement technical assistance, Strategic Improvement Plans, sanctions, and incentives.
Guam During this reporting period, there were 192 adult learners with at least 12 instructional hours, with 61.45% achieving at least 1 EFL.  Twenty-three students (12%) attained a high school diploma or equivalent.  Most of the students were female (58%), and the ethnicity of the participants was primarily Pacific Islanders (60%).  The percentages of adult learners’ functioning levels were 60.64% (ABE and ASE) and 64.86% in English-as-a-Secondary Language (ESL).  The 2% decrease in participants compared to the previous program year may be due to the continued rise in positive cases of COVID-19.  Some students cared for elderly parents or other family members who were immunocompromised.  Additionally, transportation continued to be a challenge for some students.  Hence, the decrease in enrollment.  Table A indicates students’ profiles extracted from NRS Tables 1 through 3.   Table A - Summary of Program Participants Program Year Participants Male Female Number who achieved at least one educational functioning level gain Number who attained AHS or HSE diploma Separated before achieving a measurable skill gain Remaining in program without Measurable Skills gain Native Hawaiian or Other Pacific Islander All other 2019-2020 224 77 147 94 29 84 17 147 77 2020-2021 196 63 133 110 25 31 30 154 42 2021-2022 192 88 111 95 23 47 27 116 76   Once again, Guam exceeded its measurable skills gain (MSG) target (50%) for this program year.  Surpassing the target may be attributed to having a dedicated counselor for the program, which is critical for retention.  Additionally, the local program’s commitment to addressing student needs and engagement with students ensures student success, especially during the pandemic.   Table B - Summary of Measurable Skills Gain by Entry Level Description PY2019-2020 PY2020-2021 PY2021-2022 ABE Level 1 (Beginning Literacy) 50.00% 100.00% 0.00% ABE Level 2 (Beginning Basic) 75% 80.95% 52.38% ABE Level 3 (Intermediate Low) 46.51% 68.97% 58.97% ABE Level 4 (Intermediate High) 53.93% 68.18% 65% ABE Level 5 (ASE Low) 29.63% 47.06% 62.50% ABE Level 6 (ASE High) 66.67% 100.00% 33.33% ABE Total 50.82% 68.45% 60.64% ESL Level 1 (Beginning Literacy) 0.00% 0.00% 0.00% ESL Level 2 (Beginning Low) 0.00% 0.00% 0.00% ESL Level 3 (Beginning High) 50% 0.00% 0.00% ESL Level 4 (Intermediate Low) 80% 100.00% 55.55% ESL Level 5 (Intermediate High) 84.21% 66.67% 84.61% ESL Level 6 (Advanced) 66.676% 66.67% 50% ESL Total 75.61% 77.78% 64.86% Grand Total 55.36% 68.88% 61.45%   For PY 2021 - 2022, the performance indicators for Employment Rate Q2, Q4, Median Earnings, and the Measurable Skill Gains exceeded the statewide target rate.  Guam did not meet the Credential Rate.  Although the exit surveys were automated for follow-up surveys in the TOPSpro Enterprise this program year and conducting personal callouts, contacting participants who exited the program was challenging.  The local program planned on establishing a system to follow up with participants who exited during the reporting period by obtaining next of kin contact information, such as phone numbers, email addresses, and mailing addresses, that may improve or increase the performance indicators in the Credential Rate.      Program Year Participants   Served:      7/1/2021 – 6/30/22 Exited:        7/1/2021-6/30/2022 Total Statewide Target     Total Statewide Actual 204 150     Employment Rate (Q2): 7/1/2020 – 6/30/2021   Number Rate Total Statewide Target 64 16% Total Statewide Actual 27 20.30%     Employment Rate (Q4): 1/1/2020 – 12/31/2020   Number Rate Total Statewide Target 64 16% Total Statewide Actual 36 19.14%   Median Earnings: 7/1/2020-6/30/2021   Earnings Total Statewide Target $1,000   $3,180       Credential Rate: PY 1/1/2020 – 12/31/2020   Number Rate Total Statewide Target 0 26% Total Statewide Actual 0 0.00%   Measurable Skills Gains: PY 7/1/2021 – 2022   Number Rate Total Statewide Target 138 44% Total Statewide Actual 118 57.84%        
Hawaii The state met its PY 2021 - 2022 targets for median earnings and the credential rate. The state did not meet its targets for employment rate in Q2 and Q4 and measurable skills gain (MSG). Although it did not meet the MSG target, it reduced the gap between the target and actual from PY 2020 - 2021, -5.62, and PY 2021 - 2022, -3.03. The statewide performance report indicators for Employment Rate Q2, Employment Rate Q4, Median Earnings, and Credential Rate outcomes and targets are listed in the Hawaii PY 2021 - 2022 table.   Hawaii PY 2021 – 2022                                      Total Participants Served  Cohort Period: 7/01/2021 - 6/30/2022  Total Participants Exited  Cohort Period: 4/01/2021 - 03/31/2022  Employment Rate (Q2)  Cohort Period: 07/01/2020 - 06/30/2021                                                                                                                                 Number           Rate       Total Statewide Target     1,086 28.60% Total Statewide Actual 3,370 2,018 302 14.09%                                                   Employment Rate (Q4)                 Cohort Period: 1/01/2020 - 12/31/2020                 Median Earnings                    Cohort Period: 7/01/2020 - 06/30/2021                                               Number                        Rate                                     Earnings                        Total Statewide Target 450 11.30% $5,191.00 Total Statewide Actual 243 9.72% $5,232.50                                                         Credential Rate                       Cohort Period: 1/01/2020 - 12/31/2020            Measurable Skill Gains             Cohort Period: 7/01/2021 - 06/30/2022                                                 Number                       Rate                       Number                   Rate            Total Statewide Target 38 4.70% 1,892 40.00% Total Statewide Actual 31 5.22% 1,246 36.97%   Employment Rate Q2 and Q4 The Employment Rate Q2 and Q4 declined in PY 2021 - 2022. A data matching agreement is being developed with the Hawaii Department of Labor Unemployment Insurance Division to provide better data collection for Q2 and Q4 indicators and to inform program improvement.   Employment Rate   Q2               2019 – 2020                           2020 – 2021                       2021 – 2022          Statewide Target 0% 28.60% 28.60% Statewide Actual 16.37% 16.30% 14.09% Difference Between Target and Actual +16.37% -12.30% -14.51%     Employment Rate   Q4             2019 – 2020                          2020 – 2021                        2021 – 2022           Statewide Target 0% 11.30% 11.30% Statewide Actual 16.77% 16.19% 9.72% Difference Between Target and Actual +16.77% +4.89% -1.58%   Median Earnings In PY 2021 - 2022, the median earnings were significantly lower compared with PY 2020 - 2021 but still met the target.   Median Earnings               2019 – 2020                           2020 – 2021                        2021 – 2022           Statewide Target 0 $5,090.00 $5,191.00 Statewide Actual $7,200.00 $7,930.50 $5,232.50 Difference Between Target and Actual +$7,200 +$2,840.50 +41.50   Measurable Skills Gain Although the performance target for MSG was not met in PY 2021 - 2022, the difference between the statewide target and the actual percentage improved from PY 2020 - 2021. The gap between the target and the actual decreased from -5.62% to -3.03%. MSG             2019 – 2020                            2020 – 2021                       2021 – 2022            Statewide Target   38.00% 40.00% 40.00% Statewide Actual 29.89% 34.38% 36.97% Difference Between Target and Actual -8.11% -5.62% -3.03%   Credential Rate The PY 2021 - 2022 credential rate decreased by 1.02%  from PY 2020 - 2021 but still met the performance target.     Credential Rate                2019 – 2020                          2020 – 2021                        2021 – 2022            Statewide Target 0% 4.70% 4.70% Statewide Actual 3.54% 6.24% 5.22% Difference Between Target and Actual +3.54% +1.54% +.52%   Participants In PY 2021 - 2022, participants increased by 17% from PY 2020 - 2021. Although the participant count is still below pre-pandemic numbers, it significantly increased from PY 2020 - 2021.       Participants                   2019 – 2020                           2020 - 2021                        2021 – 2022           Served 3,830 2,885 3,370 Exited 3,394 1,636 2,018 Difference Between Served and Exited -436 -1,249 -1,352 Looking toward PY 2022 – 2023, the state expects performance to change by implementing the following actions:
  • Improve data collection on participant intake by modifying the intake forms to include the 11 employment barriers criteria. This will provide local characteristics in the performance target-setting process.
  • Establish data matching using unemployment insurance data for accurate Q2 and Q4 employment outcomes.
  • Implement a media campaign to increase enrollment.
Idaho Process: The state ran monthly reports to analyze data related to core performance targets.  The TRC1 and the state director reviewed these reports for any anomalies or concerns.  Particular attention was given to enrollment numbers, retention rates, post-testing rates, and MSG level gains.  Any red flags were documented and a follow-up phone call to the local director took place.  Collaboratively, the local director and state director would discuss potential data input errors or identify strategies for resolving the concern.  These monthly reports were used as additional information to compare against the quarterly desk audits.  At the local level, data specialists or data staff review their data daily – particularly to check for new student enrollment data and accurate scores.  Accurate class placement is reviewed daily.  Any data concerns are taken to the local director and may or may not include instructor input.  If any data continues to not accurately reflect a program, a technical assistance ticket is submitted to LiteracyPro along with notification to the state that a concern has been elevated to LiteracyPro.  Results: PY21 Employ Q2 Employ Q4 Median Earnings MSGs CR Target 60.00% 45.00% $4600.00 43.00% 40.00% Actual 31.90% 40.57% $5446.07 32.49% 28.84% PY20           Actual 32.03% 34.94% $5853.12 21.59% 40%             Analysis: Comparing data to the prior program year, Idaho made some positive data increases in some performance areas while dropping in other areas.  Noteworthy accomplishments included Q4 employment 5.63% increase and MSGs 10.9% increase.  This trend is positive given the new negotiable targets for this current fiscal year.  The median earnings target was met.  Two performance areas improved while one other was met.  While some performance targets were not met, Idaho witnessed improvements from the prior program year.  Given this was a rebuilding year from the pandemic, these increases demonstrate that programs used a variety of learning platforms which created access as witnessed by the increase in enrollment.  Idaho has several industries that aggressively hired without a GED or high school diploma requirement and provided on-the-job training.  Construction, mechanic shops, food service, and healthcare facilities are examples.  The unemployment rate in Idaho was and continues to be very low, and programs saw participants leave for $15 - $ 17-hour wages.  It is critically important that AE continues its work with the Idaho workforce system and prepare students for sustainable wages and additional educational opportunities.  Action: The overall assessment of programs to improve quality and effectiveness was built into the bi-weekly AEFLA director meetings and the quarterly data desk audits.  Reviews of the data and corresponding trends at both the program and state levels allow for more timely intervention and correction of issues.  This approach contributed to the increases in some of the areas of performance.  This will continue in future reporting.  Additionally, data teams were created at the local programs to review data on a daily and weekly basis.  If students needed to be placed higher or lower in a given class, the intervention was immediate.  Likewise, if a student needed additional academic intervention, this too was immediately provided.  Additional supports were added to programs such as group and individual academic tutoring and career plans.  These interventions will continue in the future to increase retention and completion rates. 
Illinois Process: Illinois  completed ongoing data analysis through desk-top monitoring from program support staff, data staff, and the Senior Director for Adult Education to monitor data quality. The required annual submission of a Data Quality Checklist reinforced the requirement of data integrity and identified any programmatic misunderstandings of adult education data, allowing for immediate and ongoing support. Additionally, the State MIS system prevents student files with errors to be included in the final report and the Error Status report in the MIS identifies errors that are corrected throughout the fiscal year. Once the end of year reporting was completed, the ICCB Research and Analytics division compare current year data with prior year data to further test for anomalies  and address areas of concern. There were no anomalies in the data for PY21. Results: Illinois Adult Education providers’ performance were measured for outcomes by the following NRS Core Indicators of Performance:
  • Employment Rate 2nd Quarter: 29.54% of participants who exited programs were in unsubsidized employment during second quarter. This is .1% higher than PY20. Performance in this indicator exceeds the negotiated target (27.7%).
  • Employment Rate 4th Quarter: 26.00% of participants who exited programs were in unsubsidized employment during fourth quarter. This is .1% higher than PY20. Performance in this indicator falls within the 90% threshold of the negotiated target (28.4%).
  • Median Earnings: The median earnings of program participants who were in unsubsidized employment during the second quarter after exit from program was $5,739.00. This is $686.00 higher than PY20. Performance in this indicator exceeds the negotiated target ($4,651.00).
  • Credential Obtainment: 27.64% of program participants eligible to be included in this measure obtained a recognized postsecondary credential or a secondary school diploma or its recognized equivalent during participation in or within one year after exit from the program. Performance in this indicator exceeds the negotiated target (27.1%).
  • Measurable Skill Gains: 38.07% of participants who were in an education or training program achieved measurable skill gains toward a recognized postsecondary credential or employment. Performance in this indicator fell below the 90% threshold of the negotiated target (44.8%). This metric was significantly impacted by the challenges related to virtual assessment brought on by variants of the COVID-19 pandemic. However, the MSG rate was 6.49% higher than PY20. Additionally, to improve on this outcome, the ICCB has implemented a centralized, state facilitated Assessment Hub in PY22 for post-testing adult learners.
Analysis: PY21 saw an increase of 40.5% of NRS reportable learners from PY20 with English Language Learners (NRS Levels 1 – 5) increasing by 49.5%. In contrast, ABE Learners (NRS Levels 1-4) only had a 12.11% increase. Action: Throughout PY21, the Senior Director reviewed weekly statewide enrollment reports that included Adult Education and Literacy enrollment and Bridge and IET enrollment. Programs with low enrollment were referred to the Professional Development Network for targeted Technical Assistance. Additionally, the ICCB Adult Education Program Support team facilitated weekly desktop reviews on post-testing and measurable skill gains at the local level, providing technical assistance or referrals to the PDN’s ongoing targeted professional development. Additionally, as a part of the FY21 AEFLA Grant Continuation, funded programs were required to submit a work plan that identified the actionable steps toward meeting grant deliverables and continuous improvement. Progress toward goals on the work plan were reported. While there was consistent communication between program level staff and the ICCB AEL staff, these quarterly reports provided program level staff the opportunity to review their progress toward goals and communicate specific technical assistance needs. The ongoing impact of the COVID-19 variants hindered local programs and these issues remained at the forefront of the ICCB’s approach to support. In addition to the high-quality professional development, the Professional Development Network held monthly virtual office hours on Assessment, Improved Data Practices, Retention Strategies, English Language Learner Instruction, and Bridge and IETs were facilitated so local program staff could have questions answered. These office hours were designed to allow programs to also share promising practices. These supports have been impactful and will be continued as one component of the state’s approach to technical assistance. In PY22, Illinois’ student enrollment is trending toward increased numbers of economic migrants, refugees, and asylees brought to the state from Texas. As of the development of this report, the ESL enrollment is 72% of the statewide enrollment and 52% of those in our data collection system lack a social security number. This will impact our outcome measures for Employment 2nd Quarter after Enrollment and Employment 4th Quarter after enrollment. Additionally, it is unknown how long the asylees will remain in Illinois – potentially impacting both post-test rates and Measurable Skill Gains. In response to these concerns, and enrollment for ESL trending toward the NRS Level 1, the ICCB is facilitating a full day NRS Institute, reinforcing improved data practices, identifying strategies for collecting social security numbers, reminding programs of the importance of identifying students as Not in the Labor Force and Barriers to Employment. The Senior Director for Adult Education will review monthly enrollment trends which includes Measurable Skill Gains. The Professional Development Network has been directed to develop and facilitate virtual trainings on NRS Level 1 instructional strategies. Furthermore, the ICCB hired a Director for Integrated Literacy and Civics Education to add another layer of support and oversight.
Indiana Overall, Indiana continued to perform at high levels with respect to measurable skill gains, high school equivalencies, and credential attainment. Enrollment was up nearly 11 percent from the previous year, a strong emphasis on incorporating hybrid and virtual instruction, coupled with in-person classes was highlighted in professional development activities. In an assessment of core programs based on core indicators of performance, targeted instruction and a focused assessment program were key to achieving high measurable skill gain rates for ABE and ELL. These percentages exceeded core indicators of performance. The pandemic hit ELL enrollment the hardest, and while recognizing challenges faced by ELL students with fewer in-person options, professional development activities focused on ensuring all students achieved measurable skill gains. Local providers modeled promising practices that were highlighted in monthly regional and professional development facilitator meetings and statewide webinars. The measurable skill gain percentages for ABE and ELL were near the highest the state has ever achieved based on a concerted effort to ensure that learners were afforded the best instructional strategies and support services in uncertain times. While measurable skill gains were down approximately one percentage point from PY 2020, the overall performance outperformed national averages. An analysis of enrollment trends was conducted in PY 2021 as numbers decreased from the previous year and as the state office began efforts to rebuild participation levels. However, the statewide online- only programs for ABE and ELL flourished and continued to grow during this period though concerns about retention and positive outcomes of students escalated as numbers climbed. Building enrollments was a focus in PY 2021 ensuring quality and effectiveness of service. An analysis of target populations and “most in need” were an emphasis as well. The spotlight on enrollments paid off as the pandemic eased and more ELL students returned to in-person classes. Nearly 2,000 more ELL students, a 54 percent increase, were enrolled in PY 2021 than the previous program year while ABE enrollments remained steady. Though ELL fueled the increase in statewide enrollment, measurable skill gains for these students dropped slightly.    PY 2021 Performance Outcomes   19,430
  • Student Enrollments
Increase from PY 20 13,356
  • Level Gains (student may have achieved more than one gain)
Increase from PY 20   3,795
  • HSEs and HS Diplomas Awarded
Decrease from PY 20    2,005
  • WEI (Workforce Education Initiative) Enrollments
92.17% completion rate   2,984
  • IET Participants
  • 90.97% completion rate
88.46% certification rate 6,089
  • Employment Second Quarter after Exit
  • Decrease from PY 2020
  • Employment Fourth Quarter after Exit
Increase from PY 2019 and PY 2020 3,925
  • Attained Any Credential after Exit
  • Increase from PY 2020
IDWD has a system of state performance metrics in addition to the federal measures required under the National Reporting System. Above is a summary of outcomes that state staff shared broadly with practitioners in the field.  As stated earlier, this year saw a significant increase in enrollments as the pandemic subsided. The state showed strong numbers in several areas. The number of students obtaining their HSED in PY 2021, however, decreased 20 percent from the previous program year. The number can be attributed, in part, to a change in test vendors in PY 2021. Program Year # of HSEs awarded 2010 4,848 2011 5,683 2012 7,349 2013 6,759 2014 5,405 2015 5,132 2016 4,870 2017 4,989 2018 4,932 2019 4,146 2020 4,744 2021 3,795 Of the 19,430 students enrolled, 29 percent were English Language Learners (ELL), up from 21 percent in PY 2020. Of ELLs, 57 percent entered instruction at the bottom two educational functioning levels (EFL), up five percentage points from the previous year. ELLs attended an average of 75 hours of instruction. Of  ELLs served, 61 percent made a level gain down from 65 percent the previous year.  Of ABE/ASE student enrollments, 96 percent entered instruction at an EFL of Level 4 or below. The largest group (38 percent) entered at EFL Level 3. The overall ABE/ASE student group attended an average of 75 hours of instruction and 76 percent of students received a post-test. At 50 percent, students between the ages of 25-44 are the largest demographic. Particularly interesting to Indiana is the percentage of students who fell within the WIOA Youth age range. One-third of Indiana enrollments were between the ages of 16-24. This percentage is about the same from the previous year. There was a 13 percent increase in 16-17-year-old enrollments; IDWD shared this information at a local level to providers and regional workforce development boards as they looked to develop out-of-school youth programming.  NRS Table Highlights Table 4 In summary, with higher enrollments, Indiana finished PY 2021 with an overall Table 4 MSG of 68 percent, which was slightly down from the previous program year’s average. Still, Indiana exceeded the latest national average of 35 percent by 33 percentage points in PY 2020. As noted, the 11 percent increase in enrollments for PY 2021 was largely driven by ELL students. Along with state office support, three local adult education providers were housed during the winter at Camp Atterbury to assist with ELL services to about 7,000 Afghan visitors who would be relocated to Indiana communities and other areas. The quality of ABE and ELL performance remained nearly the same on Table 4 and overall performance was excellent in the final analysis.   (As a preview to PY 2022, the state augmented the mid-year incentive to include a targeted enrollment goal for each local program.) The post-pandemic months affected employment and credential attainment indicators, too. The fourth quarter after exit employment indicators were up from PY 2020 and median earnings second quarter after exit were $5,688, an increase from $4,985 in PY 2020. The credential indicator was up by nearly 750 from last year.
Iowa Performance Data Analysis In PY 2021-22, a typical adult education and literacy student served was female (52.4%), employed (50%), a minority (67.8%), averaging 25-44 years of age (51.0%), with the highest year of school completed 9-12th grade (69.8%).  Iowa had a post test rate of 60% this program year with nine of twelve educational functional levels meeting the target and an overall measurable skill gain rate of 49%. We also had an increase in high school equivalency completers with a total of 1,549. Our participants' median program hours is 63 with those making a gain having 94 hours. Both fall within the recommended hours from CASAS, our assessment vendor. Covid-19 Impact on Performance Enrollment increased this program year with 9,328 participants and has yet to reach pre-pandemic levels of 11,039. Distance education participants declined from 30.5% of participants last program year to only 12.8% of participants this program year. That said, it is higher than pre-pandemic levels. Also, the measurable skill gain rate was higher at 51.4% compared to our statewide average of 48.8% this program year and to the 40.0% rate last program year. Iowa’s Adult Education and Literacy Enrollment Overall enrollment has decreased 4.6 percent over the past five years with ABE enrollment decreasing 4.8 percent and ESL decreasing 4.3 percent. Participants increased 26.4 percent in PY 2022 with 9,328 participants compared to PY 2020-21 with 7,501 participants. Of the total number of participants that met the NRS guidelines, students in ABE, including ABE Levels 1-6, comprised the largest group by program type with 57.1 percent of the total learners served. Students enrolled in ESL, including ESL Levels 1-6, increased to 42.9 percent from 34.1 percent enrolled. This is a return to pre-pandemic levels in PY 2019-2020 with ABE at 56.1 percent and ESL at 43.9 percent. Participant Enrollment   2018 2019 2020 2021 2022 Adult Basic Education 6,486 6,507 5,319 4,942 5,328 English as a Second Language 4,761 4,532 4,159 2,559 4,000 Total 11,247 11,039 9,478 7,501 9,328 Retention Student retention is critical to the process of assessing progress. Persistence and sufficient hours increased slightly with 74.5 percent of the 12,529 individuals seeking services persisted for a minimum of 12 or more hours of instruction and a pre-test. Average hours of instruction for the 9,328 enrollees federally reported increased again this year to 74 hours which is up from 67 last program year. This remains less compared to the PY 2018-2019 where the average was 124 hours. ESL averaged the highest with 90 hours while ABE also increased to 62. Using ABE Level 1-5 and ESL levels 1-6, the posttest rate was 60%.  Of the 5,198 post tested, 70.0 percent achieved a measurable skill gain, up from 65.0 percent in program year 2020-21. Of the 663 participants at ABE Level 6, 70.5 percent achieved a measurable skill gain. Measurable Skill Gains Of the 9,328 participants, Iowa had 9,492 periods of participation. Of the 9,492 periods of participation, 48.8 percent (4,636) made a measurable skill gain. This is a 10.0 percent increase over PY 2020-2021 and exceeded our overall target of 47 percent. Measurable Skill Gains Functioning Level PY18 MSG PY19 MSG PY20 MSG PY21 MSG PY22 MSG PY22 Goal ABE Beginning Literacy 43% 36% 26% 36% 39% 38% ABE Beginning Basic 44% 35% 31% 34% 46% 45% ABE Intermediate Low 44% 36% 32% 37% 42% 43% ABE Intermediate High 42% 42% 39% 43% 51% 45% ASE Low 60% 60% 48% 49% 55% 60% ASE High   69% 61% 69% 70% 68% ESL Beginning Literacy 54% 54% 36% 53% 46% 45% ESL Low Beginning 52% 50% 39% 50% 53% 51% ESL High Beginning 54% 47% 36% 47% 45% 51% ESL Low Intermediate 47% 41% 33% 46% 46% 46% ESL High Intermediate 49% 48% 36% 46% 49% 48% ESL Advanced 33% 30% 26% 31% 35% 30%   Primary Indicators of Performance Iowa utilizes a data match process to determine if performance targets have been met for the follow-up core measures. Employment the 2nd quarter after exit was down slightly to 52.3 percent from 53.0 percent the previous program year. Median quarterly wage earnings for 2nd quarter after exit were also down to $6,069 compared to last reporting year at $6,681. IELCE participants continue to have the highest quarterly earnings with $12,773 while IET participants have a high rate of employment at 73.7 percent. Distance learners were employed at 52.3 percent earning a quarterly wage of $5,156. Overall employment in the 4th quarter after exit is 54.5 percent.  Improvements Quarterly reports have been implemented as an additional check. Our focus on achieving measurable skill gains with managed enrollment and instruction target to EFL proved successful. We exceeded our target of 47 percent measurable skill gains with 48.8 percent.  Future Directions in PY 2022- 2023
  • We will continue to focus our attention on managed enrollment and instruction targeted to EFL. Programs are also working on using a statewide student management system to enroll and communicate with students.
Kansas Despite the lingering effects of the pandemic, which particularly impacted coordination of services with corrections and employer partners, Kansas increased performance or remained steady in multiple core indicators. Employment (Second Quarter After Exit) saw an increase from 58% in PY2020 to 60.4% in PY2021, while Employment (Fourth Quarter After Exit), at 52.9% in PY2021, was within half a percent of the 53.3% from PY2020. The increase in participants finding and retaining employment, combined with rising wages in most areas of Kansas and the increased commitment of local programs to complete follow-up surveys among exited participants, resulted in a significant increase in Median Earnings, from $5510.41 in PY2020 to $5874.08 in PY2021. For the second year in a row, the Credential Attainment core indicator was the hardest hit, decreasing from 35.5% in PY2020 to 33.3% in PY2021. The majority of exiting participants indicated employment goals were higher priority at the time than educational goals. This decrease was also partially fueled by two-year institutions in Kansas, with which Adult Education programs work closely, struggling to staff sufficient in-demand courses, along with the aforementioned challenges in collaborating with workforce partners that would support participants transitioning into and continuing in postsecondary education. Measurable Skill Gains (MSGs) remained virtually the same, with 58.8% in PY2020 and 58.6% in PY2021. With Kansas Adult Education also struggling to fill staff positions, the posttesting rate dipped very slightly (from 70.5% to 70.2%), with a correlating slight decrease in total MSGs. The overall GED® pass rate remained high at 83% in PY2021, above the national average of 76%. To help address the teacher shortage in the state, Kansas launched a pilot program in PY2021 called Cross-TREK (Cross-Teaching Remote Education in Kansas) that brings together students from multiple locations from different areas of the state with an instructor at another site. Synchronous virtual classes give these learners access to high-quality, standards-aligned instruction, while supplemental assignments are completed online on the student’s own time. The pilot served ESL students from two programs, 66.7% of whom made an educational functioning level gain of one or more levels. In PY2022, Cross-TREK added a Math class for students working on diploma completion and is continuing to offer an ESL class, which concluded in December 2022 with 82% of participants achieving EFL gains. Total enrollment in PY2021 increased by 20.6% over PY2020, from 4324 to 5213, with an accompanying increase of 20.3% in high-school-equivalency diplomas earned. English Language Learners (ELLs) contributed to the greatest growth in enrollment, up 47.9%. Adult Secondary Education (ASE) participants decreased by 12.9%, while Adult Basic Education (ABE) participants saw a modest 6.4% increase. Adult Education programs continue to reach participants with barriers, populations that need more services and support to achieve outcomes. In addition to the quarterly data analysis completed by each local program and the state, in PY2021, Kansas introduced an extra measure of performance: a quarterly evaluation of each program’s progress toward goals, which encouraged celebration of incremental successes and increased efforts to areas of greatest need. At the end of the program year, the state published “Program Portraits,” a presentation of two years of data for each program and for Kansas as a whole, available to all Adult Education providers to facilitate cross-program discussion, with program leaders sharing strategies for success and seeking advice for underperforming areas. As determined by needs assessments, outreach and recruitment strategies were a major focus of professional development in PY2021, with a statewide initiative to launch standard marketing tools begun in spring 2022. The Kansas Adult Education Association (KAEA) is the lead on the project to create and disseminate these marketing tools for all programs to use. Another major concern in Kansas is the statewide shortage of Adult Education instructors. KBOR considered methods of expanding the talent pool, such as moderating state degree requirements while developing strategies to ensure high-quality instruction is maintained, with new guidelines being implemented in PY2022. The state believes increased communication, clarity, and focus on core indicators among local programs will help drive growth in performance. Each program must annually submit a Continuation Funding Application (CFA), which is reviewed by state staff and used, along with the risk assessment, as grounds for continuing funding in the next year of the grant cycle or instituting interventions or remedies for potential noncompliance. In PY2021, KBOR met individually with each program to discuss the CFA, gain greater clarification regarding the program’s goals for the next year, and set times for interim progress checks throughout PY2022. When negotiating PY2022 performance targets with local programs, the state included the sponsoring institution’s leadership to ensure awareness and support of Adult Education goals. For PY2022, the state has added another data analysis tool: intense and detailed reviews of data between state and local staff, covering 25% of programs each quarter. These discussions provide opportunity for adjustments and more strategic deployment of resources. The state continues to provide or facilitate professional development and technical assistance as needed, including more in-depth explanations of the core indicators and training for providing short-term focused secondary diploma completion classes, among other professional development.
Kentucky Process The Office of Adult Education utilized two primary methods to analyze ongoing data trends for each local provider by service area as it relates to in program year performance indications (MSG and State-specific performance indicators: enrollment and GED Attainment). Weekly and monthly reports illustrated overall statewide performance trends in enrollment, GED attainment, and MSG rate, with comparison data from previous years. State Performance Rankings were released monthly to capture progress across the network. In addition to quarterly desktop audits, in which a sample of data entry was monitored and verified for ⅓ of fiscal agents, OAE periodically ran reports such as “Level 99”, “Participants by Age”, and others as deemed necessary. OAE identified outliers in the data set to resolve anomalies. In addition, the OAE is in the process of addressing the challenges associated with the remaining US ED performance indicators (Employment-2nd/4th Quarter after exit; Median Earnings-2nd Quarter after exit; and Credential Attainment). Success has been difficult to achieve, however, courses of action and processes are being analyzed to set the conditions for greater success in the future. Notwithstanding the effects of the pandemic, Kentucky saw a significant decrease in the employment after exist rate, the median earnings, and the credential rate. The Way Ahead is to enhance the technical assistance and monitoring process as a component of the newly established Monitoring and Compliance Attestation Tool to focus on the full spectrum of US ED performance indicators and student outcomes as opposed to the previous focus on GED Attainment exclusively. The OAE has significantly modified the approach to training and engagements with the local provider network/sub-recipients to ensure that Sections 202 and 203 of WIOA, 2014 are clearly understand in support of the service delivery across the full spectrum of the definition of adult education and family literacy.   Results OAE established state performance targets for each Fiscal Agent utilizing ACS data of individuals with less than a high school diploma. The negotiated MSG rate was 51.4%. KYAE Local Providers finished Program Year 2021 with an MSG rate of 46.67%. Fourteen of the twenty-seven Local Providers met the negotiated MSG rate. Seven programs achieved 50% of their enrollment goal, and eleven Local Providers reached 50% or higher of their GED goal. Multiple variables to include the impact of the pandemic contributed to the decreased performance across the state. OAE provided targeted support to the larger providers to reverse this trend. Below is a table representing data from the largest local provider. Performance Year 2021 concluded with Kentucky failing to achieve the negotiated rates for employment, median earnings, and credential attainment. Employment-2nd Quarter was 42.96%; Employment-4th Quarter was 29.74%; Median Earnings-2nd Quarter was $3,870.60; and Credential Attainment was 20.83%. Analysis Statewide performance increased in PY21 due to a rebound from the COVID-19 pandemic, a return to services in state and local correctional facilities and increases from Integrated Education & Training and Workplace Literacy programming. At both state and local levels, KYAE experienced a significant turnover and a need for intensive staff development with the loss of veteran educators and the hiring of 75 new instructors across the state in the program year. The impact of the pandemic was significant in Kentucky consistent with challenges experienced across the nation, however, the pandemic does not address the totality of the issues in Kentucky as it relates to service delivery and performance. A new leadership philosophy has been adopted in Kentucky that focuses on US ED requirements and success across all five performance indicators vice a mindset of exclusively emphasizing GED Attainment.  Despite the challenges presented by COVID and the onboarding of new staff, the opportunity for Online Proctored GED (OP GED) testing impacted opportunities and brought some successes. According to GED Analytics, 16.8% of all tests were online proctored, demonstrating an increase of 13.68% from the previous program year. To set the conditions for greater levels of service delivery and performance, the OAE negotiated with US DOE, OCTAE to amend the following performance goals for Program Year 2022 to the following levels: ADULT EDUCATION AND LITERACY PROGRAM PERFORMANCE INDICATORS (KENTUCKY) Performance Negotiations Table: PERFORMANCE INDICATORS PY 2022/FY 2023 Expected Level PY 2022/ FY 2023 Negotiated Level Employment Rate (Q2) 45.00% 39.2% Employment Rate (Q4) 44% 41.2% Median Earnings $4,125.00 $4,000.00 Credential Attainment Rate 48.00% 38.6% Measurable Skill Gains (MSG) 51.4% 45.2%   PY 2022/FY 2023 HSE Attainment Goal and Total Enrollment of Participants: HSE Attainment Goal 3,600 Total Enrollment of Participants 20,000 Action   OAE’s monitoring and evaluation of these goals consist of: Quarterly Performance Check-ins, Desktop Audits, and On-Site Compliance Visits. KYAE has implemented additional accountability processes by requiring the LPN to adhere to KYAE operational calendars and contingency plans. In alignment with 2 CFR 200.332 (e)(1) KYAE addressed local program performance issues immediately by focusing on the bottom ⅓ of the state rankings to provide Technical Assistance. This consisted of making site visits to those providers to identify areas of improvement and provide technical assistance. Desk Audits were conducted, and Quarterly Performance Check-Ins were completed. Providers also had the opportunity to request TA and virtual meetings were conducted with program directors to provide feedback to increase performance.  
Louisiana In terms of the overall assessment of core programs based on the core indicators of performance, WorkReady U served 20,763 (Table 2A + Table 4) undereducated and underemployed citizens in FY 2021-22 with 8,975 students participating in basic skill instruction (NRS Tables 4). Data reveals that 43% of those who participated in academic services during the FY 2021-2022 program year made a measurable skill gain (MSG) (NRS Table 4). While we continued to show reduced enrollment compared to pre-pandemic levels, we are pleased to know that enrollment ticked up in FY 21-22 and the percentage of students making a measured skill gain increased by nearly 3%. Effective instruction at the local level is evident by the fact that 60% of students post-tested (NRS Table 4B) demonstrated an EFL gain. Despite the incredible challenges to intake, assessment, and instruction posed by natural disasters and the COVID-19 pandemic, a review of the 2020-2021 adult education performance data indicates some growth areas. The percentage of students with a measurable skill gain (MSG) increased by 3% and the number of students post-tested increased by 17%. Additionally, the number of adult learners who earned an industry-based credential increased by 12% over the previous fiscal year. This shows that our providers, while significantly challenged by the obstacles of the pandemic and natural disasters, have remained committed to innovating and building educational opportunities for Louisiana’s adult learners.  Louisiana was also committed to ensuring that all individuals who were undereducated and underemployed had an opportunity to receive basic skill remediation and an opportunity toward self-sufficiency as evidenced by Table 3 which shows that approximately 99% of the students served were Adult Basic Education or English Language learners. This long-term commitment to Louisiana’s most at-risk citizens is evidence that WRU is dedicated to building a stronger Louisiana by providing comprehensive adult educational services. To improve program quality and effectiveness, WRU encouraged data sharing and discussion with faculty and staff. When daily, weekly, and monthly data reports are analyzed, specific targets and goals for improvement are uncovered. Data is used to determine necessary instructional shifts as well. These data-driven shifts enable teachers to better prepare their students. Utilizing data from the Student Information System (SIS), it is our goal to see areas where we need improvement and those in which we excel. Plans to further increase performance revolve around a set of quality indicators developed jointly with providers, state staff, and stakeholders. The indicators are aligned with federal and state considerations for funding programs, and performance on the indicators will eventually become part of the funding formula. Elements of the quality indicators framework include the proportion of low-skilled students served, the number of older adult students served, implementation of digital literacy instruction and assessment in programming, demonstration of workforce preparation and occupational skills training for learners, and an analysis of program effectiveness and efficiency in the use of funds.   
Maryland All WIOA programs are measured on several core indicators of performance.  The information that follows demonstrates the performance of Maryland’s Title II programs on these core indicators. Employment Rate/Median Earnings Maryland’s Title II employment rate 2nd quarter after exit for all Periods of Participation was 33.80% in PY21.  The Title II employment rate 4th quarter after exit for all Periods of Participation was 35.99% in PY21.  Median earnings 2nd quarter after exit for all Periods of Participation was $5,525.00.  Measurable Skill Gains Considerable changes in enrollment trends, course delivery types, and remote proctoring continued in PY21.  While MSG attainment is not yet as high as Maryland would like, PY21 saw considerable growth in this area due to increasing enrollment and the return to in-person assessment. After showing a noticeable decline in enrollment from PY19, enrollment from PY20 to PY21 showed double-digit percentage increases. ABE enrollment increased year-over-year by 20.50%, ESL by 24.18%, and overall enrollment by 27.84%.   The ESL enrollment increase was especially heartening, as it saw a significant decrease during the pandemic.  Despite the impressive one-year gains, enrollment is still below pre-pandemic levels, with a decrease of 24.46% in overall enrollment when compared to PY19 levels (Overall PY19 enrollment was 21,473).  Another bright spot in enrollment trends is the average attendance rate.  Average attendance hours per participant saw an increase this year at 91.99 hours on average, an increase of 3.33 hours per participant over PY20. In PY21, 5,808 out of 16,985 Periods of Participation saw a Measurable Skill Gain, resulting in a 34.19% MSG rate.  This was below the negotiated level of performance of 43%, but it was an increase of 13.27 percentage points over PY20 performance.  While enrollment is still lower than the pre-pandemic level in PY19, this MSG rate does exceed the 28.64% MSG rate set in PY19.  For high school diploma attainment, completion levels were on par with pre-pandemic years.  A total of 417 participants earned a secondary school diploma in PY21, compared to only 230 in PY20 and 426 in PY19.  Of the 417 completers in PY21, 78 were with the National External Diploma Program and the remaining 339 were through the GED Tests Credential Attainment The credential attainment rate for all Periods of Participation was 26.98% in PY21.  This lags the negotiated target of 37%.  However, this result is considerably higher than in previous years.  Maryland credits increased data training on the importance of this metric and how to properly capture and report it in our data system in making great improvements in this level.  With the renewal of Maryland’s license with LiteracyPro’s LACES system for a new five-year contract, training on utilizing the data system to its full potential to benefit the reporting of all metrics will continue in earnest.
Michigan The adult education programs in Michigan showed signs of rebounding from the pandemic in PY 2021. Both enrollment (1+ hour of instruction) and participant counts increased by just over 20%, but are still about three thousand below pre-pandemic levels. The number of ESL participants increased by 41%, while ABE/ASE participants increased by 13% over the year. The participant barriers data collection has improved significantly in recent years since this section was updated on the Adult Learning Plan (ALP) in PY 2020. The participant barriers section now includes questions for participants that define each barrier. Adult education providers have shared that these questions have been very helpful for both staff and participants. Since PY 2019, the number of participant barriers reported has more than doubled. In PY 2021, the most prevalent barriers for adult education participants were Low Income (47%), Long-Term Unemployed (28%), Individuals with Disabilities (20%), Ex-Offenders (17%), and Single Parent (13%). The decline in correctional education programming continues to impact statewide enrollment and performance. While the number of participants in correctional education programs increased over the year from 3,384 to 4,685, this is still significantly below the pre-pandemic level of 7,002. The Michigan Department of Corrections (MDOC) enrollment increased by 26.5%, but the MSG rate continues to be well below the statewide average at 27%. As such, the statewide MSG rate continues to be below pre-pandemic levels. Adult education providers in Michigan had outperformed the statewide MSG targets in PY 2017 and PY 2018, with more than 50% of participants reporting a measurable skill gain, but that percentage dropped drastically to 35.95% in PY 2019. For the second straight year, the MSG rate has increased slightly and was 38.22% in PY 2021. Over the year, there was a significant increase in both educational gains from pre/post-testing and high school equivalency attainments at 26% and 39%, respectively. Adult education participants in Michigan also had success obtaining employment after exit from the program. Michigan met or exceeded the target for Employment 2nd Quarter After Exit, Employment 4th Quarter After Exit, and Median Earnings. The statewide average for Credential Attainment fell shy of the target with 31% achieving that outcome measure. The percentage of participants that earned a diploma or its recognized equivalent and transitioned to postsecondary within a year was only 4%, while the percentage that attained a diploma or equivalent and are employed within one year was just over 30%. With the launch of the Futures for Frontliners and Reconnect scholarship programs in Michigan, we hope to see the percentage that continue onto postsecondary education increase.  LEO-WD is also working to implement data match with community college records to improve the collection and reporting of this data. The number of distance learners declined by 21% over the year, as most providers were able to offer instruction in-person. Similarly, the MSG rate for distance learners declined over the year and fell below the statewide average for all learners. This aligns with what the state office heard anecdotally from many directors and teachers – that their distance learning participants were not making the gains that their participants attending in-person were making. Providers noticed that distance learners were not putting in the hours needed to make gains. While most providers had intended to continue offering virtual learning options based on lessons learned during the pandemic, the latest trends were forcing more providers to put more emphasis on face-to-face instruction. To support the evaluation of these trends, MAERS now allows users to run performance reports based on a range for the percentage of distance learning hours of total attendance hours. For example, the Office of Adult Education recently did an analysis of enrollments and performance of distance learning hours in increments of 25%. The highest number of learners had between 76-100% of attendance hours from distance learning, while participants with between 1-25% of distance learning attendance hours had the highest MSG rate. Participants with between 50% or less distance learning attendance hours had a higher MSG rate than the statewide average, but those with more than 50% reported gains below the statewide average. The Office of Adult Education shared this data with local providers and encouraged them to do a similar analysis to guide program improvement going forward.
Minnesota Minnesota Adult Education Performance Results. WIOA Indicator FY 2019–2020 FY 2020–2021 FY 2021–2022 Measurable skill gain (MSG) 36.89% 23.01% 30.58% Employment at second quarter after exit 37.74% 34.25% 36.16% Employment at fourth quarter after exit 36.46% 35.26% 34.06% Median quarterly earnings at second quarter after exit $6,070 $6,188 $6,889 Credential Attainment 13.41% 19.03% 17.13% Key data points for the Minnesota Adult Education system include: 
  • Adult Education met 2 of 5 WIOA indicator targets:  Employment at fourth quarter after exit and median quarterly earnings at second quarter after exit. 
  • Outcomes increased from 2020–2021 to 2021–2022 in WIOA indicators for measurable skill gains, employment at second quarter after exit, and median earnings from 2020–2021 to 2021–2022.
  • Outcomes declined from 2020–2021 to 2021–2022 in WIOA indicators for employment for fourth quarter after exit and credential attainment. This is likely due to COVID-related disruptions since these indicators most recently reported are measured for calendar year 2020 exited participants.
COVID Impact. COVID has had an ongoing impact on adult education in Minnesota. For the calendar year 2020 exited participants, we see these changes significantly in the currently reported employment at fourth quarter and in the credential attainment WIOA outcomes. Even beyond 2020, adult education programs are shifting their services to provide more Hyflex, hybrid, and virtual options due to adult enrollee expectations and needs. Adult education programs are still experiencing lower enrollees and contact hours compared to pre-COVID. Many programs are struggling to enroll new adult students. Programs are reporting that many potential adult students have barriers to attending in-person classes regularly and want more focused programming that has defined terms and explicitly connects to diploma and career pathways. In addition, adult enrollees are not participating in the same amount of instruction as before COVID. Because of this, enrollee numbers for 2021–2022 are down 26% and contact hours are down nearly 39%, compared to 2019–2020. In response, the Adult Education Leadership Team has been working with Literacy Action Network and Literacy Minnesota to increase marketing resources for adult education programs and supporting program innovations with coaching and technical assistance for local staff, including with distance learning and adult career pathway/IET programming.  Analysis and Action. Both statewide and consortium-level performance data were reviewed and analyzed by the Adult Education Leadership Team in meetings and with providers at the Fall Adult Education Manager Meeting, SPARC (Support Professional Advisory Committee) meetings, and during quarterly meetings with state database staff.  The results are also shared with providers in the state adult education report card and help define program quality expectations in the state grant application and review process. The MSG results and targets are utilized to create benchmarks in the Minnesota Adult Education Statewide Report Card. The state report card and targets can be found online on the Minnesota Adult Education Accountability page (  The majority of adult education programs have still been proctoring NRS tests using paper forms. The Adult Education Leadership Team continued to focus on building local programs’ use of computer-based tests over the last year. There were more than 13,000 computer-based CASAS and TABE tests from the prior year that were distributed to adult education programs. An additional 9,930 computer-based CASAS and TABE tests were purchased using state adult education funding in June 2022 to distribute to adult education programs. Our state assessment trainers continue to promote and offer training on computer-based testing and remote testing. Many programs are trying computer-based testing, but the providers are facing challenges with remote testing and in using the TABE Online test vouchers before they expire.  The Adult Education Leadership Team utilizes state and local performance data to create and adapt professional development offered, especially with WebChats (quarterly webinars for providers), Adult Education Manager Meetings, database trainings, the SPARC Conference, and other trainings.
Mississippi Mississippi served 8,546 (NRS Table 2A + Table 4) undereducated and underemployed citizens in FY 2021-2022 with 5,915 students participating in basic skill instruction (NRS Tables 4). Mississippi achieved a Performance Rate (MSG) of 56.98% for program year 2021-22, exceeding the required MSG target of 50% (NRS Table 4). Effective instruction at the local level is evident by the fact that 70.23% of students who post-tested (NRS Table 4B) demonstrated an EFL gain. Mississippi OAE works with the National Strategic Planning and Analysis Research Center (NSPARC) to data match for the primary indicators of performance on Table 5.  Results for PY 2021-2022 participants show: 59.94% were employed second quarter after exit; 57.85% were employed fourth quarter after exit; the median earnings second quarter after exit is $3,736.00; 36.36% attained a secondary school diploma/equivalent and enrolled in postsecondary education or training within one year of exit; 34.29% attained a secondary school diploma/equivalent and were employed within one year of exit; 38.78% attained a postsecondary credential while enrolled or within one year of exit; and 42.66% attained any credential (deduplicated).  Mississippi has 82 counties with 19 local providers, accounting for over 300 classes, providing services to students in each county.  Adult education did experience a decline in local staff in 2021-2022 with an overall total of 285.  Mississippi still experienced the effects of COVID-19 in the 2021-22 program year with programs navigating class site closures, limited seating, student’s lack of reliable technology, while at the same time ensuring adult learners and adult educators are acclimated to the virtual environment, specifically with coursework and assessments. Furthermore, students overcoming barriers such as transportation issues, participant attendance decreases due to inconsistent schedules with their children’s needs, or simply still having a fear of exposure, brought about decreased participation. Many programs have yet to recover in student attendance compared to pre-pandemic enrollment. To continue the trend of performance achievement, local programs were provided target percentages for each educational functioning level (EFL) and analyzed findings on a monthly basis through the utilization of the Desktop Monitoring Tool. This allowed programs to identify specific areas in which to focus in a timely manner. Technical assistance has become strategically focused on ways to earn an MSG, components of the NRS data tables, and ways to improve performance.  We encourage the use of the OAE-developed Instructor Class Monitoring Tool ( which allows programs to disaggregate data at the classroom, instructor, and student level in order to compare their performance to the benchmarks and identify which staff members need additional support. Important state data which is not reflected in the NRS data is the number of work ready certifications earned through adult education.  Two of the state recognized certifications in Mississippi are the National Career Readiness Certification (NCRC) and the Smart Start Certificate and/or Credential which aligns with our State WIOA Plan.  The number of National Career Readiness Certifications (NCRC) for FY 21-22, based on the ACT Work Keys assessments, was 2,801.  Also, the state awarded over 2,120 Smart Start Credentials to adult education students supporting the State’s mission of producing individuals with the necessary skills needed to be successful in the workplace as well as life.  For FY 2021-2022, students enrolled in Smart Start, specifically those below the 8th grade level, out performed classroom peers in their educational and career goals with 61.02 percent earning a MSG, as opposed to those who were not enrolled in the course with 52.32 percent MSGs. MS OAE continues to develop reports, trainings, and resources which support local programs’ abilities to analyze performance in real-time and use data to drive instruction, retention, and transition.  
Nebraska In Program Year 2021-2022, Nebraska continued to shine through the latter part of the pandemic.  Despite ongoing challenges early on with student attendance and the availability of in-person classes, Nebraska Adult Education achieved an MSG performance of 46.32%, which once again represented the highest MSG performance ever achieved by Nebraska Adult Education. Title II was also able to exceed the negotiated performance benchmark for measurable skills gains, which is an ever-present goal. Nebraska Adult Education performed at a high level on all exit-based indicators, making fourth quartile performance in all four measures. Statewide job availability and a low unemployment rate continued during Program Year 2021-2022, which again aided positive outcomes for adult learners. Industries common in the state tend to continue operations, even during pandemic circumstances. With industries such as transportation and logistics, advanced manufacturing, meat processing and agriculture, job availability continued to be a positive.  Additional industries like retail and entry-level tech allow for additional job availability. To support the goal of placing adults in long-term, sustainable employment and postsecondary education, Nebraska Adult Education continued to prioritize student learning through a robust goal setting and career pathways process, ensuring student’s needs were being met academically, while providing information and data to support positive placement post-exit. As students identified postsecondary education as their goal, it was important to understand that post-exit employment outcomes were delayed for many adult learners achieving HSE goals.  These adult learners tended to obtain lower paying jobs while focusing their time on achieving additional educational credentials, with the overall goal of securing higher paying employment after earning their HSE credential. Nebraska’s providers continued to increase post-exit performance through improved surveying of exited participants. Since Nebraska has a high percentage of students without social security numbers, surveys were necessary to show the positive outcomes of students enrolled in Title II programs. As previously stated, Nebraska Adult Education continued to prioritize the individual learning goals of adults entering programs across the State.  Assessment was an important and vital tool used to measure the progress of adults in the academic classroom.  With the ability to delivery both instruction and assessment in-person and virtually, students were able to succeed at a higher level.  With the continued negativity towards the use of assessments in adult education by employers and non-adult education critics, it was more important than ever to prioritize language learning and high school equivalency credential preparation over short-term employment placements to demonstrate the benefits of academic assessment in adult education and better support the students seeking long-term, higher paying employment.  When looking at the goal of earning a HSE credential by most adult learners, it is vital to the economy and the future of the state that educated and literate adults are transitioned into the workforce.  Short-term workforce agendas with the goal of placing adults in the workforce was a short-term solution. The benefits of integrated education and training although certainly justified and successful for certain demographics, intensive support through individualized career pathways and educationally focused instruction that includes the goals of earn a high school equivalency credential and matriculation into community college was most necessary to achieving the end-goal of long-term sustainable employment for most adults and positive exit-based outcomes for adult education.
Nevada The Nevada Business Accountability Specialist for Adult Education tracked enrollment data and MSG performance data monthly during PY21, looking at individual program data. Local programs were informed at bi-monthly shop talk calls and programs with identified data anomalies were contacted individually to resolve errors. The enrollment for PY21 was affected by the pandemic but recovered somewhat from the first year of the pandemic. Programs have continued to increase the number of students enrolled in virtual learning, although courses were offered in-person during the 2021-2022 program year. However, Measurable Skill Gains were up with the approved target met with an overall NRS Table 4 outcome of 51.64% and a target of 43%. There were 112 participants in distance education in PY19, 808 in PY20, and 955 in PY21. The significant difference can be attributed to the fact programs continued to increase the distance education available, as well as, increasing options for hybrid, hyflex learning. Many students chose to continue distance learning even with in-person classes resuming. We saw a continued improvement with the overall MSG reaching a percentage point higher than the previous year. This improvement can largely be attributed to the fact that those post tested achieved an MSG at a rate of 72% during PY21 and 70% the previous year and the post-test rate stayed at approximately 69% with the state target at 70%. Performance outcomes were up in most educational functioning levels. Gains were down slightly in ABE levels 2, 3, and 6 and ESL levels 5 and 6, with gains in all other levels increasing. The Warning, Probation, and/or Termination Status system put in place has led to significant improvement among the programs that had been placed under Warning Status. One of the programs previously under Warning status was the highest performer for PY21. The number of participants achieving high school equivalency through the GED, or HiSET assessments continues to decrease from previous years, at 129 in PY19, 122 in PY20, and 120 in PY21. After a significant increase in the number of individuals served in IET programs from 118 in PY19 to 174 in PY20, the number served in IET or IELCE went down to 130 in PY21. Programs will continue to receive additional technical assistance in developing and expanding integrated education and training programs. There was a significant increase in the outcomes on Table 5, with employed second quarter after exit going from 25.66% to 46.1%. This increase can be attributed to being able to use the SWIS data for the first time. The fourth quarter after exit employment went from 25.7% to 38.8%. The median earnings went from $6,120 up to $8,320. This is likely due to the reduction in lower paying jobs reported following the pandemic.
New Jersey The lingering effects of the pandemic continue to be felt throughout New Jersey’s adult education programs during PY 2021. Enrollment, testing, retention, and performance were impacted by the pandemic and continue to pose challenges. The digital divide exposed challenges for participants in many ways. The obstacles were overwhelming, and students and staff encountered personal, medical, educational, and employment challenges. Fortunately, in PY 2021, for the first time in a great number of years, NJ was able to provide Title II providers with state funds for loaner device programs, technology to enhance hybrid and distance learning, and the addition of technology staff positions to programs. At the start of the pandemic massive professional development plans were put together. Instructors became fluent in Google Classroom and Schoology among other resources including Kami, Nearpod, Padlet, etc. and online software tools such as Aztec and Burlington English became the norm. Agency staff had to support learners with increased flexibility and put great effort into engaging students. Initially, distance learning and remote instruction were often the only option. Agencies utilized instructional options for participants via computers, tablets, cell phones, and worksheet packets. Videoconferencing tools such as Google Meet, Zoom, etc. were the new way to provide engaging and interactive instruction. Though the option for in-person instruction has increased greatly this year there are still many programs offering hybrid instruction or remote options as this worked very well for some students and at times is the preferred option depending on a student's other responsibilities outside of their education. The overall number of participants enrolled from PY 2020 to PY 2021 remained very close. In PY 2020, 11,709 participants were enrolled. In PY 2021, 11,714 participants were enrolled. There was a decrease in the number of distance learners as more providers moved back to in-person instruction options. The number of distance learners in New Jersey decreased from 9,420 students in PY 2020 with a total of 962,252 attendance hours to 3,579 students in PY 2021 with a total of 301,799 hours. NJ had 5,841 less distance learners and 660,452 less attendance hours in PY 2021 than in PY 2020. In PY 2021, 30% of the participants were distance learners compared with 80% in PY 2020, 12% in PY 2019 and 4% in PY 2018.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Although agencies were more familiar with remote instruction than when the pandemic hit there were still some challenges to overcome and planning and coordination to be considered; especially for programs offering hybrid instruction or both in-person and remote/hybrid options. Performance was still affected amid the uncertainty of what the program year would look like, and the difficulties participants continued to face. Providers who lost staff during the pandemic often struggled to replace positions. Some providers faced challenges because of Hurricane Ida and had to re-locate and deal with other planning difficulties due to extensive flooding. Some of the relocations were not as easily accessible by public transportation, causing a hardship for students who don’t drive. The NJ Department of Corrections serves between 1,000 and 2,000 of NJ’s adult basic education students each year and in PY 2021 there were several changes made throughout NJ’s correctional system that affected performance. Southern State Correctional facility and Mountainview Correctional facility closed between June and December. Southern State Correctional facility housed 800 inmates, 400 of whom were moved to Bay Side Correctional facility and would not be able to be post tested. In addition, 4,500 inmates were released a year early because of COVID without post testing. Prison laws will continue to allow the release of inmates periodically. NJ DOC staff indicated that they foresee a shift in their Measurable Skill Gains (MSGs) performance compared with pre-COVID performance but are working diligently to develop new ways to ensure increased performance. During the pandemic NJ allowed providers to use provisional assessments but that allowance was removed for PY 2021.  All agencies are pre and post testing with NRS approved assessments. In PY 2021, NJ met its overall MSG target of 54%; 6 of 12 Educational Functioning Level (EFL) MSG targets were met. The targets that were not met were all ABE/ASE levels. NJ exceeded reaching its ESL targets by between 1% and 10% and missed meeting its ABE targets by between 5% and 14%. NJ is very proud to report that despite continued challenges, that our providers have met NJ’s overall MSG target through so much dedication, persistence, and hard work and even though NJ did not meet individual PY 2021 ABE/ASE targets there was significant improvement from last year. NJ didn’t meet any of the ABE/ASE targets last year either but the percentage ranges that the targets were missed by last year, in PY 2020, ranged from 13% to 24%, whereas the range for this year, PY 2021, was 5% to 14%. From PY 2020 to PY 2021 NJ demonstrated substantial increases in almost all of the ABE/ASE and ESL EFLs. The increases were greatest for ABE/ASE students. ABE Level 1 increased from 28% to 42%; ABE Level 2 increased from 31% to 39%; ABE Level 3 increased from 29% to 39%; ABE Level 4 increased from 34% to 43%; ABE Level 5 increased from 33% to 45% and ABE Level 6 increased from 31% to 42%. From PY 2020 to PY 2021 the ABE/ASE level increases ranged from 8% to 14%. These increases resulted in an average gain of 11% for ABE instruction which is up from a 4% increase from PY 2020. For PY 2021, ESL increases were obtained in Levels 1, 2, 4, and 5 and ranged from 2% to 9%. Although there no increases in ESL Levels 3 and 6, the decrease was just 2%. Despite the difficulties and uncertainties that remained during PY 2021, there was an 8% increase in the number of participants post tested compared to the previous year, which is very positive in light of some of the NJDOL and Hurricane Ida issues that arose. Prior to the pandemic in PY 2018, New Jersey’s adult education agencies post tested 70% of their students. In PY 2019 they were only able to posttest 54% of their students and in PY 2020 they were able to posttest 58% of their students. This year PY 2021, 66% of NJ’s participants were post tested meeting the required posttest rate of 60%. A comparison of New Jersey’s performance targets for PY 2019, 2020, and 2021 and actual performance toward meeting the targets is provided in Chart 1 below. There is a clear trend of improved performance percentages as NJ made its way through the pandemic and it is expected that NJ’s performance will continue to increase.     Chart 1:  New Jersey Adult Education Performance for the last three years   Performance Objectives PY19 Target PY19 Performance PY20 Target PY20 Performance PY21 Target PY21 Performance ABE Level 1 51% 20% 51% 28% 52% 42% ABE Level 2 49% 26% 50% 31% 51% 39% ABE Level 3 46% 29% 50% 29% 51% 39% ABE Level 4 43% 32% 47% 34% 48% 43% ABE Level 5 56% 29% 57% 33%  59% 45% ABE Level 6 45% 26% 52% 31% 53% 42% ESL Level 1 57% 54% 57% 55% 58% 64% ESL Level 2 56% 57% 56% 63% 57% 65% ESL Level 3 59% 52% 57% 63% 58% 61% ESL Level 4 57% 50% 57% 54% 58% 59% ESL Level 5 45% 44% 45% 53% 46% 56% ESL Level 6 38% 33% 40% 46% 41% 44% Overall MSG baseline 59% 53% 47% 54% 54% Employed Q2… baseline 44% 45% 38% 46% 36% Employed Q4… baseline 46% 43% 39% 44% 39% Med Earn Q2… baseline $5,930 $4,783 $6,370 $4,833 $6,747 Attained SSD & PSE/Training… baseline 3% 2% 4% 3% 3% Attained SSD & Employed… baseline 22% 18% 25% 19% 14% Attained PS Credential… baseline 40% 58% 41% 59% 25% Attained any Credential… baseline 31% 27% 34% 28% 19%   This chart shows data over the past three years on primary indicators of performance and; MSGs in the areas of: Adult Basic Education (ABE) and English Language Acquisition (ELA).  Chart 1 shows improvement in almost all EFLs, median earnings and overall MSGs, which went from 47% in PY 2020 to 54% in PY 2021. For years prior to the COVID-19 pandemic, New Jersey demonstrated consistent and significant improvement in all of the Measurable Skill Gains (MSG)/Educational Functioning Level (EFL) gain areas. For PY 2017 and PY 2018 New Jersey met all of its performance targets and was on track to meet their performance goals for PY 2019 when COVID-19 shocked all of New Jersey but especially our adult education programs. There was great improvement from PY 2019 to PY 2020 especially in regard to English Language Learners (ELLs) but there is even greater improvement from PY 2020 to PY 2021 in regard to MSGs in virtually all EFLs. During PY 2021, educational services and instruction to adults were provided through a variety of program offerings including:  ABE/ASE, ELA, volunteer literacy programs, adult high school programs, High School Equivalency (HSE) test preparation, and programs at state and local correctional facilities. Despite the pandemic, agencies continued providing adult education services consistently through the program year. The provision of supportive services and information outside of the direct realm of adult education became increasingly necessary and agency staff did an amazing job of supporting and helping participants in many ways. The ABE/ELA and Integrated English Literacy and Civics Education (IELCE) program services were delivered through 18 grantees with multiple delivery sites throughout the counties coordinated by either a lead agency heading a consortium or a single entity. Agencies included community/technical colleges or public and private non-profit agencies and New Jersey State correctional institutions.  Together, 76 agencies participated in delivering services to 11,714 participants in PY 2021. Chart 2 provides a breakdown of participants by EFL for PY 2020 and PY 2021 demonstrating the enrollment difference by levels between the years. In PY 2021, New Jersey’s WIOA Title II agencies served 11,714 adult participants. In PY 2020 they served 11,709 adult participants. The populations served for PY 2021 included 4,110 ABE participants (3,439 ABE Levels 1 - 4 participants and 671 ABE Levels 5 and 6) and 7,604 ELA participants.   Chart 2:  New Jersey Adult Education Enrolled Participants   Educational Functioning Level PY 2020 Total Enrollment PY 2021 Total Enrollment Enrollment Difference between PY20 and PY21 ABE Level 1 581 476 105 less students ABE Level 2 713 657 56 less students ABE Level 3 1,121 1,093 28 less students ABE Level 4 1,276 1,213 63 less students ABE Level 5 498 477 21 less students ABE Level 6 258 194 64 less students Total ABE 4,447 4,110 337 less students ESL Level 1 2,124 2,484 360 more students ESL Level 2 1,662 1,797 135 more students ESL Level 3 1,116 1,161 45 more students ESL Level 4 1,025 960 65 less students ESL Level 5 921 819 102 less students ESL Level 6 414 383 31 less students Total ESL 7,262 7,604 342 more students Total ABE and ESL 11,709 11,714 5 more students New Jersey’s ELA programs are offered at different venues from literacy-based volunteer programs to adult education programs.  The ELA population represented approximately 65% of all the students being served in the WIOA Title II program. This specific population may have limited reading or writing skills in any language and may function minimally or not at all in English. For the volunteer-based programs, most services are provided through tutor-learner matches that deliver one-on-one or small-group instructional services. Most of the ELA students served (33%) were assessed at the ESL Level 1 EFL. This information is reflected in chart 3. This population may only understand isolated words or phrases, may have no or minimal reading or writing skills in any language, and functions minimally or not at all in English.   Chart 3:  New Jersey Adult Basic Education English Language Learners   Educational Functioning Level Total Number of ELLs  PY 2021 Total Percentage of ELLs  PY 2021 ESL Level 1 2,484 33% ESL Level 2 1,797 24% ESL Level 3 1,161 15% ESL Level 4 960 13% ESL Level 5 819 11% ESL Level 6 383 5% Total 7,604 100% From PY 2020 to PY 2021 our programs demonstrated an increase in the overall ESL MSG from 57% to 61%. There were increases in educational gains in ESL EFLs 1, 2, and 5. There was a 9% increase at Level 1, a 2% increase at Level 2 and a 3% increase at Level 5. Of the 11,714 students served in the WIOA Title II program for PY 2021, 7,604 of them were English Language Learners (ELLs). New Jersey met all the ESL EFL gain targets for PY 2021. For PY 2020, NJ only met 4 of the ESL EFL targets.   New Jersey embraces its cultural diversity. New Jersey’s diverse makeup is consistent with its status as a gateway for immigrants. Agencies were required to integrate an English language/civics education component for their Section 243 funded IELCE programs. They are required to design programs to prepare ELLs for employment in in-demand industries and occupations that lead to economic self-sufficiency and integrate with the workforce development system. The IELCE programs support the immigrant community and other limited English proficient populations. During PY 2021, 2,070 participants were served through the IELCE program and almost 200 more students passed their citizenship exam compared with PY 2020. Two hundred and eighty-three of our ELLs passes their citizenship exam between July 1, 2021 and June 30, 2022, compared with just eighty-five students in PY 2020. The IELCE program emphasizes contextualized instruction on the rights and responsibilities of citizenship, naturalization procedures, civic participation, and U.S. history and government, in accordance with the National Standards for Civics and Government. Types of activities supported by these funds included:
  • provision of experiential learning opportunities including completing employment forms, completing the voter registration process, engaging participants in volunteer opportunities and linkages with community resources;
  • provision of computers and software for instruction;
  • citizen preparation including visiting of polling places and government buildings in the state capital;
  • field trips to promote American cultural awareness and skills; and
  • preparation of curricula focusing on issues of importance to non-citizens such as preparation for becoming a U.S. citizen, job training and placement, housing, parenting and civics preparation.
ELLs enrolled in programs that received IELCE Section 243 funding were offered the opportunity to participate in Integrated Education and Training Programs (IETs). Twenty-eight percent of ELLs served with Section 243 IELCE funds participated in an IET, which was 5% higher than PY 2020. Implementation of the IETs for PY 2021 continued to prove difficult because of lingering pandemic circumstances and challenges. Some of the hands-on components of established IET programs were still not possible. In addition, some of the agencies affected by Hurricane Ida had to postpone their IET classes for safety reasons and inaccessibility to their computer labs. Agencies tried to reach out to libraries to borrow computer lab space but they were not allowing group gathering at the time. The NJ DOL made a variety of investments to help individuals obtain their high school diploma or its recognized equivalent, improve their basic math, reading and writing skills, and attain basic English listening and speaking skills along with computer proficiencies necessary for employment including the purchase of the Aztec and Burlington English online software as an in-class instruction tool and for supplemental learning outside the classroom. The NJ DOL utilizes state funding to enhance adult education alternatives directly related to welfare registrants, dislocated workers and other jobseekers. Educational opportunities were provided to adults via programs ranging from ABE Level 1 through ABE Level 6 and ESL Level 1 through ESL Level 6 to prepare adults to:
  • enroll and participate in the appropriate level of an adult education program and receive ancillary support services if necessary;
  • move to the next EFL;
  • sit for an HSE exam and, if successful, receive a New Jersey state issued high school diploma; or graduate from an adult high school;
  • obtain gainful employment;
  • enter postsecondary education or vocational training programs;
  • attain an industry-recognized credential(s).
There were 23 Workforce Learning Link (WLL) programs throughout the state for PY 2021 compared to 24 from PY 2020.  Most of these programs are located at the One-Stop Career Centers (OSCCs). Through a combined training approach and computer-based instruction, the WLLs provide services that include basic computer literacy including typing, financial literacy, job search and employability skills, ELA instruction, and HSE preparation and testing.  This effort has allowed New Jersey to expand its literacy services to provide interactive training services that allow participants to address individual employment-related issues at their own pace. The WLLs are able to assist individuals to raise their basic skill levels, obtain employment, enter occupational training, and attain their high school diplomas by passing their HSE exams using computer labs in OSCCs throughout the state. For PY 2021, the state’s 23 WLL facilities helped approximately 1,221 individuals which was an increase of 166 compared to PY 2020. WLLs had actively offered virtual/online literacy services and training to their participants to continue to provide learning opportunities during these uncertain times.  
New Mexico Enrollment and its dynamic serve as a good indicator of overall health of the program. New Mexico’s NRS enrollment trends for the past four years are highlighted in the table below. Program Year Beginning Literacy Adult Education Adult Secondary Education ESL Total Enrollment PY 18/19 2632 3813 361 4,112 10,918 PY 19/20 2275 3462 230 3,527 9,494 PY 20/21 1367 2264 255 1,424 5,310 PY 21/22 1410 2375 259 2,010 6,054 The enrollment in PY 21/22 started showing signs of post-pandemic recovery. While the enrollment fell over 44% in PY 20/21 from PY 19/20, it increased by over 14% in PY 21/22. The largest increase was in ESL enrollment that showed growth of over 41%. While numbers have not reached the pre-pandemic levels just yet, a gradual recovery in numbers of participants is expected. Different local programs showed different performance in enrollment numbers, with a spread between 37% decrease to 160% increase in enrollment. New Mexico is hoping to use face-to-face instruction alongside the online learning experience gained through the pandemic to increase the enrollment by more than 14% in PY 22/23 and get closer to pre-pandemic levels. Another important performance indicator to examine is measurable skill gains. Four years of the aggregate data is presented in the following table. Measurable Skill Gains Program Year ABE Level 1 ABE Level 2 ABE Level 3 ABE Level 4 ABE Level 5 ABE Level 6 ESL Total Enrollment PY 18/19 37.1% 37.4% 33.3% 33.9% 42.2% 37.8% 36.4% 35.7% PY 19/20 37.9% 32.2% 28.9% 31.7% 30.2% 16.7% 29.4% 30.3% PY 20/21 40.3% 31.8% 28.3% 33.3% 27.1% 29.4% 31.1% 31% PY 21/22 37.5% 36.4% 37.1% 42.8% 31.9% 40.3% 33.9% 36.5% Overall % MSGs have improved significantly in PY21/22 - by more than 5% - and have now surpassed pre-pandemic levels. The same trend can be seen in almost all MSGs levels. The largest increase is seen in ABE Level 6. While the MSGs were either decreasing or staying steady in PY 18/19 – PY 20/21, this is a first increase seen in recent years. It can be attributed to program recovery from the pandemic as well as improved data procedures. Post-exit indicators also serve an important role in overall assessment of program performance and are presented in the next table. Indicator of Performance PY 20/21 PY 21/22 Target for PY 21/22 Difference of Targets to PY 21/22 Performance Employment 2d Quarter after Exit 22.0% 37.2% 25.0% +12.2% Employment 4th Quarter after Exit 10.6% 23.5% 35.3% -11.8% Median Earnings Employment 2d quarter after Exit 3,730.30 4,219.96 3,220.00 +999.96 Credential rate 26.86% 22.8% 37.1% -14.3% Most of the core performance indicators have increased in PY 21/22 compared to PY 20/21. The only decrease is seen in Credential attainment rate. This decrease can be attributed to the remaining effects of the pandemic since this indicator lags a year behind the reporting year. The decrease can also be attributed to difficulties with our post-secondary match. Employment 4th Quarter after-exit and Credential rate are still below the negotiated targets. Improvements in gathering after-exit information are planned for next year. Initiatives to encourage credential attainment rate are also in process during PY 22/23.
North Carolina For PY 2021-22, the COVID pandemic continued to have an adverse impact on North Carolina's MSG rates and negatively impacted the enrollment numbers of participants throughout the state. Pre-COVID, the State of North Carolina’s participant enrollment was 41,261 (PY 2019-20). During PY 2021-22, participant enrollment was 37,742. However, this is a significant increase over PY 2020-21 where the state had only 31,309 participants. Participant enrollment increased 21% for PY 2021-22 which is most encouraging for our state, providers, and students. Pre-COVID, the State of North Carolina saw continuous improvement with measurable skill gains (MSGs): PY 2016-17: 35.48%; PY 2017-18: 39.89%; PY 2018-19: 44.88%. During the pandemic the state’s MSG rates dropped significantly: PY 2019-20: 33.9% and PY 2020-21: 36.8%. North Carolina’s MSG target for PY 2021-22 was 43.6% and the state reached a final MSG of 43% which exceeds PY 2021-22 and builds confidence that the state will exceed the current PY 2022-23 MSG target of 43.6% by June 30, 2023.   ABE/ASE    Our aggregate Adult Basic Education (ABE) and Adult Secondary Education (ASE) levels overall fell below the 43.6% target for PY 2021-22 with a performance of 38% and 42% respectively. All ABE levels except for ABE Level 6 (50.3%) fell below their targets of 43.6%. Most MSGs were achieved by pre and post-tests, followed by high school equivalency diplomas and post-secondary enrollment.      ESL    Our aggregate English as a Second Language (ESL) levels well exceeded the state’s target of 43.6% for PY 2021-22 with a performance of 49.5%. ESL levels 1 through 5 surpassed the 43.6% target rate (51.7%, 57.9%, 55%, 48.2%, 47.8% respectively). Our only shortfall occurred at ESL level 6 with a performance rate of 38.8%. Most MSGs were achieved by pre and post-test EFL gains, followed by enrollment into post-secondary education after exit.       Combined Program Performance    Factoring in periods of participation (POP) for PY 2021-22, our overall MSGs were 43% which was a 5% point increase over the prior program year’s 38% performance but fell just shy of our 43.6% target. The state’s participant population for ABE/ASE was approximately 61% and ESL accounted for 39% of our participants. The number of participants in all ABE/ASE levels increased from the prior program year by 1,716 participants. Across all ESL levels, participants increased by 4,717 over PY 2020-21.   Over the past several program years, North Carolina has worked to customize and enhance our student information system database to include views of all NRS tables and to make data more easily accessible for programs. Part of this programming included the creation of additional tools and reports to aid providers in proactively monitoring and understanding their data and performance. The state provides monthly feedback, routine training sessions and open office hours throughout the year with a focus on data quality and performance improvement. The state has developed Power BI data dashboards, which allow providers to pull data to create specific data reports for programs to analyze to help identify potential problems and or to resolve problem. The Power BI data dashboards were developed to extend the functionality of the North Carolina Community College System’s (NCCCS) Performance Dashboards presented through Tableau. Within the Power BI dashboards providers can use the detailed student-level views of data that are submitted monthly to the NCCCS Data Warehouse to monitor their programs performance and outcome measures. Moreover, the State Office utilizes their statewide view of the Power BI dashboards to monitor (monthly) state performance, as well as monitoring individual providers for potential technical assistance needs throughout the program year. We are currently in the implementation phase for the new Adult Education and Literacy state MIS, ADVANSYS by Benchmark. For PY 2021-22 the state’s 58 community colleges were in the beginning stages of transitioning to the new MIS system. (The state’s 10 community-based organization, Title II providers successfully transitioned to ADVANSYS in December of 2020. The new MIS system will incorporate NRS measures using common definitions and categories. NRS tables will calculate accurately to include error checks and prevent double counting with the new state MIS. For PY 2021-22, the State Office provided biweekly ADVANSYS sprints through the Agile method of project development and management. Weekly cohort meetings for colleges first identified to convert to ADVANSYS were held for the second half of the program year and will continue through PY 2022-23 as more cohorts are onboarded. Moreover, the State Office ADVANSYS team, comprised of state staff representatives from Adult Education & Literacy, Business Systems Analysts, Research and Performance Management and Information Technology meet daily with the vendor to ensure accuracy in reporting and data management as the product is customized and finalized to meet the needs to North Carolina.  During PY 2021-22, to address areas of weak performance, state staff members conducted webinars and training sessions focused on data, performance, and instruction. Specific technical assistance was offered to individual programs who were not meeting data reporting deadlines and or performance outcomes. The State Office also continued the services of the Center of Excellence for Adult Education (CEAEP) to provide professional development and technical assistance projects to providers on improving their performance. Some of the professional development and technical assistance provided focused on performance and accountability, post-exit indicators, conquering their data challenges, and tools and strategies to increase their overall MSG rates. During the 2021-22 program year, state staff provided monthly virtual statewide meetings with providers to provide important guidance/policy updates and resources for programs regarding the development of data performance dashboards and Power BI data views.
Northern Mariana Islands Despite enrollment decreasing yet again this program year 2021-2022, we were still able to Improve the percentage of Periods of Participation (PoP) with Measurable Skills Gains (MSG) from 52.73% in PY 2020-2021 to 61.18% this year.  We worked hard in preparing those who were getting ready to achieve their goal to graduate. Our office graduated 30 students. Face-to-face courses were converted to online instruction.  It was a great challenge for all of us who were not used to this delivery mode.  We opened up all lines of communication with our students.  They were able to contact us through personal phone numbers, emails, social media messages, and WhatsApp.  They could contact most of us anytime during the week, including evenings and weekends.  Some students who were not able to do online courses were treated as independent study students and did assignments or projects through emails or through packets.  We tried to accommodate students in different ways.  We weren’t able to increase our enrollment, but we also didn’t lose too many of our students. Students were able to continue to come on campus to take their assessments.  Individual student profile reports were given to instructors to incorporate into their online lesson plans to ensure weak skills, as well as strengths, were addressed and reinforced.  Northern Marianas College’s General Education Learning Outcomes (GELOs), CASAS competencies, Adult Education CCRS, and Student Learning Outcomes (SLO) were all mapped and laid out in the program course syllabi.  Students entered the online class knowing what they were going to learn as well as the expectations from the instructor. Our intake process continues to be improved to meet student needs. Educational advising sessions were scheduled often to track progress towards goals and individual concerns were also addressed during these sessions.  We’ve found that this particular practice has had a positive effect and student retention improved overall with them posting enough hours to posttest and continuing until they fulfill their goals. We also created a separate email for potential students to send inquiries.  This helped us track the different types of requests that came in.   The office continues to communicate with the institution and a research associate from REL (Regional Educational Laboratory) for the pacific region at McREL International (a non-profit, non-partisan education research and development organization) to assist in the development of an IET program that will work and be sustainable as well as meet the needs for our community.  We plan to continue working on developing an IET program and hopefully launch it in the near future.
Ohio Highlights from the Achievements of the Core Indicators of Performance
  • In PY 21 had a 44.29% overall MSG.  As a state, we are not satisfied with Ohio’s performance.  We have had more ways for students to earn an MSG.  One piece that is different is that as a TABE only state, we are seeing less gains through an EFL gain, and overall, less eligible students receiving a progress test. 
  • Student persistence remained relatively the same at an average of 56.56 hours per student PY 21.
  • Ohio saw an increase in the number of IET students (+821) in PY 21.  
  • PY 21:  Employment rates 2nd quarter after exit:  62.36%. 
  • PY 21:  Employment rates 4th quarter after exit: 29.77%. 
  • PY 21:  Median earnings 2nd quarter after exit:  $1,789.00 
  • PY 21:  Credential attainment:  24.59%.  At the end of PY 21, our state office learned that in order for students to receive credit for the credential in LACEs, that their class level must be marked at a 9th grade level or higher.  That had not been the case in previous years.  The state office provided TA and training to address this at the end of PY 21. 
Data was reported in the NRS portal and each of the local grantees were required to enter local program data into LACES by the 10th of the month.  Program managers provided TA and data assistance throughout PY 21 to identify errors and resolve moving forward.  LACEs also provided TA and a series toward the end of PY 21 on data clean-up.  In comparison to PY 20 (SFY21), the MSG rate was a little higher.  The overall MSG was 42% in comparison to 44% in PY21.   In PY 21, Ohio saw an increase in the employment rate 2nd quarter after exit and a decrease in employment rates 4th quarter after exit.  The PY 21 median earnings were substantially less than in PY 20, but as a state we need to make sure that we are utilizing the best methods to record a more accurate $ amount.   The credential attainment rate while enrolled or within one year of exit increased from PY 20.  Areas for Improvement and Action Steps Needed
  • The focus for improvement includes MSG and employment rates. In PY 21, Ohio was a TABE only state.  Though there were new methods to earn an MSG through workplace education and through passing a technical exam in an IET, our team believes that having only one approved assessment may have impacted the EFL increase.  In addition, data demonstrated that our programs were not as successful with progress testing, and we saw many students who exited the program before they received an assessment.  Looking at NRS TABLE 4B helps our team to see that many students who receive a progress t test do achieve an MSG.  The issue is that not enough students are being progress tested.  Weekly reminders are sent out, reminding programs to look at the dashboard in LACES to see which students are eligible to progress test.  Ohio Aspire hopes to see an increase in performance outcomes by taking proactive measures now including TA and targeted progress testing communication in PY 22.
  • Increase the amount of progress toward milestone MSGs earned and increase the amount of MSG earned by passing the occupational exam for IET participants.  Ohio is looking to expand our progress toward milestone measure for FY 23 and 24.     
  • After a decrease in employment rates in 4th quarter after exit, Ohio needs to increase partnerships with local employers to help students transition into employment after exit.  Aspire continues to be a part of the Workforce Stakeholder Meetings with other WIOA Core partnerships, hopefully allowing for more opportunities to learn about employment opportunities and initiatives to help increase employment.  Continued improvement needs to occur with local program student follow-up for self-reporting of employment. 
Oklahoma Program Year 2021 - 2022
  • Total Served: 8,296
  • Achievement of at least one educational functioning level: 2,407
  • Achieved Measurable Skill Gain (MSG): 43%
Oklahoma saw an improvement in performance from FY 2021 to FY 2022. There was an increase in the overall number of students served, in those who advanced an educational functional level, and those who achieved a MSG. Overall Measurable Skill Gain (MSG) Performance Fiscal Year                                          Percentage   16-17                                                    39%   17-18                                                    41%   18-19                                                    40%   19-20                                                    41%   20-21                                                    40%   21-22                                                    43%   Over the past five years, Oklahoma’s AEFL state program has met or exceeded the negotiated rate for measurable skill gain performance. As seen in the graph above, Oklahoma’s largest percentage increase from 40% to 43% occurred during fiscal year 21-22. One aspect that might contribute to this increase in gains is an increase in attendance of students served across the state, which could likely be attributed to declines in COVID cases. “My program attendance has increased since COVID cases have declined,” stated Drumright’s AEFL program director. Oklahoma Department of Mental Health and Substance Abuse within the court system provides eligible, non-violent, felony offenders the opportunity to participate in a highly structured, court-supervised drug treatment program. Local AEFL programs have been successful in developing partnerships with local drug courts. Judges have held students accountable for attendance. This attendance rate increase has affected the number of students that are reaching a measurable skill gain. Overall English as a Second Language (ESL) MSG Performance Fiscal Year                                          Percentage   16-17                                                    31%   17-18                                                    27%   18-19                                                    26%   19-20                                                    28%   20-21                                                    42%   21-22                                                    44%   Oklahoma’s ESL performance continued to see ESL MSG improvement in 21-22 with an MSG of 44%. The state’s ESL student population has changed over the last year. Oklahoma experienced a major influx of ESL students during the 21-22 program year. According to State Department data obtained by Axios, Oklahoma received the third largest number of Afghanistan refugees that were seeking asylum in the United States (2022). Most of these students did not speak English upon their arrival. Unlike other ESL students, this population did not have family members to rely on for communication. This increased student enrollment, attendance, and student drive to succeed. Overall Adult Education (AE) MSG Performance Fiscal Year                                          Percentage   16-17                                                    43%   17-18                                                    48%   18-19                                                    45%   19-20                                                    46%   20-21                                                    40%   21-22                                                    42%   As expected, the adult education population of the state experienced a decline in performance in fiscal year 20-21 due to COVID. The following year, in 21-22, with declines in COVID cases, our adult education population saw an improvement in MSG performance to 42%. Even with increases in performance, our programs experienced many challenges. One program dealt with mold issues, which led to the discovery of asbestos. This program was forced to move classes to a high-traffic computer lab for the first six months of the school year. Conditions were not ideal, but they continued to serve their population and make performance gains. Overall High School Equivalency (HSE) Diploma Attainment Performance Fiscal Year                                          Number of Attainments   16-17                                                   1,526   17-18                                                   1,391   18-19                                                   1,171   19-20                                                   1,128   20-21                                                      702   21-22                                                      927   During the 20-21 fiscal year, testing centers across the state closed their doors due to pandemic-related restrictions. This created a situation where jobs were more plentiful, and employers were more accepting of workers without high school equivalencies. Oklahoma also had a new industry develop in the last few years with the legalization of medical marijuana. This new industry does not require a high school diploma and according to The Journal Record employees more than 16,000 people which is more than those employed in the construction trade. During FY 21-22, Oklahoma saw an increase in overall HSE attainments from 702 to 927. This increase is likely because testing centers re-opened their doors for assessment opportunities and fewer jobs were available without the requirement of a high school diploma. Overall Correctional / Institutional Education Performance Fiscal Year                                          Percentage   16-17                                                   57%   17-18                                                   65%   18-19                                                   58%   19-20                                                   61%   20-21                                                   50%   21-22                                                   57%   Oklahoma’s corrections/institutional MSG increased during the 2021-2022 fiscal year from 50% to 57%. Classes in these facilities shut down the previous year and this led to a drop in performance. This also included TABE and HSE testing. During the pandemic, students did not have access to technology. They had to receive instruction using packets and all packets had to be disinfected. This created delays and reduced the learning environment. In 2021-2022 students were given the opportunity to attend class again. Receiving instruction in person without delays has led to a positive impact on performance. Overall Employment 2nd Quarter Fiscal Year                                          Percent of Individuals Employed   16-17                                                   N/A   17-18                                                   N/A   18-19                                                   N/A   19-20                                                   35%   20-21                                                   34%   21-22                                                   12%   Overall Employment 4th Quarter Fiscal Year                                          Percent of Individuals Employed   16-17                                                   N/A   17-18                                                   N/A   18-19                                                   N/A   19-20                                                   30%   20-21                                                   36%   21-22                                                   15%   Overall Median Earnings Fiscal Year                                          Earnings   16-17                                                    N/A   17-18                                                    N/A   18-19                                                    N/A   19-20                                                    $3,769.50   20-21                                                    $3,846.50   21-22                                                    $5,563.50   Prior to FY19-20, Oklahoma was unable to collect full employment data. This explains the significant increase in employment data and wage earnings beginning FY19-20. Median earnings have steadily increased while employment percentages for 2nd and 4th quarters have decreased. The decrease in employment rates are likely due to the after-effects of the Covid related pandemic. Overall Credentials Fiscal Year                                          Percentage   16-17                                                    N/A   17-18                                                    5%                      18-19                                                    10%   19-20                                                    19%   20-21                                                    24%   21-22                                                    19%   As seen in the chart above, Oklahoma has seen a steady increase in credential rates over the last few years. Rates have increased from 5% in FY17-18 and peaked at 24% in FY20-21. In FY21-22, it is believed that the state credential rates declined due to the pandemic. It is expected that Oklahoma will see an increase in credential attainment this current fiscal year as more students are returning to the classroom. Data Quality The state of Oklahoma ensures data quality through various methods. As in previous years, Oklahoma continues to use data match, data sharing agreements, LACES training, onsite and online monitoring, technical assistance, and support. To improve data quality, ODCTE’s data specialist began providing a bimonthly virtual training called LACES and Coffee. The one-hour sessions were designed to present best practices and data collection updates. Time was also set aside for questions and assistance relating to data collection. Topics included: how to change views for reports, how to analyze and use Table Four in LACES for program improvement, FY21-22 closeout, and student intake. LACES & Coffee has increased dialogue between program data personnel and state staff. Data entry and accuracy are expected to improve due to this valuable training. To assist with the accuracy of data collection, state staff developed a barriers to employment checklist and a WIOA core programs service list. Programs have indicated that these two documents have increased students’ acknowledgment of barriers and WIOA services received.
Oregon Educational Functioning Level 2020-2021 Actual Performance 2021-2022 Target 2021-2022 Actual Performance Change from 2020-2021 Actual Performance ABE Level 1 10% 51.5% 24.8% +14.8% ABE Level 2 30% 54.5% 41.1% +11.1% ABE Level 3 20% 48.5% 39.0% +19.0% ABE Level 4 19% 44.0% 32.0% +13.0% ABE Level 5 22% 55.0% 38.7% +16.7% ASE Level 6 33% 45.0% 48.5%* +15.5% ABE/ASE Total 22%   36.7% +14.7%         ESL Level 1 2% 47.5% 13.6% +11.6% ESL Level 2 3% 54.5% 17.6% +14.6% ESL Level 3 5% 53.5% 27.3% +22.3% ESL Level 4 7% 49.5% 26.0% +19.0% ESL Level 5 3% 43.5% 24.9% +21.9% ESL Level 6 3% 23.5% 13.6% +10.6% ESL Total 4%   24.5% +20.5% *Met or exceeded federal target   Overall, performance in ABE/ASE measurable skill gains increased by 14.7%  and performance in all six ABE levels was significantly greater in 2021-2022 than in 2020-21. Additionally, Oregon exceeded the MSG federal target in ABE Level 6 by 3.5%. The next highest performing EFL was ABE Level 2 (federal target missed by 13.4%) and analysis indicates that only 55 additional students achieving a measurable skill gain in ABE Level 2 were needed to meet the federal target. Similar analysis for ABE Level 3 (federal target missed by 9.5%) indicates that 93 additional students achieving MSG were needed to meet federal targets.  Performance in ESL measurable skill gains increased overall by 20.5% in 2021-22. While federal targets were not met, performance in all six ESL levels was significantly greater in 2021-22 than in 2020-21, with the greatest increase occurring in ESL Level 3 with an increase of 22.3% year over year. Analysis indicates that while the federal target was missed by 26.2%, an additional 196 participants achieving a measurable skill gain were needed to meet the target. Similar analysis for ESL Level 6 (federal target missed by 9.9%) indicates that 96 additional participants achieving MSG were needed to meet federal targets.  In response to local program director feedback (as highlighted in the 2020-21 Federal Narrative Report), Oregon contracted with a local program to establish a statewide remote proctoring service. Available to any local ABS program in Oregon, the statewide remote proctoring service addressed barriers that local programs had with pre- and post-testing, including not having the staffing capacity to conduct widespread remote testing. As a result, pre- and post-testing pairs and assessment rates have increased. Due to the success of the initial year of statewide remote proctoring, the contract was renewed and online proctoring continues for PY2022-23. The contracted program anticipates doing additional analysis of this practice, to determine whether continuation would be in the best interest of the programs and participants. During the 2021-22 program year, the State ABS Team intentionally emphasized technical support with more frequent, consistent communication with directors and interaction with programs. The State ABS Team focused on how best to support local programs as the pandemic and its effects continued. Some activities included but were not limited to: 
  • Hosting multiple Oregon Adult College and Career Readiness Standards (OACCRS) trainings including Orientation to OACCRS, OACCRS Math trainings, and OACCRS Language Arts trainings to support instructors in learning about national and Oregon-specific aspects of the state learning standards;
  • Supporting Teaching Skills That Matter (TSTM) trained teachers in rolling out TSTM activities locally and statewide; and 
  • Fielding a team for participation in a Standards In Action training initiative.
Title II providers were required to:
  • Report on past effectiveness in performance indicators. 
  • Provide documentation of achievements and activities that demonstrated capacity and expertise in delivering adult education and literacy activities. 
  • Commit to local MSG performance targets for 2022-2023. Providers were required to justify any targets that were below state targets.  
During the 2021-22 program year, Oregon emphasized performance through several activities: 
  • Publishing an annual ABS Policy Manual which clarifies procedures for assessment, data collection, and data management.
  • Implementing a monitoring process that addresses both risk management and program improvement, as well as performance monitoring of Title II targets set by grantees in their applications. 
  • Continuing a strong statewide focus on data quality through quarterly data audits and monthly calls with local provider data professionals. 
  • Implementing a funding formula with a portion of funds awarded based on performance. 
  • Distributing a weekly Title II newsletter to program directors with details on upcoming events and webinars, state deadlines, and other important information pertinent to Title II programs in Oregon. 
Regarding Employment Second Quarter after exit, Oregon achieved 31.0% of participants who exited the program and were in unsubsidized employment during the second quarter. This represents 78% of the federally negotiated target for this indicator: 39.9%. To meet the federally negotiated target, Oregon would have needed 413 additional participants to be in unsubsidized employment during the second quarter following exit from the program.  Regarding Employment Fourth Quarter after exit, Oregon achieved 17.0% of participants who exited programs and were in unsubsidized employment during the fourth quarter. This represents 77% of the federally negotiated target for this indicator: 22.2%. To meet the federally negotiated target, Oregon would have needed 436 additional participants to be in unsubsidized employment during the fourth quarter following exit from the program. The median earnings of program participants in unsubsidized employment during the second quarter after exit from the program was $5,031.04. This exceeds the federal negotiated target for this indicator: $3,535.00. While this is an impressive increase over programmatic targets, caution should be used when considering this spike. A multitude of factors were in play during the PY2021-22, which included business re-openings and an increase of demand to fill entry level positions. What resulted was a significant increase in starting wages for employees. Other skilled trade openings saw a similar increase in starting wages. However, it has been suggested that once the economy slows, this trend may slow and/or reverse. Regarding Credential Attainment, Oregon achieved 20.4% of program participants eligible to be included in this measure attained a recognized postsecondary credential or a secondary school diploma/recognized equivalent during participation in or within one year after exit from the program. This exceeds the federal negotiated target for this indicator: 17.4%.  
Puerto Rico The AEP performed an analysis of the performance data reported this year. To perform that analysis the process conducted at the State and local levels was based on the data collection done by the service providers and uploaded to the Adult Information System (AIS) at AEP central level. A detailed data review and validation for each data indicator was performed by the central office. The specific anomalies identified were checked for corrections. The errors, when identified were resolved by telephone calls and discussion of the raw data indicators with the education centers directors, teachers and counselors at each center. The AEP completed 10 visits to education centers to provide technical assistance on compliance with data collection and data quality management for NRS purposes. The outcome reached The State performed adequately regarding its negotiated levels of performance and outcomes for each indicator for PY2021-2022. The targets and outcomes are the following:  (Table 4 NRS). Performance indicator PY 2021-2022 negotiated level (%) PY 2021-2022 outcome level (%) Percent achieving Measurable skills gains Average ABE and ESL 54.5 57.09%   COVID pandemic and post-hurricanes federal incentives impact during PY 2021-2022 The COVID pandemic effects on participants recruiting and attendance have been fading out, but during 2021-2022 program year the difficulties in participants’ recruitment continued as result of the federal economic stimulus program providing cash incentives to individuals that resulted in a lack of incentive for work in the whole economic system. In addition, Puerto Rico continues the recovery from the sequence of hurricanes that severely impacted the economic and social collective life during the PY 2021-2022. This situation provoked a challenging situation for enrollment of participants during the PY. Adult Education Participants’ profile for PY-2021-2022 The Adult Education Program served, during 2021-2022 school year, an active participants enrollment of 3,946. The previous year 2020-2021 the enrollment was higher with 5,801 participants. In conclusion, the PY 2021-2022 has experience a reduction of 1,855 participants served in comparison with the previous year. The decrease is explained by the two-fold factors explained in previous section: the impact of hurricanes and lack of interest in workforce training due the economic cash incentives provided to low income families by the federal government. The distribution by gender in participants served was as follows: 1,565 males, equivalent to 39.7% of total and 2,381 females, equivalent to 60.3%. The males’ relative participation decreased by 6.5% in relation to previous program year when it was 46.2% of total, while female participants increased by an equivalent percentage. Participants by services was as follows:
    1. Basic Education (ABE 1 thru ABE 4) – 132 participants.
    2. Secondary Education (ABE-5 thru ABE-6 and Equivalency Exam Tutoring) - 2,066 132 participants.
    3. Conversational English - 1,748 132 participants.
    4. The participants in ABE 5 and 6 represented the greater share, 52.3% of total participants, followed by the share of conversational English, equivalent to 44.3%.
Correctional education program: A total of 683 were served while in the correctional education program or in rehabilitation. (Table 6 - NRS)   Demographic Characteristics The Composition of participants by age, during school year 2021, showed that: (Table 2 - NRS)
    1. The greatest participation in the program was among participants between 16-18 years of age, a total of 1,348 participants which represented 34.2% of the enrollment.
    2. The 2nd group in relative importance was the 25-44 years of age for a total of 1,290 participants making up 32.7% of the enrollment.
    3. The 3rd group was the 45-54 years of age group with 498 participants, representing 12.6%.  
    4. The 4th group in relative importance was the group age of 19-24 with 465 participants representing 11.8% of the total.
    5. The 5th and 6th groups were the 55-59 and 60+ age representing 345 participants, a combined 8.7% of total participants.
In aggregate, these groups represented the 100% of the participants served. The 99.6% of the participants identified themselves as Hispanics or Latinos. The employment rate: The employment rate was 33.2% (greater than previous PY when it was 19.4%) with 1,310 participants. The previous PY 2020-2021 the employed participants were 1,125 participants. The unemployed were 1,156, representing 29.3% which is a lower rate in comparison with the 35.4% during the previous year. There were 1,480 participants in the category of not in the labor force, representing 37.5% of the total participants served, a significant lower percentage in comparison with the 45.2% in this category the previous year. (Table 6- NRS) The level of schooling: The level of schooling of the participants before registering in AEP is divided in nine schooling levels. The frequency of levels is as follows: (Table 6- NRS)
  1. The largest frequency was in the 9th-12th grade with no high school diploma. This group included 1,556 participants, representing this represented 39.4% of the total participants.
  2. The second most important group was the categories of participants with some or a completed university or professional degree with 1,230 combined participants, representing 30.2%.
  3. The third group in relative importance was the category of grades 6-8, with 285 participants, representing 7.2%.
  4. The fourth group was the secondary school diploma or alternative credential with 273 participants, representing 7%.
These five groups accounted for 83.8% of the total participants served.   Measurable Skill Gain (MSG) achieved. The Adult Education Program achieved the general objectives in skill gains: A total of 1,953 participants, (49.5%) achieved at least one functioning level of measurable skills gains, ABE and ESL level. (Table 4 - NRS).  A total of 1,445 participants, out of 1,748 on ELS program, equivalent to 82.7%, achieved an average level of ESL measurable skills gains. (Table 4 – NRS) In ABE, the average percent of MSG is 59.87%. Overall MSG total for ABE and ESL is 69.96%   (Table 4 – NRS) The number of participants who attained a secondary school diploma or its equivalent at ABE levels was 808 participants, out of a total 2,198 participants in total ABE levels, representing 37% of total, more than one on each 3 participants. Quality and effectiveness of the funded eligible providers and any plans to further increase performance in future reporting years. The Adult Education Program (AEP) implemented a plan to improve quality. The plan is based on a quarterly review performed by the central level staff and informing each provider on their performance. Each service provider must submit a plan explaining how it propose to improve the services. The AIS at the central office prepared a review documents that was used by the facilitators’ staff appointed at central office to visit the local service providers to discuss the deficiencies found and how to draft a correction plan.   The program staff visited 41 local services providers implementing adult education activities to provide support, technical assistance
South Carolina Assessment of Core Programs The OAE formally and informally assesses each local program throughout the year via Desktop Monitoring, subgrant Interim and Final Reports, Compliance Monitoring, and a variety of informal data reviews via the state’s data management system, LACES. The Desktop Monitoring Tool (DMT) is a quarterly report that collects data from NRS Tables, as well as secondary credentials and Career Readiness Certificates (CRC). Each local program completes and submits a quarterly DMT. At the end of each year, a program overview report is completed at the state level using the DMT data. This report provides a snapshot of each local program and assists the state with developing goals for the next year by identifying promising practices and professional development and technical assistance needs for the coming year. All programs receiving federal subgrant(s)—Adult Education, Family Literacy, Corrections Education, and Integrated English Literacy and Civics Education (IEL/CE) grants—completed an Interim Report and a year-end Final Annual Report, designed specifically for each grant. Data from these reports guide specific technical assistance delivery for programs receiving this grant. In addition to the DMT and subgrant reports, the OAE conducted data reviews in order to identify areas of strength and weakness and to measure progress toward meeting the 2021-2022 OAE Goals. Outcomes of the 2021-2022 Strategies for Increasing Performance Based on the assessments of core programs, the SCDE OAE sets goals to determine the focus for each program year in order to meet the performance targets for measurable skill gains and core follow-up outcome indicators. The 2021-2022 OAE Goals were to:
  1. Strengthen Distance Education
  2. Market and promote the GED by 23 Initiative in partnership with the SC Technical College System
  3. Advance workplace literacy opportunities and services
  4. Create awareness of Integrated Education and Training (IET) among students and partners
In support of the four goals, the processes of data collection, entry, and analysis were incorporated into all conversations with the local programs. Performance and outcome data was shared with local program directors in order to guide conversations and processes toward data-driven decision making at the local level. Training efforts on data collection and entry were further increased. In addition, much of the data trainings focused on streamlining the data entry processes across the state in order to improve the accuracy and quality of our performance data. The OAE developed several uniform processes to emphasize the need for consistency across the local programs. Key factors when determining where consistent processes were needed were: 1) Is it connected to NRS Tables or state/federal reporting?​ 2) Is it related to funding? and, 3) Is it used for program level tracking? Numerous error checks were conducted throughout the year and the results of which were used to guide additional data collection, entry, and analysis meeting agendas and training topics. These efforts proved effective in guaranteeing that the data and performance captured for distance education, statewide initiatives, Workplace Literacy, and Integrated Education and Training (IET) accurately reflected the efforts of the local programs. Goals Results
  1. Strengthen Distance Education
    1. The OAE purchased seats for three distance learning platforms and provided a multitude of professional development and technical assistance in using these platforms.
    2. Prior to the 2019-2020 program year, six local programs offered 100% virtual classes except for the students working on high school diploma courses via proficiency-based systems. By the end of the 2021-2022 program year, 16 local programs have created more flexible options to meet the needs of their students by implementing 100% distance education options for basic literacy, high school equivalency preparation, and ESL classes. Of the 16, five also provide a virtual process for registration, intake, orientation, and pre- and post-assessments.
    3. In the 2021-2022 program year, 100% of local programs offered synchronous, asynchronous, or hybrid distance learning options for students. Prior to the pandemic, 18% of enrolled students participated in distance learning options. The number of enrolled students participating in distance learning options increased to 44% of enrolled students.
  2. Market and promote the GED by 23 Initiative in partnership with the SC Technical College System
    1. By the end of program year 2021-2022, over 5,000 marketing emails about the GED by 23 Initiative scholarships and information regarding the credential and degree programs available through the SC Technical Colleges were sent out to SC Adult Education graduates. The initiative will continue through the 2022-2023 program year and enrollment into postsecondary will be monitored quarterly via the Desktop Monitoring Tool and general data system reviews.
  3. Advance workplace literacy opportunities and services
    1. Despite the barriers created by the pandemic, the number of local programs having an active partnership with an employer to deliver Workplace Literacy services increased each year, ultimately doubling from the pre-pandemic 2018-2019 to post-pandemic 2021-2022.
    2. The number of participants in these active Workplace Literacy services increased 181%, from 16 participants in 2018-2019 to 305 participants in 2021-2022.
    3. Of the 305 Workplace Literacy Participants, there were a total of 366 measurable skill gains earned. Some earned more than one MSG as a result of the employer requiring more than one certification as part of their process.
      1. 124 EFL gains
      2. 66 Progress toward milestones
      3. 176 Passed technical or occupational exams
  4. Create awareness of Integrated Education and Training (IET) among students and partners
    1. While the pandemic put a temporary halt on implementing the more than 50 approved IETs across the state, the number of local programs with an active IET program in place increased 126% from pre-pandemic numbers.
    2. From 2018-2019 to 2021-2022, the number of IET participants increased from 21 in 2018-2019 to 359 in 2021-2022. As more local programs expand their partnerships, this number is expected to continue to increase exponentially.
    3. In total, 421 measurable skill gains were earned by the 359 IET participants. As with Workplace Literacy, there are often multiple steps in these pathways and many students achieved the goals for each step.
      1. 87 EFL gains
      2. 21 secondary credentials
      3. 204 Progress toward milestones
      4. 109 Passed technical or occupational exams
Overall Outcomes While enrollment is still 34% lower than the pre-pandemic enrollment, the quantity and quality of services provided by the South Carolina local programs continue to improve as the focus shifts from just providing adult basic education to serving all of the needs of the students and communities through education, connections to support services, and workforce preparation opportunities. As a result of these efforts, overall measurable skill gains have almost returned to pre-pandemic levels (43.85% now versus 44% in 2018-2019) and all other state and federal performance measures have increased substantially. 2022-2023 Strategies for Increasing Performance For the 2022-2023 program year, the state plans to continue the use of Desktop Monitoring, the subgrant Interim and Final Reports, Compliance Monitoring, and a variety of informal data reviews via the state’s data management system, LACES. In addition, the state will continue to implement informal compliance monitoring for each program not formally monitored and continue to provide support and training for data collection, entry, and analysis. The state’s 2022-2023 goals are to:
  1. Increase participation in distance education opportunities.
  2. Continue to market and promote the GED by 23 Initiative in partnership with the SC Technical College System.
  3. Continue to highlight workplace literacy activities through promising practices.
  4. Improve the development and implementation of the Integrated Education and Training (IET) process.
  5. Continue to develop course offerings for professional development in D2L Brightspace.
  6. Monitor and collect the expanded College and Career Navigator (CCN) activities.
South Dakota In a post-vaccine milieu, it yet seems most reasonable to contrast last year with PY2020.  While South Dakota’s WIOA Title II program served 1,365 participants in PY2020, PY2021 saw 1,635 participants [for a noteworthy 19.8% increase in participation].  The Adult Education Program evidenced 34.49% Measurable Skill Gains in PY2020, while PY2021 finished at 36.44% MSG.  Similarly, South Dakota’s number of reported PY2021 GED credentials trended upward, albeit at a more modest rate. Performance Outcomes (PY2017 – PY2021) Program Year ABE MSG ESL MSG Total MSG Q2 Employment Q4 Employment Median Earnings Credential Rate 2017 41.38% 37.99% 40.25% 51.31% 46.17% $5,721.14 21.70% 2018 45.56% 30.72% 40.80% 58.70% 56.20% $5,575.85 40.52% 2019 25.82% 24.95% 25.47% 55.85% 58.80% $6,151.15 55.20% 2020 34.37% 34.89% 34.49% 50.20% 49.95% $7,220.47 43.42% 2021 37.93% 31.98% 36.44% 56.22% 50.20% $6,405.47 30.50% WIOA Title II’s High School Equivalency Attainment  Program Year   Number of GED Credentials Conferred  2017 281 2018 365 2019 126 2020 216 2021 259 Due to South Dakota’s pandemic-related recovery, unemployment rates are some of the lowest in the nation.  And like everywhere, businesses and employers are clamoring for entry-level, mid-level, and skilled workers.  Wages are seeming to increase as industries vie for qualified candidates.  And still, these workforce-related factors and our clients’ personal exigencies affect WIOA Title II’s abilities to engage students long enough to post-test, confer GED Credentials, and make significant investments in English language instruction.  South Dakota’s national rankings in the Employment Outcomes (i.e., Q2 Rate, Q4 Rate, & Median Earnings) highlight well the socio-economic realities [as best as such data-collection protocol can afford].  If someone needs or wants a job in South Dakota, the opportunities most certainly exist. The PREP initiative [noted throughout South Dakota’s PY2021 Narrative Report] is meant to support our co-enrolled learners so they can address some of the pressing needs of immediacy (e.g., food security, housing, transportation, childcare, etc.) while simultaneously concentrating on their studies in a scaffolded, tandem case-management model.  The agency continues to explore creative ways to not only support our co-enrollments to the fullest extent of the statute, but also continues to collect data to validate efforts, identify promising practices, highlight opportunities for technical assistance, and improve partnerships [for the benefit of our participants and our WIOA Joint PIRL measures].  Therefore, beyond the marked improvement in participation-figures over the prior Program Year, the South Dakota AEFLA Program is also rather proud of more than doubling its reported Percent Enrolled in More Than One Core Program on the PY2021 Statewide Performance Report.  And even though this figure is still likely a bit underreported, we are making demonstrable progress in effectively tracking these common participants.  Perhaps most importantly, we are strengthening the communication, coordination, and collaboration that underpin these efforts across the One-Stop System!
Tennessee In PY21, TDLWD’s systems and analytics specialist focused on three main goals: (1) oversight of the management information system (“Jobs4TN”), including troubleshooting issues with the system and monitoring for anomalies and errors from local data entry; (2) maintaining performance dashboards to help AE stakeholders quickly and effectively assess local program performance; and (3) establishing and measuring KPIs that directly impact student performance—the idea of “measuring what matters”.  We plan to refine these dashboards and provide additional training on how to use them effectively in the next year. TDLWD staff also provided continuous technical assistance on the data entry and evaluation process to ensure that local staff were capable of doing accurate data entry. We also included robust data assessment as part of our monitoring process. This included thorough investigations into specific local providers’ data, and when errors were found, we brought this to the locals’ attention and included the requirement for them to improve data practices as part of their corrective action plan. The Tennessee adult education program achieved the following performance outcomes in PY21:
  • MSG % all POPs: 39.17%
  • Total Reportable Individuals: 4,837
  • Total Participants: 10,368 (8,083 ABE/2,285 ESL)
  • All POPs: 10,562
  • EFL Gains: 1,464
  • HSE Diplomas: 2,482
  • IET/WPL Gains: 192
  • Employment Q2 after exit: 51.86%
  • Employment Q4 after exit: 50.33%
  • Median Earnings Q2 after exit: $4,665.84
  • Attained any credential: 44.46%
    • Attained a secondary school diploma enrolled in PSE: 1.29%
    • Attained a secondary school diploma employed: 43.87%
    • Attained a postsecondary credential while enrolled: 100% (2 participants)
PY21, while still very challenging for Tennessee’s AE programs, did see improved performance. Our negotiated MSG target for the year was 37.9%. Our final MSG was 39.17%, up from 38.24% in PY20. Providing high-quality instruction during the pandemic became the “new normal”, and local staff became more comfortable continuing distance education and embracing a hybrid model of learning for students. TDLWD also was intentional in training the local leadership, staff, and teachers how to use their data to make informed decisions about program improvement. To address local program performance issues, TDLWD leaders regularly looked at each program’s performance to determine which were struggling the most and created performance improvement plans. These plans helped focus our efforts in providing targeted technical assistance in the areas where they were struggling. Going forward, we plan to make this process even more robust to identify areas of poor performance. This includes implementing more intentional local program performance improvement plans based on low performance in enrollment, MSG, or workforce initiatives.
Texas Through TWC’s state-developed data system, we are able to see data in real-time and utilize information to target technical assistance or professional development needs of the state provider system. The TWC AEL web-based data system, Texas Educating Adults Management System (TEAMS), allows easy access for end users to input data daily with a state policy that dictates a monthly data entry requirement. Monthly data validation procedures are built into the data system providing state staff with real-time reports if local data entry is delayed. Once data is validated, it is locked for further input by local users. This allows the TWC’s AEL team to pull data reports each month to review the number of participants being reported, Measurable Skill Gains (MSGs), and posttest eligible students among other performance-related reports and quality checks. State staff analyze the reports to identify: •             significant increases or decreases in enrollment since last month; •             participants who are eligible for posttest who do not have another MSG in the system; •             time between tests in hours and days; •             IET students who have not earned an MSG or a credential; and •             concerns with retention for students who stopped attending and are in jeopardy of exiting the program early. System reports are more useful than NRS tables for programmatic discussions with providers. System-generated reports provide disaggregated data down to the instructor and participant level. TEAMS also has built-in controls to cross check for anomalies with automated data checks in place that support accurate data reporting, such as but not limited to:
  • NRS approved tests with scale/cut scores that automatically assign Educational Functioning Level (EFL) and track progress through comparison of initial test to post test
  • automatic assignment of initial test/baselines based on a first contact hour
  • MSG checks to ensure criteria is met before an MSG can be earned/credited within the data tables, particularly useful with new options for workplace literacy and IET participant gains
  • period of participation calculators and exit trackers
  • tracking of exclusionary reasons only if criteria is met
Multiple performance and management reports are available to AEL providers to track contracted measures. TWC staff create a monthly report card for all AEL grantees that provides comparison data and visualizations on both fiscal and program performance proportionally for where a provider should be within the year (proportional target) versus actual performance. AEL state staff monitor the grants for the elevation of adverse actions, such as Technical Assistance Plans (TAPs) or Corrective Action Plans (CAPs). All AEL providers have the following performance measures: Enrollment Targets – three sub targets: Basic/Popular (GED/ESL/ABE), Intensive (Workplace Literacy, Services for Internationally Trained Professionals, Re-entry), MSGs, Credential Achievement, Employed or Enrolled Q2 After Exit, Employed or Enrolled Q4 After Exit. Contracted measures are aligned to federal methodology and/or state performance as outlined in our General Appropriations Act (GAA). AEL Providers Contracted Targets Target Name PY 20-21 % of Providers Who Met Target PY 21-22 % of Providers Who Met Target MSG 41% 49% Credential 49% 35% Employed/Enrolled Q2 After Exit 81% 76% Employed/Enrolled Q4 After Exit 76% 8% Due to the lag in reporting periods for exit based outcomes, credential and employment outcomes were greatly affected in PY 21-22. During the pandemic, high school equivalency testing facilities were closed, and even with GED© offering online proctoring of tests, the usage was low. Since the PY 21-22 included reporting periods that were greatly impacted by Covid-19, some lag measures such as credential attainment and post exit employment were very low.   PY 21-22 Federal Performance Outcomes:   Indicator PY 20-21 Performance PY 21-22 Performance MSG 38.63% 39.81% Credential 41.39% 32.88% Employed Q2 After Exit 35.52% 40.78% Employed Q4 After Exit 34.11% 36.43%   TWC has made tremendous efforts to support more useful performance accountability tools. We want AEL providers to have real-time program management and performance reports available to better support local decision making and aid in planning and oversight. For this reason, we have implemented tools and practices that we feel have generated greater capacity at the local level for understanding the impact of local practices on performance indicators. In PY 21-22, we created a Detailed Program Review tool to assist AEL providers in a thorough review of grant effectiveness. The tool consists of questions related to key areas such as: •             Services / Enrollment •             Staffing Levels •             PD •             Testing Procedures •             Comprehensive Assessment •             IETs •             Distance and Remote Learning •             MSGs •             Data Entry and Data Analysis •             Protecting Personally Identifiable Information •             Compliance and Monitoring •             Student Supports During the last grant competition, TWC required specific positions related to performance. These positions included  the identification of two staff positions with one to serve as a point of contact for  Quality Assurance (QA) and a separate one for Performance Accountability(PA). Local providers were required to identify two  individuals and provide resumes as part of the solicitation and review process. The job functions were clearly laid out in the grant for these positions and helped to solidify a control group in which TWC technical assistance staff would train and convene this cohort for best practice sharing. We currently have quarterly meetings with the QA/PA staff to discuss current data collection and reporting practices, data analytics, and performance and compliance monitoring. Through targeted support to these positions, state staff has guided and facilitated a community of practice for those working in these roles and for sharing local best practices. Tableau In 2021, TWC rolled out data visualization dashboards created by using an enterprise data warehouse combined with a data visualization tool, Tableau©. Statewide and AEL grantee-level enrollment data reports were the first reports to be launched, TWC will finalize MSGs and monthly performance reports in PY 22-23. The current enrollment reports identify key activities conducted at the local provider level such as reentry services, workplace literacy, and IET activities for both Adult Basic Education/Adult Secondary Education and English language acquisition students. It also includes participant-level demographic information collected from the Participant Individual Record Layout (PIRL) that helps to support the identification of internationally trained professionals and individuals with barriers to employment. In PY 21-22, the state provided multiple technical assistance webinars on how to use the tool. Currently, more than 3,000 local administrators, data entry, and instructional staff have access to the dashboards, but usage has not been as high as we would like, therefore we plan to target and build capacity in PY22-23. We were also exploring the use of adaptive machine learning for predictive modeling. More to come on next year’s report regarding this initiative.
Virgin Islands For the reporting period of July 1, 2020 through June 30, 2021 in the USVI Adult Education programs showed significant changes from preceding years as a result of impact from COVID-19. Although learning continued to be online because of the pandemic, a hybrid model was implemented to accommodate adult learners who were not adept to technology, struggled with following simple directions and became easily frustrated with learning basic commands. The pandemic has emphasized the need for diverse and structured ways for Adult Education programs to provide learning opportunities. However, there exists genuine concern that students may become even further marginalized because they do not have a smartphone, tablet, laptop, internet access, or the necessary digital literacy to function properly in a remote learning environment. A pivotal example of Measurable Skills Gain (MSG) over the most recent two Program Years highlights  the difficulty VIDE’s AE program had in meeting program mandates due to COVID-19’s adverse effect on areas of performance (e.g.  Data, enrollment, MSG, educational functioning levels (EFL) gain, and employment gains).  Examination of measurable skill gains comparisons of program data reported for 2020-2021 in comparison with 2021-2022 showed a decline from 22 percent to 25 percent. This was also due to the instability of many power outages territory wide as the territory re-built the electrical infrastructures post the natural disasters. This also was a significant barrier that served as a contributing factor that affected the online delivery services to our adult learners. Overall, as a territory, there was a 20 percent decline in NRS participants. ESL enrollment increased by almost 22 percent and ABE enrollment increased by nearly 19 percent. There was an overall increase in students age 19-24 (22.5 percent) and students age 25-44 (23 percent). Recruitment and retention will be essential components of professional development moving forward. Additionally, concentrated monthly monitoring of the MIS per program by the State to systematically track local program’s data entry and provide one on one technical assistance for programs that have been deficient will be an area of targeted focus.
Washington Measurable Skill Gains
Washington State achieved a 27.91% (28% when rounded) overall Measurable Skill Gain (MSG) rate for the program year 2021-22, significantly lower than its MSG target of 43.8% (rounded to 44%). This is due to statewide governor-ordered suspended operations caused by the COVID-19 pandemic and our providers’ inability to CASAS test. According to Table 4B, only 3,967 participants were able to post-test in PY2021-22 out of a possible 26,251 participants in all periods of participation. This is a minuscule amount compared to the number that have been able to post-test before the pandemic. It is nearly four times the amount as last year in which our state was also ordered to suspend operations. Gains achieved using the HS Credit Option, IET MSGs, and the EFL gain for a program exit and transition to postsecondary EFL during the program year, contributed to the higher levels of gains showing in ABE 4 (39%), ABE 5 (40%), and ABE 6 (65%) [see Measurable Skill Gains: All Colleges/Organizations chart below].
  Post-exit Outcome Measures
Washington State exceeded all targets for post-exit outcome measures for PY2021-22, but it was noted in our federal program review that our Credential Rate was not calculated correctly. Table 5 shows that although performance levels were similar to last year, the cohorts are lower than the previous year by 9-38%. Our employment results are likely skewed since Washington relies solely on social security numbers (SSN) matching with the SWIS unemployment insurance dataset. English language learners made up about 44% of our program and reported SSNs at a much lower rate (38.8%) than students in our ABE programs (78.7%) according to internal research. It’s impossible to know the true rate of SSN collection among the ELL population since our state’s college data system collects SSNs and ITINs (tax ID numbers) in the same field. The strongest component of the overall credential rate is still the Attained a Postsecondary Credential while enrolled or within one year of exit measure. Strong performance is supported by our state’s special program Integrated Basic Education and Skills Training Program (I-BEST). This program allows students to work on college-level studies right away, while receiving basic skills instruction, and leads to certificates and degrees. Many of these programs were suspended during the pandemic leading to a drop in MSG as well as our Credential Rate overall.
Wisconsin The Wisconsin AEFLA program is a national leader with strong educational and employment outcomes across the core indicators of performance. As demonstrated in Table 1, the Wisconsin Adult Education program exceeded the nation across the federal WIOA primary indicators of performance. Table 1. WIOA Primary Indicators of Performance Outcome WI (PY 2021-22) Nation (PY 2020-21)[1] Difference  Measurable Skill Gain Rate 53.5% 34.7% +18.8%  2nd Quarter Employment after Exit 53.0% 29.3% +23.7%  4th Quarter Employment after Exit 43.7% 28.6% +15.1%  Median Earnings 2nd Quarter after Exit  $   6,111  $   4,899 +$1,212  Credential Attainment Rate 62.1% 25.9% +36.2% Strengthening Adult Education Recruitment   Chart 1. Wisconsin AEFLA Participation: Participants & Reportable Individuals (PY 2017-18 to 2021-22)   While the Wisconsin Adult Education program has reported strong outcomes, actual participation has plateaued with the exception of PY 2020-21 due to the COVID-19 pandemic (reference Chart 1). Stagnant participation in the Adult Education program is significant given over 354,000 Wisconsin adults are without a high school credential and over 142,000 Wisconsin adults do not speak English very well or at all (US Census American Community Survey, 2021). These data points mirror national research suggesting that only 10% of adults who could benefit from adult education skills support actually participate in the Adult Education system (Patterson, 2018).   In response to last year’s Narrative Report and provided feedback during monitoring activities, the WTCS Office coordinated a series of professional development opportunities to strengthen student recruitment efforts. During the October 2021 Common Ground Conference, the WTCS Office delivered a session titled Strategies for Adult Education Recruitment. During the session, participants were provided a step-by-step overview of how to retrieve US Census Data and use the data to inform targeted recruitment. In addition, empirical evidence from existing Adult Education recruitment research (e.g., Patterson & Song, 2018; Smith, 2019; Smith, 2020) was reviewed to build awareness of potential student recruitment strategies. The WTCS Office plans to continue delivering professional development during the 2022-23 program year focused on equity in recruitment practices and how these efforts, coupled with strong educational and employment outcomes, can enhance the economic mobility of Title II learners. The WTCS Adult Education team participated in the NRS 2022 Regional Workshop titled The Journey to Increased Enrollment: Recruitment and Service Strategies for Adult Education. The WTCS Office staff who participated in the training plan to contextualize the content for Wisconsin and scaffold the content over three professional development opportunities during the 2022-23 program year. Fully Realizing the Employment Outcomes of Title II Participants During the 2021-22 program year, the WTCS Office coordinated a technical assistance session to build the fields awareness of the three-employment based, WIOA primary indicators of performance. The indicators included 2nd quarter employment after exit, median earnings 2nd quarter after exit, and 4th quarter employment after exit. Each provider reviewed trend data across the three indicators for their program and engaged in reflection related to what might be contributing to their program’s outcomes. In addition, the WTCS Office shared an overview of each indicator’s calculation methods. Feedback from Wisconsin’s Title II providers revealed three important session findings. First, participants engaged in Integrated Education and Training had stronger employment outcomes compared with their peers not engaged in Integrated Education and Training. This finding is logical and supportive of the occupational alignment of WTCS Integrated Education and Training offerings. It also adds fuel to Wisconsin Title II’s existing Integrated Education and Training expansion efforts. Second, providers believed there was an opportunity to better partner with WIOA Title I to strengthen pathways to employment. This finding triangulates with small levels of Title II co-enrollment across WIOA Title partners; only 4 percent during PY 2021-22. Third, providers shared challenges with data quality that may be impacting employment indicator results. These included challenges in collecting Social Security Number for employment data matching purposes, which are especially challenging among ELL learners, and the limited nature of Wisconsin leveraging the state’s wage-record system for employment data. In response to employment indicator data quality feedback, the Wisconsin Title II program is planning a series of activities for PY 2022-23. The WTCS Office is planning to explore additional ways to conduct data matches that extend beyond solely matching on Social Security Number. This might include matching on name and date of birth. In addition, the WTCS Office plans to coordinate supplementary technical assistance during PY 2022-23 on strategies for collecting data at intake, including Social Security Number. Finally, the WTCS Office is exploring how it can best leverage the SWIS to complement state employment data with surrounding state data. Together, these data quality activities will help Wisconsin Title II fully realize the employment outcomes among its participants.   [1] National PY 2021-22 data unavailable at the time of writing Wisconsin’s PY 2021-22 Narrative Report
Wyoming Process: The State takes a multi-tiered approach to analyzing and monitoring data. Because data is the key to success, the State monitors and reviews data through multiple sources. This year the State implemented a ‘Roadmap to Data Success’ system which highlighted six key aspects to data analysis:
  1. Local Program Data Reviews: take place on a monthly basis where student and staff data found in the LACES database are reviewed and reconciled for accuracy. Programs then submit a monthly report that measures progress towards meeting federally negotiated performance targets for both measurable skill gains, financial tracking, and overall program comments.
  2. State Reviews:  Upon receipt of these reports, the State conducts a monthly evaluation of each provider to review student and staff data, financial aspects of the grant as well as provide technical assistance as needed. In addition, the State tracks performance  through a multi-year tracking document which evaluates local program performance, retention, enrollments in Adult Education and co-enrollments between the core partners and referrals made between core partners and other community service providers.
  3. Quarterly Reports: track successes and challenges in the local program and provide the State with the documentation needed to provide technical assistance while monitoring local program performance for the quarter.
  4. WIOA Core Program Co-Enrollment Reports: Once per quarter, Titles 1-IV develop a co-enrollment report which details those participants who were co-enrolled between the programs. These numbers are then used for federal reporting purposes, as required for Title II.
  5. Data Dives: Each local program was encouraged to conduct an extensive evaluation of programmatic data to review any one aspect of their program for the purpose of program improvement.
  6. Annual reports: Adult Education providers submit comprehensive annual reports that evaluate various aspects of the Adult Education program and include a review of compliance to WIOA Sec 231 (e), success and challenges, program improvements, program evaluation and monitoring efforts, an analysis of data shown on the NRS tables, effective educational practices, integrated programming inclusive of Integrated Education and Training programs (IET) and Integrated English Language & Civics Education (IELCE) programs, program alignments to the Unified State Plan, professional development, and career pathways.
Errors identified in these processes were discussed with the local provider and were immediately corrected. Results: Despite the challenges created by a cyber-attack on one of our local programs, each provider met federal negotiated targets within the 90% range for both EFL and outcome measures. In fact the state surpassed its targets in all areas as shown in Table 2. The State credits this performance to increased vigilance on data monitoring, monthly trainings on data validation and increased efforts on data matching and surveying. One example of this increased vigilance is that several local providers have implemented an ‘alert’ system to notify instructors that a student is approaching the time in which a post-test is needed. Table 2: Wyoming Performance FY 21/22 FY 21/22 Federal Target Actual Performance Measurable Skill Gain 48.00% 65.6% Employed 2nd Quarter After Exit 45.00% 60.34% Employed 4th Quarter After Exit 43.0% 59.34% Median Earnings $3,570.00 $4,450.76 Credential Attainment 48.00% 65.26% Local providers did a great job in capturing as many posttests as possible. Wyoming surpassed its established 60% post-test benchmark and strove to meet EFL gains through all approved means, which this year included 1.4% of eligible participants earning MSG through the new IET/Workplace measures. By the end of the year Wyoming had post tested 61.80% of all enrolled participants while another 27.6% had achieved EFL gain through the completion of a HSEC. These factors, combined, saw Wyoming surpass its performance measure target of 48% with an actual MSG performance rate of 65.6%. Because employment outcomes are tracked a year to eighteen months behind, Wyoming expected to see the full impact of the pandemic on employment measures in FY 21/22. This did not occur, at least not for the enrolled AE participants. However, recent employment reports and an increased cost of living may have a larger impact upon employment statistics for FY 22/23. Analysis & Action A comprehensive review of performance trends across the past several years, reveals that Wyoming’s AE programs are consistently increasing performance levels each year; thereby showing continuous program improvement.  Three Year MSG Trend:      FY 19/20: 49%                FY 20/21: 53%                  FY 21/22: 66% Outcome measures for the same three years also reveals improvement; however, this may be resulting from increased levels of data matching and surveying as well as from local providers developing a more comprehensive understanding of how each cell on table 5 populates. In order to compare program performance across the past several years, the State reviewed enrollment numbers, retention rates, and how participants were making MSGs. These results are shown in Table 3. Overall, Wyoming saw improvements in every aspect under review. Table 3: Two Year Comparison of Wyoming Performance   FY 20/21 FY 21/22 % Change Enrollments 1412 1586 11% increase Total MSG Gains 58.80% 65.89% 7.09% increase MSG: EFL 27.80% 28.40% .60% increase MSG: Postsecondary 6.3% 8.6% 2.3% increase MSG: HSEC 24.20% 27.6% 3.4% increase MSG: IET/Workplace 1% 1.4% .4% increase Retention 29% left before completing 21% left before completing 8% improvement Distance Learners 7.2% 8.6% 1.4% increase         The State also reviewed the demographic picture of enrollments across the past several years and found that although total enrollments took a dip during COVID, enrollment numbers rebounded in FY 21/22; going from 1,570 before the pandemic to 1,586. In addition, the State also reviewed how MSG’s were being earned and whether this has changed over time as understanding this is critical to local programs. The results of this analysis found that the number of students earning an EFL through post testing has remained relatively the same, but the number of MSG’s being earned from entry into postsecondary has increased due primarily to an increase in the number of local providers offering bridge programs. The percentage of students earning their HSEC is also seeing significant increases for a variety of reasons: increased referrals from high schools, increasing dropout rates due to COVID, and for other area specific reasons. Although providers are now able to utilize several new MSG measures for IET and workplace literacy, the State had only 1.4% of participants showing gain through Milestones or other new measures. The primary reason for this low number is that the State does not have many IET’s and those providers who have an IET must apply for and receive State approval before they can utilize ‘Milestone’s as an MSG. Although the State implemented fairly strict protocols for ‘Milestones’ there were still some challenges as one provider utilized ‘Milestone’ with at least two students who were not involved in an IET. This error was caught shortly after the close of the fiscal year and technical assistance was provided so that a modification in local program could be made for FY 22/23. It comes as no surprise that students are more successful when they don’t have to worry about things such as transportation, childcare, health care, etc. Consequently, the provision of wrap around services are critical to student success. In order to evaluate whether our students are 1) identifying their barriers and 2) are we addressing them, the state examined the highest two barriers to employment identified by students in FY 2021/22 (low income & single parents) and found that a satisfactory number of referrals were being made to help these participants overcome these barriers. One of the more significant findings of the State analysis of performance is that our students seem to be getting jobs shortly after exiting our program; however, they seem to struggle to keep that job on a long term basis. This may suggest a lack of skills in such areas as work ethics, soft skills, or possibly a change in career tracks. The implication for local programs is that they may need to increase efforts on preparing students for the workplace, which is a FY 22/23 goal for most providers. COVID brought many changes to local programs in the State and some of these changes have had positive outcomes on local performance. Hyflex classes are now the norm at several provider’s locations as they offer more flexibility to students in the more rural areas of the state. In addition, as seen during the pandemic, the State continued to see increasing numbers of students taking advantage of distance learning materials which helped to increase retention rates as shown in Table 3. Throughout the year, the State only allowed the use of NRS approved assessments for EFL placement and/or to measure gain. Testing was conducted in various fashions, inclusive of traditional face-to-face testing and virtual testing (albeit at minimum levels) utilizing local instructors as proctors for TABE online remote testing. In FY 20/21 the State transitioned to TABE CLAS-E as the only approved assessment for ESL participants. Along with this transition was a requirement that all participants be given a full battery TABE CLAS-E assessment. Several of our larger ESL providers struggled with this as instructors insisted that full battery testing took too much time. The State and the local program director, working in collaboration, addressed this through localized training and increased monitoring. By the end of the year, there was improvement in full battery testing. This will continue to be address in FY 22/23.