State Leadership Funds (Adult education and family literacy act Section 223 (AEFLA))
AEFLA Section 223(1)(a)
During the 21-22, program year activities to align adult education and literacy with other core programs and one-stop partners remained unchanged from the prior year with limited involvement of adult education in other labor-led statewide career pathways strategies. However, the Adult Education Initiatives (AEI) team engaged in several activities around IET development to support adult learner access to career pathways in the 21-22 program year. AEI also found continued success in working with the Colorado Division of Vocational Rehabilitation to support ongoing training around serving individuals with disabilities across the workforce development system. We also partnered with the Colorado Department of Labor and Employment to develop online professional development and continued conversations with WIOA partners around data sharing.
In continuing to address the November 2019 federal corrective action plan, AEI expanded the IET toolkit for local providers and continued hosting a monthly workgroup dedicated to highlighting grant requirements and best practices. Grantees in Colorado were able to enroll learners in four Integrated Education and Training (IET) programs, focused on healthcare, retail sales, food industry, and early childhood education. The 2021-22 Colorado Talent Pipeline Report identified these industries as in-demand and areas of growth supporting Colorado’s economic recovery. Some grantees developed partnerships with local workforce centers, community colleges (some of which have adult education grantees on site), and employment partners. All IELCE grantees offering IET in 21-22 utilized staff to develop curriculum that aligns to the workforce training and had career navigators who helped learners develop educational and career plans to work toward goals.
WIOA partners continued to utilize the Programmatic Accessibility Moodle Course in the 21-22 program year, with over 400 individuals completing the course since 2019. This course was created in partnership with the Colorado Department of Labor and Employment and the Rocky Mountain Regional ADA Center and focuses on how to evaluate and improve access to a site and its services for individuals with disabilities.
In 21-22 the AEI team continued to host three Moodle courses in partnership with the Colorado Workforce Development Council. The “Remote Worker Certification for Supervisors and Leaders” and “Remote Worker Certification for Workers” Moodle courses focus on equipping workers and supervisors with the skills needed to work in remote environments. The “Colorado Career Conversation” Moodle course focuses on supporting individuals engaging in career conversations with students and job seekers to understand which careers are in demand, explore careers in depth, and articulate the path(s) to enter those careers.
Members of the AEI team continued to meet with Colorado Department of Labor and Employment Regional Services Unit staff quarterly in the 21-22 program year, and presented on Title II during WIOA Agency Cross-Training in February of 2022. Members of the AEI team also continued to meet with staff at the Colorado Department of Labor and Employment and the Colorado Department of Higher Education to update and ensure Data Sharing Agreements were in place for the 21-22 annual report. Preliminary data sharing conversations have also been held with the Data Trust organized by the Colorado Workforce Development Council and staff at the Colorado Department of Human Services, though no data sharing has yet taken place with these partners.
AEFLA Section 223(1)(b)
Professional learning in the 21-22 program year continued and expanded many of the offerings from the prior year.
Each month, five hours of training was provided for instructors and their administrators through Professional Learning to Chew On, Instructor’s Corner, and Learning Network online meetings. These trainings were developed and facilitated by trainers from Hamline University and Minnesota Literacy. Reading activities in these trainings included phonemic awareness, phonemes, phonics, fluency, comprehension, inference, decoding, and building vocabulary. Math and Universal Design for Learning were also included in these training sessions. Twenty instructors and administrators completed the year-long training. The training also included a focus on the College and Career Readiness Standards for Adult Education and the English Language Proficiency Standards.
Designers’ Club and Directors’ Talk continued to be offered monthly as 1-hour online meetings. Designers’ Club covered the following topics: distinguishing English language learning needs from learning disabilities; classroom accommodations and accessibility; multi-sensory training; and Universal Design for Learning. Directors’ Talk covered activities around metacognition and growth mindset and provided opportunities for adult education program directors to connect and share their knowledge and resources across the state.
AEI offered an online and no-cost version of the EDU 134 course which covers the College and Career Readiness Standards for Adult Education and the English Language Proficiency Standards, a required component in earning Colorado’s Adult Basic Education Authorization (ABEA). The course specifically covers: listening, speaking, reading, writing, pronunciation, vocabulary, and syntax. Over 70 instructional staff, who teach across the state, have completed the course since the Spring of 2022.
AEFLA Section 223(1)(c)
AEI disseminates instructional and programmatic practices through professional learning events (described above), in weekly “AEI Updates” email communications, and via bi-monthly Office Hours webinars. The bi-monthly Office Hours webinars provided training, best practices, and resources from subject matter experts. Topics included: CCRS-aligned instruction, stress management, the Colorado Talent Pipeline Report, early childhood education credentials and pathways, partnering with libraries, data analysis, and strategies to improve enrollment and retention.
To address adult education’s role to provide access to employment, education, and training services as a required one-stop partner, the AEI team continues to seek collaboration with the Colorado Department of Labor and Employment, the Colorado Department of Higher Education, and the Colorado Department of Human Services.
To continue increasing access to digital literacy and technology, AEI purchased Northstar Digital Literacy assessments and partnered with the Colorado Department of Labor and Employment to further provide digital assessments and training throughout the state. Local workforce centers also benefit from AEI’s Northstar Digital Literacy membership using the assessments and tools in local one-stops.
AEFLA Section 223(1)(d)
AEI conducted an annual risk assessment of grantees to determine risk but did not conduct onsite monitoring based on the results due to continued COVID-related restrictions. AEI conducted quarterly monitoring calls with all grantees to ensure alignment with AEFLA requirements and to identify best practices and innovative activities in programming. This collected information was provided to grantees in follow-up reports and during Office Hours. At the request of grantees, in-person monitoring occurred during these quarterly calls. Additionally, grantees received quarterly data performance reports displaying their key NRS data with trends, prior year comparisons, performance target reminders, and key recommendations.
In the 21-22 program year, AEI continued to monitor grantee data monthly to identify any concerns about enrollment, post-testing, and measurable skill gains (MSG). AEI utilized a custom virtual grantee dashboard in the statewide data system, LACES. This monitoring was used to provide technical assistance to grantees to support improved performance and accuracy in data reporting. AEI also increased awareness around the importance of data by highlighting specific data topics in each bi-monthly Office Hours webinar and during quarterly Data Talks webinars.
AEI oversaw the implementation of four IET programs in the 21-22 program year. The team worked with grantees to ensure compliance at every level: industry selection, development of shared objectives, and the implementation of co-enrollment. The IET toolkit AEI used in 21-22 focuses on alignment between the IET program and the CCRS. AEI also defined and documented its IET toolkit review and approval processes to aid grantees in progressing from IET design to IET implementation.
AEFLA Section 223(a)(2)
In 21-22 AEI team members reviewed data from Colorado’s Labor Market Information Gateway, the 2021-22 Talent Pipeline Report, Colorado’s Eligible Training Provider List and O*Net to identify in-demand industries by regions for the development of a statewide IET which was accessible to all grantees. AEI narrowed its research to three industries - healthcare, early childhood education, and commercial driving. AEI prioritized healthcare IET development based on statewide demand, grantee request, and learner interest. The Certified Nurse Aide IET curricula development began in 21-22 and continued into the 22-23 program year.
Performance Data Analysis
From April through May annually, monthly data close out webinars and procedural resources were provided to grantees. At the local level, grantees were required to conduct end-of-year data reporting no later than the second Friday in July. Although data entered in LACES by grantees was monitored monthly for errors, AEI staff ran a final diagnostic report of learner data after that second Friday in July. Any errors identified were sent to grantees for correction in LACES by the end of July to officially close out the data reporting. Once the data was finalized and frozen, grantees were required to complete an Annual Performance Report (APR) either in writing or collaboratively through a virtual meeting. A majority of grantees elected to complete the APR collaboratively and these meetings took place in July and August of 2022. The APR required local grantees to analyze enrollment, post-testing, and MSG outcomes. This analysis covered all program types, Educational Functioning Level (EFLs), classes, and included a prior year comparison to identify areas of strength and areas needing improvement. Grantees were encouraged to document in the report outliers and anomalies as well as any plans for further research and improvement.
Grantees have access to mostly finalized Table 4 results in LACES upon the completion of their data close out in July annually. Due to current limitations with data matching partners, Table 4 postsecondary entrance MSGs and all Table 5 employment and postsecondary credential data are not available to grantees during data close out. Instead, grantees receive final aggregate program level Tables 4 and 5 after AEI submits data to the NRS. During annual Table 4 and 5 training offered in the fall and during bi-monthly Office Hours virtual meetings, grantees are encouraged to compare their Table 4 & 5 data to statewide totals and the performance of other AEFLA-funded programs.
The AEI team continues to share current, year-to-date grantee MSG rates quarterly through Office Hours meetings to provide more opportunities for grantees to visualize where their performance ranks across the state. Each also receives an individual report each quarter to show how their data compares to statewide totals and averages on a series of progress measures and outcomes including enrollment, retention to 12 hours, average instructional hours, post-testing rates, and MSG rates by level.
Participants increased from 2,973 in 20-21 to 3,715 in 21-22, with 10 out of 14 grantees increasing their participants in the 21-22 program year; and we anticipate continued gradual growth over the next 2 years as impacts of the last competition for funds and the pandemic lessen. Additionally, 11 of the 14 grantees exited fewer learners in 21-22 than compared with 20-21. The focus on retaining learners through graduation or completion was, and continues to be, a focus of statewide technical assistance and training. Improvements are needed in reducing the numbers of learners who exited specifically before making any Measurable Skill Gain as 10 of 14 grantees saw increases in this rate from the prior year.
Statewide, the MSG rate increased just slightly from 44.2% in 20-21 to 44.4% in 21-22. Twelve of 14 grantees met or exceeded the 38% MSG target for the 21-22 program year. Asian Pacific Development Center, located in the Denver Metropolitan area, and Pikes Peak Library District, located in Colorado Springs, both fell below that target at 37.43% and 37.61%, respectively. Neither grantee has been subject to technical assistance as they met 90% of the target, a threshold used by AEI to determine if required technical assistance would be implemented. Asian Pacific Development Center makes up roughly 20% of the statewide participants so technical assistance in pulling MSG rates by instructional staff and regularly monitoring these rates was provided to the grantee in July 2022. Staff turnover at Asian Pacific Development Center has presented challenges for the grantee to implement this regular monitoring.
Post-test EFL gains once again made up the majority of MSGs in 21-22 with over 88% of MSGs coming from post-testing. Post-testing increased statewide from 67% in 20-21 to 69% in 21-22. High school diploma and equivalency gains made up a little over 10% of all MSGs. Postsecondary entrance made up just 1% of MSGs and IET MSGs made up less than 1% of all MSGs in 21-22. Overall, we saw a decrease in MSG rates for ABE learners with decreases specifically at Levels 2-4 (levels 2-4 contained the majority of ABE learners in 21-22). We saw increases at Levels 1 and 5 but these groups make up far fewer of our ABE learners statewide. We have not yet identified causes for the decreases at Levels 2-4. For ESL, overall and at all levels except for ESL Level 5, we saw increases in MSG rates. Like ABE, we have not been able to identify potential causes for the decrease at ESL Level 5.
Statewide, Colorado met our 24% employment in the 2nd quarter after exit target as well as the associated $5,176.00 median earnings in the 2nd quarter after exit target at 25.40% and $6,226.00, respectively. Both are increases from the prior year. Similarly, Colorado met our 12% credential attainment rate target with a rate of 22.72% in 21-22, also an increase from the prior year. Work continues with grantees to focus on increasing these exit-based outcomes through stronger relationships across WIOA partners, in-house career coaching services, the development of more and more diverse IETs and workforce training, and expanding the match criteria and increasing the frequency of data matches.
For the 26% employment in the 4th quarter after exit target, we once again did not meet the target with just 19.82% of learners in the cohort being employed 12 months after exit. Of the cohort, a little over 42% of learners reported a Social Security Number (SSN). Currently, this is the only match criteria the Colorado Department of Labor and Employment is willing to match on which increases the challenge in meeting employment targets.
Conversations are ongoing with the Data Trust convened by the Colorado Workforce Development Council on accessing employment and wage data quarterly and on using additional learner demographics to match with the Unemployment Insurance Compensation data set. AEI continues to work with grantees on processes for collecting acceptable forms of supplemental wage information, especially with learners who are not able or willing to report an SSN.
Eighty-five percent of the cohort exited before July 2020 during the height of the pandemic and before the latest AEFLA competition. It seems that adults exiting due to the pandemic faced additional barriers in obtaining or retaining jobs a year later. Additionally, regardless of exit date in the cohort, 38% of the cohort were enrolled in programs that did not continue to receive AEFLA funding after June 2020. These grantees had little incentive to support learners in pursuing Table 5 outcomes once they were no longer receiving AEFLA funds.
AEI has had preliminary conversations around how to best support grantees who were not able to meet or exceed the Table 5 targets. Current considerations include adding Table 5 targets to the state’s risk assessment and monitoring tools, implementing supplemental employment and wage data collection, and ongoing work with grantees on SSN data collection.
Integration with One-stop Partners
For WIOA Title II, the Colorado Department of Education (CDE) is the required one-stop partner responsible for administering or supervising policy for adult education and literacy activities in the State. As outlined in the jointly developed Colorado Policy Guidance Letter#: WIOA-2016-02, Change 2, published on May 11, 2017 by CDLE, CDE delegated its one-stop partner responsibilities to each of the AEFLA-funded local provider and set the expectation (through grant agreements) that each provider negotiate and sign an MOU that addresses the service delivery elements required by WIOA with the local workforce development boards (LWDB) in each local area in which adult education services are provided. Each required one-stop partner is responsible for identifying the career services that are relevant to their programs and making those services available through the comprehensive workforce center.
In 21-22 AEI collected the Memoranda of Understanding (MOU) between local workforce development boards and grantees. These outline the infrastructure agreements with the centers, per the Colorado One-Stop System Policy Guidance Letter#: WIOA- 2016-03, Change 1 published on May 2, 2017. These MOU, including IFAs, were reviewed by AEI staff to check for adherence to the Policy Guidance Letter and to all WIOA requirements.
Integrated English Literacy and Civics Education
IELCE Funds and grants
AEI held the competition for 2020-24 in 2019-20 and the competition materials, including the Request for Applications, were reviewed and approved by the U.S. Department of Education Office of Career, Technical, and Adult Education, as outlined in the Corrective Action Plan in response to the October 2018 virtual review. The request for proposals was released on January 31, 2020. Eligibility confirmations (for demonstrated effectiveness) were due February 24, 2020. Due to COVID-19, the competition was temporarily paused and eligible providers were required to submit applications by June 8, 2020. Intents to Award were released on July 27, 2020. Grantees were evaluated on their applications which required demonstration of effectiveness, evidence of need in local areas, proposed instructional services, satisfaction of IET components and integration with the local workforce system. Six grantees were awarded IELCE funding and received funds for 2021-22 totaling $917,622.
Training activity
In 21-22, four of the six IELCE-funded programs were able to implement IET programming in the second year of the grant cycle. Five of the six grantees completed AEI’s required IET toolkit by June 30, 2022. Those five grantees now have approved IETs. The remaining grantee will receive continued technical assistance to complete and offer an IET in the next program year. Two AEI staff members lead the oversight and monitoring of the IELCE program in Colorado.
AEI used section 243 funds to establish IET programs. A combination of Section 243 and Section 231 funds were used for one IET in 2021-22. All IELCE grantees participated in the monthly online IET Workgroup to support the completion of the IET toolkit and continuous improvement in the development and implementation of IETs. In 21-22 AEI developed an asynchronous online IELCE Moodle course that IELCE grantees are required to complete once per grant cycle. This course will be released in 2022-23. AEI staff provided onsite technical assistance to one grantee struggling to design a healthcare IET including all required components. The process developed for this TA can be replicated with others as needed.
Through quarterly monitoring conversations and the monthly IET workgroup, grantees have identified the two greatest areas that are still challenging in developing IETs. The first challenge was if enough learners with similar interests were enrolled to create a specific IET program for one industry. The second was recruiting learners to participate in IETs. It may be that learners are struggling to see the value in certification and/or that they are unable to meet the requirements and time commitments of IETs. AEI structured two of its monthly IET Workgroup meetings around these topics to provide technical assistance related to combining learner feedback with local and state data to select in-demand industries and to share and discuss learner recruitment strategies and best practices. IETs offered in 2021-22 included early childcare education, certified nurse aide, food industry training, and the National Retail Federation’s certificates.
IELCE Section 243(c)(1)
In 21-22, 553 adult learners participated in IELCE programs administered by six grantees. There were 13 participants enrolled in IETs: Childhood Development Associate (4), Food Services (3), Retail Fundamentals (3), Certified Nurse Aide (3). Seven of the thirteen participants enrolled in IETs achieved an MSG other than EFL gain or secondary school diploma/equivalent. Four of the MSGs were Progress Toward Milestone MSGs and three were industry recognized credentials.
In 21-22 IELCE grantees finalized preparations for two new IETs in the early childhood education and healthcare industries for implementation in 2022-23. All IELCE grantees offering IET have career navigators who help learners develop educational and career plans to work towards goals. The navigators not only help learners connect with services, but help them identify what steps need to be taken to reach goals. AEI is hopeful that the IET Workgroup will result in more diverse, high quality IETs offered and more participants in 2022-23.
IELCE Section 243(c)(2)
The competition for the 2020-24 grant cycle included the review of local applications by their respective local workforce development boards. Applications were reviewed for alignment with: regional needs, strategies and goals identified in the local plan, proposed activities to reduce barriers to employment, and one-stop partnership responsibilities and referral processes. The boards were given the opportunity to make recommendations to promote alignment during the review process.
In 21-22 AEI worked with the Colorado Department of Labor to hold quarterly meetings to discuss policies, strategies, and best-practices for learner engagement across WIOA partner agencies, including IELCE learners. From these meetings, topics were incorporated into grantee technical assistance regarding one-stop responsibilities and best-practices.
Adult Education Standards
In 21-22, no changes were made to Colorado’s adopted standards for adult education. Adult Education providers in Colorado use the College and Career Readiness Standards (CCRS) and English Language Proficiency Standards (ELPS). Both the CCRS and ELPS have been shared with the Standards and Support Unit at the Colorado Department of Education and were determined to be in alignment with Colorado’s K-12 standards which can be found here: https://www.cde.state.co.us/cdeadult/grantees/handbook/standards
The Colorado Adult Basic Education Authorization (ABEA) provides training in CCRS and ELPS standards. To earn the ABEA, instructional staff must complete four college courses provided by institutions of higher education (EDU 131, 132, 133, and 134). Instructional staff also have the option to complete the course requirements through prior learning assessments, observation, and free online versions of the courses hosted by AEI. The content of the EDU 134 course focuses specifically on the ELPS by connecting content that is relevant to English language learners.
AEI requires each grantee to identify a staff member to serve as the grantee’s local Professional Learning Coordinator. AEI offers training to local Professional Learning Coordinators on how to observe standards in classrooms, and requires grantees to outline their processes for observation annually in their local Professional Learning Assurances and associated plan.
Programs for Corrections Education (AEFLA Section 225)
Community Education Outreach (CEO) was the only provider using Section 225 funds in 21-22. Their programs only offered Adult Basic Education services in 21-22. AEI has not been able to track rates of recidivism on correctional education participants in 21-22 or in prior years. The Colorado Department of Corrections (CDOC) defines recidivism as a return to prison or inmate status in Colorado within three years of release, which can be for either new criminal activity or a technical violation of parole, probation or non-departmental community placement. The team has not yet coordinated with CDOC to track prior year participants at this three-year mark to determine how many recidivated. We would also like to explore conversations with WIOA partners around if and how they are already tracking this data. Based on exit exclusions reported in LACES for correctional education participants in 20-21 and 21-22, so far less than five percent of correctional education participants recidivated.