U.S. Department of Labor
Office of Inspector General
Office of Audit
Report No. 06-98-003-03-340
Date Issued:  February 6, 1998

MEMORANDUM FOR:             RAYMOND J. UHALDE
                                                      Acting Assistant Secretary for
                                                           Employment and Training
 
                                                            / s /
FROM:                                         JOHN J. GETEK
                                                       Assistant Inspector General
                                                           for Audit

SUBJECT:                                 Audit of Central Texas Council of Governments
                                                      JTPA Title III Demonstration Grant No. 5502-5-00-87-60
                                                      Final Audit Report 06-98-003-03-340

The attached subject final report is submitted for your resolution action. We request a response to this report within 60 days.

Your office is responsible for submitting the report to the grantee. However, we have provided a courtesy copy of the report to the grantee this date.

If you have any questions regarding this report, please contact John Riggs, Regional Inspector General for Audit, Dallas, at 214-767-6980.

Attachment
 

Page

EXECUTIVE SUMMARY                                                                                                                        1

BACKGROUND, OBJECTIVES, SCOPE, AND METHODOLOGY                                                2

FINDINGS AND RECOMMENDATIONS                                                                                             3

1.  Cash stipends represented 20 percent of grant expenditures and
were provided to the CMA participants regardless of need or the
amount of time in class.                                                                                                                   3

2.  The grantee failed to terminate CMA participants 90 days after
the final service was provided.                                                                                                        5

3.  Program outcomes for control group participants were better than
for the CMA participants.                                                                                                                 8

ATCPIC         Austin/Travis County Private Industry Council

CMA              Career Management Account

CTCOG         Central Texas Council of Government

DOL               U.S. Department of Labor

ETA                Employment and Training Administration

JTPA             Job Training Partnership Act

OIG                Office of Inspector General

RFP               Request for Proposal

SDA               Service Delivery Area

SPIR               Standardized Program Information Record

SSG                 Substate Grantee
 

We conducted a limited scope performance audit of the Job Training Partnership Act (JTPA), Title III, Career Management Account (CMA) Demonstration Grant No. 5502-5-00-87-60 between the U.S. Department of Labor (DOL), Employment and Training Administration (ETA), and the Central Texas Council of Government (CTCOG), Service Delivery Area (SDA). Our objectives were to determine how the CMAs operated under this grant and to determine if there was a significant difference between the training strategies and outcomes for the CMA and control group participants.

We found that:

• Twenty percent of the grant funds were paid in stipends to all CMA participants regardless of need or amount of time spent in the classroom resulting in $117,785 of questioned costs.

• Participants were not terminated from the grant after 90 days of inactivity.

• Program outcomes for control group participants were better than the CMA participants.

The grantee's response attempted to provide justifications and explanations for the report findings but did not refute any facts provided in the report. The grantee's response is provided after each finding in the report and in its entirety at Appendix I. Our findings remain unchanged.

Recommendations

We recommend the Assistant Secretary for Employment and Training:

• disallow the $117,785 in questioned stipend payments during our audit period and any such additional payments after the audit period;

• ensure that all grantees operating demonstration grants currently, and in the future, follow guidelines regarding participants' termination from the programs after inactivity for 90 days to allow for non-biased comparisons of program outcomes between demonstration programs and regular JTPA programs; and
  
  
• encourage grantees operating Title III demonstration grants, and other Title III programs, to limit enrollment to those participants who are truly dislocated workers, not college students seeking a financing source for their education.

• increase customer choice,
• allow maximum flexibility in customizing services, and service delivery, to the needs of the individual dislocated worker, and
• expand the resources and sources of assistance which are available to the individual dislocated worker.

Our objectives were to determine how the CMAs operated under this grant and to determine if there was a significant difference between the training strategies and outcomes for the CMA and control group participants.

The audit period was July 1, 1995 through February 24, 1997, during which time 134 CMA and 67 control group participants were served. Since this demonstration grant was still in progress at the completion of audit fieldwork, we do not have the reported final performance statistics or dollar amounts expended.

We reviewed case files for a random sample of 76 of the 134 CMA participants and 36 of the 67 control group participants. The 112 participants in our sample consisted of 51 active and 61 terminated participants. Our purpose for the participant case file reviews was to determine what happened to the CMA and control group participants during their JTPA enrollment and compare the program outcomes for the two groups.

A separate Management Letter Report No. 06-98-004-03-340 was issued to the Assistant Secretary for Employment and Training related to ETA's management of this grant.

The audit was performed in accordance with the generally accepted auditing standards and Government Auditing Standards (1994 Revision) promulgated by the Comptroller General.
 

Neither the JTPA nor applicable regulations provide for the payment of participants' stipends without a determination of the participants' bona fide needs. Yet, by May 31, 1997, approximately $117,785 of the $578,612 grant funds (20 percent) disbursed had been spent on participants' stipends.

Title 20, Code of Federal Regulations, Section 631.20(b), provides:
In accordance with the approved substate plan, needs-related payments shall be provided to an eligible dislocated worker only in order to enable such worker to participate in training or education programs under this part. . . . [Emphasis added.]

The grantee mandated that these funds be used only for cash stipends. The grantee's policy was that each CMA limit for stipends was $1,000 but in practice the amount was usually greater if the length of training resulted in the participant exhausting this amount.
 

At least 15 of the 76 CMA participants (20 percent) whose files we reviewed attended training classes part-time, as few as 4 hours per week, yet still collected full stipends. Participants were required to have class attendance forms signed by instructors and returned to the SDA to receive a stipend. It apparently did not matter how many hours per week the participant was scheduled for training.
 

Office of Management and Budget Circular No. A-87 (1995 revision), Attachment A, paragraph C1, provides:

To be allowable under Federal awards, costs must . . . Be necessary and reasonable for proper and efficient performance and administration of the Federal awards. . . . [Emphasis added.]

Grantee's Response
 

"The ETA Request for Proposals states that the ultimate purpose of the Career Management Account Demonstration 'is to determine if career management accounts are an administratively feasible approach for achieving goals.' The RFP further states that 'demonstration grants are not subject to the cost limitations for formula-funded Title III grants at section 315 of the JTPA.... [however,] supportive services that exceed 25 percent of the funds requested in the application shall provide a narrative justification.' Additionally, while attending the initial grantees meeting in September 1995, [an ETA representative] directed grantees that funds are not 'through block grants; they are not bound by JTPA cost limitations, record keeping, or reporting, and they don't have to follow formula rules.' He further suggested that projects 'should document adjustments made to make a program work.' During the same grantees [sic] meeting, [another ETA representative] explained that while budget and extensive program modifications required written modification through grant officers, process modifications required discussing the modification with the grant program manager."
 

"When developing individual career management accounts, one of the barriers faced by CTCOG and Austin/Travis staff was the need to tell individual customers from the beginning of training how much funding is available for support services. The intent was to provide customers with the ability to make informed decisions based on a specific amount of funding to be used for supportive services and, to the extent possible, replicate a true vouchering system. The payment system, which required turning in attendance documentation, was devised as a means of ensuring attendance and participation while encouraging case management contact to document activities. The use of stipends and fixed amounts for supportive services, while not generally allowable under JTPA, Title III, is allowable as a JTPA, Title IIA, expenditure. Specifically, Section 627.310(2): (f) An SDA or SSG may set fixed levels of benefit for any supportive service. (g) (1) For purposes of title II, financial assistance is defined as a general supportive service payment for the purpose of retraining (sic) participants in training. (2) Financial assistance payments may be considered to be necessary for participation in training for title II participants, i.e., a separate, individual determination of need is not necessary."

Auditor's Conclusion

The grantee's response regarding cost limitations is not relevant to the reason the costs were questioned. We questioned these costs for the grantee's failure to assess each participant's needs before making stipend payments to all participants enrolled in training. We did not question the costs for a cost limitations exception.

The grantee's reference to Title IIA regulations is also not relevant to the finding. The JTPA Title IIA program embraces a different type of clientele (economically disadvantaged) than the
 

Finally, CMA funds earmarked for stipends were mandated by the grantee for that purpose and could not be used for such things as additional training. In our opinion, this mandate does not lend itself to a true voucher system.
 

Recommendation

We recommend the Assistant Secretary for Employment and Training disallow the $117,785 in questioned stipend payments during our audit period and any such additional payments after the audit period.
 
 2.   The grantee failed to terminate CMA participants 90 days after the final service was provided.

Of the 76 CMA participants whose case files we reviewed, 16 (21 percent) were not terminated after the participants failed to receive any training or services for more than 90 days while enrolled in the JTPA program.  These 16 participants should have been terminated negatively from the program.

Of these 16 participants, 4 were eventually terminated prior to completion of our audit fieldwork but were not terminated until after the participants found employment. Twelve participants (75 percent) still had not been terminated at the completion of audit fieldwork, having been inactively enrolled in the program as much as 1 year beyond the 90-day limitation. These facts appear to indicate that the grantee was not terminating participants as required to avoid negative terminations.

The failure to terminate these participants violates the JTPA Title III Standardized Program Information Record (SPIR) reporting requirements (Appendix C) -- an extension of the JTPA regulations -- which reads in part:

In Title III, after last receipt of either planned Basic Readjustment Service and/or Retraining, provision is made for a single period of up to 90 calendar days of inactive status prior to termination from the program.

DOL/ETA does not grant waivers for the required termination of JTPA participants beyond the time periods specified above.

Termination occurs when:

o An individual significantly fails to participate in their chosen service delivery by failing to attend, participate, or achieve in a manner that demonstrates effort on the part of the individual,

o An individual meets the goals identified in his/her Individual Service Strategy (i.e., attains GED), or
 

o Participant gains employment.

Grantee's Response
 

"The Career Management Account Request for Proposals proposed demonstration accounts that promoted: 'increased customer choice and allowed maximum flexibility in customizing services, and services delivery.' The RFP further states that 'the career management account is intended to provide maximum flexibility to the individual worker in selecting the types, timing and sources of career and employment assistance. The account is tied to the individualized worker.' Therefore, based on the criteria of the RFP, participation and/or inactivity had initially been identified by CTCOG and Austin Travis County partners as a factor of customer choice. However, Regional Department of Labor in Dallas conducted a site visit/audit in December 1996 and indicated that inactivity must result in termination and should not be considered a matter of customer choice."

"Prior to DOL review:
 

Because the control group was to be treated exactly as is expected under Title III, they were terminated for inactivity. However, because the experimental design was to reflect unlimited customer choice, experimental customers were not required to report regularly to case managers and were even given the option to select training/service timing. Stipends were based on attendance and at no time were stipends paid to those customers who were inactive."
 

An attempt was made to contact all inactive CMA customers. They were required to either begin participating or be terminated. No participants were inactive for more than 90 days after the December review and recommendation by DOL."
 

Auditor's Conclusion

We understand that the CMA should provide maximum flexibility. However, as reflected in the report, most of the participants in the "failure to terminate" category had either dropped out of the program and/or had moved and could not be contacted. This situation should not be considered a factor of a customer's choice but rather concrete grounds for terminating the participants.
 

The grantee's own proposal/statement of work calls for termination when "an individual significantly fails to participate in their chosen service delivery by failing to attend, participate, or achieve in a manner that demonstrates effort on the part of the individual."
 

We do not dispute the grantee's statement that an attempt was made to contact all inactive participants. In fact, many attempts were made to contact participants. However, we do dispute the grantee's statement that no cases were open more than 90 days after ETA's December 1996 review. The OIG's participant case file review took place during March/April 1997. While we cannot effectively comment on the cases reviewed in March (within 90 days of December 1996), the April case file reviews included some cases that had been inactive and still not terminated. For example, two participants relocated to other states while a third participant moved to another city within the state. These participants had not been successfully contacted since early to mid 1996 with their case files inactive for at least as long.

Recommendation
 

We recommend that the Assistant Secretary for Employment and Training ensure that all grantees operating demonstration grants currently, and in the future, follow guidelines regarding participants' termination from the programs after inactivity for 90 days to allow for non-biased comparisons between demonstration programs and regular JTPA programs.
 

The purpose of the CMA grant was to provide some participants greater flexibility in choosing their training curriculum and training provider. While there were exceptions, for the most part CMA participants were enrolled in the same training institutions and programs that were endorsed for the control group. Additionally, several of the CMA participants were already engaged in the training prior to JTPA enrollment, all of them attending college. It appeared that their primary motive to enroll in JTPA was simply to obtain the financial aid to continue their education. All of these participants were still enrolled in JTPA as of March 1997, with approximately 3 months remaining in the grant.
 

The major difference between the CMA and control group participants was the CMA group's ability to visually observe the funds credited and debited to their CMA; i.e., they knew how much JTPA funds had been spent on their behalf and how much was still available.
 

Although the CMA group was provided more flexibility, both the placement rate and starting hourly wage for the control group were significantly higher than for CMA participants.
 

Although we did not have access to the final performance data, we determined the outcome comparison for the two groups based upon an audited sample of 112 case files -- 76 CMAs, 36 control group. At the time audit fieldwork was performed, 78 percent (28 of 36) of the control group had been terminated from JTPA versus only 43 percent (33 of 76) of the CMA group. However, due to inactivity for more than 90 days, an additional 12 of the 33 active CMA participants should have been terminated. (See finding 2.) We found no evidence that any of the control group active participants had been inactive for more than 90 days.
 

Our determination that the control group out performed the CMA group is based on the following data.

• The SDA claimed 23 placements for the reported 28 terminated participants in the control group -- 89 percent rate. However, two claimed placements were for participants who were inactive for more than 90 days prior to termination; therefore, not a valid placement. Another claimed placement was actually a recall by the participant's former employer; not a valid placement. Consequently, only 71 percent of the control group (20 of 28) were valid placements.

• The SDA reported 30 placements for the reported 33 terminated participants in the CMA group -- 91 percent placement rate. However, an additional 12 active participants should have been terminated because of inactivity for more than 90 days. Therefore, 45 participants should have been terminated. Furthermore, the SDA claimed placements for 4 participants who were

inactive for more than 90 days prior to terminations and 3 participants were actually recalls by the participants' former employers. Consequently, only 51 percent of the CMA group who was terminated, or should have been terminated (23 of 45), were valid placements.

Additional evidence that the control group outperformed the CMA group is the difference in the average starting hourly wage for all participants claimed as placed:

• control group -- $12.65
• CMA group --$10.45

Grantee's Response

"As with any experimental design, outcomes vary based on experimental application -- Each experimental/control group was provided services in accordance with their groups service design. One could simply state that outcomes varied based on the effectiveness of approaches to service delivery. However, because of the time-limited nature of CMA funding several intervening factors have been identified as possible explanations for the diminished performance of the experimental group."

"Specifically, grant award was received on July 1, 1995 without prior notification and the initial grantee meeting followed in September. The program 'roll-out' was delayed accordingly resulting in participant enrollment beginning in October with two-year training institutions beginning Spring enrollment the following January. Therefore, CMA participants who selected two-years of training were not provided with enough time to expend funds and complete a two year training program. Because CMA criteria stresses customer choice, customers were given the option of selecting a shorter training program, possibly with less potential for success, or selecting a two-year training program and completing the program with alternate funding. While most customers selected two-year training institutions and programs very similar to those used by the control group, these selections could potentially diminish the outcomes of the experimental group in comparison to the control group -- Shorter training, for example, could result in (1) a lesser wage affecting wage at placement, or (2) a less desired employment opportunity contributing to increased customer drop out. Long-term training could result in (1) training completed after CMA termination which would reflect as a negative termination to CMA but a positive outcome for the customer, or (2) inability to continue training because of lack of available funding."
 

"Finally, experimental design, by definition, includes projected outcomes which may or may not be obtained. Variance in the performance of each of the groups has clearly been observed, but as stated above, due to the nature of the funding, there are multiple factor(s) which could be attributed to the variance."

Auditor's Conclusion

By applying JTPA Title III regulations and guidelines, we determined that the CMA group did not perform as well as the control group.

The customer groups may have been randomly selected, but the fact remains that all of the participants who were enrolled in training prior to enrollment in JTPA were in the CMA groups; none were in the control group. While we do not question the eligibility of these participants, some participant case files indicate that the only reason for enrolling in JTPA was to obtain financial aid to continue with their education. Also, three of these participants had obtained JTPA Title III eligibility status under the "long term unemployed" criteria, defined as unemployed 15 weeks of the 26 weeks immediately prior to JTPA enrollment. One might assume that if an individual was pursuing an education and not looking for work, that individual might very well be unemployed for this period of time.

The JTPA dislocated worker program, in our opinion, was never intended to serve as a means for individuals to obtain financial aid for the purpose of attending college in pursuit of various degrees, but rather to provide truly dislocated workers with appropriate services to help them obtain employment. To allow enrollment into JTPA of those participants already enrolled in college curricula, either for undergraduate or advanced degrees, diminishes available funds to those truly dislocated workers who
need the services to quickly return to the labor market.

Finally, we agree that there may be multiple factors which could attribute to the fact that the CMA participants performed worse than the control group. However, the fact remains that they did perform worse.

Recommendation

We recommend that the Assistant Secretary for Employment and Training encourage grantees operating Title III demonstration grants, and other Title III programs, to enroll only those participants who are truly dislocated workers.