The Demand for, and Impact of, Youth Internships: Evidence from a Randomized Experiment in Yemen

This paper evaluates a youth internship program in the Republic of Yemen that provided firms with a 50 percent subsidy to hire recent graduates of universities and vocational schools. The first round of the program took place in 2014 and required both firms and youth to apply for the program. The paper examines the demand for such a program, and finds that in the context of an economy facing substantial political and economic uncertainty, it appears there is an oversupply of graduates in science, technology, engineering, and mathematics, and a relative undersupply of graduates in marketing and business. Conditional on the types of graduates firms were looking to hire as interns, applicants were then randomly chosen for the program. Receiving an internship resulted in an almost doubling of work experience in 2014, and a 73 percent increase in income during this period compared with the control group. A short-term follow-up survey conducted just as civil conflict was breaking out shows that internship recipients had better employment outcomes than the control group in the first five months after the program ended.


Introduction
Youth unemployment is a global problem, and is particularly high among educated youth in the Middle East and North Africa (World Bank, 2014). Youth often face a catch-22 whereby firms are reluctant to hire them without job experience unless they have the right connections, but without being hired they cannot get this experience. The returns to this experience are believed to be high because initial jobs provide on-the-job training for youth in both job-specific skills, as well as in soft skills. But since many young people may leave a firm once they have received this experience, there is a concern that firms underprovide job experience from a public policy perspective. Firms may be particularly reluctant to invest in training workers in fragile and conflict-affected states, where political and economic uncertainty makes many types of investments additionally risky (Blattman and Ralston, 2015). This paper reports on a randomized experiment testing an internship program in the Republic of Yemen that was intended to help overcome these constraints and provide employment experience to educated youth. The program provided firms with a 50 percent subsidy towards the cost of employing an intern for six months, and a matchmaking service to link firms seeking intern positions to graduates looking for employment opportunities. The first year of this program took place in 2014, with a second year of the program canceled due to civil conflict breaking out in the country in 2015.
Evaluating the first year, we have four main findings. First, there appears to be a mismatch between the supply of skills of interns (heavily concentrated in STEM fields like engineering and IT) and the demand for these skills from firms (which required relatively more marketing and business graduates). Second, the internship program did generate new job experience, rather than just substituting for the employment applicants would have received any way. We find recipients of the internship had a 42 percentage point increase in the likelihood of working during the internship period, a 3.4 month increase in the amount worked, and a 73 percent increase in earnings during 2015. Third, we were able to field a short-term post-treatment follow-up survey just as conflict was starting, providing evidence on impacts up to five months after the internship. The results 3 show that the internship does appear to have also improved employment outcomes in this posttreatment period. Finally, we examine heterogeneity in impacts by gender. Statistical power for this comparison is low, but the results show that in the absence of the program female applicants are less likely to work than males, and that, if anything, the internship has more impact for males and so does not close this gender gap.
This paper contributes to a broader literature on the impacts of labor programs on employment in developing countries. Blattman and Ralston (2015) provide a recent review, and note that it has been hard to find skills training programs that pass a cost-benefit test. To our knowledge this is the first randomized experiment to examine the impact of just an internship program in a developing country, despite such programs being used by a number of governments. 1 In related work, Card et al. (2011) and Attanasio et al. (2011) provide results from experiments in the Dominican Republic and Colombia, respectively, that included three months of vocational training followed by two to three month internships, finding no increase in employment in the Dominican Republic, and a 7 percent increase for women in Colombia (but no increase for men) -with it being unclear how much of this effect comes from the vocational training, compared to the internship components.
Another strand of related literature provides wage subsidies to job-seekers that they can take to firms (Galasso et al., 2004;Groh et al, 2013;Levinsohn et al, 2013). Only in the Jordan case studied by Groh et al. (2013) were these subsidies actually redeemed by firms, and they find no significant post-program impacts on youth employment. The key difference with the type of internship program studied in this paper is that it is demand-driven, seeking to match applicants to jobs that firms are actively interested in filling. 2 In addition, it provides evidence from a poorer and more conflict-afflicted setting than these existing studies.
The remainder of the paper is structured as follows. Section 2 discusses the country context and details of the internship program. Section 3 compares the supply of and demand for interns. Section 4 discusses the intern selection process, the randomization procedure, take-up, and data collection.
Section 5 provides the results and Section 6 concludes.

Context and Program Details
The Republic of Yemen is the poorest country in the Arab world, with a per-capita GDP of US$1,473 (2013), more than half the population living below the poverty line, and a ranking of 154 of 187 countries in the Human Development Index (2011). In the two years preceding the launch of the program studied here, the Republic of Yemen had undergone a revolution as part of the Arab Spring, marked by protests, armed clashes and a contraction of the economy. In this environment, young graduates faced even more than the usual difficulties finding jobs. In common with elsewhere in the region, unemployment rates were estimated to be higher for educated youth relative to less skilled youth, and the female wage employment in non-agricultural sectors rate of 12 percent is the lowest in the world.

Internship Program
The Enterprise Revitalization and Employment Pilot (EREP) was designed as a two-year pilot project to help firms and recent graduates seeking jobs. The internship component of this project aimed to place 400 Yemeni youth into internship jobs with firms in Yemen, 200 in the first year and 200 in the second year, with each year the number equally divided between the two main cities of Sanaa and Aden. These internships were funded through matching grants to firms applying to the program, in which the program would pay half the costs of the internship (for up to 6 months) and the firm the other half. The program would also facilitate the process of firms finding these workers, through having recent graduates apply to the program, providing a short basic skills training course to them, and providing a matchmaking process to match these applicants to positions operated by firms. To be eligible for an internship through the program, individuals had to be Yemeni, live in either Sanaa or Aden, and have graduated from university or a vocational school in the past two years.

Program Launch
The program was launched on August 2013 by the Small and Micro Enterprise Promotion Service (SMEPS). This occurred through a project website (www.wathba-smeps.org), media events (tv, 5 radio, and newspaper), Facebook and Google Ads, workshops at universities, vocational schools, and business associations. Applicants had to fill out an online application between October 1, 2013 and December 26, 2013 (see timeline in appendix 1). The application form included contact information, schooling and field of study, work experience, and the type of work the applicant was interested in. The data from these applications form the key baseline information for our study.
Applications by youth were then subject to a simple screening process to ensure they met the eligibility criteria. The screening process started on December 29, 2013, and finished on January 5, 2014. At the end of this process, SMEPS had a total of 3,487 eligible applicants interested in undertaking an internship.

Conflict Resumes
The first year of the program took part in 2014, with internships taking place as described below.
However, in August 2014, halfway through these internships, Houthi rebels began demonstrations in Sanaa against increased fuel prices, and in September they took control of the city of Sanaa. A U.N.-brokered peace agreement was made in which they agreed to withdraw once a national unity government was formed. The situation worsened at the start of 2015, with the Houthis seizing control of the state television station, President Hadi resigning and fleeing to Aden, and conflict involving competing Yemeni factions and regional powers breaking out in late March and early April 2015. As a result of these events, the second year of the program was canceled, and as a result, this analysis focuses on the first year of the program.

The Supply of, and Demand for, Interns
We examine the characteristics of who applied for internship jobs, and then compare this to the types of jobs that firms were looking for, to examine where there was a mismatch between supply and demand for skills, and to characterize the participants in the program.

The Supply of Interns
Column 1 of Table 1 provides some basic characteristics of the 3,487 youth applying to the program. The mean and median age of the applicants is 26, with 90 percent being between 23 and 30. Thirty percent of the applicants were female, and 47 percent are from Aden (with the rest in Sanaa). The majority have a university education, rather than just vocational training, and 82 percent claim to be able to use a computer, although only half of that have more advanced computer 6 skills in terms of knowledge of specific software like CAD, Excel, Photoshop, or others. The most common specializations based on the studies individuals have undertaken are engineering and information technology (43 percent), and business (27 percent).
The first six columns of Table 2 examine the potential supply of interns in more detail, breaking down the specializations by gender and city. We see that males and females are equally likely to have taken business majors, whereas males are more likely than females to be engineering and information technology majors, and less likely to be English or teaching majors. These gender differences occur in both cities, and in general the supply of potential interns does not exhibit large differences across the two cities in terms of specialization.

The Demand for Interns
The remaining three columns of Table 2 show the demand for interns according to their specialization, for the internship positions that were actually filled. 3 Forty-five percent of all interns demanded were in positions that required a major in business. This included marketing, accounting, and general business administration. Twenty-five percent were in engineering and information technology, and 30 percent in other fields such as English and medical fields.
Comparing the potential supply of interns from the applicants to the program, and the demand in terms of positions filled, we see some mismatches between supply and demand. In particular, there appears to be an excess supply of graduates in the STEM fields of engineering and information technology, and an under-supply in business fields. In particular, marketing and business administration constitute more than four times the proportion of positions demanded as they do of potential supply, while civil engineers and information technology majors constitute less than a fifth of the proportion of positions demanded as they do of potential supply.
This mismatch of supply and demand in part reflects the volatile political and security situation in Yemen. In this environment the construction sector had halted a lot of activity, reducing the demand for engineers, whereas the retail sector was less affected and still had demand for graduates 7 to help them run their businesses and sell their products. However, it also reflects more structural factors in an economy with a weak manufacturing base and limited ICT sector.

The Intern Selection Process, Randomization, Take-up, and Data Collection
After firms had indicated their demand for interns, these jobs were mapped to specializations of study. For some positions there was a single specialization that was relevant (e.g. a firm requesting interns for a dentistry position had this job mapped just to applicants who had studied dentistry), while for others several specializations were considered suitable (e.g. for firms requiring interns in business administration positions, the specializations of business administration, office management, and management were all considered suitable). Based on this potential demand, SMEPS randomly chose a sample of 500 applicants with probability proportion to demand.
These 500 applicants were invited to attend a two-day basic skills training which occurred in

Randomization Process
The 500 applicants invited to the two-day training were randomly assigned to treatment and control groups. Random assignment effectively occurred within strata defined by specialization and city.
In all there are 231 such strata, but many of these contain few individuals in specializations with no market demand, so only 39 strata were used.
The selection algorithm proceeded as follows: First, each job was mapped to the set of suitable specializations. Then for each type of position (e.g. business administration positions in Sanaa), the aim was to allocate randomly the pool of applicants invited to training to treatment and control with a 1.5:1 ratio. More individuals were to be allocated to treatment than control to give firms a choice of candidates to choose amongst. However, for some positions there were insufficient graduates to pursue this strategy. These positions were then mapped to secondary and tertiary specializations. The selection procedure would then choose anyone with the primary specialization for treatment with certainty, before then randomly selecting from secondary and tertiary specializations. The individuals chosen with certainty then would be part of the program, but not 8 part of the experimental sample. Finally, additional individuals who had not gone through the twoday training were randomly selected to boost numbers for some positions. Randomization was done internally at SMEPS using Excel.
The resulting experimental sample consists of 583 youth, of which 430 (318 male, 112 female) were allocated to treatment, and 153 (105 male, 48 female) to control. The second and third columns of Table 2 provide summary statistics for these groups. Note that the probability of being treated varies across strata, ranging from 37.5 percent to 96.6 percent. As such, a simple comparison of means shows some differences between the two groups. For example, 7 percent of the treatment group are marketing graduates compared to 3 percent of the control group, reflecting the shortage of marketing graduates so that these individuals had higher chance of being allocated to treatment.
To account for this randomization within strata, our analysis will follow the recommendation of Bruhn and McKenzie (2009) and condition on strata (specialization*city) fixed effects. To estimate the impact of being assigned to the internship treatment group on outcome Y, we will then estimate:

∑
(1) Where T denotes assignment to the internship treatment, and I(iϵs) is a dummy variable which takes value 1 if individual i is in strata s, and zero otherwise. Column 4 of Table 1 reports the pvalue for testing the treatment effect is zero (i.e. the groups are balanced) through estimating equation (1) on our baseline variables. It shows that the treatment and control group are balanced on key characteristics like gender, age, and education after this correction.

Matchmaking and Take-up
Eleven human resource advisors were recruited by the program to help match individuals in the treatment group to firms seeking jobs. The advisors talked with the firms to check which qualities they were seeking in workers, and then called and pre-screened youth in the treatment group to identify a set of individuals that matched the firm's criteria. The CVs and details of these candidates were then sent to firms, and firms invited youth to interviews, and decided whether or not to make them an internship offer. If an individual in the treatment group was not selected by a firm, they went back into the pool of available treatment group individuals that the advisor could then attempt to match to another job.
In total 334 interviews took place, and 198 interns were accepted by firms into jobs. 4 Some of these were accepted with certainty due to insufficient numbers to also allow a control group, and so were not part of the experimental sample. In total 158 of the 430 individuals in the experimental treatment group received an internship (36.7 percent). The take-up rate did not differ significantly by gender (34.8% for females, 37.4% for males, p-value for test of equality is 0.625).
There were three main reasons why some individuals assigned to the internship treatment group did not receive an internship. First, by design there were more individuals assigned to treatment than there were internship jobs available, in order that firms could be offered several candidates to interview and find a good fit. Second, some of the youth assigned to the internship program dropped out either because they found a job without the internship, because they couldn't be reached, or because of other circumstances. Third, some of the firms that had been selected to receive matching grants for interns dropped out of the program. This resulted because several months had passed since they had applied, and either they had recruited directly from the market during this time, or had decided as a result of the changing economic situation that they could no longer afford to offer this position. When this happened SMEPS attempted to replace the firm with another firm requesting graduates from the same specialization in the same city, but in some cases this was not possible, and as a result some of individuals assigned to treatment would be in specializations where there was no longer any demand for them.

Data Collection
We use three main sources of data. The first consists of baseline information available from the applications. The second consists of administrative data from the program, which provides key information on initial take-up of the internship, and internship completion rates. The third and main source of data comes from a follow-up survey. We had originally planned surveys at 6 months and 12 months after the end of the internship. However, with the worsening situation in Yemen, we quickly launched a follow-up survey in March 2015. This timing took place 3-6 months after the end of the internships, and right before civil conflict broke out. The survey was conducted as a phone survey given that safety issues, gas shortages, and time constraints prohibited an inperson survey. The survey was conducted by Apex consulting, and had a response rate of 78.7% (78.1% treatment, 80.4% control, p-value of test of equality 0.666). We believe this response rate to be excellent given the difficult conditions, and given there is no statistical difference in response rates by treatment group, assume attrition is at random for the purposes of analysis.
The survey asked questions about employment during 2014 to capture the time of the internship, and then questions about employment in the months of December 2014 and February 2015 to capture post-treatment employment outcomes.

Results
We begin with descriptive information on what took place during the internships, followed by analysis of the impact of these internships on employment outcomes.

What did an internship entail?
The modal intern worked six months as an intern. According to the administrative data, 59 of the 198 interns placed did not work the full six months because either the firm decided to no longer retain them, or they decided to leave. In the follow-up survey, 64 percent worked in the internship position for 6 months, while 24 percent worked for three months or less.
Interns were typically working around 40 hours per week, Sunday to Thursday. The jobs appear very similar in terms of what they entailed to the types of jobs the control group was doing. Table   3 uses the survey data to compare the different characteristics of work for those who worked in the treatment group in 2014 to those who worked in the control group. This conditions on working at all (the next subsection examines the treatment effect on work).
We see that the two groups are statistically similar in terms of the characteristics of their jobs conditional on working. The jobs pay an average of 55,000-59,000 rials per month (approximately US$255-270). They are mostly with private employers, in jobs related to the studies of the intern.
In at least 80 percent of the cases, the intern says the job involves using a computer, that they learned technical skills on the job, and that they learned organizational skills on the job. However, as with jobs taken by the control group, three-quarters of the interns also say that they spent at least two hours per day on tasks they considered beneath them, and 59 percent were idle at the job at least five hours or more per week. The internships therefore appear to have involved the same mix of learning opportunities and less learning oriented activities as a typical job for this group.

Impact of Internship on Employment during the Internship Period
In order for the internship program to increase employment, it must result in additional employment for the applicants, rather than just substituting for jobs they would have found anyway. To estimate the impact of being assigned to the internship treatment on employment (the intention-to-treat effect or ITT) we estimate equation (1) for several employment outcomes. We then also estimate the impact of actually taking up the internship (the local average treatment effect or LATE) by instrumenting take-up of an internship with the random assignment.  The final column pools these different indicators together as an aggregate index of standardized zscores, and finds a significant impact on employment using this measure. Table 5 examines the heterogeneity in impact by gender. Since we have only one year of the program rather than the originally planned two, statistical power is lower than planned. We see that female applicants are less likely to work at all, work fewer months, and earn less than male applicants among the control group. The program is found to have statistically significant impacts for male applicants. All the treatment interactions suggest that the program was less effective for female applicants, but due to the relatively small sample size, we cannot reject equality of treatment effects by gender.

Post-Treatment Impact
We then turn to examining whether the added work experience gained during this internship has impacts that last beyond the end of the internship. Recall that due to the outbreak of civil conflict in Yemen, our survey had to be fielded earlier than planned, making these relatively short-term post-treatment outcomes. The survey elicited information on work in December 2014 and February 2015. December 2014 is one to three months after the end of the internship, while February 2015 is three to five months after, but during a period where the Houthis had taken over the government in Sanaa.   (2012). Appendix 2 reports these results, which show the treatment group working a statistically significant 4.7 hours more per week than the control group, and positive, but not statistically significant impacts on working at all, and on income earned. show females in the control group continuing to have lower employment rates, and earning lower income than males. As with the results in Table 5, power is low, but the point estimates suggest the treatment impacts were larger for male youth than females, so the internship program is not helping close this gender gap.

Conclusions
The internship program succeeded in generating new job experience for applicants in a context of worsening political and economic instability in Yemen. The internship jobs appear similar in terms of learning experiences to the jobs the control group managed to get, just more people in the treatment group were able to experience them. The fact that those who received internships had better employment outcomes after the end of the program shows some post-program persistence in this effect.

13
The context also requires several caveats in interpreting these results. The civil conflict that erupted in 2015 resulted in the cancellation of the second year of the program, effectively halving our intended sample size and reducing our statistical power. It also required the follow-up survey to be conducted more quickly after the end of the program that we had planned, and for a shorter survey instrument to be used. Moreover, the armed conflict in 2015 has resulted in large-scale internal displacement, more than 80 percent of the population in need of humanitarian assistance, shortages of fuel, electricity, and water, and infrastructure damage (European Commission, 2015).
As a result, employment opportunities for the population in this study are likely to have been severely affected, potentially reducing any longer-term impact. Nevertheless, we believe it important to share the results available given the paucity of evidence on internship programs in developing countries.   Robust standard errors in parentheses, after controlling for strata fixed effects. *, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively. Robust standard errors in parentheses, after controlling for strata fixed effects. *, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively. Truncated earnings truncates at the 99th percentile. Robust standard errors in parentheses, after controlling for strata fixed effects. *, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively. Truncated earnings truncates at the 99th percentile.