The demand for, and impact of, youth internships: evidence from a randomized experiment in Yemen

Abstract

This paper evaluates a youth internship program in Yemen. We examine the demand for the program and find an oversupply of graduates in science, technology, engineering, and mathematics and a relative undersupply of graduates in marketing and business. Conditional on the types of graduates firms were looking to hire, applicants were then randomly chosen for the program. Receiving an internship resulted in an almost doubling of work experience in 2014 and a 73 % increase in income. A follow-up survey shows that internship recipients had better employment outcomes than the control group in the first 5 months after the program.

JEL codes: O12, O15, J08, J16

1 Introduction

Youth unemployment is a global problem, and it is particularly high among educated youth in the Middle East and North Africa (World Bank, 2014). Youth often face a catch-22 whereby firms are reluctant to hire them without job experience unless they have the right connections, but without being hired, they cannot get this experience. The returns to this experience are believed to be high because initial jobs provide on-the-job training for youth in both job-specific skills as well as in soft skills. But since many young people may leave a firm once they have received this experience, there is a concern that firms underprovide job experience from a public policy perspective. Firms may be particularly reluctant to invest in training workers in fragile and conflict-affected states, where political and economic uncertainty makes many types of investments additionally risky (Blattman and Ralston, 2015).

This paper reports on a randomized experiment testing, an internship program in the Republic of Yemen that was intended to help overcome these constraints and provide employment experience to educated youth. The program provided firms with a 50 % subsidy towards the cost of employing an intern for 6 months and a matchmaking service to link firms seeking intern positions to graduates looking for employment opportunities. The first year of this program took place in 2014, with a second year of the program canceled due to civil conflict breaking out in the country in 2015.

Evaluating the first year, we have four main findings. First, there appears to be a mismatch between the supply of skills of interns (heavily concentrated in Science, Technology, Engineering, and Mathematics (STEM) fields like engineering and IT) and the demand for these skills from firms (which required relatively more marketing and business graduates). Second, the internship program did generate new job experience rather than just substituting for the employment applicants would have received any way. We find recipients of the internship had a 42 percentage point increase in the likelihood of working during the internship period, a 3.4-month increase in the amount worked, and a 73 % increase in earnings during 2015. Third, we were able to field a short-term post-treatment follow-up survey just as conflict was starting, providing evidence on impacts up to 5 months after the internship. The results show that the internship does appear to have also improved employment outcomes in this post-treatment period. However, we are unable to assess whether this employment came at the expense of other youth in the country. Finally, we examine heterogeneity in impacts by gender. Statistical power for this comparison is low, but the results show that in the absence of the program, female applicants are less likely to work than males and that, if anything, the internship has more impact for males and so does not close this gender gap.

This paper contributes to a broader literature on the impacts of labor programs on employment in developing countries. Blattman and Ralston (2015) provide a recent review and note that it has been hard to find skills training programs that pass a cost-benefit test. To our knowledge, this is the first randomized experiment to examine the impact of just an internship program in a developing country, despite such programs being used by a number of governments.¹ In related work, Card et al. (2011) and Attanasio et al. (2011) provide results from experiments in the Dominican Republic and Colombia, respectively, that included 3 months of vocational training followed by 2- to 3-month internships, finding no increase in employment in the Dominican Republic and a 7 % increase for women in Colombia (but no increase for men)—with it being unclear how much of this effect comes from the vocational training compared to the internship components. Another strand of related literature provides wage subsidies to job-seekers that they can take to firms (Galasso et al., 2004; Groh et al. forthcoming; Levinsohn et al, 2013). Only in the Jordan case studied by Groh et al. (forthcoming) were these subsidies actually redeemed by firms, and they find no significant post-program impacts on youth employment. The key difference with the type of internship program studied in this paper is that it is demand-driven, seeking to match applicants to jobs that firms are actively interested in filling.² In addition, it provides evidence from a poorer and more conflict-afflicted setting than these existing studies.

The remainder of the paper is structured as follows. Section 2 discusses the country context and details of the internship program. Section 3 compares the supply of and demand for interns. Section 4 discusses the intern selection process, the randomization procedure, take-up, and data collection. Section 5 provides the results, and Section 6 concludes.

2 Context and program details

The Republic of Yemen is the poorest country in the Arab world, with a per-capita GDP of US$1473 (2013), more than half the population living below the poverty line, and a ranking of 154 of 187 countries in the Human Development Index (2011). In the 2 years preceding the launch of the program studied here, the Republic of Yemen had undergone a revolution as part of the Arab Spring, marked by protests, armed clashes, and a contraction of the economy. In this environment, young graduates faced even more than the usual difficulties finding jobs. In common with elsewhere in the region, unemployment rates were estimated to be higher for educated youth relative to less skilled youth, and the female wage employment in non-agricultural sectors rate of 12 % is the lowest in the world.

2.1 Internship program

The Enterprise Revitalization and Employment Pilot (EREP) was designed as a 2-year pilot project to help firms and recent graduates seeking jobs. The internship component of this project aimed to place 400 Yemeni youth into internship jobs with firms in Yemen, 200 in the first year and 200 in the second year, with each year the number equally divided between the two main cities of Sanaa and Aden. These internships were funded through matching grants to firms applying to the program in which the program would pay half the costs of the internship (for up to 6 months) and the firm the other half. The program would also facilitate the process of firms finding these workers through having recent graduates apply to the program, providing a short basic skills training course to them, and providing a matchmaking process to match these applicants to positions operated by firms. To be eligible for an internship through the program, individuals had to be Yemeni, live in either Sanaa or Aden, and have graduated from university or a vocational school in the past 2 years.

2.2 Program launch

The program was launched in August 2013 by the Small and Micro Enterprise Promotion Service (SMEPS). This occurred through a project website, media events (tv, radio, and newspaper), Facebook and Google Ads, workshops at universities, vocational schools, and business associations. Applicants had to fill out an online application between October 1, 2013, and December 26, 2013 (see timeline in Appendix 1). The application form included contact information, schooling and field of study, work experience, and the type of work the applicant was interested in. The data from these applications form the key baseline information for our study. Applications by youth were then subject to a simple screening process to ensure they met the eligibility criteria. The screening process started on December 29, 2013, and finished on January 5, 2014. At the end of this process, SMEPS had a total of 3487 eligible applicants interested in undertaking an internship.

2.3 Conflict resumes

The first year of the program took part in 2014, with internships taking place as described below. However, in August 2014, halfway through these internships, Houthi rebels began demonstrations in Sanaa against increased fuel prices, and in September, they took control of the city of Sanaa. A U.N.-brokered peace agreement was made in which they agreed to withdraw once a national unity government was formed. The situation worsened at the start of 2015, with the Houthis seizing control of the state television station, President Hadi resigning and fleeing to Aden, and conflict involving competing Yemeni factions and regional powers breaking out in late March and early April 2015. As a result of these events, the second year of the program was canceled, and as a result, this analysis focuses on the first year of the program.

3 The supply of, and demand for, interns

We examine the characteristics of who applied for internship jobs and then compare this to the types of jobs that firms were looking for to examine where there was a mismatch between supply and demand for skills and to characterize the participants in the program.

3.1 The supply of interns

Column 1 of Table 1 provides some basic characteristics of the 3487 youth applying to the program. The mean and median age of the applicants is 26, with 90 % being between 23 and 30. Thirty percent of the applicants were female, and 47 % are from Aden (with the rest in Sanaa). The majority have a university education rather than just vocational training, and 82 % claim to be able to use a computer, although only half of that have more advanced computer skills in terms of knowledge of specific software like CAD, Excel, Photoshop, or others. The most common specializations based on the studies individuals have undertaken are engineering and information technology (43 %) and business (27 %).

Table 1 Baseline characteristics

The first six columns of Table 2 examine the potential supply of interns in more detail, breaking down the specializations by gender and city. We see that males and females are equally likely to have taken business majors, whereas males are more likely than females to be engineering and information technology majors and less likely to be English or teaching majors. These gender differences occur in both cities, and in general, the supply of potential interns does not exhibit large differences across the two cities in terms of specialization.

Table 2 The supply of interns by gender and city and demand by city

3.2 The demand for interns

The remaining three columns of Table 2 show the demand for interns according to their specialization for the internship positions that were actually filled.³ Forty-five percent of all interns demanded were in positions that required a major in business. This included marketing, accounting, and general business administration. Twenty-five percent were in engineering and information technology and 30 % in other fields such as English and medical fields.

Comparing the potential supply of interns from the applicants to the program, and the demand in terms of positions filled, we see some mismatches between supply and demand. In particular, there appears to be an excess supply of graduates in the STEM fields of engineering and information technology and an under-supply in business fields. In particular, marketing and business administration constitute more than four times the proportion of positions demanded as they do of potential supply, while civil engineers and information technology majors constitute less than a fifth of the proportion of positions demanded as they do of potential supply.

This mismatch of supply and demand in part reflects the volatile political and security situation in Yemen. In this environment, the construction sector had halted a lot of activity, reducing the demand for engineers, whereas the retail sector was less affected and still had demand for graduates to help them run their businesses and sell their products. However, it also reflects more structural factors in an economy with a weak manufacturing base and limited ICT sector.

4 The intern selection process, randomization, take-up, and data collection

After firms had indicated their demand for interns, these jobs were mapped to specializations of study. For some positions, there was a single specialization that was relevant (e.g., a firm requesting interns for a dentistry position had this job mapped just to applicants who had studied dentistry), while for others, several specializations were considered suitable (e.g., for firms requiring interns in business administration positions, the specializations of business administration, office management, and management were all considered suitable). Based on this potential demand, SMEPS randomly chose a sample of 500 applicants with probability proportion to demand.

These 500 applicants were invited to attend a 2-day basic skills training which occurred in February 2014 in Sanaa and March 2014 in Aden. Four hundred seventy-seven of the 500 invited attended. The course covered interview skills, basic time management skills, and instructions for key work behaviors. Training was provided by SEEDS in Sanaa and Mothabroom in Aden. Both are institutions specializing in offering skills development courses, English courses, and entrepreneurship courses.

4.1 Randomization process

The 500 applicants invited to the 2-day training were randomly assigned to treatment and control groups. Random assignment effectively occurred within strata defined by specialization and city. In all, there are 231 such strata, but many of these contain few individuals in specializations with no market demand, so only 39 strata were used.

The selection algorithm proceeded as follows: First, each job was mapped to the set of suitable specializations. Then for each type of position (e.g., business administration positions in Sanaa), the aim was to allocate randomly the pool of applicants invited to training to treatment and control with a 1.5:1 ratio. More individuals were to be allocated to treatment than control to give firms a choice of candidates to choose among. However, for some positions, there were insufficient graduates to pursue this strategy. These positions were then mapped to secondary and tertiary specializations. The selection procedure would then choose anyone with the primary specialization for treatment with certainty, before then randomly selecting from secondary and tertiary specializations. The individuals chosen with certainty then would be part of the program, but not part of the experimental sample. Finally, additional individuals who had not gone through the 2-day training were randomly selected to boost numbers for some positions. Randomization was done internally at SMEPS using Excel.

The resulting experimental sample consists of 583 youth, of which 430 (318 males, 112 females) were allocated to treatment and 153 (105 males, 48 females) to control. The second and third columns of Table 2 provide summary statistics for these groups. Note that the probability of being treated varies across strata, ranging from 37.5 to 96.6 %. As such, a simple comparison of means shows some differences between the two groups. For example, 7 % of the treatment group are marketing graduates compared to 3 % of the control group, reflecting the shortage of marketing graduates so that these individuals had a higher chance of being allocated to treatment.

To account for this randomization within strata, our analysis will follow the recommendation of Bruhn and McKenzie (2009) and condition on strata (specialization*city) fixed effects. To estimate the impact of being assigned to the internship treatment group on outcome Y, we will then estimate

$$ {Y}_i=b{T}_i+{\displaystyle {\sum}_{S=1}^{39}{\delta}_SI\left(i\in S\right)}+{\varepsilon}_i, $$

(1)

where T denotes assignment to the internship treatment, and I(i∈s) is a dummy variable which takes value 1 if individual i is in strata s and 0 otherwise. Column 4 of Table 1 reports the p value for testing if the treatment effect is 0 (i.e., the groups are balanced) through estimating Eq. (1) on our baseline variables. It shows that the treatment and control group are balanced on key characteristics like gender, age, and education after this correction.

Note that the standard stable unit treatment value assumption (SUTVA) assumes here that the outcomes for the control group are unaffected by those being treated. This assumption would be violated if the control group and treatment group were directly competing for the same jobs. Given our sample is small relative to both the number of job openings and job-seekers in the economy, we believe the chance that treated youth are directly competing with the control group for specific job openings is rather small and therefore that the SUTVA should be satisfied to a first order. Note that it could however still be the case that the job opportunities to the treated come at the expense of other youth in the economy (outside our experimental sample). As a result, the treatment effect will produce only the partial equilibrium effect of giving some youth additional experience and not the general equilibrium effect of what would happen if all youth took part in internships.

4.2 Matchmaking and take-up

Eleven human resource advisors were recruited by the program to help match individuals in the treatment group to firms seeking jobs. The advisors talked with the firms to check which qualities they were seeking in workers, and then, they called and pre-screened youth in the treatment group to identify a set of individuals that matched the firm’s criteria. The CVs and details of these candidates were then sent to firms, and firms invited youth to interviews and decided whether or not to make them an internship offer. If an individual in the treatment group was not selected by a firm, they went back into the pool of available treatment group individuals that the advisor could then attempt to match to another job.

In total, 334 interviews took place, and 198 interns were accepted by firms into jobs.⁴ Some of these were accepted with certainty due to insufficient numbers to also allow a control group, and so were not part of the experimental sample. In total, 158 of the 430 individuals in the experimental treatment group received an internship (36.7 %). The take-up rate did not differ significantly by gender (34.8 % for females, 37.4 % for males, p value for test of equality is 0.625).

There were three main reasons why some individuals assigned to the internship treatment group did not receive an internship. First, by design, there were more individuals assigned to treatment than there were internship jobs available in order that firms could be offered several candidates to interview and find a good fit. Second, some of the youth assigned to the internship program dropped out either because they found a job without the internship, because they could not be reached, or because of other circumstances. Third, some of the firms that had been selected to receive matching grants for interns dropped out of the program. This resulted because several months had passed since they had applied, and either they had recruited directly from the market during this time or had decided as a result of the changing economic situation that they could no longer afford to offer this position. When this happened, SMEPS attempted to replace the firm with another firm requesting graduates from the same specialization in the same city, but in some cases, this was not possible, and as a result, some of individuals assigned to treatment would be in specializations where there was no longer any demand for them.

4.3 Data collection

We use three main sources of data. The first consists of baseline information available from the applications. The second consists of administrative data from the program, which provides key information on initial take-up of the internship and internship completion rates. The third and main source of data comes from a follow-up survey. We had originally planned surveys at 6 and 12 months after the end of the internship. However, with the worsening situation in Yemen, we quickly launched a follow-up survey in March 2015. This timing took place 3–6 months after the end of the internships and right before civil conflict broke out. The survey was conducted as a phone survey given that safety issues, gas shortages, and time constraints prohibited an in-person survey. The survey was conducted by Apex consulting and had a response rate of 78.7 % (78.1 % treatment, 80.4 % control, p value of test of equality 0.666). We believe this response rate to be excellent given the difficult conditions, and given there is no statistical difference in response rates by treatment group, we assume attrition is at random for the purposes of analysis.

The survey asked questions about employment during 2014 to capture the time of the internship and then questions about employment in the months of December 2014 and February 2015 to capture post-treatment employment outcomes.

5 Results

We begin with descriptive information on what took place during the internships, followed by analysis of the impact of these internships on employment outcomes.

5.1 What did an internship entail?

The modal intern worked 6 months as an intern. According to the administrative data, 59 of the 198 interns placed did not work the full 6 months because either the firm decided to no longer retain them or they decided to leave. In the follow-up survey, 64 % worked in the internship position for 6 months, while 24 % worked for 3 months or less.

Interns were typically working around 40 h per week, Sunday to Thursday. The jobs appear very similar in terms of what they entailed to the types of jobs the control group was doing. Table 3 uses the survey data to compare the different characteristics of work for those who worked in the treatment group in 2014 to those who worked in the control group. This conditions on working at all (the next subsection examines the treatment effect on work).

Table 3 Comparison of characteristics of work in 2014 conditional on working

We see that the two groups are statistically similar in terms of the characteristics of their jobs conditional on working. The jobs pay an average of 55,000–59,000 rials per month (approximately US$255–270). They are mostly with private employers, in jobs related to the studies of the intern. In at least 80 % of the cases, the intern says the job involves using a computer that they learned technical skills on the job and that they learned organizational skills on the job. However, as with jobs taken by the control group, three-quarters of the interns also say that they spent at least 2 h per day on tasks they considered beneath them and 59 % were idle at the job at least 5 h or more per week. The internships therefore appear to have involved the same mix of learning opportunities and less learning oriented activities as a typical job for this group.

5.2 Impact of internship on employment during the internship period

In order for the internship program to increase employment, it must result in additional employment for the applicants rather than just substituting for jobs they would have found anyway. To estimate the impact of being assigned to the internship treatment on employment (the intention-to-treat effect or ITT), we estimate Eq. (1) for several employment outcomes. We then also estimate the impact of actually taking up the internship (the local average treatment effect or LATE) by instrumenting take-up of an internship with the random assignment.

Table 4 then reports the impact of the internship treatment on employment outcomes during 2014. Column 1 looks at whether the applicant worked at all in 2014. Only 61 % of the control group were able to find work. Being assigned to the treatment group increased this probability by a statistically significant 17 % age points and actually receiving the internship by 42 percentage points. Column 2 shows the program also led to a significant increase in the number of months of work done in 2014: the control group worked an average of 3.9 months of the year (including zeros for those who did not work) and the effect of taking up the internship is to almost double this amount of work experience (increasing it by 3.4 months). Columns 3 and 4 show that the program also increased the amount the applicants earned in 2014, with the LATE effect of 24,800 rials representing a 73 % increase in earnings relative to those of the control group. The final column pools these different indicators together as an aggregate index of standardized z-scores and finds a significant impact on employment using this measure.

Table 4 Impacts on employment outcomes in 2014

Table 5 examines the heterogeneity in impact by gender. Since we have only 1 year of the program rather than the originally planned two, statistical power is lower than planned. We see that female applicants are less likely to work at all, work fewer months, and earn less than male applicants among the control group. The program is found to have statistically significant impacts for male applicants. All the treatment interactions suggest that the program was less effective for female applicants, but due to the relatively small sample size, we cannot reject equality of treatment effects by gender.

Table 5 Gender impacts on 2014 employment

5.3 Post-treatment impact

We then turn to examining whether the added work experience gained during this internship has impact that last beyond the end of the internship. Recall that due to the outbreak of civil conflict in Yemen, our survey had to be fielded earlier than planned, making these relatively short-term post-treatment outcomes. The survey elicited information on work in December 2014 and February 2015. December 2014 is 1 to 3 months after the end of the internship, while February 2015 is 3 to 5 months after, but during a period where the Houthis had taken over the government in Sanaa.

Table 6 reports the results. When we consider each employment outcome separately, the point estimates are all positive, but only the impact on hours worked per week in February 2015 is significant at the 10 % level. However, when we aggregate the outcomes together into a standardized employment outcome index (again defined as the average of standardized z-scores), the internship has a positive effect which is significant at the 1 % level. The impact is relatively large in magnitude, with the ITT representing a 0.35 standard deviation increase in the index and the LATE a 0.85 standard deviation increase. Another potential approach to increasing power is to pool together the two measurements and estimate the average impact in a panel setting, as in McKenzie (2012). Appendix 2 reports these results, which show the treatment group working a statistically significant 4.7 h more per week than the control group and positive, but not statistically significant impacts on working at all and on income earned.

Table 6 Impacts post-intervention

Table 7 examines the heterogeneity in post-treatment employment impacts by gender. The results show females in the control group continuing to have lower employment rates and earning lower income than males. As with the results in Table 5, power is low, but the point estimates suggest the treatment impacts were larger for male youth than females, so the internship program is not helping close this gender gap.

Table 7 Gender ITT impacts post-intervention

6 Conclusions

The internship program succeeded in generating new job experience for applicants in a context of worsening political and economic instability in Yemen. The internship jobs appear similar in terms of learning experiences to the jobs the control group managed to get; just more people in the treatment group were able to experience them. The fact that those who received internships had better employment outcomes after the end of the program shows some post-program persistence in this effect.

The context also requires several caveats in interpreting these results. The civil conflict that erupted in 2015 resulted in the cancelation of the second year of the program, effectively halving our intended sample size and reducing our statistical power. It also required the follow-up survey to be conducted more quickly after the end of the program that we had planned and for a shorter survey instrument to be used. Moreover, the armed conflict in 2015 has resulted in large-scale internal displacement, with more than 80 % of the population in need of humanitarian assistance, shortages of fuel, electricity, and water, and infrastructure damage (European Commission, 2015). As a result, employment opportunities for the population in this study are likely to have been severely affected, potentially reducing any longer-term impact. Finally, it is possible that the employment gains to the treated youth came at the expense of other youth in the economy so that the general equilibrium impact of scaling-up such a program could be less than the effect measured here. Nevertheless, we believe it important to share the results available given the paucity of evidence on internship programs in developing countries.

Appendices

Appendix 1: Timeline

August 2013: program launched

October 1, 2013–December 26, 2013: online application process for interns

December 29, 2013–January 5, 2014: screening for eligibility

January 2014: random selection of potential interns for training

February–March 2014: 2-day basic skills training

March–May 2014: matching of treatment group interns to firms, and interviews by firms

March–June 2014: start of internships

September 2014: Houthi rebels take control of the capital of Sanaa. UN peace deal announced in which they agree to withdraw once national unity government is formed

August–December 2014: end of internships

January–February 2015: Houthis seize state TV; President Hadi resigns and flees to Aden; Houthis take charge of the government

March 2015: follow-up survey conducted

March 2015: Islamic state carries out suicide bombings (March 20). Saudi-led coalition launches air strikes and imposes naval blockade (March 26). Clashes in Aden and other cities. By April, widespread conflict

Appendix 2: Pooled estimates

Table 8 Pooled ITT impacts post-intervention in panel