- Open Access
Targeting labour market programmes — results from a randomized experiment
Swiss Journal of Economics and Statistics volume 145, pages 221–268 (2009)
We evaluate a randomized experiment of a statistical support system developed to assist caseworkers in Swiss employment offices in choosing appropriate active labour market programmes for their unemployed clients. This statistical support system predicted the labour market outcome for each programme and thereby suggested an ‘optimal’ labour market programme for each unemployed person. The support system was piloted in several employment offices. In those pilot offices, half of the caseworkers used the system and the other half acted as control group. The allocation of the caseworkers to treatment and control group was random. The experiment was designed such that caseworkers retained full discretion about the choice of active labour market programmes, and the evaluation results showed that caseworkers largely did not follow the statistical support system. This indicates that stronger incentives are needed for caseworkers to comply with statistical profiling and targeting systems.
Black, D., J. Smith, M. Berger, and B. Noel (2003), “Is the Threat of Reemployment Services More Effective than the Services Themselves? Evidence from Random Assignment in the UI System”, American Economic Review 93, pp. 1313–1327.
Black, D., J. Smith, M. Plesca, and S. Plourde (2002), “Profiling UI claimants to Allocate Employment Services: Evidence and Recommendations for the States”, Final report to the US Department of Labour.
Behncke, S., M. Frölich, and M. Lechner (2007), „Pilotprojekt Statistisch assistierte Programmselektion (SAPS)“, Swiss State Secretariat for Economic Affairs (seco), http://www.seco.admin.ch/dokumentation/publikation/00004/00005/index.html?lang=de.
Bell, S., and L. Orr (2002), “Screening (and Creaming?) Applicants to Job Training Programs: The AFDC Homemaker-Home Health Aide Demonstrations”, Labour Economics 9, pp. 279–301.
Berger, M., D. Black, and J. Smith (2001), “Evaluating Profiling as a Means of Allocating Government Services”, in: Econometric Evaluation of Labour Market Policies, M. Lechner and F. Pfeiffer (eds), Physica/Springer, Heidelberg, pp. 59–84.
Brownell, K. and T. Wadden (1991), “The Heterogeneity of Obesity: Fitting Treatments to Individuals”, Behavior Therapy, 22, pp. 153–177.
Bryson, A., and D. Kasparova (2003), “Profiling Benefit Claimants in Britain: A Feasibility Study”, Department for Work and Pensions, Research Report no. 196.
Colpitts, T. (2002), “Targeting Reemployment Services in Canada: The Service and Outcome Measurement System (SOMS) Experience”, in: Eberts, R. W., D. J. O’Leary, S. Wandner (eds), Targeting Employment Services, Kalamazoo, Michigan: W.E. Upjohn Institute, pp. 283–301.
Dehejia, RH. (2005), “Program Evaluation as a Decision Problem”, Journal of Econometrics 125, pp. 141–173.
De Koning, J. (1999), “The Chance-Meter: Measuring the Individual Chance of Long-Term Unemployment”, Paper presented at the Conference ‘Current Developments in the Evaluation of Employment Policies’, Barcelona, 19–20 July 1999, Division of Labour and Education, Rotterdam.
Dickinson, K., P. Decker, and S. Kreutzer (2002), “Evaluation of WPRS sytems”, in: Eberts, R. W., D. J. O’Leary, S. Wandner (eds), Targeting Employment Services, Kalamazoo, Michigan: W. E. Upjohn Institute, pp. 61–81.
Eberts, R. (2002), “Design, Implementation and Evaluation of the Work First Profiling Pilot Project”, Report for the US Department of Labor. http://wdr.doleta.gov/research/keyword.cfm?fuseaction=dsp_resultDetails&pub_id=2287&bas_option=Keywords&start=1&usrt=4&stype=basic&sv=1&criteria=work%20first.
Eberts, R., and C. O’Leary (2002), “A Frontline Decision Support System for Georgia Career Centres”, W. E. Upjohn Institute Staff Working Paper no. 02-84, Kalamazoo, Michigan.
Eberts, R., C. O’Leary and S. Wandner (2002), Targeting Employment Services, Kalamazoo, Michigan: W. E. Upjohn Institute.
Egger, M., and C. Lenz (2006), “Wirkungsevaluation der öffentlichen Arbeitsvermittlung – Evaluationsbericht”, SECO Publikation Arbeitsmarktpolitik No 18 (10.2006), Swiss State Secretariat for Economic Affairs (seco), http://www.seco.admin.ch/dokumentation/publikation/00004/00005/index.html?lang=de. French version: http://www.seco.admin.ch/dokumentation/publikation/00004/00005/index.html?lang=fr.
Fraser, N. (2000), “How Strong is the Case for Targeting Active Labour Market Policies”, International Journal of Manpower 20, pp. 151–164.
Fredriksson, P., and P. Johansson (2003), “Program Evaluation and Random Program Starts”, IFAU Discussion Paper 2003:1.
Frölich (2006), “Statistical Treatment Choice: An Application to Active Labour Market Programmes”, IZA Discussion Paper 2187.
Frölich (2008), “Statistical Treatment Choice: An Application to Active Labour Market Programmes”, forthcoming in the Journal of the American Statistical Association.
Frölich, M., M. Lechner, S. Behncke, S. Hammer, N. Schmidt, S. Menegale, A. Lehmann, and R. Iten (2007), “Einfluss der Rav auf die Wiedereingliederung von Stellensuchenden”, SECO Publikation Arbeitsmarktpolitik no. 20 (1.2007), Swiss State Secretariat for Economic Affairs (seco), http://www.seco.admin.ch/dokumentation/publikation/00004/00005/index.html?lang=de.
Frölich, M., M. Lechner, and H. Steiger (2003), “Statistically Assisted Programme Selection – International Experiences and Potential Benefits for Switzerland”, Swiss Journal of Economics and Statistics 139, pp. 311–331.
Horrace, W., and P. Schmidt (2000), “Multiple Comparisons with the Best, with Economic Applications”, Journal of Applied Econometrics 15, pp. 1–26.
Keum, J. (2001), “Review of the Effectiveness of Public Employment Services”, in: Park, F., Y. Park, G. Betcherman, A. Dar (eds), Labor Market Reforms in Korea: Policy Options for the Future, Korea Labor Institute.
Kreuter, M., and V. Strecher (1996), “Do Tailored Behavior Change Messages Enhance the Effectiveness of Health Risk Appraisals?: Results from a Randomized Trial”, Health Education Research 11, pp. 97–105.
Lalive, R., J. van Ours, and J. Zweimüller (2005), “The Effect Of Benefit Sanctions On The Duration Of Unemployment”, Journal of the European Economic Association 3, pp. 1386–1417.
Lechner, M., and J. Smith (2007), “What is the Value Added by Caseworkers?”, Labour Economics 14, pp. 135–151.
Manski, C. (2000), “Identification Problems and Decisions under Ambiguity: Empirical Analysis of Treatment Response and Normative Analysis of Treatment Choice”, Journal of Econometrics 95, pp. 415–442.
Manski, C. (2004), “Statistical Treatment Rules for Heterogeneous Populations”, Econometrica 72, pp. 1221–1246.
Manski, C. (2006), “Search Profiling With Partial Knowledge of Deterrence”, The Economic Journal 116, pp. F385–F401
Manski, C. (2007), “Minimax-Regret Treatment Choice with Missing Outcome Data ”, Journal of Econometrics 139, pp. 105–115.
Moffitt, R (2006), “Welfare Work Requirements with Paternalistic Government Preferences”, The Economic Journal 116, pp. F441–F458.
Moisala, J., I. Suoniemi, and R. Uusitalo (2006), “Työttömien tilastollinen profilointi”, http://www.mol.fi/mol/fi/99_pdf/fi/06_tyoministerio/06_julkaisut/06_tutkimus/tpt316.pdf.
Murphy, S. (2003), “Optimal Dynamic Treatment Regimes”, Journal of Royal Statistical Society, Series B 65, pp. 331–366.
Murphy, S., K. Lynch, D. Oslin, J. McKay, and T. TenHave (2007), “Developing Adaptive Treatment Strategies in Substance Abuse Research”, Drug and Alcohol Dependence, Supplement 2, 88S, pp. S24–S30.
OECD (1998), “Early Identification of Jobseekers at Risk of Long-Term Unemployment: The Role of Profiling”, OECD Proceedings, Paris.
O’Leary, D., P. Decker, and S. Wandner (2005), “Cost Effectiveness of Targeted Reemployment Bonuses”, Journal of Human Resources 40, pp. 270–279.
Plesca, M., and J. Smith (2005), “Rules versus Discretion in Social Programs: Empirical Evidence on Profiling in Employment and Training Programs”, unpublished, University of Michigan.
Persico, N., and P. Todd (2005), “Passenger Profiling, Imperfect Screening, and Airport Security”, American Economic Review, Papers and Proceedings 95, pp. 127–131.
Persico, N., and P. Todd (2006), “Generalising the Hit Rates Test for Racial Bias in Law Enforcement, With an Application to Vehicle Searches in Wichita”, The Economic Journal 116, pp. F351–F367.
Rudolph, H., and M. Müntnich (2001), “Profiling zur Vermeidung von Langzeitarbeitslosigkeit – erste Ergebnisse aus einem Modellprojekt”, MittAB, pp. 530–553.
Rush, A. (2005), “Algorithm-Guided Treatment in Depression: TMAP and STAR*D”, in M. Bauer, A. Berghofer and M. Adli (eds), Therapieresistente Depressionen – Aktueller Wissensstand und Leitlinien für die Behandlung in Klinik und Praxis, Springer, Heidelberg.
Smith, J. (2002), “Comments on Chapter 10”, in: Eberts, R.W., D.J. O’Leary S. Wandner (eds), Targeting Employment Services, Kalamazoo, Michigan: W.E. Upjohn Institute, 2002, pp. 303–308.
Staghøj, J., M. Svarer, and M. Rosholm (2007), “A Statistical Programme Assignment Model”, mimeo.
Stephan, G., S. Rässler, and T. Schewe (2006), “Das TrEffeR-Projekt der Bundesagentur für Arbeit. Die Wirkung von Maßnahmen aktiver Arbeitsmarktpolitik”, Zeitschrift für Arbeitsmarktforschung, 39, pp. 447–465.
U.S. Department of Labor, Employment and Training Administration (1999), “Evaluation of Worker Profiling and Reemployment Services Policy Workgroup: Final Report and Recommendations”, Washington D.C.
Velicer, W, and J. Prochaska (1999), “An Expert System Intervention for Smoking Cessation”, Patient Education and Counselling 36, pp. 119–129.
Wandner, S. (2002), “Targeting Employment Services under the Workforce Investment Act”, in: Eberts, R.W., D.J. O’Leary, S. Wandner (eds.), Targeting Employment Services, Kalamazoo, Michigan: W.E. Upjohn Institute, 2002, 1–25.
Wunsch, C. (2007), “Optimal Use of Labour Market Policies”, Discussion Paper 2007–26, Department of Economics, University of St.Gallen.
We are very grateful to Heidi Steiger for her support in the early stages of this project. We also thank Chris O’Leary and Thomas Ragni for helpful comments. We are grateful to the Swiss government (seco) for providing the administrative database as well as substantial financial support for this project. All remaining errors are our own.
About this article
Cite this article
Behncke, S., Frölich, M. & Lechner, M. Targeting labour market programmes — results from a randomized experiment. Swiss J Economics Statistics 145, 221–268 (2009). https://doi.org/10.1007/BF03399281
- active labour market programmes
- statistical treatment rules
- public employment services