Higher education research
Since the late 1990s, Jeremy Smith and Robin Naylor have collaborated on a number of research papers in the general area of the economics of higher education, with some papers also involving joint work with other co-authors (including Massimiliano Bratti, Wiji Arulampalam, Norman Ireland, Abigail McKnight, Shqiponja Telhaj). In what follows, we summarise some of the main findings of our work to date, and highlight issues which, we believe, should be the focus of further work. The papers described below are grouped into specific themes: (i) Determinants of academic performance; (ii) Dropping out of university; (iii) Skipping class; (iv) Performance Indicators for HEIs; and (v) Graduate returns - or the 'college wage premium'.
(i) Determinants of academic performance of university students
Our first focus has concerned the determinants of academic performance by students in UK universities. We were one of two teams to have discovered the accessibility to researchers of USR data, which provide detailed administrative information on whole populations of undergraduate students at all UK universities for all cohorts from the 1970s until 1993. This is a very rich dataset indeed. Our analysis of degree class determinants for the 1993 cohort concentrated on the impact of type of school attended. The first resulting paper was published as: Smith J. and R. A. Naylor , "Determinants of Degree Performance," Oxford Bulletin of Economics and Statistics, 2001, vol. 63, pp. 29-60: download. The paper was presented at the Royal Economic Society Conference, Nottingham, April, 1999, and at a very large number of departmental seminars across UK university departments of Economics.
The main finding of this paper - attracting considerable attention in the press and media, in policy circles, and in university admissions offices - was the now well-known result that students who attended state schools prior to university had a significantly higher probability of graduating with a good degree class than otherwise observationally-equivalent students who had been educated in private schools. Observationally-equivalence here signifies, inter alia: the same A-level subjects and grades; the same social class of family background; studying at the same university; reading for the same degree subject. A clear policy implication of this finding is the following: in order to admit students from across the two sectors in such a way that they have equal probabilities of achieving the same class of degree award on graduation, applicants from the state sector should be asked to have achieved, on average, 2-3 grade points below those of their private school counterparts: e.g. grades BBB instead of AAB.
Our intuitive interpretation of the result is that, simplifying, two inputs determine achievement at the end of secondary school: pupil ability and school inputs. To the extent that private school inputs are more productive than those at state schools, students from state schools who obtain the same A-level scores as those from private schools will be, on average, of higher ability. Suppose that performance at university depends on ability only: then the state-school educated will do better at university than will the private-educated. Indeed, this will be the case even if a private education has lasting benefits at university, so long as these benefits are relatively less productive at university than at school. An interesting question is whether any private school boost to A-level performance is greater than the increase in A-level performance required of the privately-educated in order to equalise their expected university performance with that of state-educated pupils. This issue awaits further exploration. A summary of work on degree performance is provided in: R. A. Naylor and Smith, J., “Determinants of educational success in higher education,”, chapter 11 in Johnes, G. and Johnes, J. (eds.), International Handbook of the Economics of Education, Edward Elgar, 2004.
In a separate paper, we show that the adverse effect of a private school education on subsequent degree performance is related systematically to the characteristics of the private school attended: higher fee schools are associated with a lower probability of a good degree class, consistent with the hypothesis that inputs are greater and hence the boost to A-level performance stronger at the more expensive schools - see R. A. Naylor and Smith J., “The determinants of degree performance in the UK,” Scottish Journal of Political Economy, vol. 51, no. 2, pp. 250-265, 2004 and R. A. Naylor and Smith J., “Schooling effects on subsequent university performance: evidence for the UK university population,” vol. 24, 549-562, Economics of Education Review, 2005.
In current work, we are considering how prior schooling affects academic performance of university students in more recent cohorts.
(ii) Dropping out of university
We have also produced a number of papers on the subject of 'Dropping Out' of university. The first paper in this series was published as: Smith J. and R. A. Naylor , "Dropping out of University: a statistical analysis of the probability of withdrawal for UK university students," Journal of the Royal Statistical Society, 2001, vol. 164, pp. 389-405: download, download from JRSS. We find evidence to support the view that both prior academic qualifications and social integration are important influences on the probability of student withdrawal from university. We also find an influence of unemployment in the county of prior residence, especially for less well-off male students.
In a second paper on dropping out, co-written with our colleague Professor Wiji Arulampalam, we show not only that weaker students (as measured by pre-university grades) are more likely to withdraw from university but also that the extent of variation in prior qualifications within the student's university degree course also exerts an influence on the individual's probability of withdrawal. See Arulampalam, W., Naylor, R. A. and Smith, J., "Effects of in-class variation and student rank on the probability of withdrawal: cross-section and time-series analysis for UK university students," presented at the Royal Economic Society Conference, University of Warwick, March, 2002, and Economics of Education Review, vol. 24, pp. 251-262, 2005.
A number of our papers on dropping out of university refer specifically to the case of medical students. See: Arulampalam, W., Naylor, R. A. and Smith, J., "A hazard model of the probability of medical school dropout in the UK," Journal of the Royal Statistical Society (Series A), vol. 167, Part 1, pp. 157-178, 2004: download; R. A. Naylor, Arulampalam, W. and Smith J., “Factors affecting the probability of first-year medical student dropout in the UK: a logistic analysis for the graduate entry cohort of 1980-1992,” Medical Education, vol. 38, pp. 492-503, 2004; and R. A. Naylor, Arulampalam, W. and Smith J., “Dropping out of medical school in the UK: explaining the changes over ten years,” Medical Education, vol. 41, pp. 385-394, 2007.
Also for the case of medical students, we have examined the questions of: (i) what determines the probability that an individual medical student will apply to a particular medical school and (ii) what determines the probability that an individual medical school applicant will receive an offer (see R. A. Naylor, Arulampalam, W. and Smith J., “Dr Who? Who gets admissions offers in UK medical schools?,” December 2005.)?
(iii) Skipping class at university
Awaiting text input: see R. A. Naylor, Arulampalam, W. and Smith J., “Am I missing something? The effects of absence form class on student performance," Economics of Education Review, vol. 31, issue 4, pp. 363-375, 2012.
(iv) University Performance Indicators
The UK Govenrment, through HEFCE, is keen to publish Performance Indicators on UK universities. Part of the motivation for this is the view that in the quasi-market for higher education, potential customers - students and their families - can make rational investment decisions only if provided with measures of univeristy performance in areas such as teaching, student progression, university research and graduate labour market transitions. Our work on student outcomes based on institutional micro-data offers a method for the construction and interpretation of performance indicators. In Smith J., McKnight, A. and Naylor, R. A., "Graduate Employability: policy and performance in higher education in the UK," Economic Journal, 110, pp. F382-411, 2000 (download), we discuss limitations of league tables of universities based on performance measures of graduate employment. We argue that if performance indicators are to be used, they should be based not on raw unadjusted institutional-level data but on student-level micro-data after adjustment for confounding factors: we find that there are large movements in the derived rank positions of universities between the adjusted and the unadjusted measures. Second, we show that the ranking of universities against the criterion of unemployment/inactivity is not well determined: there are large confidence intervals around the point estimates for university rank. Third, rankings across universities are very unstable from year to year. Finally, the rankings are, surprisingly, highly sensistive to gender. See also Bratti, M., McKnight, A., Naylor, R. and Smith, J., “Higher education outcomes, graduate employment and university performance indicators,” Journal of the Royal Statistical Society (Series A), vol. 167, part 3, pp. 475-496, 2004: download from JRSS(A).
(v) Graduate Returns - or the college wage premium
We address a number of issues concerning graduate returns. The sub-themes are: (a) cohort effects; (b) class of degree; (c) private schooling; (d) life-cycle effects; (e) variation by university and subject.
An important issue concerns the magnitude of returns to possessing a university degree. A related issue concerns how the return has varied over time. A well-known problem in estimating returns to any level of education is that associated with ability bias: when the econometrician does not have good measures of ability, estimates of returns to education will suffer omitted variable bias. In part, the extent of this bias will depend on differences in ability at different educational attainment levels. Over time, these ability differences are likely to change and thereby impact the extent of ability bias. For example, if the proportion of young people participating in higher education rises over time then this will be likely to change the gap in average ability between graduates and non-graduates and hence impact on estimates of returns to degrees. We explore the theoretical link between ability bias and estimates of degree returns in a short paper, focussing on the sensitivity of estimates to the skewness of the underlying ability distribution. We also calibrate the model for the case of changes in returns to degrees in the UK across the 1958 and 1970 birth cohorts. We find that the effect of changes in the ability-education composition of graduates and non-graduates offers a possible explanation for why graduate returns did not show greater sensitivity to demand-side forces which might have been expected to raise returns by more than has been observed in the 1980s and 1990s.
See: R. A. Naylor, Smith J. and Telhaj, S., “Graduate returns, degree class premia and higher education
expansion in the UK,” forthcoming, Oxford Economic Papers, 2016 and R. A. Naylor and J. P. Smith, (2009), “Skewness, ability bias, and the college wage premium," mimeo, University of Warwick, (download).
In further work on cohort effects and graduate returns (or the 'college wage premium'), we exploit NCDS and BCS70 data. This work in progress also addresses the Employer-Learning/Statistical-Discrimination (EL-SD) literature, which posits that returns to education should diminish through the working life-cycle as employers learn about workers' true productivities. This work is joint with Massimiliano Bratti.
Most empirical work on the estimation of returns to education focuses on either years of schooling or qualification level as the measure of education. Surprisingly little work addresses the issue of how labour market returns vary by academic performance at a given level of education. This is surprising given that in most countries there is a strong clustering of people at a relatively small number of qualification levels. For example, in the UK, labour market entrants today tend to have either a degree, or A-levels, or GCSEs. When an employer is recruiting, they tend to know at which of this small range of levels they wish to recruit. The problem for them then is to decide how to choose across candidates who all have the same qualification level, a degree, say. It seems plausible that, amongst other considerations, employers will base their choice across candidates on the grade score achieved at the given qualification level: in the case of graduates, this means on the class of degree awarded. This would be consistent with the literature on EL-SD (see above), with class of degree being regarded as a signal of graduates' underlying ability. This is what we examine in an analysis based on data for the 1970 British birth cohort. See: Bratti, M., Naylor, R. and Smith, J.,“Different returns to different degrees? Evidence from the British Cohort Study 1970,” Warwick Economics Research Paper no. 783, (download), 2007 - revised as "Heterogeneities in the returns to degrees: evidence from the British Cohort Study 1970," mimeo, University of Warwick, December 2008, (download). In this paper, we find that graduates awarded a 'higher' class degree (a first or upper second) receive an earnings premium of about 6.5% compared to students with a lower class degree (a lower second or below). In further work, we investigate the age-earnings profile by class of degree awarded.
We also exploit complementary datasets which enable us to investigate the links between class of degree awarded and labour market outcomes. A particularly rich dataset is that held by USR/HESA. Since 1994, the Higher Education Statistics Agency has been the UK Government agency acting as the depository of administrative data on full populations of students in all of the universities in the UK. These data are of high quality and contain a lot of detailed information on each student in each year of their study. They also include family and school background information and detailed data on pre-university qualifications - such as A-level subjects and grades. The predecessor to HESA was the USR (the Universities Statistical Record). When the USR was replaced, all its data entered into the public domain. We have accessed the data for the full cohorts of leavers for each year from 1985 to 1993 and added to this data from HESA for 1998 graduates. We have also linked these administrative records data for each cohort's responses to the first destination survey of all leavers. This enables us to address the question of how factors such as degree class award influence graduates' labour market outcomes. A paper in the EJ (2000) reports the results from a multi-nomial logit model of graduate outcomes, distinguishing between graduate and non-graduate jobs: see Smith J., McKnight, A. and Naylor, R. A., "Graduate Employability: policy and performance in higher education in the UK," Economic Journal, 110, pp. F382-411, 2000 (download) and also Bratti, M., McKnight, A., Naylor, R. and Smith, J., “Higher education outcomes, graduate employment and university performance indicators,” Journal of the Royal Statistical Society (Series A), vol. 167, part 3, pp. 475-496, 2004, download from JRSS(A).
We are also able to exploit the USR/HESA data to analyse the impact of degree class - and other factors - on occupational earnings of graduates in their first destination employment just one year after graduation. Under the EL-SD model (see above), we would expect that there might be a premium associated with the award of a 'higher' classes of degree. For the 1993 graduates, we find that this is indeed the case . . .
We then examine whether the premia for high degree classes found for the 1993 graduate cohort also obtained for earlier cohorts. We find this not to be the case: for the earliest cohorts we observe, 1985 graduates, there is no statistically significant premium associated with class of degree awarded. Over time, however, successive cohorts began to experience a significant, positive adn rising premium associated with a higher class of award. Both the theory and the evidence are presented in the paper:
. . . . to be continued. See Ireland, N.J., Naylor, R.A., Smith, J.P. and Telhaj, S. "Educational Returns, Ability Composition, and cohort effects: theory and evidence for cohorts of early-career UK graduates " Warwick Economics Research Paper no. 906 TWERP download and CEP discussion paper no 939 CEP download.
We estimate that for 1993 graduates, first destination occupational earnings were more than 3% higher for graduates who had attended private schools prior to univesity than for graduates who had been educated in a state LEA school, holding constant a wide variety of observed characteristics, including: family background, A-level subjects and grades, university, degree subject and degree class. See: Naylor, R. A., Smith J. and McKnight, A., "The impact of schooling on graduate earnings", Bulletin of Economic Research, (Special Issue), vol. 54, no. 4, pp. 315-340, 2002. Furthermore, we find considerable variation across different private schools in the size of the graduate earnings premium, especially for males, with the premium increasing in the level of school fees. Perhaps you do get what you pay for?
We are currently re-estimating the effects for more recent cohorts. We hope to have results available shortly.
Awaiting text.
(e) variation by university and subject
Awaiting text.
We have also considered the case of the tranistion to the graduate labour market for students in Italy: see R. A. Naylor, Smith J., Boero and McKnight, A., “Graduates and the graduate labour market: evidence from the UK and Italy,” Special Issue of Lavoro e Relazioni Industriali, pp. 131-172, 2001.
University Admission:
Should the UK adopt a Post-Qualification system?
Festival of Social Sciences
Session on 'Higher Education: Challenges and Opportunities' with Professor Mike Shattock and Registrar Jon Baldwin. You tube clip (24 minutes)
ESPE talk
21st June 2014
Sheffield Seminar
9th March 2011