PROOF OF CONCEPT: AN INSTITUTIONAL VALUE-ADDED RANKING SYSTEM

Higher education institution rankings have enjoyed an enormous rise in popularity over the past decades, and are considered a key source for evaluation by the public.  According to one source, US News & World Report’s 2014 ranking website drew 2.6 million unique visitors, with 18.9 million page views, during the rankings release day alone.  However, there is mounting evidence that these rankings are methodologically flawed and yield little insight about the quality of an institution to inform prospective students or their parents (see here, here, or here).  Some argue that rankings magnify economic inequality. Criticism stems in part from year-to-year changes in ranking methodology, giving rise to efforts to track such changes.  Current rankings engender varied limitations, key among which is the use of input factors that describe an institution, most prominently the institution’s admission selectivity, preparation level of enrolled students, faculty resources/research, campus diversity, and perceived reputation, to name a few, without centering the ranking on student outcomes after attending an institution.  And while some rankings use student outcome measures (e.g. student graduation rate), none of the popular rankings employ a methodology that gauges an institution’s value-added contribution to their students’ post-attendance success.  The Wall Street Journal/Times Higher Education (WSJ/THE) Ranking, relying on the largest number of weighted ranking factors, determines 60 percent of an institution’s rank for its global ranking on the basis of faculty research and publication citations alone, without consideration for post-graduate student outcomes.  And while its separate US College Ranking determines 19 percent of an institution’s rank on the basis of students’ expected post-graduate income and student debt repayment, it allocates 20 percent of the total rank to validity-plagued student self-reported ‘engagement’ measures, with the remaining weight based strictly on input factors (student enrollment and faculty resources), 

In contrast, the rankings here estimate four key academic progress and post-enrollment/graduation student outcomes after taking into account student and institutional resource input factors in order to gauge institutional performance by comparing the estimated outcomes with the reported outcomes for each institution, using the actual-versus-estimated difference score as the outcome of interest.  Thus, the measured performance evaluates the actual average outcome for each institution in light of a broad range of key inputs that influence such outcomes in order to identify and gauge the value-added contribution of the institution.  Institutions are ranked on four outcomes:  First-year student persistence (freshman retention rate), 6-year graduation rate of Bachelor-degree-seeking students, federal student loan repayment rate, and intergenerational income mobility rate.  All four outcomes look at undergraduate-program students, excluding students enrolled in graduate-level programs.  Detailed definition of the four outcomes, its associated metrics, and the statistical estimation methods employed are explained here (to be posted). 

The first page of the following interactive dashboards provides the overall rank across all four outcomes using a composite score for each institution with available data (currently over 550 institutions). Subsequent pages rank institutions on each individual outcome in tabular and geographically mapped format.  Click on the expansion arrow (bottom right) to view in full-scale mode.