Paper 1: University Rank Predicts Student Performance, or Does It? A Longitudinal Multi-University, Multi-Country Study Into Predictive Power of the University Prestige in Student Performance.

 

Problem Statement:
It is natural to assume that students at higher-ranked, more prestigious, more selective universities perform better than their counterparts at lower ranked schools. Unfortunately, it is close to impossible to test this proposition directly as you hardly ever have a representative sample of students from schools of different ranks completing the same task so that their performance could be compared. At best, we’re left with a comparison of post-graduation salary as a proxy for performance, but the higher salary of top-ranked school graduates may not be due to their better performance, but rather due to their prestigious “pedigree.” Case study and other student team competitions, often won by the students from lower-ranked schools, do not provide for a good comparison either as the students who take part in such competitions are a self-selected highly motivated group that may not be representative of the general student body.
The X-Culture project provides a unique opportunity for studying the effects of the school rank, or rather its predictive power, with respect to student performance. X-Culture involves students from over 100 universities in 40 countries on 6 continents in a given semester. All students in the course take part in the project eliminating the self-selection bias. The task is long enough and the number of participants is large enough to mitigate the effects of luck and personal abilities. Moreover, many schools participate multiple times, often supervised by different instructors, again alleviating the biases due to chance or temporary circumstances. Finally, performance in X-Culture is evaluated in many areas and along multiple dimensions, longitudinally and by many stakeholders. So if the effects of school rank differ across domains (e.g., matters with respect, but doesn’t with respect to personal outcomes, or vice versa), we will be able to detect these differences.

Research Questions:

• Does the prestige (rank) of the university predict student performance? If so, what is the effect size?
• Does the prestige (rank) of the professor’s Alma Mater affect student performance? If so, what is the effect size?
• Is the prestige-performance relationship moderated/mediated and how?

Hypotheses: 

• The higher the rank of the university, the stronger is the student performance
• The stronger the professor’s academic “pedigree”, the stronger is the student performance

The relationship is moderated by:

• Class size: stronger in larger classes as the larger the class, the less personal characteristics of the students matter and it’s more about the institutional environment
• Study level: stronger for Master’s than for Undergraduates
• Region: stronger in North America where rankings matter a lot

The relationship is mediated by:

• More prepared: students at stronger schools are more diligent and more prepared (e.g., get higher scores on the Readiness Test and English)
• Time investment: students at stronger schools/stronger professors work harder
  Instructor support: instructors at stronger schools provide more support, which in turn improves performance.

Other issues to consider: Is the effect the same with respect to

Effort

• Intellectual contribution
• Technical skills
• Peer satisfaction
• Quality of the work produced
• Probability of dropout
• Probability that the student will assume a leadership position?
• Is the difference in performance noticeable right from start, or emerges only over time?
• Does it help to mix low and high-rank school students, or it works best when top-ranked school students work with other top-rankers? For example, do “top” students get higher evaluations from their “low” peers than from other “top” peers?

Data:
X-Culture data will be used to measure performance. X-Culture records will be used and matched with external data (school rankings) to quantify the “prestige” of the participating universities and alma-mater institutions of the instructors.
Performance will be measured as:
Peer evaluations (intellectual contribution, writing quality, help with coordination, etc.)
Quality of the student reports
Readiness test results
Technical skills and English skills
Effort (diligence with respect to deadlines, input in terms of amount of work such as number of works submitted at each deadline and the like)
Leadership rating (by the team)
Co-Authors Welcome for the Following Tasks:
1. Develop a coding scheme and code the data: How do we define and measure school rank? How do we put on the same ranking scale schools from different countries?
Code the school-rank data: record “prestige” ranking to all schools in the X-Culture sample. Likewise, check what school the instructors graduated from and record his/her “prestige” rank. We will likely develop the coding scheme collectively, but a co-author will then have to code the variables and create a data file that is ready for analysis.

2. Literature review: Write a strong Literature Review section that covers the following:
Prior research on the effects of the school rank on student performance
An exhaustive summary of how school rank has been operationalized
An exhaustive summary of what outcomes (with respect to the school rank) have been considered in the literature
A separate note on “performance” in the school rankings such as American U.S. Today, Canadian McLean, Financial Times and the like (they have a performance component in their ranking system, so need to find out how and how much it matters)
A quantitative summary of the findings from earlier studies, preferably in a table format or a mini meta-analysis
Point out the limitations of the extant literature on the topic: what earlier studies can and cannot tell us

3. Theory: Develop a strong theory and write the Theory section
Clearly formulated research questions
Plausible hypotheses with compelling rationale based on strong logic and rooted in prior research
A theoretical integrated model (a figure that shows how the different components of the model relate to one another)

4. Analyses: Run the analyses and write up the Method and Results sections (I plan to cover this myself, but if a person with strong analytical skills signs up, I am ready to yield this position)

Ideally, tasks 2 and 3 are to be completed by the same co-author or a group of co-authors as the two are parts of the same process.
The following tasks will be conducted by all co-authors:
The Introduction and Discussion sections will be written up collectively once we have the rest of the paper in place. We will also collectively copy-edit the final version of the manuscript.

Co-Author Selection Criteria:
Demonstrated expertise in the area: strong research record, have successfully completed the tasks before; e.g., developed lit. review, theory or conducted analyses for papers published in good journals. A less experienced junior colleague (Ph.D. student or Assistant professor with limited publication record welcome to apply to complete Task 1). Tasks 2-4 only people with multiple publications in top journals please.
Not only interest, but also ability to contribute: will not sign up and not deliver, but will actually invest a considerable amount of time. Expect to invest at least a few dozen hours over the next few months.
Quality input: The person will deliver quality results (analysis-ready dataset OR publishable Lit. Review and Theory sections OR properly conducted tests with a publishable Method and Results Sections AND quality input in collective coding scheme development, writing of the Intro and Discussion sections, and final copy-editing of the paper).
Everyone is welcome to sign up, but know only those who will actually make a valuable contribution will be listed as co-authors of the resulting publications. If a person sign ups and makes a contribution (prepare or analyze the data, write some parts of the paper) but the quality will not be there and other co-authors will have to re-do those parts, the person will not be listed as a co-author.

Target Journals
Academy of Management Learning and Education
There will be likely two papers, so the second one may also go to AMLE but also possibly to the Journal of Higher Education.
A follow-up practitioner oriented article (more popular, less numbers) may then also be written for AIB Insights or other journals that publish short perspective pieces.

 

Co-Authors

• Vas Taras v_taras@uncg.edu (coordination and data)

Empirical Team

• Alfredo Jimenez <ajimenez@ubu.es>
• Anna Svirina <anna_svirina@yahoo.com>
• Alexander Assouad <assouad@live.com>;
• JK-uwl <jkraemer@uwlax.edu>;
• Ernesto Tavoletti <ernesto.tavoletti@unimc.it>;

 

Theory Team

• Clara Lei <clara.lei@yahoo.com.hk>;
• Clara Lei <clara@ift.edu.mo>;
• Dan Caprar <dan.caprar@unsw.edu.au>;
• Grishma Shah <grishma.shah@manhattan.edu>;
• Marjaana Gunkel <marjaana.gunkel@inkubator.leuphana.de>; <marjaana.gunkel@unibz.it>

 

Status: Launched 2015, active