Cross-Cultural Differences in Peer Evaluation Styles
Lead Author: Dirk Boehe
Basic Hypothesis: Culture has an effect on peer evaluations, so that people from some cultures tend to be more strict with their evaluations and people from other cultures tend to be more lenient in their evaluations.
Furthermore, culture may be a factor in propensity to give socially desirable peer evaluations or strategically beneficial peer evaluations (e.g., all team members get perfect peer evaluations to make the team look good, even if some team members deserve low peer evaluations). ”
Members of (global virtual) teams may become concerned if their team mates contributions to the team’s project leaves a lot to be desired. Their choice options range from ignoring or denying the issue, i.e. being a “nice” team mate who seeks to remain “friends” and protects his colleagues from their common supervisors on the one hand, to revealing their team mates’ incompetence or free-riding behavior through peer evaluations. Behind this, lies an ethical dilemma, either you are a good colleague or nice team mate, but lie in peer evaluations with regard to the perceived “true” performance of their mates, or you tell the “truth” in peer evaluations and thus become a “bad” colleague who disrupts the collegial spirit of the team.
Peer evaluation styles:
|C1||Self-promotion vs. modesty||Whether individual overestimates or underestimates own performance compared to peer evaluation of individual’s performance.||C1 = individual’s own performance score – average peer score for individual
C1 = Member 1 self-score of own performance – average of peer 2, 3, 4, and 5 evaluations of the performance of member 1Interested in:
– size and direction of C1: the larger the absolute value of C1, the larger the self-promotion effect (if C1 is positive) or the larger the modesty effect (if C1 is negative)
|C2a||Generousness (or leniency) vs. Strictness||Whether individual is a generous (lenient) or strict evaluator of a peer compared to other peers’ evaluation of the focal peer’s performance.||C2a = individual’s rating of peer – average of other peers’ rating for focal peer
Note: Need to calculate C2a also for peer 3, peer 4 and peer 5, hence, for a 5 member team, each team member will have 4 values for C2a, one for each of the other two peers (we could call these C2a-2, C2a-3, C2a-4, C2a-5)
|C2b||Generousness (or leniency) vs. Strictness alternative||Whether individual is a generous (lenient) or strict evaluator of a peer compared to other peers’ evaluation of the focal peer’s performance incl. the focal peer’s own score.||C2b = individual’s rating of peer – average of all peers’ rating for focal peer
Note: Need to calculate C2b also for peer 3, peer 4 and peer 5, hence, for a 5 member team, each team member will have 4 values for C2b, one for each of the other two peers (we could call these C2b-2, C2b-3, C2b-4, C2b-5)
|C3||Differentiation vs. Similarity||Whether individual differentiates in rating peers or whether individual rates peers in a similar way.||C3 = range of ratings assigned to peers
C3 = Standard deviation of ratings for peers 2, 3, 4 and 5Interested in:
– size: the larger the variance, the more individual differentiates when evaluating peers, the smaller the variance, the more similar individual rates peers
Alternatively, we could measure using range from mean rating for all peers. However, given that range does not capture the distribution of the ratings but merely gives the distance between the extremes, it will likely be inferior to standard deviation as a measure of differentiation.
|C4||Extreme vs. Central||Whether individual tends to give extreme scores vs. scores closer to the middle||C4=average distance from the middle point on the scale (e.g., 3 on a 1-5 point scale)
C4=average of absolute distances from the scale’s middle point of ratings for peers 2, 3, 4, and 5
Size: If the index is close to 0, it means the person tends to stay very close to the middle point (neutral evaluations). If the index is large (e.g., close to 2 for a 5-point scale), the individual tends to give extreme evaluations (very low or very high).
Note, this is different from leniency/strictness or differentiation/similarity.
|Agreement in peer evaluations among the team members||C5=average inter-rater coefficient for the peer evaluations for each of the peers.
Size: If each team member received the same evaluations from each peer, the index will be close to 1. If every team member has a different opinion about performance of other team members, the index will be close to 0.
Based on this dilemma, why do some team mates opt for either extreme remains a conundrum. This research project seeks to cast light on the following research questions:
– Why do some team mates lie while others tell the truth in peer evaluations?
– How does context affect peer evaluations, especially in team with underperforming team members?
– Context is broader than culture alone and encompasses team members’ backgrounds, incentive structures and institutional affiliations.
This project’s results likely have implications for team collaboration and peer evaluation beyond student projects. Since 360-degree evaluations have been becoming quite common in many organizations. the practical implications of the project may help us to understand under which contextual conditions 360-degree evaluations and similar tools are more and less effective, deliver more realistic information or conceal the truth and thus contribute to our understanding of why team effectiveness differs across contexts.
Data collected, analyses are being conducted