Cross-Cultural Differences in Peer Evaluation Styles
Lead Author: Dirk Boehe
Basic Hypothesis: Culture has an effect on peer evaluations, so that people from some cultures tend to be more strict with their evaluations and people from other cultures tend to be more lenient in their evaluations.
Furthermore, culture may be a factor in propensity to give socially desirable peer evaluations or strategically beneficial peer evaluations (e.g., all team members get perfect peer evaluations to make the team look good, even if some team members deserve low peer evaluations). ”
Managerial dilemma:
Members of (global virtual) teams may become concerned if their team mates contributions to the team’s project leaves a lot to be desired. Their choice options range from ignoring or denying the issue, i.e. being a “nice” team mate who seeks to remain “friends” and protects his colleagues from their common supervisors on the one hand, to revealing their team mates’ incompetence or free-riding behavior through peer evaluations. Behind this, lies an ethical dilemma, either you are a good colleague or nice team mate, but lie in peer evaluations with regard to the perceived “true” performance of their mates, or you tell the “truth” in peer evaluations and thus become a “bad” colleague who disrupts the collegial spirit of the team.
Peer evaluation styles:
| Criterion Variable |
Name | Description | Specific Measurement |
| C1 | Self-promotion vs. modesty | Whether individual overestimates or underestimates own performance compared to peer evaluation of individual’s performance. | C1 = individual’s own performance score – average peer score for individual Example: C1 = Member 1 self-score of own performance – average of peer 2, 3, 4, and 5 evaluations of the performance of member 1Interested in: – size and direction of C1: the larger the absolute value of C1, the larger the self-promotion effect (if C1 is positive) or the larger the modesty effect (if C1 is negative) |
| C2a | Generousness (or leniency) vs. Strictness | Whether individual is a generous (lenient) or strict evaluator of a peer compared to other peers’ evaluation of the focal peer’s performance. | C2a = individual’s rating of peer – average of other peers’ rating for focal peer
Example: Interested in: Note: Need to calculate C2a also for peer 3, peer 4 and peer 5, hence, for a 5 member team, each team member will have 4 values for C2a, one for each of the other two peers (we could call these C2a-2, C2a-3, C2a-4, C2a-5) |
| C2b | Generousness (or leniency) vs. Strictness alternative | Whether individual is a generous (lenient) or strict evaluator of a peer compared to other peers’ evaluation of the focal peer’s performance incl. the focal peer’s own score. | C2b = individual’s rating of peer – average of all peers’ rating for focal peer
Example: Interested in: Note: Need to calculate C2b also for peer 3, peer 4 and peer 5, hence, for a 5 member team, each team member will have 4 values for C2b, one for each of the other two peers (we could call these C2b-2, C2b-3, C2b-4, C2b-5) |
| Criterion Variable |
Name | Description | Specific Measurement |
| C3 | Differentiation vs. Similarity | Whether individual differentiates in rating peers or whether individual rates peers in a similar way. | C3 = range of ratings assigned to peers Example: C3 = Standard deviation of ratings for peers 2, 3, 4 and 5Interested in: – size: the larger the variance, the more individual differentiates when evaluating peers, the smaller the variance, the more similar individual rates peers Alternatively, we could measure using range from mean rating for all peers. However, given that range does not capture the distribution of the ratings but merely gives the distance between the extremes, it will likely be inferior to standard deviation as a measure of differentiation. |
| C4 | Extreme vs. Central | Whether individual tends to give extreme scores vs. scores closer to the middle | C4=average distance from the middle point on the scale (e.g., 3 on a 1-5 point scale)
C4=average of absolute distances from the scale’s middle point of ratings for peers 2, 3, 4, and 5
Interested in: Size: If the index is close to 0, it means the person tends to stay very close to the middle point (neutral evaluations). If the index is large (e.g., close to 2 for a 5-point scale), the individual tends to give extreme evaluations (very low or very high). Note, this is different from leniency/strictness or differentiation/similarity.
|
| C5 | Inter-rater agreement
(team level) |
Agreement in peer evaluations among the team members | C5=average inter-rater coefficient for the peer evaluations for each of the peers.
Interested in: Size: If each team member received the same evaluations from each peer, the index will be close to 1. If every team member has a different opinion about performance of other team members, the index will be close to 0.
|
Research questions:
Based on this dilemma, why do some team mates opt for either extreme remains a conundrum. This research project seeks to cast light on the following research questions:
– Why do some team mates lie while others tell the truth in peer evaluations?
– How does context affect peer evaluations, especially in team with underperforming team members?
– Context is broader than culture alone and encompasses team members’ backgrounds, incentive structures and institutional affiliations.
Relevance:
This project’s results likely have implications for team collaboration and peer evaluation beyond student projects. Since 360-degree evaluations have been becoming quite common in many organizations. the practical implications of the project may help us to understand under which contextual conditions 360-degree evaluations and similar tools are more and less effective, deliver more realistic information or conceal the truth and thus contribute to our understanding of why team effectiveness differs across contexts.
Status:
Data collected, analyses are being conducted