Business business is
Research research is
Learning learning is
Home For Researchers Research Projects Cross-Cultural Differences in Peer Evaluation Styles

Cross-Cultural Differences in Peer Evaluation Styles

Research Projects

Lead Author: Dirk Boehe

Basic Hypothesis: Culture has an effect on peer evaluations, so that people from some cultures tend to be more strict with their evaluations and people from other cultures tend to be more lenient in their evaluations.

Furthermore, culture may be a factor in propensity to give socially desirable peer evaluations or strategically beneficial peer evaluations (e.g., all team members get perfect peer evaluations to make the team look good, even if some team members deserve low peer evaluations). ”

 

Managerial dilemma:

Members of (global virtual) teams may become concerned if their team mates contributions to the team’s project leaves a lot to be desired. Their choice options range from ignoring or denying the issue, i.e. being a “nice” team mate who seeks to remain “friends” and protects his colleagues from their common supervisors on the one hand, to revealing their team mates’ incompetence or free-riding behavior through peer evaluations. Behind this, lies an ethical dilemma, either you are a good colleague or nice team mate, but lie in peer evaluations with regard to the perceived “true” performance of their mates, or you tell the “truth” in peer evaluations and thus become a “bad” colleague who disrupts the collegial spirit of the team.

 

Peer evaluation styles:

Criterion
Variable
Name Description Specific Measurement
C1 Self-promotion vs. modesty Whether individual overestimates or underestimates own performance compared to peer evaluation of individual’s performance. C1 = individual’s own performance score – average peer score for individual
Example:
C1 = Member 1 self-score of own performance – average of peer 2, 3, 4, and 5 evaluations of the performance of member 1Interested in:
– size and direction of C1: the larger the absolute value of C1, the larger the self-promotion effect (if C1 is positive) or the larger the modesty effect (if C1 is negative)
C2a Generousness (or leniency) vs. Strictness Whether individual is a generous (lenient) or strict evaluator of a peer compared to other peers’ evaluation of the focal peer’s performance. C2a = individual’s rating of peer – average of other peers’ rating for focal peer

Example:
C2a = Member 1 rating for peer 2 – average of peer 3, 4, and 5 ratings for peer 2

Interested in:
– size and direction of C2a: the larger the absolute value of C2a, the more generous (lenient) member 1 when rating peer 2 (if C2a is positive) or the stricter member 1 when rating peer 2 (if C2a is negative)

Note: Need to calculate C2a also for peer 3, peer 4 and peer 5, hence, for a 5 member team, each team member will have 4 values for C2a, one for each of the other two peers (we could call these C2a-2, C2a-3, C2a-4, C2a-5)

C2b Generousness (or leniency) vs. Strictness alternative Whether individual is a generous (lenient) or strict evaluator of a peer compared to other peers’ evaluation of the focal peer’s performance incl. the focal peer’s own score. C2b = individual’s rating of peer – average of all peers’ rating for focal peer

Example:
C2a = Member 1 rating for peer 2 – average of peer 2, 3, 4, and 5 ratings for peer 2

Interested in:
– size and direction of C2b: the larger the absolute value of C2b, the more generous (lenient) member 1 when rating peer 2 (if C2b is positive) or the stricter member 1 when rating peer 2 (if C2b is negative)

Note: Need to calculate C2b also for peer 3, peer 4 and peer 5, hence, for a 5 member team, each team member will have 4 values for C2b, one for each of the other two peers (we could call these C2b-2, C2b-3, C2b-4, C2b-5)

 

 

Criterion
Variable
Name Description Specific Measurement
C3 Differentiation vs. Similarity Whether individual differentiates in rating peers or whether individual rates peers in a similar way. C3 = range of ratings assigned to peers
Example:
C3 = Standard deviation of ratings for peers 2, 3, 4 and 5Interested in:
– size: the larger the variance, the more individual differentiates when evaluating peers, the smaller the variance, the more similar individual rates peers

Alternatively, we could measure using range from mean rating for all peers. However, given that range does not capture the distribution of the ratings but merely gives the distance between the extremes, it will likely be inferior to standard deviation as a measure of differentiation.

C4 Extreme vs. Central Whether individual tends to give extreme scores vs. scores closer to the middle C4=average distance from the middle point on the scale (e.g., 3 on a 1-5 point scale)

 

C4=average of absolute distances from the scale’s middle point of ratings for peers 2, 3, 4, and 5

 

Interested in:

Size: If the index is close to 0, it means the person tends to stay very close to the middle point (neutral evaluations). If the index is large (e.g., close to 2 for a 5-point scale), the individual tends to give extreme evaluations (very low or very high).

Note, this is different from leniency/strictness or differentiation/similarity.

 

C5 Inter-rater agreement

(team level)

Agreement in peer evaluations among the team members C5=average inter-rater coefficient for the peer evaluations for each of the peers.

 

Interested in:

Size: If each team member received the same evaluations from each peer, the index will be close to 1. If every team member has a different opinion about performance of other team members, the index will be close to 0.

 

Research questions:

Based on this dilemma, why do some team mates opt for either extreme remains a conundrum. This research project seeks to cast light on the following research questions:

– Why do some team mates lie while others tell the truth in peer evaluations?

– How does context affect peer evaluations, especially in team with underperforming team members?

– Context is broader than culture alone and encompasses team members’ backgrounds, incentive structures and institutional affiliations.

 

Relevance:

This project’s results likely have implications for team collaboration and peer evaluation beyond student projects. Since 360-degree evaluations have been becoming quite common in many organizations. the practical implications of the project may help us to understand under which contextual conditions 360-degree evaluations and similar tools are more and less effective, deliver more realistic information or conceal the truth and thus contribute to our understanding of why team effectiveness differs across contexts.

 

Status: 
Data collected, analyses are being conducted