The MannWhitney test doesn't really compare medians 

The MannWhitney test doesn't really compare medians 


You'll sometimes read that the MannWhitney test compares the medians of two groups. But this is not exactly true, as this example demonstrates.
The graph shows each value obtained from control and treated subjects. The twotail P value from the MannWhitney test is 0.0288, so you conclude that there is a statistically significant difference between the groups. But the two medians, shown by the horizontal lines, are identical. The MannWhitney test ranked all the values from low to high, and then compared the mean ranks. The mean of the ranks of the control values is much lower than the mean of the ranks of the treated values, so the P value is small, even though the medians of the two groups are identical.
It is also not entirely correct to say that the MannWhitney test asks whether the two groups come from populations with different distributions. The two groups in the graph below clearly come from different distributions, but the P value from the MannWhitney test is high (0.46). The standard deviation of the two groups is obviously very different. But since the MannWhitney test analyzes only the ranks, it does not see a substantial difference between the groups.
The MannWhitney test compares the mean ranks  it does not compare medians and does not compare distributions. More generally, the P value answers this question: What is the chance that a randomly selected value from the population with the larger mean rank is greater than a randomly selected value from the other population?
If you make an additional assumption  that the distributions of the two populations have the same shape, even if they are shifted (have different medians)  then the MannWhiteny test can be considered a test of medians. If you accept the assumption of identically shaped distributions, then a small P value from a MannWhitney test leads you to conclude that the difference between medians is statistically significant. But Michael J. Campbell pointed out, "However, if the groups have the same distribution, then a shift in location will move medians and means by the same amount and so the difference in medians is the same as the difference in means. Thus the MannWhitney test is also a test for the difference in means."
The KruskalWallis test is the corresponding nonparametric test for comparing three or more groups. Everything on this page about the MannWhitney test applies equally to the KruskalWallis test.
1. A. Hart. MannWhitney test is not just a test of medians: differences in spread can be important. BMJ (2001) vol. 323 (7309) pp. 391