Published 02/22/16

Marketing professors' research debunks

Recently published research by two marketing professors poses a serious challenge to the accuracy and usefulness of on-line ratings from ( 

Professor of Marketing Keith Murray, Ph.D., and Associate Professor of Marketing Srdan Zdravkovic, Ph.D., decided to explore the popular website used by many students to decide which professor’s class to take. Their findings were published in The Journal of Education for Business.

"The numbers shown for most professors are meaningless.”

“While pre-existing research reports a positive correlation between and existing, independent institutional instruments, the correlations are, while technically positive, practically speaking very weak and virtually worthless,” says Murray, who calls the findings "a manifestly powerful contribution to the discussion surrounding the validity of and the controversy that surrounds’s widespread acceptance.”

Murray and Zdravkovic found that, compared to conventional measures of sample-based polling data, the publicly available ratings exhibit an enormous estimate of margin for error.

More than 15 million raters reportedly participated on, rating a total of 1.8 million instructors at institutions of higher learning in the U.S., Canada, and the United Kingdom. 

“However, when one looks at the data across all professors in this research, to achieve a comparable confidence level and confidence interval to standards expected in national polling and sample reporting, would have to deliver between 15 to 20 times the number of student raters per professor than it presently does,” Zdravkovic says. “As a result, the numbers shown for most professors are meaningless.”

“Virtually no one I’ve spoken to, college students or academics, have a full appreciation of the atheoretical nature behind the site,” Murray says. “Taking the findings seriously of such an online poll is the rough equivalent of estimating the performance of the stock market or the weather patterns expected next week based on a few layperson respondents.”

Murray and Zdravkovic concluded that “better research is called for when university instructors’ professional reputations and career livelihoods are being judged.”