Blog #20: The aligned rank transform for nonparametric factorial analyses using only anova procedures

The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures

Authors - Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins

Authors Bios -

• Jacob O. Wobbrock is an Associate Professor at the University of Washington and has a PhD from Carnegie Mellon University.
• Leah Findlater will be an Assistant Professor at the University of Maryland next year, has taught at the University of Washington, and has a PhD from the University of British Columbia.
• Darren Gergle is an Associate Professor at Northwestern University and has a PhD from Carnegie Mellon University.
• James J. Higgins is a Professor of Statistics at Kansas State University and has a PhD from the University of Missouri-Columbia.

Venue - This paper was presented at the CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems.

SUMMARY

Hypothesis:
The researchers explain that common procedures used in analyzing non-parametric data tend to be error prone and propose a new system for analyzing data. This new procedure, called the Aligned Rank Transform (ART). The hypothesis is that ART is a better way to analyze data than any other method currently in use in HCI and can be very useful in real world applications

Content:

What ART does is it corrects requirements of ANOVA statistics that are not necessarily ture in HCI. An example of this is normality. To do this, residuals are computed, estimated effects for interactions are computed, aligned responses are computed, average ranks are assigned, ANOVA on this new data is performed, and correctness is check by checking sums.

Methods:

The ART procedure consists of 5 steps: 

1. Computing residuals: for each raw response Y, compute residual = Y - cell mean
2. Computing estimated effects for all main and interaction effects: these are calculated such that Ai is the mean response Yi for rows where factor A is at level i. AiBj is the mean response Yij for rows where factor A is at level i and factor B is at level j. And so on.
3. computing the aligned response Y', assigning average ranks Y'' where Y' = residual + estimated effect.
4. performing a full-factorial ANOVA on Y''

Results:
The results were analyzed using a standard ANOVA first and then an ART to see if any observable interaction concurred in the study. The ANOVA did not detect anything and noted their was no significant interaction but the ART found significant interaction and questioned the findings found earlier which agrees with the researchers initial perception that interaction was obviously present.


Conclusion: The researchers found that ART is useful when analyzing non-parametric data and has been useful in several cases that the researchers had worked on.

Discussion:
This paper was at a very techincal level of understanding. I was fortunate enough to have statistics and at least know what ANOVA was. The researchers achieve their goal of providing a better way to provide to analyze data for HCI research. Its good that they can find anomalies by using ART.