ELIS professor Lieven Eeckhout awarded the 2017 Most Influential OOPSLA Paper Award

(06-11-2017) The article "Statistically Rigorous Java Performance Evaluation" was awarded for the impact it has had over the past 10 years after publication.

The paper by Andy Georges, Dries Buytaert, Lieven Eeckhout (for 2007) has been widely influential in Java performance evaluation. Before this paper, Java performance evaluation methodologies had varied widely, leading to difficulty in comparisons and often misleading conclusions. The authors survey several past studies, carefully discuss discrepancies in experimental and reporting methodologies, and expose the pitfalls of benchmarking approaches. The fundamental problem is that without an appropriate statistical analysis, it is difficult to tell whether reported measurements in the literature reflect significant distinctions or just random fluctuations. The paper proposes a statistically rigorous approach for experimental measurements in the Java setting, ways to mitigate uncertainty, and even new reporting techniques. The paper also introduced JavaStats, publicly available software that monitors the variability in measurements to determine how many measurements will be needed in all to reach a desired level of statistical confidence.

 The ceremony took place on October 25, 2017 in Vancouver, Canada. OOPSLA is the top conference around object-oriented programming languages, systems and applications.

The article has since been cited more than 400 times. 

Read more articles about: