End-to-end tracing considered essential

Concurrency and complexity are major obstacles to understanding application performance in high-performance systems. We advocate end-to-end event tracing as the correct way to expose performance information for both human and automated analysis. We describe its advantages over traditional performance counter data, and illustrate its uses in performance visualization, prediction for capacity planning, and anomaly detection. We conclude with a look at opportu- nities and challenges presented by a widespread deployment of end-to-end tracing.