Empirical Evaluation of Algorithm Performance: Addressing Execution Time Measurement Challenges

Abstract

In this paper, we investigate the influence of various factors, such as programming language, testing environment, and input data, on the accuracy of algorithm execution measurements. To conduct this study, we used the BubbleSort algorithm as a test case and implemented it in Java, C, and x86 assembly language. We executed these implementations with various inputs and performed an empirical evaluation of the results using the ALGator system. We showed that the influence of the chosen programming language is negligible, since the Java and C implementations gave very similar results, while the assembler implementation differed only by a constant factor. Furthermore, our analysis emphasized the importance of repeating tests to obtain precise timing measurements - the more tests we do, the more accurate the measured result will be. We also discuss the impact of the input data type which can significantly affect the execution time due to the increased number of mispredictions of the branch predictor.

References

T. H. Cormen, C. E. Leiserson, R. L. Rivest,

and C. Stein. Introduction to Algorithms.

The MIT Press, 2nd edition, 2001.

T. Dobravec. Algator — an automatic algo-

rithm evaluation system. Advances in Com-

puters, 116(1):65–131, 2019.

T. Dobravec. Exact time measuring chal-

lenges. In Proceedings of the 25th Interna-

tional Multiconference Information Society,

IS MATCOS, volume I, pages 21–24, Koper,

-14 October 2022.

M. Fern ́andez. Models of Computation,

An Introduction to Computability Theory.

Springer, 2009.

D. Johnson. A theoretician’s guide to

the experimental analysis of algorithms.

DOI:10.1090/dimacs/059/11, 12 2001.

R. Kumar. Instruction Level Parallelism:

Branch Prediction and Optimization. LAP

LAMBERT Academic Publishing, 2012.

C. C. McGeoch. Experimental analysis of

algorithms. Notices of the AMS, 48(3), 2001.

B. Moret. Towards a discipline of experi-

mental algorithmics. Monograph in Discrete

Mathematics and Theoretical Computer Sci-

ence, 2001.

M. Price. Hot code is faster code - addressing

jvm warm-up. QCon, April 2016.

B. Swathi. A comparative study and analy-

sis on the performance of the algorithms. In-

ternational Journal of Computer Science and

Mobile Computing, 5(1):91–95, Januar 2016.

M. Tedre and N. Moisseinen. Experiments in

computing: A survey. The Scientific World

Journal, 2014(1):1–11, 2014.

Authors

  • Tomaž Dobravec University of Ljubljana

DOI:

https://doi.org/10.31449/inf.v48i4.4752

Downloads

Published

01/07/2025

How to Cite

Dobravec, T. (2025). Empirical Evaluation of Algorithm Performance: Addressing Execution Time Measurement Challenges. Informatica, 48(4). https://doi.org/10.31449/inf.v48i4.4752

Issue

Vol. 48 No. 4 (2024)

Section

MATCOS-22

License

I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.

I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.

In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.

I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal.  The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.


Copyright ©  Slovenian Society Informatika