A Comprehensive Study on Quality Assurance Tools for Java Article Swipe
YOU?
·
· 2023
· Open Access
·
· DOI: https://doi.org/10.1145/3597926.3598056
· OA: W4378760032
Quality assurance (QA) tools are receiving more and more attention and are\nwidely used by developers. Given the wide range of solutions for QA technology,\nit is still a question of evaluating QA tools. Most existing research is\nlimited in the following ways: (i) They compare tools without considering\nscanning rules analysis. (ii) They disagree on the effectiveness of tools due\nto the study methodology and benchmark dataset. (iii) They do not separately\nanalyze the role of the warnings. (iv) There is no large-scale study on the\nanalysis of time performance. To address these problems, in the paper, we\nsystematically select 6 free or open-source tools for a comprehensive study\nfrom a list of 148 existing Java QA tools. To carry out a comprehensive study\nand evaluate tools in multi-level dimensions, we first mapped the scanning\nrules to the CWE and analyze the coverage and granularity of the scanning\nrules. Then we conducted an experiment on 5 benchmarks, including 1,425 bugs,\nto investigate the effectiveness of these tools. Furthermore, we took\nsubstantial effort to investigate the effectiveness of warnings by comparing\nthe real labeled bugs with the warnings and investigating their role in bug\ndetection. Finally, we assessed these tools' time performance on 1,049\nprojects. The useful findings based on our comprehensive study can help\ndevelopers improve their tools and provide users with suggestions for selecting\nQA tools.\n
