A blogged previously about this study when we made it available as a pre-print. Now I'm happy to report that it has passed peer reviews and is published in PeerJ Computer Science.

We used solutions to programming problems of Google Code Jam. The solutions to a programming problem have to be functionally similar. Therefore, we can consider them functionally similar clones (i.e. the code fragments provide similar functionality but are not necessarily syntactically similar). Clones with similar syntax, usually the result of copy & paste, can be found reliably with contemporary clone detection tools. Functionally similar clones, however, are still a large challenge.

We used the clone detection functionality in ConQAT and DECKARD to investigate how much syntactic similarity is there in these functionally similar code clones. They probably have not been created by copy & paste. We found that the syntactic similarity is below 16 %; often it is much smaller. To better understand the remaining differences, we categorised them manually. We found differences in the algorithms, data structures, OO design, I/O and the usage of libraries. While this classification may seem trivial at first sight, it really does capture well what has to be addressed in detecting such clones.

We provide based on this categorisation a freely available benchmark representing these differences along with all the data used in the study. We hope this helps the clone detection research community to move the detection of functionally similar code clones forward.

We plan to use the benchmark ourselves now to work on a new detection approach for functionally similar code clones. If you would like to collaborate, please get in touch!

Also: If you are a practitioner and have seen functionally similar clones in your code, please get in touch as well. We are planning to conduct a study on the relevance of such clones in practice.

We are looking forward to your comments and feedback!

Source: https://peerj.com/articles/cs-49/?td=bl