Commit messages play an important role in several software engineering tasks\nsuch as program comprehension and understanding program evolution. However,\nprogrammers neglect to write good commit messages. Hence, several Commit\nMessage Generation (CMG) tools have been proposed. We observe that the recent\nstate of the art CMG tools use simple and easy to compute automated evaluation\nmetrics such as BLEU4 or its variants. The advances in the field of Machine\nTranslation (MT) indicate several weaknesses of BLEU4 …