Quick Update on C++ Compiler Bug Hunt (overall, 15 bugs reported, 8 already fixed)

	Author:	“No Bugs” Hare  Follow:
	Job Title:	Sarcastic Architect
	Hobbies:	Thinking Aloud, Arguing with Managers, Annoying HRs, Calling a Spade a Spade, Keeping Tongue in Cheek

Some time ago, I wrote a post about kscope – mini-project on finding and reporting bugs in modern C++ compilers. The point was that there is a way to use C++ facilities to make C++ code self-mutating (more strictly – pseudo-randomized using externally supplied ITHARE_KSCOPE_SEED macro), which allows finding certain classes of bugs during randomized testing. In the previous post, I wrote about 12 bugs reported, with 3 of them already fixed. I have to admit that since that point, I didn’t have as much time as I’d like to spend on kscope; still, 3 more bugs were reported (2 of them being codegen bugs) – and quite a few were fixed (fortunately, bugfixing is one thing which doesn’t require my participation <wink />).

As a result, as of early May, our table of reported-and-fixed bugs looks as follows:

Short observation shows that MSVC has much more bugs detected by this project than GCC and Clang; while exact reasons are of course, unknown, my really wild guess is that’s because codegen in GCC and Clang is already randomly-tested by a comparable-to-kscope (actually, better) tool – CSmith. As CSmith has already reported 79 bugs in GCC and 203 bugs in LLVM (and most of these reported bugs got fixed), it means much fewer chances for kscope to find anything else there. Bottom line:

And of course, as I already wrote, what is really important is not the number of bugs found, but the rate at which bugs are fixed – and in this respect MSVC looks pretty good.

That’s it for today…