Test Descriptions

Test	Severity	Type
Array Overflow	3 🔴	Overflow
Concat Overflow	3 🔴	Overflow
Gets Overflow	3 🔴	Overflow
Printf Format Overflow	3 🔴	Overflow
Printf String Overflow	3 🔴	Overflow
Quote Overflow 1	3 🔴	Overflow
Quote Overflow 2	3 🔴	Overflow
Quote Overflow 3	3 🔴	Overflow
Scanf Overflow	3 🔴	Overflow
Double Free 1	3 🔴	Memory
Double Free 2	3 🔴	Memory
Invalid Pointer	2 🟠	Memory
Memory Leak	1 🟡	Memory
Null Pointer	2 🟠	Memory
Realloc Invalid	2 🟠	Memory
Reallocf Free	2 🟠	Memory
Use After Free 1	3 🔴	Memory
Use After Free 2	3 🔴	Memory
Divide By Zero 1	2 🟠	Logical
Divide By Zero 2	2 🟠	Logical
Malloc Zero Access	2 🟠	Undefined Behavior
Stack Return 1	3 🔴	Undefined Behavior
Stack Return 2	3 🔴	Undefined Behavior
Stack Return 3	3 🔴	Undefined Behavior
String Literal Modify	2 🟠	Undefined Behavior
Uninitialized Value 1	2 🟠	Undefined Behavior
Uninitialized Value 2	2 🟠	Undefined Behavior
Malloc Error	1 🟡	Common User Error
Switch Fall Through	1 🟡	Common User Error

Severity Score	Description
3 🔴	There are security implications.
2 🟠	Your code may/will crash.
1 🟡	It's an issue, but the code may run.

Static Analyzer Test Results

Test	Sonarlint v3.20.2	Snyk 1.20.3	CLion 2023.2	GCC 13.2.0	Clang 16.0.0	icx 2023.1.0	msvc v19.37	clang-tidy	ChatGPT	CodeQL 2.15.1
Array Overflow	✅		✅	✅	✅	✅	✅	✅	✅
Concat Overflow							✅*	✅*	✅*
Gets Overflow	✅*	✅*		✅	✅*	✅*	✅	✅*	✅*	✅*
Printf Format Overflow	✅*	✅	✅				✅*	✅	✅	✅
Printf String Overflow							✅
Quote Overflow 1	✅			✅&
Quote Overflow 2	✅
Quote Overflow 3									✅
Scanf Overflow	✅*								✅*	✅
Double Free 1	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Double Free 2	✅	✅		✅*	✅	✅		✅	✅*
Invalid Pointer									✅*
Memory Leak	✅				✅	✅	✅&	✅	✅
Null Pointer	✅			✅	✅	✅		✅	✅*
Realloc Invalid	✅			✅	✅	✅		✅	✅
Reallocf Free	✅			✅	✅	✅		✅	✅
Use After Free 1	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
Use After Free 2			✅	✅			✅			✅*
Divide By Zero 1	✅		✅	✅	✅	✅	✅	✅	✅
Divide By Zero 2									✅*
Malloc Zero Access	✅*			✅	✅	✅		✅*	✅	✅*
Stack Return 1	✅		✅	✅	✅	✅	✅	✅	✅*	✅
Stack Return 2	✅		✅	✅	✅	✅	✅	✅	✅*	✅
Stack Return 3	✅			✅	✅	✅		✅	✅*
String Literal Modify				✅					✅*
Uninitialized Value 1	✅			✅	✅	✅		✅
Uninitialized Value 2	✅		✅	✅	✅	✅	✅	✅	✅
Malloc Error				✅			✅		✅	✅
Switch Fall Through	✅*			✅					✅

* True Positive and False Positive & Issued Complexity Warning

Detection Results

• Sonarlint v3.20.2 16 / 29 (55%)

  • Quote Overflow 1
  • Quote Overflow 2

• Snyk 1.20.3 4 / 29 (14%)

• CLion 2023.2 9 / 29 (31%)

• GCC 13.2.0 18 / 29 (62%)

  • String Literal Modify
  • Switch Fall Through

• Clang 16.0.0 15 / 29 (52%)

• icx 2023.1.0 15 / 29 (52%)

• msvc v19.37 11 / 29 (38%)

  • Printf String Overflow

• clang-tidy 15 / 29 (52%)

• ChatGPT 13 / 29 (45%)

  • Quote Overflow 3

Sanitizer Test Results

Test	No San	Clang 11.0.3	Clang 14.0.0	GCC 13.2.0
Array Overflow		✅	✅	✅
Concat Overflow	✅	✅	✅	✅
Use After Free	✅	✅	✅	✅
Null Pointer	✅	✅	✅	✅
Invalid Pointer	✅	✅	✅	✅
Uninitialized Value
printf Overflow		✅	✅	✅
malloc Error

GCC

Version 13.2.0

-fanalyzer -Wanalyzer-too-complex -Wall -Wextra -Werror -pedantic -std=c11 -O2 -g

Clang

Version 16.0.0

--analyze -Wall -Wextra -Werror -pedantic -std=c11 -O2 -g

ICX

Version 2023.2.1

--analyze -Wall -Wextra -Werror -pedantic -std=c11 -O2 -g

MSVC

Version v19.37

/Wall /WX /wd4710 /wd4820 /wd4996 /wd5045 /wd6255 /std:c11 /analyze /DEBUG /Zi

• /wd4710 is needed as otherwise a warning is given for the compiler
• /w35045 is needed to avoid warnings about speculative execution mitigation
• /wd4996 is needed to allow open() on Windows. Does disable the scanf() check.

ChatGPT

ChatGPT August 3 Version

Analyze the following C program for any issues that could cause it to crash. Please report if any security issues or other issues are present in the program.

Report on any issues that would cause a program to potentially crash. If the program has no serious issues, say "there are no issues".

*** Header comment removed

Analyzing on the first few sentences of its answer So long as it get the point, it counts as a pass, even if it did state incorrect information about the code.

Goldbolt

Godbolt was used to evaluate compilers

Goldbolt IDE

GraphQL

codeql database create codeql_scan --language cpp
codeql pack install
codeql database analyze codeql_scan --format=csv --output=scan_results.csv