Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Minimal reproducible example for Libfuzzer #65

Open
akhikolla opened this issue Nov 4, 2020 · 11 comments
Open

Minimal reproducible example for Libfuzzer #65

akhikolla opened this issue Nov 4, 2020 · 11 comments

Comments

@akhikolla
Copy link
Owner

hello @tdhock,

Here is the Testharness, Makefile used to compile the code using libfuzzer.

rcpp_read_out_of_bound_DeepState_TestHarness.cpp :


#include <fstream>
#include <RInside.h>
#include <iostream>
#include <RcppDeepState.h>
#include <qs.h>
#include <DeepState.hpp>
#include <ctime>

int rcpp_read_out_of_bound(int rbound);

TEST(testSAN_deepstate_test,rcpp_read_out_of_bound_test){
  RInside R;
  std::cout << "input starts" << std::endl;
  std::ofstream rbound_stream;
  int rbound  = RcppDeepState_int();
  rbound_stream.open("/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/inp>
  rbound_stream << rbound;
  std::cout << "rbound values: "<< rbound << std::endl;
  rbound_stream.close();
  std::cout << "input ends" << std::endl;
  try{
     rcpp_read_out_of_bound(rbound);
  }
  catch(Rcpp::exception& e){
    std::cout<<"Exception Handled"<<std::endl;
  }
}

When we run the makefile :

akhila@ml-experiments:~/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound$ make -f libfuzz.Makefile 
clang++ -g -fsanitize=address,fuzzer -I/usr/share/R/include -I/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include -I/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppArmadillo/include -I/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/qs/include -I/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RInside/include -I/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/include /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness.cpp -o /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF.o -c
clang++ -g -fsanitize=address,fuzzer -o /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF.o -I/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/include -I/home/akhila/.RcppDeepState/deepstate-master/build_libfuzzer -I/home/akhila/.RcppDeepState/deepstate-master/src/include -L/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RInside/lib -Wl,-rpath=/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RInside/lib -L/usr/share/R/lib -Wl,-rpath=/usr/share/R/lib -L/home/akhila/.RcppDeepState/deepstate-master/build_libfuzzer -Wl,-rpath=/home/akhila/.RcppDeepState/deepstate-master/build_libfuzzer -lR -lRInside -ldeepstate -ldeepstate_LF -I/usr/share/R/include -I/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/Rcpp/include -I/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppArmadillo/include -I/home/akhila/.RcppDeepState/deepstate-master/src/include /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/src/*.cpp
cd /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound && ./rcpp_read_out_of_bound_DeepState_TestHarness_LF --fuzz --fuzz_save_passing 
DEBUG: INFO: libFuzzer ignores flags that start with '--'

DEBUG: INFO: Seed: 3529313378

DEBUG: INFO: Loaded 1 modules   (1693 inline 8-bit counters): 
DEBUG: 1693 [604720, 604dbd), 
DEBUG: 

DEBUG: INFO: Loaded 1 PC tables (1693 PCs): 
DEBUG: 1693 [5b41d8,5baba8), 
DEBUG: 

DEBUG: INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes

input starts
rbound values: 0
input ends
R is already initialized
    #0 0x52d7f1 in __sanitizer_print_stack_trace (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x52d7f1)
    #1 0x478948 in fuzzer::PrintStackTrace() (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x478948)
    #2 0x45db4c in fuzzer::Fuzzer::ExitCallback() (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x45db4c)
    #3 0x7f0f9a38ca26 in __run_exit_handlers /build/glibc-ZN95T4/glibc-2.31/stdlib/exit.c:108:8
    #4 0x7f0f9a38cbdf in exit /build/glibc-ZN95T4/glibc-2.31/stdlib/exit.c:139:3
    #5 0x7f0f9a964c2a in Rf_initialize_R (/lib/libR.so+0x269c2a)
    #6 0x7f0f9a95f4ec in Rf_initEmbeddedR (/lib/libR.so+0x2644ec)
    #7 0x7f0f9a6e804f in RInside::initialize(int, char const* const*, bool, bool, bool) /tmp/RtmpppBlNo/R.INSTALL1906e07ad5449/RInside/src/RInside.cpp:155:21
    #8 0x7f0f9a6e8483 in RInside::RInside() /tmp/RtmpppBlNo/R.INSTALL1906e07ad5449/RInside/src/RInside.cpp:61:15
    #9 0x55e306 in DeepState_Test_testSAN_deepstate_test_rcpp_read_out_of_bound_test() /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness.cpp:12:11
    #10 0x556cf8 in DeepState_Run_testSAN_deepstate_test_rcpp_read_out_of_bound_test() /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness.cpp:11:1
    #11 0x56f7f7 in DeepState_RunTestNoFork (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x56f7f7)
    #12 0x56f60a in LLVMFuzzerTestOneInput (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x56f60a)
    #13 0x45f151 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x45f151)
    #14 0x45e895 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x45e895)
    #15 0x461071 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x461071)
    #16 0x461519 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x461519)
    #17 0x4501ee in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x4501ee)
    #18 0x479032 in main (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x479032)
    #19 0x7f0f9a36a0b2 in __libc_start_main /build/glibc-ZN95T4/glibc-2.31/csu/../csu/libc-start.c:308:16
    #20 0x424f8d in _start (/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/rcpp_read_out_of_bound_DeepState_TestHarness_LF+0x424f8d)

make: *** [libfuzz.Makefile:5: /home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/libfuzzer_rcpp_read_out_of_bound_log] Error 77

The code aborts because of an internal call to exit() from the RInside object initialization.
@tdhock
Copy link

tdhock commented Nov 4, 2020

hi @eddelbuettel I know this is not easily reproducible (without installing libfuzzer,deepstate) but do you have any idea why exit is being called here? libfuzzer does not like that
From the stack trace it looks like it is being called when RInside is instantiated, RInside R; which calls RInside::initialize, Rf_initEmbeddedR, Rf_initialize_R, and then exit. Why would R call exit and how can we avoid that? I'm pretty confused here.

@eddelbuettel
Copy link

The call sequence is from #1 to #20, isn't it? I think the exit() you see in #3 and #4 may be spurious.

But in any event, and you could check that via grep -r, we don't call exit() in Rcpp or RInside and I doubt R does. So I don't know, and I don't have any leads.

@tdhock
Copy link

tdhock commented Nov 4, 2020

Hi Dirk thanks for the quick response, that helped actually.
There are three places I see exit in
https://github.com/wch/r-source/blob/9c9e8ab03b499f0da3c490238d40fb5b9f904fb4/src/unix/system.c
I bet the culprit is https://github.com/wch/r-source/blob/9c9e8ab03b499f0da3c490238d40fb5b9f904fb4/src/unix/system.c#L197 which calls exit if num_initialized is non-zero, which I think means that R has already been initialized. Is libfuzzer possibly running the test harness code / RInside instantiation multiple times in the same process @akhikolla ? If you could try fixing by moving the Rinside instantiation outside/before the TEST() block.
The num_initialized is defined here

/* Protection against embedded misuse, PR#15420 */
static int num_initialized = 0;

This R-devel bug https://bugs.r-project.org/bugzilla/show_bug.cgi?id=15420 did not really help me much to understand what is going on.

@akhikolla
Copy link
Owner Author

I don't think libfuzzer instantiate R more than once. Libfuzzer is an in-process fuzzer which means fuzzing happens in only one process, for every test case, the process isn't restarted but the input values are changed in the memory.
I can work on changing the R instantiation inside the testharness and maybe I'll create the main function just for R initialization?

@eddelbuettel
Copy link

That could help. Try to do setup only once, and then just call workloads. You cannot run the <construct,destruct> pair in a loop as R was never built for that; I think some internal may not reset.

BTW I think your repo is not complete and the example is not self-contained: rbound_stream.open("/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/inp>

@akhikolla
Copy link
Owner Author

Thanks for your suggestion. Test Fixtures in deepstate provide SetUp() and TearDown() where we can initialize an object, destruct an object respectively.
I apologize. I didn't update it properly.

Here is the complete code:

#include <fstream>
#include <RInside.h>
#include <iostream>
#include <RcppDeepState.h>
#include <qs.h>
#include <DeepState.hpp>
#include <ctime>

int rcpp_read_out_of_bound(int rbound);

TEST(testSAN_deepstate_test,rcpp_read_out_of_bound_test){
  RInside R;
  std::cout << "input starts" << std::endl;
  std::ofstream rbound_stream;
  int rbound  = RcppDeepState_int();
  rbound_stream.open("/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/inputs/rbound");
  rbound_stream << rbound;
  std::cout << "rbound values: "<< rbound << std::endl;
  rbound_stream.close();
  std::cout << "input ends" << std::endl;
  try{
     rcpp_read_out_of_bound(rbound);
  }
  catch(Rcpp::exception& e){
    std::cout<<"Exception Handled"<<std::endl;
  }
}

@eddelbuettel
Copy link

What I meant is whether this file is public or not:

/home/akhila/R/x86_64-pc-linux-gnu-library/3.6/RcppDeepState/testpkgs/testSAN/inst/testfiles/rcpp_read_out_of_bound/inputs/rbound"

The path you use is not. You could access a file in a package via system.file(..., package=....) .

@akhikolla
Copy link
Owner Author

The file is not public. The testharness which I presented is automatically generated by the R code and Yes I make use of system.file to get the path and I write the obtained path to the Testharness.cpp file. For every Rcpp function in the package, the R code generates a Testharness and makes a call to the rcpp function inside the testharness.

@eddelbuettel
Copy link

The file is not public.

Is there a deeper philosphical reason I am not seeing? Everything else here is open source.

@akhikolla
Copy link
Owner Author

The R code uses system.file() to get the path to the "rbound" file in the RcppDeepState package and writes it to the Testharness.
The Testharness files generated are system-specific and paths provided in the file are system-specific too.
These Testharness files, makefile are written by R.
There is no specific reason why we are not using system.file in the Testharness.cpp directly.

All the inputs passed to the TestHarness by deepstate are usually stored in binary files(.crash/.fail/.pass files). Reading data from these files is not possible so we created a file(rbound) that stores the inputs provided by RcppDeepState_int().
We just save the inputs so that we know which input is causing the error on the code.

@FabrizioSandri
Copy link

FabrizioSandri/RcppDeepState#7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@eddelbuettel @tdhock @FabrizioSandri @akhikolla