Failure Auto Scan and Batch Mode Scripts

Goal

When using LLFI for fault injections, a user may want to learn which type of failures is applicable to a targeting program before he or she starts the fault injection simulation. This requires LLFI to be able to scan the targeting program and return the applicable instruction selector, register selector and fault injector automatically. Besides, after a list of applicable failure modes being returned, the user may want to run simulation with each failure mode one by one. To fulfill these demands, LLFI provides following commands:

• HardwareFailureAutoScan
• SoftwareFailureAutoScan
• batchInstrument
• batchProfile
• batchInjectFault

Failure Auto Scanners

###1. Hardware Failure Auto Scan HardwareFailureAutoScan will scan all the hardware fault instruction selectors, hardware fault register selectors and hardware fault injectors registered in LLFI. This command is independent from the targeting IR since hardware faults are applicable for all programs. However, one targeting IR as argument is still required to determine the working directory to save the output and keep the format unified.

<LLFI_BUILD_ROOT>/bin/HardwareFailureAutoScan --help
Usage: HardwareFailureAutoScan [OPTIONS] <source IR file>

List of options:

-outputfilename=<filename>: set the name of the file that stores the list of applicable hardware selectors (instruction selector/reg selector) (default: llfi.applicable.hardware.selectors.txt)
Note: If <filename> is a relative path instead of an absolute path, the base path of <filename> will be the path of the targeting IR file instead of the calling path.

--help: print this message.

By default the output file name is llfi.applicable.hardware.selectors.txt. Below is an example of running this command upon the test program under: <LLFI_BUILD_ROOT>/test_suite/BatchMode/SoftwareFailureAutoScan/. Note this command is independent from targeting IR:

<LLFI_BUILD_ROOT>/bin/HardwareFailureAutoScan memcpy1.ll

This will generate file: llfi.applicable.hardware.selectors.txt:

instSelMethod:
    - funcname
    - insttype
    - llfiindex
    - onlymain
regSelMethod:
    - onlyconstint
    - regloc

###2. Software Failure Auto Scan SoftwareFailureAutoScan scans all the software failures supported by LLFI (a software failure is a bundle of software instruction selector, software register selector and software fault injector), and return all the applicable software failure modes for the targeting program. By default, this will also generate a master input.yaml file which contains all the applicable failure modes to be utilized by the batch mode scripts.

<LLFI_BUILD_ROOT>/bin/SoftwareFailureAutoScan 
Usage: SoftwareFailureAutoScan [OPTIONS] <source IR file>

List of options:

-outputfilename=<filename>: set the name of the file that stores the list of applicable software failures (default: llfi.applicable.software.failures.txt)
 Note: If <filename> is a relative path instead of an absolute path, the base path of <filename> will be the path of the targeting IR file instead of the calling path.
 
-numOfRuns <number of runs>: set the number of runs for each found failure mode (default: 1)
--enable_tracing: enable tracing
--enable_forward_injection: enable injection on the forward slice of the selected injection point
--enable_backward_injection: enable injection on the backward slice of the selected injection point
--no_input_yaml: do not generate an master input.yaml automatically.

--help: print this message.

Here is an example on running this script upon the test program under <LLFI_BUILD_ROOT>/test_suite/BatchMode/SoftwareFailureAutoScan/: Run in terminal:

<LLFI_BUILD_ROOT>/bin/SoftwareFailureAutoScan -numOfRuns 100 --enable_backward_injection memcpy1.ll

This will generate two files: llfi.applicable.software.failures.txt and input.yaml: llfi.applicable.software.failures.txt:

instSelMethod:
    - BufferOverflow(API)
    - BufferOverflowMalloc(Data)
    - BufferOverflowMemmove(Data)
    - BufferUnderflow(API)
    - CPUHog(Res)
    - DataCorruption(Data)
    - HighFrequentEvent(Timing)
    - InappropriateClose(API)
    - IncorrectOutput(API)
    - IncorrectOutput(Data)
    - InvalidPointer(Res)
    - LowMemory(Res)
    - MemoryExhaustion(Res)
    - MemoryLeak(Res)
    - NoClose(API)
    - NoOpen(API)
    - NoOutput(API)
    - NoOutput(Data)
    - StalePointer(Res)
    - UnderAccumulator(Res)
    - WrongAPI(API)
    - WrongDestination(Data)
    - WrongMode(API)
    - WrongPointer(Data)
    - WrongRetrievedAddress(IO)
    - WrongRetrievedFormat(IO)
    - WrongSavedAddress(IO)
    - WrongSavedFormat(IO)
    - WrongSource(Data)

input.yaml:

compileOption:
  customRegSelector: Automatic
  includeInjectionTrace:
  - forward
  instSelMethod:
  - customInstselector:
      include:
      - BufferOverflow(API)
      - BufferOverflowMalloc(Data)
      - BufferOverflowMemmove(Data)
      - BufferUnderflow(API)
      - CPUHog(Res)
      - DataCorruption(Data)
      - HighFrequentEvent(Timing)
      - InappropriateClose(API)
      - IncorrectOutput(API)
      - IncorrectOutput(Data)
      - InvalidPointer(Res)
      - LowMemory(Res)
      - MemoryExhaustion(Res)
      - MemoryLeak(Res)
      - NoClose(API)
      - NoOpen(API)
      - NoOutput(API)
      - NoOutput(Data)
      - StalePointer(Res)
      - UnderAccumulator(Res)
      - WrongAPI(API)
      - WrongDestination(Data)
      - WrongMode(API)
      - WrongPointer(Data)
      - WrongRetrievedAddress(IO)
      - WrongRetrievedFormat(IO)
      - WrongSavedAddress(IO)
      - WrongSavedFormat(IO)
      - WrongSource(Data)
  regSelMethod: customregselector
  tracingPropagation: false
  tracingPropagationOption:
    generateCDFG: false
runOption:
- run:
    fi_type: AutoInjection
    numOfRuns: 100

Please note that HardwareFailureAutoScan won't generate input.yaml file automatically. And user can disable the auto input.yaml generation with option: --no_input_yaml.

Batch Mode Scripts

SoftwareFailureAutoScan returns a list of applicable software failure modes, then the next step is to simulate each mode of the list. This is the objective of batch mode scripts. Batch mode scripts contain three parts:

• batchInstrument
• batchProfile
• batchInjectfault

Each one corresponds one standard step of LLFI.

NOTE: batch mode script only support multiple software failures, multiple hardware instruction selectors or register selectors in single input.yaml are not supported.

###1. batchInstrument batchInstrument accepts one targeting IR path and requires an input.yaml file under the IR directory. It will make a new subdirectory under the IR directory for each software failure which are listed in the input.yaml (the one under the same directory of targeting IR.

Each new directory will contain all the files under the original targeting IR directory, which should include IR file, input files, etc. And a new input.yaml file will be created in each new directory with only ONE specific software failure selected in it. So each subdirectory will have sufficient files to run standard LLFI scripts independently. Then for each subdirectory, standard LLFI instrument script will be called to process the IR code for each software failure mode. However, users can also use batchProfile and batchInjectfault to stick on batch mode for following steps.

The naming rule of the created subdirectories is: llfi-< software failure mode name >/.

batchInstrument also accepts all the options of standard instrument. All these options will be passed to standard LLFI instrument script and be used when standard instrument script is called.

Example of running batchInstrument on the test program under: <LLFI_BUILD_ROOT>/test_suite/BatchMode/NoOpen_API_WrongMode_API_BufferUnderflow_API/

The input.yaml under NoOpen_API_WrongMode_API_BufferUnderflow_API/. Note it lists three software failures:

compileOption:
    instSelMethod:
      - customInstselector:
          include:
            - NoOpen(API)
            - WrongMode(API)
            - BufferUnderflow(API)
    regSelMethod: customregselector
    customRegSelector: Automatic
runOption:
    - run:
        numOfRuns: 5
        fi_type: AutoInjection

Run in terminal:

<LLFI_BUILD_ROOT>/bin/batchInstrument --readable memcpy1.ll

This will generate following three subdirectories, for each of them, llfi/ directory is created:

NoOpen_API_WrongMode_API_BufferUnderflow_API/
├── llfi-BufferUnderflow(API)
│   └── llfi
├── llfi-NoOpen(API)
│   └── llfi
└── llfi-WrongMode(API)
    └── llfi

###2. batchProfile batchProfile follows the same manner of standard LLFI profile script, except the user should not specify ./llfi/<targeting_IR_name>-profiling.exe <input_arguments> but <targeting_IR_name> <input_arguments> directly. Note this script should be called in the directory of the original IR file, not the subdirectories created by batchInstrument.

Here is an example of using batchProfile upon <LLFI_BUILD_ROOT>/test_suite/BatchMode/NoOpen_API_WrongMode_API_BufferUnderflow_API/:

<LLFI_BUILD_ROOT>/bin/batchProfile memcpy1.ll

This script will call standard LLFI profile script for each subdirectory one by one.

###3. batchInjectfault batchInjectfault follows the same manner of standard LLFI injectfault script, except the user should not specify ./llfi/<targeting_IR_name>-faultinjection.exe <input_arguments> but <targeting_IR_name> <input_arguments> directly. Note this script should be called in the directory of the original IR file, not the subdirectories created by batchInstrument.

Here is an example of using batchProfile upon <LLFI_BUILD_ROOT>/test_suite/BatchMode/NoOpen_API_WrongMode_API_BufferUnderflow_API/:

<LLFI_BUILD_ROOT>/bin/batchInjectfault memcpy1.ll

This script will call standard LLFI injectfault script for each subdirectory one by one. After fault injection, each subdirectory contains the results and statistics file same to using standard LLFI scripts.