Skip to content

Latest commit

 

History

History
61 lines (48 loc) · 3.99 KB

README.md

File metadata and controls

61 lines (48 loc) · 3.99 KB
Raw

Supervised Learning for Social Bot Detection - Benchmarking Framework

To run the framework

  • Clone the repository.
  • Asking the creators of MIB or TwiBot-20 dataset (Due to the terms of use, we cannot public the dataset onto this site. Dataset requests for academic purpose are adrently accepted. Futher information can be found at: MIB Dataset and TwiBot-20 dataset).
  • In main.py, uncomment a line for declaring a pipeline and a line for running that pipeline. For example:
if __name__ == "__main__":
  pipeline = BasicPipeline()
  pipeline.run(dataset_name="MIB")
  • Check the evaluation
Dataset name: MIB (3892 samples)
Accuracy: 0.9645
Precision: 0.9711
Recall: 0.8911
MCC: 0.9073
Training time 0.1681s
Inference time: 0.0331s

To implement a new detector

  • Check the list of features
  • Inherent your detector from BaseDetectorPipeline. For example:
class NewPipeline(BaseDetectorPipeline):
  pass
  • Initialize the list of used features and level of detection via below parameters:
Parameter Type Description
user_features List, None or all List of user property features that are employed, they will be included in user_df (except the ones that do not exist in the dataset). If all, all available features will be used.
tweet_metadata_features List, None or all List of tweet metadata property features that are employed, they will be included in tweet_metadata_df (except the ones that do not exist in the dataset). If all, all available features will be used.
use_tweet bool Decide to use tweet semantic or not. If True, tweet_df will return a Dataframe. Otherwise, tweet_df will be None.
use_network bool Decide to use tweet semantic or not. If True, network_df will return a Dataframe. Otherwise, network_df will be None. Force to be False if using MIB dataset.
verbose bool If True, print out the process to console.
account_level bool Clarify the classifier is account-level or not (account-level detector is such detector that making predictions on each account).
  • Override some functions:
Function Arguments Optional Description
init Any Initialization and run __init__() of BaseDetectorPipeline. See examples for more details.
feature_engineering_u user_df: Dataframe or None, training: bool ✅ Process feature engineering on user property dataframe. Returned dataframe must include label and user_id column if concatenate function is not overridden. The training parameter indicates the dataset is in training set (false if in validation and test set).
feature_engineering_ts tweet_metadata_df: Dataframe or None, training: bool ✅ Process feature engineering on tweet metadata dataframe. Returned dataframe must include label and user_id column if concatenate function is not overridden.
feature_engineering_n network_df: Dataframe or None, training: bool ✅ Process feature engineering on network dataframe. Returned dataframe must include label and user_id column if concatenate function is not overridden. Currently not in use
semantic_encoding tweet_df: Dataframe or None, training: bool ✅ Encode the text on tweet dataframe. Returned dataframe must include label and user_id column if concatenate function is not overridden.
concatenate All dataframes ✅ Concatenate all dataframes into a single dataframe. Don't need to be overriden if only one dataframe is not None.
classify X_train, X_dev, y_train, y_dev Fit the data into a model. Must be implemented.
predict X_test Return the predicted result from the test data. Must be implemented.

You can check some examples in the src/example folder.