Trying to Download Data

Figure 0:

• (Fig. 0) Alter the following:
  • Rename the batch directory path to the folder to be created to store your primary dataset
  • (Optional) If a different Tag Order is desired, then use the existing tags in the text box to configure the order as needed
  • Prepend any set of tag/s needed --- these are filled in by default (remove and/or replace as necessary)
  • Append any set of tag/s needed --- these are filled in by default (remove and/or replace as necessary)
  • (Optional) If a new settings config is desired, then change the name of the JSON file in the textbox & check the Create New Config checkbox
  • (Optional) ANY non-default configs the user wants to use can be loaded automatically with the dropdown menu
  • Always remember to Apply & Save Settings when done making manual changes

Figure 1:

• (Fig. 1) Alter the following:
  • (Optional) Resize Param, can be completely toggled OFF in (Fig. 2) if not desired. If it is desired, then note that any image with it's Length or Width above than the slider value will be resized (down)
  • Set the threshold for all sliders to be relative to the MINIMUM requirements the downloader filters for when downloading data (i.e. if the user wants everything, then drag all the sliders to the far left)
  • Always remember to Apply & Save Settings when done making manual changes

Figure 2:

• (Fig. 2) Alter the following:
  • Only checked features are applied for each of the Collection, Downloading, & Resizing Steps (ONLY Apply what is desired)
  • Always remember to Apply & Save Settings when done making manual changes

Figure 3:

• (Fig. 3) Alter the following:
  • To Add Required Tag/s, either manually enter them comma separated, or drop a text file on the right panel to parse comma separated tag/s
  • (NOTE) ALL Tag/s present on the bottom panel will be included
  • To Remove Required Tag/s, check the tag/s to remove (RED) and click the Remove Button
  • Always remember to Apply & Save Settings when done making manual changes

Figure 4:

• (Fig. 4) Alter the following:
  • To Add Blacklist Tag/s, either manually enter them comma separated, or drop a text file on the right panel to parse comma separated tag/s
  • (NOTE) ALL Tag/s present on the bottom panel will be included
  • To Remove Blacklist Tag/s, check the tag/s to remove (RED) and click the Remove Button
  • Always remember to Apply & Save Settings when done making manual changes

Figure 5:

• (Fig. 5) Alter the following:
  • (Optional) Specify a path to a file with one id/md5 entry on each line; these will be skipped if present in the (Collected Data)
  • (Optional) Specify a path to a file where id/md5 entries (Collected) will be stored
  • (Optional) Specify a path to a file with one id/md5 entry on each line; these will be a whitelist of all posts to be (Collected)
  • (Optional) Apply ALL of the above (Collection) filters
  • (Optional) Change the name if desired, for any of the subdirectories of the batch folder
  • (Optional) Automatically download a pre-configured Negative Tag/s File
  • (Optional) Specify a path to a file with one tag entry on each line; Automatically removes all tags in file from all posts (Downloaded)
  • (Optional) Specify a path to a file with two comma separated tag entries on each line; Automatically replaces all tags in file in the first column from all posts (Downloaded) and replaces them with the tags in the second column
  • Always remember to Apply & Save Settings when done making manual changes

Figure 6:

• (Fig. 6) Alter the following:
  • Specify a path to the current working directory where the repository is running (should be autofilled)
  • (Optional) User can choose to provide the path to a manually downloaded csv and/or parquet file of either the tags and/or posts file if preferred
  • (Optional) Path to Auto-Config File ->>>> Please Refer to Use Case #7
  • (Optional) User can choose to select multiple settings files to run in sequence in the bottom panel & use the Batch Run Option
  • Always remember to Apply & Save Settings when done making manual changes
  • To Run the downloaded, simply press the Run Button to acquire the data via the settings.json file