Skip to content

Latest commit

 

History

History
2177 lines (1509 loc) · 180 KB

CHANGELOG.md

File metadata and controls

2177 lines (1509 loc) · 180 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

8.9.0

Added

  • Agent: Added fuzz tests for coverage recording #3322
  • Agent: Added version checking in local tasks #3517
  • Agent: Create directories from template specification in local task if they don't exist #3522
  • CLI: Added a new command for template creation in the local task onefuzz-task local create-template #3531
  • CLI/Deployment/Service: Support for retention policies on containers #3501
  • Service: Add onefuzz service version to job created events #3504
  • Service: Added a start time to job and task records #3440

Changed

  • Agent: Improved handling of unexpected breakpoints #3493
  • Agent: Updated windows interceptor list #3528, #3549
  • Agent: Reporting coverage on task start up, ensuring coverage_data is emitted at the beginning of every task instead of when new_coverage is identified #3502
  • CLI/Deployment: Updating onefuzz cli requirements.txt to accept >= onefuzztypes versions #3477, #3486
  • Service: Improve area/iteration path validation in notifications #3489
  • Service: Remove feature flag from heartbeat metrics #3505

Fixed

8.8.0

Added

  • Agent: Added Mariner Linux support for agent VMs #3306
  • Service: Added support for custom ado fields that mark work items as duplicate #3467
  • Service: Permanently store OneFuzz job result data - # crashing input, # regression crashing input, etc. - in Azure storage #3380, #3439
  • Service: Added validation for Iteration/AreaPath on notifications when a job is submitted with a notification config and for onefuzz debug notification test_template #3386

Changed

  • Agent: Updated libfuzzer-fuzz basic template to include required args and make it match cli #3429
  • Agent: Downgraded some debug logs from warn to debug #3450
  • CLI: Removed CLI commands from the local fuzzing tasks as they can now be described via yaml template #3428
  • Service: AutoScale table entries are now deleted on VMSS shutdown #3455

Fixed

8.7.1

Fixed

  • Service: Removed deprecated Azure retention policy setting that was causing scaleset deployment errors #3452

8.7.0

Added

  • Agent: Added a snapshot-based test to coverage implementation #3368
  • Agent/CLI/Service: Added ability to capture crash dumps from libfuzzer, when provided #2793 #3409
  • CLI/Service: Implemented --with_tasks option for onefuzz jobs get command to expand the task information #3343

Changed

  • Agent: Migrated all the task types to the template model #3397
  • Agent: Removed srcview code from OneFuzz since it is not currently utilized #3376
  • Agent: Updated default windows VM image to windows 11 #3374
  • Agent: Migrated winapi to windows-rs, the newer Microsoft supported version of the Windows API bindings for Rust #3050
  • Deployment: Updated the default deployment option for EnableWorkItemCreation feature flag to be enabled #3387

Fixed

  • Agent: Deserialize the coverage files directly into the output files #3410
  • Agent/Deployment/Service: Bumped several C#, Python, and Rust dependencies as well as the Rust edition across all Rust crates #3396, #3161, #3346, #3391, #2870, #3392, #3402
  • Agent: Fixed a bug in agent DirectoryMonitor by adding error tolerance when attempting to fetch metadata for CreateKind::Any or CreateKind::Other events #3393
  • Service: Fixed tag shadowing in logging by giving precedence to the tags produced by log messages over the tags added prior to the call, when the tag names clashed #3388

8.6.3

Fixed

  • Service: Fixed another duplicate Azure DevOps work item creation case by handling Microsoft.VSTS.Common.ResolvedReason field when present #3383

8.6.2

Fixed

  • Agent: Fixed tasks hanging when shutting down by forcefully shutting down the runtime before exiting the main task #3378
  • Service: Refactored Azure DevOps template rendering to fix duplicate bugs being filed due to title truncation and added several validation tests in this area #3370

8.6.1

Added

  • Service: Added feature flag to toggle Azure DevOps work item processing #3353
  • Service: Requeue Azure DevOps notifications when the feature flag for work item processing is set to 'disabled' #3358

8.6.0

Added

  • Agent: Implemented debuginfo caching #3280

Changed

  • Agent: Limit azcopy copy buffer to 512MB of RAM as the default maximum #3293
  • Agent: Define local fuzzing tasks relationships through new templating model #3117
  • Deployment: Replaced --upgrade flag with --skip_aad_setup flag in the deploy.py setup script #3345
  • Service: Make ServiceConfiguration eagerly evaluated #3136
  • Service: Improved TimerRetention performance through several UPN changes & fixes #3289

Fixed

  • Agent: Fixed resolution of sibling .NET DLLs #3325
  • Agent/Service: Bumped several C# and Rust dependencies #3319, #3320, #3317, #3297, #3301, #3291, #3195, #3328
  • CLI: Look for azcopy.exe in environment variable AZCOPY and determine if it's actually referencing a directory #3344
  • CLI: Updated repro get_files to handle regression reports #3340
  • CLI: Fixed missing target_timeout setting in the Libfuzzer basic template #3334
  • CLI: Fixed false 'missing' dependency warning #3331
  • CLI: Fixed the debug notification test_template command expecting a task_id #3308
  • Deployment: Update App Registration redirect URIs if deployment uses a custom domain #3341
  • Service: Fixed links in bugs filed from regression reports by populating InputBlob when possible #3342
  • Service: Fixed several storage issues to improve platform performance and reduce spurious 404s #3313
  • Service: Added extra logging when System.Title is too long #3332
  • Service: Render System.Title before trying to trim it to the max allowed size #3329
  • Service: Differentiate INVALID_JOB and INVALID_TASK error codes #3318

8.5.0

Added

  • Agent: Added tool to check source allowlists #3246
  • Agent: Precache debuginfo analysis for target exe in coverage example #3225
  • Agent/CLI/Service: Allow tasks environment variables to be set #3294
  • CLI/Service: Correlate cli to service to facilitate event lookups in AppInsights #3137
  • CLI: Added --target_timeout flag for qemu_user template command #3277
  • Documentation: Updated Threat Model #3215
  • Service: Added optional Unless condition when updating/re-opening Work Items #3227
  • Service: Include the task ID in the prerequisite task failure message #3219
  • Service: Added events retention policy passed-integration-tests #3186

Changed

  • Agent: Shrink published Rust debug info #3247, #3252
  • Agent: Get rid of yanked hermit-abi versions #3270
  • Documentation: Updated coverage docs to use correct quotes #3279
  • Service: Better errors from Download: Make GetFileSasUrl nullable #3229
  • Service: Changed template rendering from async to synchronous #3241
  • Service: Log webhook exception as an "error" since we are retrying anyways #3238
  • Service: Make WebhookMessageEventGrid compatible with the event grid format #3286

Fixed

  • Agent: Improved .dll redirection by setting up .local file before invoking LibFuzzer #3269
  • Agent/Service: Bumped several C#, Rust dependencies, and Rust version to 1.71 #3278, #3281, #3221, #3230, #3231, #3203, #3240, #3239, #3199, #3254, #3257, #3273, #3258, #3271, #3292
  • CLI/Service: Fixed regression bugs, file bugs on regression_report and properly reset state on duplicates #3263
  • Service: Improve Azure DevOps validation problem reporting and resiliency #3222
  • Service: Updated KeyVault access policy for Azure WebSites service account access #3109
  • Service: Switched to default HttpCompletion, which is ResponseRead to attempt to prevent webhooks occasionally failing to send #3259
  • Service: Fixed Timestamp response from API #3237
  • Service: Trim System.Title if length is longer than 128 characters #3284

8.4.0

Added

  • Agent: Include debug info in the release binaries to improve backtraces and debuggability #3194
  • Agent: Added a timeout when closing the app insight channels #3181
  • Agent: Require input marker in arguments when given an input corpus directory #3205
  • Agent/CLI/Service: Added extra_output container, rename extra container #3064
  • Agent: Creating CustomMetrics for Rust CustomEvents #3188
  • Agent: Added prereqs for implementing caching for coverage locations and debuginfo in coverage task #3218
  • CLI: Added command onefuzz repro get_files for downloading files to locally reproduce a crash #3160
  • CLI: Added command onefuzz debug notification test_template <template> [--task_id <task_id>] [--report <report>] to allow a report to be sent when debugging #3206
  • Documentation: Added documentation on how to use the validation tools #3212

Changed

  • Agent: Removed agent traces from AppInsights #3143
  • Agent: Include debug info in the release binaries to improve backtraces and debuggability #3194
  • Agent: Make coverage-recording errors non-fatal #3166
  • Deployment/Service: Enable custom metrics app config value #3190
  • Documentation: Renamed example coverage.rs to record.rs to match documentation #3204
  • Service: Moved authentication into middleware #3133
  • Service: Store authentication information in KeyVault #3127, #3223
  • Service: Port current logging implementation to ILogger #3173
  • Service: Added improved error reporting from scale-in protection modification #3184
  • Service: Downgraded queue error to warning when retrying because the message is too large #3224

Fixed

8.3.0

Changed

  • CLI/Service: Don’t validate error codes on client side #3131

Fixed

  • Agent: Switched from unmaintained Rust dependency tui to ratatui #3155
  • Agent: Removed dependency on the abandoned Rust users crate #3150
  • Agent/CLI/Service: Bumped several C#, Python, and Rust dependencies #3118, #3132, #3088, #3106, #3140, #3120, #3145, #3151
  • CLI/Service: Include a reason when a task has never started #3148
  • Service: Fixed bug for scale-in protection #3144

8.2.0

Added

  • Service: Created CustomMetrics for the Node and Task Heartbeat. #3082
  • Service: Add an event for Repro VM creation. #3091
  • Service: Add more context to the deletion of nodes. #3102
  • Documentation: Create documentation for events 2.0 migration. #3098

Changed

  • Agent: Match the agent version to the server #3093
  • Service: Increase lock wait timeout for qemu_user setup script. #3114

Fixed

  • Service: Fixed issue that incorrectly marked tasks as failed. #3083
  • Service: Fixed bug when truncating reports. #3103
  • Service: Allow use of readonly_inputs for qemu_user template. #3116
  • Service: Fix logic to set check_fuzzer_help. #3130
  • CLI: Fix CLI failure dude to ErrorCode enums out of sync. #3129

8.1.0

Added

  • Agent: Added coverage percentage in Cobertura reports #3034
  • Agent: Added maxPerPage to ORM #3016
  • CLI: Added onefuzz containers files download command to download the blob content to a file #3060

Changed

  • Agent: Reconfigured OneFuzz agent to not consume S_LABEL symbols from PDBs #3046
  • Agent: Update elsa::sync::FrozenMap now implements Default #3044
  • Agent: Updated agent to use insta Rust crate for snapshot tests of stacktrace parsing #3027
  • Agent/CLI/Deployment: Store event payloads as blobs. Add API to download event payload given event id. #3069
  • Agent/Service: Bumped Rust version, several Rust dependencies, and several C# dependencies #3049, #3037, #3031, #3023, #2972, #2814, #3052, #3067, #3068, #3056, #2958
  • Service: Made our validation errors more specific so that we can handle them appropriately and reference them in documentation #3053
  • Service/CLI: Updated the Azure DevOps logic to consume the list of existing items once #3014
  • Service: Cap recursion in ORM #2992
  • Service: Collect additional report field in an ExtensionData property #3079

Fixed

  • Agent: Parse .NET exception stack traces when we see them in crash log outputs #2988
  • Agent: Tweaked some of the parameters for the agent's logging to avoid task logger occasionally skipping messages #3070
  • Agent: Allow libfuzzer verification to retry #3032
  • Agent: Fixed typo in AzCopy parameter name and set default value to true #3085
  • Agent/CLI: Added new endpoint to update the pool authentication in order to fix multiple stop messages from being sent after node shuts down #3059
  • CLI: Changed --check_fuzzer_help to --no_check_fuzzer_help #3063
  • Service: Include exception information when validation fails #3077
  • Service: Added another truncation case for 'Request body too large...' errors #3075
  • Service: Fixed the logic for marking task as failed #3083
  • Service: Fixed error deserializing events from the events container #3089

8.0.0

BREAKING CHANGES

This release removes the parameters --client_id, --override_authority, and override_tenant_domain from the config command.

For those accessing the CLI with a service principal, the parameters can be supplied on the command line for each of the CLI commands.

For example, if deploying a job:

onefuzz --client_id [CLIENT_ID] --client_secret [CLIENT_SECRET] template libfuzzer basic --setup_dir .....

Added

  • Agent: Added validate command to the agent to help validate a fuzzer #2948
  • CLI: Added option to libfuzzer template to specify a known crash container #2950
  • CLI: Added option to libfuzzer template to specify the duration of the tasks independently from the job duration #2997

Changed

  • Agent: Install v17 Visual Studio redistributables #2943
  • Agent/Service: Use minimized stack for crash site if no ASAN logs are available #2962
  • Agent/Service: Unified several Rust crate dependency versions across the platform #3010
  • CLI: Remove additional parameters from the config command and require them on each CLI request if accessing the CLI with a service principal #3000
  • Service: Loosen scriban template validation #2963
  • Service: Updated integration test pool size #2935
  • Service: Pass the task tags to the agent when scheduling jobs #2881

Fixed

  • Agent: Ensure custom target_options are always passed last to the fuzzer #2952
  • Agent: Removed xml-rs dependency #2936
  • Agent: Better logging of failures in the task_logger #2940
  • Agent/Service: Updates to address CVE's #2931, #2957, #2967
  • Deployment/Service: Renamed EventGrid subscription to conform with EventGrid's naming scheme #2960
  • Deployment/Service: Added required KeyVault access policy allowing OneFuzz Function App to use an SSL cert for custom domain endpoints #3004, #3006
  • Documentation: Updated 'Azure Devops Work Item creation' doc to remove an outdated template reference #2956
  • Service: Updated feature configuration package to fix an issue where 2 feature flags were using the same ID #2980
  • Service: Make GetNotification nullable to fix errors looking up non-existent notification IDs #2981
  • Service: UniqueReports should be UniqueInputs in LibFuzzer merge task #2982
  • Service: Fix Notification delete action #2987
  • Service: Added handle for missing unique field key in AdoFields #2986
  • Service: Implemented ITruncatable for JobConfig & EventJobStopped to avoid exceptions for messages being too large for Azure Queue #2993

7.0.0

BREAKING CHANGES

  • This release has fully deprecated jinja templates and will only accept scriban templates.
  • The onefuzz config command has removed the --authority and --tenant_domain parameters. The only required parameter for interactive use is the --endpoint parameters. The other values needed for authentication are now retrieved dynamically.
  • The recording components used in the coverage task have been rewritten for improved source-level reporting. The task-level API has one breaking change: the coverage_filter field has been removed and replaced by the module_allowlist and source_allowlist fields. See here for documentation of the new format.
  • The old dotnet template has been removed and dotnet_dll is now dotnet.

Added

  • Service: Added unmanaged nodes integration tests. #2780
  • CLI: Added notification get command to retrieve specific notification definitions. #2818
  • Agent: Added function allow-list to the coverage example exe. #2830
  • Service: Added feature flag, validation when new notifications are created, and CLI support for migration to scriban. #2816, #2834, #2839
  • Agent: Switch over to new coverage task. #2741
  • Service: Added --notification_config support for dotnet templates. #2842
  • Service: Report extension errors when deploying VM in a scaleset. #2846
  • Service: Semantically validate notification configurations. #2850
  • Agent: Accept optional dir of coverage test inputs. #2853
  • Service/Agent: Added extra container to tasks. #2847
  • Documentation: Document coverage crate and tool. #2904
  • Agent: Add the ability for a task to gracefully shutdown when a task is stopped. #2912

Changed

  • Service: Deprecated the job template feature. #2798
  • Service: Deploy with scriban only, removing jinja. #2809
  • Agent: Defer setting coverage breakpoints. This avoids breaking hot patching routines in the ASan interceptor initializers. #2832
  • Service: Updated remaining jinja docs. #2838
  • Service: Support another exception case when adding AssignedTo to telemetry. #2829
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies.#2849, #2855, #2274, #2544, #2857, #2876
  • Contrib: Updated contribution onefuzz config command lines. #2861
  • Agent: Removed Z3 telemetry. #2860
  • Service: Change the optional parameter names and set an expiration for the cache created on the onefuzz config command. #2835
  • Agent: Removed the function allowlist. #2859
  • Agent: Updated clap to remove suppressions. #2856
  • Agent: Removed unused telemetry data. #2863
  • CLI: Removed old libfuzzer dotnet template. #2875
  • Test: Updated C# functional testing InfoResponse. #2894
  • Service: Updated the truncating logic when getting the error so that we retrieve the last messages. #2896
  • Service: Added additional filter check for reports and regressions. #2911

Fixed

  • Agent: Removed a stray print statement. #2823
  • Deployment: Fixed a bug in registration.py when creating CLI service principals. #2828
  • Example: Fixed coverage example build. #2831
  • Service: Fixed the way we report an error when creating a Scaleset under a missing Pool. #2844
  • Service: Update SharpFuzz to a version that supports .NET7.0, and change .NET installation method. #2878
  • Deployment: Fixed an error where a variable was being referenced before being assigned. #2903
  • Service: Created a wrapper function to handle columns defined as GUID in tables. #2898
  • Service: Pass PreserveExistingOutputs to the task. #2905
  • Service: Fixed notification validation. #2914
  • Service: Fixed the custom script definition that could prevent the creation of the repro VM due to a change in the underlying extension setup processes. #2920
  • Deployment: Fixed --auto_create_cli_app flag bug used during deployment. #2921
  • Agent/Service: Updates to address CVE's. #2933
  • Service: Fixed a condition when generating a task configuration. #2925

6.4.0

Added

  • Deployment/CLI: OneFuzz Config refactored - tenant_id, tenant_domain, multi_tenant_domain, and cli_client_id are now required values in the config.json used during deployment and no longer required when running the config command. #2771, #2811
  • Agent: Fully escape allowlist rules #2784
  • Agent: Apply allowlist to all blocks within a function #2785
  • CLI: Added a cli subcommand onefuzz debug notification template to validate scriban notification templates #2800
  • Service: Added Notification failure webhook to communicate Notification failures #2628

Changed

  • Service: Include AssignedTo when failing to create a work item due to an authentication exception #2770

Fixed

  • Agent: Fixes & improvements to Expand behavior #2789
  • Agent: Triming whitespace in output from monitored process before printing #2782
  • CLI: Fixed default value of analyzer_exe #2797
  • CLI: Fixed missing readonly_inputs parameter in dotnet & dotnet_dll templates #2740
  • Service: Fixed query to get the existing proxy #2791
  • Service: Truncate webhooks message length for azure queue size compatibility #2788

6.3.0

Added

  • Service: Add Optional Analysis Task to Libfuzzer Template #2748
  • Agent: Use elsa for improved interface with debuggable_module::Loader #2703
  • Agent: Add sourceline output and logging to coverage example #2753
  • Agent: Fix Linux detection of shared library mappings #2754
  • Agent: Support AllowList extension #2756
  • Agent: Add stdio dumping to example #2757

Changed

  • Service: Update Azure Cli #2733
  • Service: Truncate Large Webhook Events #2742
  • Service: Wrap fallible ORM functions in try/catch #2745
  • Agent/Supervisor/Proxy: Updated third-party Rust dependencies. #2744

Fixed

  • Agent: Fixed Mulit-Agent Issue - Added machine_id to config_path and failure_path of the Agent #2731
  • Service: Fixed Proxy Table Query #2743
  • Service: Fix Notification Logic and Regression Reporting #2751#2758

6.2.0

Added

  • Agent: Added more into-JSON coverage conversions #2725
  • Agent: Added binary coverage merging measurements #2724
  • Agent: Added deserialization compatibility functions #2719
  • Agent: Added OS-generic CoverageRecord builder to capture output of target child process and allow Loader reuse in coverage recording #2716
  • Agent: Improve source coverage of HTML reports #2700, #2701, #2706
  • Deployment: Added support for custom domain names used as OneFuzz endpoints #2720
  • Service: Added documentation for unmanaged node deployment #2694

Changed

  • Agent: Use a custom Output type when recording coverage #2723
  • Agent: Reduce mutation in the agent state machine #2710
  • Service: Include dotnet version in info response #2693
  • Service: Use feature flags to get the node disposal strategy #2713

Fixed

  • Agent: Escape periods when converting globs #2721
  • Agent: Ignore benign recv hangup in agent timer functions #2722
  • Agent: Fix NullRef exception when getting a scaleset that does not exist #2692
  • Service: Downgrade error on "cannot delete nodes from scaleset" to a warning #2691
  • Service: Fixed build issue related to dotnet version 7.0.101 #2698
  • Service: Adding public identifier to Events to restore missing events #2705

6.1.0

Added

  • Service: Added support for feature flags which allows us to deploy new code in parts and turn it on when it's ready. #2620
  • Service: Added a validation endpoint for the notification template. #2655

Changed

  • Service: Update LLVM from v10 to v12 now that we are supporting Ubuntu 20.04 as our default image. #2617
  • Agent: Remove unused coverage recorder from input-tester. #2681
  • Agent: Rename coverage to coverage-legacy. #2685

Fixed

  • CLI: Return an error when uppercase application names are specified when using deploy.py. #2665
  • Agent: Fix local fuzzing mode. #2669
  • Service: Post the JobCreated event when a job is created. #2677
  • Service: The repro Create command will now fail if insert fails. Also add additional tests. #2678
  • Service: Added support for Contains Words in WIQL #2686

6.0.0

BREAKING CHANGES

Manual Deployment Step

When upgrading from version 5.20 a manual step is required. Before deploying 6.0 delete both Azure App Functions and the Azure App Service plan before upgrading. This is required because we have migrated the service from python to C#.

After deployment, there will be two App Functions deployed, one with the name of the deployment and a second one with the same name and a -net suffix. This is a temporary situation and the -net app function will be removed in a following release.

If you have not used the deployment parameters to deploy C# functions in 5.20, you can manually delete the -net app function immediately. Deploying the C# functions was not a default action in 5.20, for most deployments deleting the -net app function immediately is ok.

Deprecation of jinja templates

With this release we are moving from jinja templates to scriban templates. See the documentation for scriban here.

Version 6.0 will convert jinja templates on-the-fly for a short period of time. We do not guarantee that this will be successful for all jinja template options. These on-the-fly conversions are not persisted in the notifications table in this release. They will be in a following release. This will allow time for conversions of templates that are not handled by the current automatic conversion process.

CLI

The default value for the --container_type parameter to the container command has been removed. The container_type parameter is still required for the command. This change removes the ambiguity of the container information being returned.

Added

  • Agent: Added machine_id a parameter of the agent config. #2649
  • Agent: Pass the machine_id from the Agent to the Task. #2662

Changed

  • Service: Deployment enables refactored C# App Function. #2650
  • CLI: Attempt to use broker or browser login instead of device flow for authentication. Canceling the attempt with Ctrl-C will fall back to using the device flow. #2612
  • Service: Update to .NET 7. #2615
  • Service: Make Proxy TelemetryKey optional. #2619
  • Service: Update OMI to 1.6.10.2 on Ubuntu VMs. #2629
  • CLI: Make the --container_type parameter required when using the containers command. #2631
  • Service: Improve logging around notification failures. #2653
  • Service: Standardize HTTP Error Results. Better Rejection Message When Parsing Validated Strings. #2663
  • CLI: Retry on Connection Errors when acquiring auth token. #2668

Fixed

  • Service: Notification Template targetUrl parameter fix. Only use the filename instead of the absolute path in the URL. The makes the links created in ADO bugs work as expected. #2625
  • CLI: Fixed SignalR client code not reading responses correctly. #2626
  • Service: Fix a logic bug in the notification hook. #2627
  • Service: Bug fixes related to the unmanaged nodes (an unreleased feature). #2632
  • Service: Fix invocation of functionapp in the deployment script. Where the wrong value/parameter pair were used. #2645
  • Service: Fixing .NET crash report no-repro. #2642
  • Service: Check Extensions Status Before Transitioning to running state during VM setup. #2667

5.20.0

Added

  • Service: Added endpoint to download agent binaries to support the unmanaged node scenario. #2600
  • Service: Added additional error handling when updating VMSS nodes. #2607

Changed

  • Service: Added additional logging when using the decommission node policy. #2605

  • Agent/Supervisor/Proxy: Updated third-party Rust dependencies. #2608

  • Service: Added optional retry_limit when connecting to the repro machine. #2609

Fixed

  • Service: Fixed status top in C# implementation. #2604
  • Service: Only add "re-opened" comments to a bug if it was actually reopened. #2623

5.19.0

Changed

  • Service: Delete nodes once they're done with tasks instead of releasing scale-in protection. #2586
  • Service: Switch to using the package provided by Azure Functions to set up Application Insights and improve its reporting of OneFuzz transactions. #2597

Fixed

  • Service: Fix handling duplicate containers across accounts in C# functions. #2596
  • Service: Fix the notification GET request on C# endpoints. #2591

5.18.0

Added

  • Service: Use records to unpack the request parameters in AgentRegistration. #2570
  • Service: Convert ADO traces to customEvents and update notificationInfo. #2508
  • Agent: Include computer name in AgentRegistration & decode Instance ID from it. This will reduce the amount of calls to Azure minimizing throttling errors. #2557

Changed

  • Service: Improve webhook logging and accept more HTTP success codes. #2568
  • Service: Reduce fetches to VMSS #2577
  • CLI: Use the virtual env folder to store the config if it exists. #2561, #2567, #2583

Fixed

  • Service: Reduce number of ARM calls in ListVmss reducing calls to Azure to prevent throttling. #2539
  • Service: ETag updated in Update and Replace. #2562
  • Service: Don't log an error if we delete a Repro and it is already missing. #2563

5.17.0

Added

  • Service: Added exponential backoff for failed notifications. Many of the failures are a result of ADO throttling. #2555
  • Service: Add a DeleteAll operation to ORM that speeds up the deletion of multiple entities. #2519

Changed

  • Documentation: Remove suggestion to reset IterationPath upon duplicate. #2533
  • Service: Ignoring the scanning log file when reporting an issue with azcopy. #2536

Fixed

  • CLI: Fixed failures in command $ onefuzz status pool <pool_name>. #2551
  • Deployment: Fix the OneFuzz web address that is used to generate the input_url for bug reporting. #2543
  • Service: Produce an error if coverage recording failed due to a timeout. #2529
  • Service: Increased the default timeout for coverage recording from 5 seconds to 120 to prevent premature errors while parsing symbols and executables. #2556
  • Service: Fixed errors in ADO notifications to reduce duplicate bug-filing. #2534
  • Service: Handle null values better in ScalesetOperations and VmssOperations when a scaleset is in shutdown state. #2538
  • Service: Fix exception message formatting in VmssOperations. #2546
  • Service: Downgrade instance not found exception. #2549
  • Service: Lower log level on symbol region overlap findings during coverage recording. #2559

5.16.0

Added

  • Documentation: Added OneFuzz logo to the README file. #2340
  • Agent: Added try_insert function when building code coverage maps. #2510

Changed

  • Documentation: Described the importance of using the right runtime identifier (RID) when building .NET binaries. #2490
  • Service: Downgraded logging statement from error to warn and also included the http result code. #2484
  • Service/CLI: Updated python dependencies. #2470
  • Service: Updated the verbosity of azcopy logging to assist in debugging copy failures. #2598
  • Agent/Supervisor/Proxy: Updated third-party Rust dependencies.#2500
  • Service: Update the logic for checking if a blob exists before uploading to reduce contention during uploads. #2503
  • Service: Changed the way we update the scaleInProtection on a scaleset node to minimize throttling. #2505

Fixed

  • Service: Only fetch InstanceView data when required. This will reduce throttling by Azure.#2506
  • Service: Fixed github notification queries in the C# implementation (currently not turned on). #2513, #2514

5.15.1

Added

  • Service: Added support for Jinja template migration to Scriban. #2486

Fixed

  • Service: Replaced missing tab that caused ADO queries to fail to find existing work items resulting in duplicate items being created. #2492
  • Tests: Fixed integration-tests-linux. #2487

5.15.0

Added

  • Service: Use InterpolatedStringHandler to move values to CustomDimensions Tags #2450
  • Service: C# Can create ADO notifications #2456, #2458
  • Service: C# Cache VMSS VM InstanceID lookups #2464
  • CLI: Retry on connection reset #2468
  • Agent: Enable backtraces for agent errors #2437

Changed

  • Service: Bump Dependencies #2446
  • Service: Temporarily disable Pool validation #2459

Fixed

  • Service: Fix logic to retrieve partitionKey and rowKey #2447
  • Service: Permit periods in Pool names #2452
  • Service: Node state getting reset to init #2454
  • Service: Fix null ref exception in C# logging #2460
  • Service: Correct pool transitions #2462
  • Service: Fix UpdateConfigs #2463
  • Service: Allow worker loops to continue after errors #2469
  • Service: Lowercase webhooks digest header value #2471
  • Service: Fix C# Node state machine. #2476
  • Service: Adding missing caching from python code #2467

5.14.0

Added

  • Service: Implement not-implemented GetInputContainerQueues #2380
  • Service: Adding new default image config value to instance config 2434

Changed

  • Service: Port SyncAutoscaleSettings from Python to C# #2407

Fixed

  • Deployment: Updating error and fixing default value for auto_create_cli_app #2378
  • Service: Do not discard proxy objects when setting state #2441
  • Service: Do not fail task on notification failure #2435
  • Service: Cleanup queues for non-existent pools and non-existent tasks #2433
  • Service: Delete pool queue when pool is deleted #2431
  • Service: Minor fixes to service logging and error handling #2420
  • Service: Fixed linux repro extensions #2415
  • Service: Mark tasks as failed if a work unit cannot be created for the task #2409
  • Service: Fixed several bugs in C# ports for TimerProxy, TimerRetention, AgentEvents, Node, Tasks and Jobs #2406, #2392, #2379
  • Service: Fixed Azure linux instance proxy extensions provisioning failures #2401
  • Service: Fixed C# scheduling bugs #2390
  • Service: Fixed MarkDependantsFailed error checking #2389
  • Service: Fixed SearchStates querying in TaskOperations #2383
  • Service: Fixed Scaleset response Auth inclusion #2382
  • Service: Fix custom type interpolation in queries #2376
  • Service: Fixed error in C# port for DoNotRunExtensionsOnOverprovisionedVms must be false if Overprovision is false #2375

5.13.0

Added

  • Deployment: Added optional flags --onefuzz_app_id & --auto_create_cli_app for deploy.py to allow for custom app registrations. #2305
  • Deployment: Added optional flag --host_dotnet_on_windows for deploy.py that enables running dotnet functions on Windows based hosts to allow for attaching a remote debugger #2344
  • Deployment: Added optional flag --enable_profiler for deploy.py to enable memory and cpu profilers in dotnet Azure functions #2345
  • Service: Enabled AppInsights dependency tracking to enable better analysis of Azure Storage usage on OneFuzz deployments#2315
  • Service: Added Scriban templating library as a dependency stand-in for Jinja on C# ported services #2330
  • Service: Use 64-bit worker for dotnet functions #2349
  • Agent: Add Cobertura XML output to src-cov example binary #2334
  • CLI: Add onefuzz debug task download_files <task_id> <output> command to download a task’s containers #2359

Changed

  • Service: Removed some response-only properties from the Task model #2335
  • Service: Create storage tables on startup #2309
  • Service: Switched to using Graph SDK instead of manually constructing queries #2324
  • Service: Cache InstanceConfig for improved read performance #2329
  • Deployment: Updated deploy.py to set all function settings at once for faster deployment and upgrades #2325
  • Devcontainer: Move global tool installs into another script so they can be cached #2365
  • Bumped several dependencies in multiple files #2321, #2322, #2360, #2361, #2364, #2355

Fixed

  • Service: Fix az_copy syncing issues by removing the max_elapsed_time limit and relying on RETRY_COUNT instead #2332
  • Service: Implement not implemented bits in Scaleset/VMSS Operations for RemiageNodes & DeleteNodes#2341
  • Service: Fixed bugs in C# port of Proxy and TimerProxy functions #2317, #2333
  • Service: Enforce that there are no extra properties in request JSON, and that non-null properties are [Required] #2328
  • Service: Remove IDisposable from Creds #2327
  • Service: Removed required field in Requests to match python behavior #2367
  • Service: Fixed bug in Azure DevOps notification information #2368
  • Service: Fixed memory leaks in AgentEvents and several supporting libraries #2356
  • Service: Fixed inconsistencies in VMSS creation between C#/Python functions #2358
  • Service: Fixed dotnet Info function to correctly get version number from assembly attributes instead of the config #2316
  • Service: Fixed bug in python types #2319
  • Service: Fixed bugs in timer_workers to allow it to run properly #2343
  • CLI: Coverage task should have access to readonly_inputs containers #2352
  • Devcontainer: Ensure that python virtual environment is installed #2372

5.12.0

Added

  • Deployment: Add --use_dotnet_agent_functions to deploy.py. #2292
  • Service: Added Logging to az_copy calls for improved failure tracking. #2303
  • Service: Added Logging when sending ADO Notifications #2291
  • Service: Additional C# migration work. #2183, #2296, #2286, #2282, #2289

Changed

  • CLI: Changed the CLI's scaleset commands size positional parameter to max_size to better communicate its use in auto scaling properties. #2293

Fixed

  • Deployment: Fixed set_admins.py script. #2300
  • Service: Include serialization options when sending event message. #2290

5.11.0

Added

Changed

  • Agent: Increase the size of the output buffer when collecting logs from agent. #2166
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #2270

Fixed

  • Service: Use a singleton for logging to reduce memory use. #2247
  • Service: Use a singleton for the EntityConverter. #2267
  • Service: Add retries when creating a connection to a repro machine for debugging on windows. #2252
  • Deployment: Fix deploy to use the correct date formats when querying MSGraph. #2258
  • Service: Sync the Autoscale table to current scaleset settings. #2255
  • Service: Fixed the pool comparison in the scheduler. #2260
  • CLI: Fixed the way job and task state enumerations are compared. #2004
  • Service: Clone the JsonSerializerOptions instead of just modifying it. #2280
  • Service: Fixed a NullReferenceException in CreateQueue. #2283

5.10.0

Added

  • Recommendation in getting-started.md that OneFuzz users include a .onefuzz file in their project root directory for security tool detection #2236
  • Agent: New libfuzzer_dotnet_fuzz task #2221

Changed

  • CLI: Updated default Windows VM host image. #2226
  • Agent: Modified LibFuzzer struct to own its environment and option data #2219
  • Agent: Factor out generic LibFuzzer task #2214
  • Service: Enable C# migrated TimerRetention, TimerDaily, and containers functions #2228, #2220, #2197
  • Service: Finished migrating TimerRepro to C# #2222, #2216, #2218
  • Service: Change instances of NotImplementedException to more accurately be NotSupportedException exceptions #2234
  • Service: Migrated Tasks, Notifications, add_node_ssh_key, and Proxy functions to C# #2233, #2188, #2193, #2206, #2200

Fixed

  • Service: Update the autoscale settings to allow a VM scaleset to scale down to zero nodes and prevent new nodes from spinning up when in the shutdown state. #2232, #2248
  • Service: Add a missing function call to properly queue webhook events in WebhookOperations #2231
  • Service: Add a missing job state transition to the Task implementation. #2202
  • Service: Fixed the return value in the C# implementation when associating a subnet with the NSG. #2201
  • Service: Changed log level from Error to Info in TimerProxy. #2185
  • Service: Fixed TimerTasks config bugs in the C# port. #2196

5.9.0

Added

  • Agent: Depend on SharpFuzz 2.0.0 package in the LibFuzzerDotnetLoader project. #2149
  • Test: Added GoodBad C# example project to use with the LibFuzzerDotnetLoader integration tests. #2148

Changed

  • Service: Implemented the containers function in C#. #2078
  • Service/Build: Reuse agent build artifacts if nothing in the agent source tree has changed. This is to speed up dev builds and will not impact official releases. #2115
  • CLI: Default autoscale minimum value to 0. This allows a scaleset to scale-in until there are zero nodes running when no work is pending in the queue. This is important to ensure VM's have the latest patches when running. #2112, #2162
  • Service: Initial work to migrate TimerRepro function to C#. #2168
  • Service/CLI: Remove support for pre 3.0.0 style authentication. #2173
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #2164, #2056, #2175, #2127
  • Service: Updated C# dependencies. #2181
  • Service: When getting information about a task, ignore the task's state if the job_id is specified. #2171

Fixed

  • Agent: Drop the global event sender when closing the telemetry channel to ensure all events are flushed. #2125

5.8.0

Added

  • Service: Add correct routes and auth to agent C# functions. #2109
  • Service: Port agent_registration to C#. #2107
  • Agent: Add the dotnet_coverage task. #2062
  • Agent: Add multiple ways to specify LibFuzzerDotnetLoader targets. #2136
  • Agent: Add logging to LibFuzzerDotnetLoader. #2141
  • Documentation: Added documentation for LibFuzzerDotnetLoader. #2142

Changed

  • Service: Add caching to C# storage implementation so repeated queries do not get throttled. #2102
  • Service: Remove unused poolname validation. #2094
  • Service: Make the hostbuilder async in C#. #2122
  • Service: Updated the scaling policy for the App Functions. #2140
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #2022
  • Agent: Remove incomplete macOS support. #2134, #2135

Fixed

  • Service: Include State filter when searching for expired tasks and jobs. #2138
  • Service: Fix reported TLS errors. #2087
  • Service: Change the upload_file method to use the Azcopy command by default for robustness and fall back to the Azure Python SDK implementation if needed. This also addresses issues where low bandwidth connections timeout due to not being able to handle multiple concurrent upload streams. #1556
  • Service: Update the log SAS URL to last as long as the job duration. #2116, #2121
  • Service: Fixed a number of issues in the C# implementation of TimerProxy. #2133
  • Agent: Fix a race condition when monitoring files on the VM. #2105

5.7.1

This a hotpatch to the 5.7.0 release fixing SAS URL generation which had the potential to cause tasks to fail. #2116

5.7.0

Added

  • Agent: Add NodeState to Node Heartbeat to better track the current state of nodes in the system #2024, #2053
  • Service: Ported existing Python functions to C# #2061, #2072, #2076, #2066
  • Service: Enabling ported C# functions for QueueNodeHeartbeat, QueueTaskHeartbeat, and QueueSignalREvents #2046, #2047
  • Service: Add null analysis attributes to service result types to make it easier to check and use the various existing result types #2069
  • Service: Add dotnet editorconfig underscores naming rule for private fields to start with an underscore, ensuring OmniSharp will generate conformant names by default #2070

Changed

  • Agent: Update onefuzz-agent clap to version 3.2.4 #2049
  • Agent: Added scripts to install dotnet on windows and ubuntu fuzzing VMs #2038
  • Deployment: Update Getting Started instructions for deploy.py's file permissions. #2030

Fixed

  • CLI: Error output to specify that the tools are missing locally, not on the repro VM #2036
  • Service: Handle service event messages that are too big to fit in a queue message. #2020
  • Service: Removing unnecessary /obj/ directory. #2063

5.6.0

Added

  • Service: Add Function App settings to Bicep template and deploy.py. #1973
  • Agent: Add a timestamp to agent log to make it easier to correlate events. #1972

Changed

  • Agent/Supervisor/Proxy: Rename the supervisor process from onefuzz-supervisor to onefuzz-agent. #1989
  • Agent/Supervisor/Proxy: Rename the task executor that runs on the VM from onefuzz-agent to onefuzz-task. #1980
  • Agent/Supervisor/Proxy: Ensure GlobalFlag registry value is initialized for targets. #1960
  • Agent/Supervisor/Proxy: Enable full backtraces on Rust panics. #1959
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #2003, #2002, #1999, #1992, #1986, #1983, #1982, #1981, #1985, #1974, #1969, #1965
  • CLI/Service: Updated multiple first-party and third-party Python dependencies. #2009, #1996

Fixed

  • Agent: Remove stray print statement from the task_logger. #1975
  • Agent: Fix local coverage definition by removing a duplicated command line parameter. #1962
  • Service: Fix Instance Config and Management Logic. #2016

5.5.0

Added

  • Service: Added new functionality to the service port from Python to C#. #1924, #1938, #1946, #1934

Changed

  • Documentation: Update coverage filtering docs. #1950
  • Agent: Allow the agent to skip reporting directories. #1931
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1935
  • CLI: Deprecate libfuzzer_coverage task. #1944
  • CLI: Use consistent logger names. #1945
  • Service: Updated functionality to the service port from Python to C#. #1922, #1925, #1947

Fixed

  • Agent: Allow old reports to be parsed. #1943
  • Agent: Remove transitive OpenSSL dependency. #1952
  • Agent: Ensure GlobalFlag registry value is initialized when checking library dependencies. #1960
  • Service: Allow old reports to be parsed. #1940

5.4.1

This a hotpatch to the 5.4.0 release fixing the parsing failures from old crash reports.

Fixed

  • Agent: Allow old reports to be parsed #1943
  • Agent: Include LD_LIBRARY_PATH in shared library dependency check if and only if set by command. #1933
  • Service: Allow old reports to be parsed #1940

5.4.0

Added

Changed

  • CLI/Service: Updated multiple first-party and third-party Python dependencies. #1784
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1619, #1644, #1645, #1646, #1655, #1700, #1849, #1882
  • Agent: Separate llvm-symbolizer setup from sanitizer environment variable initialization. #1778
  • Agent: Set the TSan options based on the external symbolizer. #1787
  • CLI: Added the ONEFUZZ_CLIENT_SECRET environment variable and removed the client_secret field from the configuration file. This prevents accidental misuse via persisting the secret to disk outside of confidential client environments. If you have set a client secret in your configuration file in a public client, we recommending removing and revoking it. CI scripts that currently set the client secret in the config must instead pass it via the ONEFUZZ_CLIENT_SECRET environment variable or on each CLI invocation via the --client_secret argument. #1918
  • CLI: Use a SAS URL to download log files. #1920

Fixed

  • Agent: Only watch directories for change events. #1859
  • Agent: Switch to a smart constructor to minimize misuse. #1865
  • Service: Fixed an issue where jobs that do not have logs configured failed to get scheduled. #1893

5.3.0

Added

  • Agent: Add a compiler flag to generate debug info for the windows-libfuzzer load library test target. #1684
  • Agent: Add a Rust crate to debug missing dynamic library errors on Windows. #1713
  • Agent: Add support for detecting missing dynamic libraries on Linux. #1718
  • Service: Connect the auto scaling diagnostics to the log analytics workspace. #1708
  • Service: Handle the situation where a VM scale set instance is destroyed before we have removed scale-in protection. #1719
  • Service: Add additional support for auto scaling including changes to the CLI. New scale sets will automatically be created with auto scaling enabled. #1717, #1763
  • Agent/Service/CLI: Add support for generating log files that can be downloaded using the CLI. #1727, #1723, #1721
  • Service: Port ARM templates to Bicep. #1724, #1732
  • Service: Initial changes to port the service from Python to C#. #1734, #1733, #1736, #1737, #1738, #1742, #1744, #1749, #1750, #1753, #1755, #1760, #1761, #1762, #1765, #1757, #1780, #1782, #1783, #1777, #1791, #1801, #1805, #1804, #1803
  • Service: Make sure the scale set nodes are unable to accept work while in the setup state. #1731

Changed

  • Agent: Reduce the logging level down from warn to debug when we are unable to parse an ASan log. #1705
  • Service: Move the creation of the event grid topic to the deployment template from the deploy.py script. #1591
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1548, #1617, #1618
  • Service: Consolidate the two log analytics down to one. #1679
  • Service: Updated resource name in Bicep file to prevent name clash when deploying 5.3.0. #1808

Fixed

  • Service: Auto scale setting log statement is not an error changed it to info. #1745
  • Agent: Fixed Cobertera output so that coverage summary renders in Azure Devops correctly. #1728
  • Agent: Continue after non-fatal errors during static recovery of SanCov coverage sites. #1796
  • Service: Fixed name generation for a few resources in the Bicep file to increase uniqueness which prevents resource name clash. #1800

5.2.0

Added

  • Service: Added additional auto-scaling support for VM scale sets. #1686, #1698

Changed

  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1699, #1589

5.1.0

Added

  • Service: Added a new webhook message format compatible with Azure Event Grid. #1640
  • Service: Added initial auto scaling support for VM scale sets. #1647, #1661
  • Agent: Add an explicit timeout to setup scripts so hangs are easier to debug. #1659

Changed

  • CLI/Service: Updated multiple first-party and third-party Python dependencies. #1606, #1634
  • Agent: Check system-wide memory usage and fail tasks that are nearly out of memory. #1657

Fixed

  • Service: Fix task field to the correct NodeTasks type so serialization works correctly. #1627
  • Agent: Convert escaped characters when accessing the name of a blob in a URL. #1673
  • Agent: Override runs parameter when testing inputs as we only want to test them once. #1651
  • Service: Remove deprecated warn() method. #1641

5.0.0

Added

  • CLI/Service: Added fuzzer_target_options argument to the libfuzzer templates to allow passing some target options only in persistent fuzzing mode #1610

Changed

  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1530
  • CLI/Service: Updated multiple first-party and third-party Python dependencies. #1576 #1577 #1579 #1582 #1586 #1599
  • CLI/Service: Begin update of scale set instances before reimaging to ensure they match the latest scale set model. #1612

Fixed

  • Agent: Removed the process_stats telemetry event, which fixes a class of memory leaks on Windows libfuzzer_fuzz tasks. #1608
  • CLI/Service: Fixed seven day stale node reimaging check. #1616

4.1.0

Added

  • Agent: Added source line coverage data #1518 #1534 #1538 #1535 #1572
  • Agent: Added Cobertura XML output for source code visualization #1533
  • Service: Added auto configuration properties to the monitoring agents #1541
  • Service: Added tags to scalesets and VMs #1560

Changed

Fixed

  • Service: Increase reliability of integration tests. #1505
  • Agent: Avoid leaking unused file and cache data #1539
  • Agent: Fixed new clippy errors #1516

4.0.0

Added

  • Agent: Added common source coverage format. #1403
  • Service: Added class to store and retrieve rules associated with an API endpoint. This supports the ability to control who has access to an API. #1420
  • Service: Support for NSG creation during deployment, allowing restricted access to the scaleset and repro VMs. #1331, #1340, #1358, #1385, #1393, #1395, #1400, #1404, #1406, #1410
  • Service: Guest account access is disabled by default when creating the default service principal during deployment. #1425
  • Service: Group membership check added. #1074
  • Service: Exposed the target_timeout parameter in the radamsa basic template. #1499

Changed

Fixed

  • Service: Fixed Azure DevOps work item creation by adding missing client initialization. #1370
  • Service: Fixed validation of the target_exe blob name, enabling nesting in a subdirectory of the setup container. #1371
  • Service: Migrated to MS Graph, as azure-graphrbac is soon to be deprecated. #966
  • Service: Stopped ignoring unexpected errors when authenticating the client secret. #1376
  • Service: Fixed regex to correctly capture the object ID when trying to remove an invalid application ID. #1408
  • Service: Added check for service principal use during user role assignment. #1479
  • Service: Added support for Compute Gallery images. #1450

3.2.0

Changed

Fixed

  • Service: Fixed authentication when using a client secret. #1300
  • Deployment: Fixed an issue where the wrong AppRole was assigned when creating new CLI registrations. #1308
  • Deployment: Suppress a dependency's noisy logging of handled errors when deploying. #1304

3.1.0

Added

  • Agent: Added ability to handle fake crash reports generated by debugging tools during regression tasks. #1233
  • Service: Added ability to configure virtual network IP ranges. #1268
  • Deployment: Added flake8 to the deployment process to align with rest of the Python codebase linting. #1286
  • Service: Added custom extensions to enable Microsoft Security Monitoring extensions. #1184
  • CLI: Added --readonly_inputs option to the libfuzzer basic template. #1247

Changed

Fixed

  • Deployment: Fixed deployment in some regions by specifying widely-supported versions of Application Insights resources. #1291
  • Deployment: Fixed an issue with multi-tenant deployment caused by a mismatch between the identifier used to configure the app registration and value used to authenticate the CLI client. #1270
  • Service: Fixed scaleset proxy reset to reset all proxies in specified region. #1275
  • CLI: Temporarily ignore type errors from azure-storage-blob due to invalid Python type signatures. #1258

3.0.0

Changed

  • CLI/Deployment/Service: Move to using api:// for AAD Application "identifier URIs". Pre-3.0 clients will not be able to connect to newer instances. (BREAKING CHANGE) #1243
  • Agent/Supervisor/Proxy: Redact device, IP, and machine name in runtime statistics reported to Microsoft via Application Insights. #1242
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1232, #1230, #1228, #1229, #1231, #1242.

2.33.1

Fixed

  • CLI: Fixed an issue printing results that include SecretData. #1223

2.33.0

Added

Changed

Fixed

  • Deployment: Fixed the example deployment rule to include the required Azure Storage Queue support. #1207
  • CLI: Fixed an issue printing results that include set, datetime, or None. #1208, #1221

2.32.0

Added

  • CLI/Service: The Azure VM SKU used for proxies is now configurable via onefuzz instance_config. #1128
  • CLI: Added onefuzz status pool command to give status information for a pool. #1170

Changed

  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1152, #1155, #1156, #1157, #1158, #1163, #1164, #1165, #1166, #1176, #1177, #1178, #1179, #1181, #1182, #1183, #1185, #1186, #1191, #1198, #1199, #1200, #1201, #1202, #1203, #1204, #1205
  • Agent: Changed azcopy calls to always retry when source files are modified mid-copy. #1196
  • Agent: Continued development related to upcoming features. #1146
  • Agent: SAS URLs are now redacted in logged azcopy failures. #1194
  • CLI: Include the number of VMs used per-task in onefuzz status top. #1169
  • Deployment: Application credentials created during deployment are no longer logged. #1172
  • Deployment: Clarify logging when retrying AAD interactions. #1173
  • Deployment: Replaced custom Azure Storage Queue creation with ARM templates. #1193
  • Service: The validity period for SAS URLs is now back-dated to avoid time synchronization issues. #1195

Fixed

  • Deployment: Invalid preauthorized application references are removed during application registration. #1175
  • Service: Fixed an issue logging node status. #1160

2.31.0

Added

  • Supervisor: Added recording of STDOUT and STDERR of the supervisor to file. #1109
  • CLI/Service/Agent: Supervisor tasks can now optionally have a managed coverage container. #1123

Changed

  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1151, #1149, #1145, #1134, #1135, #1137, #1133, #1138, #1132, #1140
  • Service: Enabled testing of the Azure Devops work item rendering. #1144
  • Agent: Continued development related to upcoming features. #1142
  • CLI: No longer retry service API requests that fail with service-level errors. #1129
  • Agent/Supervisor/Proxy: Addressed multiple new cargo-clippy warnings. #1125
  • CLI/Service: Updated third-party Python dependencies. #1124

Fixed

  • Service: Fixed an issue with incomplete authorization in multi-tenant deployments. CVE-2021-37705 #1153

2.30.0

Changed

  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1116

Fixed

  • Service: Fixed an error when replacing notifications for a container. #1115
  • Service: Fixed Python 3.9 compatibility issues. #1117
  • Agent/Supervisor/Proxy: Addressed multiple new cargo-clippy warnings. #1118

2.29.1

Fixed

  • Agent: Fixed an issue with the "Premium" storage account utilities. #1111
  • Agent: Addressed a rate-limiting issue when using azcopy from a large number of VMs with numbers cores. #1112

2.29.0

Added

  • Service: PII is now removed from Jobs, Tasks, and Repros after 18 months. #1051
  • Service: Unused notifications are now removed after 18 months. #1051

Changed

  • Service: SignalR events are routed through an Azure Storage Queue to prevent SignalR outages from impacting the entire service. #1100, #1102
  • Service: Functionality used prior to 1.0.0 for assigning tasks to VMs rather than Pools is no longer supported. #1105
  • Service: The coverage and generic_generator tasks now verify {input} is used in target_env or target_options. #1106

Fixed

  • Service: Fixed an issue reimaging old nodes with debug_keep_node set. #1103
  • Service: Fixed an issue authenticating to Azure services. #1099
  • Service: Fixed an issue preventing Pools and Scalesets set to shutdown from being set to halt. #1104

2.28.0

Added

  • CLI: Added the ability to remove existing container notifications upon creating a notification integration. #1084
  • CLI/Documentation: Added an example generic_analysis task that demonstrates collecting LLVM source-based coverage. #1072
  • Supervisor: Added service-interaction resiliency for node commands. #1098

Changed

  • Agent/Supervisor/Proxy: Addressed multiple new cargo-clippy warnings. #1089
  • Agent: Added more context to errors in generator tasks. #1094
  • Agent: Added support for ASAN runtime identification of format string bugs. #1093
  • Agent: Added verification that {input} is provided to the application under test via target_env or target_options. #1097
  • Agent: Continued development related to upcoming features. #1090, #1091
  • CLI/Service: Updated multiple first-party and third-party Python dependencies. #1086
  • CLI: Changed job templates to replace existing notifications for the unique report container. #1084
  • Service: Added more context to Azure DevOps errors. #1082
  • Service: Notification secrets are now deleted from Azure KeyVault upon notification deletion. #1085

Fixed

  • Agent: Fixed an issue logging ASAN output upon ASAN log parse errors. #1092
  • Agent: Fixed issues handling non-UTF8 output from applications under test. #1088

2.27.0

Changed

  • Agent: Batch processing results are now saved after every 10 executions. #1076
  • Service: Optimized file_added event queueing by avoiding unnecessary Azure queries. #1075
  • Agent: Optimized directory change monitoring. #1078
  • Supervisor: Optimized agent monitoring. #1080

2.26.1

Fixed

  • CLI: Fixed an issue handling long-running requests. #1068
  • CLI/Service: Fixed an issue related to upcoming features. #1067
  • CLI: Fixed an issue handling target_options for libFuzzer jobs. #1066

2.26.0

Added

  • Supervisor: Added a panic handler to record supervisor failures. #1062

Changed

  • Agent: Added more context to file upload errors. #1063
  • CLI: Made errors locating azcopy more clear. #1061

Fixed

  • Service: Fixed an issue where long-lived VM scaleset instances could get reimaged with out-of-date VM setup scripts. #1060
  • Service: Fixed an issue where VM setup script updates were not always pushed. #1059

2.25.1

Fixed

  • Service: Fixed an issue detecting and reimaging failed nodes. #1054
  • Service: Fixed an issue with the supervisor restarting too quickly. #1055

2.25.0

Added

  • Agent: Added minimized_stack_function_lines and minimized_stack_function_lines_sha256 to crash reports. #993
  • CLI/Service: Added timestamp to Notification objects. #1043
  • Service: Added the scaleset_resize_scheduled event. #1047
  • Service: Added pool_id to Node objects. #1049

Changed

  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1040, #1052
  • CLI/Deployment/Service: Updated multiple first-party and third-party Python dependencies. #922, #1045
  • CLI/Service: Moved to using Pydantic built-in size validation for types. #1048
  • Service: Continued development related to upcoming features. #1046, #1050

Fixed

  • CLI: Fixed an issue handling column sorting in onefuzz status top. #1037
  • Service: Fixed an issue adding SSH keys to Windows VMs. #1038

2.24.0

Added

  • CLI/Service: Added instance configuration that can be managed via onefuzz instance_config. #1010
  • Service: Added automatic retry for Azure Devops notifications. #1026
  • CLI/Service: Added validation to GitHub Issues integration configuration. #1019

Changed

  • Agent/Supervisor/Proxy: Moved to rustls to enable running the Agent and Supervisor on Ubuntu 20.04. #1029
  • Agent: Continued development related to upcoming features. #1016

Fixed

  • Agent: Fixed an issue handling invalid data during coverage collection. #1032
  • Agent: Fixed retry logic on coverage recording failures #1033

2.23.1

Fixed

  • Service: Fixed an issue preventing deletion or reimaging of nodes in some cases. #1023

2.23.0

Changed

  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1018, #1009, #1004
  • Service: Tasks running on nodes without recent heartbeats are now marked as failed due to heartbeat issues. #1015
  • Service: Updated multiple first-party Python dependencies. #1012

Fixed

  • Agent: Fixed an issue where libfuzzer_fuzz tasks on Windows that found crashes too rapidly were unable recover handles. #1002
  • Agent: Fixed an issue with the regression tasks after using the onefuzz debug notification commands. #1011
  • Deployment: Fixed a configuration issue reducing log retention durations. #1007
  • Service: Fixed an issue creating GitHub Issues notifications. #1008
  • Service: Fixed an issue handling reimaging nodes that took an excessive amount of time. #1005

2.22.0

Changed

  • Service: Update node and task-related log messages to ease debugging. #988
  • Agent: Changed the log level for azcopy retry notification to DEBUG. #986
  • Agent: Updated stack minimization regular expressions from libclusterfuzz. #992
  • Agent: Added more context to synchronized directory errors. #995
  • Deployment: Reduced the Application Insights log retention duration to 30 days. #997
  • Agent: Improved tracking of threads during win32 debugging. #1000

Fixed

  • Agent: Fixed an issue using relative paths with synchronized directories. #996
  • Service: Fixed an issue creating GitHub Issues notifications #990
  • CLI/Service: Fixed an issue handling Union fields in the onefuzztypes library #982
  • Service: Fixed an issue handling manually-resized scalesets #984

2.21.0

Added

  • CLI: Added onefuzz debug job rerun command. #960

Changed

  • Agent: Added more context to coverage recording errors. #979
  • Agent: The coverage task now retries an input in the case of coverage recording failure. #978
  • Service: Nodes with the debug_keep_node flag will now be reimaged once the node is 7 days old. #968
  • Service: Updates to scalesets can now be requested while the node is in the resize state. #969

Fixed

  • Service: Fixed an issue when reimaging nodes that previously failed to reimage as expected. #970
  • Service: Fixed an issue when resizing scalesets that exceed Azure VM quotas. #967
  • Supervisor: Fixed an issue with refreshing service authentication tokens. #976

2.20.0

Added

  • Agent: Added a new coverage task that enables coverage analysis for both uninstrumented and Sancov targets on Linux and Windows. #763

Changed

  • Agent: Improved performance of the libFuzzer fuzzing tasks. #941
  • CLI: Changed the libfuzzer basic job template to use the new coverage task. #763
  • Deployment: Added automatic retry when authorizing newly-created applications during deployment. #959
  • Supervisor: Simplified the service coordination logic and added increased context upon failure. #963

2.19.0

Added

  • Agent/Supervisor: Added azcopy log recording upon azcopy failure. #945
  • CLI: Added onefuzz jobs containers delete command. #949
  • CLI: Added onefuzz jobs containers download command. #953

Changed

  • Agent/Service: Agents scheduled to shut down no longer wait for work prior to shutting down. #940
  • Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #942
  • Agent: Continued deveopment related to upcoming features. #937, #929, #919
  • CLI: Message details are now always shown in onefuzz status top. #933
  • CLI: Renamed template helper methods for uploading task setup files. #926
  • Contrib: Updated multiple third-party Python dependencies. #950
  • Service: Tasks that are stopped without ever having started are now marked as failed. #935
  • Supervisor: Added increased context when recording supervisor failures. #931

Fixed

  • CLI/Service: Worked around a third-party dependency issue in handling Python Unions in Events. #939
  • Deployment: Fixed an authentication issue during deployment. #947, #954
  • Deployment: Fixed an issue limiting application creation logs. #952
  • Service: Fixed an issue deleting nodes with expired heartbeats. #930
  • Service: Fixed an issue deleting nonexistent containers. #948
  • Service: Fixed an issue deleting proxies. #932
  • Service: Fixed an issue that prevented automatic migration of notification secrets to Azure KeyVault in some cases. #936
  • Supervisor: Fixed an issue adding multiple SSH keys to Windows VMs. #928

2.18.0

Added

  • Agent: Added setup_dir configuration value expansion for generator tasks. #901
  • CLI: Enable specifying alternate tenant configuration via command line arguments. #900
  • CLI/Service: Proxy status is now available via onefuzz scaleset_proxy list command. #905

Changes

  • Deployment: Moved to using Microsoft Graph User.Read rather than Azure AD Graph. #894
  • Service: Tasks are now stopped on nodes before task related storage queues are deleted. #801
  • Proxy: Proxies are automatically deployed and always available based on regions with active fuzzing scalesets. #839, #908, #907, #909, #904
  • CLI: Added explanations to errors generated when parsing arguments whose values are key/value pairs. #910, #911
  • Agent: Continued development related to upcoming features. #913, #918
  • Service: Updated first-party Python libraries #903

2.17.0

Added

Changes

  • Agent/Supervisor/Proxy: Addressed multiple new cargo-clippy warnings. #884
  • Agent/Supervisor/Proxy: Updated and removed third-party Rust dependencies. #892, #873, #865
  • Service: Improved the Python typing signatures used in the service. #881
  • Service: Updated multiple first-party and third-party Python libraries. #893, #889, #866, #885, #861, #890
  • Supervisor: The supervisor now includes the full error context upon failure. #879
  • Service: Cleaned up scaleset update logs. #880
  • Agent: Continued development related to upcoming features. #874, #868, #864
  • SDK/CLI: Replaced Python based directory uploading with azcopy sync. #878

Fixed

  • Service/Supervisor: Fixed an issue shrinking scalesets where idle nodes would not shut down as expected. #866
  • Deployment: Fixed an issue deploying to non-Microsoft single-tenant instances. #872, #898

2.16.0

Aded

  • Deployment: Added ability to only deploy RBAC rsources. #818
  • Agent: Continued development related to upcoming features. #855, #858

Fixed

  • Agent: Fixed issue where directory monitoring would fail due to azcopy temporary files. #859
  • Service: Fixed issue where scalesets could get stuck trying to resize if also manually deleted. #860

2.15.0

Added

Changes

  • Agent/Proxy/Supervisor : Updated multiple third-party Rust dependencies. #842, #826, #829
  • Service/Contrib: Updated multiple Python dependencies. #828, #827, #823, #822, #821, #847
  • Service: Resetting nodes no longer requires waiting for the node to acknowledge the shutdown in some cases. #834

Fixed

  • Supervisor: Fixed an issue introduced in 2.14.0 that sometimes prevents nodes from stopping processing tasks. #833
  • Service: Fixed an issue related to Azure Storage Queues being deleted while in use. #832
  • Deployment: Fixed an issue where the CLI client application role was not assigned during deployment. #825

2.14.0

Added

  • Contrib: Added a sample GitHub Actions workflow and an Azure DevOps Pipeline to demonstrate deploying OneFuzz jobs using CICD. #778
  • CLI/Service: Added creation timestamps to Job, Node, Pool, Scaleset, Repro, Task, and TaskEvent records returned by the service. #796, #805, #804
  • Agent/Proxy/Supervisor: Added additional context to web request failures to assist in debugging issues. #798
  • Service: Added task configuration to the crash_reported and regression_reported events. #793

Changes

  • Agent: The full error context is now logged upon task failure. #802
  • CLI: The libfuzzer-dotnet template no longer defaults to failing the task if the fuzzer exits with a non-zero status but no crash artifact. #807
  • Agent/Proxy/Supervisor: Updated multiple Rust dependencies. #800
  • Service: When multiple failures are reported for a given task, only the first failure is recorded. #797
  • Agent: Continued development related to upcoming features. #820, #816, #790, #809, #812, #811, #810, #794, #799, #779

Fixed

  • Deployment: Added missing actions to the example Custom Azure Role for deployment. #808
  • Service: Fixed an issue in scaleset creation with incompatible VM SKUs and VM Images. #803
  • Service: Fixed an issue removing user identity information from logging to user instances. #795

2.13.0

Added

  • Deployment: Allow specifying the Azure subscription to use for deployment, instead of always using the default #774

Changed

  • Agent/Supervisor: Added automatic retry when executing azcopy. #701
  • Service: When task setup fails, the error that caused the setup failure is now included in the Task error message. #781
  • Agent: The libfuzzer-fuzz task no longer queries the full local system status when only reporting process status. #784
  • Agent: The libfuzzer-fuzz task now limits the stderr collected to the last 1024 lines for potential failure reporting. #785
  • Agent: The libfuzzer-fuzz task now summarizes the executions per second and iteration counts from all of the workers on each VM. #786
  • Agent: The libfuzzer-coverage task no longer removes the initial copy of inputs. #788
  • Agent: Debugger scripts for extracting libFuzzer coverage are now embedded in the agent. #783
  • Agent: Continued development related to upcoming features. #787, #776, #663

Fixed

  • CLI: Fixed issue relating to line endings in the libfuzzer-qemu job template setup script. #782
  • Service: Fixed backward compatibility issue in ephemeral disk support when creating scalesets. #780
  • Deployment: Fixed issue in multi-tenant deployment support. #773

2.12.0

Added

  • Agent: LibFuzzer tasks now include a verification step that verifies the fuzzer can test a small number of seeds at the start of the task. #752
  • Integration Tests: Added verification that no errors are logged to Application Insights during testing. #700
  • Agent/Supervisor/Service/Deployment: Added support for multi-tenant authentication. #746
  • CLI/Service: Added support for Ephemeral OS Disks. #461, #761

Changed

  • Agent: Continued development related to upcoming features. #765, #762, #754, #756, #750, #744, #753
  • Contrib: Updated multiple python dependencies. #764
  • CLI/Agent: LibFuzzer fuzzing tasks no longer default to failing the task if the fuzzer exits with a non-zero status but no crash artifact. #748

Fixed

  • Agent/Proxy/Supervisor: Fixed issues prevent HTTPS retries. #766
  • Agent/Service/Proxy/Supervisor: Fixed logging and telemetry from the agent. #769

2.11.1

Fixed

  • Agent/Proxy/Supervisor: Fixed issues preventing heartbeats. #749

2.11.0

Changed

  • Agent: Continued log simplification and clarification. #736, #740, #742
  • Agent: Prevent invalid queue messages from being ignored. #731
  • Agent: Separated module and symbol names for Windows debugger-based crash reports. #723
  • Deployment/Agent: Updated AFL++ to 3.11c. #728
  • CLI/Deployment: Updated Python dependencies. #721
  • Agent: Updated stack minimization regular expressions from ClusterFuzz. #722
  • Service: Removed user's identity information from logging to user instances. #724, #725
  • Agent: Continued development related to upcoming features. #699, #729, #733, #735, #738, #739

Fixed

  • Deployment: Worked around a race condition in service principal creation. #716
  • Agent: Dotfiles are now ignored in libFuzzer-related directories. #741

2.10.0

Added

Changed

  • Agent/Proxy/Supervisor: Changed web request retry logic to include the underlying failure upon giving up retrying a request. #696
  • Supervisor: Added automatic web request retry logic when communicating to the service. #704
  • CLI/Service: Updated Python dependencies. #698, #687
  • Supervisor: Clarified log message when the supervisor unexpectedly exits. #685
  • Proxy: Simplified service communication logic. #683
  • Proxy: Increased log verbosity on proxy failure. #702
  • Agent: Increased setup script timestamp resolution. #709
  • Agent: Continued development related to an upcoming feature. #508, #688, #703, #710, #711

Fixed

  • Agent: Fixed support for libFuzzer targets that use shared objects or DLLs from the setup container. #680, #681, #682, #689, #713

2.9.0

Added

  • Contrib: Added sample Webhook Service #666
  • Agent: Add OneFuzz version and Software role to telemetry #586
  • Agent: Add multiple telemetry data types for the upcoming functionality #619
  • Agent: Added input_file_sha256 to configuration value expansion. #641
  • Agent: Added job_id to Task Heartbeat #646
  • Service: Added task information to job_stopped events #648

Changed

  • Service: task_stopped and task_failed now trigger once the task has stopped instead of upon entering the stopping state. #651
  • CLI: Authentication tokens are saved upon successful login rather than on program exit. #665
  • Service: If a task with dependent tasks fails, all of the dependent tasks are marked as failed. #650
  • Agent: Fixed PC address in crash report backtraces. #658
  • Service: Upon task completion, if all of the tasks in the associated job are completed, the job is marked as stopped. #649
  • Deployment/Agent: Updated AFL++ to 3.11c. #675
  • Agent/Proxy/Supervisor: Changed web request retry logic to always retry any request that fails, regardless of why the request failed. #674
  • Agent: Downloading files from task queues will now automatically retry on failure. #676
  • Service: User information is now stripped from Events before being logged to Application Insights. #661

Fixed

  • Service: Handle exception related to manually deleted scalesets #672
  • Agent: Fixed Rust lifetime issues exposed by an update to Rust regex library #671

2.8.0

Added

Changed

  • Agent: Clarified batch-processing logs. #622
  • Agent/Proxy: Updated multiple rust dependencies. #624
  • Service/CLI/Contrib: Updated multiple python dependencies. #607, #608, #610, #611, #612, #625, #626, #630, #640
  • Service: Update task configuration to verify target_exe is a canonicalized relative path. #613
  • Deployment/Agent: Updated AFL++ to 3.10c. #609
  • Deployment: Clarify application password creation succeeded after earlier failures. #629
  • Service: VM passwords are no longer set on Linux VMs. #620
  • Service: Clarify source of task failures when notification integration marks a task as failed. #635

Fixed

  • Agent/Proxy/Supervisor: Fixed web request retry logic when handling operating system level errors. #623
  • Service: Handle exceptions when creating scalesets fail due to Azure VM quota issues. #614

2.7.0

Added

  • CLI: Added onefuzz containers files download_dir to enable downloading the contents of a container. #598
  • Agent: Added microsoft_telemetry_key and instance_telemetry_key and expanded the availability reports_dir in configuration value expansion. #561
  • Agent/Service: Added job_id to agent-based heartbeats. #594
  • Agent/Proxy/Supervisor: Added additional context to errors during Storage Queue and service interactions to improve debugging. #601

Changed

  • Agent/Proxy/Supervisor: Renamed the Application Insights token names used for telemetry to microsoft_telemetry_key and instance_telemetry_key and the function that gated telemetry sharing to can_share_with_microsoft to make the telemetry implementation easier to understand. #587
  • Deployment: Updated multiple Python dependencies. #596
  • Service: Updated multiple Python dependencies. Addresses potential security issue CVE-2020-28493 #595
  • Service: Don't let nodes run new tasks if they are part of a scaleset or pool that is scheduled to be shut down. #583

Fixed

  • Service: Fixed the queries used to identify nodes running outdated OneFuzz releases. #597
  • Agent: Fixed an issue that would stop an agent or supervisor from performing work if an HTTPS request has failed in certain conditions. #603
  • Agent: Fixed an issue that would stop a task if the task printed a significant amount of data to stdout or stderr. #588
  • Deployment: Address deployment failures relating to cross-region Azure Active Directory resource creation delays. #585

2.6.0

Added

  • Service: Jobs that do not start within 30 days are automatically stopped. #565

Changed

  • Service: Debug proxies now use ports 28000 through 32000. #552
  • Service: Events now include the instance name and unique identifier. #577
  • Service: All task related Events now include the task configuration. #580
  • Service: Errors generated during report crash report notification due to invalid jobs or tasks now include the reason for the error. #576
  • CLI: Namespaced containers for coverage used in job templates now include build and platform in addition to project and name. #572
  • Service: User triggered node reimaging no longer waits for confirmation from the node prior to starting the reimage process. #566

Fixed

  • Service: Fixed an error condition when users recreate a container immediately after deleting it. #582
  • Service: Fixed an issue when one task on a node ended, the node was reimaged regardless of the state of other tasks running on the node. #567

2.5.0

Added

  • CLI: Added the ability to poll task status until the tasks have started to managed templates using --wait_for_running. #532
  • CLI: Added a libfuzzer-dotnet support. #535
  • Agent: Added crashes_account and crashes_container to configuration value expansion. #551
  • CLI: Added onefuzz status job and onefuzz status project to provide a user-friendly job status. #550

Changed

  • Agent: Logs and local telemetry from the agent now include the role (agent or supervisor) in recorded events. #527
  • Agent: Clarified the errors generated when libFuzzer coverage extraction fails #554

Fixed

  • Service: Handled SkuNotAvailable errors from Azure when creating scalesets. #557
  • Agent/Proxy: Updated multiple third-party Rust libraries. Addresses potential security issue RUSTSEC-2021-0023. #548

2.4.1

Changed

  • Agent: Verifying LibFuzzer targets at the start of a task using -help=1 now happens prior to sending heartbeats. #528

Fixed

  • Service: Fixed issue related to Azure Functions not always providing the JWT token via Authorization headers. #531
  • CLI: Fixed --wait_for_running in job templates. #530
  • Deployment: Fixed a log error by setting the default SignalR transport used by Azure Functions. #525
  • Agent: Fixed LibFuzzer coverage collection when instrumenting DLLs loaded at runtime. #519
  • Service: Fixed issue where the cached Azure Identity was not being used. #526
  • Service: Fixed log message related to identifying secondary corpus instances. #524

2.4.0

Added

  • Service: Handle scaleset nodes that never register, such as nodes with instance-specific setup script failures. #518

Changed

  • Agent: Added stdout/stderr logging and clarifying context during failures to the generic_analysis task. #522
  • Agent/Service/Proxy: Clarify log messages from the scaleset proxy. #520
  • Agent/Proxy: Update multiple third-party Rust libraries. #517

Fixed

  • Agent: Fixed potential race condition when single stepping when debugging during the generic_crash_reporter and generic_generator tasks running on Windows. #440

2.3.0

Changed

  • Service: Clarify log messages when the service and agent versions mismatch. #510
  • Service: Scalesets and Nodes are now updated in a consistent order during scheduled updates. #512
  • CLI/Service: Expanded the use of Primitive data types that provide data validation. #514

Fixed

  • Service: Fixed an error generated when scalesets scheduled for deletion had configurations updated. #511
  • Service: Fixed an issue where scaleset configurations were updated too frequently. #511

2.2.0

Added

  • Proxy: The logs from the proxy manager logged to Application Insights. #502

Changed

  • Agent: Updated the web request retry logic to retry requests upon connection refused errors. #506
  • Service: Improved the performance of shutting down pools. #503
  • Service: Updated azure-mgmt-compute Python dependency. #499

Fixed

  • Proxy: Fixed an issue in the proxy heartbeats that caused proxy VMs to be reset after 10 minutes. #502
  • Agent: Fixed an issue that broke libFuzzer based crash reporting that was introduced 2.1.1. #505

2.1.1

Added

Fixed

  • Service: Fixed an issue where scalesets could get in a state that would stop updating configurations. #489

2.1.0

Added

Changed

  • CLI/Service/Agent: Supervisor can now be fully self-contained fuzzing tasks, no longer requiring target_exe. Additionally, supervisor tasks can now optionally have managed report containers. #474
  • Service: Managed nodes that are unused beyond 7 days are automatically reimaged to ensure OS patch levels are maintained. #476
  • CLI/Service: Updated the default Windows VM image to MicrosoftWindowsDesktop:Windows-10:20h2-pro:latest. Existing scalesets will not be impacted by this change, only newly created scalesets using the default image. #469

Fixed

  • Agent: New inputs discovered by supervisor tasks are now saved to the inputs container. #484
  • CLI: The license is now properly set in the python package metadata. #472
  • Agent: Failure to download files via HTTP from queues now results in a failure, rather than the HTTP error being interpreted as the requested file. #485
  • Deployment: Fixed error when checking if the default CLI application exists. #488

2.0.0

Added

Changed

  • CLI/Service: Migrated onefuzz status top to use Webhook Events. (BREAKING CHANGE) #394
  • CLI/Service: New notification secrets, such as ADO tokens, are managed in Azure KeyVault and are no longer accessible to the user once created. (BREAKING CHANGE) #326, #389
  • CLI/Service: Updated multiple Python dependencies. #426, #427, #430

Fixed

  • Agent: Fixed triggering condition for new unique report events #422
  • Deployment: Mitigate issues related to deployments within conditional access policy scenarios. #447
  • Agent: Fixed an issue where unused nodes would stop requesting new work. #459
  • Service: Fixed dead node cleanup. #458
  • Service: Fixed an issue logging excessively large stdout/stderr from tasks. #460

1.11.0

Added

  • Service: Added support for sharding corpus storage accounts using "Premium" storage accounts for improved IOPs. #334
  • CLI/Service/Agent: Added the ability to optionally colocate multiple compatible tasks on a single machine. The coverage and crash reporting tasks in the LibFuzzer template make use of this functionality by default. #402
  • CLI: Added onefuzz debug log tail which enables continuously following Application Insights query results. #401
  • CLI/Agent: Support verifying LibFuzzer targets at the start of a task using -help=1, which will enable identifying non-functional LibFuzzer targets. #381
  • CLI/Agent: Support specifying whether to log a warning or fail the task when a LibFuzzer target exits with a non-zero status code (without also generating a crashing input). #381
  • Agent: The stdout and stderr for the supervisors and generators are now logged to Application Insights. #400
  • Service: Enabled per-Scaleset SSH keys on Windows VMs, similar to existing Linux support, enabling onefuzz debug node ssh to both Windows and Linux nodes. #390
  • Agent: Support ASAN odr-violation results. #380
  • CLI/Service/Agent: Added the ability add SSH keys to nodes within scalesets. #441
  • CLI: Added support for multi-tenant authentication. #346

Changed

  • Service: Updating outdated nodes is now limited to 500 nodes at a time. #397
  • Service: Restrict agent from accessing API endpoints not specific to the agent. #404
  • Service: Increased Azure Functions runtime timeout to 15 minutes. #384
  • Deployment/Agent: Updated AFL++ to 3.00c. #393
  • Agent: Added randomized initial jitter to agent heartbeats, which reduce API query storms when launching large number of nodes concurrently. #387

Fixed

  • CLI/Agent: Add support to verify LibFuzzer targets execute correctly at the start of a task using -help=1. #381
  • Service: Re-enable API endpoint used by onefuzz nodes update. #412
  • Agent: Addressed a race condition in LibFuzzer coverage analysis without initial seeds. #403
  • Agent: Prevent supervisor that fatally exits from processing additional new tasks. #378
  • Agent: Address issues handling LibFuzzer targets that produce non-UTF8 output to stderr. #379

1.10.0

Added

  • CLI: Added libfuzzer merge job template, which enables running performing libFuzzer input minimization as a batch operation. #282
  • CLI/Service: Added the instance-specific Application Insights telemetry key to onefuzz info get, which will enable logging to the instance specific application insights from the SDK. #353
  • Agent: Added support for parsing ASAN CHECK failed entries, which can occur during large amounts of memory corruption. #358
  • Agent/Service: Added support for parsing the ASAN "scariness" score and description when print_scariness=1 in ASAN_OPTIONS. #359

Changed

  • Agent: Mark tasks as failed if the application under test generates an ASAN log file that the agent is unable to parse. #351
  • Agent: Updated the libfuzzer_merge task to merge pre-existing inputs in a single pass. #282
  • CLI: Clarified the error messages when prefix-expansion fails. #342
  • Service: Rendered pydantic models as JSON when logging to prevent error=None from showing up in the error logs. #350
  • Deployment: Pinned the version of pyOpenssl to the version used by multiple Azure libraries. #348
  • CLI/Service: (PREVIEW FEATURE) Multiple updates to job template management. #354, #360, #361

Fixed

  • Agent: Fixed issue preventing the supervisor from notifying the service on some state changes. #337
  • Deployment: Fixed a regression in retrying password creation during deployment #338
  • Deployment: Fixed uploading tools when rolling back deployments. #347

1.9.0

Added

  • CLI/Service: Added Service-Managed Job Templates as a preview feature. Enable via onefuzz config --enable_feature job_templates. #226
  • Service/agent: Added internal support for unmanaged nodes. This paves the way for bring your own compute for fuzzing. #318
  • CLI: Added onefuzz debug subcommands to simplify coverage and fuzzing performance for libFuzzer jobs from Application Insights. #325
  • Service: Information about the user responsible for creating jobs and repro VMs is now associated with the Job and Repro VMs. #327

Changed

  • Deployment: deploy.py now automatically retries on failure when deploying the Azure Function App. #330

Fixed

  • Service: Address multiple minor issues previously hidden by function decorators used for caching. #322
  • Agent: Fixed libFuzzer coverage support for internal builds of MSVC #324
  • Agent: Address issue preventing instance-wide setup scripts from executing in some cases. #331

1.8.0

Added

  • CLI/Service: Added Event-based webhooks. #296
  • Service: Information about the user responsible for creating tasks is now associated with the tasks (this information is available in the task related event webhooks). #303

Changed

  • Contrib: Azure Devops deployment pipeline uses the --upgrade feature added in 1.7.0. #304

Fixed

  • Service: Fixed setting target_workers, used to configure the number of concurrent libFuzzer workers within a task. #305

1.7.0

Added

  • Deployment: deploy.py now takes --upgrade to enable simplify upgrading deployments. For now, this skips assignment of the managed identity role which only needs to be done on installation. #271
  • CLI: Added Application Insights debug CLI. See onefuzz debug logs #281
  • CLI: Added unique_inputs to the default container types for onefuzz reset --containers and onefuzz containers reset. #290
  • CLI: Added onefuzz debug node to enable debugging a node in a scaleset without having to specify the scaleset. #298

Changed

  • Service: When shutting down an individual scaleset, all of the nodes in the scaleset are now marked for shutdown. #252
  • Service: The scaleset service principal IDs are now cached as part of the respective Scaleset object #255
  • Service: The association from nodes that ran a task are now kept until the node is reimaged, enabling easily connecting to the node that ran a task after task completion. #273
  • Deployment: Pinned urllib3 version due to an incompatible new release #292
  • CLI: Removed calls to containers.list, significantly improving job template creation performance. #289
  • Service: No longer use HTTP 404 response codes during agent registration. #287
  • Agent: Heartbeats are now only sent as part of the execution loop. #283
  • Service: Refactored handlers for agent events, including much more detailed logging. #261
  • Deployment: Prevent users from enabling public access ton containers. #300

Fixed

  • Service: Fixed libfuzzer_merge tasks #240
  • Service: Fixed an issue where scheduled tasks waiting in the queue for longer than 7 days would never get scheduled. #259
  • Service: Removed stale Node references from scalesets #275

1.6.0

Added

  • Service: The service now auto-scales the number of Azure Functions instances as needed #238
  • CLI/Service/Agent: Added the ability to configure ensemble synchronization interval (including disabling ensemble altogether) #229
  • Contrib: Added sample Azure Devops pipeline to maintain instances of OneFuzz #233
  • Deployment: Added utility to create CLI application registrations #236
  • Deployment/Service/Agent: Added a per-instance uniquely generated UUID to telemetry (see docs/telemetry.md for more information) #245

Changed

  • CLI: The CLI now internally caches container authorization tokens #224
  • Service: Moved to using user-assigned managed identities for Scalesets #219
  • Agent: Added stdout to azcopy error logs #247
  • Service: Increased function timeouts to 5 minutes

1.5.0

Added

  • CLI/Service: Added the ability to prevent a VM from getting reset in order to debug tasks #201
  • SDK: Add examples directory to the python package #216
  • Agent: Added connection resiliency via automatic retry (with back-off) throughout the agent #153
  • Deployment: Added the ability to log the application passwords during registration #214
  • Agent: LibFuzzer Coverage metrics are now reported after the batch processing phase #218
  • Deployment: Added a utility to assign scalesets to roles #185
  • Contrib: Added a utility to automate deployment of new releases of OneFuzz via Azure Devops pipelines #208

Fixed

  • Agent: Addressed a race condition syncing input seeds #204

Changed

  • Agent: Instead of ignoring all access violations during libFuzzer coverage processing, stop on second-chance access violations #210
  • Agent: During libFuzzer coverage, disable default symbol paths unless _NT_SYMBOL_PATH is set via target_env. #222

1.4.0

Added

  • CLI: Added onefuzz containers reset to delete containers by type en masse. #198, #202
  • Agent: Added missing approved telemetry as to tool names & crash report identification. #203

Changed

  • Service: Enabled log sampling at the service at 20 items per second. #174

Fixed

  • Service: Fixed multiple bugs in the service, including an exception due to invalid format string proxy or repro VM creation #206

1.3.4

Fixed

  • CLI: Fixed incorrect resetting of granularly selected components introduced in 1.3.3 #193
  • Service: Fixed rate-limiting issues requesting MSI and Storage Account tokens #195

Changed

  • Service: Moved the SDK to use the same pydantic models as the service in request generation #191
  • Service: Improved performance of container validation #196

1.3.3

Fixed

  • Service: Fixed exception generated when deleting repro & proxy VMs #188

1.3.2

Added

  • Service/Agent: Non-functional nodes are now automatically re-imaged #154, #164, #30
  • CLI: Added more granularity for the onefuzz reset sub-command #161, #182
  • Deployment/Agent: Now includes AFL++ #7
  • Deployment/Agent: Now includes Radamsa for Windows #143
  • CLI: The onefuzz status top TUI now allows filtering based on job ID, project, or name #152

Changed

  • Service: Nodes no longer have to wait for the scaleset to finish setup before being able to fuzz #144
  • Agent: Agent now only notifies the service about its current state upon state change #175
  • Service: Task error messages now limit the stdout and stderr to the last 4096 bytes #170
  • Service: Replaced custom queue based event loop with timers #160, #159
  • Agent: Uploads that fail now report the failure earlier #166
  • Agent: All timers now include automatic jitter to reduce request storms #180
  • Agent: Ensemble container synchronization has been unified to once every 60 seconds (plus jitter) #180
  • Agent: Upon agent failure, it will no longer incorrectly re-register and request new work. #150, #146

Fixed

  • Deployment: Addressed an issue with nested exceptions triggered during a failed deployment #172
  • Deployment: Addressed incompatible prerequisite library warnings during deployment #167

1.3.1

Added

  • Testing: Added rust based libFuzzer in the end-to-end integration tests #132

Fixed

  • Agent: Always parse stderr when generating crash reports for LibFuzzer instead of using ASAN_OPTIONS=log_path, which fixes crash reports from non-sanitizer based crashes. #131
  • Deployment: Added data-migration script to fix notifications for pre-release installs #135

1.3.0

Added

  • Agent: Crash reports for LibFuzzer now attempts to parse stderr in addition to ASAN_OPTIONS=log_path. This enables crash reporting of go-fuzz based binaries. #127
  • Deployment: During deployment, App Insights logs can be configured to automatically export logs to the app-insights container in instance specific func storage account. #102

Changed

  • Agent: Reduced logs sent from the agent #125
  • Service: Scalesets now use multiple placement groups, allowing a scaleset to grow to 1000 nodes (or 600 if using a custom image). #121

Fixed

  • Deployment: Support deploying additional platforms (such as OSX). #126
  • Service: Fixed typing error in sorting TaskEvent. #129

1.2.0

Added

  • CLI/Service: Added creating and updating GitHub Issues based on crash reports. #110

Changed

  • Agent: LibFuzzer fuzzing that exits with a non-zero exit code without a resulting crashing input now mark the task as failed. #108
  • Service: The automatic variable repro_cmd used in crash report notifications now includes '--endpoint URL' to reduce friction for users with multiple OneFuzz instances. #113

1.1.0

Added

  • Agent/Service: Added the ability to automatically re-image nodes that are out-of-date #35
  • Deployment: Added data-migration scripts for pre-release installs #12
  • SDK/CLI: Added more onefuzz debug sub-commands to support debugging tasks #95
  • Agent: Added machine_id and version to log messages #94
  • Service: Errors in creating Azure Devops work items from reports now mark the task as failed #77
  • Service: The nodes executing a task are now included when fetching details for a task (such as onefuzz tasks get $TASKID) #54
  • SDK: Added example Azure Functions that uses the SDK #56
  • SDK/CLI: Added the ability to execute debugger commands automatically during repro #39
  • CLI: Added documentation of CLI sub-command arguments (used to describe afl_container in AFL templates #10
  • Agent: Added ONEFUZZ_TARGET_SETUP_PATH environment variable that indicates the path to the task specific setup container on the fuzzing nodes #15
  • CICD: Use sccache to speed up build times #47
  • SDK: Added end-to-end integration test script to verify full fuzzing pipelines #46
  • Documentation: Added definitions for pool, node, and scaleset #17

Changed

  • Agent/Service: Refactored state management for on-VM supervisors #96
  • Agent: Added 'done' semaphore to the agent to prevent agent from fetching additional work once the node should be reset. #86
  • Agent: Nodes now sleep longer between checking for new work. #78
  • Agent: The task execution clock is now started once the task is in the 'setting up' state #82
  • Service: Drastically reduced logs sent to App Insights from third-party libraries #63
  • Agent/Service: Added the ability to upgrade out-of-date VMs upon requesting new tasking #35
  • CICD: Non-release builds now include the GIT hash in the versions and localchanges if built locally with un-committed code. #58
  • Agent: Command replacements now use absolute rather than relative paths. #22

Fixed

  • CLI: Fixed issue using onefuzz template stop which would improperly stop jobs that had the same 'name' but different 'project' values. #97
  • Agent: Fixed input marker expansion (used in AFL templates related to handling @@). #87
  • Service: Errors generated after the task shutdown has started are ignored. #83
  • Agent: Instance specific tools now download and run on windows nodes as expected #81
  • CLI: Using --wait_for_running in onefuzz template jobs now properly waits for tasks to launch before exiting #84
  • Service: Handled more Azure Devops notification errors #80
  • Agent: WSearch service is now properly disabled by default on Windows VMs #67
  • Service: Properly deletes repro VMs #36
  • Agent: Supervisor now flushes logs to Application Insights upon exit #21
  • Agent: Task specific setup script failures now properly get recorded as a failed task and trigger the node to be re-imaged #24

1.0.0

Added

  • Initial public release