All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Agent: Added fuzz tests for coverage recording #3322
- Agent: Added version checking in local tasks #3517
- Agent: Create directories from template specification in local task if they don't exist #3522
- CLI: Added a new command for template creation in the local task
onefuzz-task local create-template
#3531 - CLI/Deployment/Service: Support for retention policies on containers #3501
- Service: Add onefuzz service version to job created events #3504
- Service: Added a start time to job and task records #3440
- Agent: Improved handling of unexpected breakpoints #3493
- Agent: Updated windows interceptor list #3528, #3549
- Agent: Reporting coverage on task start up, ensuring
coverage_data
is emitted at the beginning of every task instead of whennew_coverage
is identified #3502 - CLI/Deployment: Updating onefuzz cli requirements.txt to accept
>= onefuzztypes
versions #3477, #3486 - Service: Improve area/iteration path validation in notifications #3489
- Service: Remove feature flag from heartbeat metrics #3505
- Agent: Terminate process on timeout in Windows agents for the coverage task #3529
- Agent/Service: Bumped several C#, Python, and Rust dependencies #3425, #3424, #3411, #3437, #3436, #3435, #3478, #3484, #3414, #3474, #3434, #3488, #3503, #3520, #3521
- Agent: Fixed dependencies in
onefuzz-task
#3552 - Service: Removed unnecessary method argument in notifications processing #3473
- Service: Ignore regression work item updates when the work item is in some states #3532
- Agent: Added Mariner Linux support for agent VMs #3306
- Service: Added support for custom ado fields that mark work items as duplicate #3467
- Service: Permanently store OneFuzz job result data - # crashing input, # regression crashing input, etc. - in Azure storage #3380, #3439
- Service: Added validation for Iteration/AreaPath on notifications when a job is submitted with a notification config and for
onefuzz debug notification test_template
#3386
- Agent: Updated libfuzzer-fuzz basic template to include required args and make it match cli #3429
- Agent: Downgraded some debug logs from warn to debug #3450
- CLI: Removed CLI commands from the local fuzzing tasks as they can now be described via yaml template #3428
- Service: AutoScale table entries are now deleted on VMSS shutdown #3455
- Service: Removed deprecated Azure retention policy setting that was causing scaleset deployment errors #3452
- Agent: Added a snapshot-based test to coverage implementation #3368
- Agent/CLI/Service: Added ability to capture crash dumps from libfuzzer, when provided #2793 #3409
- CLI/Service: Implemented
--with_tasks
option foronefuzz jobs get
command to expand the task information #3343
- Agent: Migrated all the task types to the template model #3397
- Agent: Removed
srcview
code from OneFuzz since it is not currently utilized #3376 - Agent: Updated default windows VM image to windows 11 #3374
- Agent: Migrated
winapi
towindows-rs
, the newer Microsoft supported version of the Windows API bindings for Rust #3050 - Deployment: Updated the default deployment option for
EnableWorkItemCreation
feature flag to be enabled #3387
- Agent: Deserialize the coverage files directly into the output files #3410
- Agent/Deployment/Service: Bumped several C#, Python, and Rust dependencies as well as the Rust edition across all Rust crates #3396, #3161, #3346, #3391, #2870, #3392, #3402
- Agent: Fixed a bug in agent
DirectoryMonitor
by adding error tolerance when attempting to fetch metadata forCreateKind::Any
orCreateKind::Other
events #3393 - Service: Fixed tag shadowing in logging by giving precedence to the tags produced by log messages over the tags added prior to the call, when the tag names clashed #3388
- Service: Fixed another duplicate Azure DevOps work item creation case by handling
Microsoft.VSTS.Common.ResolvedReason
field when present #3383
- Agent: Fixed tasks hanging when shutting down by forcefully shutting down the runtime before exiting the main task #3378
- Service: Refactored Azure DevOps template rendering to fix duplicate bugs being filed due to title truncation and added several validation tests in this area #3370
- Service: Added feature flag to toggle Azure DevOps work item processing #3353
- Service: Requeue Azure DevOps notifications when the feature flag for work item processing is set to 'disabled' #3358
- Agent: Implemented
debuginfo
caching #3280
- Agent: Limit azcopy copy buffer to 512MB of RAM as the default maximum #3293
- Agent: Define local fuzzing tasks relationships through new templating model #3117
- Deployment: Replaced
--upgrade
flag with--skip_aad_setup
flag in the deploy.py setup script #3345 - Service: Make
ServiceConfiguration
eagerly evaluated #3136 - Service: Improved
TimerRetention
performance through several UPN changes & fixes #3289
- Agent: Fixed resolution of sibling .NET DLLs #3325
- Agent/Service: Bumped several C# and Rust dependencies #3319, #3320, #3317, #3297, #3301, #3291, #3195, #3328
- CLI: Look for azcopy.exe in environment variable
AZCOPY
and determine if it's actually referencing a directory #3344 - CLI: Updated
repro get_files
to handle regression reports #3340 - CLI: Fixed missing
target_timeout
setting in the Libfuzzer basic template #3334 - CLI: Fixed false 'missing' dependency warning #3331
- CLI: Fixed the
debug notification test_template
command expecting atask_id
#3308 - Deployment: Update App Registration redirect URIs if deployment uses a custom domain #3341
- Service: Fixed links in bugs filed from regression reports by populating
InputBlob
when possible #3342 - Service: Fixed several storage issues to improve platform performance and reduce spurious
404
s #3313 - Service: Added extra logging when
System.Title
is too long #3332 - Service: Render
System.Title
before trying to trim it to the max allowed size #3329 - Service: Differentiate
INVALID_JOB
andINVALID_TASK
error codes #3318
- Agent: Added tool to check source allowlists #3246
- Agent: Precache
debuginfo
analysis for target exe in coverage example #3225 - Agent/CLI/Service: Allow tasks environment variables to be set #3294
- CLI/Service: Correlate cli to service to facilitate event lookups in AppInsights #3137
- CLI: Added
--target_timeout
flag for qemu_user template command #3277 - Documentation: Updated Threat Model #3215
- Service: Added optional
Unless
condition when updating/re-opening Work Items #3227 - Service: Include the task ID in the prerequisite task failure message #3219
- Service: Added events retention policy passed-integration-tests #3186
- Agent: Shrink published Rust debug info #3247, #3252
- Agent: Get rid of yanked hermit-abi versions #3270
- Documentation: Updated coverage docs to use correct quotes #3279
- Service: Better errors from Download: Make
GetFileSasUrl
nullable #3229 - Service: Changed template rendering from async to synchronous #3241
- Service: Log webhook exception as an "error" since we are retrying anyways #3238
- Service: Make
WebhookMessageEventGrid
compatible with the event grid format #3286
- Agent: Improved .dll redirection by setting up .local file before invoking LibFuzzer #3269
- Agent/Service: Bumped several C#, Rust dependencies, and Rust version to 1.71 #3278, #3281, #3221, #3230, #3231, #3203, #3240, #3239, #3199, #3254, #3257, #3273, #3258, #3271, #3292
- CLI/Service: Fixed regression bugs, file bugs on
regression_report
and properly reset state on duplicates #3263 - Service: Improve Azure DevOps validation problem reporting and resiliency #3222
- Service: Updated KeyVault access policy for Azure WebSites service account access #3109
- Service: Switched to default
HttpCompletion
, which isResponseRead
to attempt to prevent webhooks occasionally failing to send #3259 - Service: Fixed
Timestamp
response from API #3237 - Service: Trim
System.Title
if length is longer than 128 characters #3284
- Agent: Include debug info in the release binaries to improve backtraces and debuggability #3194
- Agent: Added a timeout when closing the app insight channels #3181
- Agent: Require input marker in arguments when given an input corpus directory #3205
- Agent/CLI/Service: Added
extra_output
container, renameextra
container #3064 - Agent: Creating
CustomMetrics
for RustCustomEvents
#3188 - Agent: Added prereqs for implementing caching for coverage locations and debuginfo in
coverage
task #3218 - CLI: Added command
onefuzz repro get_files
for downloading files to locally reproduce a crash #3160 - CLI: Added command
onefuzz debug notification test_template <template> [--task_id <task_id>] [--report <report>]
to allow a report to be sent when debugging #3206 - Documentation: Added documentation on how to use the validation tools #3212
- Agent: Removed agent traces from AppInsights #3143
- Agent: Include debug info in the release binaries to improve backtraces and debuggability #3194
- Agent: Make coverage-recording errors non-fatal #3166
- Deployment/Service: Enable custom metrics app config value #3190
- Documentation: Renamed example
coverage.rs
torecord.rs
to match documentation #3204 - Service: Moved authentication into middleware #3133
- Service: Store authentication information in KeyVault #3127, #3223
- Service: Port current logging implementation to ILogger #3173
- Service: Added improved error reporting from scale-in protection modification #3184
- Service: Downgraded queue error to warning when retrying because the message is too large #3224
- Agent: Skip entire function if entry offset excluded #3172
- Agent: Try to kill debuggee if Linux recording times out #3177
- Agent: Apply allowlist to source conversion in coverage task #3208
- Service: Bumped C# and Rust dependencies #3200, #3165, #3168, #3153, #3169, #3185, #3191, #3163, #3209, #3146, #3198
- CLI/Service: Don’t validate error codes on client side #3131
- Agent: Switched from unmaintained Rust dependency
tui
toratatui
#3155 - Agent: Removed dependency on the abandoned Rust
users
crate #3150 - Agent/CLI/Service: Bumped several C#, Python, and Rust dependencies #3118, #3132, #3088, #3106, #3140, #3120, #3145, #3151
- CLI/Service: Include a reason when a task has never started #3148
- Service: Fixed bug for scale-in protection #3144
- Service: Created
CustomMetrics
for the Node and Task Heartbeat. #3082 - Service: Add an event for Repro VM creation. #3091
- Service: Add more context to the deletion of nodes. #3102
- Documentation: Create documentation for events 2.0 migration. #3098
- Agent: Match the agent version to the server #3093
- Service: Increase lock wait timeout for
qemu_user
setup script. #3114
- Service: Fixed issue that incorrectly marked tasks as failed. #3083
- Service: Fixed bug when truncating reports. #3103
- Service: Allow use of
readonly_inputs
forqemu_user
template. #3116 - Service: Fix logic to set
check_fuzzer_help
. #3130 - CLI: Fix CLI failure dude to ErrorCode enums out of sync. #3129
- Agent: Added coverage percentage in Cobertura reports #3034
- Agent: Added
maxPerPage
to ORM #3016 - CLI: Added
onefuzz containers files download
command to download the blob content to a file #3060
- Agent: Reconfigured OneFuzz agent to not consume
S_LABEL
symbols from PDBs #3046 - Agent: Update
elsa::sync::FrozenMap
now implements Default #3044 - Agent: Updated agent to use insta Rust crate for snapshot tests of stacktrace parsing #3027
- Agent/CLI/Deployment: Store event payloads as blobs. Add API to download event payload given event id. #3069
- Agent/Service: Bumped Rust version, several Rust dependencies, and several C# dependencies #3049, #3037, #3031, #3023, #2972, #2814, #3052, #3067, #3068, #3056, #2958
- Service: Made our validation errors more specific so that we can handle them appropriately and reference them in documentation #3053
- Service/CLI: Updated the Azure DevOps logic to consume the list of existing items once #3014
- Service: Cap recursion in ORM #2992
- Service: Collect additional report field in an
ExtensionData
property #3079
- Agent: Parse .NET exception stack traces when we see them in crash log outputs #2988
- Agent: Tweaked some of the parameters for the agent's logging to avoid task logger occasionally skipping messages #3070
- Agent: Allow libfuzzer verification to retry #3032
- Agent: Fixed typo in AzCopy parameter name and set default value to true #3085
- Agent/CLI: Added new endpoint to update the pool authentication in order to fix multiple stop messages from being sent after node shuts down #3059
- CLI: Changed
--check_fuzzer_help
to--no_check_fuzzer_help
#3063 - Service: Include exception information when validation fails #3077
- Service: Added another truncation case for 'Request body too large...' errors #3075
- Service: Fixed the logic for marking task as failed #3083
- Service: Fixed error deserializing events from the events container #3089
This release removes the parameters --client_id
, --override_authority
, and override_tenant_domain
from the config
command.
For those accessing the CLI with a service principal, the parameters can be supplied on the command line for each of the CLI commands.
For example, if deploying a job:
onefuzz --client_id [CLIENT_ID] --client_secret [CLIENT_SECRET] template libfuzzer basic --setup_dir .....
- Agent: Added
validate
command to the agent to help validate a fuzzer #2948 - CLI: Added option to libfuzzer template to specify a known crash container #2950
- CLI: Added option to libfuzzer template to specify the duration of the tasks independently from the job duration #2997
- Agent: Install v17 Visual Studio redistributables #2943
- Agent/Service: Use minimized stack for crash site if no ASAN logs are available #2962
- Agent/Service: Unified several Rust crate dependency versions across the platform #3010
- CLI: Remove additional parameters from the
config
command and require them on each CLI request if accessing the CLI with a service principal #3000 - Service: Loosen scriban template validation #2963
- Service: Updated integration test pool size #2935
- Service: Pass the task tags to the agent when scheduling jobs #2881
- Agent: Ensure custom
target_options
are always passed last to the fuzzer #2952 - Agent: Removed xml-rs dependency #2936
- Agent: Better logging of failures in the task_logger #2940
- Agent/Service: Updates to address CVE's #2931, #2957, #2967
- Deployment/Service: Renamed EventGrid subscription to conform with EventGrid's naming scheme #2960
- Deployment/Service: Added required KeyVault access policy allowing OneFuzz Function App to use an SSL cert for custom domain endpoints #3004, #3006
- Documentation: Updated 'Azure Devops Work Item creation' doc to remove an outdated template reference #2956
- Service: Updated feature configuration package to fix an issue where 2 feature flags were using the same ID #2980
- Service: Make
GetNotification
nullable to fix errors looking up non-existent notification IDs #2981 - Service: UniqueReports should be UniqueInputs in LibFuzzer merge task #2982
- Service: Fix Notification
delete
action #2987 - Service: Added handle for missing unique field key in
AdoFields
#2986 - Service: Implemented
ITruncatable
forJobConfig
&EventJobStopped
to avoid exceptions for messages being too large for Azure Queue #2993
- This release has fully deprecated
jinja
templates and will only acceptscriban
templates. - The
onefuzz config
command has removed the--authority
and--tenant_domain
parameters. The only required parameter for interactive use is the--endpoint
parameters. The other values needed for authentication are now retrieved dynamically. - The recording components used in the
coverage
task have been rewritten for improved source-level reporting. The task-level API has one breaking change: thecoverage_filter
field has been removed and replaced by themodule_allowlist
andsource_allowlist
fields. See here for documentation of the new format. - The old
dotnet
template has been removed anddotnet_dll
is nowdotnet
.
- Service: Added unmanaged nodes integration tests. #2780
- CLI: Added notification
get
command to retrieve specific notification definitions. #2818 - Agent: Added function allow-list to the coverage example exe. #2830
- Service: Added feature flag, validation when new notifications are created, and CLI support for migration to scriban. #2816, #2834, #2839
- Agent: Switch over to new
coverage
task. #2741 - Service: Added
--notification_config
support for dotnet templates. #2842 - Service: Report extension errors when deploying VM in a scaleset. #2846
- Service: Semantically validate notification configurations. #2850
- Agent: Accept optional
dir
of coverage test inputs. #2853 - Service/Agent: Added extra container to tasks. #2847
- Documentation: Document
coverage
crate and tool. #2904 - Agent: Add the ability for a task to gracefully shutdown when a task is stopped. #2912
- Service: Deprecated the job template feature. #2798
- Service: Deploy with scriban only, removing jinja. #2809
- Agent: Defer setting coverage breakpoints. This avoids breaking hot patching routines in the ASan interceptor initializers. #2832
- Service: Updated remaining jinja docs. #2838
- Service: Support another exception case when adding
AssignedTo
to telemetry. #2829 - Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies.#2849, #2855, #2274, #2544, #2857, #2876
- Contrib: Updated contribution
onefuzz config
command lines. #2861 - Agent: Removed Z3 telemetry. #2860
- Service: Change the optional parameter names and set an expiration for the cache created on the
onefuzz config
command. #2835 - Agent: Removed the function allowlist. #2859
- Agent: Updated clap to remove suppressions. #2856
- Agent: Removed unused telemetry data. #2863
- CLI: Removed old
libfuzzer dotnet
template. #2875 - Test: Updated C# functional testing InfoResponse. #2894
- Service: Updated the truncating logic when getting the error so that we retrieve the last messages. #2896
- Service: Added additional filter check for reports and regressions. #2911
- Agent: Removed a stray print statement. #2823
- Deployment: Fixed a bug in
registration.py
when creating CLI service principals. #2828 - Example: Fixed coverage example build. #2831
- Service: Fixed the way we report an error when creating a Scaleset under a missing Pool. #2844
- Service: Update SharpFuzz to a version that supports .NET7.0, and change .NET installation method. #2878
- Deployment: Fixed an error where a variable was being referenced before being assigned. #2903
- Service: Created a wrapper function to handle columns defined as GUID in tables. #2898
- Service: Pass
PreserveExistingOutputs
to the task. #2905 - Service: Fixed notification validation. #2914
- Service: Fixed the custom script definition that could prevent the creation of the repro VM due to a change in the underlying extension setup processes. #2920
- Deployment: Fixed
--auto_create_cli_app
flag bug used during deployment. #2921 - Agent/Service: Updates to address CVE's. #2933
- Service: Fixed a condition when generating a task configuration. #2925
- Deployment/CLI: OneFuzz Config refactored -
tenant_id
,tenant_domain
,multi_tenant_domain
, andcli_client_id
are now required values in the config.json used during deployment and no longer required when running the config command. #2771, #2811 - Agent: Fully escape allowlist rules #2784
- Agent: Apply allowlist to all blocks within a function #2785
- CLI: Added a cli subcommand
onefuzz debug notification template
to validate scriban notification templates #2800 - Service: Added Notification failure webhook to communicate Notification failures #2628
- Service: Include
AssignedTo
when failing to create a work item due to an authentication exception #2770
- Agent: Fixes & improvements to
Expand
behavior #2789 - Agent: Triming whitespace in output from monitored process before printing #2782
- CLI: Fixed default value of analyzer_exe #2797
- CLI: Fixed missing
readonly_inputs
parameter in dotnet & dotnet_dll templates #2740 - Service: Fixed query to get the existing proxy #2791
- Service: Truncate webhooks message length for azure queue size compatibility #2788
- Service: Add Optional Analysis Task to Libfuzzer Template #2748
- Agent: Use
elsa
for improved interface withdebuggable_module::Loader
#2703 - Agent: Add sourceline output and logging to coverage example #2753
- Agent: Fix Linux detection of shared library mappings #2754
- Agent: Support AllowList extension #2756
- Agent: Add
stdio
dumping to example #2757
- Service: Update Azure Cli #2733
- Service: Truncate Large Webhook Events #2742
- Service: Wrap fallible ORM functions in try/catch #2745
- Agent/Supervisor/Proxy: Updated third-party Rust dependencies. #2744
- Agent: Fixed Mulit-Agent Issue - Added
machine_id
to config_path and failure_path of the Agent #2731 - Service: Fixed Proxy Table Query #2743
- Service: Fix Notification Logic and Regression Reporting #2751#2758
- Agent: Added more into-JSON coverage conversions #2725
- Agent: Added binary coverage merging measurements #2724
- Agent: Added deserialization compatibility functions #2719
- Agent: Added OS-generic
CoverageRecord
builder to capture output of target child process and allowLoader
reuse in coverage recording #2716 - Agent: Improve source coverage of HTML reports #2700, #2701, #2706
- Deployment: Added support for custom domain names used as OneFuzz endpoints #2720
- Service: Added documentation for unmanaged node deployment #2694
- Agent: Use a custom
Output
type when recording coverage #2723 - Agent: Reduce mutation in the agent state machine #2710
- Service: Include dotnet version in
info
response #2693 - Service: Use feature flags to get the node disposal strategy #2713
- Agent: Escape periods when converting globs #2721
- Agent: Ignore benign recv hangup in agent timer functions #2722
- Agent: Fix NullRef exception when getting a scaleset that does not exist #2692
- Service: Downgrade error on "cannot delete nodes from scaleset" to a warning #2691
- Service: Fixed build issue related to dotnet version
7.0.101
#2698 - Service: Adding
public
identifier toEvents
to restore missing events #2705
- Service: Added support for feature flags which allows us to deploy new code in parts and turn it on when it's ready. #2620
- Service: Added a validation endpoint for the notification template. #2655
- Service: Update LLVM from v10 to v12 now that we are supporting Ubuntu 20.04 as our default image. #2617
- Agent: Remove unused coverage recorder from
input-tester
. #2681 - Agent: Rename
coverage
tocoverage-legacy
. #2685
- CLI: Return an error when uppercase application names are specified when using deploy.py. #2665
- Agent: Fix local fuzzing mode. #2669
- Service: Post the JobCreated event when a job is created. #2677
- Service: The repro
Create
command will now fail if insert fails. Also add additional tests. #2678 - Service: Added support for
Contains Words
in WIQL #2686
When upgrading from version 5.20 a manual step is required. Before deploying 6.0 delete both Azure App Functions and the Azure App Service plan before upgrading. This is required because we have migrated the service from python
to C#
.
After deployment, there will be two App Functions deployed, one with the name of the deployment and a second one with the same name and a -net
suffix. This is a temporary situation and the -net
app function will be removed in a following release.
If you have not used the deployment parameters to deploy C# functions in 5.20, you can manually delete the -net
app function immediately. Deploying the C# functions was not a default action in 5.20, for most deployments deleting the -net
app function immediately is ok.
With this release we are moving from jinja templates to scriban templates. See the documentation for scriban here.
Version 6.0 will convert jinja templates on-the-fly for a short period of time. We do not guarantee that this will be successful for all jinja template options. These on-the-fly conversions are not persisted in the notifications table in this release. They will be in a following release. This will allow time for conversions of templates that are not handled by the current automatic conversion process.
The default value for the --container_type
parameter to the container
command has been removed. The container_type
parameter is still required for the command. This change removes the ambiguity of the container information being returned.
- Agent: Added
machine_id
a parameter of the agent config. #2649 - Agent: Pass the
machine_id
from the Agent to the Task. #2662
- Service: Deployment enables refactored C# App Function. #2650
- CLI: Attempt to use broker or browser login instead of device flow for authentication. Canceling the attempt with
Ctrl-C
will fall back to using the device flow. #2612 - Service: Update to .NET 7. #2615
- Service: Make Proxy
TelemetryKey
optional. #2619 - Service: Update OMI to 1.6.10.2 on Ubuntu VMs. #2629
- CLI: Make the
--container_type
parameter required when using thecontainers
command. #2631 - Service: Improve logging around notification failures. #2653
- Service: Standardize HTTP Error Results. Better Rejection Message When Parsing Validated Strings. #2663
- CLI: Retry on Connection Errors when acquiring auth token. #2668
- Service: Notification Template
targetUrl
parameter fix. Only use the filename instead of the absolute path in the URL. The makes the links created in ADO bugs work as expected. #2625 - CLI: Fixed SignalR client code not reading responses correctly. #2626
- Service: Fix a logic bug in the notification hook. #2627
- Service: Bug fixes related to the unmanaged nodes (an unreleased feature). #2632
- Service: Fix invocation of
functionapp
in the deployment script. Where the wrong value/parameter pair were used. #2645 - Service: Fixing .NET crash report no-repro. #2642
- Service: Check Extensions Status Before Transitioning to
running
state during VM setup. #2667
- Service: Added endpoint to download agent binaries to support the unmanaged node scenario. #2600
- Service: Added additional error handling when updating VMSS nodes. #2607
-
Service: Added additional logging when using the
decommission
node policy. #2605 -
Agent/Supervisor/Proxy: Updated third-party Rust dependencies. #2608
-
Service: Added optional
retry_limit
when connecting to the repro machine. #2609
- Service: Fixed
status top
in C# implementation. #2604 - Service: Only add "re-opened" comments to a bug if it was actually reopened. #2623
- Service: Delete nodes once they're done with tasks instead of releasing scale-in protection. #2586
- Service: Switch to using the package provided by Azure Functions to set up Application Insights and improve its reporting of OneFuzz transactions. #2597
- Service: Fix handling duplicate containers across accounts in C# functions. #2596
- Service: Fix the notification GET request on C# endpoints. #2591
- Service: Use records to unpack the request parameters in
AgentRegistration
. #2570 - Service: Convert ADO traces to
customEvents
and updatenotificationInfo
. #2508 - Agent: Include computer name in
AgentRegistration
& decode Instance ID from it. This will reduce the amount of calls to Azure minimizing throttling errors. #2557
- Service: Improve webhook logging and accept more HTTP success codes. #2568
- Service: Reduce fetches to VMSS #2577
- CLI: Use the virtual env folder to store the config if it exists. #2561, #2567, #2583
- Service: Reduce number of ARM calls in
ListVmss
reducing calls to Azure to prevent throttling. #2539 - Service: ETag updated in
Update
andReplace
. #2562 - Service: Don't log an error if we delete a Repro and it is already missing. #2563
- Service: Added exponential backoff for failed notifications. Many of the failures are a result of ADO throttling. #2555
- Service: Add a
DeleteAll
operation to ORM that speeds up the deletion of multiple entities. #2519
- Documentation: Remove suggestion to reset
IterationPath
upon duplicate. #2533 - Service: Ignoring the scanning log file when reporting an issue with azcopy. #2536
- CLI: Fixed failures in command
$ onefuzz status pool <pool_name>
. #2551 - Deployment: Fix the OneFuzz web address that is used to generate the
input_url
for bug reporting. #2543 - Service: Produce an error if coverage recording failed due to a timeout. #2529
- Service: Increased the default timeout for coverage recording from 5 seconds to 120 to prevent premature errors while parsing symbols and executables. #2556
- Service: Fixed errors in ADO notifications to reduce duplicate bug-filing. #2534
- Service: Handle null values better in
ScalesetOperations
andVmssOperations
when a scaleset is in shutdown state. #2538 - Service: Fix exception message formatting in
VmssOperations
. #2546 - Service: Downgrade instance not found exception. #2549
- Service: Lower log level on symbol region overlap findings during coverage recording. #2559
- Documentation: Added OneFuzz logo to the README file. #2340
- Agent: Added try_insert function when building code coverage maps. #2510
- Documentation: Described the importance of using the right runtime identifier (RID) when building .NET binaries. #2490
- Service: Downgraded logging statement from
error
towarn
and also included the http result code. #2484 - Service/CLI: Updated python dependencies. #2470
- Service: Updated the verbosity of
azcopy
logging to assist in debugging copy failures. #2598 - Agent/Supervisor/Proxy: Updated third-party Rust dependencies.#2500
- Service: Update the logic for checking if a blob exists before uploading to reduce contention during uploads. #2503
- Service: Changed the way we update the
scaleInProtection
on a scaleset node to minimize throttling. #2505
- Service: Only fetch InstanceView data when required. This will reduce throttling by Azure.#2506
- Service: Fixed github notification queries in the C# implementation (currently not turned on). #2513, #2514
- Service: Added support for Jinja template migration to Scriban. #2486
- Service: Replaced missing tab that caused ADO queries to fail to find existing work items resulting in duplicate items being created. #2492
- Tests: Fixed
integration-tests-linux
. #2487
- Service: Use
InterpolatedStringHandler
to move values toCustomDimensions
Tags #2450 - Service: C# Can create ADO notifications #2456, #2458
- Service: C# Cache VMSS VM InstanceID lookups #2464
- CLI: Retry on connection reset #2468
- Agent: Enable backtraces for agent errors #2437
- Service: Fix logic to retrieve partitionKey and rowKey #2447
- Service: Permit periods in Pool names #2452
- Service: Node state getting reset to init #2454
- Service: Fix null ref exception in C# logging #2460
- Service: Correct pool transitions #2462
- Service: Fix UpdateConfigs #2463
- Service: Allow worker loops to continue after errors #2469
- Service: Lowercase webhooks digest header value #2471
- Service: Fix C# Node state machine. #2476
- Service: Adding missing caching from python code #2467
- Service: Implement not-implemented
GetInputContainerQueues
#2380 - Service: Adding new default image config value to instance config 2434
- Service: Port
SyncAutoscaleSettings
from Python to C# #2407
- Deployment: Updating error and fixing default value for
auto_create_cli_app
#2378 - Service: Do not discard proxy objects when setting state #2441
- Service: Do not fail task on notification failure #2435
- Service: Cleanup queues for non-existent pools and non-existent tasks #2433
- Service: Delete pool queue when pool is deleted #2431
- Service: Minor fixes to service logging and error handling #2420
- Service: Fixed linux repro extensions #2415
- Service: Mark tasks as failed if a work unit cannot be created for the task #2409
- Service: Fixed several bugs in C# ports for
TimerProxy
,TimerRetention
,AgentEvents
,Node
,Tasks
andJobs
#2406, #2392, #2379 - Service: Fixed Azure linux instance proxy extensions provisioning failures #2401
- Service: Fixed C# scheduling bugs #2390
- Service: Fixed
MarkDependantsFailed
error checking #2389 - Service: Fixed
SearchStates
querying inTaskOperations
#2383 - Service: Fixed Scaleset response Auth inclusion #2382
- Service: Fix custom type interpolation in queries #2376
- Service: Fixed error in C# port for
DoNotRunExtensionsOnOverprovisionedVms must be false if Overprovision is false
#2375
- Deployment: Added optional flags
--onefuzz_app_id
&--auto_create_cli_app
fordeploy.py
to allow for custom app registrations. #2305 - Deployment: Added optional flag
--host_dotnet_on_windows
fordeploy.py
that enables running dotnet functions on Windows based hosts to allow for attaching a remote debugger #2344 - Deployment: Added optional flag
--enable_profiler
fordeploy.py
to enable memory and cpu profilers in dotnet Azure functions #2345 - Service: Enabled AppInsights dependency tracking to enable better analysis of Azure Storage usage on OneFuzz deployments#2315
- Service: Added
Scriban
templating library as a dependency stand-in for Jinja on C# ported services #2330 - Service: Use 64-bit worker for dotnet functions #2349
- Agent: Add Cobertura XML output to
src-cov
example binary #2334 - CLI: Add
onefuzz debug task download_files <task_id> <output>
command to download a task’s containers #2359
- Service: Removed some response-only properties from the Task model #2335
- Service: Create storage tables on startup #2309
- Service: Switched to using Graph SDK instead of manually constructing queries #2324
- Service: Cache
InstanceConfig
for improved read performance #2329 - Deployment: Updated
deploy.py
to set all function settings at once for faster deployment and upgrades #2325 - Devcontainer: Move global tool installs into another script so they can be cached #2365
- Bumped several dependencies in multiple files #2321, #2322, #2360, #2361, #2364, #2355
- Service: Fix
az_copy
syncing issues by removing themax_elapsed_time limit
and relying onRETRY_COUNT
instead #2332 - Service: Implement not implemented bits in Scaleset/VMSS Operations for
RemiageNodes
&DeleteNodes
#2341 - Service: Fixed bugs in C# port of
Proxy
andTimerProxy
functions #2317, #2333 - Service: Enforce that there are no extra properties in request JSON, and that non-null properties are
[Required]
#2328 - Service: Remove
IDisposable
fromCreds
#2327 - Service: Removed required field in
Requests
to match python behavior #2367 - Service: Fixed bug in Azure DevOps notification information #2368
- Service: Fixed memory leaks in
AgentEvents
and several supporting libraries #2356 - Service: Fixed inconsistencies in VMSS creation between C#/Python functions #2358
- Service: Fixed dotnet
Info
function to correctly get version number from assembly attributes instead of the config #2316 - Service: Fixed bug in python types #2319
- Service: Fixed bugs in
timer_workers
to allow it to run properly #2343 - CLI: Coverage task should have access to
readonly_inputs
containers #2352 - Devcontainer: Ensure that python virtual environment is installed #2372
- Deployment: Add
--use_dotnet_agent_functions
to deploy.py. #2292 - Service: Added Logging to
az_copy
calls for improved failure tracking. #2303 - Service: Added Logging when sending ADO Notifications #2291
- Service: Additional C# migration work. #2183, #2296, #2286, #2282, #2289
- CLI: Changed the CLI's
scaleset
commandssize
positional parameter tomax_size
to better communicate its use in auto scaling properties. #2293
- Deployment: Fixed
set_admins.py
script. #2300 - Service: Include serialization options when sending event message. #2290
- Service: Converted remaining events to C#. #2253
- Agent: Add the dotnet crash report task. #2250
- Service: Additional C# migration work. #2235, #2257, #2254, #2191, #2262, #2263, #2269
- Agent: Increase the size of the output buffer when collecting logs from agent. #2166
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #2270
- Service: Use a singleton for logging to reduce memory use. #2247
- Service: Use a singleton for the EntityConverter. #2267
- Service: Add retries when creating a connection to a repro machine for debugging on windows. #2252
- Deployment: Fix deploy to use the correct date formats when querying MSGraph. #2258
- Service: Sync the Autoscale table to current scaleset settings. #2255
- Service: Fixed the pool comparison in the scheduler. #2260
- CLI: Fixed the way
job
andtask
state enumerations are compared. #2004 - Service: Clone the JsonSerializerOptions instead of just modifying it. #2280
- Service: Fixed a NullReferenceException in CreateQueue. #2283
- Recommendation in
getting-started.md
that OneFuzz users include a.onefuzz
file in their project root directory for security tool detection #2236 - Agent: New
libfuzzer_dotnet_fuzz
task #2221
- CLI: Updated default Windows VM host image. #2226
- Agent: Modified LibFuzzer struct to own its environment and option data #2219
- Agent: Factor out generic LibFuzzer task #2214
- Service: Enable C# migrated
TimerRetention
,TimerDaily
, andcontainers
functions #2228, #2220, #2197 - Service: Finished migrating
TimerRepro
to C# #2222, #2216, #2218 - Service: Change instances of
NotImplementedException
to more accurately beNotSupportedException
exceptions #2234 - Service: Migrated
Tasks
,Notifications
,add_node_ssh_key
, andProxy
functions to C# #2233, #2188, #2193, #2206, #2200
- Service: Update the autoscale settings to allow a VM scaleset to scale down to zero nodes and prevent new nodes from spinning up when in the
shutdown
state. #2232, #2248 - Service: Add a missing function call to properly queue webhook events in
WebhookOperations
#2231 - Service: Add a missing job state transition to the
Task
implementation. #2202 - Service: Fixed the return value in the C# implementation when associating a subnet with the NSG. #2201
- Service: Changed log level from
Error
toInfo
inTimerProxy
. #2185 - Service: Fixed
TimerTasks
config bugs in the C# port. #2196
- Agent: Depend on SharpFuzz 2.0.0 package in the
LibFuzzerDotnetLoader
project. #2149 - Test: Added
GoodBad
C# example project to use with theLibFuzzerDotnetLoader
integration tests. #2148
- Service: Implemented the
containers
function in C#. #2078 - Service/Build: Reuse agent build artifacts if nothing in the agent source tree has changed. This is to speed up dev builds and will not impact official releases. #2115
- CLI: Default autoscale minimum value to 0. This allows a scaleset to scale-in until there are zero nodes running when no work is pending in the queue. This is important to ensure VM's have the latest patches when running. #2112, #2162
- Service: Initial work to migrate
TimerRepro
function to C#. #2168 - Service/CLI: Remove support for pre 3.0.0 style authentication. #2173
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #2164, #2056, #2175, #2127
- Service: Updated C# dependencies. #2181
- Service: When getting information about a task, ignore the task's state if the
job_id
is specified. #2171
- Agent: Drop the global event sender when closing the telemetry channel to ensure all events are flushed. #2125
- Service: Add correct routes and auth to agent C# functions. #2109
- Service: Port
agent_registration
to C#. #2107 - Agent: Add the
dotnet_coverage
task. #2062 - Agent: Add multiple ways to specify LibFuzzerDotnetLoader targets. #2136
- Agent: Add logging to LibFuzzerDotnetLoader. #2141
- Documentation: Added documentation for LibFuzzerDotnetLoader. #2142
- Service: Add caching to C# storage implementation so repeated queries do not get throttled. #2102
- Service: Remove unused poolname validation. #2094
- Service: Make the hostbuilder async in C#. #2122
- Service: Updated the scaling policy for the App Functions. #2140
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #2022
- Agent: Remove incomplete macOS support. #2134, #2135
- Service: Include State filter when searching for expired tasks and jobs. #2138
- Service: Fix reported TLS errors. #2087
- Service: Change the
upload_file
method to use the Azcopy command by default for robustness and fall back to the Azure Python SDK implementation if needed. This also addresses issues where low bandwidth connections timeout due to not being able to handle multiple concurrent upload streams. #1556 - Service: Update the log SAS URL to last as long as the job duration. #2116, #2121
- Service: Fixed a number of issues in the C# implementation of
TimerProxy
. #2133 - Agent: Fix a race condition when monitoring files on the VM. #2105
This a hotpatch to the 5.7.0 release fixing SAS URL generation which had the potential to cause tasks to fail. #2116
- Agent: Add
NodeState
to Node Heartbeat to better track the current state of nodes in the system #2024, #2053 - Service: Ported existing Python functions to C# #2061, #2072, #2076, #2066
- Service: Enabling ported C# functions for
QueueNodeHeartbeat
,QueueTaskHeartbeat
, andQueueSignalREvents
#2046, #2047 - Service: Add null analysis attributes to service result types to make it easier to check and use the various existing result types #2069
- Service: Add dotnet editorconfig underscores naming rule for private fields to start with an underscore, ensuring OmniSharp will generate conformant names by default #2070
- Agent: Update onefuzz-agent clap to version
3.2.4
#2049 - Agent: Added scripts to install dotnet on windows and ubuntu fuzzing VMs #2038
- Deployment: Update Getting Started instructions for
deploy.py
's file permissions. #2030
- CLI: Error output to specify that the tools are missing locally, not on the repro VM #2036
- Service: Handle service event messages that are too big to fit in a queue message. #2020
- Service: Removing unnecessary
/obj/
directory. #2063
- Service: Add Function App settings to Bicep template and
deploy.py
. #1973 - Agent: Add a timestamp to agent log to make it easier to correlate events. #1972
- Agent/Supervisor/Proxy: Rename the supervisor process from
onefuzz-supervisor
toonefuzz-agent
. #1989 - Agent/Supervisor/Proxy: Rename the task executor that runs on the VM from
onefuzz-agent
toonefuzz-task
. #1980 - Agent/Supervisor/Proxy: Ensure
GlobalFlag
registry value is initialized for targets. #1960 - Agent/Supervisor/Proxy: Enable full backtraces on Rust panics. #1959
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #2003, #2002, #1999, #1992, #1986, #1983, #1982, #1981, #1985, #1974, #1969, #1965
- CLI/Service: Updated multiple first-party and third-party Python dependencies. #2009, #1996
- Agent: Remove stray print statement from the task_logger. #1975
- Agent: Fix local coverage definition by removing a duplicated command line parameter. #1962
- Service: Fix Instance Config and Management Logic. #2016
- Documentation: Update coverage filtering docs. #1950
- Agent: Allow the agent to skip reporting directories. #1931
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1935
- CLI: Deprecate
libfuzzer_coverage
task. #1944 - CLI: Use consistent logger names. #1945
- Service: Updated functionality to the service port from Python to C#. #1922, #1925, #1947
- Agent: Allow old reports to be parsed. #1943
- Agent: Remove transitive OpenSSL dependency. #1952
- Agent: Ensure
GlobalFlag
registry value is initialized when checking library dependencies. #1960 - Service: Allow old reports to be parsed. #1940
This a hotpatch to the 5.4.0 release fixing the parsing failures from old crash reports.
- Agent: Allow old reports to be parsed #1943
- Agent: Include
LD_LIBRARY_PATH
in shared library dependency check if and only if set by command. #1933 - Service: Allow old reports to be parsed #1940
- Agent: Added the OneFuzz version and tool name to the Crash Report. #1635
- Agent: Added a check for missing libraries when running the LibFuzzer
-help
check. #1812 - Service: Added new functionality to the service port from Python to C#. #1794, #1813, #1814, #1818, #1820, #1821, #1830, #1832, #1833, #1835, #1836, #1838, #1839, #1841, #1845, #1846, #1847, #1848, #1851, #1852, #1853, #1854, #1855, #1860, #1861, #1863, #1870, #1875, #1876, #1878, #1879, #1880, #1884, #1885, #1886, #1887, #1888, #1895, #1897, #1898, #1899, #1903, #1904, #1905, #1907, #1909, #1910, #1912
- Service: Restrict node operations to administrators. #1779
- CLI/Service: Updated multiple first-party and third-party Python dependencies. #1784
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1619, #1644, #1645, #1646, #1655, #1700, #1849, #1882
- Agent: Separate
llvm-symbolizer
setup from sanitizer environment variable initialization. #1778 - Agent: Set the TSan options based on the external symbolizer. #1787
- CLI: Added the
ONEFUZZ_CLIENT_SECRET
environment variable and removed theclient_secret
field from the configuration file. This prevents accidental misuse via persisting the secret to disk outside of confidential client environments. If you have set a client secret in your configuration file in a public client, we recommending removing and revoking it. CI scripts that currently set the client secret in the config must instead pass it via theONEFUZZ_CLIENT_SECRET
environment variable or on each CLI invocation via the--client_secret
argument. #1918 - CLI: Use a SAS URL to download log files. #1920
- Agent: Only watch directories for change events. #1859
- Agent: Switch to a smart constructor to minimize misuse. #1865
- Service: Fixed an issue where jobs that do not have logs configured failed to get scheduled. #1893
- Agent: Add a compiler flag to generate debug info for the
windows-libfuzzer
load library test target. #1684 - Agent: Add a Rust crate to debug missing dynamic library errors on Windows. #1713
- Agent: Add support for detecting missing dynamic libraries on Linux. #1718
- Service: Connect the auto scaling diagnostics to the log analytics workspace. #1708
- Service: Handle the situation where a VM scale set instance is destroyed before we have removed scale-in protection. #1719
- Service: Add additional support for auto scaling including changes to the CLI. New scale sets will automatically be created with auto scaling enabled. #1717, #1763
- Agent/Service/CLI: Add support for generating log files that can be downloaded using the CLI. #1727, #1723, #1721
- Service: Port ARM templates to Bicep. #1724, #1732
- Service: Initial changes to port the service from Python to C#. #1734, #1733, #1736, #1737, #1738, #1742, #1744, #1749, #1750, #1753, #1755, #1760, #1761, #1762, #1765, #1757, #1780, #1782, #1783, #1777, #1791, #1801, #1805, #1804, #1803
- Service: Make sure the scale set nodes are unable to accept work while in the
setup
state. #1731
- Agent: Reduce the logging level down from
warn
todebug
when we are unable to parse an ASan log. #1705 - Service: Move the creation of the event grid topic to the deployment template from the
deploy.py
script. #1591 - Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1548, #1617, #1618
- Service: Consolidate the two log analytics down to one. #1679
- Service: Updated resource name in Bicep file to prevent name clash when deploying 5.3.0. #1808
- Service: Auto scale setting log statement is not an
error
changed it toinfo
. #1745 - Agent: Fixed Cobertera output so that coverage summary renders in Azure Devops correctly. #1728
- Agent: Continue after non-fatal errors during static recovery of SanCov coverage sites. #1796
- Service: Fixed name generation for a few resources in the Bicep file to increase uniqueness which prevents resource name clash. #1800
- Service: Added a new webhook message format compatible with Azure Event Grid. #1640
- Service: Added initial auto scaling support for VM scale sets. #1647, #1661
- Agent: Add an explicit timeout to setup scripts so hangs are easier to debug. #1659
- CLI/Service: Updated multiple first-party and third-party Python dependencies. #1606, #1634
- Agent: Check system-wide memory usage and fail tasks that are nearly out of memory. #1657
- Service: Fix
task
field to the correctNodeTasks
type so serialization works correctly. #1627 - Agent: Convert escaped characters when accessing the name of a blob in a URL. #1673
- Agent: Override
runs
parameter when testing inputs as we only want to test them once. #1651 - Service: Remove deprecated
warn()
method. #1641
- CLI/Service: Added
fuzzer_target_options
argument to thelibfuzzer
templates to allow passing some target options only in persistent fuzzing mode #1610
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1530
- CLI/Service: Updated multiple first-party and third-party Python dependencies. #1576 #1577 #1579 #1582 #1586 #1599
- CLI/Service: Begin update of scale set instances before reimaging to ensure they match the latest scale set model. #1612
- Agent: Removed the
process_stats
telemetry event, which fixes a class of memory leaks on Windowslibfuzzer_fuzz
tasks. #1608 - CLI/Service: Fixed seven day stale node reimaging check. #1616
- Agent: Added source line coverage data #1518 #1534 #1538 #1535 #1572
- Agent: Added Cobertura XML output for source code visualization #1533
- Service: Added auto configuration properties to the monitoring agents #1541
- Service: Added tags to scalesets and VMs #1560
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1489 #1495 #1496 #1501 #1502 #1507 #1510 #1513 #1514 #1517 #1519 #1521 #1522 #1528 #1557 #1566
- Agent: Changed the function that gets the
machine_id
to beasync
to avoid runtime nesting #1468 - Service: Removed generic reset command from the CLI #1511
- Service: Updated the way we check for endpoint authorization #1472
- Service: Increase reliability of integration tests. #1505
- Agent: Avoid leaking unused file and cache data #1539
- Agent: Fixed new clippy errors #1516
- Agent: Added common source coverage format. #1403
- Service: Added class to store and retrieve rules associated with an API endpoint. This supports the ability to control who has access to an API. #1420
- Service: Support for NSG creation during deployment, allowing restricted access to the scaleset and repro VMs. #1331, #1340, #1358, #1385, #1393, #1395, #1400, #1404, #1406, #1410
- Service: Guest account access is disabled by default when creating the default service principal during deployment. #1425
- Service: Group membership check added. #1074
- Service: Exposed the
target_timeout
parameter in theradamsa basic
template. #1499
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1360, #1364, #1367, #1368, #1369, #1382, #1429, #1455, #1456, #1414, #1416, #1417, #1423, #1438, #1446, #1458, #1463, #1470, #1453, #1492, #1493, #1480, #1488, #1490
- Service: Fixed Azure DevOps work item creation by adding missing client initialization. #1370
- Service: Fixed validation of the
target_exe
blob name, enabling nesting in a subdirectory of thesetup
container. #1371 - Service: Migrated to MS Graph, as
azure-graphrbac
is soon to be deprecated. #966 - Service: Stopped ignoring unexpected errors when authenticating the client secret. #1376
- Service: Fixed regex to correctly capture the object ID when trying to remove an invalid application ID. #1408
- Service: Added check for service principal use during user role assignment. #1479
- Service: Added support for Compute Gallery images. #1450
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1301, #1302, #1310, #1312, #1332, #1335, #1336, #1337, #1341, #1342, #1343, #1344, #1353
- CLI/Service: Updated multiple first-party and third-party Python dependencies. #1346, #1348, #1355, #1356
- Service: Fixed authentication when using a client secret. #1300
- Deployment: Fixed an issue where the wrong AppRole was assigned when creating new CLI registrations. #1308
- Deployment: Suppress a dependency's noisy logging of handled errors when deploying. #1304
- Agent: Added ability to handle fake crash reports generated by debugging tools during regression tasks. #1233
- Service: Added ability to configure virtual network IP ranges. #1268
- Deployment: Added
flake8
to the deployment process to align with rest of the Python codebase linting. #1286 - Service: Added custom extensions to enable Microsoft Security Monitoring extensions. #1184
- CLI: Added
--readonly_inputs
option to thelibfuzzer basic
template. #1247
- CLI: Increased the default verbosity of destructive CLI commands. #1264
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1239, #1240, #1236, #1238, #1245, #1246, #1252, #1253, #1254, #1257, #1261, #1262, #1276, #1278
- Deployment: Fixed deployment in some regions by specifying widely-supported versions of Application Insights resources. #1291
- Deployment: Fixed an issue with multi-tenant deployment caused by a mismatch between the identifier used to configure the app registration and value used to authenticate the CLI client. #1270
- Service: Fixed
scaleset proxy reset
to reset all proxies in specified region. #1275 - CLI: Temporarily ignore type errors from
azure-storage-blob
due to invalid Python type signatures. #1258
- CLI/Deployment/Service: Move to using
api://
for AAD Application "identifier URIs". Pre-3.0 clients will not be able to connect to newer instances. (BREAKING CHANGE) #1243 - Agent/Supervisor/Proxy: Redact device, IP, and machine name in runtime statistics reported to Microsoft via Application Insights. #1242
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1232, #1230, #1228, #1229, #1231, #1242.
- CLI: Fixed an issue printing results that include
SecretData
. #1223
- Agent: Added
machine_id
configuration value expansion for all tasks. #1217, #1216
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1215, #1214, #1213, #1211, #1218, #1219
- Deployment: Fixed the example deployment rule to include the required Azure Storage Queue support. #1207
- CLI: Fixed an issue printing results that include
set
,datetime
, orNone
. #1208, #1221
- CLI/Service: The Azure VM SKU used for proxies is now configurable via
onefuzz instance_config
. #1128 - CLI: Added
onefuzz status pool
command to give status information for a pool. #1170
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1152, #1155, #1156, #1157, #1158, #1163, #1164, #1165, #1166, #1176, #1177, #1178, #1179, #1181, #1182, #1183, #1185, #1186, #1191, #1198, #1199, #1200, #1201, #1202, #1203, #1204, #1205
- Agent: Changed
azcopy
calls to always retry when source files are modified mid-copy. #1196 - Agent: Continued development related to upcoming features. #1146
- Agent: SAS URLs are now redacted in logged
azcopy
failures. #1194 - CLI: Include the number of VMs used per-task in
onefuzz status top
. #1169 - Deployment: Application credentials created during deployment are no longer logged. #1172
- Deployment: Clarify logging when retrying AAD interactions. #1173
- Deployment: Replaced custom Azure Storage Queue creation with ARM templates. #1193
- Service: The validity period for SAS URLs is now back-dated to avoid time synchronization issues. #1195
- Deployment: Invalid preauthorized application references are removed during application registration. #1175
- Service: Fixed an issue logging node status. #1160
- Supervisor: Added recording of STDOUT and STDERR of the supervisor to file. #1109
- CLI/Service/Agent: Supervisor tasks can now optionally have a managed coverage container. #1123
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1151, #1149, #1145, #1134, #1135, #1137, #1133, #1138, #1132, #1140
- Service: Enabled testing of the Azure Devops work item rendering. #1144
- Agent: Continued development related to upcoming features. #1142
- CLI: No longer retry service API requests that fail with service-level errors. #1129
- Agent/Supervisor/Proxy: Addressed multiple new
cargo-clippy
warnings. #1125 - CLI/Service: Updated third-party Python dependencies. #1124
- Service: Fixed an issue with incomplete authorization in multi-tenant deployments. CVE-2021-37705 #1153
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1116
- Service: Fixed an error when replacing notifications for a container. #1115
- Service: Fixed Python 3.9 compatibility issues. #1117
- Agent/Supervisor/Proxy: Addressed multiple new
cargo-clippy
warnings. #1118
- Agent: Fixed an issue with the "Premium" storage account utilities. #1111
- Agent: Addressed a rate-limiting issue when using
azcopy
from a large number of VMs with numbers cores. #1112
- Service: PII is now removed from Jobs, Tasks, and Repros after 18 months. #1051
- Service: Unused notifications are now removed after 18 months. #1051
- Service: SignalR events are routed through an Azure Storage Queue to prevent SignalR outages from impacting the entire service. #1100, #1102
- Service: Functionality used prior to 1.0.0 for assigning tasks to VMs rather than Pools is no longer supported. #1105
- Service: The
coverage
andgeneric_generator
tasks now verify{input}
is used intarget_env
ortarget_options
. #1106
- Service: Fixed an issue reimaging old nodes with
debug_keep_node
set. #1103 - Service: Fixed an issue authenticating to Azure services. #1099
- Service: Fixed an issue preventing Pools and Scalesets set to
shutdown
from being set tohalt
. #1104
- CLI: Added the ability to remove existing container notifications upon creating a notification integration. #1084
- CLI/Documentation: Added an example
generic_analysis
task that demonstrates collecting LLVM source-based coverage. #1072 - Supervisor: Added service-interaction resiliency for node commands. #1098
- Agent/Supervisor/Proxy: Addressed multiple new
cargo-clippy
warnings. #1089 - Agent: Added more context to errors in generator tasks. #1094
- Agent: Added support for ASAN runtime identification of format string bugs. #1093
- Agent: Added verification that
{input}
is provided to the application under test viatarget_env
ortarget_options
. #1097 - Agent: Continued development related to upcoming features. #1090, #1091
- CLI/Service: Updated multiple first-party and third-party Python dependencies. #1086
- CLI: Changed job templates to replace existing notifications for the unique report container. #1084
- Service: Added more context to Azure DevOps errors. #1082
- Service: Notification secrets are now deleted from Azure KeyVault upon notification deletion. #1085
- Agent: Fixed an issue logging ASAN output upon ASAN log parse errors. #1092
- Agent: Fixed issues handling non-UTF8 output from applications under test. #1088
- Agent: Batch processing results are now saved after every 10 executions. #1076
- Service: Optimized
file_added
event queueing by avoiding unnecessary Azure queries. #1075 - Agent: Optimized directory change monitoring. #1078
- Supervisor: Optimized agent monitoring. #1080
- CLI: Fixed an issue handling long-running requests. #1068
- CLI/Service: Fixed an issue related to upcoming features. #1067
- CLI: Fixed an issue handling
target_options
for libFuzzer jobs. #1066
- Supervisor: Added a
panic
handler to record supervisor failures. #1062
- Agent: Added more context to file upload errors. #1063
- CLI: Made errors locating
azcopy
more clear. #1061
- Service: Fixed an issue where long-lived VM scaleset instances could get reimaged with out-of-date VM setup scripts. #1060
- Service: Fixed an issue where VM setup script updates were not always pushed. #1059
- Service: Fixed an issue detecting and reimaging failed nodes. #1054
- Service: Fixed an issue with the supervisor restarting too quickly. #1055
- Agent: Added
minimized_stack_function_lines
andminimized_stack_function_lines_sha256
to crash reports. #993 - CLI/Service: Added
timestamp
toNotification
objects. #1043 - Service: Added the scaleset_resize_scheduled event. #1047
- Service: Added
pool_id
toNode
objects. #1049
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1040, #1052
- CLI/Deployment/Service: Updated multiple first-party and third-party Python dependencies. #922, #1045
- CLI/Service: Moved to using Pydantic built-in size validation for types. #1048
- Service: Continued development related to upcoming features. #1046, #1050
- CLI: Fixed an issue handling column sorting in
onefuzz status top
. #1037 - Service: Fixed an issue adding SSH keys to Windows VMs. #1038
- CLI/Service: Added instance configuration that can be managed via
onefuzz instance_config
. #1010 - Service: Added automatic retry for Azure Devops notifications. #1026
- CLI/Service: Added validation to GitHub Issues integration configuration. #1019
- Agent/Supervisor/Proxy: Moved to
rustls
to enable running the Agent and Supervisor on Ubuntu 20.04. #1029 - Agent: Continued development related to upcoming features. #1016
- Agent: Fixed an issue handling invalid data during coverage collection. #1032
- Agent: Fixed retry logic on coverage recording failures #1033
- Service: Fixed an issue preventing deletion or reimaging of nodes in some cases. #1023
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #1018, #1009, #1004
- Service: Tasks running on nodes without recent heartbeats are now marked as failed due to heartbeat issues. #1015
- Service: Updated multiple first-party Python dependencies. #1012
- Agent: Fixed an issue where
libfuzzer_fuzz
tasks on Windows that found crashes too rapidly were unable recover handles. #1002 - Agent: Fixed an issue with the regression tasks after using the
onefuzz debug notification
commands. #1011 - Deployment: Fixed a configuration issue reducing log retention durations. #1007
- Service: Fixed an issue creating GitHub Issues notifications. #1008
- Service: Fixed an issue handling reimaging nodes that took an excessive amount of time. #1005
- Service: Update node and task-related log messages to ease debugging. #988
- Agent: Changed the log level for
azcopy
retry notification toDEBUG
. #986 - Agent: Updated stack minimization regular expressions from
libclusterfuzz
. #992 - Agent: Added more context to synchronized directory errors. #995
- Deployment: Reduced the Application Insights log retention duration to 30 days. #997
- Agent: Improved tracking of threads during win32 debugging. #1000
- Agent: Fixed an issue using relative paths with synchronized directories. #996
- Service: Fixed an issue creating GitHub Issues notifications #990
- CLI/Service: Fixed an issue handling
Union
fields in theonefuzztypes
library #982 - Service: Fixed an issue handling manually-resized scalesets #984
- CLI: Added
onefuzz debug job rerun
command. #960
- Agent: Added more context to coverage recording errors. #979
- Agent: The coverage task now retries an input in the case of coverage recording failure. #978
- Service: Nodes with the
debug_keep_node
flag will now be reimaged once the node is 7 days old. #968 - Service: Updates to scalesets can now be requested while the node is in the
resize
state. #969
- Service: Fixed an issue when reimaging nodes that previously failed to reimage as expected. #970
- Service: Fixed an issue when resizing scalesets that exceed Azure VM quotas. #967
- Supervisor: Fixed an issue with refreshing service authentication tokens. #976
- Agent: Added a new
coverage
task that enables coverage analysis for both uninstrumented and Sancov targets on Linux and Windows. #763
- Agent: Improved performance of the libFuzzer fuzzing tasks. #941
- CLI: Changed the
libfuzzer basic
job template to use the newcoverage
task. #763 - Deployment: Added automatic retry when authorizing newly-created applications during deployment. #959
- Supervisor: Simplified the service coordination logic and added increased context upon failure. #963
- Agent/Supervisor: Added azcopy log recording upon azcopy failure. #945
- CLI: Added
onefuzz jobs containers delete
command. #949 - CLI: Added
onefuzz jobs containers download
command. #953
- Agent/Service: Agents scheduled to shut down no longer wait for work prior to shutting down. #940
- Agent/Supervisor/Proxy: Updated multiple third-party Rust dependencies. #942
- Agent: Continued deveopment related to upcoming features. #937, #929, #919
- CLI: Message details are now always shown in
onefuzz status top
. #933 - CLI: Renamed template helper methods for uploading task setup files. #926
- Contrib: Updated multiple third-party Python dependencies. #950
- Service: Tasks that are stopped without ever having started are now marked as failed. #935
- Supervisor: Added increased context when recording supervisor failures. #931
- CLI/Service: Worked around a third-party dependency issue in handling Python Unions in Events. #939
- Deployment: Fixed an authentication issue during deployment. #947, #954
- Deployment: Fixed an issue limiting application creation logs. #952
- Service: Fixed an issue deleting nodes with expired heartbeats. #930
- Service: Fixed an issue deleting nonexistent containers. #948
- Service: Fixed an issue deleting proxies. #932
- Service: Fixed an issue that prevented automatic migration of notification secrets to Azure KeyVault in some cases. #936
- Supervisor: Fixed an issue adding multiple SSH keys to Windows VMs. #928
- Agent: Added
setup_dir
configuration value expansion for generator tasks. #901 - CLI: Enable specifying alternate tenant configuration via command line arguments. #900
- CLI/Service: Proxy status is now available via
onefuzz scaleset_proxy list
command. #905
- Deployment: Moved to using Microsoft Graph
User.Read
rather than Azure AD Graph. #894 - Service: Tasks are now stopped on nodes before task related storage queues are deleted. #801
- Proxy: Proxies are automatically deployed and always available based on regions with active fuzzing scalesets. #839, #908, #907, #909, #904
- CLI: Added explanations to errors generated when parsing arguments whose values are key/value pairs. #910, #911
- Agent: Continued development related to upcoming features. #913, #918
- Service: Updated first-party Python libraries #903
- Documentation: Added descriptions for the Azure AD entities used by OneFuzz. #896
- Service: Added the scaleset_state_updated event. #882
- Agent/Supervisor/Proxy: Addressed multiple new
cargo-clippy
warnings. #884 - Agent/Supervisor/Proxy: Updated and removed third-party Rust dependencies. #892, #873, #865
- Service: Improved the Python typing signatures used in the service. #881
- Service: Updated multiple first-party and third-party Python libraries. #893, #889, #866, #885, #861, #890
- Supervisor: The supervisor now includes the full error context upon failure. #879
- Service: Cleaned up scaleset update logs. #880
- Agent: Continued development related to upcoming features. #874, #868, #864
- SDK/CLI: Replaced Python based directory uploading with
azcopy sync
. #878
- Service/Supervisor: Fixed an issue shrinking scalesets where idle nodes would not shut down as expected. #866
- Deployment: Fixed an issue deploying to non-Microsoft single-tenant instances. #872, #898
- Deployment: Added ability to only deploy RBAC rsources. #818
- Agent: Continued development related to upcoming features. #855, #858
- Agent: Fixed issue where directory monitoring would fail due to
azcopy
temporary files. #859 - Service: Fixed issue where scalesets could get stuck trying to resize if also manually deleted. #860
- Agent: Added context to errors generated during configuration value expansion. #835.
- CLI/Service: Added messages awaiting processing for a node to the node status API. #836
- Agent: Continued development related to upcoming features. #844, #852, #850, #843, #837, #838, #844
- Agent/Proxy/Supervisor : Updated multiple third-party Rust dependencies. #842, #826, #829
- Service/Contrib: Updated multiple Python dependencies. #828, #827, #823, #822, #821, #847
- Service: Resetting nodes no longer requires waiting for the node to acknowledge the shutdown in some cases. #834
- Supervisor: Fixed an issue introduced in 2.14.0 that sometimes prevents nodes from stopping processing tasks. #833
- Service: Fixed an issue related to Azure Storage Queues being deleted while in use. #832
- Deployment: Fixed an issue where the CLI client application role was not assigned during deployment. #825
- Contrib: Added a sample GitHub Actions workflow and an Azure DevOps Pipeline to demonstrate deploying OneFuzz jobs using CICD. #778
- CLI/Service: Added creation timestamps to
Job
,Node
,Pool
,Scaleset
,Repro
,Task
, andTaskEvent
records returned by the service. #796, #805, #804 - Agent/Proxy/Supervisor: Added additional context to web request failures to assist in debugging issues. #798
- Service: Added task configuration to the crash_reported and regression_reported events. #793
- Agent: The full error context is now logged upon task failure. #802
- CLI: The
libfuzzer-dotnet
template no longer defaults to failing the task if the fuzzer exits with a non-zero status but no crash artifact. #807 - Agent/Proxy/Supervisor: Updated multiple Rust dependencies. #800
- Service: When multiple failures are reported for a given task, only the first failure is recorded. #797
- Agent: Continued development related to upcoming features. #820, #816, #790, #809, #812, #811, #810, #794, #799, #779
- Deployment: Added missing actions to the example Custom Azure Role for deployment. #808
- Service: Fixed an issue in scaleset creation with incompatible VM SKUs and VM Images. #803
- Service: Fixed an issue removing user identity information from logging to user instances. #795
- Deployment: Allow specifying the Azure subscription to use for deployment, instead of always using the default #774
- Agent/Supervisor: Added automatic retry when executing
azcopy
. #701 - Service: When task setup fails, the error that caused the setup failure is now included in the Task error message. #781
- Agent: The
libfuzzer-fuzz
task no longer queries the full local system status when only reporting process status. #784 - Agent: The
libfuzzer-fuzz
task now limits the stderr collected to the last 1024 lines for potential failure reporting. #785 - Agent: The
libfuzzer-fuzz
task now summarizes the executions per second and iteration counts from all of the workers on each VM. #786 - Agent: The
libfuzzer-coverage
task no longer removes the initial copy of inputs. #788 - Agent: Debugger scripts for extracting libFuzzer coverage are now embedded in the agent. #783
- Agent: Continued development related to upcoming features. #787, #776, #663
- CLI: Fixed issue relating to line endings in the
libfuzzer-qemu
job template setup script. #782 - Service: Fixed backward compatibility issue in ephemeral disk support when creating scalesets. #780
- Deployment: Fixed issue in multi-tenant deployment support. #773
- Agent: LibFuzzer tasks now include a verification step that verifies the fuzzer can test a small number of seeds at the start of the task. #752
- Integration Tests: Added verification that no errors are logged to Application Insights during testing. #700
- Agent/Supervisor/Service/Deployment: Added support for multi-tenant authentication. #746
- CLI/Service: Added support for Ephemeral OS Disks. #461, #761
- Agent: Continued development related to upcoming features. #765, #762, #754, #756, #750, #744, #753
- Contrib: Updated multiple python dependencies. #764
- CLI/Agent: LibFuzzer fuzzing tasks no longer default to failing the task if the fuzzer exits with a non-zero status but no crash artifact. #748
- Agent/Proxy/Supervisor: Fixed issues prevent HTTPS retries. #766
- Agent/Service/Proxy/Supervisor: Fixed logging and telemetry from the agent. #769
- Agent/Proxy/Supervisor: Fixed issues preventing heartbeats. #749
- Agent: Continued log simplification and clarification. #736, #740, #742
- Agent: Prevent invalid queue messages from being ignored. #731
- Agent: Separated module and symbol names for Windows debugger-based crash reports. #723
- Deployment/Agent: Updated AFL++ to 3.11c. #728
- CLI/Deployment: Updated Python dependencies. #721
- Agent: Updated stack minimization regular expressions from ClusterFuzz. #722
- Service: Removed user's identity information from logging to user instances. #724, #725
- Agent: Continued development related to upcoming features. #699, #729, #733, #735, #738, #739
- Deployment: Worked around a race condition in service principal creation. #716
- Agent: Dotfiles are now ignored in libFuzzer-related directories. #741
- Agent/CLI/Service: Added regression testing tasks, including enabling git bisect using OneFuzz. #664, #691
- Agent/CLI/Service: Added call stack minimization using a Rust port of ClusterFuzz stack trace parsing. #591, #705, #706, #707, #714, #715, #719
- CLI: Added
onefuzz privacy_statement
command, which displays OneFuzz's privacy statement. #695 - Agent: Added installation of the
x86
andx86_64
Visual Studio C++ redistributable runtimes on Windows nodes. #686
- Agent/Proxy/Supervisor: Changed web request retry logic to include the underlying failure upon giving up retrying a request. #696
- Supervisor: Added automatic web request retry logic when communicating to the service. #704
- CLI/Service: Updated Python dependencies. #698, #687
- Supervisor: Clarified log message when the supervisor unexpectedly exits. #685
- Proxy: Simplified service communication logic. #683
- Proxy: Increased log verbosity on proxy failure. #702
- Agent: Increased setup script timestamp resolution. #709
- Agent: Continued development related to an upcoming feature. #508, #688, #703, #710, #711
- Agent: Fixed support for libFuzzer targets that use shared objects or DLLs from the setup container. #680, #681, #682, #689, #713
- Contrib: Added sample Webhook Service #666
- Agent: Add OneFuzz version and Software role to telemetry #586
- Agent: Add multiple telemetry data types for the upcoming functionality #619
- Agent: Added
input_file_sha256
to configuration value expansion. #641 - Agent: Added
job_id
to Task Heartbeat #646 - Service: Added task information to job_stopped events #648
- Service: task_stopped and task_failed now trigger once the task has stopped instead of upon entering the
stopping
state. #651 - CLI: Authentication tokens are saved upon successful login rather than on program exit. #665
- Service: If a task with dependent tasks fails, all of the dependent tasks are marked as failed. #650
- Agent: Fixed PC address in crash report backtraces. #658
- Service: Upon task completion, if all of the tasks in the associated job are completed, the job is marked as stopped. #649
- Deployment/Agent: Updated AFL++ to 3.11c. #675
- Agent/Proxy/Supervisor: Changed web request retry logic to always retry any request that fails, regardless of why the request failed. #674
- Agent: Downloading files from task queues will now automatically retry on failure. #676
- Service: User information is now stripped from Events before being logged to Application Insights. #661
- Service: Handle exception related to manually deleted scalesets #672
- Agent: Fixed Rust lifetime issues exposed by an update to Rust regex library #671
- CLI: Added support for Aarch64 libFuzzer targets using the QEMU user space emulator. #600
- Build: Added CodeQL pipeline. #617
- Service: Added node and task heartbeat events. #621
- Agent: Clarified batch-processing logs. #622
- Agent/Proxy: Updated multiple rust dependencies. #624
- Service/CLI/Contrib: Updated multiple python dependencies. #607, #608, #610, #611, #612, #625, #626, #630, #640
- Service: Update task configuration to verify
target_exe
is a canonicalized relative path. #613 - Deployment/Agent: Updated AFL++ to 3.10c. #609
- Deployment: Clarify application password creation succeeded after earlier failures. #629
- Service: VM passwords are no longer set on Linux VMs. #620
- Service: Clarify source of task failures when notification integration marks a task as failed. #635
- Agent/Proxy/Supervisor: Fixed web request retry logic when handling operating system level errors. #623
- Service: Handle exceptions when creating scalesets fail due to Azure VM quota issues. #614
- CLI: Added
onefuzz containers files download_dir
to enable downloading the contents of a container. #598 - Agent: Added
microsoft_telemetry_key
andinstance_telemetry_key
and expanded the availabilityreports_dir
in configuration value expansion. #561 - Agent/Service: Added
job_id
to agent-based heartbeats. #594 - Agent/Proxy/Supervisor: Added additional context to errors during Storage Queue and service interactions to improve debugging. #601
- Agent/Proxy/Supervisor: Renamed the Application Insights token names used for telemetry to
microsoft_telemetry_key
andinstance_telemetry_key
and the function that gated telemetry sharing tocan_share_with_microsoft
to make the telemetry implementation easier to understand. #587 - Deployment: Updated multiple Python dependencies. #596
- Service: Updated multiple Python dependencies. Addresses potential security issue CVE-2020-28493 #595
- Service: Don't let nodes run new tasks if they are part of a scaleset or pool that is scheduled to be shut down. #583
- Service: Fixed the queries used to identify nodes running outdated OneFuzz releases. #597
- Agent: Fixed an issue that would stop an agent or supervisor from performing work if an HTTPS request has failed in certain conditions. #603
- Agent: Fixed an issue that would stop a task if the task printed a significant amount of data to stdout or stderr. #588
- Deployment: Address deployment failures relating to cross-region Azure Active Directory resource creation delays. #585
- Service: Jobs that do not start within 30 days are automatically stopped. #565
- Service: Debug proxies now use ports 28000 through 32000. #552
- Service: Events now include the instance name and unique identifier. #577
- Service: All task related Events now include the task configuration. #580
- Service: Errors generated during report crash report notification due to invalid jobs or tasks now include the reason for the error. #576
- CLI: Namespaced containers for coverage used in job templates now include
build
andplatform
in addition toproject
andname
. #572 - Service: User triggered node reimaging no longer waits for confirmation from the node prior to starting the reimage process. #566
- Service: Fixed an error condition when users recreate a container immediately after deleting it. #582
- Service: Fixed an issue when one task on a node ended, the node was reimaged regardless of the state of other tasks running on the node. #567
- CLI: Added the ability to poll task status until the tasks have started to managed templates using
--wait_for_running
. #532 - CLI: Added a libfuzzer-dotnet support. #535
- Agent: Added
crashes_account
andcrashes_container
to configuration value expansion. #551 - CLI: Added
onefuzz status job
andonefuzz status project
to provide a user-friendly job status. #550
- Agent: Logs and local telemetry from the agent now include the role (
agent
orsupervisor
) in recorded events. #527 - Agent: Clarified the errors generated when libFuzzer coverage extraction fails #554
- Service: Handled
SkuNotAvailable
errors from Azure when creating scalesets. #557 - Agent/Proxy: Updated multiple third-party Rust libraries. Addresses potential security issue RUSTSEC-2021-0023. #548
- Agent: Verifying LibFuzzer targets at the start of a task using
-help=1
now happens prior to sending heartbeats. #528
- Service: Fixed issue related to Azure Functions not always providing the JWT token via Authorization headers. #531
- CLI: Fixed
--wait_for_running
in job templates. #530 - Deployment: Fixed a log error by setting the default SignalR transport used by Azure Functions. #525
- Agent: Fixed LibFuzzer coverage collection when instrumenting DLLs loaded at runtime. #519
- Service: Fixed issue where the cached Azure Identity was not being used. #526
- Service: Fixed log message related to identifying secondary corpus instances. #524
- Service: Handle scaleset nodes that never register, such as nodes with instance-specific setup script failures. #518
- Agent: Added stdout/stderr logging and clarifying context during failures to the
generic_analysis
task. #522 - Agent/Service/Proxy: Clarify log messages from the scaleset proxy. #520
- Agent/Proxy: Update multiple third-party Rust libraries. #517
- Agent: Fixed potential race condition when single stepping when debugging during the
generic_crash_reporter
andgeneric_generator
tasks running on Windows. #440
- Service: Clarify log messages when the service and agent versions mismatch. #510
- Service: Scalesets and Nodes are now updated in a consistent order during scheduled updates. #512
- CLI/Service: Expanded the use of Primitive data types that provide data validation. #514
- Service: Fixed an error generated when scalesets scheduled for deletion had configurations updated. #511
- Service: Fixed an issue where scaleset configurations were updated too frequently. #511
- Proxy: The logs from the proxy manager logged to Application Insights. #502
- Agent: Updated the web request retry logic to retry requests upon connection refused errors. #506
- Service: Improved the performance of shutting down pools. #503
- Service: Updated
azure-mgmt-compute
Python dependency. #499
- Proxy: Fixed an issue in the proxy heartbeats that caused proxy VMs to be reset after 10 minutes. #502
- Agent: Fixed an issue that broke libFuzzer based crash reporting that was introduced 2.1.1. #505
- Agent: Added Rust Clippy static analysis to CICD. #490
- CLI/Service: Added Bandit static analysis to CICD. #491
- Service: Fixed an issue where scalesets could get in a state that would stop updating configurations. #489
- Agent: Added
job_id
andtask_id
to configuration value expansion. #481 - Agent: Broadened the availability of
tools_dir
to configuration value expansion. #480 - Agent: Added clarifying context to command errors. #466
- CLI/Service/Agent: Supervisor can now be fully self-contained fuzzing tasks, no longer requiring
target_exe
. Additionally, supervisor tasks can now optionally have managed report containers. #474 - Service: Managed nodes that are unused beyond 7 days are automatically reimaged to ensure OS patch levels are maintained. #476
- CLI/Service: Updated the default Windows VM image to
MicrosoftWindowsDesktop:Windows-10:20h2-pro:latest
. Existing scalesets will not be impacted by this change, only newly created scalesets using the default image. #469
- Agent: New inputs discovered by supervisor tasks are now saved to the
inputs
container. #484 - CLI: The license is now properly set in the python package metadata. #472
- Agent: Failure to download files via HTTP from queues now results in a failure, rather than the HTTP error being interpreted as the requested file. #485
- Deployment: Fixed error when checking if the default CLI application exists. #488
- Agent: Added clarifying context to file system errors. #423
- CLI/Service: Significantly expanded the events available for webhooks. #394
- Agent: Added
{setup_dir}
to configuration value expansion #417 - Agent: Added
{tools_dir}
configuration value expansion to{supervisor_options}
and{supervisor_env}
#444
- CLI/Service: Migrated
onefuzz status top
to use Webhook Events. (BREAKING CHANGE) #394 - CLI/Service: New notification secrets, such as ADO tokens, are managed in Azure KeyVault and are no longer accessible to the user once created. (BREAKING CHANGE) #326, #389
- CLI/Service: Updated multiple Python dependencies. #426, #427, #430
- Agent: Fixed triggering condition for new unique report events #422
- Deployment: Mitigate issues related to deployments within conditional access policy scenarios. #447
- Agent: Fixed an issue where unused nodes would stop requesting new work. #459
- Service: Fixed dead node cleanup. #458
- Service: Fixed an issue logging excessively large stdout/stderr from tasks. #460
- Service: Added support for sharding corpus storage accounts using "Premium" storage accounts for improved IOPs. #334
- CLI/Service/Agent: Added the ability to optionally colocate multiple compatible tasks on a single machine. The coverage and crash reporting tasks in the LibFuzzer template make use of this functionality by default. #402
- CLI: Added
onefuzz debug log tail
which enables continuously following Application Insights query results. #401 - CLI/Agent: Support verifying LibFuzzer targets at the start of a task using
-help=1
, which will enable identifying non-functional LibFuzzer targets. #381 - CLI/Agent: Support specifying whether to log a warning or fail the task when a LibFuzzer target exits with a non-zero status code (without also generating a crashing input). #381
- Agent: The stdout and stderr for the supervisors and generators are now logged to Application Insights. #400
- Service: Enabled per-Scaleset SSH keys on Windows VMs, similar to existing Linux support, enabling
onefuzz debug node ssh
to both Windows and Linux nodes. #390 - Agent: Support ASAN odr-violation results. #380
- CLI/Service/Agent: Added the ability add SSH keys to nodes within scalesets. #441
- CLI: Added support for multi-tenant authentication. #346
- Service: Updating outdated nodes is now limited to 500 nodes at a time. #397
- Service: Restrict agent from accessing API endpoints not specific to the agent. #404
- Service: Increased Azure Functions runtime timeout to 15 minutes. #384
- Deployment/Agent: Updated AFL++ to 3.00c. #393
- Agent: Added randomized initial jitter to agent heartbeats, which reduce API query storms when launching large number of nodes concurrently. #387
- CLI/Agent: Add support to verify LibFuzzer targets execute correctly at the start of a task using
-help=1
. #381 - Service: Re-enable API endpoint used by
onefuzz nodes update
. #412 - Agent: Addressed a race condition in LibFuzzer coverage analysis without initial seeds. #403
- Agent: Prevent supervisor that fatally exits from processing additional new tasks. #378
- Agent: Address issues handling LibFuzzer targets that produce non-UTF8 output to stderr. #379
- CLI: Added
libfuzzer merge
job template, which enables running performing libFuzzer input minimization as a batch operation. #282 - CLI/Service: Added the instance-specific Application Insights telemetry key to
onefuzz info get
, which will enable logging to the instance specific application insights from the SDK. #353 - Agent: Added support for parsing ASAN
CHECK failed
entries, which can occur during large amounts of memory corruption. #358 - Agent/Service: Added support for parsing the ASAN "scariness" score and description when
print_scariness=1
inASAN_OPTIONS
. #359
- Agent: Mark tasks as failed if the application under test generates an ASAN log file that the agent is unable to parse. #351
- Agent: Updated the
libfuzzer_merge
task to merge pre-existing inputs in a single pass. #282 - CLI: Clarified the error messages when prefix-expansion fails. #342
- Service: Rendered
pydantic
models as JSON when logging to preventerror=None
from showing up in the error logs. #350 - Deployment: Pinned the version of pyOpenssl to the version used by multiple Azure libraries. #348
- CLI/Service: (PREVIEW FEATURE) Multiple updates to job template management. #354, #360, #361
- Agent: Fixed issue preventing the supervisor from notifying the service on some state changes. #337
- Deployment: Fixed a regression in retrying password creation during deployment #338
- Deployment: Fixed uploading tools when rolling back deployments. #347
- CLI/Service: Added Service-Managed Job Templates as a preview feature. Enable via
onefuzz config --enable_feature job_templates
. #226 - Service/agent: Added internal support for unmanaged nodes. This paves the way for bring your own compute for fuzzing. #318
- CLI: Added
onefuzz debug
subcommands to simplify coverage and fuzzing performance for libFuzzer jobs from Application Insights. #325 - Service: Information about the user responsible for creating jobs and repro VMs is now associated with the Job and Repro VMs. #327
- Deployment:
deploy.py
now automatically retries on failure when deploying the Azure Function App. #330
- Service: Address multiple minor issues previously hidden by function decorators used for caching. #322
- Agent: Fixed libFuzzer coverage support for internal builds of MSVC #324
- Agent: Address issue preventing instance-wide setup scripts from executing in some cases. #331
- CLI/Service: Added Event-based webhooks. #296
- Service: Information about the user responsible for creating tasks is now associated with the tasks (this information is available in the task related event webhooks). #303
- Contrib: Azure Devops deployment pipeline uses the
--upgrade
feature added in 1.7.0. #304
- Service: Fixed setting
target_workers
, used to configure the number of concurrent libFuzzer workers within a task. #305
- Deployment:
deploy.py
now takes--upgrade
to enable simplify upgrading deployments. For now, this skips assignment of the managed identity role which only needs to be done on installation. #271 - CLI: Added Application Insights debug CLI. See
onefuzz debug logs
#281 - CLI: Added unique_inputs to the default container types for
onefuzz reset --containers
andonefuzz containers reset
. #290 - CLI: Added
onefuzz debug node
to enable debugging a node in a scaleset without having to specify the scaleset. #298
- Service: When shutting down an individual scaleset, all of the nodes in the scaleset are now marked for shutdown. #252
- Service: The scaleset service principal IDs are now cached as part of the respective Scaleset object #255
- Service: The association from nodes that ran a task are now kept until the node is reimaged, enabling easily connecting to the node that ran a task after task completion. #273
- Deployment: Pinned
urllib3
version due to an incompatible new release #292 - CLI: Removed calls to
containers.list
, significantly improving job template creation performance. #289 - Service: No longer use HTTP 404 response codes during agent registration. #287
- Agent: Heartbeats are now only sent as part of the execution loop. #283
- Service: Refactored handlers for agent events, including much more detailed logging. #261
- Deployment: Prevent users from enabling public access ton containers. #300
- Service: Fixed libfuzzer_merge tasks #240
- Service: Fixed an issue where scheduled tasks waiting in the queue for longer than 7 days would never get scheduled. #259
- Service: Removed stale Node references from scalesets #275
- Service: The service now auto-scales the number of Azure Functions instances as needed #238
- CLI/Service/Agent: Added the ability to configure ensemble synchronization interval (including disabling ensemble altogether) #229
- Contrib: Added sample Azure Devops pipeline to maintain instances of OneFuzz #233
- Deployment: Added utility to create CLI application registrations #236
- Deployment/Service/Agent: Added a per-instance uniquely generated UUID to telemetry (see docs/telemetry.md for more information) #245
- CLI: The CLI now internally caches container authorization tokens #224
- Service: Moved to using user-assigned managed identities for Scalesets #219
- Agent: Added stdout to azcopy error logs #247
- Service: Increased function timeouts to 5 minutes
- CLI/Service: Added the ability to prevent a VM from getting reset in order to debug tasks #201
- SDK: Add examples directory to the python package #216
- Agent: Added connection resiliency via automatic retry (with back-off) throughout the agent #153
- Deployment: Added the ability to log the application passwords during registration #214
- Agent: LibFuzzer Coverage metrics are now reported after the batch processing phase #218
- Deployment: Added a utility to assign scalesets to roles #185
- Contrib: Added a utility to automate deployment of new releases of OneFuzz via Azure Devops pipelines #208
- Agent: Addressed a race condition syncing input seeds #204
- Agent: Instead of ignoring all access violations during libFuzzer coverage processing, stop on second-chance access violations #210
- Agent: During libFuzzer coverage, disable default symbol paths unless
_NT_SYMBOL_PATH
is set viatarget_env
. #222
- CLI: Added
onefuzz containers reset
to delete containers by type en masse. #198, #202 - Agent: Added missing approved telemetry as to tool names & crash report identification. #203
- Service: Enabled log sampling at the service at 20 items per second. #174
- Service: Fixed multiple bugs in the service, including an exception due to invalid format string proxy or repro VM creation #206
- CLI: Fixed incorrect resetting of granularly selected components introduced in 1.3.3 #193
- Service: Fixed rate-limiting issues requesting MSI and Storage Account tokens #195
- Service: Moved the SDK to use the same
pydantic
models as the service in request generation #191 - Service: Improved performance of container validation #196
- Service: Fixed exception generated when deleting repro & proxy VMs #188
- Service/Agent: Non-functional nodes are now automatically re-imaged #154, #164, #30
- CLI: Added more granularity for the
onefuzz reset
sub-command #161, #182 - Deployment/Agent: Now includes AFL++ #7
- Deployment/Agent: Now includes Radamsa for Windows #143
- CLI: The
onefuzz status top
TUI now allows filtering based on job ID, project, or name #152
- Service: Nodes no longer have to wait for the scaleset to finish setup before being able to fuzz #144
- Agent: Agent now only notifies the service about its current state upon state change #175
- Service: Task error messages now limit the stdout and stderr to the last 4096 bytes #170
- Service: Replaced custom queue based event loop with timers #160, #159
- Agent: Uploads that fail now report the failure earlier #166
- Agent: All timers now include automatic jitter to reduce request storms #180
- Agent: Ensemble container synchronization has been unified to once every 60 seconds (plus jitter) #180
- Agent: Upon agent failure, it will no longer incorrectly re-register and request new work. #150, #146
- Deployment: Addressed an issue with nested exceptions triggered during a failed deployment #172
- Deployment: Addressed incompatible prerequisite library warnings during deployment #167
- Testing: Added rust based libFuzzer in the end-to-end integration tests #132
- Agent: Always parse stderr when generating crash reports for LibFuzzer instead of using
ASAN_OPTIONS=log_path
, which fixes crash reports from non-sanitizer based crashes. #131 - Deployment: Added data-migration script to fix notifications for pre-release installs #135
- Agent: Crash reports for LibFuzzer now attempts to parse stderr in addition to
ASAN_OPTIONS=log_path
. This enables crash reporting of go-fuzz based binaries. #127 - Deployment: During deployment, App Insights logs can be configured to automatically export logs to the
app-insights
container in instance specificfunc
storage account. #102
- Agent: Reduced logs sent from the agent #125
- Service: Scalesets now use multiple placement groups, allowing a scaleset to grow to 1000 nodes (or 600 if using a custom image). #121
- Deployment: Support deploying additional platforms (such as OSX). #126
- Service: Fixed typing error in sorting TaskEvent. #129
- CLI/Service: Added creating and updating GitHub Issues based on crash reports. #110
- Agent: LibFuzzer fuzzing that exits with a non-zero exit code without a resulting crashing input now mark the task as failed. #108
- Service: The automatic variable
repro_cmd
used in crash report notifications now includes '--endpoint URL' to reduce friction for users with multiple OneFuzz instances. #113
- Agent/Service: Added the ability to automatically re-image nodes that are out-of-date #35
- Deployment: Added data-migration scripts for pre-release installs #12
- SDK/CLI: Added more
onefuzz debug
sub-commands to support debugging tasks #95 - Agent: Added machine_id and version to log messages #94
- Service: Errors in creating Azure Devops work items from reports now mark the task as failed #77
- Service: The nodes executing a task are now included when fetching details for a task (such as
onefuzz tasks get $TASKID
) #54 - SDK: Added example Azure Functions that uses the SDK #56
- SDK/CLI: Added the ability to execute debugger commands automatically during
repro
#39 - CLI: Added documentation of CLI sub-command arguments (used to describe
afl_container
in AFL templates #10 - Agent: Added
ONEFUZZ_TARGET_SETUP_PATH
environment variable that indicates the path to the task specific setup container on the fuzzing nodes #15 - CICD: Use sccache to speed up build times #47
- SDK: Added end-to-end integration test script to verify full fuzzing pipelines #46
- Documentation: Added definitions for pool, node, and scaleset #17
- Agent/Service: Refactored state management for on-VM supervisors #96
- Agent: Added 'done' semaphore to the agent to prevent agent from fetching additional work once the node should be reset. #86
- Agent: Nodes now sleep longer between checking for new work. #78
- Agent: The task execution clock is now started once the task is in the 'setting up' state #82
- Service: Drastically reduced logs sent to App Insights from third-party libraries #63
- Agent/Service: Added the ability to upgrade out-of-date VMs upon requesting new tasking #35
- CICD: Non-release builds now include the GIT hash in the versions and
localchanges
if built locally with un-committed code. #58 - Agent: Command replacements now use absolute rather than relative paths. #22
- CLI: Fixed issue using
onefuzz template stop
which would improperly stop jobs that had the same 'name' but different 'project' values. #97 - Agent: Fixed input marker expansion (used in AFL templates related to handling
@@
). #87 - Service: Errors generated after the task shutdown has started are ignored. #83
- Agent: Instance specific tools now download and run on windows nodes as expected #81
- CLI: Using
--wait_for_running
inonefuzz template
jobs now properly waits for tasks to launch before exiting #84 - Service: Handled more Azure Devops notification errors #80
- Agent: WSearch service is now properly disabled by default on Windows VMs #67
- Service: Properly deletes
repro
VMs #36 - Agent: Supervisor now flushes logs to Application Insights upon exit #21
- Agent: Task specific setup script failures now properly get recorded as a failed task and trigger the node to be re-imaged #24
- Initial public release