Adding redirects to target scope

[BretMcDanel]

If I scan https://foo/bar/baz burpa will add https://foo/bar/ to the target scope. However, if the redirect is to https://foo/ this will be out of scope.

I did not see an easy way to update the target scope. The best I could find is using _ names (eg _api) to try to force it. This does not seem to be a good way to do this.

Is this a feature that needs to be added? If so I can do that, but if there is already a way (either with a JSON config or via a call) I would rather use that.

[tristanlatr]

If I scan https://foo/bar/baz burpa will add https://foo/bar/ to the target scope. However, if the redirect is to https://foo/ this will be out of scope.

This is by design, we don't want to add the root of the website to the scope.
You can probably use JSON config file for this.

[BretMcDanel]

This is by design, we don't want to add the root of the website to the scope. You can probably use JSON config file for this.

Do you know how/where? I have been trying several different things to no avail. Documentation on the JSON configuration is weak at best.

Its not a problem in the GUI but when I copy the GUI config snippet into the working config it does not seem to override the api/SCAN defined scope. When I add it as another URL to scan I get 1 fail for out of scope and 1 that seems to work but that is less than ideal for reporting purposes.

[tristanlatr]

I don't have more insights unfortunately.
But what you can do is to use burpa internals and directly talk to the APIs and add your urls to the scope like this.

Example:

python3 -m burpa _api request include_scope --url=<URL> - json

Tell me if it works.

[BretMcDanel]

I am using burpa as a module, Burpa.scan() at least does not like the unexpected keyword "include_scope"

Would you be opposed to a patch that allows someone to add to the scope? As I currently envision it, it would be similar to the "exclude" parameter in use.

It seems to me that redirects are extremely common, though maybe that is my particular use case. Take the following:
https://tinyurl.com/mr3njrxt -> http://example.com/
If a scan is to be done on the tinyurl it will fail (example.com is out of scope so BurpScan is unable to connect to any seed URL).

[tristanlatr]

I am using burpa as a module

Then use

b = Burpa(...)
b._api.include('<url>')

Would you be opposed to a patch that allows someone to add to the scope?

Since you use burpa as a library, you don't have to patch the code actually.

It seems to me that redirects are extremely common

Yes indeed, but burp suite should be run on the actual target website. Same behaviour applies to wpscan: when it hit a redirect, it aborts the scan and ask the user to scan the actual URL.

[BretMcDanel]

That was my first thought, and referenced in my first post. It does not work. I get an INFO message indicating it is added to the scope but then it gets overridden when the scan occurs and I get an "example.com is out of scope, no seed url" error.

This is why I asked if you were opposed to a patch to make it work. Whether or not is gets used for this use case it allows the user to define the scope as they need.

[BretMcDanel]

To be more clear with specifics:

from burpa import Burpa

api_url="127.0.0.1"
api_port=8091
api_key=""
report_dir="./"
config="Crawl and Audit - Balanced"

b = Burpa(api_url="127.0.0.1", api_port=api_port, new_api_key=api_key, verbose=False, no_banner=True)
b._api.include("http://example.com")
b.scan("https://tinyurl.com/mr3njrxt", config=config, report_output_dir=report_dir, report_type='XML')

---

INFO - Running Burp Suite version 2023.9.1
INFO - http://example.com has been included to the scope
INFO - https://tinyurl.com/mr3njrxt has been included to the scope
INFO - Using scan configuration name(s): Crawl and Audit - Balanced
INFO - Initiating unauthenticated scan...
INFO - https://tinyurl.com/mr3njrxt Added to the scan queue, ID 3
INFO - Scan started
INFO - Scan status: crawling
INFO - Scan status: paused
INFO - Scan completed
INFO - Scan metrics for https://tinyurl.com/mr3njrxt :
INFO - CURRENT_URL =
INFO - CRAWL_REQUESTS_MADE = 1
INFO - CRAWL_NETWORK_ERRORS = 0
INFO - CRAWL_UNIQUE_LOCATIONS_VISITED = 0
INFO - CRAWL_REQUESTS_QUEUED = 0
INFO - AUDIT_QUEUE_ITEMS_COMPLETED = 0
INFO - AUDIT_QUEUE_ITEMS_WAITING = 0
INFO - AUDIT_REQUESTS_MADE = 0
INFO - AUDIT_NETWORK_ERRORS = 0
INFO - ISSUE_EVENTS = 0
INFO - CRAWL_AND_AUDIT_CAPTION = Paused task due to: Could not connect to any seed URLs.
INFO - CRAWL_AND_AUDIT_PROGRESS = 0
INFO - TOTAL_ELAPSED_TIME = 2
INFO - No issue could be found for the target https://tinyurl.com/mr3njrxt
Traceback (most recent call last):
File "...\burptest.py", line 11, in
b.scan("https://tinyurl.com/mr3njrxt", config=config, report_output_dir=report_dir, report_type='XML')
File "...\AppData\Roaming\Python\Python310\site-packages\burpa_burpa.py", line 342, in scan
raise BurpaError(f"Scan aborted - {record.target_url} : {caption}")
burpa._error.BurpaError: Scan aborted - https://tinyurl.com/mr3njrxt : Paused task due to: Could not connect to any seed URLs.

---

If I run where headless.mode=false I can get more information from BurpSuite and it tells me that example.com is not in the scope and that is the problem (see image).

---

If verbose=True I can see the following:
DEBUG - Constructing API call from template: post, /scan, {
"urls" : ["https://tinyurl.com/mr3njrxt"],
"scope": {
"include": [{"rule": "https://tinyurl.com/mr3njrxt", "type":"SimpleScopeDef"}],
"exclude": []
},
"scan_configurations": [{"name": "Crawl and Audit - Balanced", "type": "NamedConfiguration"}]
}

D

This leads me to believe that the include scope is being overridden for that scan, thus my offer to write a patch to resolve this. I really have not tried outside of this use case, which is the initial URL is a redirect to out of scope so I am uncertain whether this is the deciding factor in it being this way or not. Some applications set cookies then redirect to real content which complicates "just set it to the final destination" attempts. Whether or not those apps have good design, that does not negate their existence.

[tristanlatr]

Thanks for the detailed report.

So this means that my initial code is faulty as well. And yes I'm open to review an PR to fix that and provide an explicit include parameter.
But this means we nee to change the template of the API calls because it only expect one include string at this time.

Thanks for your investigations.

[tristanlatr]

Hello @BretMcDanel,

Are you still interested into working on this issue ?

Thanks