Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error in latest version #63

Open
vvhor opened this issue Mar 21, 2024 · 11 comments
Open

error in latest version #63

vvhor opened this issue Mar 21, 2024 · 11 comments

Comments

@vvhor
Copy link

vvhor commented Mar 21, 2024

Hello,

I'm trying the latest version but I got this error

thread 'main' panicked at src\api_connection.rs:59:33:
Could not parse API login reply: error decoding response body: missing field `access_token` at line 1 column 623
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

the request is:

./OfficeAuditLogCollector.exe --tenant-id "xxxxxxx" --client-id "xxxxx" --secret-key "xxxxx" --config config.yaml

config file:

collect:
  skipKnownLogs: True
  workingDir: ./
  maxThreads: 50
  globalTimeout: 5
  retries: 3
  hoursToCollect: 168
  contentTypes:
    Audit.General: True
    Audit.AzureActiveDirectory: True
    Audit.Exchange: True
    Audit.SharePoint: True
    DLP.All: True 
output:
  file:
    path: 'output.csv'
    separateByContentType: True
    separator: ';'

I'm using the client on window system

@ddbnl
Copy link
Owner

ddbnl commented Mar 22, 2024

Hi @vvhor,

Have you successfully used the older version(s) with the same app registration before, or are you trying for the first time? If it's the first time you could check if the API permissions are properly set, and if auditing is enabled for the tenant (this might take a while to sync after enabling it). Both these actions are described in README.md.

If it was already working before then we'll have to figure where it's coming from. I'm currently working on a new release with improved logging, so once that's out in the coming days I will link it here. Then hopefully we can see more with the increased logs.

@vvhor
Copy link
Author

vvhor commented Mar 22, 2024

Hi,

I've used it in previous version on other tenant.
With this tenant it's the first time.

I've followed all of the steps in the READEME some days ago

@vvhor
Copy link
Author

vvhor commented Mar 22, 2024

I'm currently working on a new release with improved logging, so once that's out in the coming days I will link it here. Then hopefully we can see more with the increased logs.

did you have an estimation for this release?

@ddbnl
Copy link
Owner

ddbnl commented Mar 22, 2024

I have released the new version with fixed logging and also extended logging, hopefully we'll be able to capture the error:

https://github.com/ddbnl/office365-audit-log-collector/releases/tag/v2.3.1

Make sure to also enable logging in the config:

log:
  path: './log.txt'
  debug: True

If you get it working consider disabling debug again, it's very noisy. Let me know what it does for you.

@vvhor
Copy link
Author

vvhor commented Mar 22, 2024

Hi,

many thanks. Now the log is very helpful. I'll do some test and let you know

@ddbnl
Copy link
Owner

ddbnl commented Mar 24, 2024

As a heads up, there's a new release that added an interactice interface that can be used for testing. If you have the new release, you can run the command as you did before, but add the '--interactive' command line parameter. This allows you to test the connection and immediately see the logs for any errors.

https://github.com/ddbnl/office365-audit-log-collector/releases/tag/v2.3.2

Screenshot

@vvhor
Copy link
Author

vvhor commented Mar 25, 2024

Hello,

I'm now having different errors in "Run Collector":

[00:00:01.339] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.339] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-20T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.339] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.339] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-19T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.341] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.341] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-18T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.349] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.349] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-21T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.350] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.350] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2024-03-20T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.350] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.350] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2024-03-21T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.352] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.352] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-22T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.352] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.352] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2024-03-19T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.356] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.356] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2024-03-18T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.358] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.358] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2024-03-19T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.398] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.398] (5f60) WARN   Retry blob 1 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-18T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.398] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.398] (5f60) WARN   Retry blob 1 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-19T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.398] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.398] (5f60) WARN   Retry blob 1 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-20T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.398] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.398] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2024-03-21T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.401] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.401] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2024-03-23T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.403] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.403] (5f60) WARN   Retry blob 1 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-21T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.407] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.407] (5f60) WARN   Retry blob 1 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2024-03-20T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.408] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.408] (5f60) WARN   Retry blob 1 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-22T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.409] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.409] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2024-03-18T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.411] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.411] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2024-03-23T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.411] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.411] (5f60) WARN   Retry blob 1 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2024-03-18T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.411] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.411] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-23T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.411] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.411] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2024-03-23T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.412] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.412] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2024-03-19T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.412] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.412] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2024-03-18T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.412] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.412] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2024-03-20T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.414] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.414] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2024-03-18T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.416] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.416] (5f60) WARN   Retry blob 1 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2024-03-19T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.416] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.416] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.SharePoint&startTime=2024-03-21T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.417] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.417] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.AzureActiveDirectory&startTime=2024-03-22T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.418] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.418] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2024-03-21T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.420] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.420] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2024-03-22T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.420] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.420] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=DLP.All&startTime=2024-03-19T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.420] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.420] (5f60) WARN   Retry blob 2 https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.Exchange&startTime=2024-03-20T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123
[00:00:01.422] (52b4) INFO   Blobs found: 0
Blobs successful: 0
Blobs failed: 0
Blobs retried: 34
Logs saved: 0

[00:00:01.422] (2e58) WARN   Err getting blob JSON error decoding response body: invalid type: map, expected a sequence at line 1 column 0
[00:00:01.424] (2e58) ERROR  Err getting blob response error sending request for url (https://manage.office.com/api/v1.0/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/activity/feed/subscriptions/content?contentType=Audit.General&startTime=2024-03-20T11:38:02Z&endTime=2024-03-25T11:38:02Z&PublisherIdentifier=12345678-1234-1234-1234-123456789123): connection error: Either the application has not called WSAStartup, or WSAStartup failed. (os error 10093)
[00:00:01.424] (2e58) ERROR  Could not resend failed blob, dropping it: send failed because receiver is gone

Non error in "Test API Connection"

@ddbnl
Copy link
Owner

ddbnl commented Mar 25, 2024

That's odd, so far I'm not able to reproduce. Best we can do is improve logging. I've added the full output of the JSON response as a debug log, in the section where you are receiving the error. This should give us the full response you are getting from the API. Can you run it again with the latest release, and enabling debug logging?

Also, just to ensure you are not being rate limited, could you use a publisher ID? For the ID you can just use your tenant ID again. This will isolate your requests to avoid rate limiting as much as possible. You can the executable like before, but adding "--publisher-id %tenant-id%"

https://github.com/ddbnl/office365-audit-log-collector/releases/tag/v2.3.3

@vvhor
Copy link
Author

vvhor commented Mar 27, 2024

PS. The new version not write the log in interactive mode.

in attach the full log
log.txt

@vvhor
Copy link
Author

vvhor commented Mar 28, 2024

Hi,

just to ask if you have any news about the errors..

@vvhor
Copy link
Author

vvhor commented Apr 4, 2024

can I do any other test to help you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants