'packageRules > schedule' isn't behaving as expected

[denver-HJS]

Which Renovate are you using?

Renovate Open Source CLI

Which platform are you using?

Bitbucket Cloud

Have you checked the logs? Don't forget to include them if relevant

These log statements appear in a time-ordered sequence (sorry for the string escaping). It looks like it tries to fetch Bitbucket issues but is unable to authenticate, however I didn't think this would be causing any problems related to the scheduling.

{"id":"48280aca-0a01-4973-b232-5c8cf810f693","level":"info","message":" WARN: Error finding issues (repository=acme/acme-page-driver-wdio)\n       \"err\": {\n         \"name\": \"HTTPError\",\n         \"hostname\": \"api.bitbucket.org\",\n         \"method\": \"GET\",\n         \"path\": \"/2.0/repositories/acme/acme-page-driver-wdio/issues?q=title%3D%22Action%20Required%3A%20Fix%20Renovate%20Configuration%22%20AND%20(state%20%3D%20%22new%22%20OR%20state%20%3D%20%22open%22)%20AND%20reporter.username%3D%22acme_renovate_bot_sa%22\",\n         \"protocol\": \"https:\",\n         \"url\": \"https://api.bitbucket.org/2.0/repositories/acme/acme-page-driver-wdio/issues?q=title%3D%22Action%20Required%3A%20Fix%20Renovate%20Configuration%22%20AND%20(state%20%3D%20%22new%22%20OR%20state%20%3D%20%22open%22)%20AND%20reporter.username%3D%22acme_renovate_bot_sa%22\",\n         \"gotOptions\": {\n           \"path\": \"/2.0/repositories/acme/acme-page-driver-wdio/issues?q=title%3D%22Action%20Required%3A%20Fix%20Renovate%20Configuration%22%20AND%20(state%20%3D%20%22new%22%20OR%20state%20%3D%20%22open%22)%20AND%20reporter.username%3D%22acme_renovate_bot_sa%22\",\n           \"protocol\": \"https:\",\n           \"hostname\": \"api.bitbucket.org\",\n           \"hash\": \"\",\n           \"search\": \"?q=title%3D%22Action%20Required%3A%20Fix%20Renovate%20Configuration%22%20AND%20(state%20%3D%20%22new%22%20OR%20state%20%3D%20%22open%22)%20AND%20reporter.username%3D%22renovate_bot_sa%22\",\n           \"pathname\": \"/2.0/repositories/acme/acme-page-driver-wdio/issues\",\n           \"href\": \"https://api.bitbucket.org/2.0/repositories/acme/acme-page-driver-wdio/issues?q=title%3D%22Action%20Required%3A%20Fix%20Renovate%20Configuration%22%20AND%20(state%20%3D%20%22new%22%20OR%20state%20%3D%20%22open%22)%20AND%20reporter.username%3D%22acme_renovate_bot_sa%22\",\n           \"retry\": {\n             \"methods\": {},\n             \"statusCodes\": {},\n             \"errorCodes\": {},\n             \"maxRetryAfter\": 60000\n           },\n           \"headers\": {\n             \"user-agent\": \"https://github.com/renovatebot/renovate\",\n             \"accept\": \"application/json\",\n             \"accept-encoding\": \"gzip, deflate\"\n           },\n           \"hooks\": {\n             \"beforeRequest\": [],\n             \"beforeRedirect\": [null],\n             \"beforeRetry\": [],\n             \"afterResponse\": [],\n             \"beforeError\": [],\n             \"init\": []\n           },\n           \"decompress\": true,\n           \"throwHttpErrors\": true,\n           \"followRedirect\": true,\n           \"stream\": false,\n           \"form\": false,\n           \"json\": true,\n           \"cache\": false,\n           \"useElectronNet\": false,\n           \"method\": \"GET\",\n           \"hostType\": \"bitbucket\",\n           \"baseUrl\": \"https://api.bitbucket.org/\",\n           \"auth\": \"acme_renovate_bot_sa:***********\",\n           \"gotTimeout\": {\"request\": 60000}\n         },\n         \"statusCode\": 403,\n         \"statusMessage\": \"Forbidden\",\n         \"headers\": {\n           \"server\": \"nginx\",\n           \"vary\": \"Authorization\",\n           \"content-type\": \"application/json; charset=utf-8\",\n           \"x-oauth-scopes\": \"pullrequest:write\",\n           \"x-b3-traceid\": \"0f4b7f4149b53bd7\",\n           \"x-dc-location\": \"ash2\",\n           \"strict-transport-security\": \"max-age=31536000; includeSubDomains; preload\",\n           \"acmee\": \"Wed, 23 Sep 2020 16:52:08 GMT\",\n           \"x-served-by\": \"app-3018\",\n           \"etag\": \"\\\"430e2709edbceff6cefb79aef573a6b1\\\"\",\n           \"x-static-version\": \"2bb6abf92372\",\n           \"x-credential-type\": \"apppassword\",\n           \"x-render-time\": \"0.0213220119476\",\n           \"x-accepted-oauth-scopes\": \"issue\",\n           \"connection\": \"close\",\n           \"x-version\": \"2bb6abf92372\",\n           \"x-request-count\": \"1069\",\n           \"x-frame-options\": \"SAMEORIGIN\",\n           \"content-length\": \"172\"\n         },\n         \"body\": {\n           \"type\": \"error\",\n           \"error\": {\n             \"message\": \"Your credentials lack one or more required privilege scopes.\",\n             \"detail\": {\"granted\": [\"pullrequest:write\"], \"required\": [\"issue\"]}\n           }\n         },\n         \"message\": \"Response code 403 (Forbidden)\",\n         \"stack\": \"HTTPError: Response code 403 (Forbidden)\\n    at EventEmitter.<anonymous> (/app/node_modules/got/source/as-promise.js:74:19)\\n    at runMicrotasks (<anonymous>)\\n    at processTicksAndRejections (internal/process/task_queues.js:97:5)\"\n       }\n","appId":"Unknown","timestamp":"2020-09-23T16:52:08.272Z"}

{"id":"956f8e03-ab31-408e-8f4d-d9e70a43b6da","level":"info","message":" INFO: Dependency extraction complete (repository=acme/acme-page-driver-wdio)\n \"baseBranch\": \"master\",\n \"stats\": {\n \"managers\": {\n \"dockerfile\": {\"fileCount\": 1, \"depCount\": 1},\n \"npm\": {\"fileCount\": 1, \"depCount\": 18}\n },\n \"total\": {\"fileCount\": 2, \"depCount\": 19}\n }\n","appId":"Unknown","timestamp":"2020-09-23T16:51:54.990Z"}

{"id":"8af159d6-dd22-487b-a2c3-6f86cdbb91ab","level":"info","message":" INFO: Repository finished (repository=acme/acme-page-driver-wdio)\n       \"durationMs\": 19808\n","appId":"Unknown","timestamp":"2020-09-23T16:52:08.285Z"}

What would you like to do?

At a high level, our goal is to use a global schedule that opens PRs outside of working hours with the exception of our internal npm packages.

The way we're trying to accomplish this is with the following configuration:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "branchPrefix": "renovate-",
  "hostRules": [
    {
      "domainName": "github.com",
      "token": "<my github token>"
    }
  ],
  "prConcurrentLimit": 6,
  "timezone": "America/Los_Angeles",
  "schedule": [
    "after 10pm every weekday",
    "before 5am every weekday",
    "every weekend"
  ],
  "packageRules": [
    {
      "depTypeList": [ "dependencies" ],
      "rangeStrategy": "replace"
    },
    {
      "depTypeList": [ "devDependencies" ],
      "rangeStrategy": "pin"
    },
    {
      "depTypeList": [ "peerDependencies" ],
      "rangeStrategy": "widen"
    },
    {
      "packagePatterns": [ "@acme" ],
      "groupName": "internal",
      "schedule": [ "at any time" ]
    },
    {
      "packagePatterns": [ "mocha" ],
      "groupName": "mocha"
    },
    ...
    {
      "packagePatterns": [ "aws-sdk" ],
      "schedule": [ "schedule:weekly" ]
    }
  ]
}

However when we publish our internal packages, we don't see PR's opened as expected per the "at any time" rule.

note: we've also tried without placing the rule in an array:

    {
      "packagePatterns": [ "@acme" ],
      "groupName": "internal",
      "schedule": "at any time"
    },

[rarkins]

Can you reproduce this behavior in a public repo on github.com? e.g.

• Config has schedule array at its base
• One package rule with the override schedule for some scoped package (Can be any public one)
• One scoped dependency matching the package rule, one not. Each of them out of date.

The desired outcome with the above should be that the scoped package is PR'd right away and the other not. If you reproduce the problem, then neither is created right away.

[denver-HJS]

Alright I think I've set this up in line with your suggestion, but I had a lot of pre-existing PR's that I declined to keep the dependency versions outdated, so I'm not sure if that will interfere with the Renovate Bot's process?

• Public renovate config
• Public package.json

I guess I should mention our self hosted configuration is using this semver value for the CLI: ^21.33.15

[denver-HJS]

It looks like the Renovate Bot behaved as documented in this case. I assume more PR's will be opened up tonight and we'll lose our "control" in the experiment, so I'm attaching the screenshot here. I'm not sure why two PR's were opened for the Angular monorepo, maybe because it's a predefined monorepo and there's also specificpackageRules defined? Either way, I'm not sure why the CLI in our self-hosted environment is showing different behavior.

I updated our self-hosted process to use the latest version in NPM (23.36.0) so we'll see if that naturally resolves itself. 🤞

[rarkins]

We probably should have made the default one monthly or something.. however it's always possible to rename and delete/close PRs to "reset" things. It's possible that the pre existing grouping for angular has interfered but I suspect it's something else about your setup instead, hence wanting to try to reduce and reproduce.

[denver-HJS]

@rarkins I'm hoping to avoid letting this issue auto-close. Do you have any recommendations for how we can proceed? It looks like there's still only the two pull requests opened in the public repository. I'm not sure if that's because there's been no new versions of the other dependencies published in the last ~4 days or if there's a different reason.

[rarkins]

@denver-HJS is this still a problem for you? And if so then can you reset or refresh your reproduction so that it can be debugged? Please make the default schedule be as delayed as possible so that the reproduction lasts as long as possible.

[denver-HJS]

Thank you for the reminder @rarkins. I've updated the reproduction by closing all open pull requests, decreasing the global frequency to "once quarterly" and leaving Angular packageRules with "at any time" (see renovate.json):

{
...
  "schedule": [
    "every 3 months on the first day of the month"
  ],
  "packageRules": [
    {
      "packagePatterns": [ "@angular" ],
      "groupName": "Angular",
      "schedule": [ "at any time" ]
    },
    ...
}

[denver-HJS]

Just an update here, it seems like this is working just fine in context of using the GitHub bot. Could there be a difference where we're self hosting and running the CLI? I'll try to tinker with the ruleset we're defining when I get a chance to see if it's operator error after all.

[rarkins]

Maybe if you defined rules in your backend config too