Sync gpgme error: Unusable secret key

[tfoote]

I'm refiling osrf/infrastructure#3 (comment) here so that we can track this.

A somewhat similar thing happened to Melodic over the weekend; see https://discourse.ros.org/t/testing-repository-empty-amd64/20908 for some more of the details.

I'm wondering if it would make sense to have the sync-to-testing jobs run nightly, as a backup in case something failed. That way the time frame of the breakage (if it does occur) would at least be time-limited.

In the failed melodic log noticed that there's actually a real error that we should not be having

13:57:15 gpgme gave error GPGME:54:  Unusable secret key

We should make sure that doesn't happen.

[tfoote]

There was just another instance of this today:

https://build.ros2.org/job/Grel_sync-packages-to-testing_focal_arm64/104/consoleFull

23:29:18 removing 'ros-galactic-urg-node' from 'focal|main|source'..gpgme gave error GPGME:54:  Unusable secret key
23:29:24 ERROR: Could not finish exporting 'focal'!

[clalancette]

There was another instance of this on Melodic as well: https://build.ros.org/job/Mrel_sync-packages-to-testing_bionic_amd64/726/console

23:28:38 removing 'ros-melodic-yosemite-valley' from gpgme gave error GPGME:54:  Unusable secret key
23:28:38 ERROR: Could not finish exporting 'bionic'!

[tfoote]

These last jobs failed within a minute of each other on separate days. Do we have a server maintenance task managing the gpg keys at that time? The earlier case is also just before the hour too.

[nuclearsandwich]

I saw a similar but different GPG error importing the jammy repositories today. https://build.ros2.org/job/import_upstream/357/

I am going to block out some time for this toward the start of next year.

[ijnek]

This error suddenly came up in one of my packages today: https://build.ros2.org/job/Hrel_import-package/4137/consoleFull

This is the job that triggered the Hrel_import_package: https://build.ros2.org/job/Hbin_ujv8_uJv8__rqt_image_overlay_layer__ubuntu_jammy_arm64__binary/4/consoleFull

---

Update

The build passed in the next build "Started by user SYSTEM": https://build.ros2.org/job/Hbin_ujv8_uJv8__rqt_image_overlay_layer__ubuntu_jammy_arm64__binary/5/ without any changes.

[ijnek]

Just caught another instance of this: https://build.ros2.org/job/Rbin_ujv8_uJv8__rcss3d_agent_msgs__ubuntu_jammy_arm64__binary/32/

[tfoote]

We had another instance of this for noetic sync to testing: https://build.ros.org/view/Nbin_uF64/job/Nrel_sync-packages-to-testing_focal_amd64/1134/console

It's likely still happening in the background for individual package syncs but they retry sooner so as not to be noticed.