Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch and log error thrown while parsing dates from metadata #2186

Merged
merged 1 commit into from
Dec 14, 2023

Conversation

come-nc
Copy link
Contributor

@come-nc come-nc commented Dec 11, 2023

Fix #2185

@come-nc come-nc added the 3. to review Waiting for reviews label Dec 11, 2023
@come-nc come-nc added this to the Nextcloud 29 milestone Dec 11, 2023
@come-nc come-nc self-assigned this Dec 11, 2023
Signed-off-by: Côme Chilliet <come.chilliet@nextcloud.com>
@come-nc come-nc force-pushed the fix/catch-and-log-date-errors branch from a6f16e7 to 5c9bb54 Compare December 11, 2023 10:40
@come-nc
Copy link
Contributor Author

come-nc commented Dec 11, 2023

@MichaIng Could you test this fix?

@come-nc come-nc requested review from artonge and MichaIng December 11, 2023 10:55
@come-nc
Copy link
Contributor Author

come-nc commented Dec 11, 2023

/backport to stable28

@backportbot-nextcloud backportbot-nextcloud bot added the backport-request Pending backport by the backport-bot label Dec 11, 2023
Copy link
Member

@MichaIng MichaIng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works. I get now the expected warnings and see the affected files, indeed all from the same camera. Here one example:

{
  "reqId": "IyHWBbnfR5zAjQH8yezr",
  "level": 2,
  "time": "2023-12-11T13:45:20+01:00",
  "remoteAddr": "",
  "user": "--",
  "app": "photos",
  "method": "",
  "url": "--",
  "message": "Failed to parse date 0000:00:00 00:00:00 for file /user/files/path/to/image.jpg",
  "userAgent": "--",
  "version": "28.0.0.10",
  "exception": {
    "Exception": "ValueError",
    "Message": "Epoch doesn't fit in a PHP integer",
    "Code": 0,
    "Trace": [
      {
        "file": "/var/www/nextcloud/apps/photos/lib/Listener/OriginalDateTimeMetadataProvider.php",
        "line": 53,
        "function": "getTimestamp",
        "class": "DateTime",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/apps/photos/lib/Listener/OriginalDateTimeMetadataProvider.php",
        "line": 90,
        "function": "dateToTimestamp",
        "class": "OCA\\Photos\\Listener\\OriginalDateTimeMetadataProvider",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/EventDispatcher/ServiceEventListener.php",
        "line": 86,
        "function": "handle",
        "class": "OCA\\Photos\\Listener\\OriginalDateTimeMetadataProvider",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
        "line": 230,
        "function": "__invoke",
        "class": "OC\\EventDispatcher\\ServiceEventListener",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/symfony/event-dispatcher/EventDispatcher.php",
        "line": 59,
        "function": "callListeners",
        "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/EventDispatcher/EventDispatcher.php",
        "line": 94,
        "function": "dispatch",
        "class": "Symfony\\Component\\EventDispatcher\\EventDispatcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/EventDispatcher/EventDispatcher.php",
        "line": 106,
        "function": "dispatch",
        "class": "OC\\EventDispatcher\\EventDispatcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/FilesMetadata/FilesMetadataManager.php",
        "line": 115,
        "function": "dispatchTyped",
        "class": "OC\\EventDispatcher\\EventDispatcher",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/core/BackgroundJobs/GenerateMetadataJob.php",
        "line": 105,
        "function": "refreshMetadata",
        "class": "OC\\FilesMetadata\\FilesMetadataManager",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/core/BackgroundJobs/GenerateMetadataJob.php",
        "line": 101,
        "function": "scanFolder",
        "class": "OC\\Core\\BackgroundJobs\\GenerateMetadataJob",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/core/BackgroundJobs/GenerateMetadataJob.php",
        "line": 101,
        "function": "scanFolder",
        "class": "OC\\Core\\BackgroundJobs\\GenerateMetadataJob",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/core/BackgroundJobs/GenerateMetadataJob.php",
        "line": 90,
        "function": "scanFolder",
        "class": "OC\\Core\\BackgroundJobs\\GenerateMetadataJob",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/core/BackgroundJobs/GenerateMetadataJob.php",
        "line": 81,
        "function": "scanFilesForUser",
        "class": "OC\\Core\\BackgroundJobs\\GenerateMetadataJob",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/public/BackgroundJob/Job.php",
        "line": 81,
        "function": "run",
        "class": "OC\\Core\\BackgroundJobs\\GenerateMetadataJob",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
        "line": 102,
        "function": "start",
        "class": "OCP\\BackgroundJob\\Job",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
        "line": 92,
        "function": "start",
        "class": "OCP\\BackgroundJob\\TimedJob",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/core/Command/Background/Job.php",
        "line": 89,
        "function": "execute",
        "class": "OCP\\BackgroundJob\\TimedJob",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/symfony/console/Command/Command.php",
        "line": 298,
        "function": "execute",
        "class": "OC\\Core\\Command\\Background\\Job",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/symfony/console/Application.php",
        "line": 1040,
        "function": "run",
        "class": "Symfony\\Component\\Console\\Command\\Command",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/symfony/console/Application.php",
        "line": 301,
        "function": "doRunCommand",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/3rdparty/symfony/console/Application.php",
        "line": 171,
        "function": "doRun",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/lib/private/Console/Application.php",
        "line": 206,
        "function": "run",
        "class": "Symfony\\Component\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/console.php",
        "line": 100,
        "function": "run",
        "class": "OC\\Console\\Application",
        "type": "->"
      },
      {
        "file": "/var/www/nextcloud/occ",
        "line": 11,
        "args": [
          "/var/www/nextcloud/console.php"
        ],
        "function": "require_once"
      }
    ],
    "File": "/var/www/nextcloud/apps/photos/lib/Listener/OriginalDateTimeMetadataProvider.php",
    "Line": 53,
    "message": "Failed to parse date {date} for file {path}",
    "exception": {},
    "CustomMessage": "Failed to parse date 0000:00:00 00:00:00 for file /user/files/path/to/image.jpg"
  }
}

I guess the date 0000:00:00 00:00:00 is expected in case it failed completely. Although, exiftool also shows this. Is it probably an issue, because the year 0 is too far before 1970?

Modify Date                     : 0000:00:00 00:00:00
Y Cb Cr Positioning             : Co-sited
Exposure Time                   : 1/250
F Number                        : 2.7
Exposure Program                : Program AE
ISO                             : 200
Sensitivity Type                : Standard Output Sensitivity
Exif Version                    : 0230
Date/Time Original              : 0000:00:00 00:00:00
Create Date                     : 0000:00:00 00:00:00

Should be fine to see this warning once for every affected file. I guess that I saw this eery day was just because the background job crashed at the same point every time?

@come-nc
Copy link
Contributor Author

come-nc commented Dec 12, 2023

@MichaIng Thank you for testing!

On 64bits for this all 0 dates I get a timestamp of -62169987208. On 32bits I get the DateRangeException.

@artonge Should we add a special case for this date value? Even on 64bits it won’t help people to have all their pictures dated back to year 0. But it’s hard to know if this issue is very specific to this camera or if it’s common to have this value.

@artonge
Copy link
Collaborator

artonge commented Dec 12, 2023

I think we can just ignore the 0 value, as t does not make sense. I think it's better to have nothing than a wrong value and we'll default to the mtime anyway.

@artonge
Copy link
Collaborator

artonge commented Dec 13, 2023

Ok to merge @come-nc?

@come-nc
Copy link
Contributor Author

come-nc commented Dec 14, 2023

Ok to merge @come-nc?

We should still add something to ignore all-zero timestamps, but in the mean time we should merge this yeah.

@come-nc come-nc merged commit 3ee3ae4 into master Dec 14, 2023
29 checks passed
@come-nc come-nc deleted the fix/catch-and-log-date-errors branch December 14, 2023 13:23
@backportbot-nextcloud backportbot-nextcloud bot removed the backport-request Pending backport by the backport-bot label Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Nextcloud 28] New metadata background job can trigger error on 32-bit systems
3 participants