GPS geo location wrong: GPSLongitudeRef and GPSLatitudeRef not taken into account

[skycryer]

The bug

So I have it looks like around 1k images that are shown on the wrong place on the map. I checked my Apple Photos and Export data. All have the correct gps data and are shown correct on Apple Photos. But on Immich they are completely off, not just a few miles but a few thousand.

Here is the metadata of the image

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 6.0.0">
   <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
      <rdf:Description rdf:about=""
            xmlns:exif="http://ns.adobe.com/exif/1.0/"
            xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"
            xmlns:dc="http://purl.org/dc/elements/1.1/">
         <exif:GPSLongitude>14.160669499999999</exif:GPSLongitude>
         <exif:GPSLongitudeRef>W</exif:GPSLongitudeRef>
         <exif:GPSHPositioningError>1</exif:GPSHPositioningError>
         <exif:GPSLatitudeRef>N</exif:GPSLatitudeRef>
         <exif:GPSLatitude>28.184478333333335</exif:GPSLatitude>
         <exif:GPSTimeStamp>2024-09-25T09:16:21Z</exif:GPSTimeStamp>
         <photoshop:DateCreated>2019-05-19T12:40:50+01:00</photoshop:DateCreated>
         <dc:subject>
            <rdf:Seq>
               <rdf:li>Elitsa</rdf:li>
            </rdf:Seq>
         </dc:subject>
      </rdf:Description>
   </rdf:RDF>
</x:xmpmeta>

The majority of my images seem to be on the correct location on the map, but this 1k are not. I have no idea why.

The OS that Immich Server is running on

Ubuntu 22.x

Version of Immich Server

v116.2

Version of Immich Mobile App

latest

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

https://github.com/alexbelgium/hassio-addons/tree/master/immich

Your .env content

https://github.com/alexbelgium/hassio-addons/tree/master/immich

Reproduction steps

Add the gps data from the xml file, see above, and you see the image will be located in Lybia, not on Fuerteventrura,

Relevant log output

No response

Additional information

No response

[skycryer]

After digging trough the map view I saw it has way more images of location then I thought.

I did the import to Immich over the cli to also import the xml file. And it looks like it is not using the fields for

GPSLongitudeRef

correct. Because when I post the Lat, LNG Data straight to google maps it also shows wrong location like

28°11'04.1"N 14°09'38.4"E

But it should be West, not East. When I change E to W at the end in google it is the right location. Maybe that helps?

[bo0tzz]

There's actually a TODO comment in the code for this :P

immich/server/src/services/metadata.service.ts

Lines 658 to 662 in 5bcbe77

	private async getGeo(tags: ImmichTags, reverseGeocoding: SystemConfig['reverseGeocoding']) {
	let latitude = validate(tags.GPSLatitude);
	let longitude = validate(tags.GPSLongitude);
	// TODO take ref into account

[skycryer]

So any plan TODO the in the near future :-), makes my map look really weird. And I am sure we have to reimport our data, or will there be an update button for the map data?

[bo0tzz]

It'll be done whenever someone decides to work on it ;) You can re-extract metadata by running the appropriate job from the admin panel, so there will be no need to reimport anything.

[dreadster3]

@skycryer I am kind of a newbie here but maybe this can help.

I was looking into this and seems that the issue might actually be the XMP itself not the code.

I did a few trial and error upload of pictures while modifying the exif data and the exiftool seems to behave correctly by default, so after looking into the XMP, I found this:
https://photo.stackexchange.com/questions/114069/using-exiftool-to-import-date-gps-and-tags-from-xmp-files
https://web.archive.org/web/20180921145139if_/http://www.cipa.jp:80/std/documents/e/DC-010-2017_E.pdf

In the exif metadata XMP specification it shows that the tags GPSLongitudeRef and GPSLatitudeRef are not used

I believe you will have to append what is in GPSLongitudeRef in GPSLongitude and the same for GPSLatitude

[Snowknight26]

The information might not be stored in XMP, it might be in EXIF only.

I imagine the exiftool-vendored correctly reads the data, but whatever's stored in XMP is incorrect (since GPSLatitudeRef/GPSLongitudeRef are unused, meaning GPSLatitude/GPSLongitude should include the position) whereas the EXIF (not in XMP) is correct, but XMP might trump EXIF when reading tags.

Can you provide an example image that exhibits the issue?

Can also run exiftool on an image with the issue with the following arguments to see actual tags/locations: -a -s -G0:1:2:3:4:5:7 "-gps*" -e

[C-Otto]

exiftool-vendored contains a test for this situation (positive number for west longitude):

https://github.com/photostructure/exiftool-vendored.js/blob/ed71b97ac8fa9e27f1590835648b9488b3a0d6d9/src/ReadTask.spec.ts#L98-L108

As you can see, the positive longitude is returned (and not transformed into the negative one), and the result includes a warning. It seems there's a good reason to reject this input instead of fixing it (or "tak[ing] ref into account" as per the TODO).

[C-Otto]

Related:
photostructure/exiftool-vendored.js#165

[aehiggi]

Hi, new contributor here. Thanks for the helpful comments above. I've been investigating this and AFAICT the root cause is that exiftool ignores GPSLatitudeRef and GPSLongitudeRef in XMP data when doing a "normal" read. Thanks @dreadster3.

Regardless of what value the XMP data contains for GPSLongitudeRef ("W" | "West" | "Kermit"), exiftool will always output "East" or "E". This is true of both exiftool-vendored and exiftool cli. According to the exiftool forums this is by design - because the GPSLat/LongRef tags are invalid it will ignore them and determine their values from the values in GPSLatitude and GPSLongitude themselves. These need to be either signed numbers or have a trailing "W" | "E" | "N" | "S". In this case they have neither, so the positive value for GPSLatitude is used to infer "East".

I guess one approach here is to say that the XMP data is invalid and leave as-is, but if this data is created by some popular tool then trying to handle it may be a better option to avoid creating bad experiences for users of that tool.

I've created a PR demonstrating attempting to handle the provided XMP data, using the "-e" switch for exiftool in order to read the values of the GPSLat/LongRef tags "raw" from the XMP file. It does use two reads of the sidecar (the second one being the raw read) which I'm not thrilled about, but I wanted to avoid affecting the values for other tags and just use the raw values for GPSLat/LongRef - if they are valid.

Following that the GPSLat/LongRef tags become available in getGeo() at the TODO line pointed out by @bo0tzz and they can be taken into account when doing the reverse geocode.

[zackpollard]

@aehiggi are there any popular tools you know of that do this? I'm hesitant to include a "fix" that exiftool itself has decided against. We try wherever possible to avoid these kind of custom fixes as they can cause more harm than good. Exiftool vendored also decided against making changes to support this for similar reasons.

[skycryer]

I did an export of apple Photos and generated the xml files where I got that problem of West/East GPS Locations

[aehiggi]

@zackpollard I completely understand but it would be good if Immich could help innocent users who end up with XMP files like this.

An alternative solution could be to enable users to correct the data themselves through the UI. Perhaps an extension to the Change Location dialog adding something like "Swap E <-> W" and "Swap N <-> S", so they can change the hemisphere for one or a batch of images, without affecting the Lat/Long values themselves.

[C-Otto]

I'd urge Apple to fix their code. While that's not done, feel free to fix the files using some kind of tool, possibly as a plugin to Immich (no idea if that's feasible, just an idea). I don't think that Immich should support broken data.

[aehiggi]

Apple Photos users: discussion of this issue and tips on fixing your XMP files over at the apple forums https://discussions.apple.com/thread/255792059?answerId=260820004022#260820004022

[C-Otto]

Great! Then let's wait for Apple to fix their software?

[mceachen]

That's another issue. The location name is derived from the GPS coordinates using https://github.com/photostructure/tz-lookup

tz-lookup should only be used for timezone extraction (and it has data quality issues--see the readme).

You should be using the geolocation feature if you need location metadata: https://photostructure.github.io/exiftool-vendored.js/interfaces/ExifToolOptions.html#geolocation

Back to the topic: it turns out ExifTool has magick that fixes the Apple XMP GPS, but only if geolocation is enabled. The next release of exiftool-vendored will use the hemisphere from GeolocationPosition if available, which may address this issue.

[bo0tzz]

There's a misunderstanding on the location name - it's determined in Immich using data from the geonames database, not with tz-lookup. No issues there ;)