Items appear in trash and can not be restored

[cwallace]

The bug

I have several hundred items listed under the Trash section in Immich. When attempting to restore these items they appear to restore, but after refreshing the page they reappear. The info button next to the "Trash" says that I have 0 pictures and 0 videos in Trash.

Screen.Recording.2024-09-29.at.11.00.57.AM.mp4

The OS that Immich Server is running on

Ubuntu 22.04

Version of Immich Server

v1.116.2

Version of Immich Mobile App

v1.116.2

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

name: immich
services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: nvenc # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cuda # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:2d1463258f2764328496376f5d965f20c6a67f66ea2b06dc42af351f75248792
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/storage/pictures
# The location where your database files are stored
DB_DATA_LOCATION=/mnt/storage/pictures/database

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=America/Denver

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=Hunter2

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. Delete items
2. Navigate to trash
3. Attempt to restore
4. Refresh the page

Relevant log output

No response

Additional information

No response

[AnimeshKotka]

I have the issue and have started working on it.
cc: @jrasm91 @alextran1502

[aviv926]

I tried to reproduce your problem, without success.
Can you check if you can reproduce the problem in the demo version?
https://demo.immich.app/trash

[cwallace]

I tried to reproduce your problem, without success. Can you check if you can reproduce the problem in the demo version? https://demo.immich.app/trash

So far I have not been able to reproduce this in the demo version. There are a couple of items that I'm unable to test in the demo environment however:

1. Upgrading Immich with items in the Trash
2. Deleting items using duplicate detection

[jrasm91]

I'm guessing this is specific to external library assets that are "offline". They are in the trash for now while we work on making a dedicated page for them, which will be home to the new "system integrity" page.

[cwallace]

I'm guessing this is specific to external library assets that are "offline". They are in the trash for now while we work on making a dedicated page for them, which will be home to the new "system integrity" page.

I don't believe I have any external library assets, especially none that are offline. This is a relatively new instance (created last weekend) and all of my media has been added via immich-go or the immich mobile application. Everything exists on my zfs pool (/mnt/storage/) including media and database.

[jrasm91]

Interesting. The only thing I have seen that could cause this is a timezone issue where "now" is an incorrect date and the database is in another timezone or something. Can you confirm the timezone of the immich server?

[cwallace]

Interesting. The only thing I have seen that could cause this is a timezone issue where "now" is an incorrect date and the database is in another timezone or something. Can you confirm the timezone of the immich server?

Wow! No idea how you determined that, but after uploading most (all?) of my images, I noticed that I had forgot to set the correct timezone on the host VM and ran dpkg-reconfigure tzdata.

I've tried restarting the immich stack as well as setting the timezone back to UTC and restarting the stack, but I'm still having the issue with these stuck in the trash. Any suggestions on how to resolve the issue?

edit: I gave up and simply set deletedAt to null for all assets, which at least appears to have solved having these photos being stuck in the trash. Not sure if you want to leave this open to solve the bigger problem of handling timezone changes, but I'll leave that up to you.

I do have TZ=America/Denver in my .env.

Just checked the docker containers:
immich-server is America/Denver
immich-machine-learning is America/Denver
immich-postgres is UTC
immich-redis is UTC

The host machine is America/Denver

[piyush97]

I'm facing the same issue, any solution?

[jrasm91]

You should be able to select the assets individually or hit "select all" and then delete them. I believe the issue is specifically with the "empty trash" button when the database is in a different timezone than the server.

[dpad]

Just wanted to report that I have the same (or similar) issue, but seemingly without the timezone factor?

For me, there were many assets in trash that were not counted in the Trash statistics (showed as "0 photos, 0 videos") and could not be deleted by the "empty trash" button. I had to delete them by selecting all and clicking the delete button instead.

I think that all my containers are running in UTC though:

ubuntu@immich:~$ date
Sun Oct  6 06:20:59 UTC 2024
ubuntu@immich:~$ docker exec immich_server date
Sun Oct  6 06:21:14 UTC 2024
ubuntu@immich:~$ docker exec immich_redis date
Sun Oct  6 06:21:20 UTC 2024
ubuntu@immich:~$ docker exec immich_postgres date
Sun Oct  6 06:21:25 AM UTC 2024
ubuntu@immich:~$ docker exec immich_postgres psql -U postgres -c "SHOW TIMEZONE;"
 TimeZone
----------
 Etc/UTC
(1 row)

In case it helps, I only noticed this after upgrading today from 1.115.0 to 1.117.0 (though it may have been present before, I just didn't try to empty the trash until now).

[alextran1502]

@dpad For additional information, are you using external library?

[dpad]

@alextran1502 Thanks for checking. No, I'm not using an external library.

[dpad]

I'll expand on what I did today in case it helps.

1. I upgraded using docker compose from 1.115.0 -> 1.117.0.
2. After the upgrade, I deleted a couple of items.
3. These items seemingly showed up in the Trash statistics (e.g. "1 video, 5 photos").
4. However, the Trash contained these items I just deleted, plus many hundreds more that I had deleted ~2 weeks ago.
5. When I clicked on the "Empty Trash" button, it deleted the items from today, but left all the other items from 2 weeks ago.
6. When I checked the Trash statistics again, it said "0 videos, 0 photos". I refreshed, tried "Empty Trash" again and it said something like "removed 0 items". All the items deleted 2 weeks ago were still in the Trash.
7. I found this issue and saw the recommendation to select all the items and click delete manually, so I did that. In total, it deleted ~600 items from the Trash.

Based on the above, I would suspect it may be something due to the upgrade from 1.115.0 -> 1.117.0. It's possible that this issue was present before, but I only noticed and tried emptying the trash today after the upgrade.

[dpad]

@alextran1502 By the way, I have database backups going back at least several weeks. I'd prefer not to share them directly, but I'm happy to run some queries on them if you need.

[zionun]

Hello, same problem here. Trash info says "0 video, 0 photo" but there's 185 items in it. They are not in an external library and I do not want to delete them. When I restore them, they soon reappear in the trash. The Info section for each photo says that the resource is no longer on the disk, but the file is exactly at the path shown in the error panel and permissions are the same of all other thousands of pictures that are correctly shown in the timeline. My folder is on a NAS and I tried to mount it both with smb and nfs.