Random error message "Root resource was present, but now absent" on virtual repository creation

[jmvitali]

Describe the bug
Random error message "Root resource was present, but now absent" on virtual repository creation.

Requirements for and issue

• Random error message "Root resource was present, but now absent" on virtual repository creation
• A fully functioning terraform snippet that can be copy&pasted (no outside files or ENV vars unless that's part of the issue).

   resource "artifactory_virtual_docker_repository" "create_virtual_repo" {
       artifactory_requests_can_retrieve_remote_artifacts = false
       default_deployment_repo                            = "ipp-generic-prod-ipp-nce-docker-prod-toa-nce"
       description                                        = "Description"
       id                                                 = (known after apply)
       includes_pattern                                   = "**/*"
       key                                                = "ipp-generic-prod-ipp-nce-docker-prod-toa"
       notes                                              = "managed by terraform"
       package_type                                       = (known after apply)
       project_environments                               = (known after apply)
       project_key                                        = "default"
       repo_layout_ref                                    = "simple-default"
       repositories                                       = [
           "ipp-generic-prod-ipp-nce-docker-prod-toa-nce",
        ]
       resolve_docker_tags_by_timestamp                   = false
    }

• Your version of artifactory : 8.9.1
• Your version of terraform : 1.6.5

Expected behavior
Virtual repository creation each time.

Thanks a lot in advance for your help.
Best Regards,
JM

[alexhung]

@jmvitali Thanks for the report. Can you share the output of the terraform plan or terraform apply command? i.e. The entire output of the error message.

[jmvitali]

Hello @alexhung,
Thanks to take care on that request.
We reproduce this issue this morning.
Please find the output of the related terraform apply command:

Terraform will perform the following actions:

  # restapi_object.add_repo_properties will be created
  + resource "restapi_object" "add_repo_properties" {
      + api_data        = (known after apply)
      + api_response    = (known after apply)
      + create_response = (known after apply)
      + data            = jsonencode(
            {
              + cleanupAqlLimit = "100000"
              + cleanupDays     = "1095"
              + cleanupDryrun   = "false"
              + cleanupStrategy = "1"
              + iz              = "arti"
              + repoQuota       = "53687091200"
            }
        )
      + id              = (known after apply)
      + object_id       = "testremote-docker-release-arti-nce_set_properties"
      + path            = "/properties?repository=testremote-docker-release-arti-nce"
    }

  # time_sleep.wait_5_seconds will be created
  + resource "time_sleep" "wait_5_seconds" {
      + create_duration = "5s"
      + id              = (known after apply)
    }

  # module.create-remote-repo-docker_blr.artifactory_remote_docker_repository.create_remote_repository will be created
  + resource "artifactory_remote_docker_repository" "create_remote_repository" {
      + allow_any_host_auth                   = false
      + assumed_offline_period_secs           = 300
      + blacked_out                           = false
      + block_mismatching_mime_types          = true
      + block_pushing_schema1                 = (known after apply)
      + bypass_head_requests                  = false
      + cdn_redirect                          = false
      + client_tls_certificate                = (known after apply)
      + description                           = "description"
      + disable_proxy                         = false
      + disable_url_normalization             = false
      + download_direct                       = false
      + enable_cookie_management              = false
      + enable_token_authentication           = (known after apply)
      + external_dependencies_enabled         = false
      + hard_fail                             = false
      + id                                    = (known after apply)
      + includes_pattern                      = "**/*"
      + key                                   = "testremote-docker-release-arti-nce-remote"
      + list_remote_folder_items              = true
      + metadata_retrieval_timeout_secs       = 60
      + missed_cache_period_seconds           = 1800
      + notes                                 = "ticket=ControlTower"
      + offline                               = false
      + package_type                          = (known after apply)
      + password                              = (sensitive value)
      + priority_resolution                   = false
      + project_environments                  = (known after apply)
      + project_key                           = "default"
      + repo_layout_ref                       = "simple-default"
      + retrieval_cache_period_seconds        = 7200
      + share_configuration                   = (known after apply)
      + socket_timeout_millis                 = 15000
      + store_artifacts_locally               = true
      + synchronize_properties                = false
      + unused_artifacts_cleanup_period_hours = 48
      + url                                   = "https://*******/artifactory/api/docker/testremote-docker-release-arti-nce"
      + username                              = "app-artifactory"
      + xray_index                            = false

      + content_synchronisation {
          + enabled                         = true
          + properties_enabled              = true
          + source_origin_absence_detection = true
          + statistics_enabled              = false
        }
    }

  # module.create-remote-repo-docker_blr.artifactory_virtual_docker_repository.create_virtual_repo will be created
  + resource "artifactory_virtual_docker_repository" "create_virtual_repo" {
      + artifactory_requests_can_retrieve_remote_artifacts = false
      + description                                        = "Description"
      + id                                                 = (known after apply)
      + includes_pattern                                   = "**/*"
      + key                                                = "testremote-docker-release-arti"
      + notes                                              = "managed by terraform"
      + package_type                                       = (known after apply)
      + project_environments                               = (known after apply)
      + project_key                                        = "default"
      + repo_layout_ref                                    = "simple-default"
      + repositories                                       = [
          + "testremote-docker-release-arti-nce-remote",
        ]
      + resolve_docker_tags_by_timestamp                   = false
    }

  # module.create-remote-repo-docker_blr.time_sleep.wait_5_seconds will be created
  + resource "time_sleep" "wait_5_seconds" {
      + create_duration = "5s"
      + id              = (known after apply)
    }

  # module.create-remote-repo-docker_muc.artifactory_remote_docker_repository.create_remote_repository will be created
  + resource "artifactory_remote_docker_repository" "create_remote_repository" {
      + allow_any_host_auth                   = false
      + assumed_offline_period_secs           = 300
      + blacked_out                           = false
      + block_mismatching_mime_types          = true
      + block_pushing_schema1                 = (known after apply)
      + bypass_head_requests                  = false
      + cdn_redirect                          = false
      + client_tls_certificate                = (known after apply)
      + description                           = "description"
      + disable_proxy                         = false
      + disable_url_normalization             = false
      + download_direct                       = false
      + enable_cookie_management              = false
      + enable_token_authentication           = (known after apply)
      + external_dependencies_enabled         = false
      + hard_fail                             = false
      + id                                    = (known after apply)
      + includes_pattern                      = "**/*"
      + key                                   = "testremote-docker-release-arti-nce-remote"
      + list_remote_folder_items              = true
      + metadata_retrieval_timeout_secs       = 60
      + missed_cache_period_seconds           = 1800
      + notes                                 = "ticket=ControlTower"
      + offline                               = false
      + package_type                          = (known after apply)
      + password                              = (sensitive value)
      + priority_resolution                   = false
      + project_environments                  = (known after apply)
      + project_key                           = "default"
      + repo_layout_ref                       = "simple-default"
      + retrieval_cache_period_seconds        = 7200
      + share_configuration                   = (known after apply)
      + socket_timeout_millis                 = 15000
      + store_artifacts_locally               = true
      + synchronize_properties                = false
      + unused_artifacts_cleanup_period_hours = 48
      + url                                   = "https://****/artifactory/api/docker/testremote-docker-release-arti-nce"
      + username                              = "app-artifactory"
      + xray_index                            = false

      + content_synchronisation {
          + enabled                         = true
          + properties_enabled              = true
          + source_origin_absence_detection = true
          + statistics_enabled              = false
        }
    }

  # module.create-remote-repo-docker_muc.artifactory_virtual_docker_repository.create_virtual_repo will be created
  + resource "artifactory_virtual_docker_repository" "create_virtual_repo" {
      + artifactory_requests_can_retrieve_remote_artifacts = false
      + description                                        = "Description"
      + id                                                 = (known after apply)
      + includes_pattern                                   = "**/*"
      + key                                                = "testremote-docker-release-arti"
      + notes                                              = "managed by terraform"
      + package_type                                       = (known after apply)
      + project_environments                               = (known after apply)
      + project_key                                        = "default"
      + repo_layout_ref                                    = "simple-default"
      + repositories                                       = [
          + "testremote-docker-release-arti-nce-remote",
        ]
      + resolve_docker_tags_by_timestamp                   = false
    }

  # module.create-remote-repo-docker_muc.time_sleep.wait_5_seconds will be created
  + resource "time_sleep" "wait_5_seconds" {
      + create_duration = "5s"
      + id              = (known after apply)
    }

  # module.create-repo-docker.artifactory_local_docker_v2_repository.create_local_repository will be created
  + resource "artifactory_local_docker_v2_repository" "create_local_repository" {
      + api_version           = (known after apply)
      + blacked_out           = false
      + block_pushing_schema1 = (known after apply)
      + cdn_redirect          = false
      + description           = "description"
      + excludes_pattern      = (known after apply)
      + id                    = (known after apply)
      + includes_pattern      = "**/*"
      + key                   = "testremote-docker-release-arti-nce"
      + max_unique_tags       = 0
      + notes                 = "ticket=ControlTower"
      + package_type          = (known after apply)
      + priority_resolution   = false
      + project_environments  = (known after apply)
      + project_key           = "default"
      + repo_layout_ref       = "simple-default"
      + xray_index            = false
    }

  # module.create-repo-docker.artifactory_virtual_docker_repository.create_virtual_repo will be created
  + resource "artifactory_virtual_docker_repository" "create_virtual_repo" {
      + artifactory_requests_can_retrieve_remote_artifacts = false
      + default_deployment_repo                            = "testremote-docker-release-arti-nce"
      + description                                        = "Description"
      + id                                                 = (known after apply)
      + includes_pattern                                   = "**/*"
      + key                                                = "testremote-docker-release-arti"
      + notes                                              = "managed by terraform"
      + package_type                                       = (known after apply)
      + project_environments                               = (known after apply)
      + project_key                                        = "default"
      + repo_layout_ref                                    = "simple-default"
      + repositories                                       = [
          + "testremote-docker-release-arti-nce",
        ]
      + resolve_docker_tags_by_timestamp                   = false
    }

  # module.create-repo-docker.time_sleep.wait_5_seconds will be created
  + resource "time_sleep" "wait_5_seconds" {
      + create_duration = "5s"
      + id              = (known after apply)
    }

  # module.set-repo-permissions.artifactory_group.check_iz_group_admins will be created
  + resource "artifactory_group" "check_iz_group_admins" {
      + admin_privileges = false
      + auto_join        = false
      + description      = "IZ group"
      + id               = (known after apply)
      + name             = "swb2-izga-arti"
      + policy_manager   = false
      + realm            = "ldapimport"
      + reports_manager  = false
      + watch_manager    = false
    }

  # module.set-repo-permissions.artifactory_group.check_iz_group_members will be created
  + resource "artifactory_group" "check_iz_group_members" {
      + admin_privileges = false
      + auto_join        = false
      + description      = "IZ group"
      + id               = (known after apply)
      + name             = "swb2-izg-arti"
      + policy_manager   = false
      + realm            = "ldapimport"
      + reports_manager  = false
      + watch_manager    = false
    }

  # module.set-repo-permissions.artifactory_group.check_iz_group_owner will be created
  + resource "artifactory_group" "check_iz_group_owner" {
      + admin_privileges = false
      + auto_join        = false
      + description      = "IZ group"
      + id               = (known after apply)
      + name             = "swb2-izgo-arti"
      + policy_manager   = false
      + realm            = "ldapimport"
      + reports_manager  = false
      + watch_manager    = false
    }

  # module.set-repo-permissions.artifactory_permission_target.set-repo-permissions will be created
  + resource "artifactory_permission_target" "set-repo-permissions" {
      + id   = (known after apply)
      + name = "testremote-docker-release-arti-nce"

      + repo {
          + excludes_pattern = []
          + includes_pattern = [
              + "**",
            ]
          + repositories     = [
              + "testremote-docker-release-arti-nce",
            ]

          + actions {
              + groups {
                  + name        = "every-logged-user"
                  + permissions = [
                      + "read",
                    ]
                }
              + groups {
                  + name        = "everyone"
                  + permissions = [
                      + "read",
                    ]
                }
              + groups {
                  + name        = "readers"
                  + permissions = [
                      + "annotate",
                      + "read",
                    ]
                }
              + groups {
                  + name        = "swb2-izg-arti"
                  + permissions = [
                      + "read",
                    ]
                }
              + groups {
                  + name        = "swb2-izga-arti"
                  + permissions = [
                      + "annotate",
                      + "delete",
                      + "manage",
                      + "read",
                      + "write",
                    ]
                }
              + groups {
                  + name        = "swb2-izgo-arti"
                  + permissions = [
                      + "annotate",
                      + "delete",
                      + "manage",
                      + "read",
                      + "write",
                    ]
                }

              + users {
                  + name        = "searchuser"
                  + permissions = [
                      + "read",
                    ]
                }
              + users {
                  + name        = "swb2-izu-arti"
                  + permissions = [
                      + "annotate",
                      + "delete",
                      + "read",
                      + "write",
                    ]
                }
            }
        }
    }

  # module.set-repo-permissions.time_sleep.wait_5_seconds will be created
  + resource "time_sleep" "wait_5_seconds" {
      + create_duration = "5s"
      + id              = (known after apply)
    }

Plan: 16 to add, 0 to change, 0 to destroy.
time_sleep.wait_5_seconds: Creating...
module.create-remote-repo-docker_muc.artifactory_remote_docker_repository.create_remote_repository: Creating...
time_sleep.wait_5_seconds: Creation complete after 5s [id=2024-02-16T10:10:24Z]
module.create-repo-docker.artifactory_local_docker_v2_repository.create_local_repository: Creating...
module.create-remote-repo-docker_muc.artifactory_remote_docker_repository.create_remote_repository: Creation complete after 4s [id=testremote-docker-release-arti-nce-remote]
restapi_object.add_repo_properties: Creating...
restapi_object.add_repo_properties: Creation complete after 1s [id=testremote-docker-release-arti-nce_set_properties]
module.create-remote-repo-docker_blr.artifactory_remote_docker_repository.create_remote_repository: Creating...
module.create-remote-repo-docker_blr.artifactory_remote_docker_repository.create_remote_repository: Creation complete after 8s [id=testremote-docker-release-arti-nce-remote]
module.create-remote-repo-docker_muc.time_sleep.wait_5_seconds: Creating...
module.create-repo-docker.artifactory_local_docker_v2_repository.create_local_repository: Still creating... [10s elapsed]
module.create-remote-repo-docker_muc.time_sleep.wait_5_seconds: Creation complete after 5s [id=2024-02-16T10:10:39Z]
module.create-remote-repo-docker_blr.time_sleep.wait_5_seconds: Creating...
module.create-remote-repo-docker_blr.time_sleep.wait_5_seconds: Creation complete after 5s [id=2024-02-16T10:10:44Z]
module.create-remote-repo-docker_muc.artifactory_virtual_docker_repository.create_virtual_repo: Creating...
module.create-repo-docker.artifactory_local_docker_v2_repository.create_local_repository: Still creating... [20s elapsed]
module.create-remote-repo-docker_muc.artifactory_virtual_docker_repository.create_virtual_repo: Creation complete after 2s [id=testremote-docker-release-arti]
module.set-repo-permissions.time_sleep.wait_5_seconds: Creating...
module.set-repo-permissions.artifactory_group.check_iz_group_owner: Creating...
module.set-repo-permissions.artifactory_group.check_iz_group_owner: Creation complete after 0s [id=swb2-izgo-arti]
module.set-repo-permissions.artifactory_group.check_iz_group_members: Creating...
module.set-repo-permissions.artifactory_group.check_iz_group_members: Creation complete after 1s [id=swb2-izg-arti]
module.set-repo-permissions.artifactory_group.check_iz_group_admins: Creating...
module.set-repo-permissions.time_sleep.wait_5_seconds: Creation complete after 5s [id=2024-02-16T10:10:50Z]
module.create-remote-repo-docker_blr.artifactory_virtual_docker_repository.create_virtual_repo: Creating...
module.set-repo-permissions.artifactory_group.check_iz_group_admins: Creation complete after 0s [id=swb2-izga-arti]
module.create-remote-repo-docker_blr.artifactory_virtual_docker_repository.create_virtual_repo: Creation complete after 1s [id=testremote-docker-release-arti]
module.set-repo-permissions.artifactory_permission_target.set-repo-permissions: Creating...
module.set-repo-permissions.artifactory_permission_target.set-repo-permissions: Still creating... [10s elapsed]
module.set-repo-permissions.artifactory_permission_target.set-repo-permissions: Creation complete after 15s [id=testremote-docker-release-arti-nce]
���
��� Error: Provider produced inconsistent result after apply
��� 
��� When applying changes to
��� module.create-repo-docker.artifactory_local_docker_v2_repository.create_local_repository,
��� provider "provider[\"registry.terraform.io/jfrog/artifactory\"]" produced
��� an unexpected new value: Root resource was present, but now absent.
��� 
��� This is a bug in the provider, which should be reported in the provider's
��� own issue tracker.
���

Thanks and Best Regards,
JM

[alexhung]

@jmvitali Thanks! That helps me pinning down the issue.

[alexhung]

@jmvitali I have not been able to reproduce this issue by creating the Docker v2 repo itself. However, this may be a timing issue as suggested in this post. The log messages suggests the provider was taking a long time to create the repo so it is possible Artifactory was also doing something weird.

Not sure if this is related but I notice you have time_sleep resources between your JFrog resources. Is this to ensure resources are created in correct order?

[alexhung]

@jmvitali Quick question: are you using provider alias? i.e. provider = artifactory.foo in your resource? I wonder if this is related to #892

[jmvitali]

Hello @alexhung ,
Sorry for the late answer.
Yes you are right, we are using time_sleep to wait some seconds between local repository creation and related virtual repository creation.

I'm going to try increasing the time_sleep and see the result.

Yes, we are using provider alias but the name is something like provider = "site1".

I'll keep you informed.
Thanks for your time to help on that issue.
Best Regards,
JM

[alexhung]

@jmvitali Perhaps it isn't possible in your setup/configuration, the use of resource references or depends_on meta argument will help Terraform figures out the dependencies between resources and sequence them correctly. This is more deterministic than arbitrary sleep periods.

[alexhung]

@jmvitali Can you give v10.3.2 (or later) a try and let me know if the problem still exists?

[jmvitali]

Hello Alex,
Sorry for the late answer.
Thanks a lot for the management of that ticket.
I'll test the version v10.3.2 and check if the issue will be solved.
Best Regards,
JM

[jmvitali]

Hello Alex,
For information no more issue has been identified since the implementation of the version v10.3.2 :)
Thanks a lot.
Best Regards,
JM