Active Storage Aws::S3::Errors::NoSuchKey error for variant when deleting an attachment

Question

I have a user model with an attachment as follows in my rails 7.1 application

class User
  has_one_attached :avatar
end

I am usign Active Storage with S3-compatible storage from Scaleway.

I can create attachments and also delete them. When I delete an attachment, the files are gone in my object storage and the counts in the database for ActiveStorage::Attachment.count are correct.

My logs are showing me the following errors

10:42:10 sidekiq.1 | I, [2024-01-04T10:42:10.084390 #7886]  INFO -- :   S3 Storage (116.8ms) Deleted file from key: s9jppxubel7hc96a6dulvcd0hy1g
10:42:10 sidekiq.1 | I, [2024-01-04T10:42:10.123300 #7886]  INFO -- :   S3 Storage (126.2ms) Deleted file from key: 9ye6w1rfcxufsbjkh58xdq79jugm
10:42:10 sidekiq.1 | I, [2024-01-04T10:42:10.186220 #7886]  INFO -- :   S3 Storage (62.7ms) Deleted files by key prefix: variants/9ye6w1rfcxufsbjkh58xdq79jugm/
10:42:10 sidekiq.1 | E, [2024-01-04T10:42:10.186404 #7886] ERROR -- : Error performing ActiveStorage::PurgeJob (Job ID: 158ef8d0-0d3b-4438-b5f9-cfbbeed36561) from Sidekiq(default) in 199.24ms: Aws::S3::Errors::NoSuchKey (The specified key does not exist.):

I am not expecting this error from the sidekiq job (basically ActiveJob using sidekiq as the backend).

I read that for object storage the files are not being placed in subfolders but I didn't manage to google anything relevant.

Anyone seen these error logs and knows how to tackle them? Any help appreciated

Edit: Here's my Active Storage configuration:

scaleway:
  service: S3
  access_key_id: <%= ENV.fetch("SCALEWAY_S3_ACCESS_KEY_ID") %>
  secret_access_key: <%= ENV.fetch("SCALEWAY_S3_ACCESS_KEY_SECRET") %>
  region: <%= ENV.fetch("SCALEWAY_S3_REGION") %>
  bucket: <%= ENV.fetch("SCALEWAY_S3_BUCKET_NAME") %>
  endpoint: <%= ENV.fetch("SCALEWAY_S3_ENDPOINT") %>

EDIT 2: I confirmed the behaviour using DigitalOcean spaces. Variant files are not getting created in a variants subfolder (as done with the disk service). This boils down the delete_prefixed method in Active Storage

EDIT 3: I confirmed the behaviour with a vanilla rails 7.1.2 application. See https://github.com/basiszwo/active-storage-object-storage-demo