Making note of what I need to do to migrate my Discourse server (for t/suki) to S3 Storage.

To set up S3 storage using Backblaze:

meta.discourse.org:

You need to skip CORS and configure it manually.

There are reports of clean up orphan uploads not working correctly with BackBlaze. You must change lifecycle rules for your bucket for orphan cleanup to work.

Example configuration:

 DISCOURSE_USE_S3: true
 DISCOURSE_S3_REGION: "us-west-002"
 DISCOURSE_S3_INSTALL_CORS_RULE: false
 DISCOURSE_S3_CONFIGURE_TOMBSTONE_POLICY: false
 DISCOURSE_S3_ENDPOINT: https://s3.us-west-002.backblazeb2.com
 DISCOURSE_S3_ACCESS_KEY_ID: myaccesskey
 DISCOURSE_S3_SECRET_ACCESS_KEY: mysecretkey
 DISCOURSE_S3_CDN_URL: https://falcoland-files-cdn.falco.dev
 DISCOURSE_S3_BUCKET: falcoland-files
 DISCOURSE_S3_BACKUP_BUCKET: falcoland-files/backup
 DISCOURSE_BACKUP_LOCATION: s3

Note: During initial migration to B2, you may hit the 2500 free daily class C transactions limit. You will need to add a payment method to remove caps.

As for the orphan uploads not working correctly, it appears that I need to turn lifecycle rules off:

@V1ktor on meta.discourse.org:

By by default, Backblaze sets “Keep all files” lifecycle rule for newly created buckets. That’s probably why everyone has issues with orphans. Changing lifecycle rule to “Keep only the last version of the file” deletes orphans after 24 hours once it’s hidden.

After S3 is set up, the following steps will migrate existing local assets to S3:

@rsmithlal on meta.discourse.org

I just switched my instance from local to S3 uploads and I could not for the life of me figure out how to get the S3 migrator to work. In the interest of anyone else who stops by this topic, I will summarize the required steps here.

1. Enable S3 Uploads in the admin interface and set all of the required S3 keys and settings so new uploads are sent to S3

2. Add the required DISCOURSE_S3_ env variables to the env section of app.yml

   Example:

   DISCOURSE_S3_BUCKET: your-bucket-name
   DISCOURSE_S3_REGION: your-s3-bucket-region
   DISCOURSE_S3_ACCESS_KEY_ID: your-s3-access-key-id
   DISCOURSE_S3_SECRET_ACCESS_KEY: your-s3-secret-access-key
   

3. Rebuild your app with ./launcher rebuild app

4. Enter the app with ./launcher enter app

5. Run rails uploads:migrate_to_s3 from within the docker container.

I initially tried to run the command using ./launcher run app rails uploads:migrate_to_s3 but it couldn’t find the task.