Backing up uploaded files (hundreds of gigs) to S3 using Backup gem without duplication on HD
Let's say I have 400GB of uploaded images inside a 800GB VPN machine. So we're confortably using 50% of storage.
Even tough we have our hosting's backup enabled, we want to have an extra backup sitting on S3, just because we can't get enough of backups right?
So, using `backup` gem, we are already sucessfully backing up Mysql and storing it on S3, so now we want to create a second model to backup these files.
The question is: if we create a zipped file with all theses images prior to uploading to S3, we would run out of disk space right?
Is there a way to just upload all 400gb of images to S3 and incrementally update it, on a daily basis, without having to duplicate all files (zipped or not) on hard drive?
Is RSync better suited for this, in which case I should opt out of S3 and buy a node (like in Digital Ocean) ?
What are our options?
Even tough we have our hosting's backup enabled, we want to have an extra backup sitting on S3, just because we can't get enough of backups right?
So, using `backup` gem, we are already sucessfully backing up Mysql and storing it on S3, so now we want to create a second model to backup these files.
The question is: if we create a zipped file with all theses images prior to uploading to S3, we would run out of disk space right?
Is there a way to just upload all 400gb of images to S3 and incrementally update it, on a daily basis, without having to duplicate all files (zipped or not) on hard drive?
Is RSync better suited for this, in which case I should opt out of S3 and buy a node (like in Digital Ocean) ?
What are our options?
Hey TL,
You're looking for the backup gem's Syncers concept: http://backup.github.io/backup/v4/syncers/
You can't use rsync with S3, but you can use this gem's S3 syncer to mirror all the files on disk with S3.
The S3 Syncer just takes some directories that you want to sync and will handle all the uploads. http://backup.github.io/backup/v4/syncer-s3/
You're looking for the backup gem's Syncers concept: http://backup.github.io/backup/v4/syncers/
You can't use rsync with S3, but you can use this gem's S3 syncer to mirror all the files on disk with S3.
The S3 Syncer just takes some directories that you want to sync and will handle all the uploads. http://backup.github.io/backup/v4/syncer-s3/