Redis on Digital Ocean vs. AWS/ElastiCache

I'm running a Rails app on AWS Elastic Beanstalk / EC2 which also uses ElastiCache / Redis to run background jobs with Sidekiq. If I'm not mistaken, ElastiCache is a completely separate service than the core EC2 compute engine, so any Redis storage or I/O activity shouldn't impact the EC2 compute instance performance or storage requirements, other than of course the jobs themselves being executed on EC2.

How does this work on Digital Ocean? I've setup a droplet with the identical Rails application along with Redis. However, it seems like the Redis database is on the same droplet as the compute engine on digital ocean. If for some reason there should happen to be a large number of jobs queued in a short period of time, will the Redis activity negatively impact the performance of the droplet since it doesn't seem to be extracted to its own service like it is with ElastiCache on AWS?

By the way, the Hatch service from GoRails makes it super easy to deploy an app to Digital Ocean. Definitely recommend it for anyone who isn't a seasoned DevOps engineer or Linux guru. I have no idea what I'm doing with Linux and I was able to deploy my app to digital ocean with very little pain / effort.

So Redis is an in-memory database. Sidekiq simply uses it as a store of the jobs that need to be run or are running. That information is actually quite minimal because it's really just the name of the class (the Job) to run and the arguments for it (which are just IDs of database records generally). You probably won't have hardly any space used with it, unless you're processing hundreds of thousands of jobs.

ElastiCache is basically a dedicated server just for this service. You can do the same thing by running Redis on a separate machine, but you'll want to setup firewall rules and things for that. EC does that for you basically.

You might want to run this on a separate server if you have a bunch of servers starting jobs. They can all talk to the same Redis instance just like you would have a central database for a bunch of webservers.

Hatch starts you out on the simplest path: Redis on the same machine. Once you grow out of it you can pretty easily migrate to a store on another server.

Thanks, Chris. This is what I suspected. Redis on the same machine seems like a good easy solution. And although it would be a nice problem to need to process hundreds of thousands of jobs, realistically I'll never need that.