How to use Rails for ETL and internal tools
I need to build some internal tools to generate reports, csv's, etc. I need to ingest data from a few different APIs, into a central DB, and then build some tools for end users on top of that data.
For example, once I have all of the data from the APIs, the end user would fill out a form to provide some data, and then the app will generate a csv for the end user to download.
I am primarily a frontend dev and could use some help/advice here. My approach thus far has been to use Rails services and Rake tasks that fetch the data from the APIs, but am wondering if there is a better approach. Also I am not sure what gems may or may not be useful.
I have a license for jumpstart pro and was thinking of maybe using that to give me a "head start?"
If anyone has done anything like this before and could "point me in the right direction" I would really appreciate it!
Thanks in advance :)
Your current approach with Rails services is solid. For better performance, use Sidekiq for background API data fetching. Consider HTTParty or Faraday for API integration, and use FasterCSV for CSV generation. Stick with Rails' Active Record for your DB and PostgreSQL if you need advanced queries. Jumpstart Pro is a great choice for building internal tools quickly with features like authentication already in place.