ETL Restful API

How do I extract the data from an API and then store the data into a database? I see there is a API course but there is no lesson or section showing how to take the data and load it into a database.

Hi, @CrimsonGee

Actually you just need to combine your knowledge separately from the courses on databases and separately from the courses on working with API.

But if you move on, your question is somewhat more complicated.

  1. The approach may vary when using different databases. Especially if it is SQL and NoSQL databases.
  2. If you have a multi-threaded, and the database is designed for a large number of users, then you need to separately think about the logic of connecting to the database, loading data and disconnect.

But with a simple variant of use, the following is enough for you.

  1. Get data and API in a convenient format
  2. Create DataFrame
  3. Connect to an SQL database
  4. use DataFrame.to_sql() for write - https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_sql.html.
1 Like