Read a CSV file with API

Hi there. I’m trying to get data from: https://www.fema.gov/openfema-data-page/registration-intake-and-individuals-household-program-ri-ihp-v2. I’m working with Jupyter Notebook with this command code but takes forever (sand clock loading eternally) fema_individual_and_housaholds_programs = pd.read_csv(‘https://www.fema.gov/api/open/v1/IndividualsAndHouseholdsProgramValidRegistrations.csv’)

What I’m I doing wrong?

How big file is? It may take longer if it is large file. You can download it by direct visiting - https://www.fema.gov/api/open/v1/IndividualsAndHouseholdsProgramValidRegistrations.csv

Thank you @DishinGoyani. The problem with trying to download it directly is that the file has aprx. 4 million rows and excel/csv supports just one million. So this way (if I download it directly) I just get 25% of the data and I need the whole thing.

@DishinGoyani. Actually the file is 16 million rows. Check it out in the Full Data Section (Record Count) https://www.fema.gov/openfema-data-page/individuals-and-households-program-valid-registrations-v1

@santamariaots this article may help (involves using dask instead).

2 Likes