Noteable is a newer Notebook platform that lets you write SQL directly on Excel or CSV files. No need to upload files to a server, create a table, then insert data into the table…
Try it on noteable.io
Here is an example notebook: https://app.noteable.io/f/2d4395fe-a6bf-473d-aee5-4ecbb715eaa5/Use-SQL-to-query-CSV-files-and-Dataframes-.ipynb
Thank you for sharing the link to Noteable. Have you found some use cases within your workflow where you are using Noteable .
I am going to dig a little deeper but I can think of so many ways this is more helpful than having to create tables first. I don’t want to sound “too extreme” but this seems like a big deal and a huge help but I am new to all the available data tools outside of Jupyter .
Thanks for sharing…
Yes, I do believe this is HUGE.
I have been an analyst → statistician → data scientist → data science manager → director for over 12 years and I can tell you first hand that in all those roles, this would have been a game changer.
Use Case # 1: When you capture data at a point from database, aggregate it, and want to store it in a flexible manner. In these cases, you write the query, extract the data and store as CSV files (or excel). In most of my impactful projects, I have never worked with a file that is less than 1 GB. You can’t explore files of that size in Excel without pulling a couple of hair out. Being able to write SQL to then explore and further aggregate CSV/Excel datasets is a huge convenience.
Use Case # 2: When you are prototyping, a large part of the work happens in CSV files. A data scientist rarely has the time to go set up static tables in DBs - its just a non value add activity to have the data and the create a copy of that data in the DB just to be able to use that data.
In my experience, about 30-40% of data science workflows involve working heavily with static data files (CSV/Excel). And, finally, we have a solution that enables data practitioners to move fast by using SQL directly on the files.
I find it super interesting what you have shared and in fact some time ago with the practice of the CIA I was able to link locally (on my computer) libraries of graphical functions along with SQL, I must say that it took me a while and that your proposal is certainly very interesting.
One thing that automatically activates me is to know what version of python is using the libraries and what kind of function library it uses for the graphics since as we all know it is possible that trying to do something with the version that is not the correct one can be a drama. Even so I thank you very much that you have hung it here, it is super interesting to see the things that people do.
Thank a lot.