Feedback on Web scraping, data analysis and deploying the dashboard on heroku

Hi everyone, I recently finished a project and I am looking forward to get some feedback from you.

In this project, first I scraped the data. Then did some explanatory data analysis to understand the problem better. Later I used plotly dash to create the charts and deploy it on Heroku for others to use. I am thinking about including this project on my resume and really want your honest opinion on this.

  1. Project website link - the final result and summary of the project.
  2. Project code - which is hosted on github.

what each folders contains-
a. Notebook - it contains two EDA files for data exploration.
b. web scraping - it contains two notebook which was used for initial web scraping. And a final python script which anybody can use to create the data set by running it.
c. ipl-app.py - python script for creating the dashboard app.

Thanks,
Bhola

3 Likes

Incredible job @bholaprasad26! Especially the web scraping script, I will use it to learn a lot about the web-scraping.

How much time did it take you to create it?

1 Like

Thanks @artur.sannikov96 for the kind words. It is hard to tell because I was not working continuously on the project but if you do I would say approximately 2 weeks is enough to complete the whole project.

1 Like

Only two weeks of work? Really impressive. I would like to be able to make things like that.

I am a complete noob in IPL, I don’t know what it is! Maybe you could start by describing quickly the sport your are talking about and make a short glossary containing the tecnical terms you will use later in the app. The purpose of the app is to help gamblers to make bets based upon statistics? It’s not clear for me, but I guess this is because I am not familiar with IPL universe.

2 Likes

Hi @WilfriedF , I have included a data description markdown file in the data folder. It contains the meaning of all the metrics that has been used in the project. Btw, thanks for the idea, i will include a IPL explainer video link on the website for all the people who are not familiar with IPL. Thanks again, have a great day.

Thanks for the precisions @bholaprasad26 ! Was thinking: you may be interested by the new Kaggle competition, since you have just finished this work: NFL Big Data Bowl 2010

Not the same sport, but you already have good insights about how to read and analyse players performances.

Have a great day too!

2 Likes

Thanks @WilfriedF for the suggestion. It’s a good competition with lots of money :grinning: Definitely think about it.

1 Like

Driven Data usually presents a benchmark at each new competition, so the competitor has some guidelines helping to start with. But with Kaggle apparently it’s different. I didn’t found any benchmark which I could follow for the NFL competition. Since I’m not ready yet to start ex nihilo such a competition with very experimented participants, I will wait for a better opportunity. But you are right, prizepool is very tempting!

2 Likes

@WilfriedF Currently, i am doing one driven data competition - modeling earthquake damage. It does not have any prize but I am learning a lot by doing this competition.

1 Like

Oh I was taking a look yesterday searching for competition with a confortable deadline! If I decide to enter, I will let you know. I was interested too by the Pump it Up: Data Mining the Water Table, but again I didn’t found the benchmark!

1 Like

Why do not you look at the leader board ? just compare your performance with other competitors. you can also create a dummy classifier using scikit-learn to get a baseline, than compare your accuracy against it. But, leader board is still best to compare performance. Hope this help.

1 Like

I know, I know. What I’m afraid of is having to start with raw data, sometimes it takes me hours to fix little things with pandas. The same goes for calculating the score and submitting the final csv file, these are crucial steps and having guidelines already done let you more time to put your hands in the core of the problem. But you’re still right, I shouldn’t be afraid to start from scratch and I should just try, maybe it’ll be easier than I think. Thanks for the dummy classifier link, I didn’t know about.

2 Likes

@WilfriedF we all feel afraid and i know it can feel intimidating to start from scratch but it is better to fail and learn now than later. You have more time and opportunity to learn and grow. Start working on it and whenever feel stuck, just ask here or on stackoverflow, people are more than happy to help others. This is the best thing I like about the programming community. I hope you have a good time, best of luck.

3 Likes