How to figure out if we need to analyze further or not?

Screen Link:

This is more like a concept doubt. The screen asks me if I need to analyze further or not. I almost thought that there was no further analysis required, but the next screen suggests that we also need to think about how much the leaners are willing to spend on courses. I thought this was impressive how the teachers who created the guided project mission think.

My question, how to think like a data analyst/scientist? There have been several times that I experienced “Oh, how come I did not think about this!” throughout the path, like the above example where I had to analyze further.

Now, after I complete my path, I will someday become a scientist/analyst, and I am scared I won’ be good one. So, are there any tips that Dataquest could give me based on the workflow or how the thought process of a data analyst/scientist should be?
Thank you!


Those are excellent questions. And it’s great that you are thinking about those.

While I am not a professional, yet, in DS, I do have a bit of experience related to education and learning. So, my suggestions are focused more on the latter.

But, broadly speaking there are 3 things to act upon -

  1. Practice. Pick datasets and practice on them.
  2. Ask questions.

See a column? Ask a question - what is it about? What values does it have? What type of data is it?

See another column? Ask a question again - what is it about? What values does it have? What type of data is it? How does it relate to that first column? Does it relate or not? How can I check for that from what I have learned?

Pick a column and try to ask questions about it. Questions that relate to the column itself or the rest of the data. Questions about how the column relates to the broad category the dataset might fall into. For example, looking at dataset corresponding to student loans and see a column about age or gender? Think of questions that come to mind about how age or gender might relate to what kind of student loans those groups get and have to pay? How much they have to pay over a period of time? What other aspects could influence the loan payments for those groups? Is there a column for highest degree they received/earned? Hmm… that might explain the loan amount perhaps. Check on it. But wait, what else could influence the loan amount? Maybe the university they went to? Check on that as well. Dataset doesn’t have that data… Could I find another dataset that I could use for that part? Maybe. Let’s explore more.

Ask about what kind of insights you can gather from those columns. But how do you identify insights? Create visualizations based on questions you have had. If you don’t have questions, not a problem. Think about what kind of values the column stores or represents, and think of visualizations to plot out that data. Based on the visualizations you might see a pattern. Ask more questions based on the pattern - Why is it showing that trend? Shouldn’t it show something else? What could explain that pattern? Does any other variable/column explain that pattern?

Well, because -

  • You didn’t spend enough time thinking about it.
  • Maybe you gave up too early
  • Maybe, and this is important, you do better with writing things down to facilitate your thinking than just sitting there looking at the screen and thinking about something. So, write out questions you can think of.
  • Maybe you don’t yet have all the experience you could possibly have. And that’s ok. Things take time and effort.

A lot of us go through that. Don’t worry. It’s an unfortunate side-effect of living (in general) and established education systems.

Repeat the process over multiple, different kinds of datasets and asking questions will become easier. But it needs to start from you. You can read a lot, you can read a lot of blog posts and see how others think, you can have discussions with others. And all of that will feed back into you in a positive way when you are focusing more on asking questions. Have trouble remembering everything and there’s information overload? Create a good habit of taking good notes then.

The above is a very broad generalization. But it’s important. Extremely so. Because when I said I have experience in education and how people learn, this is a very common problem I notice in learners (including myself) - not spending time thinking about the problem at hand and, instead, trying to move forward as quickly as possible. How can you spend time thinking about the problem? Asking yourself questions about it!

There will inevitably be people you will come across who will make it look very easy. You are already focusing on that based on your post. How did the instructors think of that?

  • You don’t know what kind of experience they have
  • You don’t know what other domain expertise they have
  • You don’t know how many people interacted with each other that led to good discussions on features to focus on that led to a particular insight

What you do know - the instructor probably didn’t create the project without any kind of reasonable experience. And experience comes with the [varied, deliberate] effort you put into something over time.

Being impressed by those people, unfortunately, doesn’t help others (like you, me, and a lot of other learners) to actually understand and internalize the process behind this.

They would seem more analytical, more intelligent, more curious, more creative. And they might be - some, maybe all. But that doesn’t mean those things are necessarily their innate talents.

Those attributes can be learned through deliberate practice.

As you go through the Path(s) here, you will learn quite a bit. Including how to think like an analyst/scientitst. But don’t make the most crucial mistake so many others make, and which brings me to my third point on what to act upon -

  1. Do your best to not let the above be a habit. Don’t let it get to you that you are not making progress and internalizing that you won’t be a good XYZ because of it.

You took the first step - you wanted to learn. That’s a massive step. Now you need to refine your process further. Going through DQ or any online course (even if they aren’t good enough) is a set of steps. You will have to continue climbing through your own efforts, and that needs to start with you.

I would also suggest that even if all of the above seems very difficult to do or go through, try to also create a habit of sharing your work and asking for feedback. DQ has the Social Category for this purpose. Share your work and projects and get feedback and iterate on your work based on that.