Appropriate to use corr() to find relation between tags? 469-8

davidaguilaratx · January 18, 2021, 10:16am

This is in reference to page 8: Relations Between Tags of the Popular Data Science Questions Guided Project. https://app.dataquest.io/m/469/guided-project%3A-popular-data-science-questions/8/relations-between-tags

Here goes my question:

For finding tag relationships, does it make sense to calculate and visualize a Pearson correlation coefficient for the relations_most_used dataframe found in the solutions. Why or why not?

github.com

dataquestio/solutions/blob/master/Mission469Solutions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Popular Data Science Questions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our goal in this project is to use [Data Science Stack Exchange](https://datascience.stackexchange.com) to determine what content should a data science education company create, based on interest by subject."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [

This file has been truncated. show original

# Calculate the Pearson correlation coefficients between tags
most_used_correlate = relations_most_used.corr()

# Delete all values where the tag relates to itself
for i in range(most_used_correlate.shape[0]):
    correlate.iloc[i,i] = pd.np.nan
    
# Plot the heatmap    
sns.heatmap(correlate, cmap='Greens')