Using data analysis to see if my dad is right about today’s music

While the type of music that we listen to is usually subject to our own personal preference, I’m sure that everyone out there knows of at least one other person who has really strong opinions on the state of music today. Instead of that leather-clad metalhead or the pretentious hipster that lists Folk or Jazz music as a personality, this happened to be my old man.

In a 10-minute rant during a car ride, he went on to highlight that

  1. There was a quality in songs back in his day that’s missing in today’s music.
  2. Young people (i.e., 35 and under) have horrible taste in music (basically anything after the early 1990s).

While I shrugged this off as another one of his “OK Boomer” moments, it did give me an idea to see what the data has to say.

Using some data from one of the biggest music streaming platforms, (Spotify), which also happens to have the majority of users being young adults, I will examine trends in audio and track features based on release year. Specifically, comparing some of the popular artists from the past 10 years with the most popular artists in his day (i.e., 1970s and 1980s). Furthermore, I will also look at what are some of the most popular artists in 2020.

The data

A data set that was scrapped from Spotify Web API was made available on Kaggle that contained information from over 175,000 songs that released between 1921-2020. Aside from who’s the credited artists and the year of release, this data set also contained scores for various audio features relating to a given track as shown in the developer’s blog. I’ve included a list and explanation below:

Audio/Track Feature Description
acousticness to what degree is the sound of the track being produced by non-electric means
danceability how suitable is the track for the purpose of dancing based on tempo, rhythm stability, beats per minute and overall regularity
duration length of the track (in milliseconds)
energy the perceptual measure of intensity and activity of the track based on dynamic range, perceived loudness, timbre, onset rate, and general entropy
explicit does the track contain explicit content
featured artist^ does the have a featured artist
instrumentalness to what degree does the track contain no vocal elements
key musical concept describing the collection of pitches that form a framework for melody and harmony and tension/resolution, which may be based around any root pitch
liveness the degree of presence of a live audience on the track
loudness the overall averaged loudness of the track in decibels (dB)
mode the relative key in which the track is being played in (i.e., Is it in major or minor?)
one-hit wonder^ did the track come from an artist with less than 3 songs on Spotify
speechiness the degree to which spoken words are present in the track
tempo the overall speed or pace that the music is being played (in beats per minute)
valence the degree to which the sound conveys a sense of positive mood

Note: The majority of these features are scored along a scale from 0 to 1 (low-to-high).
^ Added features during the data cleaning step

As there was a fair amount of data-cleaning done to do this analysis, I’ve only included the findings in this article. For the complete codes used, check out my GitHub.

How has songs changed over the years?

In order to test my dad’s first claim that there is a missing quality in today’s music that wasn’t in the past, I’ve investigated to see any changes in the trends of each attribute listed above across the release year. Specifically, I took the mean of most of these features and plotting it across the release year. Below are several plots to demonstrate these changes.

image

image

image

image

A few observations from these plots showed that:

  1. There seems to be a renaissance in the uptick of artist collaboration on a track in recent years as compared to the period from the late-60s to mid-90s.
  2. Both acousticness and instrumentalness are at an all-time low as compared to in the past.
  3. Conversely, energy, and loudness in songs have been on a rise in recent years.
  4. There is an all-time high in the number of explicit-content in tracks within recent years as compared to in the past

How do the greats of the past compare to artists today?

After creating a data frame that filtered out the data to include only the artists listed above and differentiating them as either “classic” or “current”, I’ve compared the differences in each audio/track feature. Below are the findings presented in a table using the kable package.

image

Overall, it appears that aside from tempo, there were significant differences across the board. Today’s top artists appear to really be into the electronic sound with more speech-like lyrical content (thanks Hip Hop + R&B). On top of that, music today appears to be “moodier” despite tracks having more energy in them (i.e., Billie Eilish or Post Malone).

So, who are young people listening to?

Working on the assumption that the vast majority of Spotify users are young people, here’s a look at the top 40 artists that had released a track in 2020, according to the average popularity score across in that year.

Note: This excluded those who did not have at least 3 tracks on Spotify.

image

A quick look here showed a mix of popular Hip Hop and R&B, pop, and Latin artists. Interestingly, it appears as though the majority of these artists debuted within the past 5 years or so. Exploring further with a correlation matrix, with the ggcorplot package, it seems that the popularity score is strongly tied to release year.

This point is further emphasized once we compare the mean popularity scores of the most popular artists in 2020 against the all-time top 40 artists.

image

image

Looking at the comparison, it seems that in spite of the skew towards more contemporary artists, many young listeners are also listening to the greats of the past like Michael Jackson, The Beatles, and Prince.

The Verdict…

Overall, it seems my old man might be right about the change in sound in today’s music. We’ve appeared to have moved away from playing instruments and that “live” sound to someone making beats with a push of a button and a louder or more energetic sound.

However, that’s not to say we don’t have any taste in music as a whole. Sure, the apparent preference for harder content is there that may not be to the taste of the older generation, but many of us still enjoy the hits of the past as well. In fact, I challenge anyone who isn’t going to break out a dance while listening to some Earth, Wind, and Fire like these folks here.

If there are any typos or inconsistencies or would like to ask or say something, just drop a comment.

Thanks for reading.

13 Likes

Congrats on your project! A great idea and an interesting article as the result!

1 Like

Thanks @ksenia.kustanovich. I figured this would be a nice change of pace to focus on the “fun with data”. Too bad that this still didn’t change my dad’s opinion about young people’s taste in music. :sweat_smile:

1 Like

This is a great review of music data. I figured I was hearing more protools tore off music than music from when I was younger, so it’s interesting to see your data say the same thing.

1 Like

Thanks, @riffology. Yeah, I figured that would be the case as more music sound way too “clean” and overly produced than before. Plus, a lot of pop songs nowadays are borrowing beats from past hits about 20-30 years ago. In any case, this was a fun little dive into music data.

Pretty cool article @michael.hoang17! A fresh and simple analysis, with good story telling :+1:

1 Like

Thanks @vcuspinera .

What a fun article. I’m probably at least as old as your dad, growing up in the classic rock era. The rock music my teenage son listened to is superior musically to classic rock: much more sophisticated drum and bass lines, multiple themes (instead of the A-B themes of classic rock), much more diversity in lyric and music styles. Only the progressive rockers of the '70s matched or exceeded the musicallity in modern rockers.

1 Like

Glad you enjoyed the article @ryandmonson. Definitely, in terms of rock, there was definitely a split where certain artists just pushed towards that technical virtuoso sound in the late 70s to 80s that just never really stopped. Times have definitely changed.