Spark and Map Reduce

What is the version being taught in spark and map-reduce course and has there been newer version to it, if yes does anything change a lot that I should be mindful of, Thanks

The version is mentioned here - https://app.dataquest.io/m/127/project%3A-spark-installation-and-jupyter-notebook-integration/3/spark

You can check the current version here - https://spark.apache.org/downloads.html

It’s difficult to be up-to-date about every single change between version updates. So, as you work with a specific version, you can go through the documentation to see what might work or not work.

You can also go through their Release Notes for a specific version to see what changes might have been made compared to the previous version.

They do have a “Migration Guide” which might be helpful - https://spark.apache.org/docs/2.4.0/sql-migration-guide-upgrade.html

It’s, unfortunately, not an easy process to be sure of what has changed and by how much unless you start working on a newer version, go through the documentation, and if things break. This is completely based on my experience, so my knowledge might be limited.