PySpark

PySpark is an interface for Apache Spark in Python. It not only allows you to write Spark applications using Python APIs, but also provides the PySpark shell for interactively analyzing your data in a distributed environment. PySpark supports most of Spark’s features such as Spark SQL, DataFrame, Streaming, MLlib (Machine Learning) and Spark Core.

Complete Courses

• For learning deep, you can go ahead with this and this paid course on Udemy.

Videos lectures

• This lecture series has very good introduction to PySpark

Documentation

• Documentation can be found here

Diving Deep

• PySpark tutorial
• Pyspark tutorial 2