PySpark-2.4.5-Notebook

This notebook is intended for learning PySpark from beginning to advanced level.

SPARKIFY_DATASET:

The dataset is a .json file that keeps track of timestamped events of the following actions performed on the digital music service:

Play a Song
Login
Listening to an advertisement
Downgrading subscription
Cancelling subscription

There are 3 different sizes of the dataset available:

mini_sparkify_event_data.json: the smallest instance of the dataset (125 mb)
medium-sparkify-event-data.json: a medium-sized instance of the dataset (237 mb)
sparkify_event_data.json: the full dataset (12 gb)

Download Links for Datasets:

Link to small-sized subset of Sparkify data (125 mb): https://drive.google.com/open?id=1FwuyO5apNwy8q6BpG-_EIqqN-ED0X9tx
Link to medium-sized subset of Sparkify data (237 mb): https://drive.google.com/open?id=17Lys6v7LOcAWFMHXwXwslUWj04LoNWp1
Link to full Sparkify dataset on AWS (12 gb): s3n://dsnd-sparkify/sparkify_event_data.json

Name		Name	Last commit message	Last commit date
Latest commit History 9 Commits
.ipynb_checkpoints		.ipynb_checkpoints
img		img
01.Spark-Intro.ipynb		01.Spark-Intro.ipynb
02.Loading data into RDDs.ipynb		02.Loading data into RDDs.ipynb
03.textFilevswholeTextFile.ipynb		03.textFilevswholeTextFile.ipynb
04.RelationalDB.ipynb		04.RelationalDB.ipynb
05.JSONRead.ipynb		05.JSONRead.ipynb
06.RDDProgrammatically.ipynb		06.RDDProgrammatically.ipynb
07.Persistence.ipynb		07.Persistence.ipynb
08.Lineage(Theory).ipynb		08.Lineage(Theory).ipynb
09.RDD TYPES(Theory).ipynb		09.RDD TYPES(Theory).ipynb
10a.BasicRDDTransformations(List).ipynb		10a.BasicRDDTransformations(List).ipynb
10b.mapFilterflatMap.ipynb		10b.mapFilterflatMap.ipynb
11.Distinct.ipynb		11.Distinct.ipynb
12.groupBy.ipynb		12.groupBy.ipynb
13.sortBy.ipynb		13.sortBy.ipynb
14.Basic RDD Action(List).ipynb		14.Basic RDD Action(List).ipynb
15.count.ipynb		15.count.ipynb
16.collect.ipynb		16.collect.ipynb
17.take.ipynb		17.take.ipynb
18.top.ipynb		18.top.ipynb
19.first.ipynb		19.first.ipynb
20.reduceAndFold.ipynb		20.reduceAndFold.ipynb
21.forEach.ipynb		21.forEach.ipynb
22.PariRDDs(Theory).ipynb		22.PariRDDs(Theory).ipynb
23.keysAndValues.ipynb		23.keysAndValues.ipynb
24.keyBy.ipynb		24.keyBy.ipynb
25.mapValues flatMapValues groupByKey reduceByKey foldByKey sortByKey.ipynb		25.mapValues flatMapValues groupByKey reduceByKey foldByKey sortByKey.ipynb
26.RegEx_Overview.ipynb		26.RegEx_Overview.ipynb
27.WordCount.ipynb		27.WordCount.ipynb
28.read_with_dataframe.ipynb		28.read_with_dataframe.ipynb
29.data_wrangling.ipynb		29.data_wrangling.ipynb
30.data_wrangling-sql.ipynb		30.data_wrangling-sql.ipynb
31.JDBC Connection-Relational-DB.ipynb		31.JDBC Connection-Relational-DB.ipynb
32.Join(Theory).ipynb		32.Join(Theory).ipynb
33.All-Joins-Explained.ipynb		33.All-Joins-Explained.ipynb
34.cogroup and cartesian.ipynb		34.cogroup and cartesian.ipynb
35.Joining Datasets in Spark.ipynb		35.Joining Datasets in Spark.ipynb
36.Transformation on Sets.ipynb		36.Transformation on Sets.ipynb
37.Transformations on Numeric RDDs.ipynb		37.Transformations on Numeric RDDs.ipynb
38.Parquet Files.ipynb		38.Parquet Files.ipynb
README.md		README.md

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

PySpark-2.4.5-Notebook

SPARKIFY_DATASET:

About

Releases

Packages

Languages

usman87626/PySpark-2.4.5-Notebook

Folders and files

Latest commit

History

Repository files navigation

PySpark-2.4.5-Notebook

SPARKIFY_DATASET:

About

Resources

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages