PYSPARK REALTIME STREAMING SENTIMENT ANALYSIS

⏱ Real-Time Sentiment Analysis using PySpark and FastAPI

PROTOTYPE VIDEO

video.mov

HOW TO EXECUTE

Terminal

git clone https://github.com/raghavtwenty/pyspark-realtime-streaming-sentiment-analysis.git

cd code/

pip install -r requirements.txt

Run Fast API Server

uvicorn _1_fastapi_server:app --reload

Run PySpark

python _3_sentiment_analysis_pyspark.py

DOMAIN

Big Data Analytics

OBJECTIVE

Perform sentiment analysis for the realtime streaming data

INTRODUCTION

In the era of big data, the ability to process and analyze real-time data streams is crucial for gaining actionable insights. This project aims to demonstrate a real-time streaming sentiment analysis application using PySpark and FastAPI. The sentiment analysis model processes incoming data in real time, determining the sentiment polarity (positive, negative, or neutral) of each data point. This prototype is particularly beneficial for applications such as social media monitoring, customer feedback analysis, and other domains where timely sentiment information is critical.

I haven't used Twitter/X API, Instead simulated the same using FastAPI. Free of cost.

FEATURES

• Highly Scalable
• Real-time
• Parallel Processing
  
  
  

TECHNOLOGIES USED

• Spark
  
• Fast API
  
  
  

END USERS

1. Students
2. Data Analyst
3. Data Scientists
   
   
   

OUTPUTS

• Fast API Server
  
  

• PySpark
  
  

• PySpark
  
  

END OF README