Download required data sources from Kaggle into the datasource/ directory and prepare the datasets for their usage. The required steps are described in the Datasources Readme file.
./gradlew clean build bootBuildImage
chmod -R g+r,o+r datasource/
chmod -R g+r,o+r news-db/
chmod -R g+r,o+r monitoring/
docker-compose pull --ignore-pull-failures
docker-compose up -d
docker ps -a --format 'table {{.Names}}\t{{.Status}}'
NAMES STATUS interest-feed Up 16 seconds user-interest-db-init Exited (0) 10 seconds ago news Up 11 seconds prometheus Up 16 seconds grafana Up 16 seconds mariadb-prometheus-exporter Up 16 seconds news-db Up 17 seconds user-interest-db Up 16 seconds redis-prometheus-exporter Up 16 seconds node_exporter Up 17 seconds cadvisor Up 17 seconds
Hint: The setup only works with Docker
and has only been tested on Linux
.
Websites:
-
Grafana Dashboards: http://localhost:3000/dashboards
-
Prometheus Targets: http://localhost:9090/targets
curl -s -XGET -H "Accept: application/json" 'http://localhost:8081/news/recommended?topics=HOME%20%26%20LIVING,COMEDY&fromDate=2013-07-29&untilDate=2013-08-05&limit=20'
Hint: In a previous version we used a POST request for the /recommendedNews
endpoint to simplify specification of parameters.
However, Http client-side cache is not possible for POST requests. Therefore, we switched to a GET request.
curl -s -XGET -H "Accept: application/json" 'http://localhost:8081/news/latest?untilDate=2013-08-05&limit=3'
curl -s -XGET -H "Accept: application/json" 'http://localhost:8082/feed/mariameyer000?date=2013-08-05'
curl -s -XGET -H "Accept: application/json" 'http://localhost:8082/usernames?limit=10'
curl -s -XGET -H "Accept: application/json" 'http://localhost:8082/interests?limit=10'
curl -s -XGET -H "Accept: application/json" 'http://localhost:8082/interests/mariameyer000'
-
✓ Feed should always contain news (use latest feeds API)
-
✓ API to show all users (and maybe their interests)
-
✓ Benchmark app
-
✓ Monitoring setup and metrics for CPU time, CPU usage and memory of services and dbs
-
✓ Metrics for service and db internals like http connections, …
-
❏ UI for Feeds and Usernames
-
❏ Implementation of simple Feed caching (enable via config)
-
✓ Http client cache in Interest-Feed service for News service endpoints
-
✓ Http client cache in Benchmark for Interest-Feed service endpoints
-
❏ Redis client-side cache in Interest-Feed service
-
❏ MariaDB client-side cache in News service
-
❏ Http independent cache for Interest-Feed and News service
-
-
❏ Monitoring for Watt and CO2e
-
❏ Implementation of predictive caching
-
❏ Setup for Kubernetes
-
❏ Documentation with images