subhub finished

README.md

@@ -1 +1,73 @@
-# subhub
+# WebSub Hub demo
+
+Subhub is a small WebSub hub demo implemented in Go using echo, http/net, and Postgres. A live version of the hub is running [here](https://subhub.henriknorrman.com), listening for subscription to the topic "advice".
+
+### WebSub
+[WebSub](https://www.w3.org/TR/websub/) provides a common mechanism for communication between publishers of any kind of Web content and their subscribers, based on HTTP web hooks. Subscription requests are relayed through hubs, which validate and verify the request. Hubs then distribute new and updated content to subscribers when it becomes available. WebSub was previously known as PubSubHubbub.
+
+### Features
+ * Accepts all subscriptions with `hub.callback`, `hub.mode`, and `hub.topic` defined.
+ * If provided, stores `hub.secret` for HMAC signature when broadcasting published content.
+ * Does NOT implement a lease period on subscription requests.
+ * Supports unsubscription requests.
+ * Sends distributed content as JSON.
+ * Sends `X-Hub-Signature` header with all content distributions.
+ * Allows user to resubscribe to already subscribed topics.
+ * Verifies every valid (required params are provided) subscription and unsubscription request.
+ * Generates and broadcasts content to all subscribers of the `hub.topic` "advice".
+
+### Endpoints
+
+`POST /`: Reads header for required parameters and handle subscription requests. 
+`GET /publish`: Acts as the publisher for testing, broadcasts content to subscribers.
+
+### Files
+**server.go** is the entry point to the server. It contains the `main()` function to the server where the port listener and routes are defined.
+
+**hub.go** holds all WebSub Hub logic, and most of the code. The Hub is not concerned with how or where the subscriber data is stored, as long as the following functions are defined:
+* `type hubStore struct {}`
+* `(h *hubStore) init()`
+* `(h *hubStore) addSubscriber(callback string, secret string, topic string, timestamp int64)`
+* `(h *hubStore) removeSubscriber(callback string, topic string)`
+* `(h *hubStore) getAllSubsByTopic(topic string) []subscription`
+
+
+**db.go** holds all Postgres logic: connecting to the database, creating tables, and making queries. All SQL and database code is contained within this file. 
+
+
+**utils.go** contains a few general utility functions, such as random hex string generator, sha256 hasher, and random advice fetcher.
+
+### Flow
+The following diagrams shows the main action flow through the hub, from client to database. The three columns represent the different code files and how they interract.
+
+#### Sucessful subscription requests
+The diagram shows a successful subscription `POST` request to `/`. The request is first passed to `handleSubscriber()` where parameters are checked and read. If the requests contains everything to be verified, the information is then passed to `verifyIntent()` for verification. This function in turn uses `sendGET()` to send off the `GET` request. If everything is successful, `verifyIntent()` then calls `addSubscriber()` which sends `INSERT` query to the database.
+
+<div align="center">
+<img src="./subscribe-success.png" width="700">
+</div>
+
+
+#### Successful publish requests
+The diagram shows a successful `/publish` call. The `dummyPublisher()` fetches all subscriber data from the database using `getAllSubsByTopic()`. For every stored subscription, `sendContent()` is called in parallel to broadcast the content to all relevant subscribers.
+
+<div align="center">
+<img src="./publish-success.png" width="700">
+</div>
+
+
+### Demo
+#### Requirements
+* Docker
+
+#### Run demo
+
+The local demo uses `modfin/websub-client:latest` as a subscriber and runs entirely in Docker containers. To run the demo:
+```
+docker compose up --build
+```
+
+#### Live version
+A live version of the hub (without the subscriber container) is running on https://subhub.henriknorrman.com.
+
+

docker-compose.yml

@@ -38,8 +38,8 @@ services:
  - subscriber_data:/var/lib/postgresql/data
  healthcheck:
  test: ["CMD-SHELL", "sh -c 'pg_isready -U postgres -d hub'"]
- interval: 3s
- timeout: 2s
+ interval: 5s
+ timeout: 3s
  retries: 5
 
 

hub/db.go

@@ -12,7 +12,7 @@ import (
 //
 // Holds a pointer to the postgres connection pool
 type hubStore struct {
- store *pgxpool.Pool
+ db *pgxpool.Pool
 }
 
 // Initiate the database instance
@@ -24,15 +24,15 @@ func (h *hubStore) init() {
  fmt.Println("Connecting to database...")
  pgURL := "postgres://postgres:password@database:5432/hub"
  var err error
- h.store, err = pgxpool.Connect(context.Background(), pgURL)
+ h.db, err = pgxpool.Connect(context.Background(), pgURL)
  if err != nil {
- h.store.Close()
+ h.db.Close()
  fmt.Println(err)
  os.Exit(1)
  }
 
  // Query to create subscription table
- _, err = h.store.Exec(context.Background(), `
+ _, err = h.db.Exec(context.Background(), `
  CREATE TABLE IF NOT EXISTS subscription (
  id SERIAL PRIMARY KEY,
  subscriber VARCHAR(128) NOT NULL,
@@ -42,7 +42,7 @@ func (h *hubStore) init() {
  UNIQUE (subscriber, topic)
  )`)
  if err != nil {
- h.store.Close()
+ h.db.Close()
  fmt.Println(err)
  os.Exit(1)
  }
@@ -55,7 +55,7 @@ func (h *hubStore) init() {
 // "MUST allow subscribers to re-request already active subscriptions."
 func (h *hubStore) addSubscriber(callback string, secret string, topic string, timestamp int64) {
  // Query to add new subscriber. Ignores old/delayed subscriptions and updates timestamp on resubscriptions.
- _, err := h.store.Exec(context.Background(), `
+ _, err := h.db.Exec(context.Background(), `
  INSERT INTO subscription
  (subscriber, secret, topic, timestamp) 
  VALUES 
@@ -73,7 +73,7 @@ func (h *hubStore) addSubscriber(callback string, secret string, topic string, t
  END;
  `, callback, secret, topic, timestamp)
  if err != nil {
- h.store.Close()
+ h.db.Close()
  fmt.Println(err)
  os.Exit(1)
  }
@@ -85,12 +85,12 @@ func (h *hubStore) addSubscriber(callback string, secret string, topic string, t
 // subscriptions keep their other topic subscriptions.
 func (h *hubStore) removeSubscriber(callback string, topic string) {
  // Query to remove a subscription
- _, err := h.store.Exec(context.Background(), `
+ _, err := h.db.Exec(context.Background(), `
  DELETE FROM subscription
  WHERE subscriber=$1 AND topic=$2
  `, callback, topic)
  if err != nil {
- h.store.Close()
+ h.db.Close()
  fmt.Println(err)
  os.Exit(1)
  }
@@ -101,13 +101,13 @@ func (h *hubStore) removeSubscriber(callback string, topic string) {
 // Return all topic subscriptions in the database
 func (h *hubStore) getAllSubsByTopic(topic string) []subscription {
  // Query to fetch all subscriptions
- rows, err := h.store.Query(context.Background(), `
+ rows, err := h.db.Query(context.Background(), `
  SELECT subscriber, secret, topic 
  FROM subscription 
  WHERE topic=$1
  `, topic)
  if err != nil {
- h.store.Close()
+ h.db.Close()
  fmt.Println(err)
  os.Exit(1)
  }