web-crawler

Presentation Link

Prerequisites

• Go 1.16+
• MySQL
• redis download link v5.0.7
• air (for live reload) installation link

Migrations

• Install

go install -tags 'mysql' github.com/golang-migrate/migrate/v4/cmd/migrate@latest

• Create Migrations

migrate create -ext sql -dir ./migrations <MIGRATION_NAME>

• Up Migrations

migrate -path "./migrations" -database "mysql://root:YOUR_MYSQL_PASSWORD@/webcrawler" up

• Down Migrations

migarte -path "./migrations"  -database "mysql://root:YOUR_MYSQL_PASSWORD@/webcrawler" down

• start server

  • development

    air

  • production

    go run main.go --threads <no of threads>

Docker Setup

• Install docker and docker-compose
• Run cp .docker.sample.env .docker.env, fill the env varaiables
• Run docker-compose up to start the services