Skip to content

Commit

Permalink
support hashed web passwd
Browse files Browse the repository at this point in the history
  • Loading branch information
@umputun
umputun committed Dec 9, 2024
1 parent b1e2bbc commit 4585647
Show file tree
Hide file tree
Showing 162 changed files with 15,961 additions and 813 deletions.
25 changes: 19 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ Success! The new status is: DISABLED. /help

```
--admin.group= admin group name, or channel id [$ADMIN_GROUP]
--disable-admin-spam-forward disable forwarding spam messages to admin group [$DISABLE_ADMIN_SPAM_FORWARD]
--disable-admin-spam-forward disable handling messages forwarded to admin group as spam [$DISABLE_ADMIN_SPAM_FORWARD]
--testing-id= testing ids, allow bot to reply to them [$TESTING_ID]
--history-duration= history duration (default: 24h) [$HISTORY_DURATION]
--history-min-size= history minimal size to keep (default: 1000) [$HISTORY_MIN_SIZE]
Expand All @@ -247,12 +247,11 @@ Success! The new status is: DISABLED. /help
--min-msg-len= min message length to check (default: 50) [$MIN_MSG_LEN]
--max-emoji= max emoji count in message, -1 to disable check (default: 2) [$MAX_EMOJI]
--min-probability= min spam probability percent to ban (default: 50) [$MIN_PROBABILITY]
--multi-lang= number of words in different languages to consider as spam, 0 to disable (default: 0) [$MULTI_LANG]
--multi-lang= number of words in different languages to consider as spam (default: 0) [$MULTI_LANG]
--paranoid paranoid mode, check all messages [$PARANOID]
--first-messages-count= number of first messages to check (default: 1) [$FIRST_MESSAGES_COUNT]
--training training mode, passive spam detection only [$TRAINING]
--soft-ban soft ban mode, restrict user actions but not ban [$SOFT_BAN]
--dry dry mode, no bans [$DRY]
--dbg debug mode [$DEBUG]
--tg-dbg telegram debug mode [$TG_DEBUG]
Expand All @@ -276,8 +275,8 @@ cas:
meta:
--meta.links-limit= max links in message, disabled by default (default: -1) [$META_LINKS_LIMIT]
--meta.image-only enable image only check [$META_IMAGE_ONLY]
--meta.video-only enable video only check [$META_VIDEO_ONLY]
--meta.links-only enable links only check [$META_LINKS_ONLY]
--meta.video-only enable video only check [$META_VIDEO_ONLY]
openai:
--openai.token= openai token, disabled if not set [$OPENAI_TOKEN]
Expand Down Expand Up @@ -306,11 +305,11 @@ server:
--server.enabled enable web server [$SERVER_ENABLED]
--server.listen= listen address (default: :8080) [$SERVER_LISTEN]
--server.auth= basic auth password for user 'tg-spam' (default: auto) [$SERVER_AUTH]
--server.auth-hash= basic auth password hash for user 'tg-spam' [$SERVER_AUTH_HASH]
Help Options:
-h, --help Show this help message
```

### Application Options in details
Expand Down Expand Up @@ -355,7 +354,21 @@ Pls note: Missed spam messages forwarded to the admin chat will be removed from

The bot can be run with a webapi server. This is useful for integration with other tools. The server is disabled by default, to enable it pass `--server.enabled [$SERVER_ENABLED]`. The server will listen on the port specified by `--server.listen [$SERVER_LISTEN]` parameter (default is `:8080`).

By default, the server is protected by basic auth with user `tg-spam` and randomly generated password. This password is printed to the console on startup. If user wants to set a custom auth password, it can be done with `--server.auth [$SERVER_AUTH]` parameter. Setting it to empty string will disable basic auth protection.
By default, the server is protected by basic auth with user `tg-spam` and randomly generated password. This password and the hash are printed to the console on startup. If user wants to set a custom auth password, it can be done with `--server.auth [$SERVER_AUTH]` parameter. Setting it to empty string will disable basic auth protection.

For better security, it is possible to set the password hash instead, with `--server.auth-hash [$SERVER_AUTH_HASH]` parameter. The hash should be generated with any command what can make bcrypt hash. For example, the following command will generate a hash for the password `your_password`: `htpasswd -n -B -b tg-spam your_password | cut -d':' -f2`

alternatively, it is possible to use one of the following commands to generate the hash:
```
htpasswd -bnBC 10 "" your_password | tr -d ':\n'
mkpasswd --method=bcrypt your_password
openssl passwd -apr1 your_password
```

In case if both `--server.auth` and `--server.auth-hash` are set, the hash will be used.

```bash

It is truly a **bad idea** to run the server without basic auth protection, as it allows adding/removing users and updating spam samples to anyone who knows the endpoint. The only reason to run it without protection is inside the trusted network or for testing purposes. Exposing the server directly to the internet is not recommended either, as basic auth is not secure enough if used without SSL. It is better to use a reverse proxy with TLS termination in front of the server.

Expand Down
16 changes: 14 additions & 2 deletions app/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (
tbapi "github.com/OvyFlash/telegram-bot-api"
"github.com/fatih/color"
"github.com/go-pkgz/lgr"
"github.com/go-pkgz/rest"
"github.com/jmoiron/sqlx"
"github.com/sashabaranov/go-openai"
"github.com/umputun/go-flags"
Expand Down Expand Up @@ -110,6 +111,7 @@ type options struct {
Enabled bool `long:"enabled" env:"ENABLED" description:"enable web server"`
ListenAddr string `long:"listen" env:"LISTEN" default:":8080" description:"listen address"`
AuthPasswd string `long:"auth" env:"AUTH" default:"auto" description:"basic auth password for user 'tg-spam'"`
AuthHash string `long:"auth-hash" env:"AUTH_HASH" default:"" description:"basic auth password hash for user 'tg-spam'"`
} `group:"server" namespace:"server" env-namespace:"SERVER"`

Training bool `long:"training" env:"TRAINING" description:"training mode, passive spam detection only"`
Expand Down Expand Up @@ -146,9 +148,14 @@ func main() {
}

masked := []string{opts.Telegram.Token, opts.OpenAI.Token}
if opts.Server.AuthPasswd != "auto" && opts.Server.AuthPasswd != "" { // auto passwd should not be masked as we print it
if opts.Server.AuthPasswd != "auto" && opts.Server.AuthPasswd != "" {
// auto passwd should not be masked as we print it
masked = append(masked, opts.Server.AuthPasswd)
}
if opts.Server.AuthHash != "" {
masked = append(masked, opts.Server.AuthHash)
}

setupLog(opts.Dbg, masked...)

log.Printf("[DEBUG] options: %+v", opts)
Expand Down Expand Up @@ -341,7 +348,11 @@ func activateServer(ctx context.Context, opts options, sf *bot.SpamFilter, loc *
if err != nil {
return fmt.Errorf("can't generate random password, %w", err)
}
log.Printf("[WARN] generated basic auth password for user tg-spam: %q", authPassswd)
authHash, err := rest.GenerateBcryptHash(authPassswd)
if err != nil {
return fmt.Errorf("can't generate bcrypt hash for password, %w", err)
}
log.Printf("[WARN] generated basic auth password for user tg-spam: %q, bcrypt hash: %s", authPassswd, authHash)
}

// make store and load approved users
Expand Down Expand Up @@ -385,6 +396,7 @@ func activateServer(ctx context.Context, opts options, sf *bot.SpamFilter, loc *
Locator: loc,
DetectedSpam: detectedSpamStore,
AuthPasswd: authPassswd,
AuthHash: opts.Server.AuthHash,
Version: revision,
Dbg: opts.Dbg,
Settings: settings,
Expand Down
9 changes: 7 additions & 2 deletions app/webapi/webapi.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ type Config struct {
DetectedSpam DetectedSpam // detected spam accessor
Locator Locator // locator for user info
AuthPasswd string // basic auth password for user "tg-spam"
AuthHash string // basic auth hash for user "tg-spam". If both AuthPasswd and AuthHash are provided, AuthHash is used
Dbg bool // debug mode
Settings Settings // application settings
}
Expand Down Expand Up @@ -130,9 +131,13 @@ func (s *Server) Run(ctx context.Context) error {
router.Use(tollbooth_chi.LimitHandler(tollbooth.NewLimiter(50, nil)))
router.Use(rest.SizeLimit(1024 * 1024)) // 1M max request size

if s.AuthPasswd != "" {
if s.AuthPasswd != "" || s.AuthHash != "" {
log.Printf("[INFO] basic auth enabled for webapi server")
router.Use(rest.BasicAuthWithPrompt("tg-spam", s.AuthPasswd))
if s.AuthHash != "" {
router.Use(rest.BasicAuthWithBcryptHashAndPrompt("tg-spam", s.AuthHash))
} else {
router.Use(rest.BasicAuthWithPrompt("tg-spam", s.AuthPasswd))
}
} else {
log.Printf("[WARN] basic auth disabled, access to webapi is not protected")
}
Expand Down
166 changes: 112 additions & 54 deletions app/webapi/webapi_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
Expand All @@ -14,6 +15,7 @@ import (
"time"

"github.com/go-chi/chi/v5"
"github.com/go-pkgz/rest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

Expand Down Expand Up @@ -63,64 +65,120 @@ func TestServer_RunAuth(t *testing.T) {
}
mockSpamFilter := &mocks.SpamFilterMock{}

srv := NewServer(Config{ListenAddr: ":9877", Version: "dev", Detector: mockDetector, SpamFilter: mockSpamFilter, AuthPasswd: "test"})
done := make(chan struct{})
go func() {
err := srv.Run(ctx)
assert.NoError(t, err)
close(done)
}()
time.Sleep(100 * time.Millisecond)

t.Run("ping", func(t *testing.T) {
resp, err := http.Get("http://localhost:9877/ping")
assert.NoError(t, err)
t.Log(resp)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode) // no auth on ping
})

t.Run("check unauthorized, no basic auth", func(t *testing.T) {
resp, err := http.Get("http://localhost:9877/check")
assert.NoError(t, err)
t.Log(resp)
defer resp.Body.Close()
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
})
hashedPassword, err := rest.GenerateBcryptHash("test")
require.NoError(t, err)
t.Logf("hashed password: %s", string(hashedPassword))

t.Run("check authorized", func(t *testing.T) {
reqBody, err := json.Marshal(map[string]string{
"msg": "spam example",
"user_id": "user123",
})
require.NoError(t, err)
req, err := http.NewRequest("POST", "http://localhost:9877/check", bytes.NewBuffer(reqBody))
assert.NoError(t, err)
req.SetBasicAuth("tg-spam", "test")
resp, err := http.DefaultClient.Do(req)
assert.NoError(t, err)
t.Log(resp)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
})
// run two servers - one with plain password and one with bcrypt hash
tests := []struct {
name string
srv *Server
port string
authType string
password string
useHashed bool
}{
{
name: "plain password auth",
srv: NewServer(Config{
ListenAddr: ":9877",
Version: "dev",
Detector: mockDetector,
SpamFilter: mockSpamFilter,
AuthPasswd: "test",
}),
port: "9877",
authType: "plain",
password: "test",
},
{
name: "bcrypt hash auth",
srv: NewServer(Config{
ListenAddr: ":9878",
Version: "dev",
Detector: mockDetector,
SpamFilter: mockSpamFilter,
AuthHash: string(hashedPassword),
}),
port: "9878",
authType: "hash",
password: "test",
useHashed: true,
},
}

t.Run("wrong basic auth", func(t *testing.T) {
reqBody, err := json.Marshal(map[string]string{
"msg": "spam example",
"user_id": "user123",
var doneChannels []chan struct{}
for _, tc := range tests {
done := make(chan struct{})
doneChannels = append(doneChannels, done)
t.Run(tc.authType, func(t *testing.T) {
go func() {
err := tc.srv.Run(ctx)
assert.NoError(t, err)
close(done)
}()
time.Sleep(100 * time.Millisecond)

t.Run("ping", func(t *testing.T) {
resp, err := http.Get(fmt.Sprintf("http://localhost:%s/ping", tc.port))
assert.NoError(t, err)
t.Log(resp)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode) // no auth on ping
})

t.Run("check unauthorized, no basic auth", func(t *testing.T) {
resp, err := http.Get(fmt.Sprintf("http://localhost:%s/check", tc.port))
assert.NoError(t, err)
t.Log(resp)
defer resp.Body.Close()
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
if tc.useHashed {
assert.Equal(t, `Basic realm="restricted", charset="UTF-8"`, resp.Header.Get("WWW-Authenticate"))
}
})

t.Run("check authorized", func(t *testing.T) {
reqBody, err := json.Marshal(map[string]string{
"msg": "spam example",
"user_id": "user123",
})
require.NoError(t, err)
req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%s/check", tc.port), bytes.NewBuffer(reqBody))
assert.NoError(t, err)
req.SetBasicAuth("tg-spam", tc.password)
resp, err := http.DefaultClient.Do(req)
assert.NoError(t, err)
t.Log(resp)
defer resp.Body.Close()
assert.Equal(t, http.StatusOK, resp.StatusCode)
})

t.Run("wrong basic auth", func(t *testing.T) {
reqBody, err := json.Marshal(map[string]string{
"msg": "spam example",
"user_id": "user123",
})
require.NoError(t, err)
req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%s/check", tc.port), bytes.NewBuffer(reqBody))
assert.NoError(t, err)
req.SetBasicAuth("tg-spam", "bad")
resp, err := http.DefaultClient.Do(req)
assert.NoError(t, err)
t.Log(resp)
defer resp.Body.Close()
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
if tc.useHashed {
assert.Equal(t, `Basic realm="restricted", charset="UTF-8"`, resp.Header.Get("WWW-Authenticate"))
}
})
})
require.NoError(t, err)
req, err := http.NewRequest("POST", "http://localhost:9877/check", bytes.NewBuffer(reqBody))
assert.NoError(t, err)
req.SetBasicAuth("tg-spam", "bad")
resp, err := http.DefaultClient.Do(req)
assert.NoError(t, err)
t.Log(resp)
defer resp.Body.Close()
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode)
})
}
cancel()
<-done
// wait for all servers to complete
for _, done := range doneChannels {
<-done
}
}

func TestServer_routes(t *testing.T) {
Expand Down
15 changes: 8 additions & 7 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,15 @@ require (
github.com/fsnotify/fsnotify v1.8.0
github.com/go-chi/chi/v5 v5.1.0
github.com/go-pkgz/lgr v0.11.1
github.com/go-pkgz/rest v1.19.0
github.com/go-pkgz/rest v1.20.1
github.com/hashicorp/go-multierror v1.1.1
github.com/jmoiron/sqlx v1.4.0
github.com/sandwich-go/gpt3-encoder v0.0.0-20230203030618-cd99729dd0dd
github.com/sashabaranov/go-openai v1.32.5
github.com/stretchr/testify v1.9.0
github.com/stretchr/testify v1.10.0
github.com/umputun/go-flags v1.5.1
gopkg.in/natefinch/lumberjack.v2 v2.2.1
modernc.org/sqlite v1.33.1
modernc.org/sqlite v1.34.2
)

require (
Expand All @@ -35,12 +35,13 @@ require (
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/samber/lo v1.47.0 // indirect
golang.org/x/exp v0.0.0-20241009180824-f66d83c29e7c // indirect
golang.org/x/sys v0.26.0 // indirect
golang.org/x/text v0.19.0 // indirect
golang.org/x/crypto v0.30.0 // indirect
golang.org/x/exp v0.0.0-20241204233417-43b7b7cde48d // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/text v0.21.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/gc/v3 v3.0.0-20241004144649-1aea3fae8852 // indirect
modernc.org/libc v1.61.0 // indirect
modernc.org/libc v1.61.4 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
Expand Down
Loading

0 comments on commit 4585647

Please sign in to comment.