Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error 500 on Broker page #1357

Open
scorpp opened this issue Feb 27, 2024 · 7 comments
Open

Error 500 on Broker page #1357

scorpp opened this issue Feb 27, 2024 · 7 comments
Labels

Comments

@scorpp
Copy link

scorpp commented Feb 27, 2024

Describe the bug
Flower is deployed on Heroku and makes use of Heroku Redis as broker.

Heroku Redis uses self-signed certificates (hence the broker_use_ssl = {"ssl_cert_reqs": "none"} part in config below). And Workers / Tasks tabs work fine. Just the broker part is broken.

To Reproduce
Steps to reproduce the behavior:

  1. Go to 'Broker' tab
  2. See error
   [W 240227 06:47:49 connection:268] Secure redis scheme specified (rediss) with no ssl options, defaulting to insecure SSL behaviour.
   [W 240227 06:47:49 connection:268] Secure redis scheme specified (rediss) with no ssl options, defaulting to insecure SSL behaviour.
   [E 240227 06:47:49 broker:31] Unable to get queues: 'Error 1 connecting to ec2-52-45-116-60.compute-1.amazonaws.com:6379. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1006).'
   [W 240227 06:47:49 connection:268] Secure redis scheme specified (rediss) with no ssl options, defaulting to insecure SSL behaviour.
   [E 240227 06:47:49 web:1875] Uncaught exception GET /broker ([10.1.39.250](https://my.papertrailapp.com/systems/flower-heroku/events?q=%2210.1.39.250%22&focus=1699101004400787463&selected=1699101004400787463))
       HTTPServerRequest(protocol='http', host='flower-heroku.herokuapp.com', method='GET', uri='/broker', version='HTTP/1.1', remote_ip='[10.1.39.250](https://my.papertrailapp.com/systems/flower-heroku/events?q=%2210.1.39.250%22&focus=1699101004400787464&selected=1699101004400787464)')
       Traceback (most recent call last):
         File "/app/.heroku/python/lib/python3.11/site-packages/tornado/web.py", line 1790, in _execute
           result = await result
                    ^^^^^^^^^^^^
         File "/app/.heroku/python/lib/python3.11/site-packages/flower/views/broker.py", line 35, in get
           queues=queues)
                  ^^^^^^
       UnboundLocalError: cannot access local variable 'queues' where it is not associated with a value

Expected behavior
Broker info is shown

Screenshots
image

System information

$ python -c 'from flower.utils import bugreport; print(bugreport())'
flower   -> flower:2.0.1 tornado:6.4 humanize:4.9.0
software -> celery:5.3.6 (emerald-rush) kombu:5.3.5 py:3.11.8
            billiard:4.2.0 py-amqp:5.2.0
platform -> system:Linux arch:64bit, ELF
            kernel version:5.15.0-1053-aws imp:CPython
loader   -> celery.loaders.app.AppLoader
settings -> transport:amqp results:disabled

deprecated_settings: None

Command line

celery --broker=$REDIS_URL/1 flower --port=$PORT --config flowerconfig

floweconfig.py

import os

basic_auth  = [os.environ['BASIC_AUTH']]

broker_url  = os.environ['REDIS_URL'] + '/1'
broker_use_ssl = {"ssl_cert_reqs": "none"}

timezone    = os.getenv('TIMEZONE', 'US/Eastern')
@scorpp scorpp added the bug label Feb 27, 2024
@jspablo
Copy link

jspablo commented Apr 7, 2024

Facing same error with RabbitMQ as broker when visiting Broker page

@jorge-menjivar
Copy link

Facing the same issue with Redis on Redis Cloud

@Legrems
Copy link

Legrems commented Jun 19, 2024

Ensure that the broker_api is set accordingly to your config:
example:

--broker="amqp://guest:guest@127.0.0.1:5672/

and

--broker-api="http://127.0.0.1:15672/api/

Note that the port change, as well as the auth isn't present in the settings (if you have one)

@scorpp
Copy link
Author

scorpp commented Jun 19, 2024

i'm not using AMQP. my broker is Redis

@fjcapdevila
Copy link
Contributor

Same problem here using Flower flower==2.0.1 with Sentinel+Redis as broker

[E 240723 14:46:33 web:1875] Uncaught exception GET /flower/broker (1.2.3.4)
    HTTPServerRequest(protocol='http', host='flower.example.com', method='GET', uri='/flower/broker', version='HTTP/1.1', remote_ip='1.2.3.4')
    Traceback (most recent call last):
      File "/home/vocalo/.venv/lib/python3.10/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
      File "/home/vocalo/.venv/lib/python3.10/site-packages/flower/views/broker.py", line 35, in get
        queues=queues)
    UnboundLocalError: local variable 'queues' referenced before assignment

@fjcapdevila
Copy link
Contributor

Quick update:
I've also found logs like this:

Unable to get queues: 'unsupported operand type(s) for +: 'NoneType' and 'str''

I was able to fix the issue by setting global_keyprefix = ''
I guess global_keyprefix is set to None by default and it makes Broker page fail.

@Afoucaul
Copy link

Afoucaul commented Oct 24, 2024

Same problem here using Flower flower==2.0.1 with Sentinel+Redis as broker

[E 240723 14:46:33 web:1875] Uncaught exception GET /flower/broker (1.2.3.4)
    HTTPServerRequest(protocol='http', host='flower.example.com', method='GET', uri='/flower/broker', version='HTTP/1.1', remote_ip='1.2.3.4')
    Traceback (most recent call last):
      File "/home/vocalo/.venv/lib/python3.10/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
      File "/home/vocalo/.venv/lib/python3.10/site-packages/flower/views/broker.py", line 35, in get
        queues=queues)
    UnboundLocalError: local variable 'queues' referenced before assignment

I had the same issue, and realized I had --broker-api=amqp://guest:guest@rabbitmq:15672/api, which I replaced with --broker-api=http://guest:guest@rabbitmq:15672/api (was amqp://, should have been http://) and the issue went away.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants