Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix master and improve CI #22

Closed
wants to merge 3 commits into from
Closed

Conversation

ynezz
Copy link
Member

@ynezz ynezz commented Dec 22, 2023

Currently master is unable to start due to some Twisted dependency hell. I've integrated a workaround which should be reverted once we bump to buildbot >= 3.10

To prevent such regressions again in future I've improved the CI testing with a simple, but extensible cram based test suite.

Example of the cram test output with current master failure:

tests/cram/master/01-logs.t: failed
--- tests/cram/master/01-logs.t
+++ tests/cram/master/01-logs.t.err
@@ -5,3 +5,35 @@
   creating /master/master.cfg.sample
   creating database (sqlite:///state.sqlite)
   buildmaster configured in /master
+  Unhandled error in Deferred:
+  
+  Traceback (most recent call last):
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/scripts/create_master.py", line 84, in createDB
+      master = BuildMaster(config['basedir'])
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/master.py", line 102, in __init__
+      self._services_d = self.create_child_services()
+    File "/usr/local/lib/python3.9/dist-packages/twisted/internet/defer.py", line 2256, in unwindGenerator
+      return _cancellableInlineCallbacks(gen)
+    File "/usr/local/lib/python3.9/dist-packages/twisted/internet/defer.py", line 2168, in _cancellableInlineCallbacks
+      _inlineCallbacks(None, gen, status, _copy_context())
+  --- <exception caught here> ---
+    File "/usr/local/lib/python3.9/dist-packages/twisted/internet/defer.py", line 2000, in _inlineCallbacks
+      result = context.run(gen.send, result)
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/master.py", line 188, in create_child_services
+      self.www = wwwservice.WWWService()
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/www/service.py", line 196, in __init__
+      self.apps = get_plugins('www', None, load_now=True)
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/plugins/db.py", line 356, in get_plugins
+      return _DB.add_namespace(namespace, interface, check_extras, load_now)
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/plugins/db.py", line 306, in add_namespace
+      tempo.load()
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/plugins/db.py", line 242, in load
+      self._tree.load()
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/plugins/db.py", line 112, in load
+      child.load()
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/plugins/db.py", line 45, in load
+      self._value = self._loader(self._entry)
+    File "/usr/local/lib/python3.9/dist-packages/buildbot/plugins/db.py", line 214, in _load_entry
+      raise PluginDBError('Requirements are not satisfied '
+  buildbot.errors.PluginDBError: Requirements are not satisfied for buildbot.www:base: The 'zope-interface>=5' distribution was not found and is required by Twisted
+  
# Ran 1 tests, 0 skipped, 1 failed.

Get rid of following annyoing errors during testing:

  chown: cannot access '/config': No such file or directory
  chown: cannot access '/certs': No such file or directory
  chmod: cannot access '/config': No such file or directory
  chmod: cannot access '/certs': No such file or directory

As those dirs doesn't exist in the container, they're being provided as
volumes during deployment.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
Currently we've broken master container but we're not aware about it as
current tests are very lame, so lets improve it a bit with some more
reliable, extensible solution.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
@ynezz
Copy link
Member Author

ynezz commented Dec 22, 2023

buildbot.errors.PluginDBError: Requirements are not satisfied for buildbot.www:base: The 'zope-interface>=5' distribution was not found and is required by Twisted

Quite puzzled:

root@e3219f2dd8ae:/# pip install zope-interface
Requirement already satisfied: zope-interface in /usr/local/lib/python3.9/dist-packages (6.1)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from zope-interface) (52.0.0)
root@e3219f2dd8ae:/# pip freeze | grep zope
zope.interface==6.1

@ynezz
Copy link
Member Author

ynezz commented Dec 22, 2023

Seems to be fixed in buildbot/buildbot@94e2d59

Force twisted==22.10.0 which fixes following buildbot master startup
issue:

Unhandled error in Deferred:

Traceback (most recent call last):
  File "/.../site-packages/buildbot/scripts/create_master.py", line 84, in createDB
    master = BuildMaster(config['basedir'])
  File "/.../site-packages/buildbot/master.py", line 102, in __init__
    self._services_d = self.create_child_services()
  File "/.../site-packages/twisted/internet/defer.py", line 2245, in unwindGenerator
    return _cancellableInlineCallbacks(gen)
  File "/.../site-packages/twisted/internet/defer.py", line 2157, in _cancellableInlineCallbacks
    _inlineCallbacks(None, gen, status, _copy_context())
--- <exception caught here> ---
  File "/.../site-packages/twisted/internet/defer.py", line 1997, in _inlineCallbacks
    result = context.run(gen.send, result)
  File "/.../site-packages/buildbot/master.py", line 188, in create_child_services
    self.www = wwwservice.WWWService()
  File "/.../site-packages/buildbot/www/service.py", line 196, in __init__
    self.apps = get_plugins('www', None, load_now=True)
  File "/.../site-packages/buildbot/plugins/db.py", line 356, in get_plugins
    return _DB.add_namespace(namespace, interface, check_extras, load_now)
  File "/.../site-packages/buildbot/plugins/db.py", line 306, in add_namespace
    tempo.load()
  File "/.../site-packages/buildbot/plugins/db.py", line 242, in load
    self._tree.load()
  File "/.../site-packages/buildbot/plugins/db.py", line 112, in load
    child.load()
  File "/.../site-packages/buildbot/plugins/db.py", line 45, in load
    self._value = self._loader(self._entry)
  File "/.../site-packages/buildbot/plugins/db.py", line 214, in _load_entry
    raise PluginDBError('Requirements are not satisfied '
buildbot.errors.PluginDBError: Requirements are not satisfied for buildbot.www:base:
The 'zope-interface>=5' distribution was not found and is required by Twisted

References: buildbot/buildbot@94e2d59
Signed-off-by: Petr Štetiar <ynezz@true.cz>
@openwrt-bot openwrt-bot deleted the ynezz/fix-master-improve-ci branch December 22, 2023 10:42
@ynezz ynezz restored the ynezz/fix-master-improve-ci branch December 22, 2023 10:43
@ynezz ynezz reopened this Dec 22, 2023
@ynezz ynezz closed this Dec 22, 2023
@ynezz ynezz deleted the ynezz/fix-master-improve-ci branch December 22, 2023 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants