Update dependency twisted to v24 [SECURITY] #286

redhat-renovate-bot · 2024-11-27T16:08:05Z

This PR contains the following updates:

Package	Update	Change
twisted (changelog)	major	`==23.10.0` -> `==24.7.0`

GitHub Vulnerability Alerts

CVE-2024-41671

Summary

The HTTP 1.0 and 1.1 server provided by twisted.web could process pipelined HTTP requests out-of-order, possibly resulting in information disclosure.

PoC

Start a fresh Debian container:

docker run --workdir /repro --rm -it debian:bookworm-slim

Install twisted and its dependencies:

apt -y update && apt -y install ncat git python3 python3-pip \
    && git clone --recurse-submodules https://github.com/twisted/twisted \
    && cd twisted \
    && pip3 install --break-system-packages .

Run a twisted.web HTTP server that echos received requests' methods. e.g., the following:

from twisted.web import server, resource
from twisted.internet import reactor

class TheResource(resource.Resource):
    isLeaf = True

    def render_GET(self, request) -> bytes:
        return b"GET"

    def render_POST(self, request) -> bytes:
        return b"POST"

site = server.Site(TheResource())
reactor.listenTCP(80, site)
reactor.run()

Send it a POST request with a chunked message body, pipelined with another POST request, wait a second, then send a GET request on the same connection:

(printf 'POST / HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n0\r\n\r\nPOST / HTTP/1.1\r\nContent-Length: 0\r\n\r\n'; sleep 1; printf 'GET / HTTP/1.1\r\n\r\n'; sleep 1) | nc localhost 80

Observe that the responses arrive out of order:

HTTP/1.1 200 OK
Server: TwistedWeb/24.3.0.post0
Date: Tue, 09 Jul 2024 06:19:41 GMT
Content-Length: 5
Content-Type: text/html

POST
HTTP/1.1 200 OK
Server: TwistedWeb/24.3.0.post0
Date: Tue, 09 Jul 2024 06:19:42 GMT
Content-Length: 4
Content-Type: text/html

GET
HTTP/1.1 200 OK
Server: TwistedWeb/24.3.0.post0
Date: Tue, 09 Jul 2024 06:19:42 GMT
Content-Length: 5
Content-Type: text/html

POST

Impact

See GHSA-xc8x-vp79-p3wm. Further, for instances of twisted.web HTTP servers deployed behind reverse proxies that implement connection pooling, it may be possible for remote attackers to receive responses intended for other clients of the twisted.web server.

CVE-2024-41810

Summary

The twisted.web.util.redirectTo function contains an HTML injection vulnerability. If application code allows an attacker to control the redirect URL this vulnerability may result in Reflected Cross-Site Scripting (XSS) in the redirect response HTML body.

Details

Twisted’s redirectTo function generates an HTTP 302 Redirect response. The response contains an HTML body, built for exceptional cases where the browser doesn’t properly handle the redirect, allowing the user to click a link, navigating them to the specified destination.

The function reflects the destination URL in the HTML body without any output encoding.

# https://github.com/twisted/twisted/blob/trunk/src/twisted/web/_template_util.py#L88
def redirectTo(URL: bytes, request: IRequest) -> bytes:
    # ---snip---
    content = b"""
<html>
    <head>
        <meta http-equiv=\"refresh\" content=\"0;URL=%(url)s\">
    </head>
    <body bgcolor=\"#FFFFFF\" text=\"#&#8203;000000\">
    <a href=\"%(url)s\">click here</a>
    </body>
</html>
""" % {
        b"url": URL
    }
    return content

If an attacker has full or partial control over redirect location due to an application bug, also known as an “Open Redirect”, they may inject arbitrary HTML into the response’s body, ultimately leading to an XSS attack.

It’s worth noting that the issue is known to maintainers and tracked with GitHub Issue#9839. The issue description, however, does not make any mention of exploitability and simply states: “…Browsers don't seem to actually render that page…”

PoC

The issue can be reproduced by running the following Twisted-based HTTP server locally:

from twisted.web import server, resource
from twisted.internet import reactor
from twisted.web.util import redirectTo

class Simple(resource.Resource):
    isLeaf = True
    def render_GET(self, request):
        url = request.args[b'url'][0]  # <-- open redirect
        return redirectTo(url, request)

site = server.Site(Simple())
reactor.listenTCP(9009, site)
reactor.run()

Once running, navigate to the following URL: http://127.0.0.1:9009?url=ws://example.com/"><script>alert(document.location)</script>, and verify that the “alert” dialog was displayed.

Note: Due to the different ways browsers validate the redirect Location header, this attack is possible only in Firefox. All other tested browsers will display an error message to the user and will not render the HTML body.

Impact

If successfully exploited, the issue will allow malicious JavaScript to run in the context of the victim's session. This will in turn lead to unauthorized access/modification to victim's account and information associated with it, or allow for unauthorized operations to be performed within the context of the victim's session.

Release Notes

twisted/twisted (twisted)

`v24.7.0`: Twisted 24.7.0

Compare Source

Twisted 24.7.0 (2024-08-08)

24.7.0.rc2 fixed an unreleased regression caused by PR 12109. (#12279)
No other changes since 24.7.0.rc2

Features

twisted.protocols.ftp now supports the IPv6 extensions defined in RFC 2428. (#9645)
twisted.internet.defer.inlineCallbacks can now yield a coroutine. (#9972)
twisted.python._shellcomp.ZshArgumentsGenerator was updated for Python 3.13. (#12065)
twisted.web.wsgi request environment now contains the peer port number as REMOTE_PORT. (#12096)
twisted.internet.defer.Deferred.callback() and twisted.internet.defer.Deferred.addCallbacks() no longer use assert to check the type of the arguments. You should now use type checking to validate your code. These changes were done to reduce the CPU usage. (#12122)
Added two new methods, twisted.logger.Logger.failuresHandled and twisted.logger.Logger.failureHandler, which allow for more concise and convenient handling of exceptions when dispatching out to application code. The former can arbitrarily customize failure handling at the call site, and the latter can be used for performance-sensitive cases where no additional information needs to be logged. (#12188)
twisted.internet.defer.Deferred.addCallback now runs about 10% faster. (#12223)
twisted.internet.defer.Deferred error handling is now faster, taking 40% less time to run. (#12227)

Bugfixes

Fixed unreleased regression caused by PR #12109. (#12279)
twisted.internet.ssl.Certificate.repr can now handle certificates without a common name (CN) in the certificate itself or the signing CA. (#5851)
Type annotations have been added to twisted.conch.interfaces.IKnownHostEntry and its implementations, twisted.conch.client.knownhosts.PlainHost and twisted.conch.client.knownhosts.HashedHost, correcting a variety of type confusion issues throughout the conch client code. (#9713)
twisted.python.failure.Failure once again utilizes the custom pickling logic it used to in the past. (#12112)
twisted.conch.client.knownhosts.KnownHostsFile.verifyHostKey no longer logs an exception when automatically adding an IP address host key, which means the interactive conch command-line no longer will either. (#12141)

Improved Documentation

The IRC server example found in the documentation was updated for readability. (#12097)
Remove contextvars from list of optional dependencies. (#12128)
The documentation for installing Twisted was moved into a single page. (#12145)
The project's compatibility policy now clearly indicates that the GitHub Actions test matrix defines the supported platforms. (#12167)
Updated imap4client.py example, it no longer references Python 2. (#12252)

Deprecations and Removals

twisted.internet.defer.returnValue has been deprecated. You can replace it with the standard return statement. (#9930)
The twisted-iocpsupport is no longer a hard dependency on Windows.
The IOCP support is now installed together with the other Windows soft
dependencies via twisted[windows-platform]. (#11893)
twisted.python.deprecate helper function will now always strip whitespaces from the docstrings.
This is done to have the same behaviour as with Python 3.13. (#12063)
twisted.conch.manhole.ManholeInterpreter.write, twisted.conch.manhole.ManholeInterpreter.addOutput, twisted.mail.imap4.IMAP4Server.sendUntaggedResponse async argument, deprecated since 18.9.0, has been removed. (#12130)
twisted.web.soap was removed.
The SOAP support was already broken, for at least the last 4 years.
The SOAP support in Twisted has no active maintainer. (#12146)

Misc

#11744, #11771, #12113, #12154, #12169, #12179, #12193, #12195, #12197, #12215, #12221, #12243, #12249, #12254, #12259, #12669

Conch

Bugfixes


- twisted.conch.insults.window.Widget.functionKeyReceived now dispatches functional key events to corresponding `func_KEYNAME` methods, where `KEYNAME` can be `F1`, `F2`, `HOME`, `UP_ARROW` etc. This is a regression introduced with #&#8203;8214 in Twisted 16.5.0, where events changed from `const` objects to bytestrings in square brackets like `[F1]`. (#&#8203;12046)

Web
---

Features

twisted.web.agent.Agent now allows duplicate Content-Length headers having the same value, per RFC 9110 section 8.6. It is otherwise more strict when parsing Content-Length header values. (#9064)
twisted.web.client.HTTPConnectionPool used by HTTP clients now runs faster by using a little less CPU. (#12108)
twisted.web.http_headers now uses less CPU, making a small HTTP client request 10% faster or so. (#12116)
twisted.web's HTTP/1.1 server now runs a little faster, with about 10% lower CPU overhead. (#12133)
twisted.web's HTTP 1.1 server is an additional 5% faster. (#12155)

Bugfixes


- twisted.web.util.redirectTo now HTML-escapes the provided URL in the fallback response body it returns (GHSA-cf56-g6w6-pqq2, CVE-2024-41810). (#&#8203;9839)
- twisted.web.http.IM_A_TEAPOT was added and returns `I'm a teapot`
  as default message for the status code 418,
  as defined in RFC 2324 section 2.3.2. (#&#8203;12104)
- The HTTP 1.0/1.1 server provided by twisted.web is now more picky about the first line of a request, improving compliance with RFC 9112. (#&#8203;12233)
- The HTTP 1.0/1.1 server provided by twisted.web now contains the characters set of HTTP header names, improving compliance with RFC 9110. (#&#8203;12235)
- The HTTP 1.0 and 1.1 server provided by twisted.web could process pipelined HTTP requests out-of-order, possibly resulting in information disclosure (CVE-2024-41671/GHSA-c8m8-j448-xjx7) (#&#8203;12248)
- twisted.web.util.redirectTo now HTML-escapes the provided URL in the fallback response body it returns (GHSA-cf56-g6w6-pqq2). The issue is being tracked with CVE-2024-41810. (#&#8203;12263)

Improved Documentation

Fix ReverseProxyResource example in developer guide. (#12152)

Deprecations and Removals


- twisted.web.util.ChildRedirector, which has never worked on Python 3, has been removed. (#&#8203;9591)
- ``twisted.web.http.Request.setResponseCode()`` no longer validates the types of inputs; we encourage you to use a type checker like mypy to catch these sort of errors. The long-deprecated ``twisted.web.server.string_date_time()`` and ``twisted.web.server.date_time_string()`` APIs were removed altogether. (#&#8203;12133)
- twisted.web.http.HTTPClient is now deprecated in favor of twisted.web.client.Agent (#&#8203;12158)

Misc
~~~~

- #&#8203;12098, #&#8203;12194, #&#8203;12200, #&#8203;12241, #&#8203;12257

Mail
----

No significant changes.

Words
-----

No significant changes.

Names
-----

No significant changes.

Trial
-----

No significant changes.

`v24.3.0`: Twisted 24.3.0

Compare Source

Twisted 24.3.0 (2024-03-01)

This release supports PyPy v7.3.14.

Bugfixes

twisted.logger.formatEvent now honors dotted method names, not just
flat function names, in format strings, as it has long been
explicitly documented to do. So, you will now get the expected
result from [formatEvent("here's the result of calling a method at
log-format time: {obj.method()}", obj=...)]{.title-ref} (#9347)
twisted.web.http.HTTPChannel now ignores the trailer headers
provided in the last chunk of a chunked encoded response, rather
than raising an exception. (#11997)
twisted.protocols.tls.BufferingTLSTransport, used by default by
twisted.protocols.tls.TLSMemoryBIOFactory, was refactored for
improved performance when doing a high number of small writes.
(#12011)
twisted.python.failure.Failure now throws exception for generators
without triggering a deprecation warnings on Python 3.12. (#12026)
twisted.internet.process.Process, used by reactor.spawnProcess,
now copies the parent environment when the [env=None]{.title-ref}
argument is passed on Posix systems and os.posix_spawnp is used
internally. (#12068)
twisted.internet.defer.inlineCallbacks.returnValue's stack
introspection was adjusted for the latest PyPy 7.3.14 release,
allowing legacy @inlineCallbacks to run on new PyPY versions.
(#12084)

Deprecations and Removals

twisted.trial.reporter.TestRun.startTest() is no longer called for
tests with skip annotation or skip attribute for Python 3.12.1 or
newer. This is the result of upstream Python gh-106584 change. The
behavior is not change in 3.12.0 or older. (#12052)

Misc

#11902, #12018, #12023, #12031, #12032, #12052, #12056,
#12067, #12076, #12078, #12087, #12095

Conch

No significant changes.

Web

Bugfixes

The documentation for twisted.web.client.CookieAgent no longer
references long-deprecated cookielib and urllib2 standard
library modules. (#12044)

Deprecations and Removals

twisted.web.http.Request now parses the
[multipart/form-data]{.title-ref} using
[email.message_from_bytes]{.title-ref}. The usage of
[cgi.parse_multipart]{.title-ref} was removed as the
[cgi]{.title-ref} module will be removed in Python 3.13. (#11848)

Misc

#12015

Mail

No significant changes.

Words

Improved Documentation

The documented IRC example was updated for Python3 usage. (#12070)

Names

No significant changes.

Trial

No significant changes.

Configuration

📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Signed-off-by: redhat-renovate-bot <redhat-internal-renovate@redhat.com>

Update dependency twisted to v24 [SECURITY]

70a5044

Signed-off-by: redhat-renovate-bot <redhat-internal-renovate@redhat.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update dependency twisted to v24 [SECURITY] #286

Update dependency twisted to v24 [SECURITY] #286

redhat-renovate-bot commented Nov 27, 2024

Update dependency twisted to v24 [SECURITY] #286

Are you sure you want to change the base?

Update dependency twisted to v24 [SECURITY] #286

Conversation

redhat-renovate-bot commented Nov 27, 2024

GitHub Vulnerability Alerts

CVE-2024-41671

Summary

PoC

Impact

CVE-2024-41810

Summary

Details

PoC

Impact

Release Notes

v24.7.0: Twisted 24.7.0

Twisted 24.7.0 (2024-08-08)

Features

Bugfixes

Improved Documentation

Deprecations and Removals

Misc

Conch

v24.3.0: Twisted 24.3.0

Twisted 24.3.0 (2024-03-01)

Bugfixes

Deprecations and Removals

Misc

Conch

Web

Bugfixes

Deprecations and Removals

Misc

Mail

Words

Improved Documentation

Names

Trial

Configuration

`v24.7.0`: Twisted 24.7.0

`v24.3.0`: Twisted 24.3.0