Fix flaky HTTPRouteInvalidCrossNamespaceParentRef test

[arkodg]

Description:

2023-11-01T02:05:54.6219408Z === RUN   TestGatewayAPIConformance/HTTPRouteInvalidCrossNamespaceParentRef/HTTPRoute_should_have_an_Accepted:_false_condition_with_reason_NotAllowedByListeners
2023-11-01T02:05:54.6245490Z === RUN   TestGatewayAPIConformance/HTTPRouteInvalidCrossNamespaceParentRef/Route_should_not_have_Parents_set_in_status
2023-11-01T02:05:54.6282765Z     helpers.go:534: Conditions matched expectations
2023-11-01T02:05:54.6284291Z === RUN   TestGatewayAPIConformance/HTTPRouteInvalidCrossNamespaceParentRef/Gateway_should_have_0_Routes_attached
2023-11-01T02:06:54.6297376Z     helpers.go:493: Error waiting for gateway, got Gateway Status &{[{0xc0002f1330 172.18.255.200}] [{Accepted True 1 2023-11-01 02:05:52 +0000 UTC Accepted The Gateway has been scheduled by Envoy Gateway} {Programmed True 1 2023-11-01 02:05:52 +0000 UTC Programmed Address assigned to the Gateway, 1/1 envoy Deployment replicas available}] [{http [{0xc0002f13f0 HTTPRoute} {0xc0002f1410 GRPCRoute}] 1 [{Programmed True 1 2023-11-01 02:05:52 +0000 UTC Programmed Sending translated listener configuration to the data plane} {Accepted True 1 2023-11-01 02:05:52 +0000 UTC Accepted Listener has been successfully translated} {ResolvedRefs True 1 2023-11-01 02:05:52 +0000 UTC ResolvedRefs Listener references have been resolved}]}]}, want zero listeners or exactly 1 listener with zero routes

Repro steps:

Include sample requests, environment, etc. All data and inputs
required to reproduce the bug.

Note: If there are privacy concerns, sanitize the data prior to
sharing.

Environment:

Include the environment like gateway version, envoy version and so on.

Logs:

Include the access logs and the Envoy logs.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.

[shawnh2]

dup of #2269