WFS getcapabilities response

[smnorris]

FYI, perhaps it is just a temporary outage but it seems like the response to a WFS GetCapabilities request has changed, and no longer returns info about the WFS service

https://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&version=2.0.0&request=GetCapabilities
smnorris/bcdata#141

Python bcdata is more dependent on this response, but R package might be affected as well?
Certainly when getting the bcdata.single_download_limit

[ateucher]

Thanks @smnorris for the heads up. Just tested it now and it seems ok? We use it in listing the WFS records, and getting the single download limit, here.

Eg:

bcdata:::bcdc_get_wfs_records()
#> # A tibble: 861 × 3
#>    whse_name                                               title         cat_url
#>    <chr>                                                   <chr>         <chr>  
#>  1 WHSE_BASEMAPPING.DBM_BC_7H_MIL_BATHYMETRC_POLY          (7.5M) Bathy… https:…
#>  2 WHSE_BASEMAPPING.DBM_BC_7H_MIL_BOUNDARIES_LINE          (7.5M) Bound… https:…
#>  3 WHSE_BASEMAPPING.DBM_BC_7H_MIL_DRAINAGE_LINE            (7.5M) Drain… https:…
#>  4 WHSE_BASEMAPPING.DBM_BC_7H_MIL_DRAINAGE_POLY            (7.5M) Drain… https:…
#>  5 WHSE_BASEMAPPING.DBM_BC_7H_MIL_POPULATION_POINT         (7.5M) Major… https:…
#>  6 WHSE_BASEMAPPING.DBM_BC_7H_MIL_ROADS_LINE               (7.5M) Major… https:…
#>  7 WHSE_BASEMAPPING.DBM_BC_7H_MIL_PARK_POLY                (7.5M) Natio… https:…
#>  8 WHSE_BASEMAPPING.DBM_BC_7H_MIL_POLITICAL_POLY           (7.5M) Provi… https:…
#>  9 WHSE_LEGAL_ADMIN_BOUNDARIES.OATS_ALR_POLYS              ALC ALR Poly… https:…
#> 10 WHSE_LEGAL_ADMIN_BOUNDARIES.OATS_ALR_BOUNDARY_LINES_SVW ALC Agricult… https:…
#> # ℹ 851 more rows
bcdata:::bcdc_single_download_limit()
#> [1] 10000

^{Created on 2023-08-22 with reprex v2.0.2}

[smnorris]

Glad it is still working.

I was using getCapabilities to:

1) Get the schema of a layer and find geometry column name
Schemas are no longer included in the response but I saw you use DescribeFeatureType for that so I adjusted to do the same. Edit - actually no, owslib was doing DescribeFeaturetype internally, this didn't come from GetCapabilities

2. list WFS layers
   I can adjust my code to use the current response to generate a list of layers as per bcdata:::bcdc_get_wfs_records()

But the current response does not seem to be a valid response to a WFS getcapabilites request and I don't think you'll be able to get the single download limit, the OperationsMetadata tag is not present. I've alerted DataBC - smnorris/bcdata#142 (comment)

[ateucher]

I don't think you'll be able to get the single download limit, the OperationsMetadata tag is not present.

Weird. I can't find it manually but I stepped through bcdc_single_download_liimit() and it is working (and not using a cached value).

(I fully admit though that I don't really understand xml parsing).

[smnorris]

Weird indeed, I presumed it was picking up the default when I tried.

I don't want to parse xml either, and was using owslib to do that for me. It bails when it cannot find OperationsMetadata

[smnorris]

This is what the response used to look like:
https://gist.githubusercontent.com/smnorris/57b7ff6df0354c4d13dab5130e55c8af/raw/682ce6d4c23cb670242e54bb6c3c6d829031d3c6/capabilities_wfs.xml

[ateucher]

Alright, so this is the endpoint that our code is hitting: http://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&version=2.0.0&request=Getcapabilities, and that has the information in it. It differs in two ways:

1. http rather than https.
2. Case: request=Getcapabilities vs request=GetCapabilities

🤯

[smnorris]

Oh dear. I think I standardized that url to https in my code, so there may not have been a change to the service?

Something definitely broke on Friday before I made changes but perhaps it was something else that has since been resolved.

[ateucher]

Having a totally different response with http vs https is... weird? Also I would think it should all be https anyway!

[ateucher]

And it does appear to be a combination of http AND the case of get[Cc]apabilities. This returns WMS 1.1 info:

http://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&version=2.0.0&request=GetCapabilities

And this appears to return nothing or time out:

https://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&version=2.0.0&request=Getcapabilities

[smnorris]

Based on my limited sysadmin experience, it sounds like an issue with reverse proxy setup.

[ateucher]

It seems very fragile! Makes you wonder which is the intended endpoint? Hopefully the one we want ;)

[smnorris]

Thanks for digging into this - I've forwarded the source of the issue and a link to this thread on to the DataBC JIRA ticket.

[ateucher]

Awesome, thanks. If you get a resolution, do you mind updating this issue so we can update the url if needbe?

[smnorris]

Will do.

I think the handling of the get[Cc]apabilites casing did change, it still fails after I switch my base url to plain http.

owslib generates the request as GetCapabilites: https://github.com/geopython/OWSLib/blob/master/owslib/feature/common.py#L35

[ateucher]

Thanks!

Interesting - looks like you could supply the request param directly though as Getcapabilities?

[smnorris]

yes, but no luck so far - hopefully I can find the right combination of parameters.

[rustyjux]

My apologies - a quick fix was put in Aug 18 by Data Systems and Services to cache the WMS GetCapabilities request to support the delivery of wildfire services. The apache rewrite rule also erroneously captured WFS GetCapabilities requests. I removed these changes on Sept 6 - again, apologies for any impacts.