Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix jore3-importer logic related to validity period inconsistencies b…
…etween transformed lines and routes. In the Jore3 data model, the line itself (the `network_lines` table in the internal database of jore3-importer) has no validity period. Instead, the line- specific headers (`network_line_headers` table) have validity periods that do not overlap (within one line). Originally, it was thought that a separate line instance would be created for each line header into the Jore4 database. However, it appears that Jore3 route directions (`network_route_directions` table) may overlap with multiple line headers. This causes a transformation problem, because in Jore4 the validity period of a route MUST NOT extend further into the past or future than the line to which the route belongs. Basically, there are two options to solve the problem: (1) For each line in Jore3 (and all related line headers), create only single line in Jore4. (2) Group and join line headers together by validity periods in such a way that no (transformed) route overlaps multiple line instances transformed to Jore4. ATM, we have chosen the option (2) to import as much (version) data as possible into Jore4. In the SQL, we group the line headers into temporally continuous clusters. The last line header of each cluster is given a Jore4 ID that can be referenced by routes. In addition, it has been found that in some cases the validity period of the latest line header (for a single line) does not extend as far into the future as an associated route direction, causing database constraint violations during the route export stage if no date corrections are made. Therefore, for each line, the latest validity date of the route directions associated with it is determined, and if necessary, it is used as the end date of the last header of the line, if it is later than what is originally in the header. Schema changes: (i) Add Jore4 ID to the `network_line_headers` (and the corresponding history) table. (ii) Remove Jore4 ID from the `network_lines` (and corresponding history) table. Resolves HSLdevcom/jore4#1659
- Loading branch information