Update XML definition

[WJSchakel]

Let's first collect several small issues, so we can do this in one batch. Items with strikethrough have been solved. You can edit this message.

• CONNECTOR tag needs a TYPE (link type) so GTU compatibility can be defined on it. Connector constructor also needs it.¹

• CENTROID tag need to be defined, as part of NETWORK.

• default_gtutypes.xml: remove NONE, name all others NL.XXX.¹

• default_linktypes.xml: remove NONE, name all others NL.XXX, match specifics with defaults in DefaultsNl.¹

• default_lanetypes.xml: remove NONE, name all others NL.XXX, match specifics with defaults in DefaultsRoadNl.¹

• default_detectortypes.xml: name all NL.XXX.¹

• Add DETECTORTYPE, use it in CONTROLTYPE.SENSOR, SINK and OD.¹

• Add defaults_detectortypes.xml, match specifics with defaults in DefaultsRoadNl.¹

• Remove DIRECTION in COMPATIBILITY of both LINKTYPE and LANETYPE (Issue Remove direction / directionality from XML format #16). (Edit: and DetectorType. Note that we need a way to define INcompatibility in XML.)

• Remove deprecated defaults from DefaultsNl that are now only there so current XML defaults can be parsed.

• Provide a default_lanebiases.xml (see below which). See also TODO in LaneParser.¹

• For LinkType and LaneType the DIRECTION is ignored in DefinitionsParser. Therefore, NONE direction now results in positive compatibility. XML should allow to define a positive compatibility (addCompatibleGtuType()) or a negative compatibility (addIncompatibleGtuType()).¹

• Inclusion of an OUTPUTFILE type. This can be used under two new fields: LOOPDETECTORS (which has no other content than an output file) and SAMPLER (which needs further details on what to sample, and should have an optional Frequency for sampling, if not given sampling is move-based). We could place all this under an OUTPUT tag.¹

• Rename SENSOR tag to DETECTOR.¹

• Files default_lanetypes.xml and default_linktypes.xml have commented out old difinitions.

• ROADLAYOUT has contradicting description: "Lanes to the right of the design line have negativeoffset. Those to the right have positive offset."

• NETWORKDEMAND.LISTGENERATOR.GTUTEMPPLATEMIX has double PP in GTUTEMPPLATEMIX. But should change LISTGENERATOR to INJECTIONGENERATOR anyway. LISTGENERATOR is not yet parsed.

• xsd:keyref "signalGroupLinkIdRef" refers to "SIGNALGROUP.TRAFFICLIGHT.LINK", should be "CONTROLTYPE.SIGNALGROUP.TRAFFICLIGHT.LINK". (This check works in the editor, Schema is unable to find it as a type because it registers it under ControlType.SignalGroup.TrafficLight.Link. But the elements that are a ControlType have a different name.)

• GTUCOLORERTYPE makes no sense. Either choose a standard option, or point to a class. Id attribute is strange, it needs no id. Including parameters is pointless; a specific colorer will know what it should do.

• In BASICROADLAYOUT we have <xsd:sequence minOccurs="1" maxOccurs="unbounded"> with in it a <xsd:choice>. This is the same as as single <xsd:choice minOccurs="1" maxOccurs="unbounded">. There are no other such cases in the spec.

• An xsd:choice element specifies no default choice. In automatic programs, it would be good if the order of choice elements is logically defined in XSD, and that the first defined option is the default option in all xsd:choice elements.

• GENERATOR and LISTGENERATOR need an ID. We now use lane.getFullId() for this.¹ Edit: This is fine, we have a xsd:unique on the combination of Link and Lane.

• NETWORKDEMAND, CONTROL, MODEL and SCENARIO can appear often at the level of OTS. This may clutter the tree at the highest level. They should become xsd:sequence or similar alternative themselves.¹

• OTS.MODEL.MODELPARAMETERS make no sense. Each parameter type may appear once in the choice. It should be more like OTS.SCENARIO.INPUTPARAMETERS. Also OTS.MODEL.PARAMETERS is an unused type. Model parameters should refer to parameter types.

• We must supply more defaults, e.g. default class names for traffic light and traffic light sensor. Now, the user has no way of knowing what to do for a mandatory field. OTS.RUN.RUNLENGTH should have a default value so the XML is not invalid right away when started from the schema.

• OTS.NETWORKDEMAND.ROUTE.NODE has non-required ID. It should be required.

• LENGTHBEGINENDTYPE and FRACTIONTYPE have a pattern that ends with "|(100(\.\0*)?))\s*\%)", the "\" after "." should be removed. Pattern.matches() crashes because of it.

• SPEEDLIMIT.LEGALSPEEDLIMIT should be required.

• Certain values in OTS.RUN lack proper constraints (i.e. positive simulation duration).

• GTUTEMPLATE has weight property which is not restricted to be positive.

• CSESTRIPE has strange single-option choices.

• MARKOV.GTUTYPE and MARKOV.PARENT need an xsd:keyref to GTU types.

• CONNECTOR needs to link up to 1 centroid, either the start or end. This should be defined with an xsd:source.

• Add descriptive xsd:appinfo for patterns that could not be matched.

• Tags that should not have any content (e.g. <STRAIGHT/> should be defined as a xsd:complexType with nothing specified. Otherwise a tag value is permissible: <STRAIGHT>inappropriate value</STRAIGHT>. Also Model.StrategicalPlanner.Route.None/Shortest, IDM, Wrap.

• Tag names with Pascal-case.

• Rename NETWORKDEMAND to Demand, and OD.DEMAND to Od.Cell.

• ots-types.xsd contains both a DoublePositive and a PositiveDouble type. Both 0 exclusive.

• keyref for OdOptions.OdOptionsItem.LaneBiases.LaneBias.GtuType to GTU types.

• Run.RandomStreams.RandomStream.Id and Run.RandomStreams.RandomStream.Replication.Id should each be a key. Everywhere where a random stream is used, it needs to be one of those Ids. Can we still allow a default and generation stream in case of 1 replication?

• LaneBias.StickyLanes should be at least 1 (0 is now allowed).

• Remove CseLane.DesignDirection.

• Supply a lot more descriptions.

• OdOptionsItem.LinkType/Origin/Lane have fields that should point to keys.

• Change patterns as XXX|YYY|ZZZ in to enumerations.

• Enhance RoomCheckerType to include time value for TTC option.

• Enhance types with a CLASS enumeration that then requires a field while others do not.

• Provide <xsd:complexType name="EmptyType" /> and use it where elements with <xsd:complexType /> are used.

• Add IdmPlusMulti.

• Add Correlation under Models.Model.ModelParameters.

• Add global flattener.

¹) Needs a change in the XML parser.

Content of default_lanebiases.xml:

• TRUCK: FROMRIGHT=0.0, BIAS=5.0, STICKY_LANES=2
• ROAD_USER: FROMLEFT=0.0, BIAS=1.0

[WJSchakel]

In XSD we have a great number of restrictions that are defined with patterns. In case of a pure choice:

  <xsd:simpleType name="LANEKEEPINGTYPE">
    <xsd:restriction base="xsd:string">
      <xsd:pattern value="KEEPLEFT|KEEPRIGHT|KEEPLANE"></xsd:pattern>
    </xsd:restriction>
  </xsd:simpleType>

It would help to define this as an enumeration:

  <xsd:simpleType name="LANEKEEPINGTYPE">
    <xsd:restriction base="xsd:string">
      <xsd:enumeration value="KEEPLEFT" />
      <xsd:enumeration value="KEEPRIGHT" />
      <xsd:enumeration value="KEEPLANE" />
    </xsd:restriction>
  </xsd:simpleType>

In this way, we can present a dropdown-menu/pick-list to users.

[averbraeck]

Excellent idea! But please wait till I addressed Issue #55. The XSD files are being changed for that issue at the moment.

[WJSchakel]

Yeah sure. I think everything in this issue should be done in one go at some later point.

[WJSchakel]

In case of an xsd:choice we want to present the user with a dropdown-menu/pick-list. However, if one of the options is a xsd:sequence, it has no name to show whatsoever. From child elements, some name can be derived. For example:

<xsd:choice>
  <xsd:element name="WIDTH" type="POSITIVELENGTHTYPE" />
  <xsd:sequence>
    <xsd:element name="WIDTHSTART" type="LENGTHTYPE" />
    <xsd:element name="WIDTHEND" type="LENGTHTYPE" />
  </xsd:sequence>
</xsd:choice>

The second option could become "widthstart | widthend". Another way to deal with this, is to specify it with an xsd:annotation:

<xsd:choice>
  <xsd:element name="WIDTH" type="POSITIVELENGTHTYPE" />
  <xsd:sequence>
    <xsd:annotation>
      <xsd:appinfo source="name">define at start and end</xsd:appinfo>
    </xsd:annotation>
    <xsd:element name="WIDTHSTART" type="LENGTHTYPE" />
    <xsd:element name="WIDTHEND" type="LENGTHTYPE" />
  </xsd:sequence>
</xsd:choice>

[Peter-Knoppers]

This is also a test to see if replies go to the right place… I would go for the annotation approach. This ensures that you will be aware that you are about to break things if you want to change that part of the xsd. Of course, it does not prevent you from breaking things, but all the structure and interpretation of this part of the xsd is all in one place (unlike having the interpretation embedded in the code of the ots-editor). -- PK From: Wouter Schakel ***@***.***> Sent: Thursday, February 23, 2023 23:08 To: averbraeck/opentrafficsim ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [averbraeck/opentrafficsim] Update XML definition (Issue #40) In case of an xsd:choice we want to present the user with a dropdown-menu/pick-list. However, if one of the options is a xsd:sequence, it has no name to show whatsoever. From child elements, some name can be derived. For example: <xsd:choice> <xsd:element name="WIDTH" type="POSITIVELENGTHTYPE" /> <xsd:sequence> <xsd:element name="WIDTHSTART" type="LENGTHTYPE" /> <xsd:element name="WIDTHEND" type="LENGTHTYPE" /> </xsd:sequence> </xsd:choice> The second option could become "widthstart | widthend". Another way to deal with this, is to specify it with an xsd:annotation: <xsd:choice> <xsd:element name="WIDTH" type="POSITIVELENGTHTYPE" /> <xsd:sequence> <xsd:annotation> <xsd:appinfo source="name">define at start and end</xsd:appinfo> </xsd:annotation> <xsd:element name="WIDTHSTART" type="LENGTHTYPE" /> <xsd:element name="WIDTHEND" type="LENGTHTYPE" /> </xsd:sequence> </xsd:choice> — Reply to this email directly, view it on GitHub<#40 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABET3YOHZQNMFX2RDQFIF3DWY7NSRANCNFSM6AAAAAAULT7H7A>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.******@***.***>>

[WJSchakel]

Yes, reply works! It does come with the text that you replay to, though. That clutters things so I'd suggest removing that text.

Right now, the code looks for the annotation. If it does not find it, it constructs a name from the sequence. With a max length of 64 characters. But yes, let's add these annotations where relevant when we work on a new XML spec.

[WJSchakel]

XML-tags are now Pascal-case. In the parser this has required in some places to address classes by their full name. For example Node (regular from ots-core) and org.opentrafficsim.xml.generated.Node are used in the same class.

[WJSchakel]

Centroid is added to XML specification. Checks on Connectors towards Node id's have been removed, as Connectors can connect to Centroids too.

[WJSchakel]

DetectorType tag fully integrated in parsing. "default_detectortypes.xml" added to all XML files.

[WJSchakel]

File default_lanebiases.xml added. There are now LaneBiases.LaneBias under definitions in XML. Within OdOptions the LaneBiases can be given as a set of choices between LaneBias or DefinedLaneBias. In the latter case, by GtuType a LaneBias from definitions is referred. Keys are defined for LaneBias.GtuType referring to GtuType.Id, and DefinedLaneBias.GtuType referring to LaneBias.GtuType.

[WJSchakel]

When a value does not match a pattern, a sensible message can now be defined as:

  <xsd:simpleType name="LengthType">
    <xsd:annotation>
      <xsd:appinfo source="pattern">Provide a valid Length, such as "3.5m".</xsd:appinfo>
    </xsd:annotation>
    <xsd:restriction base="xsd:string">
      <xsd:pattern value="[+-]?[0-9]*\.?[0-9]+([eE]......"></xsd:pattern>
    </xsd:restriction>
  </xsd:simpleType>

[WJSchakel]

Random streams can be defined, or defaults ("default" or "generation") can be used. To be able to check whether a selected random stream is defined, an xsd:keyref is defined. The default streams can be used by leaving the option blank, in which case the default for the particular type ("default" or "generation") will be used.

All constant and discrete distribution types stem from a single DistType with RandomStream attribute. An xsd:keyref to this attribute is defined. The editor now also deal with keyref's in extended types.

[WJSchakel]

RandomStream definitions has been enhanced in XML as per issue #47.

[WJSchakel]

There is now a list Ots.Scenarios.DefaultInputParameters. Two things may point to this:

1. Values that are expressions that change between scenarios, using { }.
2. Ots.Scenarios.Scenario.InputParameters to overwrite default values for a specific scenario.