Quirk to ignore annotations from ONAP

[jpc4242]

@tliron , another one about ONAP.
ONAP decided to extend tosca with annotations with the bold :

assumed that standard parsers will ignore the annotations

I personally consider this a bad decission. Even if a parser works in "relaxed mode" and ignores whatever is not defined in the standard, if future versions of TOSCA want to use that name there will be a clash.
But, anyway, that's the situation we are facing now and as a result puccini chokes on CSARs generated from ONAP, with messages like:

  zip:file: xxx.zip!/Definitions/annotations.yml
    @17,1 annotation_types: unsupported field

I think that adding a ignore_annotation_types quirk is something that even I with my limited golang knowledge would be able to do. But before even trying I'd like to know if you would accept a PR. (and if so, tell me the exact quirk name that you would like to have)

TIA,
JPC

EDIT: Already Implemented via 6eca57e

[tliron]

Ugh, I agree with it being a bad decision. I agree that it's best for it to be ignored. Rather than implementing a general "ignore-unknown-keywords" I think we should specifically just ignore this keyword. What do you think?

[jpc4242]

I totally agree. A quirk to ignore annotation_types at top level and just that. And if in the future OASIS decide to reuse that name in futurr TOSCA versions, then when detecting those versions puccini will abort if the quirk is set.
Do you find annotation_types.ignore a valid name?

BTW: Do you remember that we spoke in the past about having a "meta-quirk" that would activate all quirks needed for ETSI SOL001? Maybe an ONAP meta-quirk will be needed also if we continue finding special needs for ONAP CSARs

[tliron]

Sigh, I'm not excited about these "meta-quirks" but they are very easy to implement and if they can help other projects then I am very happy to add them. The idea of Puccini is to enable functionality, not to disable it.

I should have time this week to implement this, I can probably do it in just minutes of work.

[tliron]

Could you help by providing a list of which quirks should go under "etsi" and which should go under "onap"?

[tliron]

Support for quirk "annotations.ignore" was just commited.

[jpc4242]

Thanks, Tal:
About the metaquirks. It is also easy (just a slight inconvenience) to activate all the quirks individually with some wrapper script; this is not reducing puccini functionality. So if you are not excited about them and prefer to keep puccini as clean as possible, that's fine.

But in case you decide to have the metaquirks:

• ONAP quirks (so far):
  • annotations.ignore
  • imports.sequencedlist (BTW: thanks for that one, too!!)
• ETSI SOL001 (see below) quirks (so far):
  • imports.topology_template.ignore
  • data_types.string.permissive
  • substitution_mappings.requirements.permissive
  • substitution_mappings.requirements.list. A comment about this one: Current quirk means accepting a syntax that would have been rejected without the quirk, but at the same time implies rejecting the syntax that would have been accepted without the quirk. In ETSI SOL001 (and its examples, and the vendors using it in real world) I have seen both formats equally used. So it would be great if the quirk is modified (or a new quirk created) than accepts both syntaxes. But again this is not a big deal as it can be solved externally (we have a script that tries without the quirk first and, if this is the only type of error reported, it tries again with the quirk)

When deciding a sensible name for the ETSI SOL001 metaquirk, SOL001 refers only to the format of the tosca yaml files. The CSAR themselves are defined in SOL004 and SOL007. I suggest "ETSINFV" as the name.

HTH,
JPC

[tliron]

Thank you, much appreciated! I will try to follow your suggestions exactly.

I'm not excited about quirks, but I did decide to create them because I am excited about TOSCA adoption. I will do my best to make Puccini useful for you, and will always accept PRs that make it more useful.

[tliron]

OK, I've committed support for the combo quirks as you suggested and also made "substitution_mappings.requirements.list" permissive (it will accept both notations). Please help me by testing that it all works!

[jpc4242]

Today I performed the first test with a CSAR coming from ONAP with the new version and the onap metaquirk. The results are discouraging, but puccini is not to blame for it (ONAP is):
- uses constructs defined only in tosca 1.2 but not in tosca 1.1 (namely properties under substitution_mappings) in a 1.1 file. I had to modify the version in the yaml file. Then I got a lot of include mismatch errors which fortunately puccini could ignore thanks to -x imports.version.permissive.
- It is also having topology_templates in included file. So @tliron , we need to include imports.topology_template.ignoreas part of the ONAP meta-quirk
- Wrong syntaxes when redefining default values. SO for example instead of :

  tosca.capabilities.Endpoint.Public:
    derived_from: tosca.capabilities.Endpoint
    properties:
      network_name:
        default: PUBLIC
.....

what the ONAP template had was:

  tosca.capabilities.Endpoint.Public:
    derived_from: tosca.capabilities.Endpoint
    properties:
      network_name: PUBLIC
.....

• misc typos (e.g using valid_source_typeinstead of valid_source_types inside a capability definitions

Tal, except for adding the quirk to the metaquirk as specified above, I don't think that puccini should stretch so far as to "cover for" all these bugs in ONAP. ONAP instead should fix them (and maybe they already did a version newer than the one I tested with). What's your view?

[tliron]

I agree with you. In upgrading to TOSCA 1.3 I recommend that ONAP clean up its act and make sure it is compatible with other TOSCA parsers.

[tliron]

(I added that extra quirk to "-x onap")