Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dom-shape metric to markup #140

Merged
merged 2 commits into from
Aug 26, 2024
Merged

Conversation

tunetheweb
Copy link
Member

@tunetheweb tunetheweb commented Aug 20, 2024

@tunetheweb
Copy link
Member Author

Hi @1cg, I've added this custom metric as provided (except I put the "helper functions" to the top).

It seems to be working but let me know if there's some other site you want to test this on.

Some observations/questions from me:

  • Should the branchFactorByDepth show the depth as a property?
  • The branchFactorByDepth can get quite large for some sites (look at the cnn example). Is all this data needed?
  • Are you wanting this data long term (i.e. we should leave this to run ever month), or only as a once off (in which case we ran remove it after the next crawl)?

After you answer those and confirm this is how you want the data we can get some of the other maintainers (@rviscomi , @pmeenan , @paulcalvano ) to weigh in with their thoughts.

@1cg
Copy link

1cg commented Aug 20, 2024

Hi @tunetheweb thank you very much for including this. To answer your questions:

  • The depth of the branchFactorByDepth array is implied by the index, but if you think that it would make sense to add it to make querying easier that would be easy to do, just let me know.
  • Yes, in order to calculate the "shape" of the page we do need all that data. However we could do something less computationally and storage intensive (e.g. just node count per depth and then estimating means, although that would perclude calculating skew, range, etc. for fanout)
  • This can be a once off run, although doing it yearly might be nice to update the paper we are going to produce based on this data

Another thing that would be super useful would be metadata about the sites: total traffic, category, etc. so we can do statistical analysis across those sorts of variables.

I wanted to note that you and I have had off-github discussions about the performance of this analysis: if it is too heavy I am happy to dial it back.

Thank you!

@tunetheweb
Copy link
Member Author

Yes, in order to calculate the "shape" of the page we do need all that data. However we could do something less computationally and storage intensive (e.g. just node count per depth and then estimating means, although that would perclude calculating skew, range, etc. for fanout)

OK let's leave as is. It's not that much data compared to other stuff.

This can be a once off run, although doing it yearly might be nice to update the paper we are going to produce based on this data

Let's add and leave it to run monthly for now as it does fit in with our mission. Can remove again if it proves less useful than we hoped.

Another thing that would be super useful would be metadata about the sites: total traffic, category, etc. so we can do statistical analysis across those sorts of variables.

For "total traffic" we have rank which is a broad measure of popularity in poer of 10 buckets, with half step intervals (so top 1k, top 5k, top 10k, top 50k, top 100k...etc.).

For category, we recently added classification via the Topics API, however some have noted accuracy issues with that classificaiton with this once you get out of the most popular pages.

I wanted to note that you and I have had off-github discussions about the performance of this analysis: if it is too heavy I am happy to dial it back.

My tests seem to show it's super quick and had a quick discussion with the other maintainers and they don't seem concerned either.

Copy link

Custom metrics for https://almanac.httparchive.org/en/2022/

WPT test run results: http://webpagetest.httparchive.org/results.php?test=240820_Z2_5
Changed custom metrics values:

{
  "_markup": {
    "favicon": true,
    "rel_alternate_mobile": false,
    "compatMode": "CSS1Compat",
    "noscripts": {
      "iframe_googletagmanager_count": 0,
      "total": 0
    },
    "buttons": {
      "types": {
        "button": 9,
        "submit": 4
      },
      "total": 13
    },
    "form": {
      "target": {},
      "method": {},
      "elements": [
        {
          "tagNames": {
            "label": 1,
            "input": 1,
            "button": 1
          },
          "types": {
            "search": 1,
            "submit": 1
          },
          "total": 3
        },
        {
          "tagNames": {
            "label": 1,
            "input": 1,
            "button": 1
          },
          "types": {
            "search": 1,
            "submit": 1
          },
          "total": 3
        },
        {
          "tagNames": {
            "label": 1,
            "input": 1,
            "button": 1
          },
          "types": {
            "search": 1,
            "submit": 1
          },
          "total": 3
        },
        {
          "tagNames": {
            "label": 1,
            "input": 1,
            "button": 1
          },
          "types": {
            "search": 1,
            "submit": 1
          },
          "total": 3
        }
      ],
      "total": 4
    },
    "dirs": {
      "html_dir": null,
      "body_nodes_dir": {
        "values": {},
        "total": 0
      }
    },
    "inputs": {
      "types": {
        "search": 4
      },
      "total": 4
    },
    "audios": {
      "autoplay": {},
      "total": 0
    },
    "classes": {
      "unique_names_total": 76,
      "references_total": 247
    },
    "ids": {
      "ids_total": 52,
      "duplicate_ids_total": 0,
      "unique_ids_total": 52
    },
    "images": {
      "picture": {
        "total": 0
      },
      "source": {
        "total": 0,
        "src_total": 0,
        "srcset_total": 0,
        "media_total": 0,
        "type_total": 0
      },
      "img": {
        "total": 5,
        "src_total": 5,
        "srcset_total": 0,
        "alt": {
          "missing": 0,
          "blank": 5,
          "present": 0
        },
        "loading": {
          "auto": 0,
          "lazy": 4,
          "eager": 0,
          "invalid": 0,
          "missing": 1,
          "blank": 0
        },
        "dimensions": {
          "missing_width": 0,
          "missing_height": 0
        }
      }
    },
    "iframes": {
      "total": 0,
      "loading": {
        "auto": 0,
        "lazy": 0,
        "eager": 0,
        "invalid": 0,
        "missing": 0,
        "blank": 0
      }
    },
    "amp": {
      "html_amp_attribute_present": false,
      "html_amp_emoji_attribute_present": false,
      "amp_page": false,
      "rel_amphtml": null
    },
    "data_nosnippet": {
      "valid": 0,
      "wrong_tag_type": 0
    },
    "obsolete_elements": {},
    "svgs": {
      "svg_element_total": 15,
      "svg_img_total": 0,
      "svg_object_total": 0,
      "svg_embed_total": 0,
      "svg_iframe_total": 0,
      "svg_total": 15
    },
    "app": {
      "app_id_present": false,
      "meta_theme_color": null
    },
    "anchors": {
      "hrefs_without_special_scheme": []
    },
    "dom-shape": {
      "bodyNodesCount": 586,
      "bodyLeafNodesCount": 328,
      "depth": {
        "mean": 7.804878048780488,
        "median": 9,
        "mode": 9,
        "stddev": 2.4203861029940903,
        "skew": -0.49377326606738925,
        "range": {
          "min": 1,
          "max": 13
        }
      },
      "bodyNonLeafNodesCount": 258,
      "branchFactor": {
        "mean": 5.4689922480620154,
        "median": 3,
        "mode": 3,
        "stddev": 8.796196310171327,
        "skew": 0.2806886250602479,
        "range": {
          "min": 1,
          "max": 71
        }
      },
      "branchFactorByDepth": [
        {
          "depth": 0,
          "count": 1,
          "mean": 31,
          "stddev": 0,
          "skew": null,
          "median": 31,
          "mode": 31,
          "range": {
            "min": 31,
            "max": 31
          }
        },
        {
          "depth": 1,
          "count": 5,
          "mean": 12.2,
          "stddev": 16.61806246227279,
          "skew": 0.5536144794789603,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 45
          }
        },
        {
          "depth": 2,
          "count": 17,
          "mean": 4.647058823529412,
          "stddev": 3.77205165691327,
          "skew": 0.43664800308626384,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 19
          }
        },
        {
          "depth": 3,
          "count": 13,
          "mean": 6.384615384615385,
          "stddev": 3.2708140962260464,
          "skew": 1.034792955221957,
          "median": 5,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 11
          }
        },
        {
          "depth": 4,
          "count": 19,
          "mean": 5.578947368421052,
          "stddev": 3.0316885945083514,
          "skew": 0.8506636773620477,
          "median": 5,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 13
          }
        },
        {
          "depth": 5,
          "count": 27,
          "mean": 4.37037037037037,
          "stddev": 3.1049321355755093,
          "skew": 0.44135276087648456,
          "median": 4,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 15
          }
        },
        {
          "depth": 6,
          "count": 25,
          "mean": 3.68,
          "stddev": 1.5419468213917102,
          "skew": -0.8560606511764209,
          "median": 4,
          "mode": 5,
          "range": {
            "min": 1,
            "max": 7
          }
        },
        {
          "depth": 7,
          "count": 19,
          "mean": 14.052631578947368,
          "stddev": 19.06629344435806,
          "skew": 0.4747976634979437,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 3,
            "max": 71
          }
        },
        {
          "depth": 8,
          "count": 63,
          "mean": 5.984126984126984,
          "stddev": 11.693835119130028,
          "skew": 0.25518805026122093,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 71
          }
        },
        {
          "depth": 9,
          "count": 57,
          "mean": 3.1052631578947367,
          "stddev": 0.692926935373035,
          "skew": 0.15191090506254984,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 7
          }
        },
        {
          "depth": 10,
          "count": 6,
          "mean": 3.6666666666666665,
          "stddev": 1.8856180831641267,
          "skew": 0.3535533905932737,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 7
          }
        },
        {
          "depth": 11,
          "count": 4,
          "mean": 3.5,
          "stddev": 0.8660254037844386,
          "skew": 0.5773502691896258,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 5
          }
        },
        {
          "depth": 12,
          "count": 3,
          "mean": 5,
          "stddev": 0,
          "skew": null,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 5,
            "max": 5
          }
        }
      ]
    }
  }
}
Custom metrics for https://example.com

WPT test run results: http://webpagetest.httparchive.org/results.php?test=240820_SK_6
Changed custom metrics values:

{
  "_markup": {
    "favicon": false,
    "rel_alternate_mobile": false,
    "compatMode": "CSS1Compat",
    "noscripts": {
      "iframe_googletagmanager_count": 0,
      "total": 0
    },
    "buttons": {
      "types": {},
      "total": 0
    },
    "form": {
      "target": {},
      "method": {},
      "elements": [],
      "total": 0
    },
    "dirs": {
      "html_dir": null,
      "body_nodes_dir": {
        "values": {},
        "total": 0
      }
    },
    "inputs": {
      "types": {},
      "total": 0
    },
    "audios": {
      "autoplay": {},
      "total": 0
    },
    "classes": {
      "unique_names_total": 0,
      "references_total": 0
    },
    "ids": {
      "ids_total": 0,
      "duplicate_ids_total": 0,
      "unique_ids_total": 0
    },
    "images": {
      "picture": {
        "total": 0
      },
      "source": {
        "total": 0,
        "src_total": 0,
        "srcset_total": 0,
        "media_total": 0,
        "type_total": 0
      },
      "img": {
        "total": 0,
        "src_total": 0,
        "srcset_total": 0,
        "alt": {
          "missing": 0,
          "blank": 0,
          "present": 0
        },
        "loading": {
          "auto": 0,
          "lazy": 0,
          "eager": 0,
          "invalid": 0,
          "missing": 0,
          "blank": 0
        },
        "dimensions": {
          "missing_width": 0,
          "missing_height": 0
        }
      }
    },
    "iframes": {
      "total": 0,
      "loading": {
        "auto": 0,
        "lazy": 0,
        "eager": 0,
        "invalid": 0,
        "missing": 0,
        "blank": 0
      }
    },
    "amp": {
      "html_amp_attribute_present": false,
      "html_amp_emoji_attribute_present": false,
      "amp_page": false,
      "rel_amphtml": null
    },
    "data_nosnippet": {
      "valid": 0,
      "wrong_tag_type": 0
    },
    "obsolete_elements": {},
    "svgs": {
      "svg_element_total": 0,
      "svg_img_total": 0,
      "svg_object_total": 0,
      "svg_embed_total": 0,
      "svg_iframe_total": 0,
      "svg_total": 0
    },
    "app": {
      "app_id_present": false,
      "meta_theme_color": null
    },
    "anchors": {
      "hrefs_without_special_scheme": []
    },
    "dom-shape": {
      "bodyNodesCount": 5,
      "bodyLeafNodesCount": 3,
      "depth": {
        "mean": 2.3333333333333335,
        "median": 2,
        "mode": 2,
        "stddev": 0.4714045207910317,
        "skew": 0.7071067811865478,
        "range": {
          "min": 2,
          "max": 3
        }
      },
      "bodyNonLeafNodesCount": 2,
      "branchFactor": {
        "mean": 4,
        "median": 4,
        "mode": 1,
        "stddev": 3,
        "skew": 1,
        "range": {
          "min": 1,
          "max": 7
        }
      },
      "branchFactorByDepth": [
        {
          "depth": 0,
          "count": 1,
          "mean": 3,
          "stddev": 0,
          "skew": null,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 3
          }
        },
        {
          "depth": 1,
          "count": 1,
          "mean": 7,
          "stddev": 0,
          "skew": null,
          "median": 7,
          "mode": 7,
          "range": {
            "min": 7,
            "max": 7
          }
        },
        {
          "depth": 2,
          "count": 1,
          "mean": 1,
          "stddev": 0,
          "skew": null,
          "median": 1,
          "mode": 1,
          "range": {
            "min": 1,
            "max": 1
          }
        }
      ]
    }
  }
}
Custom metrics for https://www.cnn.com

WPT test run results: http://webpagetest.httparchive.org/results.php?test=240820_BY_7
Changed custom metrics values:

{
  "_markup": {
    "favicon": true,
    "rel_alternate_mobile": false,
    "compatMode": "CSS1Compat",
    "noscripts": {
      "iframe_googletagmanager_count": 0,
      "total": 0
    },
    "buttons": {
      "types": {
        "button": 1,
        "submit": 3
      },
      "total": 33
    },
    "form": {
      "target": {},
      "method": {},
      "elements": [
        {
          "tagNames": {
            "input": 17,
            "label": 17,
            "textarea": 1,
            "button": 2
          },
          "types": {
            "radio": 5,
            "checkbox": 12,
            "button": 1,
            "submit": 1
          },
          "total": 37
        },
        {
          "tagNames": {
            "input": 1,
            "button": 1
          },
          "types": {
            "text": 1,
            "submit": 1
          },
          "total": 2
        },
        {
          "tagNames": {
            "input": 1,
            "button": 1
          },
          "types": {
            "text": 1,
            "submit": 1
          },
          "total": 2
        }
      ],
      "total": 3
    },
    "dirs": {
      "html_dir": null,
      "body_nodes_dir": {
        "values": {},
        "total": 0
      }
    },
    "inputs": {
      "types": {
        "radio": 5,
        "checkbox": 16,
        "text": 3
      },
      "total": 24
    },
    "audios": {
      "autoplay": {},
      "total": 0
    },
    "classes": {
      "unique_names_total": 718,
      "references_total": 5892
    },
    "ids": {
      "ids_total": 114,
      "duplicate_ids_total": 1,
      "unique_ids_total": 113
    },
    "images": {
      "picture": {
        "total": 48
      },
      "source": {
        "total": 195,
        "src_total": 3,
        "srcset_total": 192,
        "media_total": 192,
        "type_total": 3
      },
      "img": {
        "total": 59,
        "src_total": 59,
        "srcset_total": 0,
        "alt": {
          "missing": 8,
          "blank": 0,
          "present": 51
        },
        "loading": {
          "auto": 0,
          "lazy": 0,
          "eager": 0,
          "invalid": 0,
          "missing": 59,
          "blank": 0
        },
        "dimensions": {
          "missing_width": 10,
          "missing_height": 10
        }
      }
    },
    "iframes": {
      "total": 28,
      "loading": {
        "auto": 0,
        "lazy": 0,
        "eager": 0,
        "invalid": 0,
        "missing": 28,
        "blank": 0
      }
    },
    "amp": {
      "html_amp_attribute_present": false,
      "html_amp_emoji_attribute_present": false,
      "amp_page": false,
      "rel_amphtml": null
    },
    "data_nosnippet": {
      "valid": 0,
      "wrong_tag_type": 0
    },
    "obsolete_elements": {},
    "svgs": {
      "svg_element_total": 57,
      "svg_img_total": 1,
      "svg_object_total": 0,
      "svg_embed_total": 0,
      "svg_iframe_total": 0,
      "svg_total": 58
    },
    "app": {
      "app_id_present": false,
      "meta_theme_color": null
    },
    "anchors": {
      "hrefs_without_special_scheme": []
    },
    "dom-shape": {
      "bodyNodesCount": 3347,
      "bodyLeafNodesCount": 1302,
      "depth": {
        "mean": 16.966973886328724,
        "median": 16,
        "mode": 26,
        "stddev": 7.590193966722312,
        "skew": -1.1900916041506677,
        "range": {
          "min": 1,
          "max": 35
        }
      },
      "bodyNonLeafNodesCount": 2045,
      "branchFactor": {
        "mean": 3.902689486552567,
        "median": 3,
        "mode": 3,
        "stddev": 2.905995421981754,
        "skew": 0.3106300442610384,
        "range": {
          "min": 1,
          "max": 45
        }
      },
      "branchFactorByDepth": [
        {
          "depth": 0,
          "count": 1,
          "mean": 60,
          "stddev": 0,
          "skew": null,
          "median": 60,
          "mode": 60,
          "range": {
            "min": 60,
            "max": 60
          }
        },
        {
          "depth": 1,
          "count": 3,
          "mean": 5.333333333333333,
          "stddev": 4.0276819911981905,
          "skew": 0.827605888602368,
          "median": 3,
          "mode": 2,
          "range": {
            "min": 2,
            "max": 11
          }
        },
        {
          "depth": 2,
          "count": 5,
          "mean": 3.6,
          "stddev": 1.7435595774162693,
          "skew": 0.3441236008058427,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 2,
            "max": 7
          }
        },
        {
          "depth": 3,
          "count": 6,
          "mean": 5.166666666666667,
          "stddev": 4.844813951249544,
          "skew": 0.447213595499958,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 16
          }
        },
        {
          "depth": 4,
          "count": 9,
          "mean": 5.888888888888889,
          "stddev": 7.504731018119236,
          "skew": 0.38494236261286746,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 2,
            "max": 27
          }
        },
        {
          "depth": 5,
          "count": 18,
          "mean": 4.611111111111111,
          "stddev": 4.124976617965904,
          "skew": 0.3905746044945042,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 17
          }
        },
        {
          "depth": 6,
          "count": 21,
          "mean": 4.285714285714286,
          "stddev": 2.7969857439216024,
          "skew": -0.25537695922762466,
          "median": 4,
          "mode": 5,
          "range": {
            "min": 1,
            "max": 13
          }
        },
        {
          "depth": 7,
          "count": 27,
          "mean": 7.185185185185185,
          "stddev": 8.96510534542653,
          "skew": 0.24374339184981672,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 1,
            "max": 45
          }
        },
        {
          "depth": 8,
          "count": 66,
          "mean": 4.893939393939394,
          "stddev": 3.5638283291936594,
          "skew": -0.02976030163736965,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 1,
            "max": 31
          }
        },
        {
          "depth": 9,
          "count": 68,
          "mean": 7.088235294117647,
          "stddev": 7.821041178839763,
          "skew": 0.5227226402002053,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 45
          }
        },
        {
          "depth": 10,
          "count": 194,
          "mean": 3.5721649484536084,
          "stddev": 1.318953214120741,
          "skew": 0.43380230801820596,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 9
          }
        },
        {
          "depth": 11,
          "count": 74,
          "mean": 6.256756756756757,
          "stddev": 5.93901623214633,
          "skew": 0.5483663673334972,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 35
          }
        },
        {
          "depth": 12,
          "count": 169,
          "mean": 3,
          "stddev": 0.7692307692307693,
          "skew": 0,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 7
          }
        },
        {
          "depth": 13,
          "count": 50,
          "mean": 4.36,
          "stddev": 1.9774731350893235,
          "skew": -0.3236453576250939,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 1,
            "max": 13
          }
        },
        {
          "depth": 14,
          "count": 52,
          "mean": 3.1153846153846154,
          "stddev": 0.9933208900111509,
          "skew": 0.11616046389935494,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 7
          }
        },
        {
          "depth": 15,
          "count": 57,
          "mean": 6.333333333333333,
          "stddev": 2.408561726558669,
          "skew": -1.1071614388213236,
          "median": 7,
          "mode": 9,
          "range": {
            "min": 3,
            "max": 11
          }
        },
        {
          "depth": 16,
          "count": 83,
          "mean": 3.2650602409638556,
          "stddev": 1.0651583665204982,
          "skew": 0.24884585174851998,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 9
          }
        },
        {
          "depth": 17,
          "count": 77,
          "mean": 3.6493506493506493,
          "stddev": 1.4925462751749237,
          "skew": 0.43506232279099466,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 10
          }
        },
        {
          "depth": 18,
          "count": 65,
          "mean": 5.984615384615385,
          "stddev": 4.240799299451091,
          "skew": 0.7037860492482395,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 19
          }
        },
        {
          "depth": 19,
          "count": 160,
          "mean": 3.6125,
          "stddev": 0.9218697033746142,
          "skew": 0.6644105970267494,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 5
          }
        },
        {
          "depth": 20,
          "count": 190,
          "mean": 3.094736842105263,
          "stddev": 0.6419326774100766,
          "skew": 0.14758065049357758,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 2,
            "max": 7
          }
        },
        {
          "depth": 21,
          "count": 197,
          "mean": 3.020304568527919,
          "stddev": 0.20049154982874617,
          "skew": 0.10127393670836697,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 5
          }
        },
        {
          "depth": 22,
          "count": 190,
          "mean": 3.4526315789473685,
          "stddev": 1.2249031845963783,
          "skew": 0.36952437110082015,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 9
          }
        },
        {
          "depth": 23,
          "count": 63,
          "mean": 4.301587301587301,
          "stddev": 1.7148735029929538,
          "skew": -0.4072677647615203,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 1,
            "max": 9
          }
        },
        {
          "depth": 24,
          "count": 94,
          "mean": 2.9893617021276597,
          "stddev": 0.10259202937226551,
          "skew": -0.10369516947304103,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 2,
            "max": 3
          }
        },
        {
          "depth": 25,
          "count": 54,
          "mean": 5.018518518518518,
          "stddev": 1.0272354258043455,
          "skew": 0.018027531034590046,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 2,
            "max": 7
          }
        },
        {
          "depth": 26,
          "count": 10,
          "mean": 2.7,
          "stddev": 0.6403124237432849,
          "skew": -0.4685212856658179,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 3
          }
        },
        {
          "depth": 27,
          "count": 4,
          "mean": 1.75,
          "stddev": 0.82915619758885,
          "skew": 0.9045340337332909,
          "median": 1.5,
          "mode": 1,
          "range": {
            "min": 1,
            "max": 3
          }
        },
        {
          "depth": 28,
          "count": 3,
          "mean": 3.6666666666666665,
          "stddev": 0.9428090415820634,
          "skew": 0.7071067811865474,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 5
          }
        },
        {
          "depth": 29,
          "count": 5,
          "mean": 2.6,
          "stddev": 0.8,
          "skew": 0.7500000000000001,
          "median": 2,
          "mode": 2,
          "range": {
            "min": 2,
            "max": 4
          }
        },
        {
          "depth": 30,
          "count": 11,
          "mean": 1.2727272727272727,
          "stddev": 0.4453617714151233,
          "skew": 0.6123724356957945,
          "median": 1,
          "mode": 1,
          "range": {
            "min": 1,
            "max": 2
          }
        },
        {
          "depth": 31,
          "count": 8,
          "mean": 2.125,
          "stddev": 1.165922381636102,
          "skew": 0.10721125348377948,
          "median": 2,
          "mode": 2,
          "range": {
            "min": 1,
            "max": 5
          }
        },
        {
          "depth": 32,
          "count": 5,
          "mean": 1.2,
          "stddev": 0.4,
          "skew": 0.4999999999999999,
          "median": 1,
          "mode": 1,
          "range": {
            "min": 1,
            "max": 2
          }
        },
        {
          "depth": 33,
          "count": 5,
          "mean": 1.6,
          "stddev": 0.4898979485566356,
          "skew": -0.8164965809277259,
          "median": 2,
          "mode": 2,
          "range": {
            "min": 1,
            "max": 2
          }
        },
        {
          "depth": 34,
          "count": 2,
          "mean": 1.5,
          "stddev": 0.5,
          "skew": 1,
          "median": 1.5,
          "mode": 1,
          "range": {
            "min": 1,
            "max": 2
          }
        }
      ]
    }
  }
}
Custom metrics for https://web.dev

WPT test run results: http://webpagetest.httparchive.org/results.php?test=240820_DH_8
Changed custom metrics values:

{
  "_markup": {
    "favicon": true,
    "rel_alternate_mobile": false,
    "compatMode": "CSS1Compat",
    "noscripts": {
      "iframe_googletagmanager_count": 0,
      "total": 0
    },
    "buttons": {
      "types": {
        "button": 4
      },
      "total": 5
    },
    "form": {
      "target": {},
      "method": {
        "GET": 1
      },
      "elements": [
        {
          "tagNames": {
            "button": 1,
            "input": 1
          },
          "types": {
            "button": 1,
            "text": 1
          },
          "total": 2
        }
      ],
      "total": 1
    },
    "dirs": {
      "html_dir": "ltr",
      "body_nodes_dir": {
        "values": {},
        "total": 0
      }
    },
    "inputs": {
      "types": {
        "text": 2
      },
      "total": 2
    },
    "audios": {
      "autoplay": {},
      "total": 0
    },
    "classes": {
      "unique_names_total": 136,
      "references_total": 522
    },
    "ids": {
      "ids_total": 41,
      "duplicate_ids_total": 0,
      "unique_ids_total": 41
    },
    "images": {
      "picture": {
        "total": 8
      },
      "source": {
        "total": 2,
        "src_total": 0,
        "srcset_total": 2,
        "media_total": 2,
        "type_total": 0
      },
      "img": {
        "total": 8,
        "src_total": 8,
        "srcset_total": 6,
        "alt": {
          "missing": 0,
          "blank": 6,
          "present": 2
        },
        "loading": {
          "auto": 0,
          "lazy": 6,
          "eager": 0,
          "invalid": 0,
          "missing": 2,
          "blank": 0
        },
        "dimensions": {
          "missing_width": 8,
          "missing_height": 8
        }
      }
    },
    "iframes": {
      "total": 1,
      "loading": {
        "auto": 0,
        "lazy": 0,
        "eager": 0,
        "invalid": 0,
        "missing": 1,
        "blank": 0
      }
    },
    "amp": {
      "html_amp_attribute_present": false,
      "html_amp_emoji_attribute_present": false,
      "amp_page": false,
      "rel_amphtml": null
    },
    "data_nosnippet": {
      "valid": 0,
      "wrong_tag_type": 0
    },
    "obsolete_elements": {},
    "svgs": {
      "svg_element_total": 0,
      "svg_img_total": 2,
      "svg_object_total": 0,
      "svg_embed_total": 0,
      "svg_iframe_total": 0,
      "svg_total": 2
    },
    "app": {
      "app_id_present": false,
      "meta_theme_color": "#3740ff"
    },
    "anchors": {
      "hrefs_without_special_scheme": []
    },
    "dom-shape": {
      "bodyNodesCount": 535,
      "bodyLeafNodesCount": 202,
      "depth": {
        "mean": 10.128712871287128,
        "median": 10,
        "mode": 9,
        "stddev": 3.4972786204762483,
        "skew": 0.3227403343498618,
        "range": {
          "min": 1,
          "max": 15
        }
      },
      "bodyNonLeafNodesCount": 333,
      "branchFactor": {
        "mean": 4.111111111111111,
        "median": 3,
        "mode": 3,
        "stddev": 3.606337801566247,
        "skew": 0.30809956588885007,
        "range": {
          "min": 1,
          "max": 43
        }
      },
      "branchFactorByDepth": [
        {
          "depth": 0,
          "count": 1,
          "mean": 21,
          "stddev": 0,
          "skew": null,
          "median": 21,
          "mode": 21,
          "range": {
            "min": 21,
            "max": 21
          }
        },
        {
          "depth": 1,
          "count": 2,
          "mean": 8,
          "stddev": 3,
          "skew": 1,
          "median": 8,
          "mode": 5,
          "range": {
            "min": 5,
            "max": 11
          }
        },
        {
          "depth": 2,
          "count": 4,
          "mean": 5,
          "stddev": 3.7416573867739413,
          "skew": 1.0690449676496976,
          "median": 4,
          "mode": 1,
          "range": {
            "min": 1,
            "max": 11
          }
        },
        {
          "depth": 3,
          "count": 6,
          "mean": 4.666666666666667,
          "stddev": 1.3743685418725535,
          "skew": -0.24253562503633275,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 3,
            "max": 7
          }
        },
        {
          "depth": 4,
          "count": 8,
          "mean": 3.75,
          "stddev": 1.3919410907075054,
          "skew": 0.5388159060803247,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 7
          }
        },
        {
          "depth": 5,
          "count": 9,
          "mean": 5.333333333333333,
          "stddev": 2.4037008503093262,
          "skew": 0.13867504905630715,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 3,
            "max": 10
          }
        },
        {
          "depth": 6,
          "count": 14,
          "mean": 6.571428571428571,
          "stddev": 5.02443011287685,
          "skew": 0.7108126675452293,
          "median": 5,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 19
          }
        },
        {
          "depth": 7,
          "count": 33,
          "mean": 4.7272727272727275,
          "stddev": 6.916264218767358,
          "skew": 0.24974070865970596,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 43
          }
        },
        {
          "depth": 8,
          "count": 55,
          "mean": 4.0181818181818185,
          "stddev": 5.3648380780137055,
          "skew": 0.18978798677159578,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 43
          }
        },
        {
          "depth": 9,
          "count": 41,
          "mean": 4.170731707317073,
          "stddev": 2.0109752399828085,
          "skew": 0.5821711197831962,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 9
          }
        },
        {
          "depth": 10,
          "count": 41,
          "mean": 3.8536585365853657,
          "stddev": 1.8942914964826922,
          "skew": 0.45064792729652914,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 14
          }
        },
        {
          "depth": 11,
          "count": 40,
          "mean": 3.125,
          "stddev": 0.8422440263961508,
          "skew": 0.1484130442988812,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 1,
            "max": 7
          }
        },
        {
          "depth": 12,
          "count": 34,
          "mean": 4.882352941176471,
          "stddev": 1.2779741754353195,
          "skew": -0.09205746178983201,
          "median": 5,
          "mode": 5,
          "range": {
            "min": 3,
            "max": 7
          }
        },
        {
          "depth": 13,
          "count": 40,
          "mean": 3,
          "stddev": 0,
          "skew": null,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 3
          }
        },
        {
          "depth": 14,
          "count": 6,
          "mean": 3,
          "stddev": 0,
          "skew": null,
          "median": 3,
          "mode": 3,
          "range": {
            "min": 3,
            "max": 3
          }
        }
      ]
    }
  }
}
Custom metrics for https://html.spec.whatwg.org

WPT test run results: http://webpagetest.httparchive.org/results.php?test=240820_PF_9
Changed custom metrics values:

{}

Copy link
Member

@pmeenan pmeenan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tunetheweb tunetheweb merged commit 21e084c into main Aug 26, 2024
4 checks passed
@tunetheweb tunetheweb deleted the add-dom-shape-metric-to-markup branch August 26, 2024 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants