Instrumenting binaries issue

[Piotr-Miller]

Hi Guys,

Got issue with instrumenting binaries on build server (Win Server 2019 Standard) - couple days ago altcover stopped seeing all of my dlls and see only third party ones: CefSharp.Core.dll, Ben.Demystifier.dll, etc, which I need filter out anyway.

After running command:
altcover --inplace -i "Path to release folder" -s "xxxx" --single
coverage.xml file looks like below:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<CoverageSession xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="1" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" minCrapScore="0" maxCrapScore="0" />
  <Modules />
</CoverageSession>

First of my thoughts was something wrong happend with binaries, but on my local machine (Windows 10 Enterprise) all is working flawlessly:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<CoverageSession xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <Summary numSequencePoints="50858" visitedSequencePoints="0" numBranchPoints="32610" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="358" minCyclomaticComplexity="1" visitedClasses="0" numClasses="1609" visitedMethods="0" numMethods="10730" minCrapScore="0" maxCrapScore="0" />
  <Modules>
    <Module hash="63-AF-83-07-9C-67-6A-43-0A-7F-E0-A3-E3-9C-CB-4D-1B-B8-A5-02">
      <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="1" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" minCrapScore="0" maxCrapScore="0" />
      <ModulePath>C:\xxx\__Saved\xxx.dll</ModulePath>
      <ModuleTime>2022-06-02T11:01:34.1420363Z</ModuleTime>
      <ModuleName>xxx</ModuleName>
      <Files />
      <Classes>
        <Class>
          <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="0" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" minCrapScore="0" maxCrapScore="0" />
          <FullName>yyyyy</FullName>
          ...

I'm using on both locations (my dev machine and build server) the same set of dlls, the same version of altcover.

I'm realising that most probable source of this issue is related with build server configuration, not altcover itself,
but decided to ask you guys anyway.

Am I missing something?

Thanks in advance for your support.

BR,
Piotr Miller

[SteveGilham]

The most obvious cause for an assembly to be skipped is that matching debug symbols -- the link between the IL in the assembly and the source code being covered -- are not being found. It is even possible that they are not being generated at all on the build server, as they are switched off by default for the Release build configuration.

The other assemblies you mention do come packaged with embedded debug information -- like this

which is the first place the instrumentation process looks. Next is the symbol file location from the assembly debug header (typically, the intermediate build folder), next is in the same directory as the assembly, next are any folders listed as --symbolDirectory parameters, stopping at the first suitably named file found.

[Piotr-Miller]

Hi Steve,

You were right, process on build server is trying to pick up symbol files from subsequent compilation:
Assembly E:\xxx.Build189__Saved\zzz.Connectivity.dll symbols from image 'E:\GitLab_builds_folder\net6.0-windows10.0.17763.0\win-x64\zzz.Connectivity.pdb'
On my dev machine process is using __Saved folder, and both binary file and symbol are matched:
Assembly E:\xxx.Build189__Saved\zzz.Connectivity.dll symbols from folder '"E:\xxx.Build189__Saved\zzzz.Connectivity.pdb"'

Build server:

E:\xxx.Build189_Saved\zzz.Connectivity.dll : beginning process
Assembly E:\xxx.Build189_Saved\zzz.Connectivity.dll symbols from image 'E:\GitLab_builds_folder\net6.0-windows10.0.17763.0\win-x64\zzz.Connectivity.pdb'
E:\xxx.Build189_Saved\zzz.Connectivity.dll : ** not a valid assembly because Mono.Cecil.Cil.SymbolsNotMatchingException: Symbols were found but are not matching the assembly
at Mono.Cecil.ModuleDefinition.ReadSymbols(ISymbolReader reader, Boolean throwIfSymbolsAreNotMaching)
at AltCover.ProgramDatabase.readSymbols@106.Invoke(String pdbpath) in ///AltCover.Engine/ProgramDatabase.fs:line 116
at AltCover.Main.I.assemblies@543-4.Invoke(Unit unitVar0) in ///AltCover.Engine/Main.fs:line 551
at AltCover.Main.I.imageLoadResilient[a](FSharpFunc2 f, FSharpFunc2 tidy) in ///AltCover.Engine/Main.fs:line 404 **
E:\xxx.Build189__Saved\zzz.Connectivity.pdb : beginning process
E:\xxx.Build189__Saved\zzz.Connectivity.pdb : ** not a valid assembly because System.BadImageFormatException: Format of the executable (.exe) or library (.dll) is invalid. **

Coverage Report: C:\Users\pmiller\coverage.xml

Local machine:

E:\xxx.Build189__Saved\zzz.Connectivity.dll : beginning process
Assembly E:\xxx.Build189__Saved\zzz.Connectivity.dll symbols from folder '"E:\xxx.Build189__Saved\zzzz.Connectivity.pdb"'
=> E:\xxx.Build189__Saved\zzz.Connectivity.dll
E:\xxx.Build189__Saved\zzz.Connectivity.pdb : beginning process
E:\xxx.Build189__Saved\zzz.Connectivity.pdb : ** not a valid assembly because System.BadImageFormatException: Format of the executable (.exe) or library (.dll) is invalid. **

Coverage Report: C:\coverage.xml

Assembly symbols from image 'AltCover.Recorder.pdb'
Assembly E:\xxx.Build189__Saved\zzz.Connectivity.dll symbols from folder '"E:\xxx.Build189__Saved\zzz.Connectivity.pdb"'
E:\xxx.Build189\zzz.Connectivity.dll
<= zzz.Connectivity, Version=1.0.8193.24506, Culture=neutral, PublicKeyToken=null

For build server passing explicitly --symbolDirectory parameter as E:\xxx.Build189 did not do the job, unfortunately.

BTW, huge thanks for your support.

[SteveGilham]

No problem.

The code as it stands just works through a list of locations - the one specified in the debug header, co-located with the assembly, then each of the --symbolDirectory values, stopping as soon as it finds one with the expected name, on the -- now invalidated -- assumption that there would not be cross-contamination from other builds which might be found first. As Mono.Cecil doesn't give a convenient API for doing this, I'm in the process of writing code to minimally interpret a .pdb file to check whether it has the same GUID as in the assembly debug header, as well as simply having the expected name.

For the moment, the work-round is to change the build, at least for the configuration under test, to generate embedded .pdb information

    <DebugType>embedded</DebugType>

which puts the debug information in the first place that the process looks, and avoids all the problems involved with separate files.

[Piotr-Miller]

Thanks Steve, now it's clear for me what is going on behind the scene.
It would be good to force altcover to search symbols in input directory firstly.

[SteveGilham]

Here's a build for you to evaluate where I've reworked the symbol file selection, and should resolve the original underlying issue

altcover.8.3.677-github-pre.nupkg.zip

[SteveGilham]

Now in release v8.3.838