How to handle Coveragescanner error messages and warnings

What does the warning "Instrumentation is different" mean?

Sometimes during compilation of a C/C++ program, Coco issues a warning of the form

Warning (Squish Coco): Compilation of 'insertargs/header.h': Instrumentation of source file 'insertargs/header.h' is different

followed by several more lines. What does this mean?

Explanation

In C and C++, the content of a source file does not completely determine to what code it compiles. This is because preprocessor symbols can be set at compile time and determine – usually via #ifdef statements – which source code gets actually translated.

This means that it is possible that a source file is compiled several times with different preprocessor settings. This may lead to different compiled code and, if Coco is involved, to different instrumentations of the same file. If Coco notices this, it issues the "Instrumentation is different" message.

The message is however only a warning. Coco treats the different versions of the file as different files and when they are displayed in CoverageBrowser or in a report, the versions are distinguished by numbers at the end. A file header.h could be displayed as header.h#1 and header.h#2 if there are two versions of it.

Example with injection of compiler arguments

What exactly is going on, and how to understand the error message, are best seen from an example. First, we have a source file, header.h that can be compiled to different code depending on an external setting. It defines a function, letter(), the compilation of which depends on whether the preprocessor symbol A is defined or not:

#ifndef HEADER_H
#define HEADER_H

static inline char letter()
{
#ifdef A
    return 'a';
#else
    return 'b';
#endif
}

#endif

Then there are two files that use this header. One is a.cpp,

#include "a.h"
#include "header.h"
#include <iostream>

void a()
{
    std::cout << "Letter A: " << letter() << "\n";
}

the other one, b.cpp. (The header files, a.h and b.h, are here omitted.)

#include "b.h"
#include "header.h"
#include <iostream>

void b()
{
    std::cout << "Letter B: " << letter() << "\n";
}

Note that the .cpp files are almost the same, and by themselves, do not contain code that is influenced by the symbol A. However, both files include header.h which does. Therefore, the preprocessed versions of a.cpp and b.cpp – the versions that CoverageScanner sees and instruments – contain differently-preprocessed versions of header.h.

Finally we have a main.cpp, that uses them both.

#include "a.h"
#include "b.h"

int main()
{
    a();
    b();
    return 0;
}

If a.cpp and b.cpp are compiled with different flags, where the symbol A is defined for a.cpp but not for b.cpp, and we are also instrumenting with Coco, we expect to see the above warning.

If this project was compiled with a Makefile, it might look like this:

main: main.cpp a.o b.o
        g++ -o main main.cpp a.o b.o

a.o: a.cpp a.h header.h
        g++ -c -o a.o a.cpp

b.o: b.cpp b.h header.h
        g++ -c -o b.o b.cpp

To build with code coverage, we prepared an instrumented script that resembles this:

#!/bin/bash

# this script assumes the COCO wrappers are already in your PATH.

export COVERAGESCANNER_ARGS='--cs-on'
export COVERAGESCANNER_ARGS+=' --cs-mcdc'
export COVERAGESCANNER_ARGS+=' --cs-mcc'
export COVERAGESCANNER_ARGS+=' --cs-inject-arg=/a.cpp/-DA'
"$@"

Here, we use the coveragescanner --cs-inject-arg option. For any g++ line containing a.cpp, we add an argument -DA.

When such a project is compiled with instrumentation enabled, the warning from above appears. Its full text is:

coco/samples/insertargs> ./instrumented make
g++ -c -o a.o a.cpp
g++ -c -o b.o b.cpp
g++ -o main main.cpp a.o b.o
Warning (Squish Coco): Compilation of 'samples/insertargs/header.h': Instrumentation of source file 'samples/insertargs/header.h' is different
Warning (Squish Coco):   Source line 7 of file 'samples/insertargs/header.h' is differently instrumented in the database 'main.csmes' and 'b.o.csmes'
Warning (Squish Coco): Original:
Warning (Squish Coco):     4: static inline char letter()
Warning (Squish Coco):     5: {
Warning (Squish Coco):     6: #ifdef A
Warning (Squish Coco): >>> 7:     return 'a';
Warning (Squish Coco):     8: #else
Warning (Squish Coco):     9:     return 'b';
Warning (Squish Coco):     10: #endif
Warning (Squish Coco): [ALL] Inconsistent instrumentations in: samples/insertargs/header.h
Warning (Squish Coco):
Warning (Squish Coco): [1/2] Source files:                     samples/insertargs/a.cpp
Warning (Squish Coco): [1/2] Distinct define flags:            -DA
Warning (Squish Coco):
Warning (Squish Coco): [2/2] Source files:                     samples/insertargs/b.cpp
Warning (Squish Coco):
Warning (Squish Coco): [ALL] Common instrumentation options:   --cs-combine-switch-cases
Warning (Squish Coco):                                         --cs-count
Warning (Squish Coco):                                         --cs-function-coverage
Warning (Squish Coco):                                         --cs-line-coverage
Warning (Squish Coco):                                         --cs-mcc

One sees here:

• The first line shows that the error occurred during the compilation of the file header.h.
• The second line shows that the problem was found when the information from the file b.o.csmes was merged into main.csmes, and that it occurred on line 7.

  What actually happened was that at first, main.csmes did not contain any information about header.h. Then, during the link step, a.o.csmes got merged. It contained information about header.h in the version with the symbol A defined. And after that, b.o.csmes got merged too – with a different version of header.h. This then caused the inconsistency.

• The next lines show the line that is instrumented differently: it is the line with return 'a'; and the first line that is only part of the code when A is defined.

  The three lines before and three lines after this line are also shown. The number of context lines can be set with the flag –cs-verbose-source-lines.

• These differences could be caused by compiler options. The following lines, beginning with the first line that starts with [ALL], summarizes the options in a form that is easy to read.

  The summary consists of several sections, each prefixed by an expression in square braces. The prefix is either [ALL] or something like [1/2]. Lines that start with [ALL] refer to all files while others refer to a group of files that were compiled with the same command line options. The first number is the number of the file group and the second is the number of all file groups. This means that [1/2] refers to the first of two groups.

  All file group sections have the same form: First there is a line that lists the contents of the group. In our example, the first group (and also the second) consists of just one file:

  Warning (Squish Coco): [1/2] Source files:                     insertargs/a.cpp

  The following line then lists the distinct define flags, that is those that do not appear in all groups. In the first group, this is -DA, as expected, since we have defined the symbol A only for the file a.cpp. The second group has no distinct flags.

  There are actually four kinds of command line options that can be listed for each group:

  • Distinct define flags: The flags that define compiler symbols.
  • Distinct include directories: This is for the flags the define the directories in which #include finds header files. (The directories are sorted alphabetically, so that the rare errors in which the compilation depends on the order of the include paths cannot be caught this way.)
  • Distinct instrumentation options: This is for coveragescanner options.
  • Distinct compiler flags (others): For the rest of the flags.

  After the file groups and their options, the command line options common to all the source files are listed with the prefix [ALL].

  In our example, only instrumentation options are listed.