Gcov

From ASCEND
Jump to: navigation, search

We can do coverage testing of ASCEND using the 'gcov' tool. To do this, build ASCEND using 'scons GCOV=1' (see building ASCEND for more details) and the coverage 'instrumentation' will be added.

To get coverage information in plaintext form, for a single code file, you can use the gcov command, eg

# run all this from the top-level working directory
cd ~/ascend
# compiler with coverage instrumentation enabled
scons -j4 DEBUG=1 GCOV=1 test solvers models ascend
# run the test
test/test solver_qrslv.bug564
# process the covereage data
gcov -p ascend/compiler/mergeinst.c --object-directory ascend/compiler
# view the results
vim ascend\#compiler\#merginst.c.gcov

Important notes

  • You don't get complete gcov data if a program exits or aborts prematurely. You can get a trace though by calling __gcov_flush() at some point close to where the error occurs (maybe you can even call it from GDB, haven't tried that). See this page.
  • gcov shows results one file at a time. Tools like ggcov, lcov, gcovr can give some aggregated coveraged data, create GUI/HTML output, and convert to 'cobertura' output format for use with other tools.
  • See also gprof if you want information about time spent in different functions.

lcov

The lcov tool can be used to produce nice, easy-to-read coverage reports from our CUnit test suite (and probably more).

Note: we found that the version of LCOV available in the Ubuntu repositories was not functional, and the newer code from git clone https://github.com/linux-test-project/lcov.git needed to be used. It can be built and installed using sudo make install, which will install it to /usr/local/bin/lcov.

To run lcov with ASCEND, try the following:

#!/bin/bash

LCOV=/usr/local/bin/lcov
GENHTML=/usr/local/bin/genhtml

scons MALLOC_DEBUG=1 GCOV=1 CC=gcc CXX=g++ -j4 -c test ascend models solvers
scons -j4 test ascend models solvers
$LCOV -c -i -d . mycov-1.info
ASCENDLIBRARY=models:solvers/qrslv LD_LIBRARY_PATH=. test/test general_color general_dstring general_hashpjw general_list general_listio \
general_mem general_pool general_pretty general_stack general_table \
general_tm_time general_ospath general_env general_ltmatrix general_ascMalloc \
utilities_ascEnvVar utilities_ascPrint \
utilities_ascSignal utilities_readln utilities_set linear_qrrank linear_mtx \
compiler_basics  compiler_expr 	compiler_fixfree  compiler_fixassign
$LCOV -c -d . mycov-2.info
$LCOV -a mycov-1.info -a mycov-2.info -o mycov-tot.info
$LCOV -r mycov-tot.info "*stdout*" -o mycov-tot2.info
$GENHTML mycov-tot2.info

Note that the command $LCOV -r mycov-tot.info "*stdout*" -o mycov-tot2.info is required, because of the way the ascend/compiler/scanner.l file is compiled. In the gcov output it is shown with a source-code filename of <stdout>, which is incorrect/false.

The above approach has been implemented in a Python script relerrorlist:test/test-cov.py which be run to produce full coverage reports for the CUnit test suite in ASCEND. So far we don't have any coverage testing from the Python or C++ parts.

ggcov

A GUI for 'gcov' exists, ggcov, see also this presentation for details. The following process can be used on Ubuntu 12.04:

  1. download ggcov 0.8.4 from here (note that the GIT code linked from the ggcov SF.net page is currently buggy)
  2. install the Glade and GNOME UI developer libraries for your distro. On Ubuntu, use apt-get install libglade2-dev libgnomeui-dev.
  3. ./configure --prefix=/usr/local
  4. make -j4

Currently, ggcov gets all confused if there aren't *.o files to match the *.gcno files produced by the --coverage instrumentation in GCC. You can quickly create at least the need .o files by copying all the .os files that SCons produces to corresponding .o files:

find . -name "*.os" | xargs rename 's/\.os$/\.o/'

If you do that, then ggcov gives bunch of warning due to symbol conflicts (static functions are used in ASCEND with the same name in more that one location, probably we can fix these with some renaming and/or merging):

Callgraph name collision: /home/john/ascend/ascend/compiler/logrelation.c:DoBreakPoint and /home/john/ascend/ascend/compiler/instance_io.c:DoBreakPoint
Callgraph name collision: /home/john/ascend/ascend/compiler/rel_common.c:CmpP and /home/john/ascend/ascend/compiler/exprsym.c:CmpP
Callgraph name collision: /home/john/ascend/ascend/compiler/relation.c:realloc_term_stack and /home/john/ascend/ascend/compiler/logrelation.c:realloc_term_stack

Other than that, ggcov seems to work OK now.

Error creating thumbnail: File missing
GGCOV showing the results of test/test general_ospath, which has fairly reasonable coverage in testing at this stage.

To get fresh tallies of covered code you might need to remove the coverage data, eg

find . -name "*.gcda" | xargs rm