Code Coverage

The Simulator in the MPLAB® X IDE has a code coverage feature that can be used to indicate those sections of your program that have been executed and those sections that have not.

Programs contain a large amount of conditional code, i.e., code that is executed only if some condition is met. You would be familiar with C's compound statements, which encapsulate statements executed conditionally, such as those following, shown in their general form.

if(condition) {
  statement;    <--- executes only if condition is true
} else {
  statement;    <--- executes only if condition is false
}
while(condition) {
  statement;    <--- executes only if condition is true
}
switch(condition) {
case value:
  statement;    <--- executes only if condition is equal to value
}

Not quite so obvious are some statements that contain conditional expressions, i.e., expressions inside the statement that are only evaluated if some condition is met, even if the statement itself executes unconditionally.

condition ?
    expression:    <--- evaluated only if condition is true
    expression;    <--- evaluated only if condition is false
if(
    expression    <--- evaluated unconditionally
    &&
    expression    <--- evaluated only if first expression is true
) { ...

And then there are plain-looking C statements and expressions that you might be surprised to discover could be encoded into sequences that contain conditional assembly instructions. For example, consider the following statement that simply copies a single-bit value from one bit-field to another.

dest.b = src.b;

By itself, this statement executes unconditionally, but it is possible that execution could take either of two paths through the assembly code generated for this statement, as is indeed the case when compiling for an 8-bit device using the MPLAB XC8 compiler. For this statement, that compiler might produce an assembly sequence that executes in a manner similar to the following equivalent C code:

if(src.b) {
  dest.b = 1;        <--- executes only if scr.b is true
} else {
  dest.b = 0;        <--- executes only if src.b is false
}

When validating projects, it is essential that all parts of your program have been executed to ensure that they are working as expected and that they do not adversely affect other parts of the program. Expressions within a statement can have side effects (especially if that expression is a function call!), and when they execute, they could affect variables used by other sections of your program. Since a conditional statement can be nested inside other conditional statements, the number of condition combinations to test becomes very large very quickly. This is why a code coverage tool is important. It confirms that your program has encountered every condition and that the code associated with those conditions has been executed. This coverage information reinforces the validity of tests that simply show a correct result.

Usage

Here is how you can use the code coverage feature when using the MPLAB X IDE simulator:

Open the project properties to enable the code coverage feature. Select the Simulator (1) category, the Code Coverage Options option category (2), and finally select a Code Coverage mode (3).

project_properties.png

There are two modes that enable the code coverage feature. They differ only in how the code coverage data is collected. The Enabled/Reset on Run mode will display only the code that has been executed (and not executed) since the location where execution last recommended (irrespective of whether your program was reset or stopped at a breakpoint). The Enabled/Reset on POR mode will display the cumulative total of all the code locations which have been executed (and not executed) since the device was reset.

Once this feature is enabled, run your code as appropriate.

Display

Code coverage information in shown in the Output window (1) under the simulator’s Debugger Console tab (2) and is available after execution has being paused or reaches a breakpoint.

The following is an example of a program’s Code Not Covered Map, which indicates the code that has not been executed. (There will also be a similar Code Coverage Map, which shows the code that was executed.)

debugger_console_coverage.png

The first section in each map is labeled Address Ranges and is a list of program address ranges. The assembly instructions found at these addresses are those that were not executed (since we are using the Code Not Covered Map as an example). For example, the instruction at address 0x76A did not execute, nor did the instructions found at address 0x770 through to address 0x773.

If C source information is available for any of the address ranges shown in the first section, the IDE will show the corresponding C line numbers and source file name for those ranges in the second section, labeled Source Files. In the figure above, you can see that the second address range listed (0x770-0x773) actually corresponds to the C statement on line 46 of main.c. The from Line and to Line are links that you can click to take you to those source lines in the editor.

C source information will not be shown if the address range contains assembly instructions that have no corresponding source code (e.g., the instructions form part of the runtime startup code linked in by the compiler), or if the assembly instructions represent only some of those which make up the statement. For MPLAB XC16/32, any library code linked in to the project will appear only in the Address Ranges section; however, MPLAB XC8 library code will additionally show C source information if that is relevant. The last reference in the above figure shows that the C source lines 14 through 42 in the XC8 integer division library source file (awdiv.c, in the compiler’s sources directory) was not executed.

If there is no C source information for an address range, you can find the relevant assembly instructions in the IDE’s Program Memory window. The following figure shows the assembly instructions that correspond to the second address range (0x770 - 0x773) in the earlier Code Not Covered Map. The BTFSS and GOTO instructions above this range of assembly create a conditional branch in the execution path.

program_memory.png

If you need more information to help you interpret the assembly code, you can use the assembly list file if you are using the MPLAB XC8 compiler. You can find this file by going to the Files window (1), navigating to the dist/default/debug directory (2), and opening the file with the .lst extension (3) in the editor by double-clicking its icon.

assembly_list_file_location.png

If you are using MPLAB XC16/32, open the disassembly window, available under Window > Debugging > Disassembly.

dissasembly_menu.png

If you are compiling your project with the optimizers enabled, there is the potential that the C source information reported by the IDE might not accurately match the address ranges specified. This is the same issue you might experience when trying to set breakpoints, step, or watch variables in a project that was built with the optimizer enabled. You can consider switching to level 1 optimization if you are using the XC16/32 compiler, or turning on the debug optimization setting or disabling the assembler optimizations entirely if you are using the XC8 compiler, but remember that with different optimizations selected, the compiler might produce different assembly output, and the code coverage results will only be valid for these optimization settings.

© 2017 Microchip Technology, Inc.
Notice: ARM and Cortex are the registered trademarks of ARM Limited in the EU and other countries.
Information contained on this site regarding device applications and the like is provided only for your convenience and may be superseded by updates. It is your responsibility to ensure that your application meets with your specifications. MICROCHIP MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WHETHER EXPRESS OR IMPLIED, WRITTEN OR ORAL, STATUTORY OR OTHERWISE, RELATED TO THE INFORMATION, INCLUDING BUT NOT LIMITED TO ITS CONDITION, QUALITY, PERFORMANCE, MERCHANTABILITY OR FITNESS FOR PURPOSE. Microchip disclaims all liability arising from this information and its use. Use of Microchip devices in life support and/or safety applications is entirely at the buyer's risk, and the buyer agrees to defend, indemnify and hold harmless Microchip from any and all damages, claims, suits, or expenses resulting from such use. No licenses are conveyed, implicitly or otherwise, under any Microchip intellectual property rights.