BullseyeCoverage Code Coverage Analyzer

Bullseye Code Coverage is not integrated with the MxSuite in versions 3.40.1.42946 and newer. However, you can still use Bullseye as a standalone tool.

hmtoggle_plus1Setting up the Bullseye Floating License

Full details on the Bullseye licensing strategy can be found at: http://www.bullseye.com/help/licenseManager.html. You can download the Bullseye installer for Windows from www.bullseye.com/download.

Briefly there are two steps to installing the floating license on your server.  Have your IT person perform these steps:

1.At the Server Computer, create a folder to store shared license files for MxVDev.

2.Share the folder. (It is advisable to suffix the share name with the '$' sign, since then it cannot be seen by people browsing the domain.)

3.Change the folder permissions so that any Bullseye user has full permission read and write to the folder.

1.Run the Bullseye installer on the Desktop PC.

2.Click Next.

3.Enter the license key in the License Key field.

4.Check the ‘Create the file’ checkbox.

5.Enter the full path of the license file the installer is to create in the field 'License Manager File'. (The license file should be under the share folder on the Server Computer; for example: \\ServerName\ShareName\Bullseye.lic)

6.Click Next.

7.Continue with the installation.

This creates and installs the shared license file on the server.  Further licenses can be added to the same license share folder. This is done by repeating the installation process with the second License Key.  (To do this, re-run the installer, you do not need to de-install.)
hmtoggle_plus1Accessing the Bullseye License

The following instructions are for anyone wishing to access the Bullseye license from their Desktop PC:

1.Run the Bullseye installer on your Desktop PC.

2.Click Next.

3.Leave the License Key field blank.

4.Leave the ‘Create the file’ checkbox unchecked.

5.Enter the full path of the existing license file in the field 'License Manager File' (for example: \\ServerName\ShareName\Bullseye.lic)

6.Continue with the installation.

If you need to change the path to the license file at any time the easiest way is to uninstall Bullseye and repeat steps 1 to 6 above.
hmtoggle_plus1Using Bullseye

MxVDev and Bullseye Coverage are standalone products. However, both tools have some features integrated with Visual Studio.

Note: When installing Bullseye Coverage, ensure that you select the option to install the integration with Visual Studio.

Taking Coverage Measurements

Use the following steps to get the coverage measurements that you need:

Step

Notes

1. Set COVFILE

Set COVFILE environment variable, then start MSVC:

•The environment variable COVFILE is used by Bullseye Coverage to determine the location of the Bullseye Coverage capture file.

•An initial value for COVFILE is set during Bullseye installation.

•This environment variable needs to be set to the proper value before MSVC is launched.

•If the COVFILE environment variable is changed after MSVC is launched, MSVC retains the previous setting.

•If you use a relative path, it is relative to the MSVC project directory. For example, if the MSVC project is in \MxSuiteSamples\TurnDoorSample\SUT\MSVC80, the setting below stores the capture file in the MxVDev project directory (TurnDoorSample):

..\..\test.cov

•The COVAUTOSAVE environment variable should always remain set to 0.

2. Start MSVC

Start MSVC and enable Bullseye Coverage:

•Start Visual Studio and open the solution for your MxVDev testing project.

•From the tools menu, select Enable/Disable Bullseye Coverage Build. (If the command is not in the menu, see http://www.bullseye.com/help/tool-microsoft.html.)

•Enable Build for Code Coverage.

3. Rebuild

Rebuild your MxVDev project.

•When you rebuild the SUT, your software is automatically instrumented to make code coverage measurements.

•It is important to perform a full rebuild of the project to instrument all the source files for coverage.

4. Clear

Clear the previous coverage measurements results:

•After building, check that the coverage file shows 0% coverage.

•To reset the coverage to 0%, from the Bullseye Coverage review tool, Right-click on the node in the Files Tree and select 'Reset Measurements'.

5. Execute

Execute MxVDev scenarios:

•You can run the scenarios individually, as part of a Regression script, or from the command line.  The coverage results are accumulated.

•All VMCs will be rebuilt with instrumentation code. Bullseye’s documentation explains how to tell Bullseye to ignore specific parts assemblies or parts of assemblies.

•Results will all be saved to the same coverage file.

In Visual Studio, select Tools->Enable/Disable Bullseye Coverage Build from the main menu.

Bullseye-enable

 

 
hmtoggle_plus1Troubleshooting

Description

After enabling Coverage Build and rebuilding your project, you do not see any entries in the coverage file. During the build, you do not see this start-up banner:

BullseyeCoverage Compile C++ version platform
Copyright (c) Bullseye Testing Technology 1990-year

Cause

One or more of the following conditions can cause this problem.

1.The Microsoft Visual Studio Executable Directories setting is missing the BullseyeCoverage/bin directory.

2.The PATH environment variable is not set properly.

3.Your makefile or build script invokes the real compiler with its absolute path.

4.A compiler interceptor is not installed.

Solution

1.Reinstall BullseyeCoverage. If that does not solve the problem, manually add the BullseyeCoverage/bin to the Executable Directories list in Visual Studio. To do this:

a.Launch Visual Studio.

b.Select Tools->Options from the Visual Studio menu.

c.Navigate to Projects and Solutions (8.0) or Projects (7.1).

d.Select VC++ Directories.

e.Add the BullseyeCoverage\bin to the top of the list.

BullseyeToolsOptions

2.The BullseyeCoverage/bin directory must occur in the PATH before the real compiler. The simplest way to accomplish this is to put BullseyeCoverage/bin first.
 
For example, in Windows:

PATH=c:\Program Files\BullseyeCoverage\bin;c:\Program Files\Microsoft Visual Studio\VC\bin;...

3.Modify your makefile or build script to use the basename of the compiler rather than its absolute path. For example, in a makefile:

CC=/usr/bin/cc   # wrong

CC=cc            # right

4.The Windows setup program creates compiler interceptors for the compilers selected on the Choose Compilers page.
 
To manually create a compiler interceptor:

cd BullseyeCoverage\bin
copy covc.exe compiler.exe

This is a bug in the Visual Studio linker.

Solution

1.Check for and install available updates for Visual Studio.

2.Disable incremental linking. In Visual Studio, select Project -> Properties -> Configuration Properties -> Linker General -> Enable Incremental Linking ->No

Bullseye.LinkError

For more information, see Bullseye Help: http://www.bullseye.com/help/trouble-lnk1000.html

 

 

Related Topics:

Regression Testing

RapiCover