Setting Up for Model Coverage

Effective Release 3.36.43.27730

hmtoggle_plus11. Enable Coverage in the Transform's Properties

In MxTransIt, select the S-function Transform and set the Coverage Information properties:

Simulink-Coverage

•To enable coverage, set the first property to True.

•To generate a cumulative report, set Generate Cumulative HTML Report for all scenarios run to True. The report is updated as each scenario completes.

•To generate separate reports for each scenario, set Generate HTML Report On Scenario Completion to True.

There are three properties that control how MxVDev generates coverage information. These properties are set in the Simulink transform itself. Note that the transform turns off all workspace and HTML code generation properties in the harnessed model (MxVDev version) REGARDLESS of these settings, meaning that there is no coverage information sent to the workspace nor HTML files generated through the harnessed model (the original model is untouched). The reason for this is to prevent Simulink Verification and Validation from preventing COM object communications. In addition, a project now supports multiple Simulink transforms to generate coverage. (See below for explanation of the data saved.)

1.Enable Coverage – when this parameter is set to true, coverage generation is enabled. (To use the other two properties, this property must be true). If coverage is enabled in the transform, coverage data is saved at the Scenario level after each Scenario is run. The coverage data is saved in the file

<ModelPath>\Coverage\<ScenarioName>_<TransformName>.cvt

where <ModelPath> is the full path to the harnessed model, <ScenarioName> is the root name of the Scenario that generated coverage information, and <TransformName> is the name of the S-Function transform in MxVDev. Note that only the Scenario file name is used, meaning that if there are two scenarios in a project that have the same name but reside in different subdirectories, the second will overwrite the first.

2.Generate HTML Report On Scenario Completion – When “Enable Coverage” and this parameter are both true, HTML reports are generated for each Scenario that is run in the project. Two reports are generated:

•One that contains the totals between each successive module reset and final scenario totals (named <ModelPath>\Coverage\<ScenarioName>_<TransformName>.html), and

•One report that contains coverage differences between each successive module reset (named <ModelPath>\Coverage\<ScenarioName>_<TransformName>_diffs.html). Explanation of how to read the report is below.

3.Generate Cumulative HTML Report for all scenarios run – When “Enable Coverage” and this parameter are both true, two HTML reports are generated for cumulative Scenario runs while the Harness is active. These are the successive totals as each Scenario is run. The reports are in these files:

•<ModelPath>\Coverage\<HarnessedModelName>.html, which includes each Scenario and totals, and

•<ModelPath>\Coverage\< HarnessedModelName>_diffs.html, which is the report of Scenarios along with the differences in coverage between them.

Note that the reports use the harnessed model name so that multiple transforms that reference the same model do not overwrite each other.

Scripts

There are two scripts that are used to generate the HTML reports. They are located in a directory named Harness\Matlab, which is in the MxSuite installation directory.

•After each Scenario, if the "Generate HTML Report On Scenario Completion" parameter is true, MxVGenerateHTMLCoverage.m is  run. This generates the two scenario reports (the script is run with the path to the <ScenarioName>_<TransformName>.cvt file). This report is on a per scenario basis.

•After each Scenario, if the "Generate Cumulative HTML Report for all scenarios" parameter is true, MxVGenerateHTMLCoverageTotals.m is run. This loads all of the .cvt files that have been generated to that point in the MxVDev session. The first parameter is the harnessed model name, the variable length parameters are the cvt files that have been generated. This is useful for a Regression Test that needs cumulative results.

These scripts may be used as a starting point to generate your own reports. The information that is in the coverage reports is described above. MxVDev bypasses the MATLAB HTML code generation due to two factors:

•HTML generation within Matlab prevents MxVDev from advancing the Scenario due to the fact that it is a background operation (as opposed to issuing the command itself).

•The coverage data generated cannot necessarily be captured by MxVDev depending upon the settings. By having MxVDev control the recording and saving of the code coverage data for later post processing, all code coverage data is recorded and available for the user. You can use it to perform additional reporting beyond that of the default reports that MxVDev provides.
hmtoggle_plus12. Set Coverage Settings in the Model

Access the Coverage Settings dialog. The settings must be set in the harnessed model. If you specify the settings in the original model, they are copied when it is harnessed.

At least one of the Coverage check boxes on the Coverage tab must be selected. If using Coverage for MATLAB files, select the desired Coverage metrics.

2011-screen1

Coverage data is saved based upon the parameters set in the harnessed model. (The original model may have coverage disabled while the harnessed model may not).  Note that these settings control the data that is saved (and therefore the data that is presented in the HTML reports). The settings on the  Results and Reporting tabs are controlled by MxVDev.

In addition, model coverage options on the Options tab are in effect:

Simulink-OptionsTab

The following data is saved for each property that is set:

•The “Enable Coverage” option saves the coverage and test data that is later loadable by the Matlab cvload function. With the exception of the last item, each test in the coverage data cell array corresponds to a module reset point. The first item covers from model start to the first module reset in the Scenario. The second value corresponds to the coverage between the first module reset point, etc., with the second to last value corresponding to the coverage between the last module reset (or Scenario start if the scenario does not contain any module reset signals) and the Scenario end. The last item is the coverage totals of the entire Scenario.

•HTML reports for single Scenarios contain the test data for each test run that was saved for the Scenario. These are the coverage data broken down, Test 1 corresponds to the Scenario start to the first module reset, Test 2 corresponds to the data between the first and second module reset, etc. The final test (listed as “Test n” in the summary but labeled “Totals” in the details) contains the full totals for the scenario.

•HTML reports with diffs for single Scenarios contain the same results as above, however between each test is a difference in coverage report. Test 1 corresponds to the coverage from the start to first reset, Test 3 corresponds to the coverage between the first reset and second reset and Test 2 corresponds to the difference between those two tests. Test 5 corresponds to the coverage between the second reset and third reset, and Test 4 corresponds to the differences in coverage between Tests 5 and 3, etc. Again, the final test is the Totals for the Scenario.

•HTML reports for cumulative Scenario runs are the same as above, however each test in the report corresponds to only the final Scenario results in the order that the Scenarios are run. For example, if Scenarios A, B, and C are run in that order, the final report corresponds to Test 1 being the total coverage for Scenario A, Test 2 being the total coverage for Scenario B, Test 3 being the total coverage for Scenario C, and Test 4 Totals being the total coverage for all Scenarios. The difference reports work the same way, Test 1 corresponds to Scenario A, Test 2 corresponds to the differences between B and A, Test 3 corresponds to Scenario B, Test 4 corresponds to the differences between Scenarios C and B, Test 5 corresponds to Scenario C and Test 6 (Totals) corresponds to the cumulative coverage.

Any additional advanced coverage examination, manipulation, or reporting (for example, broader difference reports, or coverage intersection analysis) can be performed by loading the MxVDev harnessed model that generated the coverage data in MATLAB, and calling cvload to load the coverage data files that were generated during the Scenario run. See the Simulink Verification and Validation documentation for more information.

Notes:

•For coverage to be saved, BOTH the “Enable Coverage” property must be true *and* the “Coverage for this model” in the GUI above must be checked. The data saved is the data generated by the settings above.

•The Coverage reports are stored in the "Coverage" subfolder of the model folder.

 
hmtoggle_plus1Coverage for Referenced Models

The MxSuite does not activate coverage for referenced models nor any of the options of what metrics are saved, this is a manual step. MxSuite only changes the overall points at which coverage data is saved. To activate coverage for submodels, perform the following steps:

1.Edit the ORIGINAL top level model. Note that it is important to do this in the original model, because on an MxVDev reharness, these settings revert back to the original if they are changed in the MxVDev model.

2.In the Coverage Settings panel, ensure that "Coverage for referenced models" is checked. You can exclude/include sub models as necessary using the Select Models button.

Simulink-Coverage-Referenced

Related Topic:

S-Function Connector to Simulink