This utility is delivered with the MxSuite; it is not separately licensed.  Its purpose is to automatically examine the results of regression runs.

Suppose you have a suite of 2000 Scenarios, the majority of which pass, but there are many that fail.  Your team is working on the failures.  If you run your regression on a daily basis, you need to know not just that the number of failures is decreasing, but more importantly that particular things that are currently working do not start failing.  MxVDiff is a powerful compare utility that compares the outputs of two regression runs.  The graphical interface allows you to 'drill down' find the scenarios, the test cases, the Signals, and the actual times where the two sets of results differ.

You can use MxVDiff as a Windows program or command line utility. It operates on regression logs (.mxrlg files) and performs two basic functions: Validate and Compare.

Validate

Checks that a log indicates no failures occurred during a regression run. Success means that there were no failures. This mode is only available from a command line or batch file.

Compare

Examines and reports on the differences between two regression run log files.  For Compare, in general Success means that the results are the same for the two regression runs. There is one important exception, which is this:

When comparing two Regression Log Files we refer to one as the Baseline and one as Subsequent.  If the Subsequent log shows more failures than the baseline log, but the additional failures are in Scenarios that were not included in the Baseline regression test, then the comparison is considered a Success.

Using MxVDiff

There are several ways you can use MxVDiff in the MxSuite environment:

hmtoggle_plus1Stand Alone

You can use MxVDiff as a stand alone program.  To use it as such:

1.Select Start->All Programs->MicroMax->MxVDev->MxVDiff

2.Select the first (earlier) Regression results file (.mxrlg), called the Baseline Regression Log.

3.Select the second (later) Regression results file (.mxrlg), called the Subsequent Regression Log.

4.Click the compare button.  Drill down to locate differences identified with red X marker.
hmtoggle_plus1Launch from MxVDev

Individual regression runs are displayed in the Project Explorer under the Regression/Runs folder.  You can launch MxDiff and compare the results of two Regression runs as follows:

1.Right Click on the first (earlier) of the two runs and select Compare - Select Baseline Results

2.Right Click on the second (later) of the two runs and select Compare - Select Subsequent Results

3.MxDiff launches. enabling you to navigate down to the differences if they exist.  Drill down to locate differences identified with red X mark.
hmtoggle_plus1Command Line Interface

There are two ways to use MxVDiff from the command line:

•Validation Mode

MxVDiff.exe <filename>.mxrlg

Returns 0 if the regression log shows no failures, otherwise returns 1

•Compare Mode

MxVDiff.exe <baseline>.mxrlg <subsequent>.mxrlg [--showform]

Returns Success (0) if the regression logs indicate that both regression runs had the same failures, otherwise returns 1.  But see important exception above.

Example

These commands run MxVDiff in validation mode. These commands could be put into a batch file.

cd C:\Users\Documents\MxSuiteSamples\TurnDoorSample\Regression\Regression-Set1-2016-01-22-13-51-59

"C:\Program Files (x86)\MicroMax\MxSuite 3.38.1.35613\Bin\mxvdiff.exe" regression.mxlrg

if %ERRORLEVEL% EQU 0 echo VALIDATION OK

 

 

Related Topic:

Regression Testing

Automation