2.8. Expectations on the Command Line

When you run QMTest on the command line, it automatically creates a results file called results.qmr. You can specify a different filename with the -o option. Run this command:


> qmtest run -o expected.qmr
   
to save the results to a file named expected.qmr instead of the default results.qmr.

Now, when you rerun the tests you can tell QMTest to use expected.qmr as the expected results file, like this:


> qmtest run -O expected.qmr
   
QMTest will rerun the tests, but this time it will not mention the failure of exec1. The output will look like:


--- TEST RESULTS -------------------------------------------------------------

  exec0                                         : PASS    

  exec1                                         : XFAIL   
    Expression evaluates to false.

    ExecTest.expr:
      x + x == 5

    ExecTest.value:
      0

    qmtest.target:
      local

  exec2                                         : PASS    

--- TESTS WITH UNEXPECTED OUTCOMES -------------------------------------------

  None.


--- STATISTICS ---------------------------------------------------------------

       3        tests total
       3 (100%) tests as expected

Note that QMTest indicates that there were no tests with unexpected outcomes, even though exec1 still fails. The XFAIL notation indicates that the test failed, but that failure was expected. In contrast, XPASS means that a test passed unexpectedly.