How can a test script inform R CMD check that it should emit a custom message?
- by mariotomo
I'm writing a R package (delftfews) here at office. we are using svUnit for unit testing.
our process for describing new functionality: we define new unit tests, initially marked as DEACTIVATED; one block of tests at a time we activate them and implement the function described by the tests. almost all the time we have a small amount of DEACTIVATED tests, relative to functions that might be dropped or will be implemented.
my problem/question is: can I alter the doSvUnit.R so that R CMD check pkg emits a NOTE (i.e. a custom message "NOTE" instead of "OK") in case there are DEACTIVATED tests?
as of now, we see only that the active tests don't give error:
.
.
* checking for unstated dependencies in tests ... OK
* checking tests ...
Running ‘doSvUnit.R’
OK
* checking PDF version of manual ... OK
which is all right if all tests succeed, but less all right if there are skipped tests and definitely wrong if there are failing tests. In this case, I'd actually like to see a NOTE or a WARNING like the following:
.
.
* checking for unstated dependencies in tests ... OK
* checking tests ...
Running ‘doSvUnit.R’
NOTE
6 test(s) were skipped.
WARNING
1 test(s) are failing.
* checking PDF version of manual ... OK
As of now, we have to open the doSvUnit.Rout to check the real test results.
I contacted two of the maintainers at r-forge and CRAN and they pointed me to the sources of R, in particular the testing.R script.
if I understand it correctly, to answer this question we need patching the tools package:
scripts in the tests directory are called using a system call,
output (stdout and stderr) go to one single file,
there are two possible outcomes: ok or not ok,
so I opened a change request on R, proposing something like bit-coding the return status, bit-0 for ERROR (as it is now), bit-1 for WARNING, bit-2 for NOTE.
with my modification, it would be easy producing this output:
.
.
* checking for unstated dependencies in tests ... OK
* checking tests ...
Running ‘doSvUnit.R’
NOTE - please check doSvUnit.Rout.
WARNING - please check doSvUnit.Rout.
* checking PDF version of manual ... OK
Brian Ripley replied "There are however several packages with properly written unit tests
that do signal as required. Please do take this discussion elsewhere: R-bugs is not the place to ask
questions." and closed the change request.
anybody has hints?
