Hi, I am running hundreds of tests against a large publishing system and would like to paralellize the tests using TestNG. However, I cannot find any easy way of doing this. Each test case instanciates an instance of this publisher, send some messages, wait for those messages to be published, then dump out the contents of the publish queues and compare against expected outcome. Doing this with so many tests (even if I paralellize using threads, still takes a very long time to complete (1 day or more)).

We've found that in testing this sort of system, it's best to start up system once, run all tests to send their messages, wait for publish to do its thing, dump all outputs, and match outputs with tests and verify. For example, instead of the following:

@Test
public void testRule1() {
  Publisher pub = new Publisher();
  pub.sendRule(new Rule("test1-a"));
  sleep(10); // wait 10 seconds
  pub.dumpRules();
  verifyRule("test1-a");
}

We wanted to do something like the following:

@Test
public void testRule1(bool sendMode) {
  if(sendMode) {
    this.pub.sendRule(new Rule("test1-a"));
  }
  else {
    verifyRule("test1-a");
  }
}

Where you have a dataProvider run through all the tests with sendMode = true and then perform dumpAllRules() followed by running through all of the tests again with sendMode = false. The problem is, TestNG calls the same method twice, once with sendMode = true followed by sendMode = false. Is there anyway to accomplish this in TestNG?

Thanks!