Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable time-based load test mechanism in openjdk-systemtest load tests #396

Merged
merged 1 commit into from
Jan 29, 2021
Merged

Enable time-based load test mechanism in openjdk-systemtest load tests #396

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions openjdk.build/makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -591,7 +591,7 @@ test.LockingLoadTest:
echo Target $@ completed
test.MathLoadTest_all:
echo Running target $@
$(STF_COMMAND) -test=MathLoadTest $(LOG)
$(STF_COMMAND) -test=MathLoadTest -test-args="workload=math" $(LOG)
echo Target $@ completed
test.MathLoadTest_autosimd:
echo Running target $@
Expand All @@ -603,15 +603,15 @@ test.MathLoadTest_bigdecimal:
echo Target $@ completed
test.MauveSingleThreadLoadTest:
echo Running target $@
$(STF_COMMAND) -test=MauveSingleThreadLoadTest $(LOG)
$(STF_COMMAND) -test=MauveSingleThrdLoad $(LOG)
echo Target $@ completed
test.MauveSingleInvocationLoadTest:
echo Running target $@
$(STF_COMMAND) -test=MauveSingleInvocationLoadTest $(LOG)
$(STF_COMMAND) -test=MauveSingleInvocLoad $(LOG)
echo Target $@ completed
test.MauveMultiThreadLoadTest:
echo Running target $@
$(STF_COMMAND) -test=MauveMultiThreadLoadTest $(LOG)
$(STF_COMMAND) -test=MauveMultiThrdLoad $(LOG)
echo Target $@ completed
test.NioLoadTest:
echo Running target $@
Expand Down
37 changes: 18 additions & 19 deletions openjdk.test.load/src/test.load/net/adoptopenjdk/stf/ClassloadingLoadTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,10 @@
import java.util.ArrayList;

import net.adoptopenjdk.loadTest.InventoryData;
import net.adoptopenjdk.stf.codeGeneration.Stage;
import net.adoptopenjdk.loadTest.TimeBasedLoadTest;
import net.adoptopenjdk.stf.environment.DirectoryRef;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension.Echo;
import net.adoptopenjdk.stf.plugin.interfaces.StfPluginInterface;
import net.adoptopenjdk.stf.processes.ExpectedOutcome;
import net.adoptopenjdk.stf.processes.definitions.JavaProcessDefinition;
import net.adoptopenjdk.stf.processes.definitions.LoadTestProcessDefinition;
Expand All @@ -31,7 +30,7 @@
* This is a simple test plugin that runs a workload of tests
* from the test.classloading project.
*/
public class ClassloadingLoadTest implements StfPluginInterface {
public class ClassloadingLoadTest extends TimeBasedLoadTest {

private int testCountMultiplier = 3000;
private boolean specialTest = false;
Expand All @@ -42,12 +41,6 @@ public void help(HelpTextGenerator help) throws StfException {
+ "from the test.classloading project.");
}

public void pluginInit(StfCoreExtension stf) throws StfException {
}

public void setUp(StfCoreExtension test) throws StfException {
}

public void execute(StfCoreExtension test) throws StfException {
String jvmOptionsInUse = test.getJavaArgs(test.env().primaryJvm());

Expand Down Expand Up @@ -88,19 +81,25 @@ public void execute(StfCoreExtension test) throws StfException {
.addModules(modulesAdd)
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.JUNIT)
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.HAMCREST)
.addProjectToClasspath("openjdk.test.classloading")
.setAbortIfOutOfMemory(false)
.addProjectToClasspath("openjdk.test.classloading");

if (isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
}

loadTestInvocation = loadTestInvocation.setAbortIfOutOfMemory(false)
.addSuite("classloading")
.setSuiteThreadCount(cpuCount - 1, 10)
.setSuiteInventory(inventoryFile)
.setSuiteNumTests(totalTests * testCountMultiplier)
.setSuiteRandomSelection();
.setSuiteInventory(inventoryFile);

if (!isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setSuiteNumTests(totalTests * testCountMultiplier);
}
Comment on lines +86 to +97
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to have a simpler logic?

if (isTimeBasedLoadTest) { 
...
} else {
...
}

Same for the rest of PR

Copy link
Contributor Author

@Mesbah-Alam Mesbah-Alam Jan 26, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done the way it is because STF framework enforces a particular sequence in which we can add methods in a process definition. For example, in the code snippet above, if instead we did the following:

if (isTimeBasedLoadTest) { 
			loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
		} else { 
			loadTestInvocation = loadTestInvocation.setSuiteNumTests(totalTests * testCountMultiplier);
		}
		
		loadTestInvocation = loadTestInvocation.setAbortIfOutOfMemory(false)
				.addSuite("classloading")
				.setSuiteThreadCount(cpuCount - 1, 10)
				.setSuiteInventory(inventoryFile);

We will have an error like this:

GEN 16:00:20.802 - Using Mode NoOptions. Values = ''
GEN stderr Exception in thread "main" net.adoptopenjdk.stf.StfException: Java invocation built out of sequence. LOAD_TEST_ARGS cannot be set after SUITE
GEN stderr 	at net.adoptopenjdk.stf.processes.definitions.LoadTestProcessDefinition.checkAndUpdateLevel(LoadTestProcessDefinition.java:784)
GEN stderr 	at net.adoptopenjdk.stf.processes.definitions.LoadTestProcessDefinition.setAbortIfOutOfMemory(LoadTestProcessDefinition.java:297)
GEN stderr 	at net.adoptopenjdk.stf.ClassloadingLoadTest.execute(ClassloadingLoadTest.java:92)
GEN stderr 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
GEN stderr 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
GEN stderr 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
GEN stderr 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
GEN stderr 	at net.adoptopenjdk.stf.runner.StfRunner.runStage(StfRunner.java:549)
GEN stderr 	at net.adoptopenjdk.stf.runner.StfRunner.executeTest(StfRunner.java:297)
GEN stderr 	at net.adoptopenjdk.stf.runner.StfRunner.main(StfRunner.java:69)
Generation failed

Please see here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess, we can have a separate issue to investigate if we need to revisit this feature in STF.

Copy link
Contributor

@llxia llxia Jan 26, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should. Please open an issue in STF. Thanks
Also, could you try one last thing? Could you try to move if-else code after setAbortIfOutOfMemory but before addSuite? According to STF code, setAbortIfOutOfMemory is set to LOAD_TEST_ARGS , setSuiteNumTests is set to SUITE , setTimeLimit is set to LOAD_TEST_ARGS and addSuite is set to SUITE .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move if-else code after setAbortIfOutOfMemory but before addSuite?

This had the same error as before. But it worked if I moved the if-else after addSuite and before setSuiteThreadCount :

LoadTestProcessDefinition loadTestInvocation = test.createLoadTestSpecification()
				.addJvmOption("-Djava.classloading.dir=" + notonclasspathDirRoot)  // Expose the bin_notonclasspath root directory to the deadlock test
				.addJvmOption("-Djava.version.number=" + javaVersion)
				.addModules(modulesAdd)
				.addPrereqJarToClasspath(JavaProcessDefinition.JarId.JUNIT)
				.addPrereqJarToClasspath(JavaProcessDefinition.JarId.HAMCREST)
				.addProjectToClasspath("openjdk.test.classloading")	
				.setAbortIfOutOfMemory(false)
				.addSuite("classloading");
		
		if (isTimeBasedLoadTest) { 
			loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
		} else { 
			loadTestInvocation = loadTestInvocation.setSuiteNumTests(totalTests * testCountMultiplier);
		}
		
		loadTestInvocation = loadTestInvocation.setSuiteThreadCount(cpuCount - 1, 10)
				.setSuiteInventory(inventoryFile)
				.setSuiteRandomSelection();

So, this simplifies the logic too. I will make similar update in the rest of the test classes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Mesbah-Alam

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New STF issue raised: adoptium/STF#101

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to recap conversation with Lan about the above issue over slack:

I tried the change proposed above locally, and the if-else doesn't really work for both the scenarios when a timeLimit is provided (time-based test) and when it's no (iteration based test). Meaning, one particular sequence works when isTimeBasedLoadTest==true, and a different sequence is expected by STF when isTimeBasedLoadTest=false. Hence we have decided to stick to what the original change was.

Further discussion to commence in adoptium/STF#101


loadTestInvocation = loadTestInvocation.setSuiteRandomSelection();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we set setSuiteRandomSelection , setAbortIfOutOfMemory , and others together in line 78? Same for the rest of PR


test.doRunForegroundProcess("Run classloading tests", "CLT", Echo.ECHO_ON,
ExpectedOutcome.cleanRun().within("2h"),
ExpectedOutcome.cleanRun().within(finalTimeout),
loadTestInvocation);
}

public void tearDown(StfCoreExtension stf) throws StfException {
}
}
}
39 changes: 19 additions & 20 deletions openjdk.test.load/src/test.load/net/adoptopenjdk/stf/ConcurrentLoadTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,27 +14,21 @@

package net.adoptopenjdk.stf;

import net.adoptopenjdk.loadTest.InventoryData;
import net.adoptopenjdk.loadTest.TimeBasedLoadTest;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension.Echo;
import net.adoptopenjdk.stf.plugin.interfaces.StfPluginInterface;
import net.adoptopenjdk.stf.processes.ExpectedOutcome;
import net.adoptopenjdk.stf.processes.definitions.JavaProcessDefinition;
import net.adoptopenjdk.stf.processes.definitions.LoadTestProcessDefinition;
import net.adoptopenjdk.stf.runner.modes.HelpTextGenerator;
import net.adoptopenjdk.loadTest.InventoryData;

public class ConcurrentLoadTest implements StfPluginInterface {
public class ConcurrentLoadTest extends TimeBasedLoadTest {
public void help(HelpTextGenerator help) throws StfException {
help.outputSection("ConcurrentLoadTest runs concurrency unit tests");
help.outputText("");
}

public void pluginInit(StfCoreExtension stf) throws StfException {
}

public void setUp(StfCoreExtension test) throws StfException {
}

public void execute(StfCoreExtension test) throws StfException {
String inventoryFile = "/openjdk.test.load/config/inventories/concurrent/concurrent.xml";
int totalTests = InventoryData.getNumberOfTests(test, inventoryFile);
Expand All @@ -43,20 +37,25 @@ public void execute(StfCoreExtension test) throws StfException {
LoadTestProcessDefinition loadTestInvocation = test.createLoadTestSpecification()
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.JUNIT)
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.HAMCREST)
.addProjectToClasspath("openjdk.test.concurrent")
.setAbortIfOutOfMemory(false)
.addProjectToClasspath("openjdk.test.concurrent");

if (isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
}

loadTestInvocation = loadTestInvocation.setAbortIfOutOfMemory(false)
.addSuite("concurrent")
.setSuiteThreadCount(cpuCount - 2, 20) // Leave 1 cpu for the JIT. i for GC and set min 20
.setSuiteInventory(inventoryFile) // Point at the file which lists the tests
.setSuiteNumTests(totalTests * 20) // Run for about 2 minutes with no -X options
.setSuiteRandomSelection(); // Randomly pick the next test each time
.setSuiteInventory(inventoryFile); // Point at the file which lists the tests

if (!isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setSuiteNumTests(totalTests * 20); // Run for about 2 minutes with no -X options
}

loadTestInvocation = loadTestInvocation.setSuiteRandomSelection(); // Randomly pick the next test each time

//.setSuiteNumTests(totalTests * 50) // Run for about 5 minutes
test.doRunForegroundProcess("Run concurrency unit tests", "LT", Echo.ECHO_ON,
ExpectedOutcome.cleanRun().within("1h"),
ExpectedOutcome.cleanRun().within(finalTimeout),
loadTestInvocation);
}

public void tearDown(StfCoreExtension stf) throws StfException {
}
}
}
35 changes: 18 additions & 17 deletions openjdk.test.load/src/test.load/net/adoptopenjdk/stf/DirectByteBufferLoadTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
package net.adoptopenjdk.stf;

import net.adoptopenjdk.loadTest.InventoryData;
import net.adoptopenjdk.loadTest.TimeBasedLoadTest;
import net.adoptopenjdk.stf.environment.FileRef;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension.Echo;
import net.adoptopenjdk.stf.plugin.interfaces.StfPluginInterface;
import net.adoptopenjdk.stf.processes.ExpectedOutcome;
import net.adoptopenjdk.stf.processes.definitions.JavaProcessDefinition;
import net.adoptopenjdk.stf.processes.definitions.LoadTestProcessDefinition;
Expand All @@ -29,7 +29,7 @@
*
* The DirectMemoryTest allocates direct memory ByteBuffers and dereferences them.
*/
public class DirectByteBufferLoadTest implements StfPluginInterface {
public class DirectByteBufferLoadTest extends TimeBasedLoadTest {
public void help(HelpTextGenerator help) throws StfException {
String testName = DirectByteBufferLoadTest.class.getSimpleName();

Expand All @@ -38,12 +38,6 @@ public void help(HelpTextGenerator help) throws StfException {
help.outputText("This test is primarily aimed at stressing the Garbage Collector.");
}

public void pluginInit(StfCoreExtension test) throws StfException {
}

public void setUp(StfCoreExtension test) throws Exception {
}

public void execute(StfCoreExtension test) throws StfException {
// The NIO workload has a dependency on filesystem.jar
FileRef filesystemJar = test.env().findTestFile("openjdk.test.nio/bin/lib/filesystem.jar");
Expand All @@ -58,19 +52,26 @@ public void execute(StfCoreExtension test) throws StfException {
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.HAMCREST)
.addJarToClasspath(filesystemJar)
.addProjectToClasspath("openjdk.test.gc")
.addProjectToClasspath("openjdk.test.nio")
.addSuite("DirectByteBuffer")
.addProjectToClasspath("openjdk.test.nio");

if (isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
}

loadTestInvocation = loadTestInvocation.addSuite("DirectByteBuffer")
.setSuiteThinkingTime("1ms", "1ms")// Waiting for 1ms between tests to give the system a chance to reclaim unused native memory.
.setSuiteThreadCount(cpuCount - 1, 2)
.setSuiteNumTests(numTests)
.setSuiteInventory(inventory)
.setSuiteThreadCount(cpuCount - 1, 2);

if (!isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setSuiteNumTests(numTests);
}

loadTestInvocation = loadTestInvocation.setSuiteInventory(inventory)
.setSuiteRandomSelection();

test.doRunForegroundProcess("Run DirectByteBuffer load test", "DBLT", Echo.ECHO_ON,
ExpectedOutcome.cleanRun().within("5m"),
ExpectedOutcome.cleanRun().within(finalTimeout),
loadTestInvocation);
}

public void tearDown(StfCoreExtension test) throws StfException {
}
}

37 changes: 18 additions & 19 deletions openjdk.test.load/src/test.load/net/adoptopenjdk/stf/LambdaLoadTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,47 +14,46 @@

package net.adoptopenjdk.stf;

import net.adoptopenjdk.loadTest.TimeBasedLoadTest;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension.Echo;
import net.adoptopenjdk.stf.plugin.interfaces.StfPluginInterface;
import net.adoptopenjdk.stf.processes.ExpectedOutcome;
import net.adoptopenjdk.stf.processes.definitions.JavaProcessDefinition;
import net.adoptopenjdk.stf.processes.definitions.LoadTestProcessDefinition;
import net.adoptopenjdk.stf.runner.modes.HelpTextGenerator;

public class LambdaLoadTest implements StfPluginInterface {
public class LambdaLoadTest extends TimeBasedLoadTest {
public void help(HelpTextGenerator help) throws StfException {
help.outputSection("LambdaLoadTest runs unit tests for Lambda and Streams");
help.outputText("");
}

public void pluginInit(StfCoreExtension test) throws StfException {
}

public void setUp(StfCoreExtension test) throws StfException {
}

public void execute(StfCoreExtension test) throws StfException {
String inventoryFile = "/openjdk.test.load/config/inventories/lambdasAndStreams/lambda.xml";
int cpuCount = Runtime.getRuntime().availableProcessors();

LoadTestProcessDefinition loadTestInvocation = test.createLoadTestSpecification()
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.JUNIT)
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.HAMCREST)
.addProjectToClasspath("openjdk.test.lambdasAndStreams")
.setInactivityLimit("60m") // Since this test is run using -Xint as well, set a larger inactivity limit than default
.addProjectToClasspath("openjdk.test.lambdasAndStreams");

if (isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
}

loadTestInvocation = loadTestInvocation.setInactivityLimit("60m") // Since this test is run using -Xint as well, set a larger inactivity limit than default
.addSuite("lambda") // Start args for the first suite
.setSuiteThreadCount(cpuCount - 2, 2) // Leave 1 cpu for the JIT, 1 cpu for GC and set min 2
.setSuiteInventory(inventoryFile) // Point at the file which lists the tests
.setSuiteNumTests(200) // Run this many tests
.setSuiteRandomSelection(); // Randomly pick the next test each time

.setSuiteInventory(inventoryFile); // Point at the file which lists the tests

if (!isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setSuiteNumTests(200); // Run this many tests
}

loadTestInvocation = loadTestInvocation.setSuiteRandomSelection(); // Randomly pick the next test each time

test.doRunForegroundProcess("Run lambda and stream load test", "LT", Echo.ECHO_ON,
ExpectedOutcome.cleanRun().within("60m"),
ExpectedOutcome.cleanRun().within(finalTimeout),
loadTestInvocation);
}

public void tearDown(StfCoreExtension test) throws StfException {
}
}
}
36 changes: 18 additions & 18 deletions openjdk.test.load/src/test.load/net/adoptopenjdk/stf/LangLoadTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@
package net.adoptopenjdk.stf;

import net.adoptopenjdk.loadTest.InventoryData;
import net.adoptopenjdk.loadTest.TimeBasedLoadTest;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension;
import net.adoptopenjdk.stf.extensions.core.StfCoreExtension.Echo;
import net.adoptopenjdk.stf.plugin.interfaces.StfPluginInterface;
import net.adoptopenjdk.stf.processes.ExpectedOutcome;
import net.adoptopenjdk.stf.processes.definitions.JavaProcessDefinition;
import net.adoptopenjdk.stf.processes.definitions.LoadTestProcessDefinition;
Expand All @@ -27,18 +27,12 @@
* This is a test plugin for Java.lang related tests, it runs a workload of
* invoke tests and project coin tests.
*/
public class LangLoadTest implements StfPluginInterface {
public class LangLoadTest extends TimeBasedLoadTest {
public void help(HelpTextGenerator help) throws StfException {
help.outputSection("LangLoadTest runs a workload of Java.lang related tests.");
help.outputText("");
}

public void pluginInit(StfCoreExtension stf) throws StfException {
}

public void setUp(StfCoreExtension test) throws StfException {
}

public void execute(StfCoreExtension test) throws StfException {
String inventoryFile = "/openjdk.test.load/config/inventories/lang/lang.xml";
int numlangTests = InventoryData.getNumberOfTests(test, inventoryFile);
Expand All @@ -47,18 +41,24 @@ public void execute(StfCoreExtension test) throws StfException {
LoadTestProcessDefinition loadTestInvocation = test.createLoadTestSpecification()
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.JUNIT)
.addPrereqJarToClasspath(JavaProcessDefinition.JarId.HAMCREST)
.addProjectToClasspath("openjdk.test.lang")
.addSuite("lang")
.setSuiteThreadCount(cpuCount - 1, 2)
.setSuiteNumTests(numlangTests * 100)
.setSuiteInventory(inventoryFile)
.addProjectToClasspath("openjdk.test.lang");

if (isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setTimeLimit(timeLimit); // If it's a time based test, stop execution after given time duration
}

loadTestInvocation = loadTestInvocation.addSuite("lang")
.setSuiteThreadCount(cpuCount - 1, 2);

if (!isTimeBasedLoadTest) {
loadTestInvocation = loadTestInvocation.setSuiteNumTests(numlangTests * 100);
}

loadTestInvocation = loadTestInvocation.setSuiteInventory(inventoryFile)
.setSuiteRandomSelection();

test.doRunForegroundProcess("Run lang load test", "LLT", Echo.ECHO_ON,
ExpectedOutcome.cleanRun().within("10m"),
ExpectedOutcome.cleanRun().within(finalTimeout),
loadTestInvocation);
}

public void tearDown(StfCoreExtension stf) throws StfException {
}
}
}
Loading