JCStress

Java Concurrency Stress

Posted by Vipin Sharma on Saturday, December 26, 2020 Tags: OpenJDK jcstress   8 minute read

Draft (Work in progress)

What is JCStress:

Concurrent code is hard to test, it is possible some code works for years and one bad day it fails. For this JCStress is a stress test framework for concurrent programs developed by JDK developers.


Starting with first JCStress test

Following are important annotations help us to understand our first JCStress test.

@Actor	: It annotates methods that hold the actions done by the threads.

@State  : It annotates the class that holds the data mutated/read by the tests.

@Result	: This annotation marks the result object.

@JCStressTest	: Mark the class as JCStress test.

@Outcome    : It describes the test outcome, and how to deal with it.

First JCStress test:

@JCStressTest

// These are the test outcomes.
@Outcome(id = "1, 1", expect = Expect.ACCEPTABLE_INTERESTING, desc = "Both actors came up with the same value: atomicity failure.")
@Outcome(id = "1, 2", expect = Expect.ACCEPTABLE, desc = "actor1 incremented, then actor2.")
@Outcome(id = "2, 1", expect = Expect.ACCEPTABLE, desc = "actor2 incremented, then actor1.")

// This is a state object
@State
public class APISample_01_Simple {

    int v;

    @Actor
    public void actor1(II_Result r) {
        r.r1 = ++v; // record result from actor1 to field r1
    }

    @Actor
    public void actor2(II_Result r) {
        r.r2 = ++v; // record result from actor2 to field r2
    }

}

Following are few important points about the above test:

  1. Class APISample_01_Simple has field int v, it is annotated with @State, and it stores state of object.

  2. For each test executed:
    1. Methods actor1 and actor2 are executed exactly once.
    2. Methods actor1 and actor2 are called by particular threads, e.g. Thread t1 calls actor1 and Thread t2 calls actor2.
    3. Per @State object (here instance of APISample_01_Simple) JCStress framework calls actor1 and actor2 only once.
    4. The invocation order of methods actor1 and actor2 is deliberately not specified.
    5. It stores state in the same instance of APISample_01_Simple and records result in same instance of II_Result.
    6. Result is available in the instance of II_Result after actor1 and actor2 are completed.
  3. Class II_Result is annotated with @Result, to store result of test it usages 2 int type fields r1 and r2.

  4. Depending upon output of the test we tag it with corresponding @Outcome. e.g. for r1=1 and r2=2, result is Expect.ACCEPTABLE. for r1=1 and r2=1, result is Expect.ACCEPTABLE_INTERESTING.


Setting up project and how to run JCStress test

To use jcstress in your project, you are recommended to create the submodule with the jcstress tests, which would use jcstress libraries and build steps.

Easy and quik way to create jcstress project is to create standalone JCStress project using maven archetype, for this you need maven and JDK 8+.

mvn archetype:generate \
 -DinteractiveMode=false \
 -DarchetypeGroupId=org.openjdk.jcstress \
 -DarchetypeArtifactId=jcstress-java-test-archetype \
 -DgroupId=com.jfeatures \
 -DartifactId=jcstresstest \
 -Dversion=1.0

This is Github project created using above command.

Running tests, if we have not created any test it will execute default test available.

cd jcstresstest
mvn clean verify
java -jar target/jcstress.jar

In case we want to execute a particular test use -t option like below

java -jar target/jcstress.jar -t ConcurrencyTest

Open test report:

firefox results/index.html


More JCStress APIs

JCStress provides useful APIs to develop Java concurrency tests. This sections we are going to discuss these APIs and how they solve different type of problems.

Using Result classes in JCStress

JCStress ships lots of pre-canned result classes. e.g. II_Result and III_Result, here I represent an int. Similarly, following is list of different letters used in JCStress result classes.

I 	: 	int
Z 	: 	boolean
F	:	float
J	:	long
S	: 	short
B	:	byte
C	:	char
D	:	double
L	:	object

With help of this we can try to find pre canned Result class in JCStress. e.g

  1. for Result having int(I) and boolean(Z) we can use IZ_Result.
  2. for Result having float(F) and long(J) we can use FJ_Result.

Arbiters

Method annotated with @Arbiter run after all method annotated with @Actor, and therefore can observe the final result. This is sample test using Arbiters.

Signal

Signal is useful for delivering a termination signal to Actor in Mode.Termination tests. It will run after Actor in question started executing. For termination test we use annotation @JCStressTest(Mode.Termination). This is sample test using Signal.

Nested tests

It is sometimes convenient to put the tests in the same source file for better comparison. JCStress allows to nest the tests. This is sample test showing nested tests in JCStress.

Shared Metadata

When tests share the outcomes and other metadata. To use common metadata for such tests, there is a special @JCStressMeta annotation that can use metadata of another class. This

Adding test descriptions and references

JCStress also allows to put the descriptions and references right at the test. This helps to identify the goal for the test, as well as the discussions about the behavior in question.

Following example showing how we can add test descriptions and references

@Description("Sample Hello World test")
@Ref("http://openjdk.java.net/projects/code-tools/jcstress/")


VM options and arguments used by JCStress

Vm options

  1. -XX:+StressLCM -XX:+StressGCM

    LCM stands for Local Code Motion, GCM stand for Global Code Motion.

    As performance optimization compiler may reorder independent instructions without changing the semantics of the code. Compiler tries to find the most optimal order of instructions.

    When we use StressLCM and StressGCM options, the instruction scheduling works differently. It chooses a random instruction order which is allowed under Java memory model. It helps to simulate different scenarios for our concurrency test.

  2. -XX:-TieredCompilation -XX:TieredStopAtLevel=1 Use -XX:-TieredCompilation or -XX:TieredStopAtLevel=1 to disable C1 or C2 respectively.

  3. -Xint Runs the application in interpreted-only mode. Compilation to native code is disabled, and all bytecode is executed by the interpreter. The performance benefits offered by the just in time (JIT) compiler are not present in this mode.

Command line options

JCStress has got a couple of helpful command line options. we can pass argument -h to get full list of command line options.

  java -jar target/jcstress.jar -h

Some important options are:

  Modes (-m)
  Test name(-t)
  verbose result (-v)
  Number of CPUs used (-c)

Let’s look at one important command line option -m.

It sets test mode, from sanity to stress it becomes more and more rigorous, and we know for concurrency issues more rigorous is more helpful. Following are sample commands to run different modes.

java -jar target/jcstress.jar -t ConcurrencyTest -m sanity
java -jar target/jcstress.jar -t ConcurrencyTest -m quick
java -jar target/jcstress.jar -t ConcurrencyTest -m default
java -jar target/jcstress.jar -t ConcurrencyTest -m tough
java -jar target/jcstress.jar -t ConcurrencyTest -m stress

Conclusion

Knowing features like this helps you get the best java jobs, that’s why to help you I wrote ebook 5 steps to Best Java Jobs. Download this step by step guide for free!

Resources:

  1. JCStress GitHub