Menguji Kod Multi-Threaded di Java

1. Pengenalan

Dalam tutorial ini, kita akan membahas beberapa asas menguji program serentak. Kami akan memfokuskan pada kesesuaian berasaskan utas dan masalah yang dihadapinya dalam ujian.

Kami juga akan memahami bagaimana kami dapat menyelesaikan beberapa masalah ini dan menguji kod multi-utas dengan berkesan di Java.

2. Pengaturcaraan Serentak

Pengaturcaraan serentak merujuk kepada pengaturcaraan di mana kita membahagikan sebahagian besar pengiraan menjadi pengiraan yang lebih kecil dan relatif bebas .

Tujuan latihan ini adalah untuk menjalankan pengiraan yang lebih kecil ini secara serentak, bahkan mungkin secara selari. Walaupun terdapat beberapa cara untuk mencapai ini, objektifnya adalah untuk menjalankan program dengan lebih pantas.

2.1. Threads dan Pengaturcaraan Serentak

Dengan pemproses membungkus lebih banyak teras daripada sebelumnya, pengaturcaraan serentak berada di barisan hadapan untuk memanfaatkannya dengan cekap. Namun, hakikatnya program serentak jauh lebih sukar untuk mereka bentuk, menulis, menguji dan menyelenggara . Oleh itu, jika kita dapat menulis kes ujian berkesan dan automatik untuk program serentak, kita dapat menyelesaikan sebahagian besar masalah ini.

Jadi, apa yang menjadikan ujian menulis untuk kod serentak begitu sukar? Untuk memahami perkara itu, kita mesti memahami bagaimana kita mencapai kesesuaian dalam program kita. Salah satu teknik pengaturcaraan serentak yang paling popular melibatkan penggunaan utas.

Sekarang, utas boleh jadi asli, dalam hal ini dijadualkan oleh sistem operasi yang mendasari. Kami juga dapat menggunakan apa yang dikenali sebagai benang hijau, yang dijadualkan secara langsung waktu berjalan.

2.2. Kesukaran Menguji Program Serentak

Tanpa mengira jenis benang yang kami gunakan, apa yang menjadikannya sukar untuk digunakan adalah komunikasi utas. Sekiranya kita berjaya menulis program yang melibatkan utas tetapi tidak ada komunikasi utas, tidak ada yang lebih baik! Lebih realistik, utas biasanya perlu berkomunikasi. Terdapat dua cara untuk mencapai ini - memori bersama dan penghantaran mesej.

Sebilangan besar masalah yang berkaitan dengan pengaturcaraan serentak berpunca daripada penggunaan benang asli dengan memori bersama . Menguji program sedemikian sukar untuk alasan yang sama. Beberapa utas dengan akses ke memori bersama umumnya memerlukan pengecualian bersama. Kami biasanya mencapainya melalui beberapa mekanisme pelindung menggunakan kunci.

Tetapi ini masih boleh menyebabkan pelbagai masalah seperti keadaan perlumbaan, kunci hidup, kebuntuan, dan kelaparan benang, untuk beberapa nama. Lebih-lebih lagi, masalah ini berselang-seling, kerana penjadualan benang dalam kes benang asli sama sekali tidak bersifat deterministik.

Oleh itu, menulis ujian yang berkesan untuk program serentak yang dapat mengesan masalah-masalah ini secara deterministik memang menjadi cabaran!

2.3. Anatomi Benang Benang

Kami tahu bahawa utas asli dapat dijadualkan oleh sistem operasi dengan tidak dapat diramalkan. Sekiranya benang ini mengakses dan mengubah data yang dikongsi, ia akan menghasilkan interleaving benang yang menarik . Walaupun beberapa interleavings ini boleh diterima sepenuhnya, yang lain mungkin meninggalkan data akhir dalam keadaan yang tidak diingini.

Mari kita ambil contoh. Katakan kita mempunyai kaunter global yang ditingkatkan oleh setiap utas. Pada akhir pemprosesan, kami ingin keadaan kaunter ini sama dengan bilangan utas yang telah dilaksanakan:

private int counter; public void increment() { counter++; }

Sekarang, untuk menambahkan bilangan bulat primitif di Java bukanlah operasi atom . Ini terdiri daripada membaca nilai, meningkatkannya, dan akhirnya menyimpannya. Walaupun beberapa utas melakukan operasi yang sama, ia mungkin menimbulkan banyak kemungkinan interleavings:

Walaupun interleaving tertentu ini menghasilkan hasil yang dapat diterima sepenuhnya, bagaimana dengan yang satu ini:

Ini bukan yang kita harapkan. Sekarang, bayangkan beratus-ratus utas menjalankan kod yang jauh lebih kompleks daripada ini. Ini akan menimbulkan cara-cara yang tidak dapat dibayangkan bahawa benang-benang itu menjalin.

Terdapat beberapa cara untuk menulis kod yang dapat mengelakkan masalah ini, tetapi itu bukan tajuk tutorial ini. Penyegerakan menggunakan kunci adalah salah satu yang biasa, tetapi ia mempunyai masalah yang berkaitan dengan keadaan perlumbaan.

3. Menguji Kod Berbilang Benang

Setelah memahami cabaran asas dalam menguji kod berbilang utas, kita akan melihat cara mengatasinya. Kami akan membina kes penggunaan yang mudah dan cuba mensimulasikan seberapa banyak masalah yang berkaitan dengan serentak mungkin.

Mari kita mulakan dengan menentukan kelas sederhana yang dapat mengira apa sahaja:

public class MyCounter { private int count; public void increment() { int temp = count; count = temp + 1; } // Getter for count }

Ini adalah sekeping kod yang nampaknya tidak berbahaya, tetapi tidak sukar untuk difahami bahawa ia tidak selamat dari benang . Sekiranya kita menulis program serentak dengan kelas ini, pasti cacat. Tujuan pengujian di sini adalah untuk mengenal pasti kecacatan tersebut.

3.1. Menguji Bahagian Tidak Serentak

Sebagai peraturan, selalu disarankan untuk menguji kod dengan mengasingkannya dari sebarang tingkah laku yang serentak . Ini adalah untuk memastikan bahawa tidak ada kecacatan lain dalam kod yang tidak berkaitan dengan kesesuaian. Mari lihat bagaimana kita boleh melakukannya:

@Test public void testCounter() { MyCounter counter = new MyCounter(); for (int i = 0; i < 500; i++) { counter.increment(); } assertEquals(500, counter.getCount()); }

Walaupun tidak banyak yang berlaku di sini, ujian ini memberi kita keyakinan bahawa ia berfungsi sekurang-kurangnya tanpa adanya persetujuan.

3.2. Percubaan Pertama Menguji Bersama

Mari kita teruskan untuk menguji kod yang sama sekali lagi, kali ini dalam penyediaan serentak. Kami akan cuba mengakses contoh kelas yang sama dengan pelbagai utas dan melihat bagaimana tingkah lakunya:

@Test public void testCounterWithConcurrency() throws InterruptedException { int numberOfThreads = 10; ExecutorService service = Executors.newFixedThreadPool(10); CountDownLatch latch = new CountDownLatch(numberOfThreads); MyCounter counter = new MyCounter(); for (int i = 0; i  { counter.increment(); latch.countDown(); }); } latch.await(); assertEquals(numberOfThreads, counter.getCount()); }

This test is reasonable, as we're trying to operate on shared data with several threads. As we keep the number of threads low, like 10, we will notice that it passes almost all the time. Interestingly, if we start increasing the number of threads, say to 100, we will see that the test starts to fail most of the time.

3.3. A Better Attempt at Testing With Concurrency

While the previous test did reveal that our code isn't thread-safe, there's a problem with this teat. This test isn't deterministic because the underlying threads interleave in a non-deterministic manner. We really can't rely on this test for our program.

What we need is a way to control the interleaving of threads so that we can reveal concurrency issues in a deterministic manner with much fewer threads. We'll begin by tweaking the code we are testing a little bit:

public synchronized void increment() throws InterruptedException { int temp = count; wait(100); count = temp + 1; }

Here, we've made the method synchronized and introduced a wait between the two steps within the method. The synchronized keyword ensures that only one thread can modify the count variable at a time, and the wait introduces a delay between each thread execution.

Please note that we don't necessarily have to modify the code we intend to test. However, since there aren't many ways we can affect thread scheduling, we're resorting to this.

In a later section, we'll see how we can do this without altering the code.

Now, let's similarly test this code as we did earlier:

@Test public void testSummationWithConcurrency() throws InterruptedException { int numberOfThreads = 2; ExecutorService service = Executors.newFixedThreadPool(10); CountDownLatch latch = new CountDownLatch(numberOfThreads); MyCounter counter = new MyCounter(); for (int i = 0; i  { try { counter.increment(); } catch (InterruptedException e) { // Handle exception } latch.countDown(); }); } latch.await(); assertEquals(numberOfThreads, counter.getCount()); }

Here, we're running this just with just two threads, and the chances are that we'll be able to get the defect we've been missing. What we've done here is to try achieving a specific thread interleaving, which we know can affect us. While good for the demonstration, we may not find this useful for practical purposes.

4. Testing Tools Available

As the number of threads grows, the possible number of ways they may interleave grows exponentially. It's just not possible to figure out all such interleavings and test for them. We have to rely on tools to undertake the same or similar effort for us. Fortunately, there are a couple of them available to make our lives easier.

There are two broad categories of tools available to us for testing concurrent code. The first enables us to produce reasonably high stress on the concurrent code with many threads. Stress increases the likelihood of rare interleaving and, thus, increases our chances of finding defects.

The second enables us to simulate specific thread interleaving, thereby helping us find defects with more certainty.

4.1. tempus-fugit

The tempus-fugit Java library helps us to write and test concurrent code with ease. We'll just focus on the test part of this library here. We saw earlier that producing stress on code with multiple threads increases the chances of finding defects related to concurrency.

While we can write utilities to produce the stress ourselves, tempus-fugit provides convenient ways to achieve the same.

Let's revisit the same code we tried to produce stress for earlier and understand how can we achieve the same using tempus-fugit:

public class MyCounterTests { @Rule public ConcurrentRule concurrently = new ConcurrentRule(); @Rule public RepeatingRule rule = new RepeatingRule(); private static MyCounter counter = new MyCounter(); @Test @Concurrent(count = 10) @Repeating(repetition = 10) public void runsMultipleTimes() { counter.increment(); } @AfterClass public static void annotatedTestRunsMultipleTimes() throws InterruptedException { assertEquals(counter.getCount(), 100); } }

Here, we are using two of the Rules available to us from tempus-fugit. These rules intercept the tests and help us apply the desired behaviors, like repetition and concurrency. So, effectively, we are repeating the operation under test ten times each from ten different threads.

As we increase the repetition and concurrency, our chances of detecting defects related to concurrency will increase.

4.2. Thread Weaver

Thread Weaver is essentially a Java framework for testing multi-threaded code. We've seen previously that thread interleaving is quite unpredictable, and hence, we may never find certain defects through regular tests. What we effectively need is a way to control the interleaves and test all possible interleaving. This has proven to be quite a complex task in our previous attempt.

Let's see how Thread Weaver can help us here. Thread Weaver allows us to interleave the execution of two separate threads in a large number of ways, without having to worry about how. It also gives us the possibility of having fine-grained control over how we want the threads to interleave.

Let's see how can we improve upon our previous, naive attempt:

public class MyCounterTests { private MyCounter counter; @ThreadedBefore public void before() { counter = new MyCounter(); } @ThreadedMain public void mainThread() { counter.increment(); } @ThreadedSecondary public void secondThread() { counter.increment(); } @ThreadedAfter public void after() { assertEquals(2, counter.getCount()); } @Test public void testCounter() { new AnnotatedTestRunner().runTests(this.getClass(), MyCounter.class); } }

Here, we've defined two threads that try to increment our counter. Thread Weaver will try to run this test with these threads in all possible interleaving scenarios. Possibly in one of the interleaves, we will get the defect, which is quite obvious in our code.

4.3. MultithreadedTC

MultithreadedTC is yet another framework for testing concurrent applications. It features a metronome that is used to provide fine control over the sequence of activities in multiple threads. It supports test cases that exercise a specific interleaving of threads. Hence, we should ideally be able to test every significant interleaving in a separate thread deterministically.

Now, a complete introduction to this feature-rich library is beyond the scope of this tutorial. But, we can certainly see how to quickly set up tests that provide us the possible interleavings between executing threads.

Let's see how can we test our code more deterministically with MultithreadedTC:

public class MyTests extends MultithreadedTestCase { private MyCounter counter; @Override public void initialize() { counter = new MyCounter(); } public void thread1() throws InterruptedException { counter.increment(); } public void thread2() throws InterruptedException { counter.increment(); } @Override public void finish() { assertEquals(2, counter.getCount()); } @Test public void testCounter() throws Throwable { TestFramework.runManyTimes(new MyTests(), 1000); } }

Here, we are setting up two threads to operate on the shared counter and increment it. We've configured MultithreadedTC to execute this test with these threads for up to a thousand different interleavings until it detects one which fails.

4.4. Java jcstress

OpenJDK maintains Code Tool Project to provide developer tools for working on the OpenJDK projects. There are several useful tools under this project, including the Java Concurrency Stress Tests (jcstress). This is being developed as an experimental harness and suite of tests to investigate the correctness of concurrency support in Java.

Although this is an experimental tool, we can still leverage this to analyze concurrent code and write tests to fund defects related to it. Let's see how we can test the code that we've been using so far in this tutorial. The concept is pretty similar from a usage perspective:

@JCStressTest @Outcome(id = "1", expect = ACCEPTABLE_INTERESTING, desc = "One update lost.") @Outcome(id = "2", expect = ACCEPTABLE, desc = "Both updates.") @State public class MyCounterTests { private MyCounter counter; @Actor public void actor1() { counter.increment(); } @Actor public void actor2() { counter.increment(); } @Arbiter public void arbiter(I_Result r) { r.r1 = counter.getCount(); } }

Here, we've marked the class with an annotation State, which indicates that it holds data that is mutated by multiple threads. Also, we're using an annotation Actor, which marks the methods that hold the actions done by different threads.

Finally, we have a method marked with an annotation Arbiter, which essentially only visits the state once all Actors have visited it. We have also used annotation Outcome to define our expectations.

Overall, the setup is quite simple and intuitive to follow. We can run this using a test harness, given by the framework, that finds all classes annotated with JCStressTest and executes them in several iterations to obtain all possible interleavings.

5. Other Ways to Detect Concurrency Issues

Writing tests for concurrent code is difficult but possible. We've seen the challenges and some of the popular ways to overcome them. However, we may not be able to identify all possible concurrency issues through tests alone — especially when the incremental costs of writing more tests start to outweigh their benefits.

Hence, together with a reasonable number of automated tests, we can employ other techniques to identify concurrency issues. This will boost our chances of finding concurrency issues without getting too much deeper into the complexity of automated tests. We'll cover some of these in this section.

5.1. Static Analysis

Static analysis refers to the analysis of a program without actually executing it. Now, what good can such an analysis do? We will come to that, but let's first understand how it contrasts with dynamic analysis. The unit tests we've written so far need to be run with actual execution of the program they test. This is the reason they are part of what we largely refer to as dynamic analysis.

Please note that static analysis is in no way any replacement for dynamic analysis. However, it provides an invaluable tool to examine the code structure and identify possible defects long before we even execute the code. The static analysis makes use of a host of templates that are curated with experience and understanding.

While it's quite possible to just look through the code and compare against the best practices and rules we've curated, we must admit that it's not plausible for larger programs. There are, however, several tools available to perform this analysis for us. They are fairly mature, with a vast chest of rules for most of the popular programming languages.

A prevalent static analysis tool for Java is FindBugs. FindBugs looks for instances of “bug patterns”. A bug pattern is a code idiom that is quite often an error. This may arise due to several reasons like difficult language features, misunderstood methods, and misunderstood invariants.

FindBugs inspects the Java bytecode for occurrences of bug patterns without actually executing the bytecode. This is quite convenient to use and fast to run. FindBugs reports bugs belonging to many categories like conditions, design, and duplicated code.

It also includes defects related to concurrency. It must, however, be noted that FindBugs can report false positives. These are fewer in practice but must be correlated with manual analysis.

5.2. Model Checking

Model Checking is a method of checking whether a finite-state model of a system meets a given specification. Now, this definition may sound too academic, but bear with it for a while!

We can typically represent a computational problem as a finite-state machine. Although this is a vast area in itself, it gives us a model with a finite set of states and rules of transition between them with clearly defined start and end states.

Now, the specification defines how a model should behave for it to be considered as correct. Essentially, this specification holds all the requirements of the system that the model represents. One of the ways to capture specifications is using the temporal logic formula, developed by Amir Pnueli.

While it's logically possible to perform model checking manually, it's quite impractical. Fortunately, there are many tools available to help us here. One such tool available for Java is Java PathFinder (JPF). JPF was developed with years of experience and research at NASA.

Specifically, JPF is a model checker for Java bytecode. It runs a program in all possible ways, thereby checking for property violations like deadlock and unhandled exceptions along all possible execution paths. It can, therefore, prove to be quite useful in finding defects related to concurrency in any program.

6. Afterthoughts

By now, it shouldn't be a surprise to us that it's best to avoid complexities related to multi-threaded code as much as possible. Developing programs with simpler designs, which are easier to test and maintain, should be our prime objective. We have to agree that concurrent programming is often necessary for modern-day applications.

However, we can adopt several best practices and principles while developing concurrent programs that can make our life easier. In this section, we will go through some of these best practices, but we should keep in mind that this list is far from complete!

6.1. Reduce Complexity

Complexity is a factor that can make testing a program difficult even without any concurrent elements. This just compounds in the face of concurrency. It's not difficult to understand why simpler and smaller programs are easier to reason about and, hence, to test effectively. There are several best patterns that can help us here, like SRP (Single Responsibility Pattern) and KISS (Keep It Stupid Simple), to just name a few.

Now, while these do not address the issue of writing tests for concurrent code directly, they make the job easier to attempt.

6.2. Consider Atomic Operations

Atomic operations are operations that run completely independently of each other. Hence, the difficulties of predicting and testing interleaving can be simply avoided. Compare-and-swap is one such widely-used atomic instruction. Simply put, it compares the contents of a memory location with a given value and, only if they are the same, modifies the contents of that memory location.

Most modern microprocessors offer some variant of this instruction. Java offers a range of atomic classes like AtomicInteger and AtomicBoolean, offering the benefits of compare-and-swap instructions underneath.

6.3. Embrace Immutability

In multi-threaded programming, shared data that can be altered always leaves room for errors. Immutability refers to the condition where a data structure cannot be modified after instantiation. This is a match made in heaven for concurrent programs. If the state of an object can't be altered after its creation, competing threads do not have to apply for mutual exclusion on them. This greatly simplifies writing and testing concurrent programs.

However, please note that we may not always have the liberty to choose immutability, but we must opt for it when it's possible.

6.4. Avoid Shared Memory

Most of the issues related to multi-threaded programming can be attributed to the fact that we have shared memory between competing threads. What if we could just get rid of them! Well, we still need some mechanism for threads to communicate.

There are alternate design patterns for concurrent applications that offer us this possibility. One of the popular ones is the Actor Model, which prescribes the actor as the basic unit of concurrency. In this model, actors interact with each other by sending messages.

Akka is a framework written in Scala that leverages the Actor Model to offer better concurrency primitives.

7. Conclusion

Dalam tutorial ini, kami membahas beberapa asas yang berkaitan dengan pengaturcaraan serentak. Kami membincangkan serentak multi-utas di Java secara terperinci. Kami menempuh cabaran yang dihadapi kepada kami sambil menguji kod tersebut, terutama dengan data yang dikongsi. Selanjutnya, kami melalui beberapa alat dan teknik yang ada untuk menguji kod serentak.

Kami juga membincangkan cara lain untuk mengelakkan masalah bersamaan, termasuk alat dan teknik selain ujian automatik. Akhirnya, kami menjalani beberapa amalan terbaik pengaturcaraan yang berkaitan dengan pengaturcaraan serentak.

Kod sumber untuk artikel ini boleh didapati di GitHub.