Peningkatan Log Java dengan Konteks Diagnostik yang dipetakan (MDC)

1. Gambaran keseluruhan

Dalam artikel ini, kami akan meneroka penggunaan Konteks Diagnostik yang Dipetakan (MDC) untuk meningkatkan pembalakan aplikasi.

Idea asas Konteks Diagnostik yang dipetakan adalah menyediakan cara untuk memperkaya mesej log dengan maklumat yang mungkin tidak tersedia dalam skop di mana pembalakan itu sebenarnya berlaku, tetapi itu memang berguna untuk melacak pelaksanaan program dengan lebih baik.

2. Mengapa Menggunakan MDC

Mari kita mulakan dengan contoh. Anggaplah kita harus menulis perisian yang memindahkan wang. Kami menubuhkan kelas Transfer untuk mewakili beberapa maklumat asas: id pemindahan unik dan nama pengirim:

public class Transfer { private String transactionId; private String sender; private Long amount; public Transfer(String transactionId, String sender, long amount) { this.transactionId = transactionId; this.sender = sender; this.amount = amount; } public String getSender() { return sender; } public String getTransactionId() { return transactionId; } public Long getAmount() { return amount; } } 

Untuk melakukan pemindahan, kita perlu menggunakan perkhidmatan yang disokong oleh API mudah:

public abstract class TransferService { public boolean transfer(long amount) { // connects to the remote service to actually transfer money } abstract protected void beforeTransfer(long amount); abstract protected void afterTransfer(long amount, boolean outcome); } 

Kaedah sebelumTransfer () dan afterTransfer () boleh diganti untuk menjalankan kod tersuai tepat sebelum dan tepat setelah pemindahan selesai.

Kami akan memanfaatkan sebelumTransfer () dan afterTransfer () untuk mencatat beberapa maklumat mengenai pemindahan .

Mari buat pelaksanaan perkhidmatan:

import org.apache.log4j.Logger; import com.baeldung.mdc.TransferService; public class Log4JTransferService extends TransferService { private Logger logger = Logger.getLogger(Log4JTransferService.class); @Override protected void beforeTransfer(long amount) { logger.info("Preparing to transfer " + amount + "$."); } @Override protected void afterTransfer(long amount, boolean outcome) { logger.info( "Has transfer of " + amount + "$ completed successfully ? " + outcome + "."); } } 

Isu utama yang perlu diperhatikan di sini adalah bahawa apabila mesej log dibuat, tidak mungkin untuk mengakses objek Transfer - hanya jumlah yang dapat diakses, sehingga mustahil untuk log sama ada transaksi transaksi atau pengirimnya.

Mari sediakan fail log4j.properties biasa untuk log masuk konsol:

log4j.appender.consoleAppender=org.apache.log4j.ConsoleAppender log4j.appender.consoleAppender.layout=org.apache.log4j.PatternLayout log4j.appender.consoleAppender.layout.ConversionPattern=%-4r [%t] %5p %c %x - %m%n log4j.rootLogger = TRACE, consoleAppender 

Mari akhirnya siapkan aplikasi kecil yang dapat menjalankan beberapa pemindahan pada masa yang sama melalui ExecutorService :

public class TransferDemo { public static void main(String[] args) { ExecutorService executor = Executors.newFixedThreadPool(3); TransactionFactory transactionFactory = new TransactionFactory(); for (int i = 0; i < 10; i++) { Transfer tx = transactionFactory.newInstance(); Runnable task = new Log4JRunnable(tx); executor.submit(task); } executor.shutdown(); } }

Kami perhatikan bahawa untuk menggunakan ExecutorService , kita perlu menyelesaikan pelaksanaan Log4JTransferService dalam penyesuai kerana executor.submit () mengharapkan Runnable :

public class Log4JRunnable implements Runnable { private Transfer tx; public Log4JRunnable(Transfer tx) { this.tx = tx; } public void run() { log4jBusinessService.transfer(tx.getAmount()); } } 

Apabila kami menjalankan aplikasi demo kami yang menguruskan banyak pemindahan pada masa yang sama, kami dengan cepat mengetahui bahawa log tidak berguna seperti yang kami mahukan . Sangat sukar untuk melacak pelaksanaan setiap pemindahan kerana satu-satunya maklumat berguna yang dicatat adalah jumlah wang yang dipindahkan dan nama utas yang melakukan pemindahan tersebut.

Lebih-lebih lagi, mustahil untuk membezakan antara dua transaksi berbeza dengan jumlah yang sama yang dilaksanakan oleh utas yang sama kerana garis log yang berkaitan pada dasarnya sama:

... 519 [pool-1-thread-3] INFO Log4JBusinessService - Preparing to transfer 1393$. 911 [pool-1-thread-2] INFO Log4JBusinessService - Has transfer of 1065$ completed successfully ? true. 911 [pool-1-thread-2] INFO Log4JBusinessService - Preparing to transfer 1189$. 989 [pool-1-thread-1] INFO Log4JBusinessService - Has transfer of 1350$ completed successfully ? true. 989 [pool-1-thread-1] INFO Log4JBusinessService - Preparing to transfer 1178$. 1245 [pool-1-thread-3] INFO Log4JBusinessService - Has transfer of 1393$ completed successfully ? true. 1246 [pool-1-thread-3] INFO Log4JBusinessService - Preparing to transfer 1133$. 1507 [pool-1-thread-2] INFO Log4JBusinessService - Has transfer of 1189$ completed successfully ? true. 1508 [pool-1-thread-2] INFO Log4JBusinessService - Preparing to transfer 1907$. 1639 [pool-1-thread-1] INFO Log4JBusinessService - Has transfer of 1178$ completed successfully ? true. 1640 [pool-1-thread-1] INFO Log4JBusinessService - Preparing to transfer 674$. ... 

Nasib baik, MDC dapat membantu.

3. MDC di Log4j

Mari memperkenalkan MDC .

MDC di Log4j membolehkan kami mengisi struktur seperti peta dengan maklumat yang dapat diakses oleh pelekat ketika mesej log sebenarnya ditulis.

Struktur MDC dilampirkan secara dalaman ke thread pelaksanaan dengan cara yang sama seperti pemboleh ubah ThreadLocal .

Oleh itu, idea peringkat tinggi adalah:

  1. untuk mengisi MDC dengan maklumat yang ingin kami sediakan untuk pelampir
  2. kemudian log mesej
  3. dan akhirnya, kosongkan MDC

Pola pelekat harus diubah dengan jelas untuk mendapatkan pemboleh ubah yang disimpan di MDC.

Oleh itu, mari ubah kod mengikut garis panduan ini:

import org.apache.log4j.MDC; public class Log4JRunnable implements Runnable { private Transfer tx; private static Log4JTransferService log4jBusinessService = new Log4JTransferService(); public Log4JRunnable(Transfer tx) { this.tx = tx; } public void run() { MDC.put("transaction.id", tx.getTransactionId()); MDC.put("transaction.owner", tx.getSender()); log4jBusinessService.transfer(tx.getAmount()); MDC.clear(); } } 

MDC.put () tidak mengejutkan digunakan untuk menambahkan kunci dan nilai yang sesuai dalam MDC sementara MDC.clear () mengosongkan MDC.

Let's now change the log4j.properties to print the information that we've just store in the MDC. It is enough to change the conversion pattern, using the %X{} placeholder for each entry contained in the MDC we would like to be logged:

log4j.appender.consoleAppender.layout.ConversionPattern= %-4r [%t] %5p %c{1} %x - %m - tx.id=%X{transaction.id} tx.owner=%X{transaction.owner}%n

Now, if we run the application, we'll note that each line carries also the information about the transaction being processed making far easier for us to track the execution of the application:

638 [pool-1-thread-2] INFO Log4JBusinessService - Has transfer of 1104$ completed successfully ? true. - tx.id=2 tx.owner=Marc 638 [pool-1-thread-2] INFO Log4JBusinessService - Preparing to transfer 1685$. - tx.id=4 tx.owner=John 666 [pool-1-thread-1] INFO Log4JBusinessService - Has transfer of 1985$ completed successfully ? true. - tx.id=1 tx.owner=Marc 666 [pool-1-thread-1] INFO Log4JBusinessService - Preparing to transfer 958$. - tx.id=5 tx.owner=Susan 739 [pool-1-thread-3] INFO Log4JBusinessService - Has transfer of 783$ completed successfully ? true. - tx.id=3 tx.owner=Samantha 739 [pool-1-thread-3] INFO Log4JBusinessService - Preparing to transfer 1024$. - tx.id=6 tx.owner=John 1259 [pool-1-thread-2] INFO Log4JBusinessService - Has transfer of 1685$ completed successfully ? false. - tx.id=4 tx.owner=John 1260 [pool-1-thread-2] INFO Log4JBusinessService - Preparing to transfer 1667$. - tx.id=7 tx.owner=Marc 

4. MDC in Log4j2

The very same feature is available in Log4j2 too, so let's see how to use it.

Let's firstly set up a TransferService subclass that logs using Log4j2:

import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; public class Log4J2TransferService extends TransferService { private static final Logger logger = LogManager.getLogger(); @Override protected void beforeTransfer(long amount) { logger.info("Preparing to transfer {}$.", amount); } @Override protected void afterTransfer(long amount, boolean outcome) { logger.info("Has transfer of {}$ completed successfully ? {}.", amount, outcome); } } 

Let's then change the code that uses the MDC, which is actually called ThreadContext in Log4j2:

import org.apache.log4j.MDC; public class Log4J2Runnable implements Runnable { private final Transaction tx; private Log4J2BusinessService log4j2BusinessService = new Log4J2BusinessService(); public Log4J2Runnable(Transaction tx) { this.tx = tx; } public void run() { ThreadContext.put("transaction.id", tx.getTransactionId()); ThreadContext.put("transaction.owner", tx.getOwner()); log4j2BusinessService.transfer(tx.getAmount()); ThreadContext.clearAll(); } } 

Again, ThreadContext.put() adds an entry in the MDC and ThreadContext.clearAll() removes all the existing entries.

We still miss the log4j2.xml file to configure the logging. As we can note, the syntax to specify which MDC entries should be logged is the same than the one used in Log4j:

Again, let's execute the application and we'll see the MDC information being printed in the log:

1119 [pool-1-thread-3] INFO Log4J2BusinessService - Has transfer of 1198$ completed successfully ? true. - tx.id=3 tx.owner=Samantha 1120 [pool-1-thread-3] INFO Log4J2BusinessService - Preparing to transfer 1723$. - tx.id=5 tx.owner=Samantha 1170 [pool-1-thread-2] INFO Log4J2BusinessService - Has transfer of 701$ completed successfully ? true. - tx.id=2 tx.owner=Susan 1171 [pool-1-thread-2] INFO Log4J2BusinessService - Preparing to transfer 1108$. - tx.id=6 tx.owner=Susan 1794 [pool-1-thread-1] INFO Log4J2BusinessService - Has transfer of 645$ completed successfully ? true. - tx.id=4 tx.owner=Susan 

5. MDC in SLF4J/Logback

MDC is available in SLF4J, too, under the condition that it is supported by the underlying logging library.

Both Logback and Log4j support MDC as we've just seen, so we need nothing special to use it with a standard set up.

Let's prepare the usual TransferService subclass, this time using the Simple Logging Facade for Java:

import org.slf4j.Logger; import org.slf4j.LoggerFactory; final class Slf4TransferService extends TransferService { private static final Logger logger = LoggerFactory.getLogger(Slf4TransferService.class); @Override protected void beforeTransfer(long amount) { logger.info("Preparing to transfer {}$.", amount); } @Override protected void afterTransfer(long amount, boolean outcome) { logger.info("Has transfer of {}$ completed successfully ? {}.", amount, outcome); } } 

Let's now use the SLF4J's flavor of MDC. In this case, the syntax and semantics are the same as that in log4j:

import org.slf4j.MDC; public class Slf4jRunnable implements Runnable { private final Transaction tx; public Slf4jRunnable(Transaction tx) { this.tx = tx; } public void run() { MDC.put("transaction.id", tx.getTransactionId()); MDC.put("transaction.owner", tx.getOwner()); new Slf4TransferService().transfer(tx.getAmount()); MDC.clear(); } } 

We have to provide the Logback configuration file, logback.xml:

   %-4r [%t] %5p %c{1} - %m - tx.id=%X{transaction.id} tx.owner=%X{transaction.owner}%n       

Again, we'll see that the information in the MDC is properly added to the logged messages, even though this information is not explicitly provided in the log.info() method:

1020 [pool-1-thread-3] INFO c.b.m.s.Slf4jBusinessService - Has transfer of 1869$ completed successfully ? true. - tx.id=3 tx.owner=John 1021 [pool-1-thread-3] INFO c.b.m.s.Slf4jBusinessService - Preparing to transfer 1303$. - tx.id=6 tx.owner=Samantha 1221 [pool-1-thread-1] INFO c.b.m.s.Slf4jBusinessService - Has transfer of 1498$ completed successfully ? true. - tx.id=4 tx.owner=Marc 1221 [pool-1-thread-1] INFO c.b.m.s.Slf4jBusinessService - Preparing to transfer 1528$. - tx.id=7 tx.owner=Samantha 1492 [pool-1-thread-2] INFO c.b.m.s.Slf4jBusinessService - Has transfer of 1110$ completed successfully ? true. - tx.id=5 tx.owner=Samantha 1493 [pool-1-thread-2] INFO c.b.m.s.Slf4jBusinessService - Preparing to transfer 644$. - tx.id=8 tx.owner=John

It is worth noting that in case we set up the SLF4J back-end to a logging system that does not support MDC, all the related invocations will be simply skipped without side effects.

6. MDC and Thread Pools

MDC implementations are usually using ThreadLocals to store the contextual information. That's an easy and reasonable way to achieve thread-safety. However, we should be careful using MDC with thread pools.

Let's see how the combination of ThreadLocal-based MDCs and thread pools can be dangerous:

  1. We get a thread from the thread pool.
  2. Then we store some contextual information in MDC using MDC.put() or ThreadContext.put().
  3. We use this information in some logs and somehow we forgot to clear the MDC context.
  4. The borrowed thread comes back to the thread pool.
  5. After a while, the application gets the same thread from the pool.
  6. Since we didn't clean up the MDC last time, this thread still owns some data from the previous execution.

This may cause some unexpected inconsistencies between executions. One way to prevent this is to always remember to clean up the MDC context at the end of each execution. This approach usually needs rigorous human supervision and, therefore, is error-prone.

Another approach is to use ThreadPoolExecutor hooks and perform necessary cleanups after each execution. To do that, we can extend the ThreadPoolExecutor class and override the afterExecute() hook:

public class MdcAwareThreadPoolExecutor extends ThreadPoolExecutor { public MdcAwareThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) { super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler); } @Override protected void afterExecute(Runnable r, Throwable t) { System.out.println("Cleaning the MDC context"); MDC.clear(); org.apache.log4j.MDC.clear(); ThreadContext.clearAll(); } }

This way, the MDC cleanup would happen after each normal or exceptional execution automatically. So, there is no need to do it manually:

@Override public void run() { MDC.put("transaction.id", tx.getTransactionId()); MDC.put("transaction.owner", tx.getSender()); new Slf4TransferService().transfer(tx.getAmount()); }

Sekarang kita dapat menulis semula demo yang sama dengan pelaksanaan pelaksana baru kita:

ExecutorService executor = new MdcAwareThreadPoolExecutor(3, 3, 0, MINUTES, new LinkedBlockingQueue(), Thread::new, new AbortPolicy()); TransactionFactory transactionFactory = new TransactionFactory(); for (int i = 0; i < 10; i++) { Transfer tx = transactionFactory.newInstance(); Runnable task = new Slf4jRunnable(tx); executor.submit(task); } executor.shutdown();

7. Kesimpulannya

MDC mempunyai banyak aplikasi, terutamanya dalam senario di mana pelaksanaan beberapa utas yang berbeza menyebabkan mesej log yang disisipkan yang sukar dibaca.

Dan seperti yang kita lihat, ia disokong oleh tiga kerangka kerja pembalakan yang paling banyak digunakan di Java.

Seperti biasa, anda akan menemui sumbernya di GitHub.