days
-4
-5
hours
0
-2
minutes
0
-1
seconds
-4
-5
search
Plus JAX London Blind Bird special

Tune your Garbage Collector

Ram Lakshmanan
© Shutterstock / Pretty Vectors

Developers take advantage of the JVM argument -XX:+UseGCLogFileRotation to rotate GC log files. However, this approach has a few challenges. In this article, JAX London speaker Ram Lakshmanan has some recommendations on how to deal with those issues.

qweq

Have you heard of JAX London?

JAX London is our four-day conference for cutting edge software engineers and enterprise-level professionals.

Interested? We have a special offer for you! Don’t miss out the Blind Bird Special offer that ends on the 6th of June!

But for now, here’s a little taste of what JAX London is all about with this mini tutorial by JAX London speaker Ram Lakshmanan.

~ ~ ~

Developers take advantage of the JVM argument -XX:+UseGCLogFileRotation to rotate GC log files. Example:

-XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/home/GCEASY/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=20M

As per the above example, JVM will rotate GC log file whenever its size reaches 20MB. It will generate files up to 5 files, with extensions gc.log.0, gc.log.1, gc.log.2, gc.log.3, gc.log.4.

But this approach has few challenges. Let’s have a look.

a) Losing old GC logs

Suppose if you had configured -XX:NumberOfGCLogFiles=5 then, over a period of time, 5 GC log files will be created:

gc.log.0 <– oldest GC Log content
gc.log.1
gc.log.2
gc.log.3
gc.log.4 <– latest GC Log content

Most recent GC log contents will be written to ‘gc.log.4’ and old GC log content will be present in ‘gc.log.0’.

When the application starts to generate more GC logs than the configured ‘-XX:NumberOfGCLogFiles’ in this case 5, then old GC log contents in gc.log.0 will be deleted. New GC events will be written to gc.log.0. It means you will end up not having all the generated GC logs. You will loose the visibility of all events.

b) Mixed-up GC logs

Suppose application has created 5 gc log files i.e.

gc.log.0
gc.log.1
gc.log.2
gc.log.3
gc.log.4

then, let’s say you are restarting the application. Now new GC logs will be written to gc.log.0 file and old GC log content will be present in gc.log.1, gc.log.2, gc.log.3, gc.log.4 i.e.

gc.log.0 <– GC log file content after restart
gc.log.1 <– GC log file content before restart
gc.log.2 <– GC log file content before restart
gc.log.3 <– GC log file content before restart
gc.log.4 <– GC log file content before restart

So your new GC log contents get mixed up with old GC logs. Thus, to mitigate this problem you might have to move all the old GC logs to a different folder before you restart the application.

c) Forwarding GC logs to central location

In this approach, current active file to which GC logs are written is marked with extension “.current”. Example, if GC events are currently written to file ‘gc.log.3’ it would be named as: ‘gc.log.3.current’.

If you want to forward GC logs from each server to a central location, then most DevOps engineers use ‘rsyslog’. However, this file naming convention poses a significant challenge to use ‘rsyslog’, as described in this blog.

d) Tooling

Now to analyze the GC log file using the GC tools such as (gceasy.io, GCViewer….), you will have to upload multiple GC log files instead of just one single GC Log file.

Recommended solution

We can suffix the GC log file with the time stamp at which the JVM was restarted then, GC Log file locations will become unique. Then new GC logs will not over-ride the old GC logs. It can be achieved by suffixing ‘%t’ to the GC log file name as shown below:

"-XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/home/GCEASY/gc-%t.log"

‘%t’ suffixes timestamp to the GC log file in the format: ‘YYYY-MM-DD_HH-MM-SS’. So generated GC log file name will start to look like: ‘gc-2019-01-29_20-41-47.log’

This simple solution addresses all the shortcomings of -XX:+UseGCLogFileRotation.

Author
java

Ram Lakshmanan

Every single day, millions & millions of people in North America—bank, travel, and commerce—use the applications that Ram Lakshmanan has architected. Ram is an acclaimed speaker in major conferences on scalability, availability, and performance topics. Recently, he has founded a startup, which specializes in troubleshooting performance problems.


Leave a Reply

Be the First to Comment!

avatar
400
  Subscribe  
Notify of