How to send Logging Messages to SysLog using Log4j2 SysLogAppender in Linux - Java

Sometimes you may want to route your log messages to Syslog in UNIX based environment e.g. Linux. Since logger allows multiple appenders, you can print the same log messages to log file, console and route it to Syslog at the same time. In order to send log messages to Syslog using log4j2, you need to make some changes in your log4j2.xml file. This change is to include Syslog appender and configure that, once you do that you also need to enable TCP or UDP reception by editing rsyslog.conf file. For faster transmission you can choose UDP protocol and configure a port number on which your Syslog is listening, In our example, Syslog is listening on port 618. Let's see the complete step by step guide to configure Syslog logging in log4j logger.


How to send logger messages to Syslog using Log4j2 SysLogAppender in Linux

Here is the step by step instructor to configure Log4j for your Java application to send login messages to SysLog in Linux environment:


1) Configure SysLogAppender in log4j2.xml file


<Syslog name="Syslog" format="RFC5424" host="localhost" port="514"
 protocol="UDP" appName="yourApp" newLine="true" />

Syslog appender can only be used with two layouts RFC5424 or BSD, the format attribute is used to specify that. If set to "RFC5424" the data will be formatted in accordance with RFC 5424. Otherwise, it will be formatted as a BSD Syslog record. Note that although BSD Syslog records are required to be 1024 bytes or shorter the SyslogLayout does not truncate them.

The RFC5424Layout also does not truncate records since the receiver must accept records of up to 2048 bytes and may accept records that are longer. protocol can be either TCP or UDP, accordingly, you need to edit /etc/rsyslog.conf file to enable TCP or UDP reception.



2) Depending upon, which protocol you are using to publish log messages to Syslog, enable them into file /etc/rsyslog.conf

vi /etc/rsyslog.conf
 
 
# Provides UDP syslog reception
 
$ModLoad imudp.so
 
$UDPServerRun 514
 
 
# Provides TCP syslog reception
 
#$ModLoad imtcp.so
 
#$InputTCPServerRun 514


Since we are using UDP to publish log messages to Syslog, we have enabled UDP Syslog reception, by the way, you may need to use root login to edit this file. You can use su root to switch to root login. Pay attention to port value 514 as well, you need to provide the same port number to your syslog appender in log4j2.xml using the port attribute, as shown above, port="514" protocol="UDP".


3) Restart Syslog in Linux


$ /etc/init.d/rsyslog restart





4) Tail Syslog


Now, it's time to see the output by tailing Syslog file, you will most likely see

tail -f /var/log/messages
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level


Now, if you see this you will find that output is really simple, it's only printing log messages and not useful metadata like class, thread or even logging level. Thankfully you can configure that using LoggerFields nested element for RFC5424Layout. Just change your Syslog appender as following :

<Syslog name="Syslog" format="RFC5424" host="localhost" port="514" 
protocol="UDP" appName="This is Warning messages, printed at WARN level" 
newLine="true" />

<LoggerFields>
<KeyValuePair key="" value="%d{yyyy-MM-dd HH:mm:ss,SSS} [%t] %-4p %c " />
</LoggerFields>
</Syslog>



This will print formatted input, where values taken from the MDC data structure, which is a HashMap.

Output

<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp 
- [mdc@27030 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] 
WARN  com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp 
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN  
com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp 
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN  
com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp 
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN 
 com.startup.Hello"] This is Warning messages, printed at WARN level 

You can customize format depending upon your need e.g. if you only need logging level then you can just put %p in value.

Since windows environment doesn't have a Syslog facility, you can use networking tool which can listen to UDP messages on a port, socat is one such tool, you can run socat as

$ socat stdio udp-recv:514 

to receive UDP messages sent to port 514

How to send logger messages to syslog using Log4j2 SysLogAppender in Linux


Issue: Not seeing Structured data in Syslog

Even after correct setup and using LoggerFields, if you don't see structured data required to show metadata related to logging statements e.g. log level, category, thread and mapped diagnostic context data, it's time to check if your rsyslog supports structured data or not. Apparently, older versions of rsyslog, below 5.6.6 are not supporting structured data, so if you are running on the older version, it's time to update rsyslog binaries as well.

In order to confirm that whether your log4j setup is correct or not, I recommend listening UDP messages on port 514, since syslogger also listens on the same port, please stop it before starting your program, maybe a Perl or python script or even a Java program to listen UDP traffic, failing to do so may throw error "address already in use".

If you see structured data correctly then your log4j2 syslog appender is correctly setup and publishing data, and it's rsyslog which is not supporting full RFC5424 format, especially structured data. On the other hand, if you don't see structured data and only see space e.g. (-) then it's time to look your syslog configuration in log4j2.xml file. You must enable includeMDC field as true.


Further Learning
5 Free Linux Courses for Programmers
Linux Command Line Interface (CLI) Fundamentals
How Linux Works: What Every Superuser Should Know 
Learn Linux in 5 Days and Level Up Your Career


Other Java Logging Tutorials and Best Practices you may like
  • 5 Java Performance tuning books for experienced Programmers (list)
  • Why use Log4j logging vs System.out.println in Java? (answer)
  • Why use SLF4j over log4j for logging in Java? (answer)
  • How to enable SSL debugging log in Java Virtual Machine? (tips)
  • How to configure Log4j without XML or Properties file in Java? (tutorial)
  • Difference between Functional and Non-Functional Requirement (article)
  • 10 Essential JVM Options for Real-world Java Applications (tips)
  • How to use MDC or Mapped Diagnostic Context in Log4j? (tutorial)
  • 10 Tips for logging in Java application (tips)
  • How to fix Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory in Java (solution)
Thanks for reading this article so far. If you have liked this article then you may want to like our Facebook page (Javarevisited) as well to receive updates and information about Java and related technology. If you have any suggestions or feedback then please drop a comment.

No comments:

Post a Comment