Friday, July 30, 2021

How to send Logging Messages to SysLog using Log4j2 SysLogAppender in Linux ? Java Example

Sometimes you may want to route your log messages to Syslog in UNIX-based environment like Linux. Since logger allows multiple appenders, you can print the same log messages to a log file, console, and route it to Syslog at the same time. In order to send log messages to Syslog using log4j2, you need to make some changes in your log4j2.xml file. This change is to include the Syslog appender and configure that, once you do that you also need to enable TCP or UDP reception by editing rsyslog.conf file. For faster transmission you can choose UDP protocol and configure a port number on which your Syslog is listening, In our example, Syslog is listening on port 618. Let's see the complete step-by-step guide to configure Syslog logging in log4j logger.


How to send logger messages to Syslog using Log4j2 SysLogAppender in Linux

Here is the step by step instructor to configure Log4j for your Java application to send login messages to SysLog in the Linux environment:


1. Configure SysLogAppender in log4j2.xml file

<Syslog name="Syslog" format="RFC5424" host="localhost" port="514"
 protocol="UDP" appName="yourApp" newLine="true" />

Syslog appender can only be used with two layouts RFC5424 or BSD, the format attribute is used to specify that. If set to "RFC5424" the data will be formatted in accordance with RFC 5424. Otherwise, it will be formatted as a BSD Syslog record. Note that although BSD Syslog records are required to be 1024 bytes or shorter the SyslogLayout does not truncate them.

The RFC5424Layout also does not truncate records since the receiver must accept records of up to 2048 bytes and may accept records that are longer. the protocol can be either TCP or UDP, accordingly, you need to edit /etc/rsyslog.conf file to enable TCP or UDP reception.



2. Depending upon, which protocol you are using to publish log messages to Syslog, enable them into file /etc/rsyslog.conf

vi /etc/rsyslog.conf
 
 
# Provides UDP syslog reception
 
$ModLoad imudp.so
 
$UDPServerRun 514
 
 
# Provides TCP syslog reception
 
#$ModLoad imtcp.so
 
#$InputTCPServerRun 514


Since we are using UDP to publish log messages to Syslog, we have enabled UDP Syslog reception, by the way, you may need to use root login to edit this file. You can use su root to switch to the root login. Pay attention to port value 514 as well, you need to provide the same port number to your syslog appender in log4j2.xml using the port attribute, as shown above, port="514" protocol="UDP".


3. Restart Syslog in Linux


$ /etc/init.d/rsyslog restart





4. Tail Syslog

Now, it's time to see the output by tailing the Syslog file, you will most likely see

tail -f /var/log/messages
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level
Nov 19 12:20:12 TestHost yourApp - This is Warning messages, printed at WARN level

Now, if you see this you will find that output is really simple, it's only printing log messages and not useful metadata like class, thread or even logging level. Thankfully you can configure that using LoggerFields nested element for RFC5424Layout. Just change your Syslog appender as following :

<Syslog name="Syslog" format="RFC5424" host="localhost" port="514" 
protocol="UDP" appName="This is Warning messages, printed at WARN level" 
newLine="true" />

<LoggerFields>
<KeyValuePair key="" value="%d{yyyy-MM-dd HH:mm:ss,SSS} [%t] %-4p %c " />
</LoggerFields>
</Syslog>



This will print formatted input, where values are taken from the MDC data structure, which is a HashMap.

Output

<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp 
- [mdc@27030 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] 
WARN  com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp 
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN  
com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp 
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN  
com.startup.Hello"] This is Warning messages, printed at WARN level
<182>1 2019-11-18T11:46:43.011+08:00 TestHost yourApp 
- [mdc@18060 ="2019-11-18 11:46:43,011 [pool-1 thread-1\] WARN 
 com.startup.Hello"] This is Warning messages, printed at WARN level 

You can customize the format depending upon your need e.g. if you only need logging level then you can just put %p in value.

Since windows environment doesn't have a Syslog facility, you can use a networking tool that can listen to UDP messages on a port, socat is one such tool, you can run socat as

$ socat stdio udp-recv:514 

to receive UDP messages sent to port 514. You can further see these free Linux courses to learn more about essential Linux commands. 

How to send logger messages to syslog using Log4j2 SysLogAppender in Linux


Issue: Not seeing Structured data in Syslog

Even after correct setup and using LoggerFields, if you don't see structured data required to show metadata related to logging statements e.g. log level, category, thread, and mapped diagnostic context data, it's time to check if your rsyslog supports structured data or not. Apparently, older versions of rsyslog, below 5.6.6 are not supporting structured data, so if you are running on the older version, it's time to update rsyslog binaries as well.

In order to confirm that whether your log4j setup is correct or not, I recommend listening to UDP messages on port 514, since syslogger also listens on the same port, please stop it before starting your program, maybe a Perl or python script or even a Java program to listen to UDP traffic, failing to do so may throw error "address already in use".

If you see structured data correctly then your log4j2 syslog appender is correctly set up and publishing data, and it's rsyslog which is not supporting full RFC5424 format, especially structured data. On the other hand, if you don't see structured data and only see space e.g. (-) then it's time to look at your Syslog configuration in log4j2.xml file. You must enable includeMDC field as true.



Other Java Logging Tutorials and Best Practices you may like
  • 5 Java Performance tuning books for experienced Programmers (list)
  • Why use Log4j logging vs System.out.println in Java? (answer)
  • Why use SLF4j over log4j for logging in Java? (answer)
  • How to enable SSL debugging log in Java Virtual Machine? (tips)
  • How to configure Log4j without XML or Properties file in Java? (tutorial)
  • Difference between Functional and Non-Functional Requirement (article)
  • 10 Essential JVM Options for Real-world Java Applications (tips)
  • How to use MDC or Mapped Diagnostic Context in Log4j? (tutorial)
  • 10 Tips for logging in Java application (tips)
  • How to fix Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory in Java (solution)
Thanks for reading this article so far. If you have liked this article then you may want to like our Facebook page (Javarevisited) as well as to receive updates and information about Java and related technology. If you have any suggestions or feedback then please drop a comment.

No comments:

Post a Comment

Feel free to comment, ask questions if you have any doubt.