How to Manage System Logs with Amazon S3?

Manage System Logs With AWS S3

Viewing Log File Contents

Linux system administrators often need to look at log files for troubleshooting purposes. In fact, this is the first thing any sysadmin would do.

Linux and the applications that run on it can generate all different types of messages, which are recorded in various log files. Linux uses a set of configuration files, directories, programs, commands and daemons to create, store and recycle these log messages. Knowing where the system keeps its log files and how to make use of related commands can therefore help save valuable time during troubleshooting.

In this tutorial, we will have a look at different parts of the Linux logging mechanism.
Disclaimer

The commands in this tutorial were tested in plain vanilla installations of CentOS 6.4, Ubuntu 12 and Debian 7.

The rsyslog Daemon

At the heart of the logging mechanism is the rsyslog daemon. This service is responsible for listening to log messages from different parts of a Linux system and routing the message to an appropriate log file in the /var/log directory. It can also forward log messages to another Linux server.

Dec 16 01:21:08 debian kernel: [    9.584074] Bluetooth: BNEP (Ethernet Emulation) ver 1.3  
Dec 16 01:21:08 debian kernel: [    9.584074] Bluetooth: BNEP filters: protocol multicast  
Dec 16 01:21:08 debian kernel: [    9.648220] Bridge firewalling registered
Dec 16 01:21:08 debian kernel: [    9.696728] Bluetooth: SCO (Voice Link) ver 0.6 
Dec 16 01:21:08 debian kernel: [    9.696728] Bluetooth: SCO socket layer initialized  
Dec 16 01:21:08 debian kernel: [    9.832215] lp: driver loaded but no devices found  
Dec 16 01:21:08 debian kernel: [    9.868897] ppdev: user-space parallel port driver  
Dec 16 01:21:11 debian kernel: [   12.748833] [drm] Initialized drm 1.1.0 20060810
Dec 16 01:21:11 debian kernel: [   12.754412] pci 0000:00:02.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, low) -> IRQ 11  
Dec 16 01:21:11 debian kernel: [   12.754412] [drm] Initialized vboxvideo 1.0.0 20090303 for 0000:00:02.0 on minor 0

The rsyslog Configuration File

The rsyslog daemon gets its configuration information from the rsyslog.conf file. The file is located under the /etc directory.

Basically, the rsyslog.conf file tells the rsyslog daemon where to save its log messages. This instruction comes from a series of two-part lines within the file.

This file can be found at rsyslog.d/50-default.conf on ubuntu.

The two part instruction is made up of a selector and an action. The two parts are separated by white space.

The selector part specifies what’s the source and importance of the log message and the action part says what to do with the message.

The selector itself is again divided into two parts separated by a dot (.). The first part before the dot is called *acility (the origin of the message) and the second part after the dot is called priority (the severity of the message).

Together, the facility/priority and the action pair tell rsyslog what to do when a log message matching the criteria is generated.

Here is excerpt from a CentOS rsyslog.conf file:

# rsyslog v5 configuration file
...  
...    
# Include all config files in /etc/rsyslog.d/  
IncludeConfig /etc/rsyslog.d/*.conf  

#### RULES ####  
# Log all kernel messages to the console.  
# Logging much else clutters up the screen.  
#kern.*  /dev/console  

# Log anything (except mail) of level info or higher.  
# Don't log private authentication messages!  
*.info;mail.none;authpriv.none;cron.none                /var/log/messages  

# The authpriv file has restricted access.  
authpriv.*                                              /var/log/secure  

# Log all the mail messages in one place.  
mail.*                                                  -/var/log/maillog  


# Log cron stuff  
cron.*                                                  /var/log/cron  

# Everybody gets emergency messages  
*.emerg                                                 *  

# Save news errors of level crit and higher in a special file.  
uucp,news.crit                                          /var/log/spooler  

# Save boot messages also to boot.log  
local7.*                                                /var/log/boot.log  
...  
...

Creating and Testing Your Own Log Messages

The rsyslog daemon gets its configuration information from the rsyslog.conf file. The file is located under the /etc directory.

Basically, the rsyslog.conf file tells the rsyslog daemon where to save its log messages. This instruction comes from a series of two-part lines within the file.

This file can be found at rsyslog.d/50-default.conf on ubuntu.

The two part instruction is made up of a selector and an action. The two parts are separated by white space.

The selector part specifies what’s the source and importance of the log message and the action part says what to do with the message.

The selector itself is again divided into two parts separated by a dot (.). The first part before the dot is called *acility (the origin of the message) and the second part after the dot is called priority (the severity of the message).

Together, the facility/priority and the action pair tell rsyslog what to do when a log message matching the criteria is generated.

Here is excerpt from a CentOS rsyslog.conf file:

# rsyslog v5 configuration file
...  
...    
# Include all config files in /etc/rsyslog.d/  
IncludeConfig /etc/rsyslog.d/*.conf  

#### RULES ####  
# Log all kernel messages to the console.  
# Logging much else clutters up the screen.  
#kern.*  /dev/console  

# Log anything (except mail) of level info or higher.  
# Don't log private authentication messages!  
*.info;mail.none;authpriv.none;cron.none                /var/log/messages  

# The authpriv file has restricted access.  
authpriv.*                                              /var/log/secure  

# Log all the mail messages in one place.  
mail.*                                                  -/var/log/maillog  


# Log cron stuff  
cron.*                                                  /var/log/cron  

# Everybody gets emergency messages  
*.emerg                                                 *  

# Save news errors of level crit and higher in a special file.  
uucp,news.crit                                          /var/log/spooler  

# Save boot messages also to boot.log  
local7.*                                                /var/log/boot.log  
...  
...

Rotating Log Files

The rsyslog daemon gets its configuration information from the rsyslog.conf file. The file is located under the /etc directory.

Basically, the rsyslog.conf file tells the rsyslog daemon where to save its log messages. This instruction comes from a series of two-part lines within the file.

This file can be found at rsyslog.d/50-default.conf on ubuntu.

The two part instruction is made up of a selector and an action. The two parts are separated by white space.

The selector part specifies what’s the source and importance of the log message and the action part says what to do with the message.

The selector itself is again divided into two parts separated by a dot (.). The first part before the dot is called *acility (the origin of the message) and the second part after the dot is called priority (the severity of the message).

Together, the facility/priority and the action pair tell rsyslog what to do when a log message matching the criteria is generated.

Here is excerpt from a CentOS rsyslog.conf file:

# rsyslog v5 configuration file
...  
...    
# Include all config files in /etc/rsyslog.d/  
IncludeConfig /etc/rsyslog.d/*.conf  

#### RULES ####  
# Log all kernel messages to the console.  
# Logging much else clutters up the screen.  
#kern.*  /dev/console  

# Log anything (except mail) of level info or higher.  
# Don't log private authentication messages!  
*.info;mail.none;authpriv.none;cron.none                /var/log/messages  

# The authpriv file has restricted access.  
authpriv.*                                              /var/log/secure  

# Log all the mail messages in one place.  
mail.*                                                  -/var/log/maillog  


# Log cron stuff  
cron.*                                                  /var/log/cron  

# Everybody gets emergency messages  
*.emerg                                                 *  

# Save news errors of level crit and higher in a special file.  
uucp,news.crit                                          /var/log/spooler  

# Save boot messages also to boot.log  
local7.*                                                /var/log/boot.log  
...  
...

Related Posts