Howto: Remote logging with syslog-ng

As anyone who uses any flavor of GNU/Linux should know, system logs are stored in /var/log. If you are like me, you may have more than one machine at home. Maybe you have a file server, or a gateway box, or maybe a workstation that you allow your kids to use so they don't mess up your own. Being able to review system logs is an important task. It allows you to debug problems, as well as track things that could alert you to potential problems. If you have more than one PC, this can be a pain in the rear. I found numerous HOWTO's on the internet on how to setup remote logging... some were a little out dated, and others gave too many options that made it all a bit confusing. This little howto is how I setup my remote system logging to make life a little simpler.

My GNU/Linux distro of choice is Gentoo, but this should work on any flavor of linux that uses the logging daemon syslog-ng. Check your package manager to see if that is what you are using. If you're using something else, like SYSKLOGD, you will need to replace it.

First you will need a remote-logging server. This could very well be your desktop computer, or some other dedicated machine. This will be the machine that stores the logs from all your other computers. I will refer to this machine as the log server. Remote machines that you wish to keep track of (file servers, gateways, proxy server, whatever) will be refered to as remote. So lets get started.

Log Server Setup

1. If you are using Gentoo, switch on the hardened use flag for syslog-ng. It makes syslog-ng logs a little more organized. Why this is not done by default is beyond me.
2. The syslog-ng config file should be located at /etc/syslog-ng/syslog-ng.conf. Open it up with your favorite text editor. You should see several lines that look like this example:
   
source src { unix-stream("/dev/log"); internal(); };
source kernsrc { file("/proc/kmsg"); };
destination authlog { file("/var/log/auth.log"); };
filter f_auth { facility(auth); };
filter f_authpriv { facility(auth, authpriv); };
log { source(src); filter(f_authpriv); destination(authlog); };
log { source(src); filter(f_syslog); destination(syslog); };

   It's ok if you have more, or slightly different information. this is just an example.
   
3. At the bottom of the file, you are going to add the following:
   
source remote_log {
udp( port(1999) );
};

   This tells syslog-ng that we are going to listen on udp port 1999 for incoming log information.
   Next we are going to add destination files for the logs we receive over the network.
   
#define destinations.
destination remote_authlog { file("/var/log/remote.d/$HOST/auth.log"); };
destination remote_syslog { file("/var/log/remote.d/$HOST/syslog"); };
destination remote_cron { file("/var/log/remote.d/$HOST/cron.log"); };
destination remote_daemon { file("/var/log/remote.d/$HOST/daemon.log");};
destination remote_kern { file("/var/log/remote.d/$HOST/kern.log"); };
destination remote_lpr { file("/var/log/remote.d/$HOST/lpr.log"); };
destination remote_user { file("/var/log/remote.d/$HOST/user.log"); };
# Should be remote_maillog (Without dot) as it was the default on logwatch
destination remote_mail { file("/var/log/remote.d/$HOST/maillog"); };
destination remote_mailinfo { file("/var/log/remote.d/$HOST/mail.info");};
destination remote_mailwarn { file("/var/log/remote.d/$HOST/mail.warn");};
destination remote_mailerr { file("/var/log/remote.d/$HOST/mail.err");};
destination remote_newscrit { file("/var/log/remote.d/$HOST/news/news.crit");};
destination remote_newserr { file("/var/log/remote.d/$HOST/news/news.err");};
destination remote_newsnotice { file("/var/log/remote.d/$HOST/news/news.notice");};
destination remote_debug { file("/var/log/remote.d/$HOST/debug");};
destination remote_messages { file("/var/log/remote.d/$HOST/messages"); };
destination remote_grsec { file("/var/log/remote.d/$HOST/grsec.log"); };
destination remote_pax { file("/var/log/remote.d/$HOST/pax.log"); };

   Next we want to setup log filters, which tell syslog what data goes to what file.
   
   
#connect filter and destination
log { source(remote_log); filter(f_authpriv); destination(remote_authlog); };
log { source(remote_log); filter(f_syslog); destination(remote_syslog); };
log { source(remote_log); filter(f_cron); destination(remote_cron); };
log { source(remote_log); filter(f_daemon); destination(remote_daemon); };
log { source(remote_log); filter(f_kern); destination(remote_kern); };
log { source(remote_log); filter(f_lpr); destination(remote_lpr); };
log { source(remote_log); filter(f_mail); destination(remote_mail); };
log { source(remote_log); filter(f_user); destination(remote_user); };
log { source(remote_log); filter(f_mail); filter(f_info); destination(remote_mailinfo); };
log { source(remote_log); filter(f_mail); filter(f_warn); destination(remote_mailwarn); };
log { source(remote_log); filter(f_mail); filter(f_err); destination(remote_mailerr); };
log { source(remote_log); filter(f_debug); destination(remote_debug); };
log { source(remote_log); filter(f_messages); destination(remote_messages);};
log { source(remote_log); filter(f_pax); destination(remote_pax);};
log { source(remote_log); filter(f_grsec); destination(remote_grsec);};

4. Save your file. you're done with it.
5. Make the log directory for remote machines. with the command:
   mkdir /var/log/remote.d
   
6. restart syslog-ng: /etc/init.d/syslog-ng restart

Remote Setup

1. Setting up the remote machines is way easier. Again, you will need syslog-ng on these machines too. Open up /etc/syslog-ng/syslog-ng.conf with your favorite editor.
2. Add the following lines at the bottom of the config:
   
destination remote {
udp("10.1.1.2" port(1999));
};
log {source(src);destination(remote);};

   You will need to change the IP address '10.1.1.2' to the real address of your log server.
   
3. Save the file, you're done with it.
4. Restart syslog-ng: /etc/init.d/syslog-ng restart
   
5. Repeat these steps on all machines you want to report back to your log server.
   

It might take a few minutes, but you should soon see logs appearing in /var/log/remote.d/hostname from your remote machines.

Optional Steps

Keep syslog-ng.conf from being accidently (or intentionally) overwritten:
chattr +i /etc/syslog-ng/syslog-ng.conf

Install a GUI log parser to make reading your logs easier. A program such as ksystemlog if you're a KDE fan.

Additional Notes
there is a plethora of other options, including encryption if you plan on remote-logging over the internet. I intentionally left this information out, because this howto is geared more to home-networks. If you want to setup remote-logging over the internet, I highly recommend you find a howto that includes SSL options :)

This howto was adapted from articles listed at:
http://www.campin.net/syslog-ng/faq.html
http://gentoo-wiki.com/HOWTO_create_a_logserver_with_syslog-ng