Log like a paranoid Lumberjack!
Ok, so maybe I’m a touch paranoid, but I like logging. I also like monitoring, and statistics. I like to know what’s going on, when and how. I don’t mind a little noise, as long as I can quickly assess what’s happening with my servers.
I had, until recently, two forms of monitoring going on. I previously set up Zabbix (after getting frustrated with Nagios) and I had LogicMonitor as well. Unfortunately, Zabbix was starting to overload the server that it was running on, and I didn’t want to move it. So I switched exclusively to LogicMonitor, and I’ve been happy since.
When I set up my new systems, I set up centralized logging via rSyslog. I chose rSyslog because It allowed me to use standard Syslog on other systems, so I didn’t have to install anything special on my other hosts. It also allowed me to log into MySQL. Unfortunately, I didn’t do the smart thing and set up MySQL partitioning, so that data got unwieldy. I also had no reliable front end or analyzer that could take advantage of MySQL, so I did away with that component.
I do administer Red Hat machines professionally, but not personally. One thing I like about Red Hat machines is LogWatch. It has succinct log summaries with some analysis and is nice to read. It is analogous to the FreeBSD system and security log summaries, which I also like.
Still, LogWatch wasn’t enough for me. Enter OSSEC from TrendMicro. It has a lot of security functionality, but the one that’s most useful to me is the log analysis. It tells me about possible security issues, system errors, and miscellaneous issues. It runs on my centralized log host, so I only need to install and maintain it once. It can be a little noisy by default, but it often tells me important things that I should know. It has notified me of misconfiguration, Apache eating all available memory on hosts, processes that have run amok. All easy things to fix, and logged, but hard to find amongst the tons of data on my multiple hosts. Fortunately, no security issues have come up, but I’m sure I’ll get a good idea that something is wrong should there be one. Remember, your servers and firewalls are *always* under attack.
It’s good to have a tool that can help put my paranoid mind at ease a little. Some of the default alerts are a little… verbose. I love getting the “multiple attempts from a blacklisted host” when a server on an RBL does a dictionary attack against one of my spam-trap domains. That alert is fairly reasonable, only firing off after a few attempts.
Now, to get an IPS system working…