People often ask me how I like to setup OSSEC or how I use it internally on my own servers. I always do a set of customizations to make sure I use it the best way possible.
In this article I will show step by step those steps and hopefully it can be helpful to other OSSEC users out there.
Note that I will be focusing on a standalone install, but the principles apply to the agent/manager setup as well.
Installing OSSEC is pretty simple, so I won’t spend too much time on it. I always like to use the latest package from here:
$ wget https://dcid.me/ossec-packages/ossec-hids-latest.tar.gz
$ tar -zxvf ossec-hids-latest.tar.gz
$ cd ossec-hids*
# ./install.sh
If you run into issues, check if make, gcc and libc are installed:
# yum install gcc make libc-dev (for Redhat/fedora/centos)
# apt-get install gcc make libc-dev
Once OSSEC is installed, we can start tunning it to the server. The first thing I do is find what is running and what type of logs are available.
I generally use lsof to list what is there:
# lsof | grep log
httpd 30764 apache 19w REG 202,0 5856044 2088893 /var/log/httpd/site4.access.log
httpd 30764 apache 20w REG 202,0 413775 2088897 /var/log/httpd/site3.access.log
httpd 32200 apache 2w REG 202,0 2894 2088895 /var/log/httpd/error_log
httpd 32200 apache 7w REG 202,0 2894 2088895 /var/log/httpd/error_log
httpd 32200 apache 8w REG 202,0 11293853 2088903 /var/log/httpd/site2.error.log
syslogd 2220 root 2w REG 202,0 317 121533 /var/log/messages
syslogd 2220 root 3w REG 202,0 3474 121774 /var/log/secure
syslogd 2220 root 4w REG 202,0 147252 121855 /var/log/maillog
syslogd 2220 root 5w REG 202,0 46954 121875 /var/log/cron
syslogd 2220 root 6w REG 202,0 0 121873 /var/log/spooler
syslogd 2220 root 7w REG 202,0 0 121874 /var/log/boot.log
mysqld 2359 mysql 1w REG 202,0 20162 121188 /var/log/mysqld.log
mysqld 2359 mysql 2w REG 202,0 20162 121188 /var/log/mysqld.log
.. and many more...
And I compare with what OSSEC added automatically
# cat /var/ossec/etc/ossec.conf |grep "<location>/"
<location>/var/log/auth.log</location>
<location>/var/log/syslog</location>
<location>/var/log/dpkg.log</location>
The logs that are missing I add manually using the util.sh tool that comes with OSSEC:
# /var/ossec/bin/util.sh addfile /var/log/httpd/site4.access.log
/var/ossec/bin/util.sh: File /var/log/httpd/site4.access.log added.
# /var/ossec/bin/util.sh addfile /var/log/mysqld.log
/var/ossec/bin/util.sh: File /var/log/mysqld.log added.
.. for all others ..
I also check using netstat -tanep to see what is running on the server to see if we might have missed something:
# netstat -tanep |grep LISTEN
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 27 4659 2359/mysqld
tcp 0 0 :::22 0.0.0.0:* LISTEN 0 4561 2278/sshd
tcp 0 0 :::80 :::* LISTEN 0 4793 2411/httpd
Before even putting into production, I like to test if OSSEC is able to parse all the logs properly.
For that, I use the tool ossec-logtest with the -a option to analyse old events and compare with a manual audit of the logs.
On my desktop, if I run it against the /var/log/syslog I get:
# cat /var/log/syslog | /var/ossec/bin/ossec-logtest -a
2012/03/29 17:49:48 ossec-testrule: INFO: Reading local decoder file.
2012/03/29 17:49:48 ossec-testrule: INFO: Started (pid: 7440).
** Alert 1333054188.1: mail - syslog,errors,
2012 Mar 29 17:49:48 goiabada->stdin
Rule: 1002 (level 2) -> 'Unknown problem somewhere in the system.'
Mar 29 10:11:04 goiabada kernel: [604548.729517] chrome[11541]: segfault at 60 ip b3d27f27 sp bf940a50
error 4 in libgtk-x11-2.0.so.0.2400.6[b3c34000+461000]
** Alert 1333054188.2: mail - syslog,errors,
2012 Mar 29 17:49:48 goiabada->stdin
Rule: 1002 (level 2) -> 'Unknown problem somewhere in the system.'
Mar 29 10:29:06 goiabada kernel: [605630.394325] chrome[20438]: segfault at cc ip b37a6d87 sp bfa6e024
error 4 in libpthread-2.13.so[b379e000+17000]
I can also run without the -a to see if it is properly decoding the logs:
I use the command monitoring on OSSEC to be able to track state changes and to get a full picture of all my agents.
Those are the commands I like to monitor:
<localfile>
<log_format>full_command</log_format>
<command>/sbin/iptables -nL</command>
</localfile>
<localfile>
<log_format>full_command</log_format>
<command>netstat -tan |grep LISTEN |grep -v 127.0.0.1 | sort</command>
</localfile>
<localfile>
<log_format>full_command</log_format>
<command>last -n 5</command>
</localfile>
I specially like the last, netstat, df, iptables and similar commands since they allow me to view the current state of the system.
I can go to the queue directory and see the state from all the systems, like who logged in on all of them:
# cat /var/ossec/queue/diff/*/535/last-entry
ossec: output: 'last -n 5':
root pts/0 1.2.3.4 Mon Jun 4 21:00 - 03:04 (06:04)
root pts/0 2.3.4.5 Mon May 7 20:59 - 20:46 (4+23:46)
daniel pts/1 2.3.3.4 Fri May 4 22:34 - 22:34 (00:00)
Integrity checking is one of the most misused options on OSSEC. It can range from very annoying to completely useless if you don’t do it properly. The default options from OSSEC don’t help much with that as well.
The real goal of integrity checking is to verify that the integrity of the files are still intact (they haven’t changed). However, if a file changes many times per week, it becomes very hard to distinguish between a normal and a malicious change.
How I use integrity checking:
1- On real time
2- Only on a few set of files
3- Storing the changes so I can compare if needed
The real time option is specially important, because I can tie it with my policy rules to alert when they happen on times it shouldn’t.
So, on my web servers, I add the following in there:
<directories realtime="yes" report_changes="yes" restrict=".php|.js">/var/www</directories>
Which restricts the integrity checking to the files I care (generally .php, .js, .htm, etc) and monitors those in real time. So when a file changes I get the alert showing what was modified as well.
This is not a finished document and I will keep updating it as I have time. Any questions, just email me dcid@dcid.me.