OSSEC, the free and open source IDS
Intrusion detection software is meant to monitor network traffic or host activities for malicious actions, such as successful or unsuccessful intrusion attempts, hostile traffic (i.e., malicious scans and denials of service), unauthorized configuration changes, malware symptoms, and user policy violations. An intrusion detection system (IDS) typically can produce reports describing the details of the potentially hazardous activity which generated alerts. OSSEC is particularly useful in this context for many reasons. First, it is an established, reputable product with a proven track record (OSSEC was first released in 2004 and has been owned by Trend Micro since 2009). Second, it is free and open source. Third, it is compatible with most modern operating systems such as Linux, Windows (Server 2008, Server 2003, 7, Vista, XP, 2000) BSD (Free/Open/Net), Unix (Solaris, HP-UX, AIX), and MacOS.
One of the key tenets of IT security is to keep intruders from gaining access to your organization’s network. Not only must the network’s edge be hardened to resist a myriad of attacks, but measures must be put into place to detect attackers who have successfully breached the perimeter. These two measures are important steps in achieving a “defense in depth” security posture, and OSSEC is an effective and affordable option to fulfill the IDS role.
Intrusion detection systems (IDS) are generally classified as either network-based or host-based in nature. A network-based IDS (NIDS) attempts to discover unauthorized access to a network by analyzing traffic as it flows between nodes for signs of malicious activity. A host-based IDS (HIDS), on the other hand, is designed to detect threats occurring on the hosts where they are installed (on servers, for example). A HIDS monitors local actions and attempts to identify those which could be hazardous. In this way a HIDS is similar to antivirus applications that identify and block certain attack patterns and raise alarms to alert users and administrators.
You may wonder how NIDS and HIDS are able to recognize when attacks are happening. In other words, how do they differentiate between hostile, prohibited actions on the one hand and normal, benign behavior on the other? Just as an antivirus application must be kept updated with the latest virus definitions, an IDS that is signature-based relies on signatures of known attack patterns to enable it to recognize threats. Alternatively, an anomaly-based IDS detects actions which occur outside a baseline of normal, expected behavior.
OSSEC is a HIDS that functions using both signature and anomaly detection (the book OSSEC HIDS Host Based Intrusion Guide states on page 161 that OSSEC’s “kernel-level checks do not use any signatures and instead rely on anomaly detection technology to look for rootkits”). OSSEC provides both host agent and file integrity agent (integrity checking) capabilities. It can also detect rootkits and perform log analysis. OSSEC can be deployed as a stand-alone agent or as part of a distributed network of agents with a central OSSEC server controlling their configurations and settings. In server mode, a central OSSEC server manages one or more remote OSSEC agents. These agents generate updates and status reports which are transmitted to the server. If any of these notifications are deemed suspicious by the server, it generates alerts.
OSSEC is only available as a server or stand-alone installation on Linux/BSD. You can install the OSSEC agent on Windows hosts to be monitored by OSSEC servers.
Let’s take a look at the OSSEC installation process on Linux. As usual, it is simplest and quickest to check if OSSEC is available in your Linux distribution’s software repositories by searching for it with your package management application (such as Synaptic). If it is not there, you can download it with the wget utility as follows:
Unpack the package, change into the resulting directory, and start the installation routine:
The first thing you need to select is your language; just hit Enter for English. The next screen advises that you must have a C compiler installed to proceed. Press Enter again.
The next screen prompts you for the desired type of installation:
Type ‘server’ if you want to set up an OSSEC server that will manage and monitor remote OSSEC agents on other hosts. Type ‘agent’ if you want to install the OSSEC agent that will be controlled by an OSSEC server. Type ‘local’ if you want to avoid an OSSEC client/server environment and just run OSSEC on a single host. Type ‘hybrid’ if you want to deploy an OSSEC server that also contains an agent (which answers to another OSSEC server).
The next step lets you specify the installation location; the default is /var/ossec. You can now specify the location you want or just accept the default and press Enter.
Next you will be asked to configure email notifications and specify the required email address and SMTP server. OSSEC uses email notifications to alert you regarding events which triggered alerts.
The next steps allow you to specify which components of OSSEC are enabled.
The integrity check daemon is responsible for monitoring and reporting changes in system files.
The rootkit detection engine regularly performs tests looking for signs of rootkits.
Log analysis is enabled by default. It automatically analyzes the contents of these log files and alerts on any anomalies detected.
OSSEC is now running and your host is being monitored for intrusions and anomalies. However, it is functioning in a default (untuned) state and should be tuned with custom settings for your environment. Some ways you can customize your installation of OSSEC include editing the rules and signatures to reflect the combination of applications and services running on your host, specifying additional logging sources and adjusting the criticality of alerts to reflect those issues most important to your environment (such as data/host criticality and sensitivity). Simply add or adjust the rules contained in the XML files in the
/var/ossec/rules directory (the rule format is explained in the OSSEC online user manual). New rules can be obtained from BitBucket.
OSSEC Agent for Windows
If you configure your Linux/BSD host as an OSSEC server and you want it to monitor Windows hosts, you need to install the OSSEC agent on them. These agents connect to the server through an encrypted connection on UDP port 1514 (adjust any firewall rules accordingly). The server and agents are authenticated using a symmetric key that is defined on the server and then copied to the agents.
The agent .exe installation file is also available on the OSSEC download page. After you install it, you must specify the IP address and authentication key of an OSSEC server.
To generate an authentication key on an OSSEC server, you use the manage_agents command as shown below.
Now you return to the manage_agents menu and this time select ‘E’.
You can confirm that the connection has succeeded by reviewing the contents of the
/var/ossec/logs/ossec.log file on the agent and server.
OSSEC Web User Interface
To manage the OSSEC server or local installation with a graphical user interface, you can download the Web user interface (WUI) and then follow these steps.
Extract the contents of the gzipped tar file:
Move and rename the OSSEC-WUI directory to the WWW directory which is accessible by Apache:
Change your working directory and start the setup routine:
The installation routine will prompt you as follows:
Now add the web server user account (such as apache or www-data) to the ossec group in
Then change the permissions on the OSSEC temporary directory:
Lastly, restart the Apache web daemon:
Ddpbsd.blogspot.com, Watching for Potentially Malicious Domains with OSSEC
Devio.us, OSSEC online manual
Github.com, OSSEC email abuse script
HackerTarget.com, Defending WordPress with OSSEC
HowToForge.com, Securing Your Server With OSSEC
Linuxdrops.com, AnaLogi web interface for OSSEC
Mousesecurity.com, Using OSSEC for File Integrity Monitoring
ReadTheDocs.org, OSSEC online manual
Rootshell.be, Multiple OSSEC articles
TAMU.edu, Protecting web servers with OSSEC
If you found the content of this article helpful and want to expand your knowledge further, please consider buying a relevant book using the links below. Thanks!