Logs for SOC1 Analysis

/ CyberSecurity / By

It’s essential to review and analyze logs, not only to identify errors but also to ensure the system, installed applications, and services are secure and functioning correctly. Logs can be monitored manually through the console or automatically with third-party applications.

Logs Overview

Today, we’ll cover where to find logs, highlight the most crucial ones, and demonstrate how to view them in the console. Additionally, we’ll guide you on setting up Logwatch for daily email summaries of log activities.

Linux log files are generally categorized into four types:

  • Application Logs
  • Event Logs
  • Service Logs
  • System Logs

Monitoring and analyzing these can be complex due to their volume and diversity.

Viewing Logs in the Console

You can display all log files in the console using one of the following commands:

  • more: Views text files in the command prompt, displaying one screen at a time for large files. It allows scrolling through the content.
  • less: Similar to more, but faster because it doesn’t load the entire file at once and supports page navigation.
  • cat (catenate): Reads data from files and outputs their contents, useful for displaying a file’s content at the command line.
  • grep (global regular expression print): Processes text line by line, printing lines that match a specified pattern.
  • tail: Shows the last part of files and can monitor real-time updates to the file.

These commands can be combined for various purposes. For example:

Important Log Files to Monitor

  1. System Logs
    Stores informational and non-critical system messages, useful for tracking non-kernel boot errors and startup messages.
  2. Authentication
    Logs all authentication-related events on Debian and Ubuntu servers, useful for investigating security concerns like failed login attempts.
  3. Boot
    Contains booting related information and messages logged during the startup process.
  4. Kernel
    Ideal for troubleshooting kernel-related errors and warnings.
  5. Mail
    Stores mail server-related logs.
  6. Database
    Logs all MySQL/MariaDB related messages, including debug, failure, and success notices.
  7. otherLogs
    Additional log files may be present depending on installed applications

Logwatch Setup

Logwatch simplifies log management by analyzing and reporting daily digests of your machine’s activities.

Installation

Install from the repository or download the latest version from SourceForge:

sudo apt install logwatch

Configuration

Copy the default configuration to prevent overwrites during updates:

sudo cp /usr/share/logwatch/default.conf/logwatch.conf /etc/logwatch/conf/logwatch.conf

Edit the configuration file as needed, paying attention to the output format, email settings, report range, detail level, and services to monitor.

Generating Reports

Test the setup by generating a report:

sudo logwatch --output stdout --detail med --format text

By default, Logwatch sends daily reports via cron. Adjustments can be made to the cron settings if necessary.

Logwatch Configuration and Usage

After installing Logwatch, you’ll need to personalize its configuration to suit your monitoring needs. Here’s how to do it:

Configuration Steps

  1. Create a Configuration Copy: To prevent your custom settings from being overwritten by updates, first copy the default configuration file:
   sudo cp /usr/share/logwatch/default.conf/logwatch.conf /etc/logwatch/conf/logwatch.conf
  1. Edit the Configuration: Open the copied configuration file in an editor, like nano:
   sudo nano /etc/logwatch/conf/logwatch.conf
  1. Customize the Settings: Adjust the following settings within the file to match your requirements:
  • Output: Choose between mail (to send reports via email) or stdout (to display on the screen).
  • Format: Select html for a web page format or text for plain text emails.
  • MailTo: Enter your email address to receive the reports.
  • MailFrom: Specify the sender email address, which could also be your own.
  • Range: Set this to yesterday to receive reports about the previous day’s logs, or choose All for all available logs, and Today for the current day.
  • Detail: Choose the detail level of the reports (Low, Medium, or High).
  • Service: Decide which services you want to monitor. By default, all are included. You can specify individual services by modifying this setting.

Example Service Configuration

For specifying individual services, your configuration might look like this:

Service = sendmail
Service = http
Service = identd
Service = sshd2
Service = sudo

To exclude daily reports, you can deactivate them by uncommenting:

DailyReport = No

Preparing for Reports

  • Create a Cache Directory: Logwatch requires a cache folder, which you can create with:
  sudo mkdir /var/cache/logwatch
  • Test Run: Generate a test report to ensure everything is set up correctly:
  sudo logwatch --output stdout --detail med --format text

Automating Reports

By default, Logwatch is scheduled to send reports daily via a cron job located at /etc/cron.daily/00logwatch. If you wish to adjust the timing, remove this file and add a new cron job. For example, to receive reports at 6:00 AM:

0 6 * * * root /usr/sbin/logwatch --output mail

Final Notes

With these configurations, Logwatch will send you daily summaries of log activities, tailored to your specified detail level and focusing on the services you’re most interested in. This ensures you’re well-informed about the operational health and security posture of your systems.

Conclusion

Following these steps, you can effectively monitor and analyze log reports through your email, ensuring your system’s integrity and performance.