This command will search for the string «ErrorLog» (case-insensitive) in the file /etc/httpd/conf/httpd.conf and output the line containing the path to the error log file.

How to Check Apache Error Logs on a CentOS VDS?

Setting up and maintaining an Apache web server on a CentOS VDS requires constant monitoring. Apache error logs are your main source of information for identifying and troubleshooting issues affecting website performance. In this article, we will take a detailed look at how to effectively check and analyze Apache error logs on your CentOS VDS to ensure the stable and uninterrupted operation of your web applications.

We will cover the main log files, tools for analyzing them, as well as methods for filtering and interpreting the information contained in the logs. By following these instructions, you will be able to quickly diagnose problems, prevent failures, and maintain optimal performance of your web server.

Main Apache Log Files

Apache records information about its operation in several log files. The most important ones are the error log and the access log. Let’s examine each of them in more detail, and also consider where they are located by default on a CentOS VDS.

Error log (error.log): This file contains records of all errors, warnings, and other important events related to Apache’s operation. It is the primary source of information for diagnosing problems.

Access log (access.log): This file contains records of all requests that have been processed by the Apache server. It can be useful for analyzing traffic, identifying bots, and other monitoring purposes.

Default location of log files: On CentOS, the standard location for Apache log files is usually as follows:

  • /var/log/httpd/error_log — Error Log
  • /var/log/httpd/access_log — Access Log
However, this location can be changed in the Apache configuration file. Let’s look at how to find the actual location of the log files.

Finding the Location of Log Files in Apache Configuration

To find the exact location of the log files, you need to view the Apache configuration file. It is usually located at /etc/httpd/conf/httpd.conf or in files located in the /etc/httpd/conf.d/ directory.

Example 1: Searching for ErrorLog in httpd.conf 

grep -i "ErrorLog" /etc/httpd/conf/httpd.conf
This command will search for the string «ErrorLog» (case-insensitive) in the file /etc/httpd/conf/httpd.conf and output the line containing the path to the error log file.

Example result: 

ErrorLog "logs/error_log"
In this case, the path to the error log file is relative to the ServerRoot directive, which is also defined in the configuration file. Usually, ServerRoot points to /etc/httpd. Therefore, the actual path to the file will be /etc/httpd/logs/error_log.

Example 2: Searching for CustomLog for AccessLog 

grep -i "CustomLog" /etc/httpd/conf/httpd.conf
This command will search for the string «CustomLog» (case-insensitive) in the file /etc/httpd/conf/httpd.conf and output the line containing the path to the access log file.

Example result: 

CustomLog "logs/access_log" combined
Similarly, the path to the access log file is relative to the ServerRoot directive. In this case, the actual path to the file will be /etc/httpd/logs/access_log.

Example 3: Checking Configuration Files in conf.d 

grep -i -r "ErrorLog" /etc/httpd/conf.d/
This command will recursively search for the string «ErrorLog» (case-insensitive) in all files in the /etc/httpd/conf.d/ directory. This is useful if the Apache configuration is split into several files.

Once you have determined the location of the log files, you can proceed to check and analyze them.

Viewing and Analyzing Logs Using the Command Line

The command line provides powerful tools for viewing and analyzing Apache logs. Let’s consider some of the most useful commands.

Basic Commands for Viewing Logs

tail: Shows the last lines of a file. This is useful for viewing new errors in real time.

cat: Outputs the content of the entire file. Used to view the entire log, but may be impractical for large files.

less: Opens the file for viewing page by page. Allows you to easily navigate through the file and search for text.

grep: Searches for lines that match a given pattern. Used to filter logs by keywords or regular expressions.

Example 1: Viewing the last 100 lines of the error log 

tail -n 100 /var/log/httpd/error_log
This command will output the last 100 lines of the /var/log/httpd/error_log file. You can replace 100 with any other number to change the number of displayed lines.

Example 2: Filtering the error log by the keyword «PHP» 

grep "PHP" /var/log/httpd/error_log
This command will output all lines from the /var/log/httpd/error_log file that contain the word «PHP». This is useful for finding errors related to PHP scripts.

Example 3: Viewing the error log in real time and filtering by IP address 

tail -f /var/log/httpd/error_log | grep "192.168.1.100"
This command will start viewing the /var/log/httpd/error_log file in real time (tail -f) and filter the output, showing only lines containing the IP address 192.168.1.100. This can be useful for tracking errors related to a specific user or host.

Example 4: Counting the number of errors of a specific type 

grep "File not found" /var/log/httpd/error_log | wc -l
This command will count the number of lines in the /var/log/httpd/error_log file that contain the phrase «File not found». wc -l is used to count the number of lines. This can help determine the frequency of certain errors.

Expert tip: Use regular expressions with grep for more complex searches. For example, to search for all 404 errors, you can use the command grep "404" /var/log/httpd/access_log.

Analyzing Access Logs

The access log contains information about each request to the server. You can use it to analyze traffic, identify performance issues, and detect suspicious activity.

Example 5: Viewing the most frequently requested pages 

awk '{print $7}' /var/log/httpd/access_log | sort | uniq -c | sort -nr | head -n 10
This command does the following:

  • awk '{print $7}': Extracts the seventh column from each log line (the request URL).
  • sort: Sorts the list of URLs.
  • uniq -c: Counts the number of identical URLs.
  • sort -nr: Sorts the results in descending order of the number of requests.
  • head -n 10: Outputs the 10 most frequently requested URLs.
This allows you to quickly determine which pages are most popular on your site.

Example 6: Analyzing HTTP response codes 

awk '{print $9}' /var/log/httpd/access_log | sort | uniq -c | sort -nr
This command is similar to the previous one, but extracts the ninth column (the HTTP response code) from each log line. This allows you to see the distribution of response codes (200 OK, 404 Not Found, 500 Internal Server Error, etc.). Analyzing response codes helps identify problems with page availability or server errors.

Example 7: Identifying IP addresses generating the most requests 

awk '{print $1}' /var/log/httpd/access_log | sort | uniq -c | sort -nr | head -n 10
This command extracts the first column (the client IP address) from each log line and counts the number of requests from each IP address. This can help identify bots or users abusing server resources.

Using Log Analysis Tools

Although the command line provides powerful tools for log analysis, there are specialized tools that automate many tasks and provide a more convenient interface. Let’s consider two popular tools: GoAccess and AWStats.

GoAccess

GoAccess is a fast and flexible log analyzer that can generate interactive reports in real-time in the terminal or in HTML format.

Installing GoAccess on CentOS: 

yum install goaccess
Analyzing the access log with GoAccess: 

goaccess /var/log/httpd/access_log
This command will start GoAccess and open an interactive report in the terminal. You can navigate the report using the cursor control keys.

Creating an HTML report: 

goaccess /var/log/httpd/access_log -o report.html
This command will create an HTML file with the report (report.html), which can be opened in a browser.

Example GoAccess configuration (goaccess.conf): 

time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h %l %u %t "%r" %s %b "%R" "%U"

# Ignore certain IP addresses
ignore-ip 192.168.1.100
ignore-ip 10.0.0.5
This configuration file allows you to configure the log format and ignore certain IP addresses during analysis. Save it to a file named goaccess.conf and run GoAccess with the -f option: 

goaccess -f goaccess.conf /var/log/httpd/access_log -o report.html
Key features of GoAccess:

  • Interactive real-time report
  • Support for various log formats
  • HTML report generation
  • Filtering by IP addresses, URLs, HTTP codes, and other parameters

AWStats

AWStats is a powerful and free log analyzer that generates detailed statistical reports in HTML format. It requires preliminary configuration and periodic updating of the statistics database.

Installing AWStats on CentOS: 

yum install awstats
Configuring AWStats:

1. Create a configuration file for your site (for example, /etc/awstats/awstats.yoursite.com.conf) based on the /etc/awstats/awstats.model.conf template. 2. Edit the configuration file and specify the path to the access log file: 

LogFile="/var/log/httpd/access_log"
SiteDomain="yoursite.com"
HostAliases="www.yoursite.com localhost 127.0.0.1"
3. Update the statistics database: 

/usr/lib64/awstats/awstats.pl -config=yoursite.com -update
4. Configure the web server for access to AWStats reports (for example, create a virtual host or alias):

Example Apache virtual host configuration for AWStats: 

<VirtualHost *:80>
    ServerName awstats.yoursite.com
    DocumentRoot /usr/share/awstats/wwwroot

    <Directory /usr/share/awstats/wwwroot>
        Options FollowSymLinks
        AllowOverride None
        Require all granted
    </Directory>

    Alias /awstats-icon/ /usr/share/awstats/wwwroot/icon/
</VirtualHost>
Key features of AWStats:

  • Detailed website traffic statistics
  • Search query analysis
  • Analysis of bots and search engine crawlers
  • Geographic distribution of visitors
  • 404 error statistics
Comparison of GoAccess and AWStats:

FunctionGoAccessAWStats
Operating ModeInteractive in real timeHTML report generation (requires database updates)
Configuration ComplexitySimpleMore Complex
Resource IntensityLowMedium
FunctionalityBasic analysis functionsWider range of functions
Real TimeYesNo (requires periodic updates)
The choice between GoAccess and AWStats depends on your needs and preferences. GoAccess is an excellent choice for quick real-time log analysis, while AWStats is for obtaining detailed statistical information about website traffic.

Interpreting Common Errors

Understanding common errors logged by Apache greatly simplifies the debugging process. Let’s look at the most common errors and how to fix them.

404 Not Found

A 404 Not Found error means that the requested resource (file, page, image, etc.) was not found on the server.

Possible causes:

  • Incorrect URL in the request
  • The resource has been deleted or moved
  • Incorrect virtual host settings
  • Problems with file access permissions
Example entry in the access log: 

192.168.1.100 - - [20/04/2024:10:00:00 +0000] "GET /nonexistent_page.html HTTP/1.1" 404 200 "-" "Mozilla/5.0"
Troubleshooting:

  • Check that the URL in the request is correct.
  • Make sure the resource exists and is accessible.
  • Check the virtual host settings.
  • Check file access permissions.
  • Use redirects (Redirect or Alias) for moved resources.

500 Internal Server Error

A 500 Internal Server Error means that an unexpected error occurred on the server that prevents the request from being processed.

Possible causes:

  • Errors in PHP scripts (syntax errors, runtime errors)
  • Problems with Apache configuration
  • Lack of server resources (memory, CPU time)
  • Problems with file access permissions
  • Error in .htaccess
Example entry in the error log: 

[Fri Apr 20 10:00:00.000000 2024] [php7:error] [pid 12345] [client 192.168.1.100:50000] PHP Fatal error:  Call to undefined function nonexistent_function() in /var/www/html/index.php on line 10
Troubleshooting:

  • Check the PHP error logs for syntax errors or runtime errors.
  • Make sure all required PHP modules are installed and enabled.
  • Check the Apache configuration for errors.
  • Increase server resource limits (memory, CPU time).
  • Check file access permissions.
  • Check the .htaccess file for syntax errors or incorrect directives.

403 Forbidden

A 403 Forbidden error means that the server has denied access to the requested resource.

Possible causes:

  • Insufficient access permissions for the file or directory.
  • Directory access denied in the Apache configuration (Require directive).
  • Incorrect .htaccess settings.
  • Access denied by IP address.
Example entry in the error log: 

[Fri Apr 20 10:00:00.000000 2024] [access_compat:error] [pid 12345] [client 192.168.1.100:50000] AH01797: client denied by server configuration: /var/www/html/secret_file.txt
Troubleshooting:

  • Check file or directory access permissions (read and execute permissions should be set for the web server).
  • Check the Apache configuration for Require directives that restrict access to the resource.
  • Check the .htaccess file for errors or incorrect directives.
  • Make sure the client IP address is not blocked in the Apache configuration or firewall.

PHP Warning/Notice

PHP Warnings and Notices are warnings and notifications that indicate potential problems in PHP scripts. They do not cause the script to terminate immediately, but may indicate errors that could cause problems in the future.

Possible causes:

  • Using uninitialized variables.
  • Incorrect use of PHP functions.
  • Outdated code.
  • Incompatibility between PHP versions and used libraries.
Example entry in the error log: 

[Fri Apr 20 10:00:00.000000 2024] [php7:warn] [pid 12345] [client 192.168.1.100:50000] PHP Warning:  Undefined variable: username in /var/www/html/index.php on line 5
Troubleshooting:

  • Check the PHP code for uninitialized variables and other potential problems.
  • Use the correct PHP functions and follow the recommendations for their use.
  • Update the code to the current version.
  • Make sure that all libraries and PHP modules are compatible with the PHP version being used.
  • Enable PHP error reporting in development mode for more detailed error information.

Configuring Logging to Improve Debugging

Configuring Apache logging parameters can greatly simplify the process of debugging and identifying problems. Let’s consider which parameters can be changed and how to do it.

Changing the Logging Level

The LogLevel directive defines which types of messages will be written to the error log. By default, the warn or error level is usually used. For more detailed debugging, you can set the debug level, but this will significantly increase the size of the log.

Available logging levels (from least to most detailed):

  • emerg: Emergency situations requiring immediate intervention.
  • alert: Warnings about serious problems.
  • crit: Critical errors.
  • error: Errors.
  • warn: Warnings.
  • notice: Notifications.
  • info: Informational messages.
  • debug: Debug messages.
  • trace1, trace2, …, trace8: Most detailed debug messages (available when the mod_log_config module is enabled).
Example of changing the logging level in httpd.conf: 

LogLevel debug
After changing the configuration file, you need to restart Apache: 

systemctl restart httpd
Attention! Using the debug level in a production environment can quickly fill up disk space. It is recommended to use this level only temporarily, during debugging.

Configuring the Access Log Format

The CustomLog directive defines the format of entries in the access log. You can configure the log format to include additional information, such as request processing time, User-Agent, Referer, etc.

Example of changing the access log format: 

LogFormat "%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-Agent}i\" %D" combined_with_time
CustomLog "logs/access_log" combined_with_time
In this example, we defined a new log format combined_with_time, which includes the request processing time (%D) in microseconds, and use it for the access log. %{Referer}i and %{User-Agent}i add information about the Referer and User-Agent to the log.

Most useful parameters for the log format:

  • %h: Client IP address.
  • %l: Client name (not recommended as it requires reverse DNS lookup).
  • %u: User name (if authentication is required).
  • %t: Request time and date.
  • %r: Request string (method, URL, protocol).
  • %s: HTTP response code.
  • %b: Response size in bytes.
  • %{Referer}i: Referer.
  • %{User-Agent}i: User-Agent.
  • %D: Request processing time in microseconds.
  • %T: Request processing time in seconds.
Expert tip: Use the mod_rewrite directive for logging POST requests, which is especially useful for debugging forms and APIs.

Log Rotation