Intermittent 404 on select assets, LAMP stack

Posted by Tom Lagier on Server Fault See other posts from Server Fault or by Tom Lagier
Published on 2014-05-28T20:48:34Z Indexed on 2014/05/28 21:31 UTC
Read the original article Hit count: 426

We have a LAMP stack WordPress server that is serving most assets correctly. However, one plugin's CSS file and several images are returning soft 404s roughly 20% of the time. I can't find any reference to the 404 in the access logs, but the browser is definitely receiving a 404 response from somewhere (WordPress, I would assume).

When I use an alias URL that does not match the site URL but does resolve to the asset path, the resource loads correctly 100% of the time. However, using the site url only resolves for the select, problematic assets 20% of the time.

You can test one of the problematic assets here: http://www.mreco.org/wp-content/uploads/2014/05/zero-cost.jpg

However the alias link always resolves correctly: http://mr-eco.wordpress.promocampaigns.com/wp-content/uploads/2014/05/zero-cost.jpg

Stranger, if I attempt to access outdated content that definitely does not exist on the server, at the live URL it returns the content roughly 50% of the time. Using the alias link, it 404s 100% of the time - the correct behavior.

Error log and PHP error log are clean.

A sample access log (pulled from grep 'zero-cost.jpg' /var/log/httpd/mr-eco-access_log) from several refreshes of the live direct link (where I am not seeing any 404's):

10.166.202.202 - - [28/May/2014:20:27:41 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 304 -
10.166.202.202 - - [28/May/2014:20:27:42 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 304 -
10.166.202.202 - - [28/May/2014:20:27:43 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 304 -
10.166.202.202 - - [28/May/2014:20:27:43 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 304 -
10.176.201.37 - - [28/May/2014:20:27:56 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 200 57027

Chrome's dev tools list the following network activity before displaying 404 page content:

zero-cost.jpg /wp-content/uploads/2014/05 GET 404 Not Found text/html Other 15.9?KB 73.2?KB 953?ms 947?ms

My Apache configuration is standard, I've listed the virtual host entry and .htaccess file below. I can provide other parts of Apache config if necessary.

Virtual host:

<VirtualHost *:80>
    DocumentRoot /var/www/public_html/mr-eco.wordpress.promocampaigns.com
    ServerName www.mreco.org
    ServerAlias mreco.org mr-eco.wordpress.promocampaigns.com
    ErrorLog logs/mr-eco-error_log
    CustomLog logs/mr-eco-access_log common
    <Directory /var/www/public_html/mr-eco.wordpress.promocampaigns.com>
        AllowOverride All
        SetOutputFilter DEFLATE
    </Directory>
</VirtualHost>

.htaccess:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress

I have checked for multiple A records and can confirm that there is a single A record pointing at the domain:

;; ANSWER SECTION:
mreco.org.      60  IN  A   50.18.58.174

I'm fairly new to systems administration, and at a complete loss as to what could cause this. In the past, inconsistently 404ing assets have been because of out-of-sync instances behind a load balancer. In this case, it is a single instance behind the load balancer.

Because of the inconsistency, it feels like a caching issue. We don't make use of Apache caching, and as far as I know WordPress should not be caching either.

What I've done so far:

  • Reset WordPress permalinks
  • Disabled WordPress plugins
  • Re-generated WordPress .htaccess file
  • Swapped ServerName and ServerAlias directives
  • Cleared browser cache
  • Confirmed disk location of resources
  • Checked PHP, access, and error logs
  • Confirmed correct DNS setup (can post if necessary)

I'm at a total loss. Thanks for helping me out!

© Server Fault or respective owner

Related posts about apache-2.2

Related posts about amazon-web-services