Intermittent 404 on select assets, LAMP stack
- by Tom Lagier
We have a LAMP stack WordPress server that is serving most assets correctly. However, one plugin's CSS file and several images are returning soft 404s roughly 20% of the time. I can't find any reference to the 404 in the access logs, but the browser is definitely receiving a 404 response from somewhere (WordPress, I would assume).
When I use an alias URL that does not match the site URL but does resolve to the asset path, the resource loads correctly 100% of the time. However, using the site url only resolves for the select, problematic assets 20% of the time.
You can test one of the problematic assets here: http://www.mreco.org/wp-content/uploads/2014/05/zero-cost.jpg
However the alias link always resolves correctly: http://mr-eco.wordpress.promocampaigns.com/wp-content/uploads/2014/05/zero-cost.jpg
Stranger, if I attempt to access outdated content that definitely does not exist on the server, at the live URL it returns the content roughly 50% of the time. Using the alias link, it 404s 100% of the time - the correct behavior.
Error log and PHP error log are clean.
A sample access log (pulled from grep 'zero-cost.jpg' /var/log/httpd/mr-eco-access_log) from several refreshes of the live direct link (where I am not seeing any 404's):
10.166.202.202 - - [28/May/2014:20:27:41 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 304 -
10.166.202.202 - - [28/May/2014:20:27:42 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 304 -
10.166.202.202 - - [28/May/2014:20:27:43 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 304 -
10.166.202.202 - - [28/May/2014:20:27:43 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 304 -
10.176.201.37 - - [28/May/2014:20:27:56 +0000] "GET /wp-content/uploads/2014/05/zero-cost.jpg HTTP/1.1" 200 57027
Chrome's dev tools list the following network activity before displaying 404 page content:
zero-cost.jpg /wp-content/uploads/2014/05 GET 404 Not Found text/html Other 15.9?KB 73.2?KB 953?ms 947?ms
My Apache configuration is standard, I've listed the virtual host entry and .htaccess file below. I can provide other parts of Apache config if necessary.
Virtual host:
<VirtualHost *:80>
DocumentRoot /var/www/public_html/mr-eco.wordpress.promocampaigns.com
ServerName www.mreco.org
ServerAlias mreco.org mr-eco.wordpress.promocampaigns.com
ErrorLog logs/mr-eco-error_log
CustomLog logs/mr-eco-access_log common
<Directory /var/www/public_html/mr-eco.wordpress.promocampaigns.com>
AllowOverride All
SetOutputFilter DEFLATE
</Directory>
</VirtualHost>
.htaccess:
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress
I have checked for multiple A records and can confirm that there is a single A record pointing at the domain:
;; ANSWER SECTION:
mreco.org. 60 IN A 50.18.58.174
I'm fairly new to systems administration, and at a complete loss as to what could cause this. In the past, inconsistently 404ing assets have been because of out-of-sync instances behind a load balancer. In this case, it is a single instance behind the load balancer.
Because of the inconsistency, it feels like a caching issue. We don't make use of Apache caching, and as far as I know WordPress should not be caching either.
What I've done so far:
Reset WordPress permalinks
Disabled WordPress plugins
Re-generated WordPress .htaccess file
Swapped ServerName and ServerAlias directives
Cleared browser cache
Confirmed disk location of resources
Checked PHP, access, and error logs
Confirmed correct DNS setup (can post if necessary)
I'm at a total loss. Thanks for helping me out!
