How mod_cache working with "must-revalidate" and "max-age"?
- by Dmitriy Sosunov
Quick question before I will explain my flow: ?an mod_cache perform revalidate with if-none-match only if max-age is expired in case if it configured in reverse proxy mode?
My goal is to reduce a number of revalidation requests to our the origin server.
For instance:
The first request goes to the origin server and then mod_cache save a response in to the cache according to header cache-control: max-age. And only when max-age is expired then mod_cache will revalidate with if-none-match.
Currently, mod_cache revalidate each request, regardless that max-age is defined or not.
My configuration of Apache 2.4.3 (Windows), on linux I see the same behavior that I will show below.
ServerName proxy.lo
ProxyRequests Off
ProxyPreserveHost Off
Header set Vary "Accept, Content-Type, Content-Encoding, Accept-Language"
RequestHeader set X-Forwarded-Proto "http"
# modify header for user agent's
Header set Cache-Control "private, no-cache, no-store, no-transform"
CacheQuickHandler off
CacheDefaultExpire 300
# the origin server do not provide last-modified
CacheIgnoreNoLastMod On
CacheIgnoreCacheControl On
# the origin server define cache-control: private, no-store only for user agents
# Therefore, I would like ignore those headers on the proxy server.
CacheStorePrivate On
CacheStoreNoStore On
CacheEnable disk /
CacheRoot "C:/Apache.Cache"
CacheDirLevels 5
CacheDirLength 4
CacheMinExpire 15
CacheDetailHeader on
CacheHeader on
KeepAlive Off
ProxyPass / http://origin.lo/
ProxyPassReverse / http://origin.lo/
Also, I have turned on debug log level to see how mod_cache handles a content for caching:
I provided this to show that mod_proxy always decides that a content isn't fresh. Why?I provided this to show that mod_proxy always decide that a content isn't fresh. Why? max-age was provided (see below).
[Sun Nov 04 11:58:42.899890 2012] [cache:debug] [pid 6492:tid 1400] cache_storage.c(624): [client 192.168.1.100:63741] AH00698: cache: Key for entity /testpage?(null) is http://proxy.lo/testpage?
[Sun Nov 04 11:58:42.899890 2012] [cache_disk:debug] [pid 6492:tid 1400] mod_cache_disk.c(569): [client 192.168.1.100:63741] AH00709: Recalled cached URL info header http://proxy.lo/testpage?
[Sun Nov 04 11:58:42.899890 2012] [cache_disk:debug] [pid 6492:tid 1400] mod_cache_disk.c(865): [client 192.168.1.100:63741] AH00720: Recalled headers for URL http://proxy.lo/testpage?
[Sun Nov 04 11:58:42.899890 2012] [cache:debug] [pid 6492:tid 1400] cache_storage.c(320): [client 192.168.1.100:63741] AH00695: Cached response for /testpage isn't fresh. Adding/replacing conditional request headers.
[Sun Nov 04 11:58:42.899890 2012] [cache:debug] [pid 6492:tid 1400] mod_cache.c(414): [client 192.168.1.100:63741] AH00757: Adding CACHE_SAVE filter for /testpage
[Sun Nov 04 11:58:42.899890 2012] [cache:debug] [pid 6492:tid 1400] mod_cache.c(448): [client 192.168.1.100:63741] AH00759: Adding CACHE_REMOVE_URL filter for /testpage
[Sun Nov 04 11:58:42.899890 2012] [proxy:debug] [pid 6492:tid 1400] mod_proxy.c(1068): [client 192.168.1.100:63741] AH01143: Running scheme http handler (attempt 0)
[Sun Nov 04 11:58:42.899890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(1976): AH00942: HTTP: has acquired connection for (origin.lo)
[Sun Nov 04 11:58:42.899890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(2029): [client 192.168.1.100:63741] AH00944: connecting http://origin.lo/testpage to origin.lo:80
[Sun Nov 04 11:58:42.901890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(2151): [client 192.168.1.100:63741] AH00947: connected /testpage to origin.lo:80
[Sun Nov 04 11:58:42.901890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(2554): AH00962: HTTP: connection complete to 192.168.1.100:80 (origin.lo)
[Sun Nov 04 11:58:42.903890 2012] [proxy:debug] [pid 6492:tid 1400] proxy_util.c(1991): AH00943: http: has released connection for (origin.lo)
[Sun Nov 04 11:58:42.903890 2012] [headers:debug] [pid 6492:tid 1400] mod_headers.c(800): AH01502: headers: ap_headers_output_filter()
[Sun Nov 04 11:58:42.903890 2012] [cache:debug] [pid 6492:tid 1400] mod_cache.c(1190): [client 192.168.1.100:63741] AH00769: cache: Caching url: /testpage
[Sun Nov 04 11:58:42.903890 2012] [cache:debug] [pid 6492:tid 1400] mod_cache.c(1196): [client 192.168.1.100:63741] AH00770: cache: Removing CACHE_REMOVE_URL filter.
[Sun Nov 04 11:58:42.904890 2012] [cache_disk:debug] [pid 6492:tid 1400] mod_cache_disk.c(1318): [client 192.168.1.100:63741] AH00737: commit_entity: Headers and body for URL http://proxy.lo/testpage? cached.
The first request to the origin server without mod_proxy to http://origin.lo/
GET http://origin.lo/testpage HTTP/1.1
Host: origin.lo
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/json
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
The first response from the origin without mod_proxy
HTTP/1.1 200 OK
Cache-Control: must-revalidate, proxy-revalidate, max-age=30
Content-Type: application/json; charset=utf-8
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sun, 04 Nov 2012 10:11:01 GMT
Content-Length: 1877
So, I assumed that revalidation must be occur only in 30 seconds after the success response.
Is't right?
Let's check it:)
Within 30 sec, the Google Chrome didn't perform any requests to the origin server to revalidate a request and has return the response from local cache.
When max-age is expired, the Google Chrome perform a request to revalidate:
GET http://origin.lo/testpage HTTP/1.1
Host: origin.lo
Connection: keep-alive
Cache-Control: max-age=0
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/xml
If-None-Match: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
and response:
HTTP/1.1 304 Not Modified
Cache-Control: must-revalidate, proxy-revalidate, max-age=30
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Sun, 04 Nov 2012 10:16:20 GMT
As you can see, all works as expected. User agent revalidates request only when max-age is expired.
Let's now try perform the folling flow though mod_proxy (see configuration above).
The first request:
GET http://proxy.lo/testpage HTTP/1.1
Host: proxy.lo
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/json
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
and the response was:
HTTP/1.1 200 OK
Date: Sun, 04 Nov 2012 10:23:36 GMT
Server: Apache
Cache-Control: private, no-cache, no-store, no-transform
Content-Type: application/json; charset=utf-8
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Content-Length: 1932
Vary: Accept,Content-Type,Content-Encoding,Accept-Language
X-Cache: MISS from proxy.lo
X-Cache-Detail: "cache miss: attempting entity save" from proxy.lo
Connection: close
Ok, let's see to the disk cache and try to see how request and response was stored.
(I cut binary data)
http://proxy.lo/testpage?
Cache-Control: private, no-cache, no-store, no-transform
Content-Type: application/json; charset=utf-8
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Date: Sun, 04 Nov 2012 10:27:15 GMT
Content-Length: 1932
Vary: Accept, Content-Type, Content-Encoding, Accept-Language
Host: proxy.lo
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/json
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
X-Forwarded-Proto: http
Cache-Control: max-age=300, must-revalidate
X-Forwarded-For: 192.168.1.100
X-Forwarded-Host: proxy.lo
X-Forwarded-Server: origin.lo
Ok, what we see?
We see that the first request was performed with max-age=300 & must-revalidate
Ok, looks good, as for me, lets perform the next call:
GET http://proxy.lo/testpage HTTP/1.1
Host: proxy.lo
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
Accept: application/json
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
and the second response from mod_proxy:
HTTP/1.1 200 OK
Date: Sun, 04 Nov 2012 10:31:58 GMT
Server: Apache
Cache-Control: private, no-cache, no-store, no-transform
ETag: "7cf651e2-176f-4ac1-808e-0e0c17cfd0a2"
Content-Length: 1932
Vary: Accept,Content-Type,Content-Encoding,Accept-Language
X-Cache: REVALIDATE from proxy.lo
X-Cache-Detail: "conditional cache hit: entity refreshed" from proxy.lo
Connection: close
Content-Type: application/json; charset=utf-8
SO, MY QUESTION IS: WHY mod_proxy perform revalidation on each request regardless that max-age is defined?
N.B. Apache 2.4.3
Thanks,
I would be grateful for any help.
