TYPO3 static file cache doesn't work for each request

Question

I have a strange problem with static file cache and TYPO3. It works, but not all the time. During tests of the delivered pages I recognized, that not every time the cached html is delivered.

I've then done several tests in differnt browsers which were never logged in to the TYPO3 backend. Everywhere the same. Some pages are cached, some not. Even when I do multiple requests to the same page, e.g. via curl, sometimes I get the static variant, sometimes TYPO3 delivers the page. But it's not reproduceable.

I've created a small shell script to automate this testing for me:

#!/bin/sh
url="https://domain.tld/foo/bar/"
for i in `seq 10`;
do
  echo $i
  curl -s $url | grep statically
done

Expected behaviour would be to get the static page on each request, but reality looks like this:

➜  ~ ./test.sh
1
<!-- cached statically on: 14-12-16 08:46 -->
2
3
4
5
<!-- cached statically on: 14-12-16 08:46 -->
6
<!-- cached statically on: 14-12-16 08:46 -->
7
8
9
<!-- cached statically on: 14-12-16 08:46 -->
10
<!-- cached statically on: 14-12-16 08:46 -->

This is what the access log looks like:

xx.xxx.xx.xx    -   14/Dec/2016:11:52:44 +0100  GET /foo/bar/   HTTP/1.1    200 26892   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:44 +0100  GET /foo/bar/   HTTP/1.1    200 26892   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:43 +0100  GET /foo/bar/   HTTP/1.1    200 26892   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:43 +0100  GET /foo/bar/   HTTP/1.1    200 26661   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:43 +0100  GET /foo/bar/   HTTP/1.1    200 26661   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:42 +0100  GET /foo/bar/   HTTP/1.1    200 26661   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:42 +0100  GET /foo/bar/   HTTP/1.1    200 26661   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:42 +0100  GET /foo/bar/   HTTP/1.1    200 26661   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:42 +0100  GET /foo/bar/   HTTP/1.1    200 26892   -   curl/7.43.0
xx.xxx.xx.xx    -   14/Dec/2016:11:52:41 +0100  GET /foo/bar/   HTTP/1.1    200 26892   -   curl/7.43.0

The only difference is the size of the delivered content.

This is how the response header looks like. The first is broken, the second is correct:

➜  ~ curl -I https://domain.tld/foo/bar/
HTTP/1.1 200 OK
Date: Wed, 14 Dec 2016 12:14:17 GMT
Server: Apache/2.4.20
X-Powered-By: PHP/7.0.6
Content-Language: de
Content-Length: 21850
Strict-Transport-Security: max-age=31536000
Connection: close
Content-Type: text/html; charset=utf-8

➜  Downloads curl -I https://domain.tld/foo/bar/
HTTP/1.1 200 OK
Date: Wed, 14 Dec 2016 12:14:19 GMT
Server: Apache/2.4.20
Strict-Transport-Security: max-age=31536000
Vary: Host,Accept-Encoding
Last-Modified: Wed, 14 Dec 2016 07:46:38 GMT
Accept-Ranges: bytes
Content-Length: 21932
Cache-Control: max-age=38741
Expires: Wed, 14 Dec 2016 23:00:01 GMT
X-UA-Compatible: IE=edge
X-Content-Type-Options: nosniff
Connection: close
Content-Type: text/html; charset=utf-8

My .htaccess file with the mod_rewrite rules can be found here:

http://pastebin.com/5G6f3b4W

It's basically the default .htacess file shipped with TYPO3 7.6 and some custom additions.

Since the hosting is done on a managed server, there is no access to the vhost configuration to enable RewriteLog.

Tl;dr

• creation of static file cache files works, the files are present in the file system
• it's not reproduceable, why the static files aren't delivered at any request

I'm now searching for hints what I can do to further track down the problem.