TYPO3 cache warmup with crawler not working

Question

I have a strange behaviour with the cache in my TYPO3 installation:

Situation:

• TYPO3 v9.5.22
• the cache for the page exists
• the page is only accessible for a certain FE usergroup
• cache is build with EXT:crawler (the configuration is setup with the needed FE usergroup)
  
  

First test

• I am logged in with a FE user with the needed FE usergroup

Result:
Page needs long time on first request, runs fast on second request → that feels as if the cache does not exist but is build with the first request

Second test

• still logged in with same FE user from first test
• also logged in as BE user with activated adminPanel
• settings in the adminPanel:
  • "No cache" is not checked (so cache should be used)

Result:
Same behaviour as in first test, adminPanel → TypoScript shows that page is build (Page generation), so cache is not used

Third test

• all settings from second test
• settings in adminPanel:
  • simulate FE usergroup → choose the needed FE usergroup

Result:
Fast response, adminPanel show that cache is used.

My conclusion:

• cache in general works
• warm up fails somehow → crawler configuration might be falsy?

What can be the reasons for this behaviour?
Is my configuration falsy?
Can some other settings in TYPO3 create this situation?

Answer 1

The problem was following:

• the page is access restricted with a single FE usergroup
• in the crawler configuration I added only the single FE usergroup which was needed to access the page
• the FE user had two FE usergroups.

The reason why the cache from the crawler is not taken into account depends on the way how TYPO3 caching works:

• TYPO3 gets all FE usergroups of a user.
• These FE usergroups are one of the input parameter to caculate a hash.
• This hash is an identifier. It is used to check whether a cache was already build with these parameters.
• My crawler was set up with only one FE usergroup but my test user has two of them.
• That's why the hash will be different. So the cache which was created by the crawler is not taken into account.

So the solution would be: Create a crawler configuration for every possible FE usergroup configuration which could users have.