Make LRU memory-based cache cleaning compatible with ECS cache
The structure of ECS cache means that non-ECS RRsets will tend to be preferred over ECS RRsets when LRU (Least Recently Used) cache cleaning is initiated when a cache size reaches configured max-cache-size
Essentially, LRU maintains a list of nodes as potential early-expire candidates based on when they were last accessed. When a node is accessed by a client query it drops to the bottom of the list. When named needs to expire some content early, in order to make room for more, it picks the node from the top of the list.
Enter ECS cache. Here we have, sitting attached to a single node, another dimension of variable size, depending on the range of clients querying for the same name where ECS has been enabled and also depending on the effective prefix size for the scoped RRsets sitting there. Every time we access that node to pull out (or add) an RRset, we move the node to the end of the LRU list. If there are a lot of different RRsets being maintained, even though some of them individually would have been candidates for LRU deletion, newer access/additions to the node will move it back to the bottom of the LRU list.
The outcome is that under cache memory pressure, we could end up expiring most of the other (usable!) cache content but never removing the older ECS-scoped content that has created the memory pressure in the first place. This could lead to:
- Cache thrash and poor performance due to repeated adds and deletions of some important cache content.
- Worst case - we seldom manage to complete the series of fetches needed to populate cache with the RRsets needed to answer a client query because the content is vanishing as fast as we put it in there.
- Very worst case - we don't manage to prime the roots because as fast as we do, they're LRU-expiring again (or some other problem with getting them into cache to be used).
The mitigation (as it is anyway, for sites whose caches are over-constrained by max-cache-size) is to make sure that max-cache-size is large enough to prevent reaching the limit, other than exceptionally.
ECS cache, however, is much more vulnerable to becoming unusable under cache memory pressure because of the way that the ECS-scoped content drives up cache memory use but then fails to be managed effectively by LRU-cleaning, so we end up with 'islands' of untouched ECS cache and nothing much else.
We do need to fix this, but (perhaps) also make recommendations to users of ECS that they must increase max-cache-size when enabling ECS and then monitor, since reaching max-cache-size could be bad for their resolvers because cache cleaning in that situation is not as effective as for non-ECS caches.
Similarly we need to cater for multiple RDATA types for the same name being cached at the same node, since those are subject to the same problem, although mostly to a much lesser degree.
Relates to (now closed) Support case 00001412 Single View broken