Cache management is actually a key element of the database management, Laurent Ribardière presents at 4D Summit Portland 2016 the new cache manager he implemented in the 4D v16 64-bit version.
Before watching the technical presentation with details about the implementation, we recommend to start with these two blog posts, describing the feature benefit itself:
Behavior of Cache Manager in previous versions
Up to 4D v16, the cache used to be split into 4 large blocks of same size. For instance, a cache of 4 GB used to be split into four blocks of 1 GB each. Basically objects are allocated in one of the block and whenever memory space is needed, in order to avoid fragmentation, data of one big block is wiped out all at once.
In some case, you could have memory remaining in this block (because a record is being used for instance) and this is where fragmentation could happen. The more tables, current selections and connected users you have on a server, the more this fragmentation risk is increasing. And in the end, whenever you need to load a big object in memory, you may be stuck. Even if you put more memory, the problem remains the same.
The other drawback of this mechanism is that you always release one big block at once. For example if you have a 128 GB cache, you will release 32 GB at once so you loose a lot of information loaded in cache memory whereas it was not necessary.
4D v16 fully optimized Cache Manager
There might be a lot of reason why some objects in memory cannot be freed right away and have to stay in cache memory. So the idea is the following: instead of trying to avoid fragmentation, let’s try to leave with fragmentation. The new cache manager only works in 64-bit, where the virtual memory space is up to 264, so almost infinite. Of course, physical memory is always limited but internally objects will be allocated inside this huge virtual space. We actually use the Memory Management Unit (MMU) of the processor to map the virtual memory to physical memory.
The good thing for 4D internal implementation is that the system is handling the physical memory defragmentation itself, by rearranging objects in memory whenever it is needed, but without changing the addresses in virtual memory. So defragmentation is totally transparent for us in the 4D internal C++ code. This is how we can work with a very large cache without bothering about fragmentation as it is fully handled by the system and without having to often release objects from memory.
The cache manager internal algorithm is actually based on a object priority concept. Object with a high priority will stay in cache longer than objects with a low priority. And we will even provide commands to give the 4D developer the control over the priority of tables or indexes. You will actually be able to change the priority in cache of a table or index on startup or even dynamically whenever you need it.
The new cache manager will actually mostly benefit to very large databases. It starts to be useful when the data doesn’t fit into the cache as there are choice to do about which object to keep or release in memory.