Cache Memory Tutorial - 2
-Aviral Mittal (avimit att yhaoo dat cam)
Connect @

Cache Memories - 2.
In the previous Section, it was explained what is a cache memory, why a cache memory is needed with an example 256 bit register working as a cache memory and very basic working principles of cache.
This chapter is about 2 very commonly used Cache related terms, i.e. Spatial Locality and Temporal Locality.

In the previous Section, a 256 bit register was used as a cache memory. It is of very limited use, as the size of this memory is very small, so the number of times there will be a Cache Miss will be quite large. That is to say that the Cache Miss rate will be large which will result in a large number of fetches from slower memory, which in turn wont be very efficient.
If a memory is organized in such a way that it has several of these 128 bit 'lines' or may be 256 bit 'lines', the the performance of the system can be greatly enhanced, as its obvious that larger cache memory would increase the probability of a Cache Hit and would decrease the probability of Cache Miss. Having larger cache memory would also increase the cost of the system, hence there is always this trade-off to decide what size of cache should a system or a processor in a system should use. The size of the cache memory is one factor which can affect Cache Hit rate Cache Miss rate, another factor is the characteristics of the code the processor is running. For example if the code is very loopy, then it is possible that the entire loop is fetched in the cache, and thorough-out  the time the processor is executing the loop, all the code fetches are only from cache memory. This means a Cache Hit rate of 100% while the processor is executing the loop.

What is Spatial Locality and Temporal Locality?

Spatial Locality
The characteristic or the property of the code, where the code-instructions for a task or a function are stored in  close proximity in memory, such as code corresponding to a loop in programing languages etc, is termed as 'Spatial Locality' . Higher the spatial locality, the higher is the Cache Hit rate, resulting in performance boost and power saving(s). 'Spatial' is related to 'space'. By the way loops is just one example of increasing the 'spatial locality'.
If the code is written in a way that it jumps to far off locations, then the code is supposed to have low Spatial Locality, and this would decrease the Cache Hit rate, resulting in performance de-gradation and higher power consumption.

Temporal Locality

If the code is written in a way that it uses functions which are used very frequently, then their presence in cache memory can again help in boosting the system performance. The term 'temporal' is related to time. Temporal Locality is the characteristic or property of the code which exploits the use of multiple functions in quick succession i.e. very frequently. Hence if the code is written in this way, then its supposed to exhibit high degree of 'Temporal Locality'. Again this property of the code is very suitable to be used with cache memories, as it will obviously increase the Cache Hit rate, boost performance and save power.

Conclusion: Ya, the industry and academia loves complex jargon e.g. 'spatial locality' or 'temporal locality' to make things look more sophisticated, but things really at times are as simple as that.

Note: Its to be noted that at times there may or may not be a choice to write code to have higher Spatial Locality/Temporal Locality, hence the 'spatial locality/temporal locality' of the code is just not a programmer's competence, its also depends upon the application which is being coded.

Typical Cache Hit Rate(s).
In embedded world of ARM Cortex-M processor based SoCs/Systems a Cache Hit rate of > 90% is quite common.

<= Cache Memory Primer                             Next => Cache Organization, Direct Mapped Cache 

Cache Organizations:
4-Way-Set-Associative Cache
2-Way-Set-Associative Cache

What is a cache memory
How does a cache memory work
What is Cache Miss
What is Cache Hit
What is 'tag'.