How Does Memory Hierarchy Reduce Access Time?

TL;DR

Memory hierarchy bridges the speed mismatch between a fast processor and slower memory at a reasonable cost. Registers provide the shortest access time, followed by SRAM cache, DRAM main memory, and secondary storage. Effective access time depends on each level's access time, hit ratio, miss ratio, and whether the levels are searched simultaneously or sequentially.

Transcript

hello everyone welcome back in this session we are going to learn about first the memory hierarchy and then we will have a brief introduction regarding how the different memory units are interfaced with the processor so let's get to learning now the term hierarchy if put simply means ranking also ranking can be done based on different parameters fo... Read More

Key Insights

• Memory hierarchy is a ranking of storage levels based on properties such as access time, size, cost, and usage frequency. The hierarchy exists to bridge the speed mismatch between a fast processor and slower memory while keeping the overall memory system reasonably cost effective.
• Registers have the shortest access time because they are made from flip-flops and embedded within the processor. SRAM caches follow registers, DRAM main memory follows caches, and secondary storage occupies the final level in the access-time hierarchy described in the lesson.
• Cost and usage frequency increase toward the upper levels of the memory hierarchy. This ordering is the reverse of the hierarchy based on access time and size, illustrating why a memory system combines several storage technologies instead of relying on only one level.
• Memory interfacing is the organization of connections between memory levels, the processor, and input-output peripherals. Its goal is to supply instructions to a processor whose speed is measured in millions of instructions per second, while maintaining fast access and reasonable cost.
• Information at level n is a subset of the information available at level n+1. When the processor finds a requested instruction or data item at level n, the result is a hit. When it is absent, the result is a miss and the next level is consulted.
• Hit ratio is the probability that requested information is available at a particular memory level. If 80 of a program's 100 instructions are present in main memory, its illustrated hit ratio is 80 divided by 100, or 0.8, which is 80 percent.
• Parallel interfacing connects the processor to different memory levels simultaneously. For three levels, the average access time weights t1 by h1, t2 by the probability of missing M1 and hitting M2, and t3 by the probability of missing both M1 and M2.
• Level-wise interfacing searches memory sequentially after each miss. Accessing a later level means earlier levels were already checked unsuccessfully, so their access times must also be included. Its effective access-time expression therefore uses cumulative times for requests resolved at M2 or M3.

Questions & Answers

Q: What is memory hierarchy in computer organization?

Memory hierarchy is the ranking of storage units according to properties such as access time, size, cost, and frequency of use. In the hierarchy presented, registers are closest to the processor and require the least access time, followed by SRAM caches, DRAM main memory, and secondary storage. The arrangement bridges the speed mismatch between the fast processor and slower memory at a reasonable cost.

Q: Why is a memory hierarchy needed?

A memory hierarchy is needed because the processor is very fast while memory is comparatively slow. Its purpose is to bridge that speed mismatch without making the entire storage system excessively costly. Frequently required information can be kept in faster upper levels, while larger collections remain in slower lower levels, allowing the processor to obtain many instructions quickly through a cost-effective organization.

Q: What is the order of levels in the memory hierarchy?

The access-time order begins with processor registers, which are made from flip-flops and embedded within the processor. SRAM cache comes next, followed by DRAM main memory, and then secondary storage. Registers take the least time to access. In terms of cost and frequency of use, the direction is reversed, with those properties increasing toward the upper levels.

Q: What are a memory hit and a memory miss?

A memory hit occurs when the processor requests an instruction or data item and finds it at the memory level being checked. A memory miss occurs when the requested information is not present at that level. Following a miss, the processor searches the next level, identified as level n+1, because information at level n is treated as a subset of the information available there.

Q: How is the hit ratio of a memory level calculated?

The hit ratio represents the proportion or probability of requested information being present at a memory level. In the lesson's example, a program contains 100 instructions in secondary storage and 80 are brought into main memory. The main-memory hit ratio is therefore 80 divided by 100, equal to 0.8 or 80 percent. The corresponding miss ratio is one minus the hit ratio.

Q: How does parallel memory interfacing work?

Parallel memory interfacing connects the processor simultaneously to multiple memory units, such as M1, M2, and M3. When information is requested, the processor can look across the different levels side by side. If their access times are t1, t2, and t3, with t1 less than t2 and t2 less than t3, the effective access time weights each time by the probability that its level supplies the request.

Q: How is effective access time calculated for parallel interfacing?

For three simultaneously connected levels, M1 contributes h1 multiplied by t1. M2 contributes the probability of missing M1 and hitting M2, which is (1-h1) multiplied by h2 multiplied by t2. M3 contributes the probability of missing both earlier levels, which is (1-h1) multiplied by (1-h2) multiplied by t3. Adding these contributions gives the average memory access time.

Q: How does level-wise memory interfacing differ from parallel interfacing?

Level-wise interfacing searches the hierarchy sequentially. The processor first checks M1, proceeds to M2 only after an M1 miss, and continues to M3 only if the information is also absent from M2. Because reaching a later level means the earlier levels were already searched, their access times are included cumulatively. Parallel interfacing instead connects all levels simultaneously and allows their searches to occur side by side.

Summary & Key Takeaways

• Memory units can be ranked by access time, size, cost, and frequency of use. Registers, which are made from flip-flops and embedded in the processor, have the shortest access time. SRAM caches come next, followed by DRAM main memory and secondary storage. Cost and usage frequency increase toward the upper levels.

• The information held at one memory level is a subset of the information available at the next level. Finding requested data or an instruction at the current level is a hit. Failing to find it is a miss, which causes the processor to search the next level of the hierarchy.

• Memory levels can be interfaced with the processor simultaneously or level by level. Simultaneous interfacing allows searches to occur in parallel, while level-wise interfacing searches each level only after a miss at the preceding level. Effective memory access time therefore depends on the chosen organization, access times, and hit ratios.