Cache

What is Caching and How Does it Work? 
Caching makes your computer faster 

If you have purchased a computer recently, you may have heard the word "cache." It is usually mentioned as part of the specifications. Caching is a technology used with the memory subsystem of your computer. The main purpose of a cache is to accelerate your computer while keeping the price of the computer low. Caching allows you to do your computer tasks more rapidly. 

The Basic Concepts

To understand the basic idea behind a cache system, let's start with a super-simple example that uses a librarian to demonstrate caching concepts. Imagine a librarian behind his desk. He is there to give you the books you ask for. For the sake of simplicity, let's say you can't get the books yourself -- you have to ask the librarian for any book you want to read, and he gets it for you from the stacks in a storeroom (the library of congress in Washington, D.C., is set up this way). First, let's start with a librarian without cache. 

The first customer arrives. He asks for the book Moby Dick. The librarian goes into the storeroom, gets the book, returns to the counter and gives the book to the customer. Later, the client comes back to return the book. The librarian takes the book and returns it to the storeroom. He then returns to his counter waiting for another customer. Let's say the next customer asks for Moby Dick (you saw it coming...). The librarian then has to return to the storeroom to get the book he recently handled and give it to the client. Under this model, the librarian has to make a complete round trip to fetch every book -- even very popular ones that are requested frequently. Is there a way to improve the performance of the librarian? 
Yes, there's a way -- we can put a cache on the librarian.

Let's give the librarian a backpack into which he will be able to store 10 books (in computer terms, the librarian now has a 10-book cache). In this backpack, he will put the books the clients return to him, up to a maximum of 10. Let's use the prior example, but now with our

new-and-improved caching librarian. 

The day starts. The backpack of the librarian is empty. Our first client arrives and asks for Moby Dick. No magic here -- the librarian has to go to the storeroom to get the book. He gives it to the client. Later, the client returns and gives the book back to the librarian. Instead of returning to the storeroom to return the book, the librarian puts the book in his backpack and stands there (he checks first to see if the bag is full -- more on that later). Another client arrives and asks for Moby Dick. Before going to the storeroom, the librarian checks to see if this title is in his backpack. He finds it! All he has to do is take the book from the backpack and give it to the client. There's no journey into the storeroom, so the client is served more efficiently. 

What if the client asked for a title not in the cache (the backpack)? In this case, the librarian is less efficient with a cache than without one, because the librarian takes the time to look for the book in his backpack first. One of the challenges of cache design is to minimize the impact of cache searches, and modern hardware has reduced this time delay to practically zero. Even in our simple librarian example, the latency time (the waiting time) of searching the cache is so small compared to the time to walk back to the storeroom that it is irrelevant. The cache is small (10 books), and the time it takes to notice a miss is only a tiny fraction of the time that a journey to the storeroom takes. 
From this example you can see several important facts about caching: 

- Cache technology is the use of a faster but smaller memory type to accelerate a slower but larger memory type.  It also helps when data is transferred from a relatively slow data channel, like the Internet.
- When using a cache, you must check the cache to see if an item is in there. If it is there, it's called a cache hit. If not, it is called a cache miss and the computer must wait for a round trip from the larger, slower memory area. 
- A cache has some maximum size that is much smaller than the larger storage area. 
- It is possible to have multiple layers of cache. With our librarian example, the smaller but faster memory type is the backpack, and the storeroom represents the larger and slower memory type. This is a one-level cache. There might be another layer of cache consisting of a shelf that can hold 100 books behind the counter. The librarian can check the backpack, then the shelf and then the storeroom. This would be a two-level cache. 

Internet Caching

Caching can also improve performance when you retrieve information from the Internet. Your Internet connection is the slowest link in your computer. So your browser (Internet Explorer, Netscape, Opera, etc.) uses the hard disk to store HTML pages, putting them into a special folder on your disk. The first time you ask for an HTML page, your browser renders it and a copy of it is also stored on your disk. The next time you request access to this page, your browser checks if the date of the file on the Internet is newer than the one cached. If the date is the same, your browser uses the one on your hard disk instead of downloading it from Internet. In this case, the smaller but faster memory system is your hard disk and the larger and slower one is the Internet. 

The browser cache works well with individual computers, but is not efficient when many people access similar web sites.  To provide caching for many users on the network, you need to set up a separate computer server that does the job of caching web pages for everyone. This type of special server is called a web-caching appliance.  The Avantis ContentCache™ system is an example of this type of server

The ContentCache web-caching appliance provides one of the functions of a proxy server. It provides a store-and-forward cache. When you configure your web browser to use the caching appliance, it never connects to the URL. Instead, it always connects to this special proxy server, and asks it to get the URL (web page) for you. 

How the web-caching appliance works 

The web-caching appliance receives a request for an Internet service (such as a Web page request) from a user. The server looks in its local cache of previously downloaded Web pages. If it finds the page, it returns it to the user without needing to forward the request to the Internet. If the page is not in the cache, the web-caching server, acting as a client on behalf of the user, uses one of its own IP addresses to request the page from the server out on the Internet. When the page is returned, server relates it to the original request and forwards it on to the user. It also adds this new page to its cache. If the cache is full the web-caching server uses special rules to decide what other page to remove from the cache.  Just like the librarians backpack cache, if the bag becomes full they have to remove one of the books. For example they may remove the book that has been requested the least.

To the user, the web-caching server is invisible; all Internet requests and returned responses appear to be directly with the addressed Internet server. (The web-caching server is not quite invisible; its IP address has to be specified as a configuration option to the browser or other protocol program.) 

Even organizations that already have very fast internet connections, can benefit from web caching because it effectively removes upstream contention and all the interaction delays associated with traversing the internet (typically 10-20 hops between the user’s web browser and the origin server). Therefore, it will take longer for web content to travel through the Internet than the near instantaneous LAN delivery when caching is deployed.

Caching systems must ensure that users always receive identical content to what would have been delivered had the request been forwarded to the originating server via the Internet. Web servers deliver a ‘caching directive’ with their web pages that determine how long content is to be cached. For example, the directive may instruct the web caching appliance not to cache the content or for the content to time-out after a period of time. A professional web caching solution, such as the Avantis ContentCache™ will follow the originating server’s caching directive when setting all attributes of cached content to ensure that content is always ‘fresh’ and identical to the originating server. If required, ContentCache™ can also be configured to determine how often the content should be re-loaded e.g. on a daily, weekly or monthly basis.

How is it different from a Proxy Server?

The Web-caching server provides some of the functions of a proxy server, but is not as difficult to set up and use.  Microsoft provides Proxy Server software which runs on a server.  It usually requires a very experienced network technician to get this working correctly.  The web-caching appliance, as it name implies, is as easy to use as an appliance.  It is a plug and play device that includes the hardware and software necessary to perform the Proxy Server function.

Benefits

In a typical organization up to 90% of internet traffic is generated by users requesting the same web content more than once – making caching particularly advantageous. However, ContentCache™ offers much more than just dynamic caching – allowing high bandwidth content and e-learning resources to be hosted and delivered from the local cache on the network. This provides a better data gathering experience for all organizations, regardless of their Internet connection speed, and frees up the connection for other bandwidth intensive activities such as video conferencing.

Just as the librarian’s performance can be improved by adding a backpack cache, the Internet performance can be improved by adding a web-caching appliance to your network.