Caching block flaw in handling maximumElementsInCacheBeforeScavenging

Topics: Caching Application Block
Apr 26, 2007 at 3:36 AM
I ran into a nasty flaw while testing the caching application block of enterprise library 2.0 which still exists in enterprise library 3.0.

I wanted to use the cache to speed up access to a very slow, arcane API. The items I will be storing will number in the tens of thousands and have very low expiration times (2 minutes to 10 minutes). I ran some tests where I added hundreds of thousands of the objects I wanted to cache and it seemed to work quite nicely until it got to the end of my run, where the number of items I had put in the cache had reached the limit specified in maximumElementsInCacheBeforeScavenging. What I noticed was that even though my console app was sitting on a Console.Read() statement, the CPU usage was up around 50% constantly. I let it run for about 5 minutes with no abatement.

I probed around the source a bit to see what was going on and I found the problem and I think I've solved it. If the number of items in the cache goes over the number of items specified in the maximumElementsInCacheBeforeScavenging config property, a request to start scavenging will be queued in the BackgroundScheduler, however for every subsequent object which is added to the cache beyond the threshhold specified in that directive, another request for scavenging will be queued. Put another way, if you have set maximumElementsInCacheBeforeScavenging to 10,000 and you have 10,000 objects in your cache, if 2,000 get added quicker than scavenging can finish there will be 2,000 queued requests to start scavenging in the queue.

I could see this getting out of hand very quickly in my scenario of a rapidly changing cache, so I came up with a fix of sorts. I decided to post it here in the hope that a: somebody might benefit from it and b: somebody might be able to spot any flaws in it that I haven't seen.

Please note: I have made my changes to the caching block in Ent Lib 3.0.

In CacheCapacityScavengingPolicy.cs, add the following private field underneath the maximumElementsInCacheBeforeScavenging field:
      private bool isOverMaximum;

In CacheCapacityScavengingPolicy.cs, add the following after the IsScavengingNeeded method:
        /// <summary>
        /// Determines if scavanging is needed.
        /// </summary>
        public bool IsScavengingNeeded(int currentCacheItemCount, bool testIfOverMaximum)
        {
            if (currentCacheItemCount > MaximumItemsAllowedBeforeScavenging) {
                if (isOverMaximum) {
                    return false;
                } else {
                    isOverMaximum = true;
                    return true;
                }
            } else {
                isOverMaximum = false;
                return false;
            }
        }
 
        /// <summary>
        ///  Informs the policy that scavenging has been completed
        /// </summary>
        public void ScavengingComplete()
        {
            isOverMaximum = false;
        }

In ScavengerTask.cs, change DoScavenging to this:
        /// <summary>
        /// Performs the scavenging.
        /// </summary>
        public void DoScavenging()
        {
            if (NumberOfItemsToBeScavenged == 0) {
                scavengingPolicy.ScavengingComplete();
                return;
            }
 
            Hashtable liveCacheRepresentation = cacheOperations.CurrentCacheState;
 
            int currentNumberItemsInCache = liveCacheRepresentation.Count;
 
            if (scavengingPolicy.IsScavengingNeeded(currentNumberItemsInCache))
            {
                ResetScavengingFlagInCacheItems(liveCacheRepresentation);
                SortedList scavengableItems = SortItemsForScavenging(liveCacheRepresentation);
                RemoveScavengableItems(scavengableItems);
                scavengingPolicy.ScavengingComplete();
            }
        }

In Cache.cs, change lines 167 to 170 from
                if (scavengingPolicy.IsScavengingNeeded(inMemoryCache.Count))
                {
                    cacheScavenger.StartScavenging();
                }
to
                if (scavengingPolicy.IsScavengingNeeded(inMemoryCache.Count, true))
                {
                    cacheScavenger.StartScavenging();
                }
Apr 27, 2007 at 1:54 PM
I tossed it in the IssueTracker just to make sure it is known.

http://www.codeplex.com/entlib/WorkItem/View.aspx?WorkItemId=9925

Regards,

Dave

____________________________

David Hayden
Microsoft MVP C#
Jun 27, 2007 at 11:13 AM
Thanks David, most useful!

/rob