Database BackingStore memory consumption

Topics: Caching Application Block
Sep 1, 2009 at 1:44 PM
Edited Sep 1, 2009 at 1:46 PM

Well, I know using a DB backing store is not a DB cache, but a memory cache with a persistent storage implementation.

The thing is, this is not the most common usage scenario, as evidenced by the misinterpretation of the behavior & usage of this feature and the several complains of the developers trying to "easily" get a shared persistent cache mechanism.

Ok, first, the loading of the cached items at startup is not a good idea, much better would be loading just keys and expiration metadata from DB and lazy load the actual item only when needed.

Second, load the ENTIRE set of cached objetcts with a DATASET! Common guys! you can do better than that....  I have a service that's using this configuration and stores about 100Mb of cached items. Those items consume about 300Mb of memory in the EntLib caching block (a little overhead here...) but when initializing the cache this baby consumes 1.6Gb (yes, Gigabytes) of ram. After a couple minutes the GC collect those GBs of wasted memory but in some cases where the server is under a little more presure the service FAILS to start.

I'm willing to help & contribute with some code here, clearly we need to solve this issue ASAP and I think many can benefit from fixing this.

We can go two different routes here:

  1. Modify the Database backing store / manager to use a lazy load approach (and NOT USE a DATASET)
  2. Create a new implementation of a Cache that stores the cached item only in DB and not in memory.

Any ideas, comments?

Thanks a lot,

Andrés G Vettori

Sep 2, 2009 at 7:27 AM


I'm not sure what do you mean by creating a new implementation of Cache, are you saying to modify the Cache class? Also, wouldn't it create more overhead if you will store the cached item to the db and you dont have a memory representation of that items? and whenever you will access a item, you would do data access. Doesn't that defeat the purpose of caching?

Valiant Dudan
Global Technology and Solutions
Avanade, Inc.


Sep 2, 2009 at 4:16 PM

I will try to clarify a little the two options:

  1. Modify the Database backing store / manager to use a lazy load approach (this option seems clear to me)
  2. Change the Caching block so it becomes a more general caching mechanism (not just a memory cache) allowing to define different storage mechanisms like memory, file system, isolated storage, database, etc, etc.
  3. Knowing that a common pattern in caching is to have two or more caching levels, maybe would be a nice idea to implement explicitely that pattern. I mean, explicitely define different levels of caching each one being a cache storage provider (the first one would be memory, the second file system, and the third one database, for example). I'm thinking on a CacheManager that implements something like a chain of responsability pattern or something similar.

Well, for v4.x the only option would be #1, maybe the needed changes would be incorporated into v5. I'm even willing to contribute code for this.


Sep 3, 2009 at 12:56 AM
Edited Sep 3, 2009 at 1:04 AM

#2 is also possible to achieve by implementing your own instance of IBackingStore.  If you want to contribute, there's an Entlib Contrib project here in Codeplex -  If you want to have the features you mentioned above, you'll can log a work item in the issue tracker and get people to vote.  The entlib team also gave the community the chance to vote on features they want in preparation for the next version.  The current product backlog can be viewed here -, but I don't think there'd be much change in CAB.


Sarah Urmeneta
Global Technology and Solutions
Avanade, Inc.

Sep 3, 2009 at 1:24 AM

Well, although #2 is indeed possible to achieve with entLib 4.x it would require some important changes to basic stuff. For example, the interface IBackingStore doesn't expose a method to get a single item, something needed to achieve the ""lazy load" approach and avoid loading all cached items on startup (in this case the startup load would only get some metadata about the items, like expiration time and key).

Let's see what I can do, maybe I can convince some people to evolve a little the v.Next Caching block....   who knows...

Thanks for your comments,