How to load balance with the caching block?

Topics: Caching Application Block
Apr 21, 2008 at 10:47 PM
Edited Apr 21, 2008 at 10:48 PM
I have a WCF web service that caches a list of customers read from a database into the memory of each server node where the service is hosted. This cache needs to be invalidated every time the list of customers in the database changes (this change can be triggered by any application including the web service).

I have the web service hosted in multiple servers behind a load balancer (Big IP). How do I ensure that the changes to the list of customers invalidates the cache in all the servers?

Any suggestions? The caching scenarios listed with the caching block documentation (including the single writer) do not support this scenario, so what is the best solution with/without the caching block to this relatively common enterprise scaling problem?
Apr 22, 2008 at 9:28 PM

rcarun wrote:
I have a WCF web service that caches a list of customers read from a database into the memory of each server node where the service is hosted. This cache needs to be invalidated every time the list of customers in the database changes (this change can be triggered by any application including the web service).

I have the web service hosted in multiple servers behind a load balancer (Big IP). How do I ensure that the changes to the list of customers invalidates the cache in all the servers?

Any suggestions? The caching scenarios listed with the caching block documentation (including the single writer) do not support this scenario, so what is the best solution with/without the caching block to this relatively common enterprise scaling problem?


I would really love to see this added to the caching block in a future release. It would be an extremely useful feature to have.

To answer your question though, look into sending and receiving UDP notifications between servers. We don't use the CachingBlock, but you could probably extend the entlib one to support this. Basically, each time a cache item is invalidated or expires, a cache instance on a server would send out a multicast UDP notification that would be picked up by the other caches. The data sent would have to include the key name of the item. The other caches would then invalidate the item, if it existed.

I hope this helps, and perhaps we should log this as a built-in feature request.
Apr 23, 2008 at 6:18 AM
Although I have not used it, indeXus.Net Shared Cache - high-performance, distributed caching http://www.codeplex.com/SharedCache looks like something I would try.


rcarun wrote:
I have a WCF web service that caches a list of customers read from a database into the memory of each server node where the service is hosted. This cache needs to be invalidated every time the list of customers in the database changes (this change can be triggered by any application including the web service).

I have the web service hosted in multiple servers behind a load balancer (Big IP). How do I ensure that the changes to the list of customers invalidates the cache in all the servers?

Any suggestions? The caching scenarios listed with the caching block documentation (including the single writer) do not support this scenario, so what is the best solution with/without the caching block to this relatively common enterprise scaling problem?


Apr 28, 2008 at 1:41 AM
I found myself in precisely the same situation. We have a web service farm and need to flush the caches after reloading certain data. If a high-performance cache isn't in the cards for your company then you may want to try what we've implemented. F5 BigIP boxes support two sets of web services called iRules and iControl. iControl contains a large set of web service methods that allow a remote app to query, configure and control BigIP boxes. We use a small portion of the iControl services to query the BigIP for the members in a local load-balancing pool. This returns a set of IP/port pairs which we turn around and use to send out notifications. You'll need to google for BigIP iControl and look around a bit for the documentation (my apologies for not being able to supply a link). The essence of the code is this; add one method to your web service that is used to gather all the member information by querying BigIP via iControl and distribute the request (ExecuteDistributedRequest) to all of the members in the pool. Add a second method that executes the request (ExecuteDynamicRequest). In my case I use reflection so I can add a method somewhere and just invoke it (instead of having an ugly switch statement). I've wrapped most of the implementation in a class called (boringly enough) DistributedRequestManager. Then code or tools invoke the ExecuteDistributedRequest method passing in a method name and an array of objects that match the method's signature. This in turn generates a sequence of calls to ExecuteDynamicRequest. There are some object lessons in this code (particularly the ServerCertificateValidationCallback) which you may need to include depending on how your BigIP box is configured.

Here's some snippets:

Web Service Methods

WebMethod
public DistributedRequestResponse ExecuteDistributedRequest(string[] poolNames, string methodName, object[] parameters)
{
DistributedRequestResponse response = null;

try
{
response = DistributedRequestManager.ExecuteDistributedRequest(poolNames, methodName, parameters);
}
catch (Exception ex)
{
if (ExceptionPolicy.HandleException(ex, "ProductWS"))
throw;
}

return response;
}

WebMethod
public object ExecuteDynamicRequest(string methodName, object[] parameters)
{
object response = null;

try
{
// Get the Type and MethodInfo.
Type type = typeof(DistributedRequestManager);
MethodInfo methodInfo = type.GetMethod(methodName, BindingFlags.Static | BindingFlags.NonPublic);

response = methodInfo.Invoke(null, parameters);
}
catch (Exception ex)
{
if (ExceptionPolicy.HandleException(ex, "ProductWS"))
throw;
}

return response;
}

The DistributedRequestManager

public class DistributedRequestManager
{
static LocalLBPool LocalLBPool = null;

static ProductClient.ProductWS ProductWS = null;
static string ProductWSUrlMask = null;

static DistributedRequestManager()
{
// Install a certificate policy for validating certificates when we connect to BigIP
System.Net.ServicePointManager.ServerCertificateValidationCallback =
new RemoteCertificateValidationCallback(DistributedRequestManager.ValidateServerCertificate);

// Set up an instance of the BigIP local load balancing pool and related network credentials
LocalLBPool = new LocalLBPool();

// Local LB Pool URL
string localLBPoolServiceUrl = ConfigurationManager.AppSettings"BigIP.LocalLBPoolServiceURL";
Uri uri = new Uri(localLBPoolServiceUrl);
LocalLBPool.Url = uri.AbsoluteUri;

// Local LB Pool Coolkies
LocalLBPool.CookieContainer = new CookieContainer();

// Local LB Pool authentication credentials
string username = ConfigurationManager.AppSettings"BigIP.Username";
string password = ConfigurationManager.AppSettings"BigIP.Password";
NetworkCredential credential = new NetworkCredential(username, password);

CredentialCache credentialsCache = new CredentialCache();
credentialsCache.Add(uri, "Basic", credential);
credentialsCache.Add(uri, "Digest", credential);

LocalLBPool.Credentials = credentialsCache;
LocalLBPool.PreAuthenticate = true;

// Get an instance of the product web service and the URL mask
ProductWS = new ProductClient.ProductWS();
ProductWSUrlMask = ConfigurationManager.AppSettings"ProductWS.UrlMask";
}

internal static DistributedRequestResponse ExecuteDistributedRequest(string[] poolNames, string methodName, object[] parameters)
{
// Get the pool members
//CommonIPPortDefinition[][] members = GetLocalMachine(poolNames);
CommonIPPortDefinition[][] members = LocalLBPool.get_member(poolNames);

if (members.Length < 1 || members0.Length < 1)
return null;

// For each pool member invoke ExecuteDynamicRequest()
object[][] responses = NotifyMembers(members, methodName, parameters);

// Construct an aggregate response
DistributedRequestResponse drr = new DistributedRequestResponse();
drr.PoolNames = poolNames;

drr.Request = new DynamicRequest();
drr.Request.MethodName = methodName;
drr.Request.Parameters = parameters;

drr.ResourceResponses = new ResourceResponsemembers.Length[];
CommonIPPortDefinition member = null;

for (int p = 0; p < members.Length; p++)
{
drr.ResourceResponsesp = new ResourceResponsemembers[p].Length;

for (int m = 0; m < membersp.Length; m++)
{
member = memberspm;
drr.ResourceResponsespm = new ResourceResponse(poolNamesp, member.address, member.port, responsespm);
}
}

return drr;
}

private static object[][] NotifyMembers(CommonIPPortDefinition[][] members, string methodName, object[] parameters)
{
// The members array is a two-dimensional array. The first dimension
// represents the pool and corresponds to poolNamex submitted in get_member(poolNames)
// The second dimension is the actual pool member (ip and port spec)
CommonIPPortDefinition member = null;
object[][] responses = new objectmembers.Length[];

for (int p = 0; p < members.Length; p++)
{
responsesp = new objectmembers[p].Length;

for (int m = 0; m < membersp.Length; m++)
{
member = memberspm;

// Configure this member's url
ProductWS.Url = string.Format(ProductWSUrlMask, member.address, member.port.ToString());

// execute the dynamic request
responsespm = ProductWS.ExecuteDynamicRequest(methodName, parameters);
}
}

return responses;
}

internal static bool ValidateServerCertificate(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
return true;
}

If all of this leaves you bewildered, feel free to email me.