WASABi Reactive Rule Scaling By

Topics: Windows Azure Integration Pack
Dec 19, 2011 at 4:32 PM

Hi,

I'm using the Windows Azure Scaling Application Block in my application. There I have a reactiveRule which reacts on CPU Timer of a WebRole.

The action looks like followed:

<action target="WebRole" by="1" />

When the rule is met, the instance count is increased by one and stays there.

That's my problem, how to specify: Whenever the CPU Time is met, increase the instance count by one (until Maximum defined in default rule). So if the CPU Time hits the limit another time, I want a third instance to come up and so one.

Any hints on achieving this?

greets

Dec 19, 2011 at 5:08 PM

What's your reactive rule conditional?

If you specify it using the 'greater' clause (see below) than it will keep scaling up till it hits the maximum defined by the constraint rules.

<rule name="Example Scaling Rule" rank="100">
  <when>
    <greater operand="CPU_WebRole" than="80"/>
  </when>
  <actions>
    <scale target="WebRole" by="1"/>
  </actions>
</rule>

Dec 19, 2011 at 7:54 PM

Hi,

Thanks for your answer. My rule condition looks exactly like yours. For testing purpose I've set the condition to greater than 0, but it stops scaling after 1 step.

The condition rules ranges from 1 to 5.

Any idea of whats wrong?

greets

 

Dec 19, 2011 at 8:02 PM

How's your stabilizer configured?

Do you want to post the whole autoscaling config?

Dec 19, 2011 at 8:41 PM
Rules:
<?xml version="1.0" encoding="utf-8" ?>
<rules xmlns="http://schemas.microsoft.com/practices/2011/entlib/autoscaling/rules" enabled="true">
  <constraintRules>
    <rule name="Default" description="Always active"
          enabled="true" rank="1">
      <actions>
        <range min="1" max="5" target="WebRole"/>
      </actions>
    </rule>
  </constraintRules>

  <reactiveRules>
    <rule name="ScaleUp" description="Increases instance count" enabled="true">
      <when>
        <greaterOrEqual operand="Avg_CPU_Web" than="1"/>
      </when>
      <actions>
        <scale target="WebRole" by="1"/>
      </actions>
    </rule>
  </reactiveRules>

  <operands>
    <performanceCounter alias="Avg_CPU_Web"
      performanceCounterName="\Processor(_Total)\% Processor Time"
      aggregate="Average" source="WebRole" timespan="00:10:00"/>
  </operands>
</rules>

Service Information:
<?xml version="1.0" encoding="utf-8" ?>
<serviceModel xmlns="http://schemas.microsoft.com/practices/2011/entlib/autoscaling/serviceModel">
  <subscriptions>
    <subscription name="xxx"
                  subscriptionId="xxx"
                  certificateStoreLocation="LocalMachine"
                  certificateStoreName="My"
                  certificateThumbprint="xxx">
      <services>
        <service dnsPrefix="xxx" slot="Production" scalingMode="Scale">
          <roles>
            <role alias="WebRole" roleName="WebRole" wadStorageAccountName="storage"/>
          </roles>
        </service>
      </services>
      <storageAccounts>
        <storageAccount alias="storage" connectionString="xxx" />
      </storageAccounts>
    </subscription>
  </subscriptions>
</serviceModel>

I dont have a stabilizer configured. Am I supposed to?
greets
Dec 19, 2011 at 9:28 PM

If you don't specify a cooldown period in the service information file, a default of 20 minutes will be used. So maybe that is why you are not seeing this scaling... did you wait for more than 20 minutes to see if it scales after that?

If you want to change this 20-minute default, you can configure the stabilizer. More info on it here: http://msdn.microsoft.com/en-us/library/hh680945(v=PandP.50).aspx#sec46

Hope it helps,
Julian

Dec 20, 2011 at 10:54 AM

Okay thanks, it seems to work now :)

Another thing I don't know how to do:

I configured my Autoscaler to use the System Diagnostics Logger. I started my WorkerRole and also started the DiagnosticsMonitor to Log into my TableStorage. But I don't find any information about the Autoscaler or messages when there is was a scale action performed. Where do I find this?

 

Dec 20, 2011 at 11:10 AM

Unfortunately the Sample Config for the Diagnostics Logger is missing on this page:

Sample Configuration Settings for System Diagnostics Logging

http://msdn.microsoft.com/en-us/library/hh680883(v=PandP.50).aspx

Dec 20, 2011 at 3:30 PM

@crazyfx Thanks for catching that. We'll update.

In the meantime, here's the code snippet :

<system.diagnostics>
  <sources>
    <source name="Autoscaling General"  switchName="SourceSwitch" 
        switchType="System.Diagnostics.SourceSwitch" >
      <listeners>
        <add name="AzureDiag" />
        <remove name ="Default" />
      </listeners>
    </source>
    <source name="Autoscaling Updates"  switchName="SourceSwitch" 
        switchType="System.Diagnostics.SourceSwitch" >
      <listeners>
        <add name="AzureDiag" />
        <remove name ="Default" />
      </listeners>
    </source>
  </sources>
  <switches>
    <add name="SourceSwitch" 
        value="Verbose, Information, Warning, Error, Critical" />
  </switches>
  <sharedListeners>
    <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, 
        Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, 
        PublicKeyToken=31bf3856ad364e35"
        name="AzureDiag"/>
  </sharedListeners>

  <trace>
    <listeners>
      <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, 
          Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, 
          PublicKeyToken=31bf3856ad364e35"
          name="AzureDiagnostics">
        <filter type="" />
      </add>
    </listeners>
  </trace>
</system.diagnostics>

 

 


Jan 11, 2012 at 2:06 AM

FYI, the MSDN article has been updated on Dec 20 to include the missing code snippet.

Jun 27, 2012 at 11:59 AM

Hi,

I have also question about reactive rules. 

I'm using Windows Azure Scaling Application Block (WASABi) in my application for a while now and the issue that I am facing is with a reactive rule where you can specify degree of scaling ie how many new instances will be added to my current number of instances. That degree can be represented by number of instances or with percentage. However, i'm trying to set that field to a value that will be calculated using average number of messages in a specified queue and current role instances ie to a variable that is not known in advance. Is there any way that i could dinamically change deegre of scaling od my roles (by 1, 3, 4... n instances)? I do not want it to be hard coded prior to deployment of my application. If I can not do something like that, WASABI is useless to me.

greets

Jun 28, 2012 at 5:44 PM
Edited Jun 28, 2012 at 5:44 PM

@DukeTitoAndolini

Instead of the performanceCounter operand, use the queueLength and the roleInstanceCount operands that we ship in the box.

For a sample, see "the operands element" section of http://msdn.microsoft.com/en-us/library/hh680955(PandP.50).aspx#sec10

Grigori

 

 

Jun 29, 2012 at 7:44 AM

I know that. That is ok. But when I get values for queueLength and the roleInstanceCount I need those values to use them to calculate by how many instances will my role be scaled. Problem is that I don't want to specify in rule that I want to scale role by for example 1,2,6 instances or 25%, 35%. I want scaling number to change dinamically, depending on some calculation performed with values from queueLength and roleInstanceCount.

For example here:

<reactiveRules>
  <rule name="Example Scaling Rule" rank="100">
    <when>
      <greater operand="Calculation" than="6"/>
    </when>
    <actions>
      <scale target="WorkerRoleA" by="2"/>
    </actions>
  </rule>
  ...
</reactiveRules>

I want instead of this rule and value '2' to be able to put some variable that will change during time depending on operand 'Calculation'. Operand 'Calculation' represents calculation value that we get using queueLength and roleInstanceCount. For calculation I need operations such as subtraction, multiplication and division. Another question is how to calculate that because as I can see there is no possibility to do that. 

Is that currently possible in WASABi?

Jun 29, 2012 at 2:33 PM

You cannot parametrize the value of the  by attribute in the scale action out of the box. You can, however, provide a custom action based on our implementation that does that. If you want to suggest this as a feature request, pls do so via the uservoice site.

You can, however, use simple expressions in the <when> clause with multiple operands to compare. So, you can do something like this:

<all>
   <greater operand="queueLength" than="0.5 * customOperand"/>
   <greater operand="roleInstanceCount" than="3 * someOtherCustomOperandOrPerfCounter"/>
</all>

See the "Specifying Comparison Values" of the reference doc for additional samples.

Supported operations are multiplication and division. Usingthe source code for the block provided, you can easily extend it to support subtraction.

 

Jul 2, 2012 at 8:58 AM

 

Thank you for your answer. Maybe custom action and operand can be useful for me. In the the particular situation, I have formula: queueLength/10 - roleInstanceCount = scalingNumber. Using that calculation I can get, as you can see, number by which I should scale role. Do you think it is possible to achieve that using custom actions and custom operands?