How to reduce Enterprise Library 5.0 Logging memory usage?

Topics: Building and extending application blocks, Data Access Application Block, Enterprise Library Core, General discussion, Logging Application Block, Validation Application Block
Jan 25, 2012 at 10:16 PM

I am using enterprise logging 5.0 within a .net 4.0 console application. I notice very high memory usage within my application. I was able to determine that the cause was due to the following call:

var logWriter = EnterpriseLibraryContainer.Current.GetInstance<LogWriter>();

After some profiling and manual testing with a simple console application, I was able to determine that memory usage drop from **45mb to 10mb** when the following dlls were removed from the execution folder:

- Microsoft.Practices.EnterpriseLibrary.Validation.dll
- Microsoft.Practices.EnterpriseLibrary.Data.dll

The log initialization is my first call to Enterprise library apis. My console application does not make any calls to the Data.dll and Validation.dll. They exist in my execution folder, because they are references for other class libraries and our deployment setup. 

I am assuming that EnterpriseLibraryContainer.Current is initializing based on what is found in the execution folder. I tried creating my logwriter with the following but I got the same result:

var configSource = new FileConfigurationSource(configPath)
var logWriterFactory = new LogWriterFactory(configSource);
var logWriter = logWriterFactory.Create();

Is it possible to initialize a logwriter without increasing the memory usage with the validation and data dlls present in the execution folder?

Jan 26, 2012 at 12:18 AM
Edited Jan 26, 2012 at 12:47 AM

So after some debugging within the entlib source. I believe the following is finding the dll and instantiating the Validation.dll, and Data.dll, despite not being referenced at all in the project.

From EntLib50Src\Blocks\Common\Src\Configuration\ContainerModel\TypeLoadingLocator.cs

        private IEnumerable<TypeRegistration> GetRegistrationsInternal(IConfigurationSource configurationSource,
            Func<ITypeRegistrationsProvider, IConfigurationSource, IEnumerable<TypeRegistration>> registrationAccessor)
        {
            Type providerType = Type.GetType(Name);
            if (providerType == null) return new TypeRegistration[0];

            var provider = (ITypeRegistrationsProvider)Activator.CreateInstance(providerType);
            return registrationAccessor(provider, configurationSource);
        }

The call to Type.GetType(Name) looks in the Executing Assembly location

my stack trace leading up to the method above,  which orginates from :

 

var configSource = new FileConfigurationSource(configPath);
var logWriterFactory = new LogWriterFactory(configSource);
var logWriter = logWriterFactory.Create();

 

 

> Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.TypeLoadingLocator.GetRegistrationsInternal(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource, System.Func<Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider,Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource,System.Collections.Generic.IEnumerable<Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.TypeRegistration>> registrationAccessor) Line 85 C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.TypeLoadingLocator.GetRegistrations(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 67 + 0x70 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.CompositeTypeRegistrationsProviderLocator.GetRegistrations.AnonymousMethod__0(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider l, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource cs) Line 128 + 0xc bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.CompositeTypeRegistrationsProviderLocator.GetRegistrationsInternal.AnonymousMethod__4(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider l) Line 147 + 0x1d bytes C#  [External Code]   Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.Unity.UnityContainerConfigurator.RegisterAllCore(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider rootProvider) Line 61 + 0x4e bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ChangeTrackingContainerConfigurator.RegisterAll(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider rootProvider) Line 66 + 0x10 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.EnterpriseLibraryContainer.ConfigureContainer(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider locator, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.IContainerConfigurator configurator, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configSource) Line 83 + 0xf bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.EnterpriseLibraryContainer.ConfigureContainer(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.IContainerConfigurator configurator, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configSource) Line 62 + 0x1c bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.Unity.EnterpriseLibraryCoreExtension.Initialize() Line 60 + 0xe bytes C#  [External Code]   Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.Unity.EnterpriseLibraryBlockExtension.Initialize() Line 29 + 0x37 bytes C#  Microsoft.Practices.EnterpriseLibrary.Validation.dll!Microsoft.Practices.EnterpriseLibrary.Validation.Configuration.Unity.ValidationBlockExtension.Initialize() Line 34 + 0x8 bytes C#  [External Code]   Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.Unity.UnityContainerConfigurator.AddValidationExtension() Line 208 + 0xf bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.Unity.UnityContainerConfigurator.UnityContainerConfigurator(Microsoft.Practices.Unity.IUnityContainer container) Line 44 + 0x8 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.EnterpriseLibraryContainer.CreateDefaultContainer(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 121 + 0x18 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ContainerBasedInstanceFactory<Microsoft.Practices.EnterpriseLibrary.Logging.LogWriter>.ContainerBasedInstanceFactory(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 48 + 0xe bytes C#  Microsoft.Practices.EnterpriseLibrary.Logging.dll!Microsoft.Practices.EnterpriseLibrary.Logging.LogWriterFactory.LogWriterFactory(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 34 + 0xb bytes C#

Jan 26, 2012 at 12:32 AM
Edited Jan 26, 2012 at 12:48 AM

After Debugging further my original application which contains connection strings with Oracle ODP.net Providers.  (which I failed to mention from the start)

(my current application execution makes no calls or references to data access, connection strings are defined because application uses dynamic calls to other dlls which need connection strings., but for my test I am not invoking any of those calls)

Since Microsoft.Practices.EnterpriseLibrary.Data.dll is found, EnterpriseLibrary continues default registration of types for dataaccess and I found that the following call is the cause for the huge memory spike:

\EntLib50Src\Blocks\Data\Src\Data\Configuration\DatabaseSyntheticConfigSettings.cs

private static DbProviderMapping GetDefaultMapping(string dbProviderName)
        {
            // try to short circuit by default name
            if (DbProviderMapping.DefaultSqlProviderName.Equals(dbProviderName))
                return defaultSqlMapping;

            if (DbProviderMapping.DefaultOracleProviderName.Equals(dbProviderName))
                return defaultOracleMapping;

            // get the default based on type
            var providerFactory = DbProviderFactories.GetFactory(dbProviderName);
            

            if (SqlClientFactory.Instance == providerFactory)
                return defaultSqlMapping;

            if (OracleClientFactory.Instance == providerFactory)
                return defaultOracleMapping;

            return null;
        }

The DbProviderFactories.GetFactory(dbProviderName) call when dbProviderName=Oracle.DataAccess.Client.OracleClientFactory causes the huge memory spike.

So looks like the reason for the huge memory spike was due to odp.net and the fact that its registering DBFactories.  

It seems like I cannot create a logger without registering everything present in the executing assembly location.  Ideally I would like to not register data access unless its explicitly told to.

my stack trace leading up to the GetDefaultMapping call:

> Microsoft.Practices.EnterpriseLibrary.Data.dll!Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSyntheticConfigSettings.GetDefaultMapping(string dbProviderName) Line 258 C#  Microsoft.Practices.EnterpriseLibrary.Data.dll!Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSyntheticConfigSettings.GetProviderMapping(string dbProviderName, Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSettings databaseSettings) Line 242 + 0x8 bytes C#  Microsoft.Practices.EnterpriseLibrary.Data.dll!Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSyntheticConfigSettings.GetDatabaseData(System.Configuration.ConnectionStringSettings connectionString, Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSettings databaseSettings) Line 155 + 0x18 bytes C#  Microsoft.Practices.EnterpriseLibrary.Data.dll!Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSyntheticConfigSettings.get_Databases() Line 94 + 0x2b bytes C#  [External Code]   Microsoft.Practices.EnterpriseLibrary.Data.dll!Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSyntheticConfigSettings.DoGetRegistrations() Line 323 + 0x2d bytes C#  [External Code]   Microsoft.Practices.EnterpriseLibrary.Data.dll!Microsoft.Practices.EnterpriseLibrary.Data.Configuration.DatabaseSyntheticConfigSettings.GetRegistrations(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 307 + 0x75 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.TypeLoadingLocator.GetRegistrations.AnonymousMethod__0(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider p, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource cs) Line 67 + 0xc bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.TypeLoadingLocator.GetRegistrationsInternal(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource, System.Func<Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider,Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource,System.Collections.Generic.IEnumerable<Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.TypeRegistration>> registrationAccessor) Line 90 + 0x11 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.TypeLoadingLocator.GetRegistrations(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 67 + 0x70 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.CompositeTypeRegistrationsProviderLocator.GetRegistrations.AnonymousMethod__0(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider l, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource cs) Line 128 + 0xc bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.CompositeTypeRegistrationsProviderLocator.GetRegistrationsInternal.AnonymousMethod__4(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider l) Line 147 + 0x1d bytes C#  [External Code]   Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.Unity.UnityContainerConfigurator.RegisterAllCore(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider rootProvider) Line 61 + 0x4e bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ChangeTrackingContainerConfigurator.RegisterAll(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider rootProvider) Line 66 + 0x10 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.EnterpriseLibraryContainer.ConfigureContainer(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ITypeRegistrationsProvider locator, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.IContainerConfigurator configurator, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configSource) Line 83 + 0xf bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.EnterpriseLibraryContainer.ConfigureContainer(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.IContainerConfigurator configurator, Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configSource) Line 62 + 0x1c bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.Unity.EnterpriseLibraryCoreExtension.Initialize() Line 60 + 0xe bytes C#  [External Code]   Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.Unity.EnterpriseLibraryBlockExtension.Initialize() Line 29 + 0x37 bytes C#  Microsoft.Practices.EnterpriseLibrary.Validation.dll!Microsoft.Practices.EnterpriseLibrary.Validation.Configuration.Unity.ValidationBlockExtension.Initialize() Line 34 + 0x8 bytes C#  [External Code]   Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.Unity.UnityContainerConfigurator.AddValidationExtension() Line 208 + 0xf bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.Unity.UnityContainerConfigurator.UnityContainerConfigurator(Microsoft.Practices.Unity.IUnityContainer container) Line 44 + 0x8 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.EnterpriseLibraryContainer.CreateDefaultContainer(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 121 + 0x18 bytes C#  Microsoft.Practices.EnterpriseLibrary.Common.dll!Microsoft.Practices.EnterpriseLibrary.Common.Configuration.ContainerModel.ContainerBasedInstanceFactory<Microsoft.Practices.EnterpriseLibrary.Logging.LogWriter>.ContainerBasedInstanceFactory(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 48 + 0xe bytes C#  Microsoft.Practices.EnterpriseLibrary.Logging.dll!Microsoft.Practices.EnterpriseLibrary.Logging.LogWriterFactory.LogWriterFactory(Microsoft.Practices.EnterpriseLibrary.Common.Configuration.IConfigurationSource configurationSource) Line 34 + 0xb bytes C#

Jan 26, 2012 at 5:47 AM

I looked at the problem as well and couldn't replicate your results.  My console application increased from 1.7MB to 6MB after EntLib was loaded.  
It makes sense it was Oracle Data Provider.  Not that it's Oracle!  I mean it makes sense because I don't have ODP installed.

As you've seen the loading of the assemblies is buried deep within the Enterprise Library plumbing.  
I tried a few approaches to avoid loading those assemblies but couldn't get around it.  It might be possible to write your own UnityContainerExtension
to not load data access but I'm not how profitable that would be.

--
Randy Levy
Enterprise Library support engineer
entlib.support@live.com 

Jan 26, 2012 at 7:56 PM

thanks for checking on your side as well.  I am going to investigate to exactly why DbProviderFactories.GetFactory("Oracle.DataAccess.Client"); takes so much memory usage. I do have one have question though. In DatabaseSyntheticConfigSettings.cs for GetDefaultMapping,  since I am using ODP.net this will return null.  I am considering doing a check to by pass the GetFactory call. Do you see any downside of doing this or should I be creating a new default provider name?  I am still unsure to the purpose of the function. I don't want to make modification to the source if I don't have to. Please let me know what you think.

I am going to profile the odp.net part to see if anything can be done to reduce memory usage.

Feb 6, 2012 at 9:02 PM

mister pcapiral, any solution about it ? thx