Semantic Logging - RollingFlatFileSink - Real-Time Logging

Topics: Semantic Logging Application Block
Jan 9, 2015 at 4:55 PM
We're currently using semantic logging and are able to both save to rolling flat files and a sql database. A difference we're seeing is that the sql sink inserts every x seconds while the rolling flat file sink doesn't insert data until the file rolls over or the service stops.

Is there a way to have inserts happen in real-time for the rollingFlatFileSink?
Jan 14, 2015 at 3:26 PM
Here's the config file. Are we missing some other property in one of the elements?
<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns="http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xsi:schemaLocation="http://schemas.microsoft.com/practices/2013/entlib/semanticlogging/etw SemanticLogging-svc.xsd">

  <traceEventService/>

  <sinks>
    <!--TokenEventSource-->
    <rollingFlatFileSink fileName="C:\TokenEventSource.txt" name="TokenEventSource" rollSizeKB="5">
      <sources>
        <eventSource name="Namespace.TokenEventSource"
                     level="LogAlways" />
      </sources>
    </rollingFlatFileSink>

    <!--SecurityEventSource-->
    <rollingFlatFileSink fileName="C:\SecurityEventSource.txt" name="SecurityEventSource" rollSizeKB="5">
      <sources>
        <eventSource name="Namespace.SecurityEventSource"
                     level="LogAlways" />
      </sources>
    </rollingFlatFileSink>

    <!--HttpClientEventSource-->
    <rollingFlatFileSink fileName="C:\HttpClientEventSource.txt" name="HttpClientEventSource" rollSizeKB="5">
      <sources>
        <eventSource name="Namespace.HttpClientEventSource"
                     level="LogAlways" />
      </sources>
    </rollingFlatFileSink>

    <!--HttpEventSource-->
    <rollingFlatFileSink fileName="C:\HttpEventSource.txt" name="HttpEventSource" rollSizeKB="5">
      <sources>
        <eventSource name="Namespace.HttpEventSource"
                     level="Informational" />
      </sources>
    </rollingFlatFileSink>

    <!--ServicesEventSource-->
    <rollingFlatFileSink fileName="C:\ServicesEventSource.txt" name="ServicesEventSource" rollSizeKB="5">
      <sources>
        <eventSource name="Namespace.ServicesEventSource"
                     level="LogAlways" />
      </sources>
    </rollingFlatFileSink>

    <!--ExceptionEventSource-->
    <rollingFlatFileSink fileName="C:\ExceptionEventSource.txt" name="ExceptionEventSource" rollSizeKB="5">
      <sources>
        <eventSource name="Namespace.ExceptionEventSource"
                     level="LogAlways" />
      </sources>
    </rollingFlatFileSink>
  </sinks>
</configuration>
Jan 27, 2015 at 9:53 PM
Is this expected behavior? We still haven't figured this out.
Jan 28, 2015 at 3:21 AM
I haven't noticed that. When using the RollingFlatFileSink OOP the sink will be used in synchronous mode which should cause entries to be flushed when received. Perhaps you are having issues with the log file being locked until it is rolled or the service is shutdown.
Feb 3, 2015 at 4:32 PM
We're still seeing this issue after trying several permutations.

All EventSourceAnalyzer.InspectAlls pass
We've tried running the service in the console with flat files and rolling files
We've tried running the service as a windows service under local service account, flat files and rolling files
We've tried running the service as a windows service under system account, flat files and rolling files
The directory where the log files are places have full permissions for both local service account and system account
When the service is started, the files cannot be deleted with an error: "The action can't be completed because the file is open in Enterprise Library....."
We've redownloaded the semantic logging out of process service, grabbed the updates and reinstalled the service on the server.

The worst part is this all works on my computer at home when I go through the process and use the service started in a console. I know this should work but am running out of ideas.
Feb 3, 2015 at 4:56 PM
Edited Feb 3, 2015 at 5:10 PM
Here are some interesting items in the event viewer when installing the service with local service. Installing under a system account I don't get these errors.

EventId : 805, Level : Warning, Message : The configuration was partially successfully loaded. Check logs for further error details., Payload : , EventName : TraceEventServiceConfigurationWithFaultsInfo, Timestamp : 2015-02-03T16:54:06.0236143Z, ProcessId : 5960, ThreadId : 5528

ProviderId : d1ed7ec7-5701-5554-2c5e-796dc42511c5
EventId : 803
Keywords : 4
Level : Error
Message : A fault was detected while processing the configuration for the element 'RollingFlatFileSink'. Message: System.UnauthorizedAccessException: Access to the registry key 'HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Winevt\Publishers{659518be-d338-564b-2759-c63c10ef82e2}' is denied.
at Microsoft.Win32.RegistryKey.Win32Error(Int32 errorCode, String str)
at Microsoft.Win32.RegistryKey.CreateSubKeyInternal(String subkey, RegistryKeyPermissionCheck permissionCheck, Object registrySecurityObj, RegistryOptions registryOptions)
at Microsoft.Win32.RegistryKey.CreateSubKey(String subkey, RegistryKeyPermissionCheck permissionCheck)
at Microsoft.Win32.Registry.SetValue(String keyName, String valueName, Object value, RegistryValueKind valueKind)
at Microsoft.Diagnostics.Tracing.Session.TraceEventSession.SetFilterDataForEtwSession(String providerGuid, Byte[] data, Boolean V4_5EventSource)
at Microsoft.Diagnostics.Tracing.Session.TraceEventSession.EnableProvider(Guid providerGuid, TraceEventLevel providerLevel, UInt64 matchAnyKeywords, TraceEventProviderOptions options)
at Microsoft.Practices.EnterpriseLibrary.SemanticLogging.Etw.Utility.TraceEventUtil.EnableProvider(TraceEventSession session, Guid providerId, EventLevel level, EventKeywords matchAnyKeyword, IEnumerable1 arguments, IEnumerable1 processNamesToFilter, Boolean sendManifest)
at Microsoft.Practices.EnterpriseLibrary.SemanticLogging.Etw.TraceEventServiceWorker.Initialize()
at Microsoft.Practices.EnterpriseLibrary.SemanticLogging.Etw.TraceEventService.<>c__DisplayClass6.<AddWorkers>b__2()
at Microsoft.Practices.EnterpriseLibrary.SemanticLogging.Etw.TraceEventService.HandleException(String callerName, Action body)
Opcode : Info
Task : 64731
Version : 0
Payload : [faultedElement : RollingFlatFileSink] [message : System.UnauthorizedAccessException: Access to the registry key 'HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Winevt\Publishers{659518be-d338-564b-2759-c63c10ef82e2}' is denied.
at Microsoft.Win32.RegistryKey.Win32Error(Int32 errorCode, String str)
at Microsoft.Win32.RegistryKey.CreateSubKeyInternal(String subkey, RegistryKeyPermissionCheck permissionCheck, Object registrySecurityObj, RegistryOptions registryOptions)
at Microsoft.Win32.RegistryKey.CreateSubKey(String subkey, RegistryKeyPermissionCheck permissionCheck)
at Microsoft.Win32.Registry.SetValue(String keyName, String valueName, Object value, RegistryValueKind valueKind)
at Microsoft.Diagnostics.Tracing.Session.TraceEventSession.SetFilterDataForEtwSession(String providerGuid, Byte[] data, Boolean V4_5EventSource)
at Microsoft.Diagnostics.Tracing.Session.TraceEventSession.EnableProvider(Guid providerGuid, TraceEventLevel providerLevel, UInt64 matchAnyKeywords, TraceEventProviderOptions options)
at Microsoft.Practices.EnterpriseLibrary.SemanticLogging.Etw.Utility.TraceEventUtil.EnableProvider(TraceEventSession session, Guid providerId, EventLevel level, EventKeywords matchAnyKeyword, IEnumerable1 arguments, IEnumerable1 processNamesToFilter, Boolean sendManifest)
at Microsoft.Practices.EnterpriseLibrary.SemanticLogging.Etw.TraceEventServiceWorker.Initialize()
at Microsoft.Practices.EnterpriseLibrary.SemanticLogging.Etw.TraceEventService.<>c__DisplayClass6.<AddWorkers>b__2()
at Microsoft.Practices.EnterpriseLibrary.SemanticLogging.Etw.TraceEventService.HandleException(String callerName, Action body)]
EventName : TraceEventServiceConfigurationFaultInfo
Timestamp : 2015-02-03T16:54:05.9923297Z
ProcessId : 5960
ThreadId : 5528
Feb 3, 2015 at 5:42 PM
One more update from today:

I'm able to make this work on my home computer with Windows 8.1. However, I'm not able to make this work on a work laptop (Windows 7) or a work server (Windows Server 2012). I wonder if the work machines are locked down in some way that's preventing this from happening.
Feb 4, 2015 at 5:52 AM
Sounds like permissions. See: Installing and running the Out-of-Process Windows Service/Console Host and https://msdn.microsoft.com/en-us/library/dn775006.aspx for instructions on setting registry key permissions or adding the SLAB OOP Service User to the Performance* group.
Feb 5, 2015 at 5:03 PM
We ended up getting it working. A bit ashamed to say it may have been an oversight on our part. It seems we were looking at the file size in explorer and not seeing any changes so we assumed it was not being written. Apparently it was but we can't be sure if it was always working or if it started working after some additional changes.

We did notice this after we ran through and set permissions in the registry for the account running the OOP service. So that may have still been related as we no longer saw the errors and warnings in event viewer after doing so. Thanks for your help randylevy.