Not able to write a large string of 50000 characters into a flatfile

Topics: Logging Application Block
Mar 30, 2012 at 7:04 PM

I want to write a text message of 50000 characters into a flat file.  Not able to write that into flat file.......can anyone help with this immediately please

Mar 31, 2012 at 12:08 AM

Actually, the problem is...I am not able to understand whats going on in flatfile tracelistener.  I have a text file which have 30000 characters in it.  It is writing properly into flatfile.  When I have a text file with 50000 characters, it is not writing into flatfile.  Not able to understand whats going on in flatfile tracelistener.  Any ideas are really appreciated.

Mar 31, 2012 at 2:02 AM
Edited Mar 31, 2012 at 6:07 AM

Are you sure you are reading the whole file in?  I just did a test that wrote 100,000,000 characters without an issue.

Can you clarify what you mean by "it is not writing into flatfile"?  Does that mean, the data is truncated?  Or that there is an exception?  Or that it appears to succeed but no data is written?

Randy Levy
Enterprise Library support engineer 

Mar 31, 2012 at 2:41 PM

it is not giving any exception...looks like succceeded...but not writing.

Apr 1, 2012 at 1:14 PM


In front end....I am reading the text file of 50KB or 49000 characters in this way

 StreamReader myFile = new StreamReader(filePath);
            string ErrorMessage = myFile.ReadToEnd();

            return ErrorMessage;

I am passing this errormessage as a parameter to my logging service.


 db.AddParameter(cmd, "logmessage", DbType.String, Int32.MaxValue, ParameterDirection.Input, false, 0, 0, null, DataRowVersion.Default, logEntry.LogMessage);


private string logMessage;

public string LogMessage
            get { return this.logMessage; }
            set { this.logMessage = value; }

Database Side

	@EventID int, 
	@Priority int, 
	@Severity nvarchar(32), 
	@Title nvarchar(256), 
	@Timestamp datetime,
	@MachineName nvarchar(32), 
	@AppDomainName nvarchar(512),
	@ProcessID nvarchar(256),
	@ProcessName nvarchar(512),
	@ThreadName nvarchar(512),
	@Win32ThreadId nvarchar(128),
	@Message nvarchar(1500),
	@FormattedMessage ntext,
	@logentryID nvarchar(256),
	@logmessage nvarchar(max),
	@sourceValue nvarchar (256),
	@severityvalue nvarchar (256),
	@typeValue nvarchar (256),
	@LogId int OUTPUT




There are some unwanted things in the above stored procedure.  I am going to remove them later.  For now, our concentration is on Logmessage.

I don't know where the mismatch of datatypes are happening....when I try to do a 30000 chars or 31KB text file....its writing both flatfile and db without any problem.  When I exceed that, it is not giving any kind of exception anywhere....but not logging anything to database or flatfile.

My requirement is to log upto a 1 GB text file to both places.
Apr 1, 2012 at 1:59 PM


Finally, I figured out the problem.  The problem is with the event log.  Actually, I am writing the logs into 3 places....event log, flatfile and db.  I think...event log is having some limitation to hold the data.  So, I just removed the event log...and now i am successfully able to write upto 26MB text file.  


Above 26MB file....the appplication is failing.  It's not writing anything to db or flatfile.  The code is same as above.  Do I need to change anything to make it work?...The requirement is upto 1GB.


Please guide me.

Apr 2, 2012 at 9:08 PM
Edited Apr 3, 2012 at 3:35 AM

Yes, the EventLog has a hard size limit of 32766 on the size of the message:

In terms of reading in 1GB string there is no guarantee that this will succeed (since a contiguous 1GB block will have to be allocated).  Even if you can read in the string then there is still no guarantee that it will succeed.  I'm guessing that you are getting an OutOfMemoryException.  If I read in a big enough file I cause an OutOfMemoryException.

The reason that you could read in the complete file but then have an error in the FlatFileTraceListener is that the trace listener is performing formatting of the string so will almost always create another copy of the string which could lead to an OutOfMemoryException.

One thing to do to help is to use the LogEntry overload instead of the object overload since the object overload will call ToString() and could trigger an OutOfmemeoryException.  The second part is to process the large data in smaller chunks.  Either writing to file or database in smaller chunks to avoid allocating a large block of memory.  Since you already have a custom database trace listener you could modify that but to write to a file you would probably need to create a new trace listener to write to file in smaller chunks than the FlatFileTraceListener (and avoid string concatenation found with Formatters as well).

Randy Levy
Enterprise Library support engineer