Keeping ELMAH's Error Log Size In Check

Published 10 July 09 08:52 AM | Scott Mitchell

Error Logging Modules And Handlers (ELMAH) is a free, open source error logging library for ASP.NET applications that provides automated error logging and notification and, unlike health monitoring, offers a built-in log viewer web page along with a host of other nifty features. If you're not using ELMAH or have never heard of it I highly recommend that you check it out. It's the first thing I add to any new ASP.NET project I start.

As with any sort of logging service is is important that ELMAH's log be periodically pruned. If you let ELMAH's log grow unchecked it can reduce performance when querying the log and suck up disk space, which is especially important in hosted environments where there are typically hard disk quote limits for each user on the database server. The good news is that there are a number of techniques you can employ to help ensure that your ELMAH error log stays a reasonable size.

  • Use error filtering. ELMAH offers a rich set of error filtering rules that you can use to instruct ELMAH not to log certain types of errors, which can help keep ELMAH's log size down. I use error filtering to filter out 404 errors. While logging 404 errors can alert you to broken URLs on your site or others, if you've ever managed an Internet-facing website you know that it's not uncommon to receive a deluge of requests from bots searching for security holes.
  • Setup a weekly job in SQL Server. If your website is hosted in a dedicated environment you can (likely) setup a job on SQL Server. In past projects I've used a job that ran once a week on Sunday at 2:00 AM that would delete all ELMAH log entries older than three months.
  • Update the ELMAH_LogError stored procedure to delete old log entries. This approach works well for applications in a shared environment as you do not need permissions to create SQL jobs, but rather just permission to create/alter a stored procedure. The ELMAH_LogError stored procedure is the stored procedure used by the SQL Server provider for logging errors; it inserts a record into the ELMAH_Error table. Update this table by adding a DELETE statement that deletes all log records older than a certain threshold. For example, the following DELETE statement removes all log entries more than 90 days old.

    DELETE FROM [ELMAH_Error]
    WHERE TimeUtc < DATEADD(d, -90, getdate())


    Add the above statement after the INSERT statement in the stored procedure. Doing so will cause the error log to be pruned of entries older than 90 days anytime a new error is logged. (I've never used an ELMAH log provider other than the SQL Server provider, so I'm not certain what stored procedures or queries or other things would need to be changed to implement such log trimming using an alternate provider.)

Keep in mind that trimming the error log brings with it a tradeoff: you are removing error log entries that might be important for analysis later in time. If this is the case, if you think you might need to review that error log from more than 90 days in the past, then before deleting records from the error log you should archive them somewhere.

Happy Programming!

Filed under:

Comments

# Clayton O. said on November 15, 2010 03:24 PM:

Scott,

This was fantastic and I use the Update example quite often.  Recently we had a problem where we had several thousand records and our modified procedure was timing out before all of the old records could be deleted.  This was not allowing us to add record any new errors and causing some severe performance problems.

After putting in the research it seems that we had a day just over the 90 day mark in which we had several repeating errors.  The issue was corrected but those errors remained in the log.  As it came time to clean them up the procedure just didn't have enough time to get the task done.  We manually ran the DELETE command and found it was taking close to five minutes to delete all of those records.  It was a bad day 90 days ago.

We modified your original query to the following:

DELETE FROM [ELMAH_Error]

WHERE ErrorId NOT IN (SELECT TOP 1000 ErrorId FROM ELMAH_Error ORDER BY TimeUtc DESC)

Granted you can up the number of items you want to keep track of, but we really only need the 1000 most recent records.  We may eventually come up with a better solution, but for now this is working.  Thought I would share my experience should anyone else run into the same problem.

Thanks for the initial idea,

~Clayton

Leave a Comment

(required) 
(required) 
(optional)
(required) 

Archives

My Books

  • Teach Yourself ASP.NET 4 in 24 Hours
  • Teach Yourself ASP.NET 3.5 in 24 Hours
  • Teach Yourself ASP.NET 2.0 in 24 Hours
  • ASP.NET Data Web Controls Kick Start
  • ASP.NET: Tips, Tutorials, and Code
  • Designing Active Server Pages
  • Teach Yourself Active Server Pages 3.0 in 21 Days

I am a Microsoft MVP for ASP.NET.

I am an ASPInsider.