Creating Custom Logs in Your Applications – What Events Should You Be Tracking

It’s not uncommon for developers to code their logging modules into applications. They do this to compensate for limitations in standard logging functionality available with an operating system.

For Instance, Windows records application errors in Event Viewer logs. These logs provide information for developers and administrators to review what went wrong, but they don’t always provide the level of monitoring needed for an enterprise application.

Therefore, if you need to monitor an enterprise application, you’d most likely want to make use of custom logs.

Here are a few events that you don’t want to skip in your custom logs:

• Poor Performance and Load Times
• Failed Events During Checkout and Sales
• Cyber Security and Suspicious Events

Poor Performance and Load Times

An operating system will log any events where the application crashes due to a timeout, but these logs only tell you when the application crashes not when it’s showing signs of performance degradation.

As your user base grows and the application supports more traffic, your server hardware and resources should scale. It’s not uncommon for administrators to forget server resources as the business scales up, and this mistake can lead to long application timeouts.

With custom applications, you can write events that give administrators a heads-up when the application is taking too long to load. You determine an acceptable time for pages to load, and then create events in a log that indicate the application is taking too long to process.

For instance, if a page should only take five seconds to load, then you should write events to a log when load times reach three seconds. Administrators can monitor these events and scale resources when the server needs better CPU power, memory resources, and bandwidth.

Another advantage to logging and monitoring these events is that having them as a warning limits the possibility of your application crashing, which puts stress on your IT administrators, developers, and business revenue.

Failed Events During Checkout and Sales

Most application logs capture pages that crash, but the most important of these pages are the ones that guide a user through the buying process. It could be a page where a user contacts your sales team, or it could be one that takes a user through an e-commerce store purchase.

When one of these pages crashes, you lose revenue. One way to salvage or prevent such situations is to log events and capture the data that would otherwise be lost. With the lost information, you can then contact the buyer and complete the purchase.

When you visit any popular e-commerce store, and you’ll see that the shopping cart process is usually cut up into sections. In these sections, the user can enter some information and click “Next”; the shopping cart process is also often cut up into sections where the user enters some information and clicks “Next” or “Continue” to go to the next step in the process.

After each step, information is stored in the database. Even with this information, you still may not know why the application failed or at what step the user could no longer complete a purchase.

Logging failed sales events lets you review information from a client and identify where the sales page failed. It could be from a user-generated error such as an expired credit card, but it could also be from a bug in your application.

For instance, you could have a bug in the payment module in your application that rejects a payment with a legitimate credit card. Operating system logs won’t record this type of error, but your custom application logs can help you identify these logic errors.

Cyber Security and Suspicious Events

Any application open to the public internet should have heavy monitoring for security events. Logging suspicious events is one of the ways you can identify when a cyber-security attack is occurring.

You should have other security measures on the server and the network and coded into the application, but logging events can give you clues into an ongoing attack. Logging events can help you with forensics after a critical data breach.

The most challenging part about this type of logging is that identifying attacks from the application can lead to many false positives.

Additionally, developers must know what constitutes a suspicious action and what is legitimate traffic. Developers are not usually cyber-security experts, so it takes some collaboration between developers and administrators to get these logs right.

As a simple example, one suspicious event is a user that continues to enter an incorrect password. Some users forget their password or need a few attempts before getting it right. You could log the number of times a user incorrectly enters a password before they are successful. If these number of attempts ads up to a certain threshold, then it could alert administrators.

Logging single events is an excellent way to identify possible attacks, but most hackers launch attacks that go after numerous users at one time.

There are effective ways to detect these attacks:

• Log bot-login events: You can code this type of detection by understanding the way bots work. These account takeover tools (ATOs) takes a list of users, their credit card numbers and passwords and log into accounts continuously to test their viability.
• Log multiple failed-login events: One way to detect these tools is to log failed login events that have only a few milliseconds in between attempts. Another way is to log multiple failed credit card purchases in a short amount of time on a specific product.
• Log useragent with failed-login events: Usually, a bot does not pass a useragent variable to the website application. These types of events don’t display common user behavior patterns so they are useful for forensics or detecting a possible ongoing data breach.

What Should a Logged Event Include?

After you identify the events that you want to log, you need the right information in your log files for them to become effective forensic tools. You can log events to any location—a database, text files (make sure that they are secure), or an operating system logger such as Windows Event Viewer.

The following data points should be the baseline for your logs:

• Hostname
• IP address
• Date and time of the event
• The application name (if you have multiple applications)
• The error returned by the application
• Raw exception data from the operating system
• Customer number if available

After you create your logs, you must aggregate them to a shared location. Remember that logged events are a source of data breaches, so they should be highly secured and restricted to only administrators.

One More Thing

You can also use a third-party service to stream your logs to a SaaS system that lets you search them for specific events. Once you set up custom logs, you’ll find that you can much more effectively identify security events, salvage lost sales, and debug your system before application errors cause critical downtime.