Free Download or Try Live

Getting the Maximum from your log4j logs

In this little guide I will cover some of the many ways you can use and benefit from XpoLog 6’s new features and enhancements. I will concentrate mainly on how to get the maximum from your log4j event logs. If you download XpoLog now (totally free of charge), you can easily follow as you go along.
 
 
 
 
 
Download XpoLog Free of Charge
Once your log4j logs have been transferred to and properly defined in XpoLog Center, you can troubleshoot your java application by running Simple, Complex and Analytic Searches on your log4j data, measure your application performance and thread activity, measure code activity with class and method analytics on log4j, build analysis, create your own Apps or use XpoLog’s Apps for better monitoring, and create dashboards, charts, slide-shows, and make use of other visualization gadgets for maximum analysis.
 
I will cover the following subjects:
  • Transferring your log events to XpoLog using log4j - scroll down
  • Defining log4j Patterns for Maximum Analysis – Defining and editing Patterns in XpoLog Center, and defining log4j Patterns in SysLog - scroll down
  • Adding AppTags to log4j logs for easier and more powerful Searches (Simple and Complex Search) - scroll down
  • Auto detecting new errors, exceptions and bugs in log4j logs – from Search to Analytics and use cases - scroll down
  • Application Performance and Availability – use cases - scroll down
  • Examining the Quality of your Java Code – use cases - scroll down
  • Out of box apps and dashboards - scroll down
 

Inside XpoLog

Before I begin, I just want to give you a very quick peek at XpoLog Center, so you can easily navigate your way once you are inside. In this article, I will “visit” the four main XpoLog interfaces to show you how to use the functions I will cover, they are the Manager interface, the Search interface, the Analytics interface, and the Apps interface.

Double Click to edit this text

XpoLog Manager: We have an entire wiki guide for all the functions contained in the Manager interface, but for this particular guide, I will be showing you how to add and edit logs. 
 
XpoLog Search: The Search interface is where I will be spending the most time showing you all the information you can gather from XpoLog’s powerful search engine. 
 
XpoLog Analytics: The Analytics interface shows you details of the analytics found during the search.
 
XpoLog Apps: The Apps interface is where you can view search results with different visualization gadgets, and create your own apps and dashboards to suit your needs.
 

Transferring your log events to XpoLog using log4j

First I will show you how to transfer your log events to XpoLog using log4j. There are two ways of doing this. The first method is to allow XpoLog direct access to your files (PULL). The other method is by defining a SysLog appender and sending your events and messages to XpoLog (PUSH). XpoLog supports both methods.
 

Allowing XpoLog access to your files (Pull)

Assuming you are already using log4j to write your log events to files, to allow for XpoLog to perform analytics on your log data, you need to give XpoLog access to these files. Define the name, pattern, and data pattern so that XpoLog can read these files, collect and index the data, and start analyzing. 
 

Using direct access (Local or Remote) 

XpoLog can access a local log file, i.e. a log file that resides on the same server as XpoLog. XpoLog can also access a log file on a remote server to which it has been provided direct access, as long as XpoLog is provided with the UNC path (\\hostname\dirname) to the log files on the remote server.
 

Using SSH (Secured Shell) 

XpoLog can access log files on remote servers over SSH agent-less, provided that XpoLog has an account with a username and password or private/public key for connecting to the SSH server where the log files are situated.
 
Note that XpoLog requires Read permissions for any log it reads, regardless of the source of the log file.
 
To allow for XpoLog to pull (data from) the files, define the logger and give XpoLog access to the remote server where the logger is defined; then add the log to XpoLog.
 

Defining the Logger:

For example:
 
#Logger definition
log4j.logger.mylog=ERROR,mylog
log4j.logger.com.mylog.logcollection=ERROR,mylog
#Appender data for mylog
log4j.appender.mylog=org.apache.log4j.RollingFileAppender
log4j.appender.mylog.File=${home.path}myappoutput.log
 
log4j.appender.mylog.MaxFileSize=5000KB
log4j.appender.mylog.MaxBackupIndex=10
 
log4j.appender.mylog.layout=org.apache.log4j.PatternLayout
log4j.appender.mylog.layout.ConversionPattern=[%d] [%t] [%p] [%c] [%l] %m%n
 
(where d = date, t = thread, p = priority, c = class, l = method, m = message, and n = new line) 
 

Adding a log to XpoLog:

1. Inside XpoLog Center, go to Manager > Administration > Add Log
 
The Add Log screen opens.
 

2. Give the log a name and a parent folder, and select an AppTag (Tag to Application(s)) from the drop-down list or create a new AppTag. You can select and create any number of AppTags for the same log. You do not have to tag the log at all, but later on I will show you just how useful these AppTags can be.  

3. Select the log type to be Local and give a path (the screen capture below shows the example given).

 
4. Click Next to view the sample text from the log, the conversion pattern, and the log records analysis results, or click Save.
(Later, I will give details regarding editing the pattern in the Pattern Editor.)
 

Sending your log4j log events to XpoLog (PUSH)

To send log events and log messages to XpoLog through SysLog, define a SysLog appender that uses the XpoLog server as the SysLog host. From inside XpoLog Center, define a TCP or a UDP SysLog Listener account and make sure the port (usually 1468 for TCP or 514 for UDP) is open on XpoLog’s machine. We recommend using TCP.
 

Defining a TCP SysLog Listener account:

1. Inside XpoLog Center, go to Manager > Administration > Listeners. A Listeners accounts console opens and presents all the configured listeners available.
 
2. Click Syslog TCP. The Syslog TCP Account window opens.
 
3. Add a descriptive name for the Listener account and click Advanced Settings:
 
4. Under Advanced Settings, do the following:
 
General Information:
Enabled: Make sure the account is enabled.
Listening Interface: Select the IP address for the XpoLog listener instance.
 
Dynamic Log Creation Configuration:
Parent Folder: Select the parent folder which all logs from this listener will be placed under in the XpoLog Folders and Log trees.
Collection Policy: Select the collection policy which will be associated to all logs from this listener (used mainly for storage location and data retention).
Log Name Prefix: write a prefix which will be added to any of the logs from this listener (used to easily distinguish between multiple listener accounts logs in cases where several are created).
Split by Source Device: check to create a log for each unique source device value in the received Syslog messages or events (a log will be created per device with its associated events).
Split by Facilities: check to create a log for each unique facility value in the received Syslog messages or events.
 
Listener Data:
Listener Data Location: Browse to and select the location which data will be stored to, by default XpoLog stores it in its data directory.
[Optional] Indexing Node: This is the node in the cluster which will index the received Syslog messages (appears only if an XpoLog cluster is deployed).
Indexing Interval: Select the frequency at which you want the (received) Syslog messages to be indexed.
 
5. Click Save. The data received from the Syslog listener account will be placed under the configured parent folder you selected.
 
Now all you need to do is to make sure the SysLog events from your java application are sent to XpoLog. Configure log4j to use a SysLog appender. Here is an example configuration: 
 
log4j.rootLogger=INFO, SYSLOG
 
log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.syslogHost=119.0.0.8
log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.conversionPattern=%d{ISO8601} %-5p [%t] %c{2} %x - %m%n
log4j.appender.SYSLOG.Facility=LOCAL1
 
If you chose to PUSH your log events using SysLog, define your events and log patterns before they reach the XpoLog Center.  After the events have reached XpoLog, you are given the opportunity to edit the log pattern if you so wish. By creating the most readable data you will allow for XpoLog to perform the highest detailed analysis of your logs.
 
Since logs are written in free format, XpoLog has an advanced built-in mechanism to detect the structure, or pattern, of the incoming log. But as a user, you can edit and fine-tune these patterns to suit your needs.
 

Defining Patterns in SysLog Appenders

When sending events to XpoLog through SysLog, be sure to create a detailed conversion pattern while configuring your log4j SysLog appender. Here is an example:
 
    #Logger definition
    log4j.logger.events=INFO, SYSLOG
 
    #Appender data for syslog
    log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
    log4j.appender.SYSLOG.syslogHost=127.0.0.1
    log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
    log4j.appender.SYSLOG.layout.conversionPattern=[%t] %c - %m%n 
    log4j.appender.SYSLOG.Facility=LOCAL1
 
(t = thread, c = class, m = message, and n = new line)
 
The SysLog appender will write this event logger to the SysLog. Remember to define a SysLog Listener account inside XpoLog Center.
 

Defining and Editing log4j Patterns for Maximum Analysis

Now that your logs are in XpoLog, I will show you how to define log4j logs in XpoLog Center, to create the most readable data and thus allow for XpoLog to perform highly detailed analysis of your logs. You can define and edit the patterns in XpoLog Center.
 

Defining Patterns in XpoLog Center

To allow for XpoLog to access and pull data from your files, after defining the logger with a name, pattern and data pattern, define the log patterns in XpoLog Center.
 

Defining the log pattern in XpoLog Center:

1. In XpoLog Center, add a new log. See my instructions above. Once you have filled in the details, click Next (rather than Save) to get to the Log Pattern screen.
 
The Log Pattern screen is divided into 3 parts: 
 
a) Sample text from the log
b) Pattern Editor – this is where you can edit the pattern, or add a new pattern. You can also toggle between the Wizard view and the Manual view. 
c) Log record analysis result
 
2. In the Wizard of the Pattern Editor, define the log pattern.
 
Click Manual in the Pattern Editor and edit the XpoLog data pattern to comply with the log4j layout:
 
a)  [%d] = [{date:Date,locale=en,yyyy-MM-dd HH:mm:ss,SSS}]
b)  [%t] = [{text:Thread}]
c)  [%p]= [{priority:Priority,DEBUG;INFO;WARNING;ERROR;FATAL}]
d)  [%c]= [{string:Class}]
e)  [%l]= [{string:Method}({text:Source}:{number:LineNumber})]
f)   %m = {string:Message}
g)  %n =  new line
The XpoLog pattern in our example will be:
 
[{date:Date,locale=en,yyyy-MM-dd HH:mm:ss,SSS}] [{text:Thread}] [{priority:Priority,DEBUG;INFO;WARNING;ERROR;FATAL}] [{string:Class}] [{string:Method}({text:Source}:{number:LineNumber})] {string:Message}
 
3. Click Save.
 
You can also edit the pattern after you have added the log. 
 

Editing Patterns in XpoLog Center 

When the events arrive at XpoLog Center; they are written internally. Here is what they might look like when created by the XpoLog SysLog listener:
 
XPLG:[1436716542132] [local1] [INFO] [test-1] []: [http-30303-Processor24] audit - [Master] [-] [LOGIN] [login/logout] [SECURITY] [http-30303-Processor24] [-] [-] [-] [-] release user admin
 
XPLG:[1436716542140] [local1] [INFO] [test-1] []: [http-30303-Processor24] audit - [Master] [Admin] [LOGIN] [login/logout] [SECURITY] [http-30303-Processor24] [EDA6FECA79A7BBB4480BAFC0FFB911F1] [administrators] [127.0.0.1] [127.0.0.1] login with username admin ok
 
The text at the very beginning is the extra data added by the XpoLog Syslog listener. The other parts of the text in the SysLog file correspond to the layout you created in the log4j SysLog appender (follow the color scheme). 
 
Once the data arrives into XpoLog, a log is created with the default SysLog pattern:
 
XPLG:[{timestamp:Timestamp,MM/dd/yyyy HH:mm:ss.SSS}] [{text:Facility}] [{priority:Level,DEBUG;INFO;WARN;ERROR;FATAL}] [{text:Source Device}] {block,start,emptiness=true} {string:Message}
 
Edit the log and set the pattern to reflect the layout you defined in the log4j configuration.
 

Editing the pattern in a log in XpoLog:

1. In XpoLog Center, go to Manager > Administration and find the log under Folders and Logs in the tree in the left margin. Right-click on the log and select Edit
 

The Edit Log screen opens.

2. There is no need to change any of the basic info. Click Next to get to the Log Pattern screen (see the instructions for defining a log pattern above). The pattern can be edited in the Pattern1 field of the Pattern Editor, or you can add a new pattern in addition to the existing one by clicking the New tab.
 
You can add as many patterns as you want by clicking the New tab. XpoLog will save all these patterns as templates for forthcoming logs.
 
3. Click Save.
 
In the screen capture below you can see how to define the log data pattern. It is displayed in the Pattern1 field. The pattern for this log is the following:
 
XPLG:[{timestamp:Timestamp,MM/dd/yyyy HH:mm:ss.SSS}] [{text:Facility}] [{priority:Level,DEBUG;INFO;WARN;ERROR;FATAL}] [{text:Source Device}] {block,start,emptiness=true}{text:Application Name}[{text:Process Id}]: {block,end,emptiness=true}{string:Message}
 
Most of the pattern, up to and including {block,end,emptiness=true} is part of the SysLog protocol and functions as an prefix to the message – it contains the SysLog timestamp, facility, priority and the source device.
 
As mentioned previously, you can edit the pattern inside XpoLog Center after the event logs have been sent. If your messages all follow the same structure, we recommend further editing the pattern to include this structure, to receive a more refined parsing. Here is a more refined pattern of the log shown above:
 
XPLG:[{timestamp:Timestamp,MM/dd/yyyy HH:mm:ss.SSS}] [{text:Facility}] [{priority:Level,DEBUG;INFO;WARN;ERROR;FATAL}] [{text:Source Device}] {block,start,emptiness=true}{text:Application Name}[{text:Process Id}]: {block,end,emptiness=true}[{text:ServerIp}] [{text:User,User}] [{choice:Action Type,LOGIN;VIEW;CHANGE}] [{text:Action description,Action description}] [{choice:Context,LOGS;FOLDERS;VERIFIERS;CONFIGURATION;SECURITY;REPROTS;TASKS;JOBS;
NODES;SEARCH_ENGINE}] {string:Message}
 
The following screen capture shows the same log as above, after editing. You can see the original message has been split into the relevant columns:
 
By creating the most readable data, you will receive the most detailed analysis of your logs from XpoLog. 
 

AppTags and Search

Adding AppTags to your logs in XpoLog can make any search, simple or complex, extremely powerful.
 

Adding AppTags to log4j logs

When adding a log to XpoLog, you are given the option of tagging the log to a number of applications, and you can also create new AppTags.
 
Once your data has been transferred into XpoLog and been properly defined with regards to pattern and AppTags, you can start searching. You can do a Simple Search or a Complex Search. A good and detailed pattern together with well thought-through AppTags for a log will make either search that much more effective.  To show you the power of the AppTags, I will begin by drawing a sketch: 
 
Imagine you have three servers, Server1, Server2, and Server3, and plenty of log4j files in each server. There are times you want to search in all of the files in all of the servers, and times you want to search files in one or two of the servers. 
 
Inside XpoLog, create a log for each server, call them log4j_server1, log4j_server2, and log4j_server3. Then create an AppTag for all three logs and call it Log4J. Create an AppTag for the first log called “Georgia” and an AppTag for the second and third logs called “Atlanta”. Imagine that Georgia and Atlanta are the locations of your servers. 
 

Inside XpoLog Search

We are now moving into the XpoLog Search interface. At the top is the search field where you enter your search query. In the main section is a graph depicting the results across the selected time span. You can zoom in on specific points in time to get a list of all errors by hovering over the graph.
 
Under the graph is a table listing all events from all the logs where the anomaly from the search query was found. In the side bar, Analytics Insight lists other anomalies found during the search. You will see later how powerful this feature is, as XpoLog is already analyzing your logs before you even asked it to. 
 

Simple Search examples

Let’s do a simple search where we utilize the AppTags in the sketch above. Say we want to look for anything that appears in the log4j files in Server1. Inside XpoLog Center, go to Search, in the search field, type:
 
* in log.log4j server1
 
* = anything, hence, in this example, the * represents all anomalies in Server1. 
 
Now look for the word “ERROR” in all the files on all 3 servers. Inside XpoLog Center, go to Search, in the search field, type:
 
error in log.log4j server*
 
* = anything. Hence, in this example, the * represents any number or name you gave any server, in our example, the * refers to the numbers 1, 2, and 3.
 
The result will show you where the word “ERROR” appears in any file in any of the three servers.
 
 
Now look for the word “remote” in any file on any server that is situated in Atlanta. In our example this means Server2 and Server3. Inside XpoLog Center, go to Search, in the search field, type:
 
remote IN apptag Atlanta
 
The result will show you where the word “remote” appears in any file in any server that has been tagged with “Atlanta”.
 
The reason this AppTag is so useful is that should you add more servers in Atlanta, or Georgia, and you want to just continue looking for texts or abnormalities in these servers, the moment you give the new server the AppTag “Atlanta” or “Georgia”, XpoLog will continue its search and automatically include searching through any files placed on the new servers. The same goes for removing a server. Once a server is removed, XpoLog will automatically continue its search through all files on all servers that are still there. No further configuration is necessary.
 
Now let’s look at an example which takes the pattern into consideration. We already saw how editing the pattern adds columns to the Log records analysis result field. In the Pattern Editor, we added the priority. Let’s conduct a search where we look for all log4j logs where the priority is ERROR. Inside XpoLog Center, go to Search, in the search field, type:
 
priority=error in log.log4j server* 
 
* = anything. Hence, in this example we are searching through all the files on all the servers.
 

Complex Search example

When doing a Complex Search, XpoLog will first refine the data collected in the Simple Search by filtering the result events according to their source (logs, files, applications, or servers). Then it will go on to performing advanced complex operations and reporting on these log events according to the criteria you ask for.
 
XpoLog automatically detects errors in the search results and presents these as suggestions (tagged to low/medium/high severities) next to the search results. This technique is known as “Integrated Layers” and it boosts troubleshooting and exposes issues in the logs you may never have thought of and this in turn helps you find the source of various problems faster.
 
Running a complex search query results in a summary table, presented in a tabular format, and you can also create dashboards and other visualization gadgets for an easier, more natural view.
 
As an example, let’s look for the word “ERROR” or “Exception” in all log4j files on all servers, but also ask XpoLog to count how many errors (or exceptions) were found in each class. In the search field, type:
 
error or exception in log.log4j server* | count | group by class
 
The result will show you a table with a list of all classes where errors were found, and how many errors in each.
 
Complex Search provides the option to aggregate log data and to generate advanced statistics, trends, business intelligence, and transactions analysis on the log data.
 

Auto detecting new errors, exceptions and bugs in log4j logs

Let’s have a look at the basics of XpoLog Analytics, how to auto detect new errors, exceptions and bugs in log4j logs, and discover unknown messages.
 

From Search to Analytics

You may have noticed that when getting results from the Simple and Complex Searches using XpoLog’s powerful search engine, XpoLog suggests analytic insights you may be interested in investigating further. Look in the side bar at the Analytics Insight list.
 
To put it simply, while XpoLog’s Search Engine gives you everything you asked for, XpoLog Analytics gives you everything else.
 
If you are searching for a string, a thread, an error, etc. in one or any number of logs, folders, applications, or servers, within a given time frame, XpoLog’s search engine will find and display all cases/events of the search request within all logs. But XpoLog will also open the door for any other abnormality that may occur in these logs within this time frame, and this brings us to Analytics.
 
 In other words, as soon as you conduct a search, XpoLog will already automatically present you with all other issues that you did not even know existed, be it errors, exceptions, unknown messages, or any other anomaly. 
 
As an example, look at the simple search we did before, looking for all log4j logs where the priority was ERROR.
Quick recap: Inside XpoLog Center, in the Search interface, in the search field, we typed:
 
priority=error in log.log4j server* 
 
The result looked like this:
Below the graph, XpoLog displays all the events where the priority=ERROR.  But in the side bar (see red rectangle), XpoLog has already suggested Analytic Insights, such as ERROR, java.io.EOFException, … and the list goes on. These Insights may not necessarily appear in any of the events where priority=ERROR, but they do appear somewhere in these logs, and hence may indicate that something went wrong somewhere.
 
So already, at this stage when all you are doing is searching for something, XpoLog is already several steps ahead, analyzing, and inviting you to dig deeper to find the root of the issue.  From the Analytics Insight list, we can select one or more insights and either add them to the search, or use them to replace the search. We can then investigate the matter further, in XpoLog Analytics.
 

Inside XpoLog Analytics

The screen capture below shows the Analytics interface. The top section has a graph showing you the data distribution and the maximum severity of the events over the selected time-span. Below the graph is a table showing the logs and folders in which these events were found. Below this section is another table showing the 10 most severe errors that were found in any of these logs and folders. 
 
When listing the logs, XpoLog lists the logs containing the problems with the high severity first (red), then all those with medium priority problems (orange), and lastly, those containing low priority problems (green). XpoLog decides the severity level according to the highest severity anomaly found in the event. You may be searching for an anomaly with a medium severity, but if, in the same event, another anomaly with high severity is found, the event as a whole will be marked as high priority.
 
In the screen capture above, a search was conducted for Failed to initialize hudson, which has medium priority, but within the same event, XpoLog found a hidden message, java.lang.OutOfMemoryError, which has high priority, thus bringing the entire event up to high priority.
 

Drilling down

From the initial Analytics page, which by default shows the total summary of all anomalies in all logs of your search, you can drill down for more specific details. For the sake of our example, let’s drill down into log4j:
 
The Analytics page has now drilled down to the log4j level (see screen capture below). You can see the number of anomalies has been reduced, as has the amount of data being depicted. The first table below the graph now contains only folders of the log4j applications and the second table shows the most severe log4j problems found.
 

Log4j Use Cases

Let’s have a look at a potential use case. Inside log4j is a tomcat folder.
 
A user is complaining that tomcat will not start. We don’t know what the problem is, so the easiest way to find out is to do a general search for anything abnormal going on in tomcat in the given time frame when the user was unable to start it.
Inside the XpoLog Search interface, in the search field, we type the following query: 
 
* IN folder tomcat 8
 
In addition to the requested search results, XpoLog suggests many more analytic insights. There could be many reasons why tomcat did not start, so from the Analytics Insight list, we will select java.net.BindException and replaced the * search with this query:
 
"java.net.BindException" IN folder tomcat 8
 
The screen capture below shows how to replace the existing search query: right-click on the insight and then select Replace search:
 
The search result for "java.net.BindException" IN folder tomcat 8 will look as follows: 
 
We now have completely new events in the list and we see that for the insight "java.net.BindException" that the “address is already in use”. This is why the user was unable to start tomcat.
 
***
 
In another example, we created an App called hudson. Hudson worked for a while, but at some point it stopped. We did not know why, so we did a search for hudson. In the search field, we typed:
 
hudson IN folder tomcat 8
 
We got the following results:
 
Here we can see in the Analytics Insight list that there is a high priority error: Java.lang.OutOf MemoryError. In this particular example, this error occurs in the first event on our list of events containing “hudson”. We now already know why hudson stopped working. By zooming in and hovering over the graph, you also get a presentation of the high, middle, and low priority errors per moment in time.
 

Bottom line…

We can see from these examples and use cases that while users are simply searching for known anomalies, or even searching for any anomaly without pre-knowledge, XpoLog is already several steps ahead analyzing everything else about the logs in the search that the user never thought of. In the fraction of a second it takes XpoLog to do the requested search, a complete list of analytic insights are also created, waiting for the users and inviting them to dig deeper into their logs, folders, apps or servers, to get to the root of whatever is causing them trouble.
 
This is why we call our searches “Analytic Search”…
 

Application Performance and Availability 

I will now show you how XpoLog can measure your application performance and availability to the finest detail in just a few lone clicks with a built-in transaction detection engine.
 

Preliminary

For the sake of our use cases, we selected one of our java applications that runs various types of processes. This application uses log4j to log its activities. One of these logs is called system audit. System audit notes all the various stages of the different processes executed by the system.
 
We added the log in XpoLog Manager:
We already covered the full Add Log procedure earlier (scroll up to view again).
 

Application Performance 

Every second, tens, hundreds, and even thousands of transactions are logged in system audit. We want to measure the duration of each transaction and find out how many of them took more than 10 seconds to complete. We want to count how many such “slow” transactions occurred per hour.
 
In the Search interface, in the search field, type:
 
* IN log.system audit | transaction ("start job", "Thread","Job Started" -> "end job","Thread","Job Completed") | where transaction time > 10000 | count | interval 1 hour 
 
Where: 
  • transaction is an XpoLog function which maps the system processes in the log and displays a flow of events.
  • transaction time is measured in milliseconds, so 10000 is 10 seconds.
 
We got the following result:
We can tell from the graph that the times when we are likely to experience poorer performance are between 8 AM and 6 PM. The table below the graph lists the number of transactions that lasted more than 10 seconds, per hour.
 
Let’s say you were to decide that more than 50 occurrences of a transaction taking more than 10 seconds would cause an overall delay. You can easily tell from the results (graph and table) when the times you are most likely to experience poorer application performance are. See the areas marked in red:
 

Application Availability

Let’s continue with another system audit use case and check its availability. We want to find out which system processes never completed, and when they took place.
 
In the search field, type:
 
* IN log.system audit | transaction ("Set job state to Working Job State", "Thread","Job Started" -> "Set job state to Finish Job State","Thread","Job Completed") | where job completed = null
 
Where: 
  • transaction is an XpoLog function which maps the system processes in the log and displays a flow of events.
  • null means missing. For this search, “job completed = null”, means that the “Job Completed” step is missing from the flow.
We got the following results:
 
We see from the graph when the most occurrences of the transaction failing to complete successfully took place.
 
The table below the graph lists the exact times the transaction failed to complete successfully.
 

In a nutshell

I have already shown you how a search, simple or complex, can give you results far beyond the expected, and how XpoLog is always many steps ahead analyzing your data and suggesting analytic insights you never even thought of could be what’s causing you trouble. But now you have also seen that XpoLog’s search engine has a built-in transaction detection engine, automatically giving you insights regarding the application performance and availability.
 

Examining the Quality of your Java Code 

I will now move on to show you how XpoLog can help you find errors in you java code, which in turn will help you create better quality code.
 

Preliminary

We are using the same java application as before, the one that runs various types of processes and uses log4j to log its activities. Another one of these logs is called errors, and it notes all the errors found in each of the classes of the code.
 
We will now see the benefits of the parsing process we did earlier. Quick re-cap:
 
Remember that XpoLog has an advanced built-in mechanism to detect the structure, or pattern, of the incoming log. But as a user, you can edit and fine-tune these patterns to suit your needs
 
You can create a detailed conversion pattern while adding your log4j log in XpoLog Center, setting the pattern to reflect the layout you defined in the log4j configuration. For the sake of our use case, we made sure to parse our log error to include separate columns for the Priority (ERROR, in our case) and the Class. This screen capture shows the Log Viewer in the Manager interface:
 

Investigating the Quality of the Code

We now want to check which classes in the code of our application wrote the most errors. By locating these errors quickly, we can create high quality code significantly faster. We do this by looking into the log error.
 
For a general view, we want to see the number of errors over a certain time span, just to make sure there are not too many.
 
In XpoLog Center, in the Search interface, in the search field, type:
 
priority = error in log.errors | count | interval 1 hour  
 
Since the log may also include anomalies that are fatal, we specifically search for the anomalies that are “just” errors. 
 
We get the following result:
 
 
 
We now want to see which classes are responsible for the majority of the errors.
 
In the search field, type: 
 
priority = error in log.errors | count | group by class | order by count desc  
 
We get the following result:
 
We see that the class with the highest number of errors is xpolog.eye.data.filters.QueryFilter.
 
We would like to have a closer look at this class, and XpoLog gives you a very easy way to drill down into this particular class (or any other class on the list for that matter). All you need to do is click on the link (in the table) to xpolog.eye.data.filters.QueryFilter.
 
Once you click on the class, XpoLog will change the search-query in the Search field by adding this class to the query with the action Add to search (AND). The search field will now read as follows:
 
(priority=error) AND class contains xpolog.eye.data.filters.QueryFilter IN log.errors
 
We get the following (drill down) result:
We can now investigate the errors specifically found in the class xpolog.eye.data.filters.QueryFilter.
 
Perfect code is seldom found, but our quest for exceptionally high quality code has now been given a powerful boost.
 

Out of the box Apps and Dashboards

After all our searches and use cases, let’s have a look at how we can view all this data and information using different visualization gadgets. To do this, open the Apps interface.
 

What is an App?

First of all, what is an XpoLog App? The XpoLog App is like a container. It can contain any number of dashboards, and each dashboard can contain any number of gadgets. The gadgets can be a visual display, such as a graph or a chart, but it can also be textual information such as a table. From the moment a gadget is saved, the results are automatically updated at regular intervals.
 
XpoLog has also created unique out-of-the-box apps, some of them tailored specifically for log4j. These apps are ready for you to use, to quickly receive insights on your data, such as performance, quality and compliance.
 

From Search to App (and back)

XpoLog lets you take a search result directly into Apps and vice versa. Let us have a look at what our last search would look like in an App.  What you will do now is create a new gadget and place it in the relevant Dashboard and App by saving the search as a gadget.
 

Saving a Search as a Gadget:

1. From the actual search result in the Search interface, click Save Gadget from the top right of the screen. The Save Gadget window opens. (We are looking again at the log error.)
 
2. In the Save Gadget window, do the following:
  • In the Gadget Title field, give your gadget a meaningful name.
  • In the Time Range fields, either keep the same time range or select a new one from the drop down.
  • In the Gadget View, select either Chart or Table.
  • In the App fields, either select one of the existing apps or create a new one. Once you have selected, you will be asked to give a name to the new dashboard.
3. Click Save and View Dashboard.
 
The Apps interface opens and you can view your result: 
 
 
Creating your own Apps 
 
Saving a search as a gadget is not the only way to create a gadget. From inside the Apps console, you can create your own Apps, Dashboards and gadgets.
 

Creating an App:

1. From the Apps console, click on the Add New App icon. 
 

A new App icon opens next to it.

2. Give the new App a meaningful name. Open the new App by clicking on the icon.
3. Inside the new App, click the Add New Dashboard icon.

A new Dashboard icon opens next to it.

4. Give the new dashboard a meaningful name. Open the dashboard by clicking on the icon.
The Add A Gadget screen opens.
You can add any type and any number of gadgets to your dashboard, but if you want to use the gadget for a specific search it is best to check that the gadget is appropriate. For example, in our example, we used the XpoLog transaction function, so the best gadget to select would be the transaction visual type.
 
5. Once you have added several gadgets to your dashboard, you can view them simultaneously. You can keep adding more gadgets for different views. Click the plus-button in the top right corner and select Add Gadget. This is an example of the same search shown in many visualization types:
 

Viewing your Apps

When viewing your apps and dashboard, you can change the view in a number of ways, such as:
  • Changing the time span:
An example of changing the time span: If we started by searching for errors in the last 7 days of a search but want to narrow it down to the last 24 hours, and perhaps the last hour, simply click the          icon to select a different time frame.
 
Here is an example of 4 Spline charts of the same search query but the time frame is narrowed down from the last 7 days to the last hour:
 
  • Creating a slide show (this is useful after creating several dashboards):
 

Slide show in movement:

  • Speaking of seeing several graphs (gadgets) simultaneously, you can drag and drop and move them around as you please using the drag and drop icon        in the top left corner of each graph.
 
Other ways to view your apps are:
  • Changing the layout of the screen to show many gadgets simultaneously, here is an example of 3 columns of graphs/charts:
 
 
  • Inverting the color theme:
  • and more…

Other Cool App Functions

One of the useful functions of the self-created custom Apps is the Scheduled Export. Once you have created an App you like you can schedule it to send you reports as often as you wish. For example, every morning at 9 AM you want to receive a report for a specific search for the last 24 hours. The App can create reports and export them to PDF, CSV, and directly to your email.
 
To schedule these reports to be sent, all you need to do is click Edit App (either from the top menu bar or the side bar, depending where you are), and then, in the Export Settings section, select the Exporting Frequency. From there, you can fill in all the other details needed to create the reports.
 
Another great App function is how you can custom filter a dashboard result to suit your needs. Let’s say you have a dashboard showing the results of a search you did, but you want to slightly alter the search without going back to the Search interface.
 
Click the search icon         and enter the criteria that you want to add to the gadget's queries. Click the 'Search' to activate the filter. The gadget will show the new result accordingly:
 

Final words

I have shown you how to send your data to XpoLog, how to add logs and edit logs, how to parse them and to add AppTags to them. Then we did some Simple and Complex Searches, I introduced you briefly to Analytic Search and XpoLog Analytics. We saw how XpoLog’s Analytical Insights can find numerous anomalies you never even thought of looking for, and can give you valuable information regarding you application performance and quality of code. Last, but not least, we took a quick peek at viewing all this information in Apps.
 
I now suggest you have a look at and experiment with XpoLog on your own. You can download our software in just a few minutes and it’s absolutely free.
 

If you need any help at all, configuring, navigating, or anything else, we are here to help you, support@xpolog.com

Download XpoLog Free

Notice the View in Search link in the bottom right corner. Clicking this link brings you (back) to the Search interface.

Now let’s look at the gadget for another of our examples, system audit. First, save the search result as a gadget:

In the Save Gadget window, in the Gadget Title field, give your gadget the name transaction where job completed = null, and in the Apps field call the App System Audit and the Dashboard Transactions. If you save several gadgets for several time frames, and place them all in the Transactions dashboard, you can see several results in the Apps interface:

Here the View in Search link is in the bottom each individual right corner of each gadget. Clicking this link brings you (back) to the Search interface. (Note that if you created the App from the Apps interface, you can also see it in the Search interface by clicking the View in Search link.)