Coverting transaction logs into flat file format.

Coverting transaction logs into flat file format.

In this scenario a user writes entries in their workflow using
either the Transaction Log workflow action or the
soae-xf:write-transaction-log extension function. Either way the user
contents end up in a transaction log looking rather like this:

	
		2010-07-28T10:25:15.119000
		2537861815468032
	
	
		
			
				2010-07-28T10:25:15.124000
			
			
				NODE=poc1-s4|WORKFLOW_ID=CommonOutboundProcessing|TRANS_ID=NO_SOAP_HEADER|PAYLOAD_ID=|REF_TO_TRANS_ID=|REF_TO_PAYLOAD_ID=|BUS_PROC_ID=|INTERACTION_ID=|REF_TO_INTERACTION_ID=|STEP=LogOutboundStart|TIMING_POINT=1280309115124|TIMING_POINT_ID=service-outbound-received|COMMENT=Start of outbound processing;
			
		
	
	
		2010-07-28T10:25:15.149000
	

What the user would prefer is something more like a traditional flat
logfile with just the entries in it for whatever purpose,
be it audit, monitoring or SLA. Plus many traditional tools like
Splunk, Tivoli and OpenView are setup to integrate with flat file
logging rather than XML based logging.

To format your entries is just a pretty simple
XSLT style sheet that looks for tags with a non zero
in the header. The output is then passed to a template
rule which adds a line break between every set of
contents found.

<?xml version="1.0" encoding="ISO-8859-1"?>

	
	
 

	
		
	
 

	
		

		
	
 

We throw away zero entries because they are SOAE's
own test writes to the log file and are probably not of interest to the
user.

However, we have a problem in that if we just copy a
transaction log it's not valid XML until it's been closed by SOAE which
appends a terminating tag. We need to surround the style
sheet with some PERL to tidy the log file and to call an XSLT engine to
format the file. In fact we'll use one of SOAE's XSLT engines in the
PERL script.

#!/usr/bin/perl
#
############################################################################
#
# Name:         logformatter.pl
#
# Author:       Peter Logan, Intel Corporation (UK) plc, SOA Products Group.
#
# Date:         29 July 2010
#
# Description:  Takes SOAE XML formatted transaction log files and turns the
#               user nodes into flat file logs suitable for e.g. Splunk to consume.
#
# Usage:        logformatter.pl [-help] [option]  ...
#
# Documents:    None.
#
# Revision:     Draft version - 29 July 2010.
#
# To do:
#
############################################################################
#
# Includes
use Getopt::Long;       # Passes cmd ln arguements to script.
use File::Copy;         # Does what it says on the tin.

# Initialise variables

# Variables from command line.
$thefile;               # The file to be formatted.

# Get Command line arguments using getopts module.
$cmdline = GetOptions ("help|h" => $help,
                        "f=s" => $thefile,);

# Handle command line options.
Usage_Message() if ($help);
Usage_Message() unless ($thefile);

# Insert the closing log tag if it's required.
open FILE, $thefile or die $!;
while()
    {
    $last_line = $_ if eof;
    }
close FILE;

unless ($last_line =~ m/</log>/)
    {
    open FILE, ">>", $thefile or die$!;
    print FILE '';
    close FILE;
    }

# Now call napa engine to format the file contents.
$outfile = $thefile . "_formatted.txt";
system("/opt/scr/bin/napa1cmdd logformatter.xsl $thefile $outfile");
print "Formatted $outfilen";

exit 0;

# Subroutines
#------------------------------------------------------------------------------
sub Usage_Message
        {
        print < 
Usage: ./logformatter.pl -f file

-f      Name of source file to be formatted for log file consumer.

Example: ./logformatter.pl -f Messaging2_Events:V2010-07-26-1027_adf1280146225178:Messaging2_Events

END_OF_USAGE_MESSAGE

        exit 1;
        }

Okay so now things are starting to come together. We're able to put
the PERL script, style sheet (call it logformatter.xsl) and transaction
log file in the same directory and call it with the following command
line:

./logformatter.pl -f Messaging2_Events:V2010-07-26-1027_adf1280146225178:Messaging2_Events

Yes, don't forget to chmod your logformatter.pl to be executable! We can run it on multiple transaction logs like this:

ls  | while read file; do ./logformatter.pl -f $file; done

The last part is to apply this (and modify to your requirements) to
systems which SOAE runs on. You will probably want to add this to a
crontab which periodically copies your latest transaction logs to
somewhere for processing with the above.

- Pete
1 条帖子 / 0 全新
如需更全面地了解编译器优化,请参阅优化注意事项