mod_logxml - Log XMPP Packets to XML File

Name: mod_logxml
Purpose: Log XMPP packets to XML file
Author: Badlop
Type: Module
Requirements: ejabberd 0.7.5 or newer
Download: ejabberd-modules
	DESCRIPTION
	-----------

This module sniffs all the XMPP traffic send and received by ejabberd,
both internally and externally transmitted. It logs the XMPP packets
to a XML formatted file. It's posible to filter transmitted packets 
by orientation, stanza and direction. It's possible to configure the 
file rotation rules and intervals.

This module reuses code from mod_log_forensic, mod_stats2file, mod_muc_log


	CONFIGURABLE PARAMETERS
	-----------------------

stanza: 
    Log packets only when stanza matches
    Default value: [iq, message, presence, other]
direction: 
    Log packets only when direction matches
    Default value: [internal, vhosts, external]
orientation: 
    Log packets only when orientation matches
    Default value: [send, revc]
logdir: 
    Base filename, including absolute path
    Default value: "/tmp/jabberlogs/"
timezone:
    The time zone for the logs is configurable with this option. 
	Allowed values are 'local' and 'universal'.
	With the first value, the local time, 
	as reported to Erlang by the operating system, will be used. 
	With the latter, GMT/UTC time will be used. 
	Default value: local
rotate_days: 
    Rotate logs every X days
    Put 'no' to disable this limit.
    Default value: 1
rotate_megs: 
    Rotate when the logfile size is higher than this, in megabytes.
    Put 'no' to disable this limit.
    Default value: 10
rotate_kpackets: 
    Rotate every *1000 XMPP packets logged
    Put 'no' to disable this limit.
    Default value: 10
check_rotate_kpackets: 
    Check rotation every *1000 packets
    Default value: 1


	EXAMPLE CONFIGURATION
	---------------------

  {mod_logxml, [
     {stanza, [message, other]},
     {direction, [external]},
     {orientation, [send, recv]},
     {logdir, "/var/jabber/logs/"},
     {timezone, universal}, 
     {rotate_days, 1}, 
     {rotate_megs, 100}, 
     {rotate_kpackets, no},
     {check_rotate_kpackets, 1}
  ]},


	CHANGELOG
	---------

0.2.3 - 2007-03-20
  * The file name respects the timezone option

0.2.2 - 2007-02-13
  * Added new option: timezone

0.2.1 - 2006-08-08
  * Fixed small bug on start/2 

0.2 - 2006-03-08
  * Changed some configuration options: 
      rotate_days, rotate_mages and rotate_kpackets can now be set independently
  * New format of XML logs: now XMPP packets are enclosed in , with attributes:
      or: orientation of the packet, either 'send' or 'recv'
      ljid: local JID of the sender or receiver, depending on the orientation
      ts: timestamp when the packet was logged

0.1 - 2005-11-11
  * Initial version

Related Links

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Cluster

Anyone have a problem getting this working for ejabberd setups that are clustered?

It seems to work before I setup ejabberd in a cluster. There aren't any errors in the logs and the log file directory is completely empty. I am logging message stanzas (or trying to!)

perl script to parse

Here is simple perl script to parse logxml files. HTML based on xmpp.xsl posted below.

#!/usr/bin/perl

use strict;
use XML::Simple;
use CGI qw(param);


my $str;
open(A,"< ej.xml");
while(<A>) { $str .= $_; }
close(A);

$str .= "<log>";

my $xml = XMLin($str);


print "Content-Type: text/html\r\n\r\n";

print qq|
<html>
        <head>
                <style>
                        body {
                        margin: 10px; 
                        font-family: Verdana, Arial, Helvetica, sans-serif; 
                        font-size: 10pt; 
                        }
                        td {
                        font-family: Verdana, Arial, Helvetica, sans-serif;
                        font-size: 10pt; 
                        }
                        p {
                        color: blue;
                        }
                        td.head_right {
                        background-color: #f0f0f0; 
                        padding: 0px 3px 0px 3px; 
                        font-size: 8pt; 
                        }
                        td.head_left {
                        width: 50px; 
                        background-color: #e0e0e0; 
                        padding: 0px 3px 0px 3px; 
                        font-weight: bold; 
                        font-size: 8pt;
                        }
                        td.header {
                        border: solid 1px silver;
                        }
                        td.message {
                        padding: 3px; 
                        border: solid 1px silver;
                        }
                <style>
        <head>
        <body>
                <table align="center" width="100%" cellspacing="0" cellpadding="0" style="border: solid 1px silver; border-collapse: collapse">

|;

my $ex = param('exclude');

foreach my $packet (@{$xml->{packet}}) {
        if ($packet->{'or'} eq 'send' && $packet->{'message'}->{'body'}) {
        if ($packet->{'ljid'} =~ /$ex/ || $packet->{'message'}->{'to'} =~ /$ex/) { next; }
        if (!param('otr') && $packet->{'message'}->{'body'} =~ /\?OTR/g) { next; }
        print qq|
                <tr><td class="header">
                                                        <table border="0" width="100%" cellspacing="0" cellpadding="0">
                                                                <tr>
                                                                        <td class="head_left">ts: <td>
                                                                        <td class="head_right">$packet->{'ts'}<td>
                                                                <tr>
                                                                <tr>
                                                                        <td class="head_left">from: <td>
                                                                        <td class="head_right">$packet->{'ljid'}<td>
                                                                <tr>
                                                                <tr>
                                                                        <td class="head_left">to: <td>
                                                                        <td class="head_right">$packet->{'message'}->{'to'}<td>
                                                                <tr>
                                                        <table>
                                                <td><tr>
                                                <tr>
                                                        <td class="message">$packet->{'message'}->{'body'}<td>
                                                <tr>

                        |;

        }
}

print qq|
                <table>
        <body>
<html>
|;

Announce messages

Good day for all!
The announce messages has not stores in log, but i've many users with sending announce rights.
Is there a way to get log for announce messages?
Thanks.

Configure like this

If you configure like this, the announces are logged:

     {stanza, [message]},
     {direction, [internal]},
     {orientation, [send]},

Bug report: time shift minus 1 hour in the log name

Version: 0.2.2 - 2007-02-13
Names of logs still don't respect the timezone setting. This can be confusing sometimes - for instance first packet in today's log has timestamp ts="20070305T00:00:06", but the log is named srvis03-2007-03-04-23-00-06.xml (so in this case it even has different date in its name due to the one hour difference!)
If you corrected this, the module would be nearly perfect.

Fixed on 0.2.3

Finally, fixed on version 0.2.3. Check SVN.

Bug report: time shift minus 1 hour in the log

Briefly: it's 10:25 on the server now and when I send a message, in the log I can read the timestamp: ts="20070212T09:25:19".
I'm in the Czech Republic and it's correctly set on our Win 2003 Server - GMT+01:00 (Prague). The timestamps in "ejabberd.log" are correct. It must be something with mod_logxml.

Have a nice day
Pontiac

New timezone option on version 0.2.2

The timestamp was specified in universal time. I've added a new option to specify the timezone you want the timestamps to be: local (as you want) or universal (as it was previously).

log filename still created using universal time

But the log entries are now logged correctly (timezone local). Thanks badlop for quick response.

Pontiac

Bug report: mod_logxml doesn't finish the log file with </log>

Hello,
my server have run for the whole weekend. I have day rotation and have the Friday's log unfinished with the </log> tag as well as the two logs made on Saturday and Sunday. The server wasn't shut down.

I have mod_logxml Id 0.2.1 compiled with Erlang/OTP R10B-10. I'm running ejabberd 1.1.2 on Windows 2003 Server.

Strange bug. It works for me.

Strange bug.

It works correctly for me, and nobody else complained before. The module should add the closing tag before closing the file.

Check the ejabberd log files for any ERROR or CRASH REPORT.

If you are really interested in fixing this bug, we will have to do some live debugging. If so, join ejabberd chatroom any day between 20.00 and 22.00 GMT+1.

closing tag added

Hmm, Badlop. It's weird (but pleasant at the same time :-). The mod_logxml suddenly (from 20.2.2007) started to add the <\log> closing tag. I am going to observe it. Cannot remember of any change I did in ejabberd configuration.

We'll see.

Pontiac

PS: But anyway - is it OK that the ejabberd@conference.jabberd.ru is off-line in my PSI? Is it really off-line? Or am I supposed to set it up somehow?

No ejabberd chatroom

Hello Badlop,
I have just been trying to find chatroom ejabberd at conference.jabber.ru. All I have found so far is "ejabber" and this one is offline.
So are you sure "ejabberd@conference.jabber.ru" exists?

It exists, join it

Pontiac wrote:

I have just been trying to find chatroom ejabberd at conference.jabber.ru.
So are you sure "ejabberd@conference.jabber.ru" exists?

It exists, but it isn't listed on the MUC service. Tell your Jabber client the details of the chatroom and join it, instead of searching.

I have added it

But PSI says it is offline.

No errors

Hello,
I have found no error or crashes regarding mod_logxml in ejabberd.log.

I'll try the chatroom ASAP.

Compilation in Win version

Hello,
is there any tutorial on how to compile modules in Win32 version of Ejabberd? I suppose it is not possible to do point 2 "recompile ejabberd" on standard installed Windows version.

Try ejabberd-modules

Pontiac wrote:

Hello, is there any tutorial on how to compile modules in Win32 version of Ejabberd? I suppose it is not possible to do point 2 "recompile ejabberd" on standard installed Windows version.

On Windows: install Subversion and Erlant/OTP. Then follow this: ejabberd-modules.

mod_logxml.erl

Why does mod_logxml.erl contain "ejabberd_hooks:add(user_send_packet...." at line 123 twice?

It seems a stupid bug

I don't see any reason to do it twice, so I've removed the duplicate line and published version 0.2.1.

Thanks for the comment.

it's my xmpp.xsl for viewing XML log

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE xsl:stylesheet>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="html" indent="yes" encoding="utf-8"/>

<xsl:template match="/">
	<html>
		<head>
			<style>
				body {
				margin: 10px; 
				font-family: Verdana, Arial, Helvetica, sans-serif; 
				font-size: 10pt; 
				}
				td {
				font-family: Verdana, Arial, Helvetica, sans-serif;
				font-size: 10pt; 
				}
				p {
				color: blue;
				}
				td.head_right {
				background-color: #f0f0f0; 
				padding: 0px 3px 0px 3px; 
				font-size: 8pt; 
				}
				td.head_left {
				width: 50px; 
				background-color: #e0e0e0; 
				padding: 0px 3px 0px 3px; 
				font-weight: bold; 
				font-size: 8pt;
				}
				td.header {
				border: solid 1px silver;
				}
				td.message {
				padding: 3px; 
				border: solid 1px silver;
				}
			</style>
		</head>
		<body>
			<table align="center" width="100%" cellspacing="0" cellpadding="0" style="border: solid 1px silver; border-collapse: collapse">
				<xsl:variable name="varStartWith">send</xsl:variable>
				<xsl:for-each select="log/packet">
				<xsl:variable name="varOrient"><xsl:value-of select="@or" /></xsl:variable>
				<xsl:if test="starts-with($varOrient,$varStartWith)">
					<xsl:if test="not(count(message/body)=0)">
					<tr><td class="header">
						<table border="0" width="100%" cellspacing="0" cellpadding="0">
							<tr>
								<td class="head_left">ts: </td>
								<td class="head_right"><xsl:value-of select="@ts" /></td>
							</tr>
							<tr>
								<td class="head_left">from: </td>
								<td class="head_right"><xsl:value-of select="@ljid" /></td>
							</tr>
							<tr>
								<td class="head_left">to: </td>
								<td class="head_right"><xsl:value-of select="message/@to" /></td>
							</tr>
						</table>
					</td></tr>
					<tr>
						<td class="message"><xsl:value-of select="message/body" /></td>
					</tr>
				</xsl:if>
				</xsl:if>
				</xsl:for-each>
			</table>
		</body>
	</html>
</xsl:template> 
</xsl:stylesheet>

doesn' work

Hi,
Dunno why but it just won't work with Firefox. All I can see is a white page.

My fault

I am sorry, I didnt notice there were no message packets (only iq). So nothing could be displayed.

Syndicate content