We use our jabber server as a bridge between two systems and we push data from one system to the other through the jabber XMPP messages over ejabberd.
We will move a few GB worth of message data per day.
Our jabber server keeps crashing after consuming up to around 4 GB of memory.
This is what we see in the crash log:
=erl_crash_dump:0.1
Wed Oct 8 17:18:27 2008
Slogan: eheap_alloc: Cannot allocate 2850821240 bytes of memory (of type "heap").
System version: Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [async-threads:0] [kernel-poll:false]
Compiled: Sun Jul 20 07:56:39 2008
Atoms: 12649
=memory
total: 5587194721
processes: 5460439114
processes_used: 5460426146
system: 126755607
atom: 891921
atom_used: 861992
binary: 90576657
code: 10935423
ets: 9048352
The only configuration changes we've made to this server were to increase the max stanza size and the bit rate for the 'normal' traffic shaper.
So for ejabberd seems way too fragile to use for our needs, but I've heard of people using it for much more than we are.
Does anyone have any ideas on how to prevent ejabberd from consuming so much memory and crashing?
Thanks in advance.
-Preston
Some ideas
Probably the memory is consumed by the processes queues: if the messages can't be processed as fast as they are received, the queues increase until some limit is reached. In your case it seems the limit is imposed by the system RAM, or maybe by Erlang capability to handle 4 GB.
If you increase the available RAM, the crash will happen later, but will happen for sure.
If that's the reason of the problem, the solution is to ensure that the queues don't grow continually. Some ideas:
badlop, what queues do you
badlop, what queues do you mean? Erlang interprocess queues or something else?
Anyways:
How do I check exactly if the queues consume RAM?
Thanks.
I'm seeing this very same
I'm seeing this very same issue as well - so any help here would be greatly appreciated.
The output for the process information in webtool looks like so:
Pid Name/Spawned as State Reductions Stack+heap MsgQ Length
<0.27394.12> proc_lib:init_p/5 Garbing (limited info) 110439599 285082125 288927
The crash I think is happening because the garbage collector is trying to allocate some memory - but it can't get it.
Any help would be fantastic.