we are running ejabberd on ubuntu and using http-polling for a flexible web front end. All is working as expected however every once in a while a client will disconnect (close pop up browser) and although we push a connection.disconnect() through JSJaC to the correct place, it won't really close the session. It's not always occurring, but does happen from time to time.
In short, I was wondering if their is some sort of configuration or something along those lines where I could have an account auto close it's session if no activity is present in the form of packet traffic or http-polling presence.
I haven't been able to track down any server side errors regarding the mystery "ghost sessions" the only clearly definitive clue I've been able to sniff out is the "replacement" of an active session upon an http-polling session being start from an account that's already "in session".
Thanks in advance.
P.S. upon reconnecting and disconnecting the session there is about a 25% chance that session won't even close when forcing a connection.disconnect() from Firebug or window.onunload
Maybe it's a bug fixed in SVN
If the client closes the connection correctly, but sometimes ejabberd doesn't close it as expected, that can be considered a bug, right? :)
You don't mention what ejabberd version you use, but probably you run 1.1.4 or older. There were some updates in both ejabberd core and http_poll that may have solved that problem.
Could you compile and test ejabberd SVN in another machine, and test it with your client? The purpose is to know if current latest version is fixed or not. If compiling from SVN is a problem to you, tell me and I'll setup a test ejabberd svn server.
Regarding the automatic closing of idle sessions: I think that feature is not implemented. For that reason it's important to fix the root of the problem.
same issue at SVN
yup, im using a version from SVN, and ive found the same issue, if the user idle for some time, then it will disconnected, any idea
thx
I'm having a very similar
I'm having a very similar issue. I was having this exact same problem until I compiled the latest version from SVN. Now, when I kill a session the user will be logged out after 5 minutes. However, if that user reconnects after the server has disconnected their ghost session, it will not disconnect them again if their client is killed. At least, it does not disconnect them in the same time frame as the first session was killed.
Well it seems as if the
Well it seems as if the subsequent ghost sessions are terminated but it seems to take longer than the first. The first session is terminated after 5 minutes, it seems the subsequent ghost sessions are terminated after 10-15 minutes
try running the server in live
try booting up your server instance in "live" mode (ejabberctl live) and watch for connection and packet activity, it could be throwing errors under the hood, that's how I tracked down my issue.
As for recreating the issue, I tried the steps you described and everything seems to be running alright now.
What kind of sessions do you
What kind of sessions do you mean? I try http-poll and http-bind with jwchat and ejabberd SVN, and the sessions are closed after 5 minutes. Then I login again using the same method, kill it again, and the session timeouts in 5 minutes again.
Or do you mean a connection to port 5222 with a desktop client?
version
Revision: 969
Node Kind: directory
Schedule: normal
Last Changed Author: mremond
Last Changed Rev: 969
Last Changed Date: 2007-11-09 05:18:38 -0500 (Fri, 09 Nov 2007)
svn was pulled down in early/mid Nov. so I should be running a flavor of the latest and greatest.
updated
Revision: 969
Node Kind: directory
Schedule: normal
Last Changed Author: mremond
Last Changed Rev: 969
Last Changed Date: 2007-11-09 05:18:38 -0500 (Fri, 09 Nov 2007)
svn was pulled down in early/mid Nov. so I should be running a flavor of the latest and greatest.
Revision: 1076
Node Kind: directory
Schedule: normal
Last Changed Author: badlop
Last Changed Rev: 1076
Last Changed Date: 2007-12-17 17:23:56 -0500 (Mon, 17 Dec 2007)
just pulled down the latest revision and recompiled... let ya know what happens
same problem, newer version
still experiencing the same issue.
it seems to stem from a dropped client on an http-polling connection. If a browser crashes, or loses internet connection, or a user kills the process (thus not allow onunload to be executed) a session kill packet never gets sent to the server. Now the server will not receive anymore packets... however the session remains logged in for hours... or days at this point, which causes a lot of dropped messages due to the off line table never receiving them even though the user is technically off line. What complicates things a bit more however is if I re-establish a connection with said username to said server it will "replace" the session... now if I send a "kill/close" packet the server to "close session" the session won't die... it just sits, there logged in... forever.
=INFO REPORT==== 2007-12-19 12:38:07 ===I(<0.4064.0>:ejabberd_c2s:766) : ({socket_state,ejabberd_http_poll,{http_poll,<0.4062.0>},<0.4063.0>}) Opened session for 7705393@localhost/localhost
=INFO REPORT==== 2007-12-19 12:38:07 ===
I(<0.232.0>:ejabberd_listener:94) : (#Port<0.3180>) Accepted connection {{192,168,1,209},39801} -> {{192,168,1,220},5280}
=INFO REPORT==== 2007-12-19 12:38:07 ===
I(<0.224.0>:ejabberd_http:100) : started: {gen_tcp,#Port<0.3180>}
=INFO REPORT==== 2007-12-19 12:38:08 ===
I(<0.232.0>:ejabberd_listener:94) : (#Port<0.3181>) Accepted connection {{192,168,1,209},39816} -> {{192,168,1,220},5280}
=INFO REPORT==== 2007-12-19 12:38:08 ===
I(<0.224.0>:ejabberd_http:100) : started: {gen_tcp,#Port<0.3181>}
=INFO REPORT==== 2007-12-19 12:38:08 ===
I(<0.4064.0>:ejabberd_c2s:1266) : ({socket_state,ejabberd_http_poll,{http_poll,<0.4062.0>},<0.4063.0>}) Close session for 7705393@localhost/localhost
this is what the log entries looked like.... even though that account is still showing as logged in, the server isn't replacing the session (it was before), now it's just logging in... and logging out... however even though the server says it has closed the session... the user still shows as being online, and doesn't store off line messages.
chadillac wrote: we are
we are running ejabberd on ubuntu and using http-polling for a flexible web front end. All is working as expected however every once in a while a client will disconnect (close pop up browser) and although we push a connection.disconnect() through JSJaC to the correct place, it won't really close the session. It's not always occurring, but does happen from time to time.
P.S. upon reconnecting and disconnecting the session there is about a 25% chance that session won't even close when forcing a connection.disconnect() from Firebug or window.onunload
I tried ejabberd SVN with Tkabber 0.10 because it implements HTTP-Poll. When I close the session in Tkabber, it sends '</stream:stream>', and the session is closed. I verified in the webadmin.
Maybe this is interesting: Tkabber makes a connection every few seconds, even if the user is idle:
=INFO REPORT==== 30-Nov-2007::13:56:33 === I(<0.246.0>:ejabberd_listener:90) : (#Port<0.555>) Accepted connection {{127,0,1,1},41165} -> {{127,0,1,1},5280} =INFO REPORT==== 30-Nov-2007::13:56:33 === I(<0.239.0>:ejabberd_http:101) : started: {gen_tcp,#Port<0.555>} =INFO REPORT==== 30-Nov-2007::13:56:33 === I(<0.477.0>:ejabberd_http:202) : (#Port<0.555>) http query: 'POST' /http-pollIn short, I was wondering if their is some sort of configuration or something along those lines where I could have an account auto close it's session if no activity is present in the form of packet traffic or http-polling presence.
I think that isn't implemented.
Timeout is already implemented
Hello,
Timeout is already implemented in ejabberd HTTP poll and bind.
Default value is 5 minutes.
--
Mickaël Rémond
52 hours?
Hello,
Timeout is already implemented in ejabberd HTTP poll and bind.
Default value is 5 minutes.
--
Mickaël Rémond
If this is the case it's not working correctly, I had closed all instances of a browser (we are ONLY using http-polling connections) and left the server running over the weekend (I am the ONLY person testing this install, project), upon returning Monday (roughly 52 hours) morning, when viewing the admin "online users" page, my username was still logged in and not receiving off line messages. To complicate things further, upon launching my client and logging in, my session was once again "replaced" and could again sending and receiving messages... in most cases I have to reboot the server to kill the hung connections.
It depends on the version
It depends on the version you are using.
A bug has been fixed in older version.
--
Mickaël Rémond
version
It depends on the version you are using.
A bug has been fixed in older version.
--
Mickaël Rémond
I pulled down a fresh copy of the SVN yesterday afternoon, compiled, installed and is running... if I open a session, then ctrl-alt-del or cmd-q the browser session so the onunload event fails to fire (connection->disconnect()), my session remains live... as of right now my session has been active and logged in for 24+ hours.
edit:
what I mean my active and logged in is it shows I'm online, but there have been 0 packets sent or received from the server for that username
"should be setup"
after digging through the code I found this
ejabberd_http_poll.erl:-define(HTTP_POLL_TIMEOUT, 300000).ejabberd_http_poll.erl: Timer = erlang:start_timer(?HTTP_POLL_TIMEOUT, self(), []),
ejabberd_http_poll.erl: Timer = erlang:start_timer(?HTTP_POLL_TIMEOUT, self(), []),
Now I don't know enough about erlang to completely trouble shoot this... but from what I was able to tell the "msg" being sent appears blank... maybe a misplaced function call?
Code seems correct
after digging through the code I found this
ejabberd_http_poll.erl:-define(HTTP_POLL_TIMEOUT, 300000).ejabberd_http_poll.erl: Timer = erlang:start_timer(?HTTP_POLL_TIMEOUT, self(), []),
ejabberd_http_poll.erl: Timer = erlang:start_timer(?HTTP_POLL_TIMEOUT, self(), []),
Now I don't know enough about erlang to completely trouble shoot this... but from what I was able to tell the "msg" being sent appears blank... maybe a misplaced function call?
Do you mean the last parameter should be a string instead of just []?
Don't worry, any value passed as the third parameter is acceptable, because that parameter is not used at all later when the timeout triggers this function:
handle_info({timeout, Timer, _}, _StateName, #state{timer = Timer} = StateData) -> {stop, normal, StateData};Timeout works for me; step by step
I made a simple experiment like you, and in my case timeouts worked correctly.
I tried this with ejabberd SVN r1093:
$ firefox=INFO REPORT==== 21-Dec-2007::20:21:29 ===I(<0.474.0>:ejabberd_http_poll:154) : started: {"01d5191304b9b62c19575ee4b393c07fc667fa54",
[]}
=INFO REPORT==== 21-Dec-2007::20:21:29 ===
I(<0.476.0>:ejabberd_c2s:655) : ({socket_state,ejabberd_http_poll,{http_poll,<0.474.0>},<0.475.0>}) Accepted authentication for a1
=INFO REPORT==== 21-Dec-2007::20:21:29 ===
I(<0.476.0>:ejabberd_c2s:766) : ({socket_state,ejabberd_http_poll,{http_poll,<0.474.0>},<0.475.0>}) Opened session for a1@localhost/jwchat
=INFO REPORT==== 21-Dec-2007::20:22:09 ===I(<0.253.0>:ejabberd_listener:94) : (#Port<0.994>) Accepted connection {{127,0,0,1},33241} -> {{127,0,0,1},5280}
=INFO REPORT==== 21-Dec-2007::20:22:09 ===
I(<0.245.0>:ejabberd_http:100) : started: {gen_tcp,#Port<0.994>}
...
=INFO REPORT==== 21-Dec-2007::20:26:38 ===I(<0.476.0>:ejabberd_c2s:1266) : ({socket_state,ejabberd_http_poll,{http_poll,<0.474.0>},<0.475.0>}) Close session for a1@localhost/jwchat
I verify in the Web Admin that the session is closed.
I also tried having all the time another http-poll user, this one is loged in using Tkabber SVN with its http-poll support. Same result.
I didn't verify if that timeout feature works correctly with http-bind (using mod_http_bind), but this experience says it works correctly with http-poll.
So I don't understand why your session is not closed after 5 minutes.
If you make more tries with still that problem, we can try again once ejabberd 2.0.0-beta1 is published; we could even use the same binary installer to ensure it's not a problem related to Erlang version, for example.
strange
Erlang (BEAM) emulator version 5.4.9 [64-bit] [source] [threads:0]Eshell V5.4.9 (abort with ^G)
(ejabberd@localhost)1>
5.4.9 is the version of Erlang I'm running, and I recompiled from source today now running r1128... still without luck... I did notice that when I did my make install it errored out, after a ps aux it appears that Erlang refused to stop running even after Ejabberd was shut down, killing it and starting the server cured the issue, but I have to wonder if the bug lies in my install of Erlang as opposed to Ejabberd.
Thanks for the through answer but I'm still having no luck, after killing the browser and waiting upwards of 20 minutes the session was still shown as logged in and active...
The part that confuses me the most is the "unkillable" state of the session once such an error occurs... even if I log in and log out 10 times with said "hung" account/session the session will never show as offline in the admin panel, it is constantly logged in regardless of my interactions until I restart the server.
Any more ideas?
Thanks for all the help so far :)
log snippets
=INFO REPORT==== 2007-12-28 14:49:21 ===I(<0.360.0>:ejabberd_c2s:673) : ({socket_state,ejabberd_http_poll,{http_poll,<0.358.0>},<0.359.0>}) Accepted authentication for 7705393
.....
=INFO REPORT==== 2007-12-28 14:49:22 ===
I(<0.360.0>:ejabberd_c2s:784) : ({socket_state,ejabberd_http_poll,{http_poll,<0.358.0>},<0.359.0>}) Opened session for 7705393@localhost/localhost
....
=INFO REPORT==== 2007-12-28 14:49:24 ===
I(<0.360.0>:ejabberd_c2s:1284) : ({socket_state,ejabberd_http_poll,{http_poll,<0.358.0>},<0.359.0>}) Close session for 7705393@localhost/localhost
these are the logs after a session is already hung, it accepts authentication, opens a session, completely ignores the previous one, but does take it over, and after closing the admin panel still shows the session as active... and it doesn't store off line messages.
Once again, many thanks.
shed some light
I launched the server in Live mode to see if it would give me anymore insight rather than analyzing the logs.
sure enough after 5 minutes (to the second) of the last packet being received this came popping out.
=CRASH REPORT==== 28-Dec-2007::15:11:37 ===crasher:
pid: <0.362.0>
registered_name: []
error_info: {undef,[{gen_fsm,enter_loop,
[ejabberd_c2s,
[],
session_established,
{state,
{socket_state,
ejabberd_http_poll,
{http_poll,<0.360.0>},
<0.361.0>},
ejabberd_socket,
#Ref<0.0.0.8172>,
"1476895941",
{sasl_state,
"jabber",
"localhost",
[],
#Fun<ejabberd_c2s.1.6511860>,
#Fun<ejabberd_c2s.2.14587779>,
cyrsasl_digest,
{state,
5,
"796276665",
"7705393",
[],
#Fun<ejabberd_c2s.1.6511860>}},
c2s,
c2s_shaper,
false,
false,
false,
false,
[],
true,
{jid,
"7705393",
"localhost",
"localhost",
"7705393",
"localhost",
"localhost"},
"7705393",
"localhost",
"localhost",
{{1198,872394,758710},<0.362.0>},
{1,
{{"7705393","localhost",[]},nil,nil}},
{1,
{{"7705393","localhost",[]},nil,nil}},
{1,
{{"7705393","localhost",[]},nil,nil}},
{0,nil},
{dict,
1,
16,
16,
8,
80,
48,
{[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[]},
{{[],
[],
[],
[],
[],
[],
[[{"7705393",
"localhost",
"localhost"}|
nothing]],
[],
[],
[],
[],
[],
[],
[],
[],
[]}}},
{xmlelement,
"presence",
[],
[{xmlelement,
"show",
[],
[{xmlcdata,
<<97,118,97,105,108,97,98,108,101>>}]}]},
undefined,
undefined,
false,
{userlist,none,[]},
{{0,0,0,0},0},
[]}]},
{proc_lib,wake_up,3}]}
initial_call: {gen_fsm,enter_loop,
[ejabberd_c2s,
[],
session_established,
{state,{socket_state,
ejabberd_http_poll,
{http_poll,<0.360.0>},
<0.361.0>},
ejabberd_socket,
#Ref<0.0.0.8172>,
"1476895941",
{sasl_state,
"jabber",
"localhost",
[],
#Fun<ejabberd_c2s.1.6511860>,
#Fun<ejabberd_c2s.2.14587779>,
cyrsasl_digest,
{state,
5,
"796276665",
"7705393",
[],
#Fun<ejabberd_c2s.1.6511860>}},
c2s,
c2s_shaper,
false,
false,
false,
false,
[],
true,
{jid,
"7705393",
"localhost",
"localhost",
"7705393",
"localhost",
"localhost"},
"7705393",
"localhost",
"localhost",
{{1198,872394,758710},<0.362.0>},
{1,{{"7705393","localhost",[]},nil,nil}},
{1,{{"7705393","localhost",[]},nil,nil}},
{1,{{"7705393","localhost",[]},nil,nil}},
{0,nil},
{dict,
1,
16,
16,
8,
80,
48,
{[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[]},
{{[],
[],
[],
[],
[],
[],
[[{"7705393","localhost","localhost"}|
nothing]],
[],
[],
[],
[],
[],
[],
[],
[],
[]}}},
{xmlelement,
"presence",
[],
[{xmlelement,
"show",
[],
[{xmlcdata,
<<97,118,97,105,108,97,98,108,101>>}]}]},
undefined,
undefined,
false,
{userlist,none,[]},
{{0,0,0,0},0},
[]}]}
ancestors: [ejabberd_c2s_sup,ejabberd_sup,<0.39.0>]
messages: [{'$gen_event',closed},
{'DOWN',#Ref<0.0.0.8172>,process,<0.361.0>,normal}]
links: [<0.230.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 610
stack_size: 21
reductions: 16563
neighbours:
=SUPERVISOR REPORT==== 28-Dec-2007::15:11:37 ===
Supervisor: {local,ejabberd_c2s_sup}
Context: child_terminated
Reason: {undef,[{gen_fsm,enter_loop,
[ejabberd_c2s,
[],
session_established,
{state,
{socket_state,
ejabberd_http_poll,
{http_poll,<0.360.0>},
<0.361.0>},
ejabberd_socket,
#Ref<0.0.0.8172>,
"1476895941",
{sasl_state,
"jabber",
"localhost",
[],
#Fun<ejabberd_c2s.1.6511860>,
#Fun<ejabberd_c2s.2.14587779>,
cyrsasl_digest,
{state,
5,
"796276665",
"7705393",
[],
#Fun<ejabberd_c2s.1.6511860>}},
c2s,
c2s_shaper,
false,
false,
false,
false,
[],
true,
{jid,
"7705393",
"localhost",
"localhost",
"7705393",
"localhost",
"localhost"},
"7705393",
"localhost",
"localhost",
{{1198,872394,758710},<0.362.0>},
{1,
{{"7705393","localhost",[]},nil,nil}},
{1,
{{"7705393","localhost",[]},nil,nil}},
{1,
{{"7705393","localhost",[]},nil,nil}},
{0,nil},
{dict,
1,
16,
16,
8,
80,
48,
{[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[]},
{{[],
[],
[],
[],
[],
[],
[[{"7705393",
"localhost",
"localhost"}|
nothing]],
[],
[],
[],
[],
[],
[],
[],
[],
[]}}},
{xmlelement,
"presence",
[],
[{xmlelement,
"show",
[],
[{xmlcdata,
<<97,118,97,105,108,97,98,108,101>>}]}]},
undefined,
undefined,
false,
{userlist,none,[]},
{{0,0,0,0},0},
[]}]},
{proc_lib,wake_up,3}]}
Offender: [{pid,<0.362.0>},
{name,undefined},
{mfa,{ejabberd_c2s,start_link,
[{ejabberd_socket,
{socket_state,
ejabberd_http_poll,
{http_poll,<0.360.0>},
<0.361.0>}},
[{shaper,c2s_shaper},{access,c2s}]]}},
{restart_type,temporary},
{shutdown,brutal_kill},
{child_type,worker}]
I'm not even sure where to start to track this back, but hopefully it'll help.
Thanks :D
Can you try with Erlang/OTP R10B-9 up to R11B-5?
The error message is large, but I think this time the problem is easy to detect:
error_info: {undef,[{gen_fsm,enter_loop,This means that the function enter_loop with 4 parameters, defined in the erlang module gen_fsm seems to not be defined. The call to that function was added in .
There could be many reasons for the 'undefined' error, but I think the problem is that the function was implemented in recent Erlang/OTP versions, and you are using an old one.
gen_fsm:enter_loop/4 was added in STDLIB 1.3.11 (). That version of STDLIB seems to be included with BEAM 5.4.12. And BEAM 5.4.12 seems to be included in Erlang/OTP R10B-9. Which happens to be released in December 2005: two years ago.
You are using BEAM 5.4.9, seems to be included in Erlang/OTP R10B-7, released in August 2005. So it seems you need to update it. Note that Erlang/OTP R12B was released recently, but it includes too much new code and is not recommended for production servers yet.
The ejabberd 2.0.0-beta1 Guide says that 'Erlang/OTP R9C-2 or higher' is required to compile ejabberd. I guess this must be updated to say 'Erlang/OTP R10B-9 up to R11B-5'.
I hope you can get a newer Erlang version, and check if the error message disappears, and HTTP-Poll sessions are closed correctly.
it works!
Sure enough after going to the Erlang site and pulling down the source for R10B-10 (I was running the Ubuntu repository version previously) and booting up the server on the newly installed version all worked without a so much as a hiccup!
thanks a lot for the advice and I'm glad to see we figured it out :)
Many thanks to Badlop :)
How to change 5 minute timeout?
Hello,
Timeout is already implemented in ejabberd HTTP poll and bind.
Default value is 5 minutes.
--
Mickaël Rémond
Can someone please tell me how to change the ejabberd default timeout of 5 minutes? I am using muckl as a web client for ejabberd and have it configured for HTTP bind. All is well except when there is no activity for 5 minutes, the clients all disconnect. I would like to increase this value, but I cannot find a configurable parameter for it in ejabberd.cfg. Based on certain comments here, I thought it was max_inactivity under mod_http_bind, but that is set for 30 seconds, so that's clearly not the one I want.
Self-service configuration
Can someone please tell me how to change the ejabberd default timeout of 5 minutes? I am using muckl as a web client for ejabberd and have it configured for HTTP bind. All is well except when there is no activity for 5 minutes, the clients all disconnect.
The only value configurable in ejabberd.cfg is max_inactivity, which gets the default value indicated by MAX_INACTIVITY. The other default values indicated in ejabberd_http_bind.erl are not easily configurable:
-define(MIN_POLLING, 2000000). % don't poll faster than that or we will % shoot you (time in microsec) -define(MAX_WAIT, 3600). % max num of secs to keep a request on hold -define(MAX_INACTIVITY, 30000). % msecs to wait before terminating % idle sessions -define(MAX_PAUSE, 120). % may num of sec a client is allowed to pause % the sessionYou can change them in the source code, recompile the file and install it.
I also noticed there is another timeout. That one is associated to HTTP handling, not only HTTP-Bind. It's defined in two lines in ejabberd_http.erl:
Data = SockMod:recv(Socket, 0, 300000), ... case (State#state.sockmod):recv(State#state.socket, 0, 300000) of300000 milliseconds = 5 minutes
Maybe you can play with that one, too.
Still unable to increase timeout default
We are still experiencing session timeout after 5 minutes of no activity. I would like to bump that default up to about 30 minutes. I tried bumping up the 300000 parameter values you mentioned and recompiling, but still inactive sessions timeout after 5 minutes. Any other idea Badlop?