Home » Forums » ejabberd & XMPP » ejabberd Administration

mod_rest + mod_mam to support migration

Submitted by ncajelli on Fri, 2016-10-21 20:39

Hi,

I'm planning to migrate my current chat implementation (based on an API + MySQL) to ejabberd with mod_mam enabled.

I installed ejabberd-16.08 and I managed to run a migration from my current tables to the archive table, as well as the user authentication with extauth_program

Since my app is already live, and i need to be able to handle the scenario of users havind the old & new implementations for a while, i'm looking to do a smooth integration.

I've been doing some tests and looking at the logs, and i have some questions:

- When an user sends a message using the xmpp protocol, the message is inserted on the archive table for that user at the moment, right?
- I noticed that it inserts 2 rows on the archive per message (one for each user). At which point is the row inserted for the user that is receiving the message?

I'm struggling with 2 scenarios:

UserNew sends a message to ejabberd
-> custom module calls to myApi to insert the message in the old system
-> message is inserted on archive for UserNew
-> message on archive for UserOld is missing

UserOld sends a message to myApi
-> myApi calls ejabberd with a payload like this:

<message to="usernew@mydomain" from="userold@mydomain" type="chat">
  <body>The message</body>
  <markable xmlns="urn:xmpp:chat-markers:0" />
  <extras xmlns="urn:xmpp:extras" customvar="value" />
</message>

-> message arrives UserNew client
-> message is inserted on archive for UserNew
-> message is not inserted on archive for UserOld

If i look at mod_mam's logs:
NewUser to NewUser2

2016-10-21 20:25:46.502 [debug] <0.2551.0>@mod_mam:user_send_packet:251 user_send_packet
2016-10-21 20:25:46.502 [debug] <0.442.0>@ejabberd_sql:sql_query_internal:570 SQL: "insert into archive (username, timestamp, peer, bare_peer, xml, txt, kind, nick) values ('NewUserID', 1477081546502237, 'NewUser2@mydomain', 'NewUser2@mydomain', '<message from=''NewUser2@mydomain/android-15'' to=''NewUser2@mydomain'' xml:lang=''en'' type=''chat'' id=''cde87670-1639-4c83-87f8-0f61d6604295''><body>message body</body><markable xmlns=''urn:xmpp:chat-markers:0''/><extras xmlns=''urn:xmpp:extras'' somevar=''somevalue''/></message>', 'message body', 'chat', '')"
2016-10-21 20:25:46.550 [debug] <0.2780.0>@mod_mam:is_bare_copy:942 Recipient of message to bare JID has top priority: NewUser2@mydomain/android-8
2016-10-21 20:25:46.551 [debug] <0.2635.0>@mod_mam:is_bare_copy:950 Additional recipient of message to bare JID: NewUser2@mydomain/android-4
2016-10-21 20:25:46.577 [debug] <0.442.0>@ejabberd_sql:sql_query_internal:570 SQL: "insert into archive (username, timestamp, peer, bare_peer, xml, txt, kind, nick) values ('NewUser2ID', 1477081546550538, 'NewUser2@mydomain', 'NewUser2@mydomain', '<message from=''NewUser2@mydomain'' to=''NewUser2@mydomain'' type=''chat''><body>message body</body><markable xmlns=''urn:xmpp:chat-markers:0''/><extras xmlns=''urn:xmpp:extras'' somevar=''somevalue''/></message>', 'message body', 'chat', '')"
2016-10-21 20:25:46.633 [debug] <0.2635.0>@mod_mam:is_bare_copy:950 Additional recipient of message to bare JID: NewUser2@mydomain/android-4
2016-10-21 20:25:46.666 [debug] <0.2780.0>@ejabberd_c2s:send_text:1857 Send XML on stream = <<"<message from='NewUser2@mydomain' to='NewUser2@mydomain' type='chat'><archived by='mydomain' xmlns='urn:xmpp:mam:tmp' id='1477081546550538'/><stanza-id by='mydomain' xmlns='urn:xmpp:sid:0' id='1477081546550538'/><body>message body</body><markable xmlns='urn:xmpp:chat-markers:0'/><extras xmlns='urn:xmpp:extras' somevar='somevalue'/></message>">>
2016-10-21 20:25:46.730 [debug] <0.2780.0>@mod_mam:is_bare_copy:942 Recipient of message to bare JID has top priority: NewUser2@mydomain/android-8
2016-10-21 20:25:46.731 [debug] <0.437.0>@ejabberd_sql:sql_query_internal:570 SQL: "insert into archive (username, timestamp, peer, bare_peer, xml, txt, kind, nick) values ('NewUser2ID', 1477081546730595, 'NewUser2@mydomain/android-15', 'NewUser2@mydomain', '<message from=''NewUser2@mydomain/android-15'' to=''NewUser2@mydomain'' xml:lang=''en'' type=''chat'' id=''cde87670-1639-4c83-87f8-0f61d6604295''><body>message body</body><markable xmlns=''urn:xmpp:chat-markers:0''/><extras xmlns=''urn:xmpp:extras'' somevar=''somevalue''/></message>', 'message body', 'chat', '')"
2016-10-21 20:25:46.787 [debug] <0.2780.0>@ejabberd_c2s:send_text:1857 Send XML on stream = <<"<message from='NewUser2@mydomain/android-15' to='NewUser2@mydomain' xml:lang='en' type='chat' id='cde87670-1639-4c83-87f8-0f61d6604295'><archived by='mydomain' xmlns='urn:xmpp:mam:tmp' id='1477081546730595'/><stanza-id by='mydomain' xmlns='urn:xmpp:sid:0' id='1477081546730595'/><body>message body</body><markable xmlns='urn:xmpp:chat-markers:0'/><extras xmlns='urn:xmpp:extras' somevar='somevalue'/></message>">>
2016-10-21 20:25:46.878 [debug] <0.2780.0>@mod_mam:user_send_packet:251 user_send_packet

OldUser to NewUser

2016-10-21 20:29:52.186 [debug] <0.2551.0>@mod_mam:is_bare_copy:942 Recipient of message to bare JID has top priority: newuser@mydomain/android-15
2016-10-21 20:29:52.186 [debug] <0.450.0>@ejabberd_sql:sql_query_internal:570 SQL: "insert into archive (username, timestamp, peer, bare_peer, xml, txt, kind, nick) values ('NewUserID', 1477081792186708, 'olduser@mydomain', 'olduser@mydomain', '<message from=''olduser@mydomain'' to=''newuser@mydomain'' type=''chat''><body>Jsks</body><markable xmlns=''urn:xmpp:chat-markers:0''/><extras xmlns=''urn:xmpp:extras'' somevar=''somevalue''/></message>', 'Jsks', 'chat', '')"
2016-10-21 20:29:52.249 [debug] <0.2551.0>@ejabberd_c2s:send_text:1857 Send XML on stream = <<"<message from='olduser@mydomain' to='newuser@mydomain' type='chat'><archived by='mydomain' xmlns='urn:xmpp:mam:tmp' id='1477081792186708'/><stanza-id by='mydomain' xmlns='urn:xmpp:sid:0' id='1477081792186708'/><body>Jsks</body><markable xmlns='urn:xmpp:chat-markers:0'/><extras xmlns='urn:xmpp:extras' somevar='somevalue'/></message>">>

The main issue here is that at some point, UserOld should update the client, and he's gonna miss all the messages that have been sent through this flow.

Any tip or example would be highly appreciated

Thanks

‹ Understanding ejabberd source code Weird Connection Issues [Resolved - Somewhat] ›

I configure like

Submitted by badlop on Mon, 2016-10-24 10:49.

I configure like this:

modules:
  mod_mam: 
    default: always
  mod_offline: {}

When User1 sends a message to User2, the sender's message is stored in the mam table immediately. If User2 is online, the message is delivered to him, and also stored in MAM table.

But if User2 is offline, the message is stored in the offline table; and later when User2 logins, the message is delivered to him (from the offline table) and is stored in the MAM table.

Try this config and scenario, does it work correctly?
Otherwise, disable your custom module and try again this config and scenario, does it work now?

»

Yes, now i see the messages

Submitted by ncajelli on Mon, 2016-10-24 13:10.

Yes, now i see the messages getting queued on the table spool when a UserNew sends a message to UserOld

The only issue that remains is that when UserOld sends a message to UserNew (through mod_rest endpoint), that message is not inserted into archive nor spool.

Is there a way to "log in" UserOld through mod_rest to simulate the same behaviour as if using the xmpp native client?

Thanks for the help.

»

Hi Guys, I am getting the

Submitted by iitrsachin89 on Sun, 2017-02-19 19:40.

Hi Guys,

I am getting the same problem "The only issue that remains is that when UserOld sends a message to UserNew (through mod_rest endpoint), that message is not inserted into archive nor spool."

when UserOld is sending a message to UserNew then a message is inserted on the archive for UserNew but not forUserOld.

Please share if you have any idea to fix it.

»
Syndicate content

User login