Hello all,
I'm running a ejabberd server with ~2500 users.
As I'm thinking about migrating to MySQL: Can anybody tell my how the performance is when comparing Mnesia with MySQL?
For a test I exported the Mnesia database with mod_ctlextra to textfiles using this command:
ejabberdctl export2odbc myserver.mydomain /tmp/
Second issue:
Then I imported the data to a MySQL database.
This is the result (cut-n-pasted from phpMyAdmin):
last 2,000
privacy_default_list 0
privacy_list 0
privacy_list_data 0
private_storage 642
rostergroups 8,283
rosterusers ~46,566
spool 2,936
users 2,464
vcard 800
vcard_search 800
The numbers are the rows of each table. I'm wondering why "last", "vcard" and "vcard_search" contain a even number of rows. Is this just a coincidence or did the export not work properly?
Thank you and regards,
Martin
*SQL performs better. Compare size of tables
I'm running a ejabberd server with ~2500 users.
As I'm thinking about migrating to MySQL: Can anybody tell my how the performance is when comparing Mnesia with MySQL?
2500 concurrent users, or total registered accounts? The important factor is concurrent users.
ejabberd consumes less computational resources when using some *SQL database than when using internal Mnesia. You are probably interested in that topic when you have many concurrent users (more than 1000, for example). With few concurrent users ejabberd's CPU consumption is negligible so admins of small servers don't care to setup an external SQL server and database.
Then I imported the data to a MySQL database.
This is the result (cut-n-pasted from phpMyAdmin):
last 2,000
privacy_default_list 0
privacy_list 0
privacy_list_data 0
private_storage 642
rostergroups 8,283
rosterusers ~46,566
spool 2,936
users 2,464
vcard 800
vcard_search 800
The numbers are the rows of each table. I'm wondering why "last", "vcard" and "vcard_search" contain a even number of rows. Is this just a coincidence or did the export not work properly?
It's suspicious. You can check the number of items in the tables in Mnesia, and compare with the exported results. Go to ejabberd Web Admin -> Nodes -> your node -> Database.
Re: *SQL performs better. Compare size of tables
Hello and thanks for your reply. :-)
No, 2500 accounts at all and about 50 to 150 concurrent users on the server.
But it sounds like I'm going for MySQL after what I heard from you.
Problem now is that the Debian package of ejabberd isn't supporting MySQL and I won't compile my own package. I hope there will be a development in this issue shortly as other people also request MySQL support for the Debian package.
Thank you for the hint with the Web Admin. It really seems that I have 800 vcards even, interesting. ;-)
Regards,
Martin
With 150 concurrent users Mnesia is good. Instructions for mysql
2500 accounts at all and about 50 to 150 concurrent users on the server.
But it sounds like I'm going for MySQL after what I heard from you.
I help to administer a server with 25000 registered accounts, 500 concurrent users maximum, 2400 different accounts were used in the last month. It uses the default Mnesia, and there hasn't been any problem, and CPU/RAM usage is low.
So, I think right now with just 150 max concurrent users, performance of Mnesia is not an issue for you.
Problem now is that the Debian package of ejabberd isn't supporting MySQL and I won't compile my own package. I hope there will be a development in this issue shortly as other people also request MySQL support for the Debian package.
You already have the erlang compiler intalled. You only need to download with Subversion, compile and install with the ejabberd *.beam files:
Re: With 150 concurrent users Mnesia is good.
Hello,
I just migrated to MySQL. I got the MySQL driver from a nice German guy who already compiled it. I just had to copy the four beam files to the right directory and everything was set. Perfect! :-) But thank you for the instructions above! I'm sure other people will love and need it, too.
Then the problems started. It seems that ejabberd has problems with special database names (and maybe also with special user names). My database naming schema is localhost_ejabberd_jabber.domain.tld (localhost because no webspace domain is affected, then the program name and then the jabber domain which is affected). Usernames are similar but shorter because of the 16 character limitation. I got errors that ejabberd failed_changing_database and after finally changing database name AND username to ejabberd01 and everything went through smoothly.
I think this is a issue to fix or at least talk about in the documentation.
BTW: What Mnesia database files can I move to backup?
The export command created these files for me:
last.txt
offline.txt
passwd.txt
private_storage.txt
roster.txt
vcard_search.txt
vcard.txt
I now use the following modules as odbc version:
mod_last_odbc
mod_offline_odbc
mod_privacy_odbc
mod_private_odbc
mod_roster_odbc
mod_vcard_odbc
When I checked ejabberd's mnesia directory I saw some files which could be useless and are better stored at my backup location (as I don't want to try to remember in a year what files are still necessary and which not):
last_activity.DCD
last_activity.DCL
offline_msg.DAT
passwd.DCD
passwd.DCL
private_storage.DAT
roster.DCD
vcard.DAT
vcard_search.DCD
vcard_search.DCL
Is this list correct or is there any file missing or wrong?
Regards,
Martin
Can database name have dots?
It seems that ejabberd has problems with special database names (and maybe also with special user names). My database naming schema is localhost_ejabberd_jabber.domain.tld (localhost because no webspace domain is affected, then the program name and then the jabber domain which is affected). Usernames are similar but shorter because of the 16 character limitation. I got errors that ejabberd failed_changing_database and after finally changing database name AND username to ejabberd01 and everything went through smoothly.
I think this is a issue to fix or at least talk about in the documentation.
Note that I knoe few of MySQL. But I can't create a database with special characters:
$ mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 87
Server version: 5.0.67-1 (Debian)
mysql> CREATE DATABASE localhost_ejabberd_jabber.localhost.com DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.localhost.com DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci' at line 1
mysql> CREATE DATABASE localhost_ejabberd_jabber-localhost-com DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-localhost-com DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci' at line 1
mysql> CREATE DATABASE localhost_ejabberd_jabber_localhost_com DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
Is this list correct or is there any file missing or wrong?
Yes, all your comments are correct.
Re: Can database name have dots?
Oh I'm sorry. I was not writing what I wanted to. The database names did not contain dots. It looked like this:
localhost_ejabberd_jabber-localhost-com
Dots are replaced with dashes.
With phpMyAdmin it is possible to create datebase and user names like this. Also there is a 64 character limit for database names and a 16 character limit for usernames. ejabberd seems to have problems eigher with the length of the database and/or username or _ and/or - in database and/or user names.
Limit in database name imposed by MySQL?
Oh I'm sorry. I was not writing what I wanted to. The database names did not contain dots. It looked like this:
localhost_ejabberd_jabber-localhost-com
Dots are replaced with dashes.
With phpMyAdmin it is possible to create datebase and user names like this.
Doesn't help to use slash:
$ mysql -u root
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 88
Server version: 5.0.67-1 (Debian)
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> CREATE DATABASE localhost_ejabberd_jabber-localhost-com DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-localhost-com DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci' at line 1
mysql> CREATE DATABASE localhost_ejabberd_jabber_localhost_com DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.02 sec)
mysql> \q
Bye
So, I can't create a database with name that contains . or -
there is a 64 character limit for database names and a 16 character limit for usernames
I didn't verify the 64-char limit in database name.
Regarding the reported limit of 16-char for usernames: I can register an account in ejabberd, with MySQL storage, with up to 40 characters:
=INFO REPORT==== 23-Dec-2008::00:31:47 ===
I(<0.368.0>:ejabberd_listener:119) : (#Port<0.474>) Accepted connection {{127,0,0,1},38521} -> {{127,0,0,1},5222}
=INFO REPORT==== 23-Dec-2008::00:31:48 ===
I(<0.383.0>:ejabberd_c2s:697) : ({socket_state,gen_tcp,#Port<0.474>,<0.382.0>}) Accepted authentication for a234567890b234567890c234567890d234567890
=INFO REPORT==== 23-Dec-2008::00:31:48 ===
I(<0.383.0>:ejabberd_c2s:809) : ({socket_state,gen_tcp,#Port<0.474>,<0.382.0>}) Opened session for a234567890b234567890c234567890d234567890@localhost/Tka
The account exists in the database:
mysql> SELECT * FROM ejadb.users;
+------------------------------------------+----------+
| username | password |
+------------------------------------------+----------+
| a234567890 | aaaaaaaa |
| a2345678901234567890 | aaaaaaaa |
| a234567890b234567890c234567890d234567890 | aaaaaaaa |
| badlop | aaaaeeee |
+------------------------------------------+----------+
4 rows in set (0.00 sec)
MySQL stuff
Oh I'm sorry. I was not writing what I wanted to. The database names did not contain dots. It looked like this:
localhost_ejabberd_jabber-localhost-com
Dots are replaced with dashes.
With phpMyAdmin it is possible to create datebase and user names like this.
Doesn't help to use slash:
Hm that's interesting. phpMyAdmin does allow it. Does anybody know what database and user names are "legal" in MySQL?!?
there is a 64 character limit for database names and a 16 character limit for usernames
Regarding the reported limit of 16-char for usernames: I can register an account in ejabberd, with MySQL storage, with up to 40 characters:
NO NO! ;-)
I was talking about a 16 character limit for MySQL user names! :-)
The column for user names is limited to 16 characters so there has to be this limit. Also the column for database names is limited to 64 characters.
I did not check the limits for jabber users.