We have installed ejabberd version 17.04 on a Debian 4.9.65 machine
The number of open file descriptors in ejabberd process is steadily increasing till it reaches the limit (64k as of now). lsof, outputs numerous entries which has type as sock. On further investigation we found that, each of these fd had an established TCP6 connection, which then becomes sock at a later point of time. Note that, not all TCP connection exhibits this behavior
beam.smp 8633 ejabberd 1168u IPv6 64599259 0t0 TCP xxx.xx.x.x:xmpp-client->xxx.xxx.220.xxx:39514 (ESTABLISHED)
became
beam.smp 8633 ejabberd 1168u sock 0,8 0t0 64599632 protocol: TCPv6
If you notice, the inodes are different, even though the fd is same.
The following is ls -l and stat out for this fd
ls -l 1168
lrwx------ 1 ejabberd ejabberd 64 Feb 19 07:14 1168 -> socket:[64599632]
stat 1168
File: 1168 -> socket:[64599632]
Size: 64 Blocks: 0 IO Block: 1024 symbolic link
Device: 4h/4d Inode: 64519590 Links: 1
Access: (0700/lrwx------) Uid: ( 999/ejabberd) Gid: ( 999/ejabberd)
Access: 2018-02-20 07:42:35.143752343 +0000
Modify: 2018-02-19 07:14:22.058094796 +0000
Change: 2018-02-19 07:14:22.058094796 +0000
Birth: -
The file creation time is same as when the connection was established. The new inode (64519590) has no mention in /proc/net/tcp* or netstat. The connections from the client are getting cleanly closed (from client perspective). The mysql connections from ejabber are not getting affected.
Any pointers to debug/fix would be greatly appreciated.
Thanks
