Can anybody explain in plain words what are the settings for the mailinglist like option? What exactly should I change and where on the server? I mean I need the members to receive forum notification and reply through email and post in the forum.
That is (and its not ElkArte's fault) one of the more challenging areas since you need to enable the server to "pipe" meaning forward emails it receives to the posting program or use an imap email box like google.
Please read this https://github.com/elkarte/Elkarte/wiki/Posting-by-Email-Feature and see if that helps answer some of your questions and then feel free to ask some more.
Thanks for your reply.
I know this post but it refers to Cpanel users. I am not a Cpanel user. I am VestaCP user. What should I do then?
The main thing you need to do in the panel is "pipe" the email to the posting program. I'm not familiar with VestaCP but there should be a forwarding section in the email where you can send it to a program on the server instead of to another email address.