Openfire died

Discussion in 'General' started by cowboy47, Jul 11, 2008.

  1. cowboy47

    Joined:
    Jun 14, 2007
    Messages:
    276
    Likes Received:
    0
    The openfire server was running on my system, now, as of 10 minutes ago, when I click on the IM tab, it can't load. From the command prompt, service openfire status reports that openfire is running but it aint.

    Any ideas?
     
  2. danardf

    Joined:
    Dec 3, 2007
    Messages:
    8,069
    Likes Received:
    12
    hello.

    Same problem for me.

    Reboot server: Openfire is ok, and few munites after, openfire not responding.
     
  3. Bob

    Bob

    Joined:
    Nov 4, 2007
    Messages:
    2,400
    Likes Received:
    1
    Danardf,

    I have had a similar problem over the last few days with the IM tab just hanging. Configuration went generally well, but after that Openfire became very shaky....

    After a bit of investigation found it appeared to be related to the Java Heap Space. So I increased it.

    It appears to be due to the upgrade of the Openfire, and is probably not just Elastix related.

    In the file

    /etc/init.d/openfire

    there is a line, add the bolded part to the line

    #Lastly, prepare the full command that we are going to run.
    OPENFIRE_RUN_CMD="${JAVACMD} -Xmx512m -server ${OPENFIRE_OPTS} -classpath \"${LOCALCLASSPATH}\" -jar \"${OPENFIRE_LIB}/startup.jar\""

    Reboot and try again. I am back to 100% reliability, and seems to be a touch faster on the response (perception).

    I have not checked further, but I believe the openfire script in the /opt directory also needs this change, especially if you stop and restart Openfire, but I am not sure. Happy to have it working after a reboot.

    Regards

    Bob
     
  4. danardf

    Joined:
    Dec 3, 2007
    Messages:
    8,069
    Likes Received:
    12
    ok
    thanks for this information.

    yesterday, i make an update the openfire, and i have not the problem.
     
  5. Bob

    Bob

    Joined:
    Nov 4, 2007
    Messages:
    2,400
    Likes Received:
    1
    No worries, I realised that the thread was going stale, so I suspected you have found something to correct the issue.

    Mainly posted for others who want a quick fix.

    Regards

    Bob
     
  6. areid

    Joined:
    Apr 11, 2008
    Messages:
    117
    Likes Received:
    0
    we are having the same problem. openfire seems to stop during the night and I have to do a reboot from elastix GUI admin cp and it openfire is up and running.

    I am hoping by adding "-Xmx512m" will work for me. Any advice?
     
  7. Bob

    Bob

    Joined:
    Nov 4, 2007
    Messages:
    2,400
    Likes Received:
    1
    Areid,

    not really, reasonably straight forward. Just reboot after completion.

    Mine has been rock steady since the change....and it is being reported by other Openfire users (which are not necessarily using it within ELastix).

    Regards
    Bob
     
  8. areid

    Joined:
    Apr 11, 2008
    Messages:
    117
    Likes Received:
    0
    Bob,
    Thank you for your quick response. I did a reboot and will check tomorrow if I will have the same problem with openfire stopping overnight. I will report back to this discussions.
     
  9. areid

    Joined:
    Apr 11, 2008
    Messages:
    117
    Likes Received:
    0
    Bob,
    it worked. thank you. In the morning, it was still running.
     
  10. Bob

    Bob

    Joined:
    Nov 4, 2007
    Messages:
    2,400
    Likes Received:
    1
    Areid,

    Thanks for the feedback. Feedback like yours keeps the threads useful for others who come along with a similar issue.

    It would probably need a few more days to be absolutely sure it has corrected your issue, but I suspect that it has and you seem pretty confident that it has as well.

    I have a suspicion that the Openfire is on the borderline re: memory usage which is why some are reporting it, others are not, and it probably differs form version to version.

    It might be good to add it to the ISO anyhow (otherwise to many others it looks like an Elastix problem.)

    Regards

    Bob
     
  11. Bob

    Bob

    Joined:
    Nov 4, 2007
    Messages:
    2,400
    Likes Received:
    1
    Just keeping this thread updated. I have not had time to look at it in detail, but the fix detailed does fix the original problem it was meant to fix.

    I suspect that others who are running a later version (especially if you do the YUM updates), are still seeing the problem (if they have not applied the fix).

    In other words, it appears to correct the problem on the latest version that comes with Elastix (Openfire 3.51).

    However, it appears that if you are running Elastix on 512Mb (test machine or just been happy with 512Mb - cause it works), it may not be enough, and causes the swap file to be used up as it reaches the 512Mb....

    As I said, I have not had enough time to review the issue in depth, just going on what I see at the moment, which is one machine I have with 512Mb (a test machine), and many others that have 1Gb or 2Gb. The latter machines are not exhibiting any issue with Openfire from what I can see, whereas the 512Mb machine is.

    If you have had the problem, if you could post your memory size, so that we can confirm or squash this theory....

    Regards

    Bob
     
  12. wiseoldowl

    Joined:
    Aug 19, 2008
    Messages:
    251
    Likes Received:
    0
    It didn't work well for me so I am trying this:

    In the files /etc/init.d/openfire AND in /opt/openfire/bin/openfirectl (bold part is added):

    OPENFIRE_RUN_CMD="${JAVACMD} -Xms32m -Xmx128m -Xss128k -Xoss128k -XX:ThreadStackSize=128 -server ${OPENFIRE_OPTS} -classpath \"${LOCALCLASSPATH}\" -jar \"${OPENFIRE_LIB}/startup.jar\""

    then in etc/sysconfig/openfire I added the line at the bottom of this section:

    # If you wish to set any specific options to pass to the JVM, you can
    # set them with the following variable.
    #OPENFIRE_OPTS="-Xmx1024m"
    OPENFIRE_OPTS="-Xms32m -Xmx128m -Xss128k -Xoss128k -XX:ThreadStackSize=128"

    I hope that catches everything. Those values seem to work better in a low memory situation (~512K physical memory). I found them on this page, under "Heap Settings":
    http://www.igniterealtime.org/community/docs/DOC-1033

    After making the above changes I did the following from the CLI:

    service openfire stop
    service openfire start

    The above seems to reduce the frequency of the problems. I think maybe if I had a 4:00 AM stop/start each night it might eliminate the memory problem, except that then the clients would de-register. As I said in another thread, I sure wish the Openfire people would fix their memory leak!

    The big problem here seems to be Java, which I've always thought of as a memory pig (I generally HATE Java apps). I wish there were some alternative that was similar to Openfire, but that does not rely on Java - I'm sure it would be much more stable.
     
  13. Chilling_Silence

    Joined:
    Sep 23, 2008
    Messages:
    488
    Likes Received:
    0
    Ive got the latest stable version of Elastix (1.2), no updates or changes.
    There's 1GB Ram in my system, had the same issue with it stopping after about 10 minutes
    So I added -Xmx512m and then after about 20 hours my system looked like this:
    http://i234.photobucket.com/albums/ee23 ... yUsage.jpg

    Have tried the above mentioned settings, will see how I go over the next 24 hours
     
  14. Chilling_Silence

    Joined:
    Sep 23, 2008
    Messages:
    488
    Likes Received:
    0
    Ive had mixed results with -Xmx512m and other more detailed settings.

    On systems with 1GB Ram, it still maxes out the RAM very quickly, crashing Openfire, and then even a restart of the app wont fix it.

    Have been keeping an eye on it and so far what wiseoldowl posted here has been working well:
    http://www.elastix.org/index.php?option ... id=3#10344

    That said, Im hesitant about enabling it on a production system because of this issue :(
     
  15. kman

    Joined:
    Aug 4, 2008
    Messages:
    26
    Likes Received:
    0
    Hi folks,

    I was able to get OpenFire stable using the -Xmx512m fix some time ago, then following a yum update I removed this addition and it's been faultless. Now, I've just built a new 1.3-2 system with updates and the memory leak is evident here also. :blink:

    I've tried the -Xmx512m change, followed by the suggestion by wiseoldowl, but neither of these makes any difference. I welcome any input on this, as I don't currently have the resources to debug.

    Perhaps upgrading to 3.5.2 or 3.6.0 would be a better move?

    Thanks in advance.

    Dave.
     
  16. kman

    Joined:
    Aug 4, 2008
    Messages:
    26
    Likes Received:
    0
    Update:

    I just removed the 'Search' plugin and installed the 'SIP Phone Plugin' plugin.... now the memory utilisation appears to be stable.

    I will update again tomorrow.

    Dave.
     
  17. kman

    Joined:
    Aug 4, 2008
    Messages:
    26
    Likes Received:
    0
    Confirmed: The bug is in the Search plugin. Memory utilisation has remained stable since removing the search plugin and restarting openfire yesterday.

    I'll try to repeat the problem on a test unit later today.

    Dave.
     
  18. Chilling_Silence

    Joined:
    Sep 23, 2008
    Messages:
    488
    Likes Received:
    0
    Very cool!
    I'll have to give it a whirl myself.
    Have you done the -Xmx512m change, or anything similar? Or is this on a vanilla Openfire setup, just with the search plugin removed?
     
  19. kman

    Joined:
    Aug 4, 2008
    Messages:
    26
    Likes Received:
    0
    No changes to the Java environment at all. Here's what I did:

    1) Clean install Elastix 1.3-2
    2) Yum update
    3) Setup OpenFire using the Elasix wizard with the built-in DB
    4) Reboot, Wait...

    At this stage the memory was still stable. So next I took steps to replicate the environment in which I last saw the problem.

    5) Add the Asterisk-IM plugin
    6) Modify the OpenFire SQL script to remove the unique constraint on table create (See Elastix Without Tears)
    7) Add the localhost Asterisk Manager to OpenFire, reboot

    ... still stable :( Fortunately you can tell within only a few minutes whether it's stable.

    8) Add three basic OpenFire users to the system. Didn't even bother to reboot this time, just restart OpenFire.

    Aha! Now I can watch the memory utilisation increase steadily. What you see above is *every* step I'd taken to replicate the problem; still getting a DHCP address, default passwords, no other cofig at all.

    Then I removed the Search plugin and restarted the OpenFire service.... steady as a rock for the last ~7 hours.

    I've already added this to my build notes for future reference. :)

    Dave. [​IMG]
     
  20. Chilling_Silence

    Joined:
    Sep 23, 2008
    Messages:
    488
    Likes Received:
    0
    Brilliant stuff!
     

Share This Page