Server Crashed!

Discussion in 'General' started by savoip, Nov 18, 2009.

  1. savoip

    Joined:
    Nov 15, 2009
    Messages:
    4
    Likes Received:
    0
    hi,
    all of the sudden, i tried today to access my server, i am getting this on the web:

    Forbidden
    You don't have permission to access / on this server.

    Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.


    --------------------------------------------------------------------------------

    Apache/2.2.3 (CentOS) Server at xx.xx.xx.xx Port 443

    i also cant SSH !
    nothing happened or changed...
    how can i get in , what has happened!!???

    the funny thing is that, the server is still running normally, people can call through it..!!
    help please!!
     
  2. mbit

    Joined:
    Jul 4, 2007
    Messages:
    466
    Likes Received:
    0
    Youll have to grab a keyboard and monitor and see what comes up on the screen.

    Do you have 2 network cards in the machine. Could one have died?
     
  3. savoip

    Joined:
    Nov 15, 2009
    Messages:
    4
    Likes Received:
    0
    yes i did that,
    and i found out that the harddisk got full!
    it was just 4% yesterday, and today it's 100%
    anyway, i looked in the /proc, and found one big large file around 99GB, i couldnt delete it!
    please help..
     
  4. dicko

    Joined:
    Oct 24, 2008
    Messages:
    4,099
    Likes Received:
    0
    /proc is not a physical disk file system

    start in root ( / ) with

    du -hx --max-depth=1

    Find the "big" directory, change to it and iterate until you get to the rogue. (perhaps /var/log ?)
     
  5. savoip

    Joined:
    Nov 15, 2009
    Messages:
    4
    Likes Received:
    0
    you are right !
    i found a huge file under /var/log/asterisk called "full"
    and series of large files full.1 , full.2, etc...
    is it safe to delete it??
    what is this file ??
     
  6. ramoncio

    Joined:
    May 12, 2010
    Messages:
    1,663
    Likes Received:
    0
    I think you should learn that by yourself.
     
  7. dicko

    Joined:
    Oct 24, 2008
    Messages:
    4,099
    Likes Received:
    0
    The "correct" way would be to rotate the logs and then feel free to delete all the full.* files but not the full file (current log).
    You should look in those files and find out what is being so verbose


    to regain disk space:

    rm -rf /var/log/asterisk/full.*

    to rotate the logs:
    rasterisk -x 'logger rotate'


    to see what's wrong :
    less /var/log/asterisk/full

    fix the problem and you will be on your merry way.

    But I agree with ramoncio, you need to know how what you did works :)
     
  8. mbit

    Joined:
    Jul 4, 2007
    Messages:
    466
    Likes Received:
    0
    I dont believe the issue is the rotation. Your issue is why the logs are filling so quickly. How long has the system been in production. You should go through the logs and check to see if there are errors in there that are filling the log very quickly. I have had issues like this before where a faulty module put information in the logs at a couple of megs a minute. It became very big very quickly.
     
  9. dicko

    Joined:
    Oct 24, 2008
    Messages:
    4,099
    Likes Received:
    0
    Thank you mbit,

    But I believe you misunderstood my post; to stop savoip deleting /var/log/asterisk/full and thus breaking the log daemon.
    full.* can be deleted to make the system functional again, and after the rotation, full will be much smaller and easier to browse, but presumable still full of the verbose bits. I guess you also missed my:

    .
    .
    to see what's wrong :
    less /var/log/asterisk/full

    fix the problem and you will be on your merry way.
    .
    .

    ;) ;)
     

Share This Page