Automated offsite backups

Chilling_Silence

Joined
Sep 23, 2008
Messages
488
Likes
0
Points
0
#1
Hi all,

I'm looking for some sort of way to easily, automatically and remotely backup everything on some of my Elastix-1.6 systems.

Ideally I'd like to backup the CDRs and system recordings as well as the general configuration.

I understand there's the backups module in the Web UI but I'm after something that can be automated.
Any ideas?

Thanks


Chill.
 

dicko

Joined
Oct 24, 2008
Messages
4,099
Likes
0
Points
0
#2
I use something like this:
----------------------------------------------------------------
#!/bin/bash
# set up some variables

BUSERVER=backerupper@1.2.3.4
BUDIR=/data/clientbackups
DOW=`date +%A`
DAILYDAYS=8
ISODAYS=8
MONITORDAYS=30
LOCALBUDIR=/backup
VMDAYS=90

echo "$HOSTNAME Backup on $DOW `date`"


ssh -p 2222 $BUSERVER mkdir -p $BUDIR/$HOSTNAME


case $DOW in
Sunday)

echo "It's $DOW, lets do the big one."
echo "Begin mondoarchive to $LOCALBUDIR at `date`"
/usr/sbin/mondoarchive -OVi -d $LOCALBUDIR -E "/backup /var/lib/asterisk/backups /var/spool/asterisk /usr/src " -s 4800m -p "`hostname`_`date +%Y.%m.%d.%H.%M`"
echo "End mondoarchive at `date` "
;;
esac




echo "Begin rsync at `date` to $BUSERVER:$BUDIR"
/usr/bin/rsync -at --inplace --progress -e 'ssh -q -p 2222' $LOCALBUDIR/*.iso $BUSERVER:$BUDIR/`hostname`/
echo "On remote server"
/usr/bin/rsync -at --inplace --progress -e 'ssh -q -p 2222' $BUSERVER:$BUDIR/$HOSTNAME/daily/
/usr/bin/rsync -at --inplace --progress -e 'ssh -q -p 2222' /var/lib/asterisk/backups/daily/*.gz $BUSERVER:$BUDIR/$HOSTNAME/daily
echo "End rsync at `date` to $BUSERVER:$BUDIR"
rsync -e 'ssh -p 2222' $BUSERVER:$BUDIR/`hostname`/*.iso
#delete old backup files over 8 days
#find /var/lib/asterisk/backups/daily/*.gz -mtime +$DAILYDAYS
rsync -e 'ssh -p 2222' $BUSERVER:$BUDIR/`hostname`/daily/

echo "Will delete"
echo "`find /var/lib/asterisk/backups/daily/*.gz -mtime +$DAILYDAYS -delete`"
echo "backupfiles"
find /var/lib/asterisk/backups/daily/*.gz -mtime +$DAILYDAYS -delete

#delete old fax temp files over $ISODAYS days old
#find /var/www/html/fax/tmp/* -mtime +$ISODAYS -delete

#delete monitor files older than $MONITORDAYS
#find /var/spool/asterisk/monitor/*.wav -mtime +$MONITORDAYS -delete

#find /var/spool/asterisk/voicemail/*/*/INBOX/* -mtime +$VMDAYS -delete

/var/lib/asterisk/bin/ampbackup.pl daily yes yes yes yes yes

----------------------------------------------------------------------

kind of 2 level, (iso's once a week, the rest daily)


a little unpolished and not much error detection but it fits my particular environment, I hope you get the gist

(run as a cron job)

I also have for backing up from a working server to a semi-active "standby server", (run from the standby)
------------------------------------------------------
#!/bin/bash
service hylafax stop
service asterisk stop
service mysqld stop

rsync -av --progress --delete -e 'ssh -p 2222' 1.2.3.4:/var/www/ /var/www/
rsync -av --progress --delete -e 'ssh -p 2222' 1.2.3.4:/var/lib/mysql/ /var/lib/mysql/
rsync -av --progress --delete -e 'ssh -p 2222' 1.2.3.4:/var/spool/hylafax/ /var/spool/hylafax/

service mysqld start
service hylafax start
service asterisk start



rsync -av --progress --delete -e 'ssh -p 2222' 1.2.3.4:/tftpboot/ /tftpboot/
rsync -av --progress --delete -e 'ssh -p 2222' 1.2.3.4:/var/lib/asterisk/backups/ /var/lib/asterisk/backups/
rsync -av --progress --delete -e 'ssh -p 2222' 1.2.3.4:/var/spool/asterisk/ /var/spool/asterisk/

-------------------------------------------------------------------

one will need to do the ssh keys thing though,

for that I stole/hacked

-------------------------------------------------------

#!/bin/sh

KEY="$HOME/.ssh/id_dsa.pub"

if [ ! -f ~/.ssh/id_dsa.pub ];then
echo "private key not found at $KEY"
echo "* please create it with "ssh-keygen -t dsa" *"
echo "* to login to the remote host without a password, don't give the key you create with ssh-keygen a password! *"
exit
fi

if [ -z $1 ];then
echo "Please specify user@host.tld as the first switch to this script and port as second switch"
exit
fi

echo "Putting your key on $1... "

KEYCODE=`cat $KEY`
ssh -p $2 -q $1 "mkdir ~/.ssh 2>/dev/null; chmod 700 ~/.ssh; echo "$KEYCODE" >> ~/.ssh/authorized_keys; chmod 644 ~/.ssh/authorized_keys"

echo "done!"

------------------------------------------------------------------------------------------




regards

dicko
 

Chilling_Silence

Joined
Sep 23, 2008
Messages
488
Likes
0
Points
0
#3
Awesome that looks like a great starting point, thanks!

I'll give it a whirl and see how I go, much appreciated :)
 

dicko

Joined
Oct 24, 2008
Messages
4,099
Likes
0
Points
0
#4
Chill:

If it basically works for you, please tidy up all my crap and blog it as you do so well, (I'm a sloppy lazy slob ;) )

dicko
 

Chilling_Silence

Joined
Sep 23, 2008
Messages
488
Likes
0
Points
0
#5
Sure mate!

I've got a fair bit on today, so I probably won't be able to take a look at it immediately, but stay tuned for a blog post on it ;)

Thanks again :)
 

Chilling_Silence

Joined
Sep 23, 2008
Messages
488
Likes
0
Points
0
#6
Why stop asterisk etc though ?
 

dicko

Joined
Oct 24, 2008
Messages
4,099
Likes
0
Points
0
#7
Don't forget that is run from the "standby" so shouldn't be too impactful, and was actually a transitional script as I move to drbd/pacemaker (which is NOT EASY to get going on current Elastix kernels, you guys should stick with heartbeat for now) , to be run from the earstwhile "new" production server, and I hate "down-time".

There is a slight possibility of "collision" especially with the mysql server, so just for the hell of it I stop the perhaps problematic services, Marie Koreen (I hope I spelled her name right ) does the mysqldump more rigourously, in her expansion of the Elastix backup process.

You will notice that the longer rsync processes are generally less critical so I moved them outside of that temporal stasis.

I personally add mysql "jounalling" to mysqld :-

log-bin = /var/lib/mysql/bin.log

to /etc/my.cnf , this after being "once-bitten", this will allow an orderly rebuild from the bin-log after an eff-up, as I said, I'm a sloppy, lazy bastard :)

dicko
 

Chilling_Silence

Joined
Sep 23, 2008
Messages
488
Likes
0
Points
0
#8
Why not use something like:
mysqldump --add-drop-table -h localhost -uroot -peLaStIx.2oo7 -A > /root/database.sql

Then just backup / rsync that?
Means you don't have to take anything offline... Ideally I'd like to keep the systems up 24/7, as this will be for a 24/7 support helpdesk :-/

Or have I missed something ?
 

dicko

Joined
Oct 24, 2008
Messages
4,099
Likes
0
Points
0
#9
Perhaps you did miss something, I do that all the time :)

As stated I concur, never take down a working server unless you have to, 5 nines is what I aspire to (or better yet 60/60/24/7/52) , I never stopped the production server, just the "hot-spare" and that for just a few seconds, and restoring a mysqldump can take quite a time on some of my legacy +five year old boxes astriskcdr can run to gigabytes, it's a matter of expediency and this process plus the log-bin journal just works quicker, yet still covers your ass, (I'm also impatient ;) , but please go the safer slower route if you prefer).

In my experience a catastrophic single server failure (thuggishly pulling the plug) in a well designed HA cluster will recover in much less than 30 seconds (which will never be noticed in a real-world "damn I mis-dialed, I'll do it over" scenario, and a couple of dropped calls in progress ) thusly your customer "pissed-off level" will approach zero. Ethically you get to get one failure every two months to maintain that 5x9, I can live with that, next step a fail-safe BGP solution to match and finding a VSP or three that also achieve that 5x9 criterion. But I digress . . .

dicko
 

Chilling_Silence

Joined
Sep 23, 2008
Messages
488
Likes
0
Points
0
#10
Aside from what's in /etc/asterisk and also /var/spool/asterisk/ if I'm also dumping MySQL, is there anything that's going to change from a vanilla Elastix install if I'm purely sticking to editing things in FreePBX?

It's nice to do ISO's and all, but potentially overkill if I'm just wanting to backup configuration files so that the system can be restored (Overwriting a vanilla Elatix install) and also if the client makes a boo boo, we can go back to yesterdays configs.

I'm doing some testing now, hope to blog it all shortly :)
 

dicko

Joined
Oct 24, 2008
Messages
4,099
Likes
0
Points
0
#11
Overkill will save you're ass when a lazy underkill just wouldn't cover ones lack of prescience and you forgot to cover all your bases, don't be complacent, it's just cpu cycles.

What else changed is totally dependent on YOUR system and how much it is used,


find / -mtime -1


will tell you "what's new today for me", if anything looks pertinent then it probably is, and thus a good case to "back it up".


If you can currently do a "bare metal restore" from last week, and apply last night's state within an hour, then please share your recipe. My methodology does :) :)




dicko
 

dicko

Joined
Oct 24, 2008
Messages
4,099
Likes
0
Points
0
#12
For immediacy, but with some concern about the abilities of the local warm-ware, you can further reduce downtime by doing the mondoarchive to a "local" usb stick, just maintain control of both the local BIOS boot settings, and any warm-ware involved.

dicko
 

Chilling_Silence

Joined
Sep 23, 2008
Messages
488
Likes
0
Points
0
#13
Not quite using your scripts but check the blogs for my new post and let me know what you think.

In other news the new Forums work brilliantly on my Nexus One!!
 

dicko

Joined
Oct 24, 2008
Messages
4,099
Likes
0
Points
0
#14
Chilling:

It certainly looks functional, I like the "diff" bit

Personally I would have left the SSHPORT option in, I don't advise anyone leaving SSH on 22. I would also have left the key exchange as a script for the noobs.

Editing amportal.conf to suit (forr rsync or ftp even) and

/var/lib/asterisk/bin/ampbackup.pl daily yes yes yes yes yes

and adding the cron job is also effective if you just need the core stuff.

regards

dicko
 

Chilling_Silence

Joined
Sep 23, 2008
Messages
488
Likes
0
Points
0
#15
Yes perhaps the port could be useful, however it makes no difference if you change the SSH port on the PBX, only the port on the server everything is backing up to, but it's still a good point and a simple addition.

What does that ampbackup script do that mine doesn't?
Do you adjust /etc/amportal.conf on most of your systems?
 

dicko

Joined
Oct 24, 2008
Messages
4,099
Likes
0
Points
0
#16
:) :)


ALL my machines run SSH on "another port", and all for the very same reasons.

amportal.conf, yes I edit to default new extensions to call waiting, add SSHPORT=2222 or whatever to keep the status display functional (hopefully green). In general much of FreePBX's behavior can be "tuned" here-within.

it's pretty well self-documenting as to how to do the off site backups, adding directories, like /tftpboot, and more. the last few lines need changing to suit a drbd/heartbeat type deployment to not use sym-linked directories. I don't use the internal scp/sftp bits though, as I prefer my scripts :)

so I guess essentially the only thing can do that yours doesn't is back-up the "complete" state of the FreePBX machine including dahdi and also itself, true, at the cost of some time, disk space, bandwidth and CPU cycles. You could add them though.

If you look at one of the backups in /var/lib/asterisk/backups/<set>/ you will see what's included.

You can of course also schedule on a more granular and time sensitive fashion multiple subsets of "yes yes etc." to suit the dynamics of your systems. JM2CWAE

regards

dicko
 

Members online

No members online now.

Latest posts

Forum statistics

Threads
30,902
Messages
130,887
Members
17,565
Latest member
omarmenichetti
Top