Since then, I’ve applied to positions for an internship at school and finished my semester. Went through a bunch of interviews and finally accepted a position at ATI (i.e. AMD) which is awesome! I’m not really sure yet if I’m allowed to blog about the things I’ll be working on there but I’ll find out. Would be cool to keep track of the progress I make while there for the next 16 months. Along with that, I’ve bought myself a car to get to and from work – FREEDOM!

Elsewhere in my tech-space, I’ve begun familiarizing myself with the ASP.NET framework (in C#) and MsSQL as the project I’m working on at AMD seems to require such. I actually ended up coding a test project for an interview for another company using the framework/database and posted the project (more info below). It’s my first dive into ASP and far from the proper way of doing things, but I’ve already learned quite a bit since then so I think I’m on a good course.

I also won 2nd place in the International Student Design Competition (iSDC 2010) for my group project in CSC318! The conference is being held in Germany and our projects will be presented there soon Unfortunately, no one in our group will be able to attend but we’re going to try to get our poster printed out for display there. We’ll probably be listed on the iSDC website too once that’s done. Look for the Grocery Garden project by The Crew

Finally, I also got myself hardeepsingh.ca because well, why not? It just points to the same host as theootz.com though.

Back to list

No related posts.

What You Need

You’ll need PHP, bash, CRON access and python to get this working. And given the bash/cron requirements, that also means it assumes you’re running a *nix based server. It’s probably possible to get the requirements down, but I just wanted to get something done quick and it’s what I came up with.

The Files

backup.sh

First up is the bash script (based on this one) that does most of the work. It will tar+gzip all your site files, export your database(s), and remove old backup files (+5 days old) when necessary. You should only really have to edit the top part of the script. Make sure you remember what you set your BACKUP_DIR and FILES_DIR to – you’ll need them in other scripts later on.

#!/bin/sh

# EDIT THIS PART ------------------
# where all these backup scripts are stored
BACKUP_DIR=/home/theootz/backups

# where you want to keep all the actual backup folders
FILES_DIR=/home/theootz/backups/files

# where all your html files reside
HTML_DIR=/home/theootz/public_html

# your MySQL database name, username and password
SQL_USER1=myusername1
SQL_PASS1=mypass1
SQL_DB1=mydb1

# a second database
# Uncomment these lines if you want to back up a second database.
# Also uncomment the line specified below!
# You can also add more databases and follow the same format.
#SQL_USER2=myusername2
#SQL_PASS2=mypass2
#SQL_DB2=mydb2
# STOP EDITING ------------------

THEDATE=`date +%Y-%m-%d_%H-%M-%S`

mkdir -p $BACKUP_DIR/files

mysqldump -u${SQL_USER1} -p${SQL_PASS1} ${SQL_DB1} > ${FILES_DIR}/${THEDATE}_db1backup.sql

# Uncomment the line below if have a second database to back up
#mysqldump -u${SQL_USER2} -p${SQL_PASS2} ${SQL_DB2} > ${FILES_DIR}/${THEDATE}_db2backup.sql

tar -cf ${FILES_DIR}/${THEDATE}_site.tar ${HTML_DIR}
gzip ${FILES_DIR}/${THEDATE}_site.tar

find ${FILES_DIR}/*site* -mtime +5 -exec rm {} \;
find ${FILES_DIR}/*db* -mtime +5 -exec rm {} \;

backup_upload.py

Next is the Python script to upload the files. It uses a PHP library that I’ll give more information on below.

import sys
import os

########## Make sure these values match the ones you set in your shell script!
BACKUP_DIR = "/home/theootz/backups"
FILES_DIR = "/home/theootz/backups/files"

# this needs to be changed to reflect your settings on your s3 account
S3_BUCKET = "theootz"
S3_FOLDER = "mysite/backups"
##########

j = os.path.join
fname = os.path.basename(sys.argv[1])
cmdstr = j(BACKUP_DIR, "s3.php") + " put " + \
    j(FILES_DIR, fname) + " " + \
    j(S3_BUCKET, j(S3_FOLDER, fname.split("_")[-1]))

os.system(cmdstr)

PHP S3

The python script uses a PHP library to do all the S3 uploading work. The original is available here.

I had to edit a few paths in the php library to get it to work, so if you want to download my version it’s available here:

Regardless of which one you pick, make sure you open and edit the s3.php file! You’ll need to put your S3 keys in there (near the top of the file). It will look something like this:

// S3_CONF file
$aws_key = null;
$aws_secret = null;

You’ll want to put your access and secret keys there as strings, as follows:

// S3_CONF file
$aws_key = "my access key here";
$aws_secret = "my secret key here";

How To Get It All Working?

Ok, so you’ve got the files. Now upload them to your web server to your BACKUP_DIR folder (whatever you choose it as, I picked /home/theootz/backups). Finally, take the files from the the PHP S3 library and put them in your BACKUP_DIR as well. This should leave you with the following files in a single folder:

• backup.sh
• backup_upload.py
• s3sync.php
• s3.php
• libs3.php
• libaws.php

I’m pretty sure most of the files from the S3 library aren’t needed but once again not something I bothered to check.

Now you’ll want to enable execution for all our scripts. You can do this by running the following commands while in your BACKUP_DIR folder:

chmod u+x backup.sh
chmod u+x backup_upload.py
chmod u+x s3.php

At this point, you can go ahead and run backup.sh and see if everything worked. If it did, then we can set up the cron jobs to have this all work automatically for us

You’ll want to add two entries to cron. The first entry will run our backup.sh on a daily basis for us. This will give us 5 days worth of backups at any one point. The second script will upload our latest backup to the S3 service, replacing whatever is already there.

1       1       *       *       *       /path/to/BACKUP_DIR/backup.sh
1       1       *       *       *       find /path/to/FILES_DIR/* -mtime 0 -exec python /path/to/BACKUP_DIR/backup_upload.py '{}' \;

Make sure you edit the paths to reflect the way you have it configured for your system! For example, my cron jobs look like this:

0       0       *       *       *       /home/theootz/backups/backup.sh
0       0       *       *       0       find /home/theootz/backups/files/* -mtime 0 -exec python /home/theootz/backups/backup_upload.py '{}' \;

Done!

And that’s it Your website along with databases and other content will now be backed up daily to the server itself, and uploaded to your S3 account weekly. If you wanna be super extra paranoid you can download the backups from the FILES_DIR yourself or use rsync or something similar to automate that as well. One idea is perhaps sending it to your e-mail – use that g-mail space for something right?

Hope that’s some help to someone

No related posts.

There’s the new “Projects” section which is powered by a nice little back-end script to let me easily manage links and data to all the projects I’m working on, or have worked on.

I added a “LifeStream” – based on the same concept that facebook uses to aggregate many social sites into one ‘stream’. Ironically, I disabled facebook from showing up on the stream because it was posting way too much. Though the upside is, it gave me an idea for getting my whole cross-posting idea to work that I’m going to explore later. Originally I was trying to use my blog to push data to all the other services I use, but I figured I could also use facebook to push to my blog and from my blog to the rest of the services. I’d have to work out how I could make my blog still push unique posts to facebook and not push those specific posts but I haven’t really looked at any code yet so I’ll figure it out then.

Next, I finally figured out how to set up my blog as an OpenID provider So instead of remembering that convoluted verisign address that I *ALWAYS* forget, I can just use my blog address. Nice and simple.

I’ve added a massive caching system and found another use for my Amazon S3 account! I decided to create a new s3 bucket and linked it to Amazon’s Cloud Front. All media, most javascript, css, etc… is now served from local amazon end points (@ cloud.theootz.com) instead of my hosting provider I haven’t tested to see if this has resulted in any real speed improvement, but at least at a glance it seems to help a lot.

And to end it off, I found a wordpress plug-in for managing resume’s that I’m looking at using for posting my CV up on the site. Haven’t fiddled with it much yet but it seems to hold promise. Also another plug-in to find and display ‘related posts’ at the bottom of each post I make. Gotta love wordpress plug-ins

Related posts:

1. video+cross post test, second attempt
2. Da Stank Bank “Uploader”

http://theootz.com/dastankbank/

No related posts.