They stated that they stick by the reasoning that their traffic management hardware/software can sometimes falsely identify traffic from sources as being P2P traffic even though it isn’t. This can result in some traffic being throttled even when it shouldn’t be. Specifically:

As the Company has consistently stated, it does not intentionally shape any non-P2P file-sharing traffic as part of its network management practices and, as such, there are no websites that the Company “targets” for traffic shaping.  However, as the Company has stated in the past, the deployment of any network technology such as technical ITMPs may have some unintended consequences and the Company encourages its customers to bring such unintended consequences to its attention in order to resolve them.

BUT, they’ve done extensive testing with some of the specific sites I mentioned to them and were able to replicate the issue. Specifically hotfile.com, using both free and premium accounts.

As a result of the complaint letter, the Company has performed various tests in an attempt to recreate the reported problem. As accurately indicated in the complaint letter, www.hotfile.com is a storage type of site that leverages http (web) direct downloading mechanisms for the sharing of files. The site offers two types of access, a free access and a premium (paid for subscription) access. Whereas the free access has limitations in terms of the number of concurrent downloads as well as throughput/speed that can be reduced if the site www.hotfile.com is congested, the premium service is not limited in concurrent downloads or throughput/speed as a result of congestion at the site itself.

So they admit something is messed up for ‘some’ of the traffic. But the interesting part is they’ve claimed to already be rolling out a solution I’m wondering if this is the result of many complaints, or specifically this one, but still interesting. In the following quote, DPI is Deep Packet Inspection devices. And to give context, my original complaint was sent to them on the 19th of October, so it definitely gave them time to react to it.

In order to address the issue of inadvertent shaping of www.hotfile.com traffic, there was a requirement to update the signatures used in the various DPI platforms used by the Company. A solution was implemented for the vast majority of the Company’s DPI devices on 2 November 2010 and a second solution for the remaining DPI devices will be implemented by 30 November 2010. As such, www.hotfile.com traffic should no longer be shaped during peak hours after the 30 November 2010.

So it looks like ISPs don’t want to mess with whatever the CRTC deems they should be doing, once it’s been established. The issue is the CRTC giving in so much to ISPs when they lobby for change. UGH.

Also DPI devices freak me out. EVERY packet that leaves from your computer to go to anywhere else, encrypted or not, is inspected by these machines. They look at the actual CONTENTS of the packets to determine what type of traffic they are. It’s like having a packet sniffer constantly monitoring your connection -.-

So what else does the CRTC have planned for our internet? Why, more control for the ISPs and government of course and screwing us over! YAY……..

Currently we’ve got this wonderful issue with FURTHER tracking of what you do on the internet. And just for kicks, lets give the ISPs even MORE control over how much they can limit our bandwidth. Unlimited internet? NOPE. Rogers already doesn’t allow this, and bell charges an extra $30/month for unlimited bandwidth on their current packages. Think of our cell phone plans. $30/month for a measly 1 or 2gb or whatever the hell it is that they offer. And then charging a RIDICULOUS amount of money per additional MEGABYTE. Now imagine that happening to your internet connection on the desktop too. Hell you’re probably already in that position. Imagine it being worse. Ugh…

*edit*
Thought it might be useful to include my original complaint sent to the CRTC as well. If nothing else, could serve as a template for throttling for other sites and such. If they’re gonna throttle P2P traffic then they damned well stick to nothing but that only.

If you want to try and get throttling changed at all…then complain to the CRTC, your local government officials, whatever. Hell, complain to every government official you can get the contact info for even

This complaint is in regard to Bell Internet Services and their Internet Traffic Management Practices.

In January and February 2010, the CRTC followed up with several Canadian ISPs in order to ensure consumers have access to the information they need to make informed choices about the Internet services they purchase. Specifically, the Commission required that several ISPs more prominently display information about their ITMPs on their websites, marketing materials, and terms of service.

This is currently NOT the case with Bell’s Internet Services! Bell has decided to throttle (i.e. lower the bandwidth) on websites beyond only P2P (peer to peer) traffic as stated on their “Network Management” web page http://internet.bell.ca/index.cfm?language=en&method=content.view&content_id=12119

An example of these websites includes HOTFILE.COM.

This is a well known issue amongst it’s customers, but such practices are not at all mentioned on any of their websites, marketing materials, terms of service or acceptable use policies.

Their terms of service http://internet.bell.ca/index.cfm?method=content.view&category_id=550&content_id=11013 has several locations in which they mention the quality and speed of service one should expect:

1) Section 6: “Service” in this case refers to the internet service I have chosen from their available plans. There is no mention of the service being affected by using a particular website here.

2) Section 12: Here, they explain how they are not responsible for bottlenecks somewhere along the connection between my local system and a remote system. However this is a well known issue that affects all its customers, and ONLY their customers. Customers on competing ISPs (such as Rogers) have no such issues with the mentioned website. Also, the speeds obtained and the times for which the speeds are at a lower rate correspond to their P2P traffic management practices.

Further down the Terms of Service page is their Acceptable Use Policy. There is one section in particular which seems relevant:

1) Under the GENERAL heading, it lists actions which are prohibited from being committed while using their Service. In particular, “Restricting or inhibiting any other user from using or enjoying the Internet, impairing the operations or efficiency of the Service or creating an unusually large burden on our networks, or otherwise generating levels of Internet traffic sufficient to impede other users’ ability to transmit or receive information.”

This may be a viable reason for restricting the internet connection speed of a single individual. However, this is not the case. Connection speeds to the web services and web sites I have mentioned are throttled for everyone, regardless of how much bandwidth they use or otherwise.

It is important to note that HOTFILE.COM is not a P2P service. It is a direct download over HTTP, using standard ports. It should not be throttled according to their own practices. Also, having a premium account with the HOTFILE.COM service does not matter – Bell is actively throttling the connection to this website as a whole.

I believe Bell should be required to specify which websites it throttles on their Network Management website in order to allow customers to make truly informed decisions.

Links:
1) http://www.dslreports.com/forum/r24012235-Hotfile-IS-throttled. This page shows screen captures comparing other direct download services and their speeds, with speeds from HOTFILE.com while using a Bell internet service connection. The speeds achieved are at most 256kbit/s – in line with their P2P traffic management practices.

2) http://www.dslreports.com/forum/r24729541-So-Hotfile-is-throttled-During-What-Times Customers have found that the times correspond to the times used for their P2P traffic management practices.

No related posts.

Click to view full image

Just an experiment in some things. For the image:

• Texturing
  • This failed :/ I wanted to try to make the tree look as realistic as possible. Unfortunately, the texture I used was too low of a resolution and I couldn’t get the bump map to work out the way I wanted. Also, I couldn’t figure out how to properly texture the vines. UGH
• Lighting
  • I think this turned out ok I wanted to make the tree blend in with the background. I should have changed the angle of the lighting a bit but it’s not bad I think. Also messing around with HDRI maps, and global illumination.
• Modeling
  • Modeling of the tree, and the vines. Mostly trying to get micro-triangle modeling to work…worked out decently for the vines – couldn’t get it to work at all for the tree
• Compositing
  • Getting the tree to blend in with the background and not look out of place. Pretty much failed here too lol

This is what the image looked like before I edited it in photoshop...

The video was just to see how it the global illumination would hold up in an animation. Some methods end up producing a lot of weird artifacts once you start trying to animate and such. I still have to try to build a scene and move things around inside of it instead of just the camera and see how that turns out. I also did a small bit of post processing on the video – added some film grain, color correcting, and other little things here and there. Since I rendered the video at a very odd resolution, Youtube decided to sample it down to an even lower res. Meh. The video download probably looks a bit better.

Video: Download

Click here to view the embedded video.

Back to list

Related posts:

1. video+cross posting test
2. Da Stank Bank “Uploader”
3. Da Stank Bank Uploader – progress!

What You Need

You’ll need PHP, bash, CRON access and python to get this working. And given the bash/cron requirements, that also means it assumes you’re running a *nix based server. It’s probably possible to get the requirements down, but I just wanted to get something done quick and it’s what I came up with.

The Files

backup.sh

First up is the bash script (based on this one) that does most of the work. It will tar+gzip all your site files, export your database(s), and remove old backup files (+5 days old) when necessary. You should only really have to edit the top part of the script. Make sure you remember what you set your BACKUP_DIR and FILES_DIR to – you’ll need them in other scripts later on.

#!/bin/sh

# EDIT THIS PART ------------------
# where all these backup scripts are stored
BACKUP_DIR=/home/theootz/backups

# where you want to keep all the actual backup folders
FILES_DIR=/home/theootz/backups/files

# where all your html files reside
HTML_DIR=/home/theootz/public_html

# your MySQL database name, username and password
SQL_USER1=myusername1
SQL_PASS1=mypass1
SQL_DB1=mydb1

# a second database
# Uncomment these lines if you want to back up a second database.
# Also uncomment the line specified below!
# You can also add more databases and follow the same format.
#SQL_USER2=myusername2
#SQL_PASS2=mypass2
#SQL_DB2=mydb2
# STOP EDITING ------------------

THEDATE=`date +%Y-%m-%d_%H-%M-%S`

mkdir -p $BACKUP_DIR/files

mysqldump -u${SQL_USER1} -p${SQL_PASS1} ${SQL_DB1} > ${FILES_DIR}/${THEDATE}_db1backup.sql

# Uncomment the line below if have a second database to back up
#mysqldump -u${SQL_USER2} -p${SQL_PASS2} ${SQL_DB2} > ${FILES_DIR}/${THEDATE}_db2backup.sql

tar -cf ${FILES_DIR}/${THEDATE}_site.tar ${HTML_DIR}
gzip ${FILES_DIR}/${THEDATE}_site.tar

find ${FILES_DIR}/*site* -mtime +5 -exec rm {} \;
find ${FILES_DIR}/*db* -mtime +5 -exec rm {} \;

backup_upload.py

Next is the Python script to upload the files. It uses a PHP library that I’ll give more information on below.

import sys
import os

########## Make sure these values match the ones you set in your shell script!
BACKUP_DIR = "/home/theootz/backups"
FILES_DIR = "/home/theootz/backups/files"

# this needs to be changed to reflect your settings on your s3 account
S3_BUCKET = "theootz"
S3_FOLDER = "mysite/backups"
##########

j = os.path.join
fname = os.path.basename(sys.argv[1])
cmdstr = j(BACKUP_DIR, "s3.php") + " put " + \
    j(FILES_DIR, fname) + " " + \
    j(S3_BUCKET, j(S3_FOLDER, fname.split("_")[-1]))

os.system(cmdstr)

PHP S3

The python script uses a PHP library to do all the S3 uploading work. The original is available here.

I had to edit a few paths in the php library to get it to work, so if you want to download my version it’s available here:

Regardless of which one you pick, make sure you open and edit the s3.php file! You’ll need to put your S3 keys in there (near the top of the file). It will look something like this:

// S3_CONF file
$aws_key = null;
$aws_secret = null;

You’ll want to put your access and secret keys there as strings, as follows:

// S3_CONF file
$aws_key = "my access key here";
$aws_secret = "my secret key here";

How To Get It All Working?

Ok, so you’ve got the files. Now upload them to your web server to your BACKUP_DIR folder (whatever you choose it as, I picked /home/theootz/backups). Finally, take the files from the the PHP S3 library and put them in your BACKUP_DIR as well. This should leave you with the following files in a single folder:

• backup.sh
• backup_upload.py
• s3sync.php
• s3.php
• libs3.php
• libaws.php

I’m pretty sure most of the files from the S3 library aren’t needed but once again not something I bothered to check.

Now you’ll want to enable execution for all our scripts. You can do this by running the following commands while in your BACKUP_DIR folder:

chmod u+x backup.sh
chmod u+x backup_upload.py
chmod u+x s3.php

At this point, you can go ahead and run backup.sh and see if everything worked. If it did, then we can set up the cron jobs to have this all work automatically for us

You’ll want to add two entries to cron. The first entry will run our backup.sh on a daily basis for us. This will give us 5 days worth of backups at any one point. The second script will upload our latest backup to the S3 service, replacing whatever is already there.

1       1       *       *       *       /path/to/BACKUP_DIR/backup.sh
1       1       *       *       *       find /path/to/FILES_DIR/* -mtime 0 -exec python /path/to/BACKUP_DIR/backup_upload.py '{}' \;

Make sure you edit the paths to reflect the way you have it configured for your system! For example, my cron jobs look like this:

0       0       *       *       *       /home/theootz/backups/backup.sh
0       0       *       *       0       find /home/theootz/backups/files/* -mtime 0 -exec python /home/theootz/backups/backup_upload.py '{}' \;

Done!

And that’s it Your website along with databases and other content will now be backed up daily to the server itself, and uploaded to your S3 account weekly. If you wanna be super extra paranoid you can download the backups from the FILES_DIR yourself or use rsync or something similar to automate that as well. One idea is perhaps sending it to your e-mail – use that g-mail space for something right?

Hope that’s some help to someone

No related posts.

Click here to view the embedded video.

And of course an embed test.

Related posts:

1. video+cross posting test
2. Speed, projects and features – oh my!
3. An experiment in 3D Max :o

Higher quality vimeo link
Youtube embed…cause I think this should get embedded on tumblr?

Related posts:

1. video+cross post test, second attempt
2. Da Stank Bank “Uploader”
3. An experiment in 3D Max :o