I was faced with the conundrum of finding an automated solution for backing up essential data stored on EBS volumes that were both storage volumes and the root volume of the instance itself. If a MySQL database is being run from the instance there are special procedures one should do before doing running the scripts. Mainly stopping the MySQL server before starting and then re-enabling it after; Googling for such a circumstance turns up a script by Eric Hammond mentioned at this blog (http://alestic.com/2009/09/ec2-consistent-snapshot).

I am using RDS and Multi-AZ for my MySQL databases and am managing the automated backup and retention period quite easily, so I have no need for a more advanced script. I was wondering what scripts are out there or if I could write my own. Googling turned up a few dirty examples, but nothing worthwhile I could find. I did run across this example (http://blog.taggesell.de/index.php?/archives/86-Amazon-EC2-how-to-automatically-create-snapshots-of-attached-EBS-volumes.html) and I actually used it in the first script I ended up writing, however, the second script was the main thing. I needed something to cleanup the EBS backups because I didn’t want them piling up. I set a retention period and delete them after that period. I wrote it in bash using Amazon API tools installed on my Ubuntu server.

Of course, this means you have to have the tool setup correctly for your specific environment, (http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/setting-up-your-tools.html) and (http://blog.taggesell.de/index.php?/archives/66-Managing-Amazon-EC2-virtual-machines-101-part-1-creating-AMI-images.html) if you’re wondering what was in my ./awsapicfg file it was the environment variables needed.

Ready to see the code? First, a backup script that I can call with a schedule cron job.

#EBS Backup Script - Tyler Whitney
. ./.awsapicfg

MY_CERT=cert-X.pem
MY_KEY=pk-X.pem
# fetching the instance-id from the metadata repository
MY_INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)

# get list of locally attached volumes via EC2 API:
VOLUME_LIST=$(ec2-describe-volumes -C $MY_CERT -K $MY_KEY \
| grep ${MY_INSTANCE_ID} | awk '{ print $2 }')

sync

#actually creating the snapshots
for volume in $(echo $VOLUME_LIST); do
ec2-create-snapshot -C $MY_CERT -K $MY_KEY -d "Nightly Backup_$volume_$(date +%m-%d-%Y)" $volume
done

As you can see I am listing ALL the current volumes associated with the current instance I am running the script from. Then I loop through them and created a snapshot for each of them; giving them the “Nightly Backup” name to grep on later. If you need to do this for multiple servers (because the backup script will grep the same name) then you may want to give it a better name like “ServerA Nightly” or something like that. To keep things simple, otherwise you can check on other things in the scripts.

Now, I needed to write the script to delete all items older than a specific time period. Let’s jump right into it.

#Backup cleanup script -- Tyler Whitney
. ./.awsapicfg

MY_CERT=cert-X.pem
MY_KEY=pk-X.pem
# fetching the instance-id from the metadata repository
MY_INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)

# get list of locally attached volumes via EC2 API:
SNAP_LIST=$(ec2-describe-snapshots -C $MY_CERT -K $MY_KEY \
| grep "Nightly Backup" | awk '{ print $2"_"$5 }')

sync

#actually checking and deleting the snapshots
OLD="$(date --date='3 minutes ago' +%s)"
for volume in $(echo $SNAP_LIST); do
check="${volume#*_}"
SNAP="$(date --date=${check:0:19} +%s)"
if (( OLD > SNAP )); then
  ec2-delete-snapshot -C $MY_CERT -K $MY_KEY ${volume%_*}
fi
done

As you can see here, I am doing something very similar to the first script except I am looping through the EBS snapshots rather than the volumes of the current instance. So nothing is attaching the snapshot to the instance, so I grep on the same name I used before, hence my comments above. From there, I convert the date they were created to seconds from the Unix epoch, and I do the same with converting “3 minutes ago” which will delete anything that is older than 3 minutes. Obviously you will probably want to set this to something higher. My retention period on my production box is 8 days, so I have “8 days ago” at that location. I check via a loop and if to see if one is older than the other and if it is I delete the snapshot. Easy as that!

What do you think? Please comment, I will try to respond to any questions I see!

For those of you wondering how I did it. Well, generally when I plan things out too much I tend to screw up. So instead of waiting until somewhere around Thanksgiving, I thought I’d do it tonight. We were in the kitchen and she just got home from Potsdam from visiting her family and she was putting some groceries away and picked up our cat that walked in to say hello. I then told her that Zoey (the cat) and I had been talking, and that I consulted with Stella (the other cat) and decided that we really loved her and that she was a keeper. Then I got down on one knee and asked her to marry me… presenting the ring. She cried and yadda yadda, haha, and then said yes.

P.S. – Those of you asking if I had asked her Dad, the answer is yes, I asked him awhile before when we were all on a trip to Lake Placid for the weekend.

So, today there are a few things on my mind, and I plan on going through them rather quickly because this should be short and I don’t have my own article… yet. I’d like to complain, or rant if you will, about a few things that really get to me.

First things first, this morning at 5 a.m. the nice noise of someone mowing the lawn at West Bay Plaza rang through my apartment windows to kill the peaceful sleep that my girlfriend and I were enjoying. Let me state that whomever it was that decided to mow the lawn at 5 a.m. was not only an idiot for mowing so early in the morning (okay okay so he might just be following orders), but he is also an idiot because he hit every single rock there was in the grass, so the noise of mowing would be interrupted with the beautiful ear wrenching sound of metal to rock scraping. It was so terrible that I got up to look out the window and get a good look at this person mowing the lawn, turned out to be some guy, I didn’t recognize him other than that of the evil rock grinding man from my dreams. This nightmare turned out to be true.

On the same subject of idiots at 5 a.m. this next person has no excuse. Whereas the story above could be simply that the guy was following directions to mow the lawn at 5 a.m. the person I’m about to speak about has no way out. On many occasions, and I mean this as in almost every day, someone pulls into the parking lot of my apartment complex and pulls into a parking space. They then proceed to honk their horn obnoxiously until some other idiot comes out of a nearby apartment. What happens if the person doesn’t come out of the apartment quickly? Well its simple, the asinine idiot continues to honk the horn until the other person surely hears it and comes out. What would I like to say to this idiot honking at 5 a.m.? “Get the ?!#$ off your lazy ass and get out of the car and knock on the door like a normal person,” is the first thing that comes to my mind. I mean, there are plenty of options one has. Very simply, if both you and the person has cell phones, or at least you have a cell phone and the other person has some sort of phone in the apartment, you can tip them off that you are there very easily with a simple phone call. Of course, not everyone can afford cell phones or even wish to have one because of the bills, the contracts, or for whatever reason they might have… you can still get off your ass and use your two legs and arms to go to the door and knock. Also, I know not everyone has two arms and legs, but again I got a look at this idiot and made absolutely sure there wasn’t a good reason behind it… and I was right, there wasn’t, there isn’t, and there never will be. So stop it.

So, now that I have this massive screen I can’t work with anything less. Thats the evil of large screens, you can’t go back to the dinky ones.

I’ve also discovered that when you’re dealing with poor dart players (and/or semi-drunk friends) it takes about two hours to play a game of cricket. I think the other people at the Monopole on Saturday night might have agreed seeing as though they had to wait two hours to play a game of darts. Thats what happens when you get dragged out by friends when you simply state you can’t stay long… you play a game of darts for two hours. But its always good to catch up with old friends.

Quick note to anyone who drives on I-89 in Vermont… if you’re between Grand Isle and Burlington and you drive slow in the left lane… get the hell out of my way. The right lane is where you go when I’m behind you wanting to go faster… some of us have to get to work on time.

Now that thats out of the way, I think I’d like to say good morning to all… thank you for visiting the first post of my reformed Blog… and if you think words and to suggest if you are a little less faint of heart than some of my mild rantings and beautiful thoughts…. then you might want to also check out my other entertaining blog. Its less about me and more for a container of articles I write. They are interesting none-the-less… and you should know how to get there by now.

Thanks for Coming.
I’m on Facebook and Twitter too you know!

I of all people, know the harm of smoking. I smoked for years before I recently quit because of my wish to be healthier and feel healthier. I have bee successful so far and I still plan to continue my smoke free lifestyle.

On the other hand, I have to say I am appalled by some of the advertisement I am seeing on TV today. The cigarette advertisements I see in TV and movies? No! I am appalled by the ANTI- cigarette ads that I see on TV today… and I don’t even like cigarettes. Why would I say this? Well organizations and websites such as http://ovx.org and http://smokefreemovies.ucfs.edu from the University of California in San Francisco’s website annoy the complete fuck out of me. In fact, the commercials make me want to go and buy a pack of cigarettes just to spite the bastards. So whats the big deal some may ask? Why do you care?

First off, imagine a Fight Club without Tyler Durden smoking and drinking… and imagine a Scarface with no guns and no cigarettes? What the fuck would the movie world come to? Would all moves be like Land Before Time with no action and no fun? Cigarettes are banned from paid product placement… good. I think the cartoon camel was a bit much, and if they weren’t trying to aim at children and kids to get them started earlier than what else were they doing… I am against them there. No product like cigarettes should be aimed towards kids… just like alcohol and gun commercials should not be either (in my opinion) … but when they’re old enough to make an educated decision on their own… fucking let them. Heres the deal, the movies people see tobacco in should not be viewed by children anyway! The picture they use as an example on the website for Smoke Free Movies as I’ve linked to above shows Brad Pitt in the movie Fight Club… rated R with killing, sex, nudity, swearing, drugs, violence, guns, the works. If you’re letting your kids watch that movie you need to worry more about your parenting than whether or not companies are allowing cigarettes in movies. Sometimes cigarettes make the character who they are… sometimes cigarettes make the movie with the feel it was meant to have. Stop criticizing the movie industry and Hollywood for doing what they do best. Let the parents do a little fucking parenting and limit what internet access and TV/movie/media access kids have. Its your job after all.

When I DO see cigarettes in kids shows or movies (very rarely) its to tell them how BAD they are. I can agree with no smoking inside laws, such as the strict laws we have here in the state of New York… and I think it will be seen that more states will soon follow. As well as the no smoking withing a certain amount of feet of a public building… even that is OK. But come on people… lets let the smokers smoke on their porches and smoke in their own homes if that’s what they wish. Are we going to go around and tell them to stop smoking… our kids might see? Bullshit… its their own choice… and your children will have their own choices as well. Its your job to teach your own children what is right and what is wrong in your opinion so they can come up with their own values and morals.

On another note, when OVX states “control what goes into your own” … that statement is bullshit. I wish I could control getting their fucking annoying commercials off of TV. If I could control what goes on TV I would stop annoying mop commercials and vaccuums… or better yet, the millions of fucking prescription drug commercials I see on TV. What impact are those having on the fucking children of America? Or the bullshit ableist, racist, and culturally offensive commercials Burger King has in their pocket with small people and people for breakfast and people from different cultures for their “Whopper Big Mac Challeng.” Talk about a fucking stupid commercial… lets let our children think that these “Whopper virgins” love to eat our over 500 calorie garbage burgers and that they wish they had meat such as this! Yes indeed, lets make our children more culturally inept and more stupid so they can love these commercials and become more fat. Just what America needs is to gain more weight! While we’re at it, lets stop the stupid fucking shows on TV that lead people to believe families such as the Osbournes are real and what happens in the Real World MTV House is real. Right? Those shows are just as good for our children as fucking cigarettes. So what is the solution? Stop all of these shows and commercials and movies from being displayed unless we are all 21? No… fuck that. Parenting is the answer… better yet, if you’re offended by something… DON’T FUCKING WATCH IT. Turn the TV off… thats the answer. Don’t let your children watch the garbage you don’t like if thats your opinion… otherwise, shut the fuck up.