ec2-manage-snapshots

Based on code from Oren Solomianik’s ec2-delete-old-snapshots ec2-manage-snapshots does things a little differently.

Oren’s script would delete any snapshots older than n days for (a) given volume(s). I improved the code a little bit so as to handle a –region parameter ( –region eu-west-1 is an example) an –noop so as to tell us what it would do but not actually do it. That’s the ec2-delete-old-snapshots you’ll find in the included archive.

[ad#Amazon Cloud inline]

Because I wanted to manage snapshots differently I heavily modified the original script to create ec2-manage-snapshots so that for a given volumes, it would keep snapshots made:

  • in the last 7 days
  • the past 4 sundays
  • every first day of the month

and erase the rest.

Its behavior can be easily modified but I wanted to make sure I kept monthly snapshots, last 4 weekly & last 7 days worth. It assumes you’re doing one daily snapshot of each volume.

Download here: ec2-manage-snapshots

Failsafe
Like its ancestor, the program checks to see that each vol-id entered on the command line has at least one newer snapshot than the deletion date, to prevent deletion of all snapshots of a certain volume. The deletion process will commence for a volume only if such a snapshot was found. More importantly, you can use the --noop command line option or the constant in the code (NOOP) and the script will tell you what it would have deleted but won’t actually delete anything.

Disclaimer: This program is in development. Although it has been tested and worked on production environments, it can’t be guaranteed to perform without unexpected results. Use at your own risk.

33 thoughts on “ec2-manage-snapshots

  1. I’ll put it under Apache License as soon as I have the time. Consider it so for now. The original had no license info nor did its google code site.

  2. On my system, I was getting a fatal error:

    Fatal error: Class ‘XsltProcessor’ not found

    This was on Ubuntu 9.10 desktop. It was because I didn’t have the package php5-xsl installed. To get around this:

    sudo apt-get install php5-xsl

    Then I re-ran the script and all was working.

  3. It’s also worth noting that on most systems running PHP prior to 5.3, it is not possible to use long options, so the –noop and –region will not work. It’s actually quite dangerous for –noop to fail unexpectedly, for obvious reasons! I saw this error:

    Warning: getopt(): No support for long options in this build

    I would recommend updating to PHP 5.3, but we’re using Drupal 6 which is not compatible with it yet.

    As a workaround, I suggest adding short versions of each option, so at least people have the choice.

    • Well, Oren’s script main goal is to delete the last n days. Yours deletes n backups. The script I added (based on Oren) sees things more like backups and always keeps the monthlies, a month worth of weeklies and the last seven days of dailies. I guess it depends on what you need.

      As for the library requirements, I have to say I didn’t look too much into it since my home system and my servers seemed to have all that it needed. Your mileage (did) may varies.

  4. Hi Cloud Walker,

    I noticed one issue with the script… It will fail if the snapshot is part of an EBS AMI snapshot. The issue with 5.3 compatibility could probably be overcome by using Zend Framework’s Zend_Console_Getopt class. I write a lot of PHP and could help really spruce this up! :)

    BTW, thanks for the post.

      • Hey Cloud Walker! Sorry for the long delay. I am going to take a look at this today to see if there’s anything I can help contribute.

      • Howdy Again,

        I spent the last 8 hours really putting some polish on this script. Here’s a link to the project files. (http://www.upsellnetwork.com/ec2-manage-snapshots.export.tar.gz)

        Some of the improvements are:
        =========================================
        * OO Redesign of the logic
        * Accounts for more than one backup a day
        * Uses Amazon’s newest API
        * Handles snapshots that are in use by AMI images
        * access key and secret access key can now be included as command line arguments
        * configuration has been moved outside of the script, so there should be no editing there.
        * added validation for amazon regions with graceful exit when a bad region is supplied
        * added quiet mode

        Right now the entire Zend Framework is included under the library directory, but I bet I can weed that down.

        When unzipping the script, you will have to create a file called config.ini under the /etc directory. There is a config.ini-dist to be used as an example. To see all of the options, just run ec2-manage-snapshots.php –help

        Let me know what you guys think!!!! :)

        Best,
        Nick

    • Great work but I would have rather you put it on github since we already have a project there. Also, I’d rather we keep the same name otherwise there will be confusion?

  5. I haven’t really played with GitHub before. I’ll take a look in a few hours and try to integrate what I’ve done with your project. If that all works, I’ll just close the project at Google. Were you able to run the script?

  6. Pingback: Manage EBS snapshots with a python script

  7. Pingback: Making the move from Centos 5 to Ubuntu Server | Simon Morley | Digital Savage

  8. I am not a PHP developer and want to convert your script to C#. Before I take the time to learn PHP for the conversion, I want to make sure that your script does not prune the original snapshot. I am under the belief that pruning the original snapshot will invalidate the subsequent snapshots. Please let me know.

    • “I am under the belief that pruning the original snapshot will invalidate the subsequent snapshots”. That’s actually not true. AWS behind the scenes always keeps enough information to be able to restore a snapshot, no matter what other snapshots still exist.

  9. I believe you and I am trying to find the post that made me think I had to keep the original… no luck. Anyway, it is easy to prove with some testing. Thanks for your help and your script.

  10. I’m using Nick Millers fork of this. I’m running an Ubuntu 10.04 server with PHP 5.3

    When I run this command, this is all I get. I can’t tell if it’s working or not.

    PHP 5.3.2-1ubuntu4.7 with Suhosin-Patch (cli) (built: Jan 12 2011 18:36:08)
    Copyright (c) 1997-2009 The PHP Group
    Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies

    I have only started creating snapshots over the last couple of days, so maybe it doesn’t have anything to delete.

    Any tips appreciated.

  11. I found that I had a problem before in the script and now I’ve figured out that I wasn’t passing — args in the command. This is what I’m getting now.

    $ sudo /usr/bin/php -f /opt/scripts/bin/ec2-manage-snapshots.php — args -v vol-9cc356e5

    There was an error running /opt/scripts/bin/ec2-manage-snapshots.php
    Error Message: The provided “vol-9cc356e5″ volume does not exist

    I’m 100% sure that the volume with that volume id does exist. Any ideas?

    if I run ec2-describe-volumes –region eu-west-1, I see vol-9cc356e5 in the list of my volumes.

  12. You have a bug when using a different region than US.

    Although you give a choice to select a region in the config file, it is never passed to the Zend EC2 classes.

    This meant they were always talking to the US webservice and I never got any results back

    When I tried changing the bin/ec2-manage-snapshots.php file to include this:

    Zend_Service_Amazon_Ec2_Ebs::setRegion(‘eu-west-1′);

    It worked :)

  13. Hi,
    Great script and very useful to me. I’ve forked and modified it to handle specifying snapshots using tags. This is because of the way I manage my volumes. I have a single instance which is automatically rebuilt by Chef it if fails. To do this it starts new volumes from existing snapshots. This means that volume ids don’t work but tagging the snapshots does.

    The commit is: https://github.com/ejsarge/ec2-purge-snapshots/commit/3d942a92cc4363d7f7a2d0d0bcdf8ed12580e289

    I’ve added a pull request to your queue. Feel free to ask me for whatever licence you need.

    Note that I had to refactor the main code in order to do this so the commit looks scarier than it is.

    Note that I’m a Ruby and Git newbie so my apologies if I’ve screwed something up.

    Thanks for the hard work.

    Cheers,
    Edward

  14. Pingback: Amazon EC2 Backup: Purging old snapshots | TrailHunger.com Technical Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>