Setting up laptop backups using the EECISbak.sh script

1) BACKGROUND
This script is designed to back laptops up to the EECIS
"smb2.eecis.udel.edu" drive array, though it could be used to back up
to any machine with rsync and SSH access. It does not attempt to
perform file "versioning", rather, it relies on the underlying backup
filesystem to keep older versions of files. For "smb2", it relies on
ZFS. This script merely duplicates the files on the local machine to
the remote machine. 

A backup system is, indeally, simple enough to be robust. This script
re-uses existing tools to the greatest extent possible. The script is
written in "bash", the remote connection is made with "ssh", and the
synchronization is performed using "rsync". These are both tools with
substantial history in the UNIX arena. OSX Macs have them installed as
part of the base system, and they can be installed under Windows using
the "Cygwin" environment (described below).

The script is intended to be invoked periodically approximately once
per hour by the operating-system-dependent periodic-command scheduler
("cron" for Macs and "Task Scheduler" under Windows). The script
checks a time-stamp file to see when a backup was last performed, and
if it has been long enough (12 hours by default) a backup is
made. Laptops are off and on the net, so by trying frequently, there's
a good chance that an attempt will eventually be made when the laptop
is able to make a net connection to the server. The "rsync" software
only transfers the parts of files that have changed so after the
first, full copy, updates should be fast even over a relatively slow
link. Further, it will do an excellent job of picking up where it left
off if it is not able to finish the backup before the laptop is shut
down or disconnected from the network.

2) CONFIGURATION
All files connected to the script should be kept under the EECISbak
directory as directed in installation below. Files with "sys" in the
filename should not be modified by end users since updates to this
software may overwrite them.

The "config" file is "sourced" by the main script and can be used to
set the environment for the backup. While complex shell-script logic
could be included, it's best to use simple "VARIABLE=value" lines. The
"sysconfig" file shows default values (all commented-out with #); if
required, this file could be copied to the "config" file and then
modified.

Configuring which files are included is done by the "include",
"sysinclude", "exclude", and "sysexclude" files. Note that "includes"
and "excludes" may be listed in any file by using "+ " and "- "
respectively (see the sysexclude file for examples). Each file and
directory is matched against the list of patterns in these files, and
the first match is applied, so if the following two lines are found:

+ /Users/roosen/
- /Users/*

then the directory /Users/roosen and all contents will be included
(unless excluded later), and all other contents of /Users/ will be
excluded. If the two lines are reordered, however:

- /Users/*
+ /Users/roosen/

Then since /Users/roosen/ will match the first line, it will be
excluded. Lines ending in / will only match directories, and lines
beginning with / will be matched against the full file pathname. 

Note for Windows laptops: Files that end in ".txt" use
DOS/Windows-style line breaks and those without use UNIX-style line
breaks. Under Windows, either may be modified; if you prefer to use
Windows-native software for text-file editing, you will probably find
the ".txt" versions easier to work with, and the reverse if you
prefer to use Emacs or Vi under Cygwin. Changes will be propagated
from the .txt to the non-.txt or vice-versa automatically when the
EECISbak.sh script runs.

3) INSTALLATION
3.1) Windows
     3.1.1) The backup script depends on several "Cygwin" packages. Cygwin
     is easily obtained and installed, but you must be running as an
     administrator. If Cygwin is already installed, make sure the
     required packages have been installed.
         3.1.1.1) Begin by going to "cygwin.com" and click on the
	 "Install Cygwin Now" button in the upper-right.
	 3.1.1.2) Run "setup.exe" to install (or reconfigure) Cygwin.
	 3.1.1.3) Accept all the defaults (keep clicking "next"). Select
	 "http://mirror.mcs.anl.gov" as the mirror and click
	 "next".
	 3.1.1.4) In the "select packages" panel, click "view" until
	 it shows the "Full" view. Locate and select "bash", "cygutils",
	 "openssh", and "rsync" are selected for installation. Click
	 "next", and wait.

     3.1.2) Download EECISbak.zip from
     "http://www.eecis.udel.edu/~roosen". Unzip in your folder under
     "Documents and Settings" so that all the files are stored in a
     EECISbak folder. Files include:
     	    3.1.2.1) EECISbak.sh: this is the main backup script (bash shell).
	    3.1.2.2) EECISbak.bat: a DOS script that invokes the above
	    for ease of running directly from Windows.
	    3.1.2.3) EECISbka.bat: a DOS script that invokes the above
	    for ease of running non-interactively under Windows using
	    "Scheduled Tasks"
	    3.1.2.4) readme.txt: this help
	    3.1.2.5) sysconfig, sysconfig.txt: configuration files for
	    use by system staff in "UNIX" and "DOS" formats. Copy to
	    "config" or "config.txt" for your own modifications.
	    3.1.2.6) sysexclude, sysexclude.txt: rsync exclude files
	    for system staff defaults in "UNIX" and "DOS" formats.
	    3.1.2.7) crontab: sample crontab entry for use under MacOS.
	    3.1.2.8) pwget: keychain helper script for use under MacOS.
	    3.1.2.9) host_key: the backup host's public SSH key.

     3.1.3) Run the "EECISbak.bat" script (you can just double-click on
     it in Windows Explorer). 
	    3.1.3.1) When it finishes the first time, it will write a
	    file called "authorized_keys.txt". Post the contents of
	    that file into a EECIS Help Request ticket asking for
	    authorization to back up.

     3.1.4) Once your Help Request has been answered, run the
     EECISbak.bat once to ensure that backups will go through.

     3.1.5) Set up EECISbka.bat to run hourly to insure that regular
     backups will occur.
            3.1.4.1) Select "Start->All Programs->Accessories->System
	    Tools->Scheduled tasks", then double-click "Add Scheduled
	    Task". A "wizard" will start up. Browse to the
	    "EECISbka.bat" file as the program to be run. The basic
	    wizard doesn't let you set it to run hourly, so just say
	    "daily". Otherwise, take all the defaults, and be sure to
	    enter your username and password where asked. In the last
	    page, select "Open Advanced Properties for this Task when
	    I click Finish".
	    3.1.4.2) In the "Advanced Properties" pane, select the
	    "schedule" tab. Make sure it starts "Daily" at "12PM". Hit
	    the "Advanced" button. Click on "Repeat Task" and set it
	    to "every 1 hour" with a duration of 24-hours. Click "OK".
	    3.1.4.3) In the "Settings" tab, uncheck both boxes under
	    "Power Mangement".     

3.2) MacOS X
All required utilities are installed by default. To install:
    3.2.1) Download "http://www.eecis.udel.edu/~roosen/EECISbak.zip" and
    unzip it into your home directory so that the script will be found
    in "~/EECISbak/EECISbak.sh". Contents are listed in 3.1.2, above.

    3.2.2) Modify ~/EECISbak/exclude to fit your needs (this is just a text
    file).

    3.2.3) Open a terminal (Applications->Utilitied->Terminal.app) and run
    "~/EECISbak/EECISbak.sh". The first time it is run, it will generate
    an SSH key-pair and construct an authorized_keys.txt file that
    will need to go onto the server.

    3.2.4) Open a help-request ticket at
    https://www.eecis.udel.edu/service with the subject, "EECIS laptop
    backup setup" and post the contents of "authorized_keys.txt" into
    the description. If you have more than one laptop to back up, note
    that they need to be given different names in the server
    authorized_keys.

    3.2.5) Edit the "EECISbak/crontab" file to reflect the location of
    the EECISbak directory. If you already have a crontab file (run
    "crontab -l" to check), add the line in EECISbak/crontab to
    it. Otherwise, run "crontab ~/EECISbak/crontab". 

    3.2.6) Once your help request has been answered, run EECISbak.sh
    again ("by hand") to make sure all is well. You may be asked to
    accept an SSH key from smb2: do so.

    3.2.6) Logs will be created in "EECISbak/log-YYYYMMDDHHMM" (marked by
    the date and time). Check these logs periodically to ensure
    backups are actually taking place.

4) KNOWN PROBLEMS
   4.1) Under Windows, a window pops up every time the .bat file is
   run even if there is no output. I'm sure there's a better way to
   handle running the script that doesn't do that.

   4.2) "cron" is outdated for MacOS: "launchd" is the "right way to
   do it". 
