3 # Duplicity Control Tool -- description at far below (after "exec duplicity")
7 ** Usage: dupltool [options] source url
8 ** Source directory and store url are required, and the url
9 ** MUST be of form "pexpect+scp:/\$HOST/\$REMOTEDIR"
14 #-- Grab command line arguments
19 #-- Confirm that the STORE url was of expected format
21 if [ -n "${STORE##pexpect+scp:/*/*}" ] ; then
22 echo "** Bad store URL: $STORE" >&2
23 echo "** Required format: pexpect+scp:/$HOST/$REMOTEDIR" >&2
27 ## Helper function to read duplicity pathnames and prefix them with
28 ## their from and to months, as in ${FROM}${TO}:${FILENAME}
29 ## both $FROM and $TO are 6 digits (yyyymm).
30 remote_from_to_month_prefix() {
31 sftp $SFTP <<< "ls" | while read F ; do
33 duplicity-full-signatures.*) echo "${F:26:6}${F:26:6}:$F" ; ;;
34 duplicity-full.*) echo "${F:15:6}${F:15:6}:$F" ; ;;
35 duplicity-inc.*) echo "${F:14:6}${F:34:6}:$F" ; ;;
36 duplicity-new-signatures.*) echo "${F:25:6}${F:45:6}:$F" ; ;;
41 THISMONTH="$(date -u "+%Y%m")" # UTC date
43 SFTP="${STORE##pexpect+scp://}"
45 REMOTEDIR="${SFTP#*/}"
46 SFTP="sftp://$SFTP" # openssh-client=1:8.4p1-5+deb11u1
47 #SFTP="$HOST:$REMOTEDIR" # openssh-client=1:7.6p1-4ubuntu0.7
49 # Obtain remote pathnames with from-to prefix in reverse date order,
51 FILES=( $(remote_from_to_month_prefix) )
53 if [ -n "$FILES" ] ; then
54 THATMONTH="${FILES[0]:6:6}" # to-month of newest
55 if [ "$THISMONTH" != "$THATMONTH" ] ; then
56 echo "** New monthly snapshot; stashing daily into .$THATMONTH" >&2
57 { # Generate command stream for sftp
58 echo "mkdir .$THATMONTH"
60 for F in ${FILES[@]} ; do
61 [ "${F:0:6}" != "$THATMONTH" ] && break
62 [ -n "$FX" ] && echo "rename $FX .$THATMONTH/$FX"
64 [ -z "${FX%duplicity-full*}" ] && break
67 echo "** Stashing into .$THATMONTH completed" >&2
71 exec duplicity ${DUPL[@]}
74 # Duplicity Control Tool
75 # ======================
77 # This Duplicity Control Tool should be run as a daily anacron job (or
78 # daily as a cron job) so as to make regular snapshots of a directory
79 # tree using duplicity with the pexpect+scp backend.
81 # Ideally the backup store should be set up with password-less scp
82 # access, preferrably as a non-root remote user, or at the very least
83 # be accessed by means of a local ssh-agent.
85 # The store is then managed by this Duplicity Control Tool to keep a
86 # multi-level history, which ends up as a succession of monthly
87 # snapshots with a most recent tail of daily snapshots. Specifically,
88 # at the first snapshot event each month, the store is revised by
89 # stashing away the snapshots of the most recent previous month, to
90 # leave the first one (or the most recent full) as the start point for
91 # a new snapshot, which thus becomes a "monthly" snapshot.
93 # The chain of daily snapshots from that first (monthly or full)
94 # snapshot remain stashed in the ".%Y%m" directory.
96 # Note that the Duplicity Control Tool considers "the month of the
97 # last snapshot or full" to be "the previous month" regardless of how
98 # many months back from today that might be, i.e., it will stash
99 # snapshots just as if that month was the prior month. In other words,
100 # if you would happen to have a huge temporal hole in the snapshot
101 # series you might consider running a plain duplicity snapshot first
102 # and then let the subsequent, regular runs of the Duplicity Control
103 # Tool build its multi-level snapshot history henceforth.
105 # === These are the examples of the various duplicity snapshot files:
106 # duplicity-full.20230128T010719Z.vol1.difftar.gz
107 # duplicity-full-signatures.20230128T010719Z.sigtar.gz
108 # duplicity-new-signatures.20221216T230345Z.to.20221217T230403Z.sigtar.gz
109 # duplicity-inc.20230128T010719Z.to.20230130T082241Z.manifest
110 # duplicity-inc.20230127T061825Z.to.20230130T081002Z.vol1.difftar.gz