Format of backup matters more (to me) than which software that does it

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Format of backup matters more (to me) than which software that does it

rhkramer
Two introductory things:

   * Happy Thanksgiving to all!

   * I am not very good about doing regular backups, or backing up everything
that should be backed up

I'm writing this to maybe spark a discussion which might include links to
sources that provide the answers I'm looking for (for which I'm looking ;-).

To me, I don't care too much or at all what software does my backups, I would
like to have a convenient almost one button (one cron job) method to do my
backups, and then I care more about how the backups are organized / formatted,
with the idea (among others) that a variety of software could write to or read
from the backups as organized.

I guess the exception to not caring about the software is that I do recognize
that programs that use rsync are useful at times, especially when the backup
is remote.  (But using rsync could be an option of any backup program, not
just, for example, one like rdiff, which, iiuc, is written specifically to
(always?) use rsync.)

Is there any standard for the organization of backed-up files?  Maybe even a
default standard (something that several backup programs use)?

I'm going to elaborate -- I mean things like the naming of files, the naming of
directories of files, compression of files (or tarring), encryption of files,  
whether perhaps they are stored as diffs, etc.

I want to elaborate further, but I don't know how best to do that -- let me
describe (in probably too long a writeup) how I'd like my backup files to be
stored.

First of all, I want to distinguish between backups of the system and backups
of user data (and sort of an inbetween one, backups of user / system
configuration data).

Next, let me say that I personally don't care too much about having a backup
to do a bare metal reinstall of my system, I am much more interested in what I
consider my "user data" -- files that I've created or collected (whether they
be programs, source code, text files, spreadsheets, photos, sketches, videos,
audios, ...).  

If I personally have a system "occurrence" that requires a bare metal
reinstall, I am more likely to install a more recent system from scratch.  For
example, my "daily driver" is a Debian Wheezy system -- if it dies, I will
probably install the latest Debian (Buster) and then restore my user data,
rather than restoring the Debian Wheezy system.  

(There could be exceptions or reasons not to do it that way -- I hear people
complaining about systemd stuff, and having problems (either real problems of
systemd or just new approaches that are required that a user is not yet
familiar with), so if I found a lot of problems I might go back to Wheezy (or
something inbetween, like Jessie).  

If I did go back to Wheezy, my first thought is that I'd reinstall Wheezy from
scratch, then consider adding the user / system configuration files (the stuff in
/etc and /home/<username./.*) -- that would probably be on a selective basis
as I found things that weren't working as I desired after the "from scratch"
install.

Anyway, another thought occurred to me as I write this -- maybe what's needed
(maybe what I would like to have) is a very easy to read and modify backup
configuration file, that many backup programs could read, write, and utilize.

In it, I could specify, by directory | filename | (maybe) partition, which files
to backup, how often to back them up, where to store them (and organize them),
whether to encrypt them, tar them, compress them, store them as diffs (I forget
the right terminology, so let me just say forward or reverse deltas), how to
name new directories | files that might be created, whether to use the rsync
type transmission algorithm, and maybe other things that don't come to mind
atm.

Then, knowing how the backup is stored, I'd have the freedom (and knowledge)
to restore anything that I needed to restore -- a single file from yesterday,
last week, or wheneve; all my user files; all my encrypted files, and even a
bare metal complete reinstall (including or not including my user data).

Of course, I'd like the backup program to handle at least some of those
restorations, but I'd also like the freedom to use other OS (CLI?) tools to go
into the structure of the backedup files and do whatever I wanted / needed to
do.

So, maybe rather then elaborating any further, I'll just ask if anything like
a "standard" backingup configuration file exists, or a standard organization of
backups, or if several such candidates exist, or if neither of the above, does
anyone else see the advantage of such an approach?

(One advantage that I hope I've alluded to but maybe not explicitly stated is
that, if one backup program disappears, other backup programs could continue
to work with the "standard" backingup configuration file and the standard (or
the user customized) organization of the backed up files.

(I won't go much into organization of the files, but one such "choice" to be
made is how subsequent backups are organized in directories, for example, each
new backup in a new directory, with the same basic name, but with a suffix to
indicate the date (and time?) when that backup was made.  (And, are backups
older than a certain time frame deleted, or compressed, or moved to another
storage medium?))












Reply | Threaded
Open this post in threaded view
|

Re: Format of backup matters more (to me) than which software that does it

ewl+rdiffbackup
To make it short:
* I don't know of any such standard format for backups
* rdiff-backup can't disappear as it's open source
* rdiff-backup can cover your use cases as I kind of understood them, you just need to use the right parameters and put them e.g. in a cron job
* rdiff-backup can't help you being better at making backups...

On November 28, 2019 3:01:23 PM UTC, [hidden email] wrote:

>Two introductory things:
>
>   * Happy Thanksgiving to all!
>
>* I am not very good about doing regular backups, or backing up
>everything
>that should be backed up
>
>I'm writing this to maybe spark a discussion which might include links
>to
>sources that provide the answers I'm looking for (for which I'm looking
>;-).
>
>To me, I don't care too much or at all what software does my backups, I
>would
>like to have a convenient almost one button (one cron job) method to do
>my
>backups, and then I care more about how the backups are organized /
>formatted,
>with the idea (among others) that a variety of software could write to
>or read
>from the backups as organized.
>
>I guess the exception to not caring about the software is that I do
>recognize
>that programs that use rsync are useful at times, especially when the
>backup
>is remote.  (But using rsync could be an option of any backup program,
>not
>just, for example, one like rdiff, which, iiuc, is written specifically
>to
>(always?) use rsync.)
>
>Is there any standard for the organization of backed-up files?  Maybe
>even a
>default standard (something that several backup programs use)?
>
>I'm going to elaborate -- I mean things like the naming of files, the
>naming of
>directories of files, compression of files (or tarring), encryption of
>files,  
>whether perhaps they are stored as diffs, etc.
>
>I want to elaborate further, but I don't know how best to do that --
>let me
>describe (in probably too long a writeup) how I'd like my backup files
>to be
>stored.
>
>First of all, I want to distinguish between backups of the system and
>backups
>of user data (and sort of an inbetween one, backups of user / system
>configuration data).
>
>Next, let me say that I personally don't care too much about having a
>backup
>to do a bare metal reinstall of my system, I am much more interested in
>what I
>consider my "user data" -- files that I've created or collected
>(whether they
>be programs, source code, text files, spreadsheets, photos, sketches,
>videos,
>audios, ...).  
>
>If I personally have a system "occurrence" that requires a bare metal
>reinstall, I am more likely to install a more recent system from
>scratch.  For
>example, my "daily driver" is a Debian Wheezy system -- if it dies, I
>will
>probably install the latest Debian (Buster) and then restore my user
>data,
>rather than restoring the Debian Wheezy system.  
>
>(There could be exceptions or reasons not to do it that way -- I hear
>people
>complaining about systemd stuff, and having problems (either real
>problems of
>systemd or just new approaches that are required that a user is not yet
>
>familiar with), so if I found a lot of problems I might go back to
>Wheezy (or
>something inbetween, like Jessie).  
>
>If I did go back to Wheezy, my first thought is that I'd reinstall
>Wheezy from
>scratch, then consider adding the user / system configuration files
>(the stuff in
>/etc and /home/<username./.*) -- that would probably be on a selective
>basis
>as I found things that weren't working as I desired after the "from
>scratch"
>install.
>
>Anyway, another thought occurred to me as I write this -- maybe what's
>needed
>(maybe what I would like to have) is a very easy to read and modify
>backup
>configuration file, that many backup programs could read, write, and
>utilize.
>
>In it, I could specify, by directory | filename | (maybe) partition,
>which files
>to backup, how often to back them up, where to store them (and organize
>them),
>whether to encrypt them, tar them, compress them, store them as diffs
>(I forget
>the right terminology, so let me just say forward or reverse deltas),
>how to
>name new directories | files that might be created, whether to use the
>rsync
>type transmission algorithm, and maybe other things that don't come to
>mind
>atm.
>
>Then, knowing how the backup is stored, I'd have the freedom (and
>knowledge)
>to restore anything that I needed to restore -- a single file from
>yesterday,
>last week, or wheneve; all my user files; all my encrypted files, and
>even a
>bare metal complete reinstall (including or not including my user
>data).
>
>Of course, I'd like the backup program to handle at least some of those
>
>restorations, but I'd also like the freedom to use other OS (CLI?)
>tools to go
>into the structure of the backedup files and do whatever I wanted /
>needed to
>do.
>
>So, maybe rather then elaborating any further, I'll just ask if
>anything like
>a "standard" backingup configuration file exists, or a standard
>organization of
>backups, or if several such candidates exist, or if neither of the
>above, does
>anyone else see the advantage of such an approach?
>
>(One advantage that I hope I've alluded to but maybe not explicitly
>stated is
>that, if one backup program disappears, other backup programs could
>continue
>to work with the "standard" backingup configuration file and the
>standard (or
>the user customized) organization of the backed up files.
>
>(I won't go much into organization of the files, but one such "choice"
>to be
>made is how subsequent backups are organized in directories, for
>example, each
>new backup in a new directory, with the same basic name, but with a
>suffix to
>indicate the date (and time?) when that backup was made.  (And, are
>backups
>older than a certain time frame deleted, or compressed, or moved to
>another
>storage medium?))