How much extra space should there be on a target file system?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How much extra space should there be on a target file system?

Theodore Wynnychenko
Hello

I have been searching for an answer or guidance to this question, but have
not been able to find anything, so I will ask here.

I am sorry if it has been asked/answered before.

In any case, I am planning on "updating" my backup hardware, and will
probably switch from rsync to rdiff (I have used rsync for many years, and
have been happy with it, except for that one time years ago when I deleted a
bunch of files and did not notice until about a day later, and 10 minutes
after the final backup had been synced - a bit disappointing).

Anyway, I am wondering if there is any guidance on how much larger the
target file system should be compared with the source file system when using
rdiff?

Clearly, rdiff stores more information, so, in theory, if you have two file
systems of exactly the same size, and the source is completely full, then
rdiff would have nowhere to store any historical changes.

Is there a rule-of-thumb, or some sort of calculation that will give
guidance on how much extra space there needs to be on the target file system
to reliably not have an instance when the target file system becomes full,
while the source still has space on it?

It seems this would be based on the frequency of updates, the size of the
file system, the number and size of files, the types of changes made to
those files, how long backups are maintained, and probably other things.

I am not looking for anything exact, more just a way to guesstimate.  But, I
am also not opposed to math either, if there is a calculation that can help
determine the target file system's appropriate size.

Thanks
Ted




_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: How much extra space should there be on a target file system?

rdiff-backup-users mailing list
On Sun, 1 Sep 2019 at 19:39, Theodore Wynnychenko <[hidden email]> wrote:

> Hello
>
> I have been searching for an answer or guidance to this question, but have
> not been able to find anything, so I will ask here.
>
> I am sorry if it has been asked/answered before.
>
> In any case, I am planning on "updating" my backup hardware, and will
> probably switch from rsync to rdiff (I have used rsync for many years, and
> have been happy with it, except for that one time years ago when I deleted
> a
> bunch of files and did not notice until about a day later, and 10 minutes
> after the final backup had been synced - a bit disappointing).
>
> Anyway, I am wondering if there is any guidance on how much larger the
> target file system should be compared with the source file system when
> using
> rdiff?
>
> Clearly, rdiff stores more information, so, in theory, if you have two file
> systems of exactly the same size, and the source is completely full, then
> rdiff would have nowhere to store any historical changes.
>
> Is there a rule-of-thumb, or some sort of calculation that will give
> guidance on how much extra space there needs to be on the target file
> system
> to reliably not have an instance when the target file system becomes full,
> while the source still has space on it?
>
> It seems this would be based on the frequency of updates, the size of the
> file system, the number and size of files, the types of changes made to
> those files, how long backups are maintained, and probably other things.
>
> I am not looking for anything exact, more just a way to guesstimate.  But,
> I
> am also not opposed to math either, if there is a calculation that can help
> determine the target file system's appropriate size.
>
> Thanks
> Ted
>

Just a user here who backups up their home Linux desktop using
rdiff-backup.

So it all depends on your rate of change of backed up data and how long
you want to keep previous backups for.

As an example here are some figures from my daily backup:

    Backup                   Size    Pct
    --------------       --------    ----
    Current mirror       42.8 GiB    100%
    Day 406 increment    65.4 GiB    150%
    Day 804 increment    87.2 GiB    200%

So for *MY* data and *MY* rate of change, 150% is enough for over a year
of daily backups.  Make sure your backup file system has enough inodes
as well as free space.

Mike
_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
Reply | Threaded
Open this post in threaded view
|

Re: How much extra space should there be on a target file system?

Patrik Dufresne-2
Hello Ted,

As you mentioned, It depends alot on your specific usage and how long you
want to store the data.

> Anyway, I am wondering if there is any guidance on how much larger the
target file system should be compared with the source file system when using
rdiff?

Here the rules I have when I have little information about he filesystem
usage:
1. For business, I multiply the average data size by how many year they
want to keep.
So if the business have around 150GiB of data and want to keep them for 5
year: 150 x 5 = 750GiB is a good start. That also account for the business
growth and files changes.

2. For desktop users is different, the data change less often. I usually
just double the capacity. For a user with 250GiB, I will plan 500GiB.

Hope it's helping you to plan the capacity.

--
Patrik Dufresne Service Logiciel inc.
http://www.patrikdufresne.com <http://patrikdufresne.com/>/
514-971-6442
130 rue Doris
St-Colomban, QC J5K 1T9


On Sun, Sep 1, 2019 at 4:55 PM Mike Fleetwood via rdiff-backup-users <
[hidden email]> wrote:

> On Sun, 1 Sep 2019 at 19:39, Theodore Wynnychenko <[hidden email]>
> wrote:
>
> > Hello
> >
> > I have been searching for an answer or guidance to this question, but
> have
> > not been able to find anything, so I will ask here.
> >
> > I am sorry if it has been asked/answered before.
> >
> > In any case, I am planning on "updating" my backup hardware, and will
> > probably switch from rsync to rdiff (I have used rsync for many years,
> and
> > have been happy with it, except for that one time years ago when I
> deleted
> > a
> > bunch of files and did not notice until about a day later, and 10 minutes
> > after the final backup had been synced - a bit disappointing).
> >
> > Anyway, I am wondering if there is any guidance on how much larger the
> > target file system should be compared with the source file system when
> > using
> > rdiff?
> >
> > Clearly, rdiff stores more information, so, in theory, if you have two
> file
> > systems of exactly the same size, and the source is completely full, then
> > rdiff would have nowhere to store any historical changes.
> >
> > Is there a rule-of-thumb, or some sort of calculation that will give
> > guidance on how much extra space there needs to be on the target file
> > system
> > to reliably not have an instance when the target file system becomes
> full,
> > while the source still has space on it?
> >
> > It seems this would be based on the frequency of updates, the size of the
> > file system, the number and size of files, the types of changes made to
> > those files, how long backups are maintained, and probably other things.
> >
> > I am not looking for anything exact, more just a way to guesstimate.
> But,
> > I
> > am also not opposed to math either, if there is a calculation that can
> help
> > determine the target file system's appropriate size.
> >
> > Thanks
> > Ted
> >
>
> Just a user here who backups up their home Linux desktop using
> rdiff-backup.
>
> So it all depends on your rate of change of backed up data and how long
> you want to keep previous backups for.
>
> As an example here are some figures from my daily backup:
>
>     Backup                   Size    Pct
>     --------------       --------    ----
>     Current mirror       42.8 GiB    100%
>     Day 406 increment    65.4 GiB    150%
>     Day 804 increment    87.2 GiB    200%
>
> So for *MY* data and *MY* rate of change, 150% is enough for over a year
> of daily backups.  Make sure your backup file system has enough inodes
> as well as free space.
>
> Mike
> _______________________________________________
> rdiff-backup-users mailing list at [hidden email]
> https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> Wiki URL:
> http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
_______________________________________________
rdiff-backup-users mailing list at [hidden email]
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki