Hard link issues (was: Incredibly slow i/o to NAS server)

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Hard link issues (was: Incredibly slow i/o to NAS server)

Robert Nichols-2
On 12/05/2016 03:26 PM, Robert Nichols wrote:
> On 12/05/2016 09:28 AM, Joe Steele wrote:
>> I believe the harlink problems you describe are identified in this bug report (that I submitted in 2009):
>> http://savannah.nongnu.org/bugs/?func=detailitem&item_id=26848
>> Attached to the report are patches to fix the issue.  Some distros (which I think include Debian, Ubuntu, & Suse) have since been including the patches as part of their packaging.
> Those patches look like they address the problems with checksums being misplaced or lost, but they don't appear to have anything to do with what I see as the greater problem of sets of hard links being broken up and inconsistencies between the hard link counts in the metadata and the mirror.  I finally got rdiff-backup to build (needed a patch for librsync >= 1.0.0), so I can do some testing later.

Well, after two months of use I can confirm that those patches _do_ appear to solve the problem of hard links being broken up or inconsistent.  I still don't understand how, but it works.  My hard link audit hasn't found any issues since I installed those patches.

Doing the hard links correctly does aggravate the problem of tiny "zero-diff" files being created, mainly whenever a file has a change in its hard link set, but also under some other, hard to pin down conditions.  In a ~500GB archive I found I had accumulated over a million (10^6) such files, using ~4.3GB of space.  I now have a new audit that gets rid of those.  I've really got to make another stab a learning Python so that I can fix those problems at the source.  So far, I've only succeeded in being devoured by a snake.

Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

rdiff-backup-users mailing list at [hidden email]
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki