Hi All, I'm using rsync (version 3.0.7-1ubuntu1.1 on my Ubuntu 10.4 system) in a shell script to perform local backups of my system on to removable hard disks. (FYI: It's a bit kludgy, but I'm VERY happy with it. Been doing it for a few years. Normally works great. It's extremely fast at creating up-to-date immediately-bootable clones/mirrors (?) of my system. I always store backups in different locations from my office. So, I'm not hogging up bandwidth, or exposing my data to others, by using the internet to do my backups.) My system is a dual boot, sharing Ubuntu with WinXP, for when I (rarely) HAVE to genuinely do something in Win. (Thanks, but no thanks. I don't want to do a virtual machine.) Are there any rsync wizards out there? My problem is this: Lately, when rsync updates my WinXP partition (by this command)... rsync -Hvxrlt --delete /mnt/tempWindows/ /mnt/backupWindows ... I get the following error messages ...
*sending incremental file list*
*pagefile.sys*
*rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)*
*deleting Documents and Settings/etc. ... *
*((Here, it seems to remove various deleted WinXP files from the target partition.)) *
*deleting Documents and Settings/etc. ...*
*rsync: write failed on “/mnt/backupWindows/pagefile.sys”: No space left on device (28)*
*rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.7]*
*rsync: connection unexpectedly closed (28 bytes received so far) [sender]*
*rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7]*
I'm confused by this, as I'm not a CLI wizard by any means. So if you offer advice, please try to dumb it down to the newbie level. You'll get an understanding of my comprehension level by the following comments... Booting into each of the two WinXP partitions proves that the clone/mirror partition is "out of date" compared to the master partition. I used ls to find out that the pagefile.sys files on both WinXP partitions are exactly the same size: 2,145,386,496 bytes. I used df to find out that (at this point in time) the source WinXP partition has 468,400 1KB blocks available, while the target WinXP partition has 2,015,600 1KB blocks available. df also informed me that the source WinXP partition is only 10,472,136 1KB blocks in total size, while the target partition is =larger=, at 10,985,712 1KB blocks. My guess is that the target can only run out of space is if rsync (1) copies an updated file into the free space of the target partition using a temp filename, (2) deletes the original file at the target partition, and (3) renames the newly-copied file to the original name. Any other explanation for that? Any suggested work-arounds? I'm thinking about cutting the Win swap file size in half, and seeing if that removes the problem at rsync time. Then limping along with that setup until I rebuild my system (when Ubuntu 12.4.1 comes out in a few weeks), with a larger partition for my "bare bones" WinXP. Thanks in advance for any help. Cheers, Carl Turney Bayswater Mobile 0427 024 735
My apologies for the asterixes that the email system seems to have put in to the error messages I quoted on this thread. Thanks again, Carl On 10/08/12 13:23, Carl Turney wrote:
Hi All,
I'm using rsync (version 3.0.7-1ubuntu1.1 on my Ubuntu 10.4 system) in a shell script to perform local backups of my system on to removable hard disks.
(FYI: It's a bit kludgy, but I'm VERY happy with it. Been doing it for a few years. Normally works great. It's extremely fast at creating up-to-date immediately-bootable clones/mirrors (?) of my system. I always store backups in different locations from my office. So, I'm not hogging up bandwidth, or exposing my data to others, by using the internet to do my backups.)
My system is a dual boot, sharing Ubuntu with WinXP, for when I (rarely) HAVE to genuinely do something in Win. (Thanks, but no thanks. I don't want to do a virtual machine.)
Are there any rsync wizards out there? My problem is this:
Lately, when rsync updates my WinXP partition (by this command)... rsync -Hvxrlt --delete /mnt/tempWindows/ /mnt/backupWindows ... I get the following error messages ...
*sending incremental file list*
*pagefile.sys*
*rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)*
*deleting Documents and Settings/etc. ... *
*((Here, it seems to remove various deleted WinXP files from the target partition.)) *
*deleting Documents and Settings/etc. ...*
*rsync: write failed on “/mnt/backupWindows/pagefile.sys”: No space left on device (28)*
*rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.7]*
*rsync: connection unexpectedly closed (28 bytes received so far) [sender]*
*rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7]*
I'm confused by this, as I'm not a CLI wizard by any means. So if you offer advice, please try to dumb it down to the newbie level. You'll get an understanding of my comprehension level by the following comments...
Booting into each of the two WinXP partitions proves that the clone/mirror partition is "out of date" compared to the master partition.
I used ls to find out that the pagefile.sys files on both WinXP partitions are exactly the same size: 2,145,386,496 bytes.
I used df to find out that (at this point in time) the source WinXP partition has 468,400 1KB blocks available, while the target WinXP partition has 2,015,600 1KB blocks available.
df also informed me that the source WinXP partition is only 10,472,136 1KB blocks in total size, while the target partition is =larger=, at 10,985,712 1KB blocks.
My guess is that the target can only run out of space is if rsync (1) copies an updated file into the free space of the target partition using a temp filename, (2) deletes the original file at the target partition, and (3) renames the newly-copied file to the original name.
Any other explanation for that? Any suggested work-arounds?
I'm thinking about cutting the Win swap file size in half, and seeing if that removes the problem at rsync time. Then limping along with that setup until I rebuild my system (when Ubuntu 12.4.1 comes out in a few weeks), with a larger partition for my "bare bones" WinXP.
Thanks in advance for any help.
Cheers,
Carl Turney Bayswater Mobile 0427 024 735
_______________________________________________ luv-main mailing list luv-main@luv.asn.au http://lists.luv.asn.au/listinfo/luv-main
On 2012-08-10 13:23, Carl Turney wrote:
Hi All,
[...]
Lately, when rsync updates my WinXP partition (by this command)... rsync -Hvxrlt --delete /mnt/tempWindows/ /mnt/backupWindows ... I get the following error messages ...
[...]
*rsync: write failed on “/mnt/backupWindows/pagefile.sys”: No space left on device (28)*
*rsync error: error in file IO (code 11) at receiver.c(302) [receiver=3.0.7]*
*rsync: connection unexpectedly closed (28 bytes received so far) [sender]*
*rsync error: error in rsync protocol data stream (code 12) at io.c(601) [sender=3.0.7]*
[...]
My guess is that the target can only run out of space is if rsync (1) copies an updated file into the free space of the target partition using a temp filename, (2) deletes the original file at the target partition, and (3) renames the newly-copied file to the original name.
Any other explanation for that? Any suggested work-arounds?
I'm thinking about cutting the Win swap file size in half, and seeing if that removes the problem at rsync time. Then limping along with that setup until I rebuild my system (when Ubuntu 12.4.1 comes out in a few weeks), with a larger partition for my "bare bones" WinXP.
Hi Carl, Given this is a swap file, and therefore its contents should only be relevent until a system is rebooted, why bother syncing it at all: rsync --exclude=pagefile.sys Secondly, if you are concerned that you don't have enough space for 2 copies of the file (so that the new copy can be moved into place only after fully transfered), you could try the --inplace option to rsync, which explicitly avoids this behaviour. Finally, I'm surprised that you don't have more issues syncing a Windows partition using the Linux NTFS drivers; I'd assume that this would ignore a while lot of permissions and ACLs which I'd expect to cause problems, or at least insecurities. Interesting to see that it boots at all, but I'd definitely be concerned about permissions issues, though this may not be a problem for your specific use-case. -- Regards, Matthew Cengia
Hi,
Lately, when rsync updates my WinXP partition (by this command)... rsync -Hvxrlt --delete /mnt/tempWindows/ /mnt/backupWindows ... I get the following error messages ...
Given this is a swap file, and therefore its contents should only be relevent until a system is rebooted, why bother syncing it at all:
rsync --exclude=pagefile.sys
Thanks. Never noticed that parameter/argument. Added it into the backup command (above), and it solved the problem. Thanks, Matthew Cengia.
Finally, I'm surprised that you don't have more issues syncing a Windows partition using the Linux NTFS drivers; I'd assume that this would ignore a while lot of permissions and ACLs which I'd expect to cause problems, or at least insecurities.
I was concerned about those sorts of things, but didn't fully understand them. But, the backups boot up well, and the programs all seem to function OK. Part of the "solution" may be because my WinXP partitions aren't NTFS, but (time to gasp) FAT32. I'm not networked with anyone else when running WinXP, except for web surfing and downloading/updating applications and utilities. Don't store any valuable data on that partition. Only potential problem is max file size of 4GB, but I do any DVD-related stuff on Linux now.
Interesting to see that it boots at all, but I'd definitely be concerned about permissions issues, though this may not be a problem for your specific use-case.
Could be the FAT32 thing making it possible to boot and run. As far as the permissions: I have up-to-date security utils for it (AVG antivirus, ZoneAlarm firewall, Advanced System Care); nothing worth stealing; and nothing (except programs) that would be missed if it were erased -- on the WinXP partition. Cheers, Carl
Hi, I've bypassed rsync-ing the WinXP swap file, and the backups don't crash any more. But I just now noticed that my clone of the WinXP partition isn't a perfect replica. Using this command... rsync -Hvxrlt --exclude=pagefile.sys --delete /mnt/tempWindows/ /mnt/backupWindows ... I end up with these two results: In the source WinXP partition... 392 hidden files 3847 folders 35269 files In the target WinXP partition... 214 hidden files 3996 folders 35307 files I've run that rsync command a couple times, without even booting into Windows between rsync's. So, it's coming up with nothing to send to the target, and nothing to delete from the target. Maybe I'm not correctly using parameters/arguments that effectively deal with creating/deleting empty folders, 0-byte files, hidden files, or similar oddities? Thought that -Hvxrlt and --delete covered it all, but apparently not. Thanks in advance, if anyone sees what I'm doing wrong here. Carl
Carl Turney <carl@boms.com.au> wrote:
I've bypassed rsync-ing the WinXP swap file, and the backups don't crash any more.
But I just now noticed that my clone of the WinXP partition isn't a perfect replica.
Using this command... rsync -Hvxrlt --exclude=pagefile.sys --delete /mnt/tempWindows/ /mnt/backupWindows ... I end up with these two results:
Does adding the -a (archive) option help?
On 10/08/12 13:23, Carl Turney wrote:
Lately, when rsync updates my WinXP partition (by this command)... rsync -Hvxrlt --delete /mnt/tempWindows/ /mnt/backupWindows ... I get the following error messages ... [...]
*rsync: write failed on “/mnt/backupWindows/pagefile.sys”: No space left on device (28)* [...] I used df to find out that (at this point in time) the source WinXP partition has 468,400 1KB blocks available, while the target WinXP partition has 2,015,600 1KB blocks available.
df also informed me that the source WinXP partition is only 10,472,136 1KB blocks in total size, while the target partition is =larger=, at 10,985,712 1KB blocks.
My guess is that you were right to think that the block size was involved, but wrong in your interpretation of what df tells you. df is for finding out how much disk space you have free, and for that purpose variable block sizes are just a source of confusion, so df pretends that that all your file systems have the same block size, and on recent linux systems that tends to mean it defaults to displaying figures with a fictional block size of 1K. You can modify the fictional block size, but df won't show you the real one. Try something like this on your linux filesystem (assuming ext/ext3/ext4): sudo /sbin/dumpe2fs /dev/fubar | head -n 100 | grep 'Block size' (putting head in there just makes it faster. dumpe2fs gives you a lot of info you don't need to grep through). I'm not sure what you'd do to find the block size for and NTFS partition from within linux. Maybe someone else here knows that? Andrew
participants (4)
-
Andrew McNaughton -
Carl Turney -
Jason White -
Matthew Cengia