gmane.network.rsync.general

Re: getting problems with lsyncd.

garvit sharma — 2013-05-23T05:28:06

Please help i have tried many times but i did not found the reason behind
not syncing up.


On Wed, May 22, 2013 at 9:45 PM, garvit sharma <garvits45< at >gmail.com> wrote:

Re: getting problems with lsyncd.

garvit sharma — 2013-05-23T04:45:06

when i am running nothing happens!! the directories wont syncs up.


On Wed, May 22, 2013 at 1:43 PM, Kevin Korb <kmk< at >sanitarium.net> wrote:

[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes[sender]: Broken pipe (32)

2013-05-23T03:50:21

https://bugzilla.samba.org/show_bug.cgi?id=5478

--- Comment #32 from Scott Wood <woodystrash< at >hotmail.com> 2013-05-23 03:50:20 UTC ---
I tried the --no-ckecksum or --blocking-io, both to no avail.  I now have an
admin at the other end of the connection onboard for troubleshooting.  If/when
we find the cause, I'll update the list.

[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes[sender]: Broken pipe (32)

2013-05-23T02:21:16

https://bugzilla.samba.org/show_bug.cgi?id=5478

--- Comment #31 from Scott Wood <woodystrash< at >hotmail.com> 2013-05-23 02:21:10 UTC ---
(In reply to comment #30)

Hey Loïc,

I haven't tried the --no-ckecksum or --blocking-io yet.  I'll give them a
whirl.  I have tried with the --no-compress, though and that did not resolve
the matter.  I'll let you know how it works with combinations of --no-ckecksum
and --blocking-io.  Thanks!

Regards,
Scott

Re: rsync behavior on copy-on-write filesystems

Kevin Korb — 2013-05-22T21:35:17

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

You might find the lsh program in the support dir of the source tree
useful.  It is essentially the same as using ssh localhost but without
the overhead.

On 05/22/13 17:25, Allen Supynuk wrote:

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin KorbPhone:    (407) 252-6853
Systems AdministratorInternet:
FutureQuest, Inc.Kevin< at >FutureQuest.net  (work)
Orlando, Floridakmk< at >sanitarium.net (personal)
Web page:http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlGdOhUACgkQVKC1jlbQAQdcCQCdH8AzLEkR67kaynV8ZCLuJtBU
qloAoMClNog68b+WlKparl07cp0SDvwT
=jUvs
-----END PGP SIGNATURE-----

Re: rsync behavior on copy-on-write filesystems

Allen Supynuk — 2013-05-22T21:25:01

Sorry for the churn and thanks for the suggestions. When I redid my
experiments over the network everything worked just as I dreamed it
would. Changing the first 4K bytes only caused a 4K change in the
copy. Changing meta-data (time stamp) only caused the time stamp to
change in the copy.

This is very nice and I expect it to save 10's of GB per archive going forward.

Kevin was right. --whole-file was the default when source and
destination are specified as local paths.

On 5/22/13, Allen Supynuk <allen.supynuk< at >gmail.com> wrote:

Re: getting problems with lsyncd.

Kevin Korb — 2013-05-22T20:43:01

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Some more details would help.  What happens when you run that?  Is
there a verbose or a debug option?

And for those of us not so familiar with lsyncd what actual rsync
command does that run?

On 05/22/13 16:00, garvit sharma wrote:

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin KorbPhone:    (407) 252-6853
Systems AdministratorInternet:
FutureQuest, Inc.Kevin< at >FutureQuest.net  (work)
Orlando, Floridakmk< at >sanitarium.net (personal)
Web page:http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlGdLdUACgkQVKC1jlbQAQe7dACgq3YaGzUBrr2AL8bq89dWGUJS
uewAoOpGTm2z+MxV0/cNtsOooNzbdk9S
=oqi9
-----END PGP SIGNATURE-----

getting problems with lsyncd.

garvit sharma — 2013-05-22T20:00:08

Hello All,

When i run lsyncd using *lsyncd -rsync /home/abc/source
/home/abc/dest*then i am able to sync the two directories of the local
system. But when i
run using
*lsyncd -rsync /home/abc/source 10.5.1.12:/home/abc/dest* where 10.5.1.12
is the ip address of the local machine then i am unable to sync the both
directories on the local machine. Please leave your suggestions.


Thank you,

Re: rsync behavior on copy-on-write filesystems

Allen Supynuk — 2013-05-22T18:43:36

Kevin,

I will try again over a remote connection to see if that makes a
difference. Not expecting -z to day much of anything based on the
random data, just wanting to be consistent with the flags in the
finished solution.

Chris,

You only get --whole-file if you specify it (or -W).

Paul,

For my first couple of days of testing I dutifully did both 'df -h .'
and 'btrfs filesystem df .' until I saw that they give the same answer
after you wait for background cleanup. In this case we are only adding
files so no background cleanup applies.

On 5/22/13, Paul Slootman <paul+rsync< at >wurtel.net> wrote:

[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes[sender]: Broken pipe (32)

2013-05-22T07:02:10

https://bugzilla.samba.org/show_bug.cgi?id=5478

--- Comment #30 from Loïc Gomez <samba-bugz< at >kyoshiro.org> 2013-05-22 07:02:09 UTC ---
Did you try with one of these options : --no-checksum --no-compress
--blocking-io ?

Just a check since you have a 4G limit : I suppose you're not rsync-ing to a
FAT32 filesystem ?

Re: rsync behavior on copy-on-write filesystems

Paul Slootman — 2013-05-22T06:41:58

Note that you need to be using "btrfs filesystem df ."
for reliable numbers; the normal df does not take into account
background cleanups etc.


Paul

[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes[sender]: Broken pipe (32)

2013-05-22T06:05:44

https://bugzilla.samba.org/show_bug.cgi?id=5478

--- Comment #29 from Scott Wood <woodystrash< at >hotmail.com> 2013-05-22 06:05:43 UTC ---
As a follow up to my second question, we added the firewall rules to allow
SYN,RST,ACK,FIN and ACK traffic form the server in question and it did not
solve the problem.

Re: rsync behavior on copy-on-write filesystems

Chris Dunlop — 2013-05-22T03:46:44

<snip>


You're testing --whole-file: that's the default with source and
destination as local paths. You want --no-whole-file in there.

Cheers,

Chris

[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes[sender]: Broken pipe (32)

2013-05-22T03:43:45

https://bugzilla.samba.org/show_bug.cgi?id=5478

Scott Wood <woodystrash< at >hotmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |woodystrash< at >hotmail.com

--- Comment #28 from Scott Wood <woodystrash< at >hotmail.com> 2013-05-22 03:43:41 UTC ---
I believe I am a recent victim of this bug.  I am doing an rsync via ssh of
large files (from 4GB, up to a couple hundred GB).  I have the -W and have
disabled compression on the file type that is failing most often.

Here is a rough example of the rsync push command:

rsync -avW --timeout=0 --rsync-path='/bin/rsync_3.0.9' --skip-compress=ext -e
"ssh -c arcfour -i /home/user/.ssh/CUSTOM_id_rsa" /path/to/bigfile.ext
remoteuser< at >theserver:/path/to/destination

And the connection is terminated with an ACK,RST packet from the server
resulting in the following on the client:

Read from remote host theserver: Connection reset by peer
rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken
pipe (32)
rsync: connection unexpectedly closed (28 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(605) [sender=3.0.9]

I have reproduced the result with a multitude of versions, including 3.0.4,
3.0.7 and 3.0.9 on the client side, and 3.04 and 3.09 on the server side.  I
have the following questions:

1) Chip Scwheiss, re: comment 23.  Do you have a RHEL bug reference so I can
see if this is resolved, or pursue it further with them under our support
contract?
2) What is the consensus on the security implications and effectiveness of
Chip's iptables suggestion?
3) If it boils down to needing to --bwlimit, does anyone have any suggestions
other than trial and error to calculate the maximum it should be able to
handle? As we have many TB to move, this would not be the ideal solution.

rsync behavior on copy-on-write filesystems

Allen Supynuk — 2013-05-21T22:06:42

I have been doing some experiments with rsync on btrfs, a
copy-on-write file system that is approaching or having just achieved
production-ready status depending on your requirements.

For my purposes the reliability appears by almost all accounts to be
there, and the compression alone makes it very compelling.

However the following two experiments show rsync behaviors that are
disappointing to the point of appearing to be bugs. Certainly rsync is
more powerful if they are fixed. Of course, this is assuming that I
have not missed something in my tests.

--

Bottom line on top: rsync with --inplace appears to (wastefully)
rewrite the entire file even when only a single block or just the
meta-data (timestamp) has changed. While this is necessary behavior on
some file systems it is wasteful on copy-on-write systems.

I propose that rsync be changed to only write blocks that have changed
when --inplace is in effect. And that only meta-data be changed if the
underlying filesystem supports it.

In the experiments below the final results would have been 20BGB - 4KB
smaller had these changes been in place.

(Running rsync 3.0.9)

################################################
## Test rsync --inline

## 1) Start with an empty filesystem

$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/jobarchive-Ajobarchivetest2
                      300G   36M  293G   1% /vol/jobarchive_Ajobarchivetest2

## 2) Create a subvolume. Put one file with 10gb of random data in it.
##    Note: Compression is turned on, but our random data defeats it.

$ btrfs subvolume create src
$ time dd if=/dev/urandom of=src/10gb bs=4k count=2621440 conv=notrunc
2621440+0 records in
2621440+0 records out
10737418240 bytes (11 GB) copied, 811.427 s, 13.2 MB/s
0.400u 806.115s 13:31.42 99.3%  0+0k 0+20971520io 0pf+0w

$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/jobarchive-Ajobarchivetest2
                      300G   11G  283G   4% /vol/jobarchive_Ajobarchivetest2

## 3) Create a second subvolume called current. Copy the first file into it.

$ btrfs subvolume create current
$ time cp --archive src/* current
0.057u 17.389s 0:42.29 41.2%    0+0k 19737984+20971520io 0pf+0w

$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/jobarchive-Ajobarchivetest2
                      300G   20G  274G   7% /vol/jobarchive_Ajobarchivetest2

## 4) Make a snapshot of the second volume called job1. Note that it
takes up almost no space.

$ btrfs subvolume snapshot current job1
$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/jobarchive-Ajobarchivetest2
                      300G   21G  273G   7% /vol/jobarchive_Ajobarchivetest2

## 5) Change the first 4k bytes of the original file

$ time dd if=/dev/urandom of=src/10gb  bs=4k count=1 conv=notrunc
1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 0.000601676 s, 6.8 MB/s
0.001u 0.001s 0:00.03 0.0%      0+0k 32+8io 1pf+0w

$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/jobarchive-Ajobarchivetest2
                      300G   21G  273G   7% /vol/jobarchive_Ajobarchivetest2

## 6) Use rsync --inplace to make a copy of the first file.
##    Note:
##    - We use --inplace to copy over the existing file
##    - We do not use -W aka --whole-file so the delta-xfer algorithm
should be in play
##    - The hope is that rsync will only rewrite the first block of the file

$ time /usr/share/sbtools-sbjobarchive/external-apps/rsync/rsync-3.0.9/install/centos5-64/bin/rsync
--stats -az --timeout=600 --inplace src/ current/

Number of files: 2
Number of files transferred: 1
Total file size: 10737418240 bytes
Total transferred file size: 10737418240 bytes
Literal data: 10737418240 bytes
Matched data: 0 bytes
File list size: 52
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 10742659175
Total bytes received: 34

sent 10742659175 bytes  received 34 bytes  11012464.59 bytes/sec
total size is 10737418240  speedup is 1.00
851.783u 79.265s 16:14.78 95.5% 0+0k 19752416+20971520io 17pf+0w

## 7) Alas the new file takes up a full extra 10 GB.

$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/jobarchive-Ajobarchivetest2
                      300G   31G  263G  11% /vol/jobarchive_Ajobarchivetest2

## Conclusion: rsync rewrote the entire file into current/10gb even
though it only needed to write the
## first 4k block. If it had we would have saved 10 GB - 4KB of disk.

################################################

## Test metadata-only change

## Start with files as above

$ btrfs subvolume snapshot current job2
Create a snapshot of 'current' in './job2'
$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/jobarchive-Ajobarchivetest2
                      300G   31G  263G  11% /vol/jobarchive_Ajobarchivetest2

## Change the meta-data of the first file then rsync with --inplace

$ touch src/10gb
$ time /usr/share/sbtools-sbjobarchive/external-apps/rsync/rsync-3.0.9/install/centos5-64/bin/rsync
--stats -az --timeout=600 --inplace src/ current/

Number of files: 2
Number of files transferred: 1
Total file size: 10737418240 bytes
Total transferred file size: 10737418240 bytes
Literal data: 10737418240 bytes
Matched data: 0 bytes
File list size: 52
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 10742659172
Total bytes received: 31

sent 10742659172 bytes  received 31 bytes  10620523.19 bytes/sec
total size is 10737418240  speedup is 1.00
920.122u 82.526s 16:50.71 99.2% 0+0k 20469728+20971520io 0pf+0w

$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/jobarchive-Ajobarchivetest2
                      300G   41G  253G  14% /vol/jobarchive_Ajobarchivetest2

## conclusion: Entire file was copied on the destination system even though only
## meta-data had changed. This is not necessary with a copy-on-write system.

rsync is too often CPU-bound

Torsten Bronger — 2013-05-21T08:20:20

Hallöchen!

While rsync works perfectly if you have little bandwidth, it is
difficult to use in a fast trustworthy local network.  Firstly, you
have to switch off compression, encryption, and checksum
calculation.  This is easily possible, although it can be hard for
people to collect all options necessary for fast-network tuning.

However, the CPU performance I observed was devastating.
Transfering big files, the NFS server uses <10% CPU, while rsync
daemon uses >80%.  Why is this?

Really critical is this with an NAS with a slow CPU.  In my case, it
limits the transfer rate from 55 MByte/s (NFS) to 12 MByte/s
(rsync).

For me, it means that I use NFS for data transmission, and use rsync
only locally on the client computer with a much faster CPU.  But
this should not be necessary in my opinion.  I think that at least
for big files, CPU and bandwidth performance of rsync and NFS should
be the same.

Tschö,
Torsten.

[Bug 9894] Rsync can silently zero out chunks in a file

2013-05-20T12:41:31

https://bugzilla.samba.org/show_bug.cgi?id=9894

--- Comment #2 from Ankur Mishra <an.m< at >outlook.com> 2013-05-20 12:41:31 UTC ---
(In reply to comment #1)

Thanks for your response Wayne. Apologies for long comment below, but it would
be really helpful if you could go through it.

My setup is rsyncd on one machine (source of a file), and rsync client on
another machine (destination). 

When this happens (checksum mismatch due to read failure) does receiver
(destination) throw an error? What I see is that receiver is not retuning an
error code. From receiver point of view this looks like a success.

I have tested this with both with and without --checksum with fresh sync (i.e.,
receiver didn't have the file)

This is the experiment I did:

1. Ran rsyncd on one machine
2. Modified the rsyncd map_ptr() to mimic read failures
3. Ran rsync from a different machine
4. I could see rsync returning without any error, and a partially zeroed out
file (of same length)
5. I observed same symptoms as #4 on a flash based embedded system during i/o
stress on server side.

Is there any specific option I should be using while running rsync to catch
checksum errors?

Thanks,
Ankur

Re: contribute to rsync

Karl O. Pinc — 2013-05-20T12:13:15

You might also consider invoking rsync from incron.


Karl <kop< at >meme.com>
Free Software:  "You don't pay back, you pay forward."
                 -- Robert A. Heinlein

Timeout on deletion

AZ 9901 — 2013-05-20T08:55:03

Hello,

I use Rsync client (3.0.9) synchronising to a Rsync daemon (3.0.8) over SSH.

I have sometimes a huge amount of files to delete.
Rsync daemon can take a lot of time to delete them, leading Rsync client into a timeout…

Is there any way to avoid this timeout ?
I would have liked not to increase timeout value on client side (which is already set to 600 seconds), as I don't really know how long the deletion will take.

Could it be possible for the client and the daemon to communicate with each other from time to time during the deletion process ?
The client would then know that deletion is still in progress, this would avoid timeout !

Thank you very much,

Best regards,

Ben

Re: IRC

Kevin Korb — 2013-05-20T05:04:57

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

#rsync on irc.freenode.net

On 05/20/13 00:53, garvit sharma wrote:

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
Kevin KorbPhone:    (407) 252-6853
Systems AdministratorInternet:
FutureQuest, Inc.Kevin< at >FutureQuest.net  (work)
Orlando, Floridakmk< at >sanitarium.net (personal)
Web page:http://www.sanitarium.net/
PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlGZrvkACgkQVKC1jlbQAQfiBQCgnzTqIjGE7+AcD2qH9zwlDoiz
YBYAnRoFjd6vSy6shUg/VAGvreCxvwbe
=NvxC
-----END PGP SIGNATURE-----

IRC

garvit sharma — 2013-05-20T04:53:47

Hello All,

                  Is there any irc group for rsync ?. If it does exist
please let me know.

Thanks,