gmane.comp.sysutils.backup.fsvs.devel

[ANNOUNCE] FSVS 1.1.17 released

Philipp Marek — 2008-10-29T12:39:31

Hello everybody, it's finally time to announce a new version. There have been a lot of internal changes (again); and that did take most of the time, because I had to shatter many hours of thinking into many small half-hour pieces, and that means a lot of wasted time. I hope to put the next version out sooner; it should have more user-visible (and fewer internal) changes. And that's already the keyword - the next version will feature some incompatibilities, and will duly be noted 1.2.0, or even 2.0. Time (and the next release announcement) will tell. Well, back to more mundane matters - what did change since 1.1.16? There are some additional features: - New "uncopy" command, to disambiguate "revert" on copied and changed entries. Manually added or "prop-set" entries are kept known. - New option "all_removed", to trim the output for deleted hierarchies. - New option "config_dir", important for https connections with client certificate authentication. - New command "delay", for use in scripts. - New command "rel-ignore"; this converts the given ($PWD-local) shell patterns to working copy root relative. - New "fsvs cat" command, to fetch really pristine copies from the repository. - A new flag for ignore patterns, for matching directories only. - And a way for ignore patterns to match the entries' mode; so eg. world-unreadable files can easily be ignored. The most important fixes are: - Bugfix for filtered commit (eg. "-f text"). Previously that stored the current meta-data of entries that didn't match the filter in the entry list, too; so they wouldn't be seen as changed afterwards. - Performance fix for "fsvs diff -rX:Y entry" - don't diff the whole working copy, only the given entries. - "diff" showed for replaced entries "only in rX" - fixed. - Bugfix for "dir_sort" option; the root directory wasn't printed. - Bugfixes for memory and file leaks in /tmp. Regarding documentation and directly user-visible changes there are - Some documentation fixes, and repairs for the man pages. There's now a bit more included in the source distribution, too. - Started a "tips & tricks" document. - User-readable error on non-writeable $FSVS_CONF. - The option "dir_sort" now uses strcoll(), ie. sorts according to the current locale. - "fsvs info" for the working copy root now prints the revision of the highest priority URL, and not "0". Not directly related to FSVS, but maybe of interest for users: there's a new web site "http://doc.fsvs-software.org", which has the (generated) documentation available as normal files on a web server - and doesn't use the (slow) "checkout" hierarchy of fsvs.tigris.org. Last, but not least, I did some small enhancements that should make life easier for some corner-cases: - Export some environment variables for use in diff, commit- and update-pipes. - Splitted "-C" into finer grained option settings "-o change_check". Changed the default to use MD5 on possibly-changed files. - FSVS now shows "maybe changed" for unreadable files or directories; should we throw a warning? - Some directory mtime handling fixes. - Another try to detect invalid "colordiff" program names, and a better message than a simple "EPIPE". I'd like to say "thank you" for the users that reported problems; in (arbitrarily alphabetic order) Maurice, Plamen, and Thomas. *** Thank you! *** You can get it on the usual place: http://freshmeat.net/projects/fsvs/; some distributions will have packages for you. every kind of feedback is highly appreciated, be it bug reports, questions, ideas or even patches; please send it to the users< at >- or the dev< at >- mailing list, as appropriate. If you create packages for $DISTRIBUTION (and want to share them), please tell me (and possibly the users-list), so that I can put the links on the download page. [ BTW, if you provide some easy way to use FSVS for eg. /etc, you might want to take a look at the example tree in the source distribution. ] To avoid angry (or curious) questions, I'll add what I'd like to see in the next version - and why that'll make FSVS incompatible with 1.1.x (and 1.0.x, of course). - The WAA and CONF layout will change a bit: - The URL list will move to the WAA. - The filename hashing may currently produce collisions, if you're using nested working copies. - I'm planning to enhance the (current) ignore patterns to "groupings", which would also serve to provide svn's "auto-props" feature. Then world-unreadable entries can be ignored, or can be sent encrypted to the repository. That'll (possibly) mean a bit of change in the way ignore patterns are stored in $FSVS_CONF; maybe I can do that compatible. Then there are lots of other features I'd like to implement - but I think that these would be enough for a new version. Regards, Phil

Re: [feature request] auto unversion of files matching new ignore pattern

Philipp Marek — 2008-10-01T16:06:41

Hello Gunnar! On Tuesday 30 September 2008 Gunnar Thielebein wrote: Yes, it would have to be some config option, because that would mean trying *all* patterns on *all* known entries ... and that can take some time, and possibly include entries that you want to keep. Yes, that's right ... Maybe the inverse - doing "unversion" with some flag puts a take-pattern in front of the list? But the shell expands wildcards, whereas FSVS doesn't ... so you'd get a whole lot of take-patterns. Well, since some time I've got a point on my TODO ... "fsvs shelf" and "unshelf" or something like that. That should mark all local changes as not-changed, so that only new changes appear (and get committed) - and on "unshelf" they get visible again. But that should work in multiple layers, and possibly many changes in a single file should be splitted into the various shelves, etc. .... Kind of what quilt does: http://savannah.nongnu.org/projects/quilt But I have no real idea how that can be sanely implemented. Currently I'd like to keep that behaviour as-is - trying a hundred patterns against some hundred thousand paths isn't nice. But if you really insist, we can discuss further which solution you could implement ;-) Regards, Phil

[feature request] auto unversion of files matching new ignore pattern

Gunnar Thielebein — 2008-09-30T20:32:30

It would be nice if fsvs would have an option that if patterns are edited files that are matched also automagically become unversioned. If I have added some pattern like It should automaticly appear as unversioned: IMO this was already requested some time ago. I think adding a config for that would also not break too much compatibility of older versions. It is that you cant have the ultimate ignore list matching all kind of distribution. Easing the use of the unversion would help in customising hosts. At the moment I need to edit the ignore list, then unversion the files and if some other admin has done work also remember the pathes i removed. What do you think about that? Regards, Gunnar

Re: Pointer arithmetic in src/direnum.c:561

Philipp Marek — 2008-09-18T15:54:58

Hello Jonty! On Thursday 18 September 2008 Jon wrote: ... Yes, you're right. It's a bit cleaner that way. Fixed in r1882. Thank you! Regards, Phil

Pointer arithmetic in src/direnum.c:561

Jon — 2008-09-18T13:36:41

Hi, I think line src/direnum.c:561 should change from: sts->name=names[i] + this->strings; to: sts->name=this->strings + names[i]; I have spent the last 3 evenings trying to get fsvs running on my Nokia N800 and kept getting segmentation faults every time I ran the command: fsvs commit -m 'yada-yada-yada' /etc Using DEBUGP and binary chopping through the code I tracked the segfault down to src/direnum.c:587 which uses sts->name. The last change to sts->name was made at 561 which does the fancy pointer arithmetic to convert an integer offset from the names[] array to a char*. I use a cross-compiler to build programs for the Nokia N800. I suspect that the compiler will only generate the code I want when the source expression looks like char*+int. I think it generates bad code when the source expression is int+char*. Thanks Jonty

Re: [PATCH] - Client certificate authentication

Philipp Marek — 2008-09-02T08:37:06

No problem. Thank you, I'll take a look. Regards, Phil

Re: [PATCH] - Client certificate authentication

Gunnar Thielebein — 2008-09-01T16:27:22

Hi Phil, sorry that it took time until i could made a test (finished a move and have no internet connection in my flat yet). I did a test with 1873 (1865 failed for some reason). When changing the config_dir option to e.g. /root/.subversion fsvs does not recognise this and still keeps the default path /etc/fsvs/auth. My configfile looks this: If you need further information please let me know. Gunnar Philipp Marek wrote:

[PATCH] - Client certificate authentication

Philipp Marek — 2008-08-23T14:43:28

Hello Gunnar, could you please test that this works for you? It's already committed in r1865, so if you're on HEAD you won't need that. Regards, Phil

Re: saving auth credentials

Philipp Marek — 2008-08-21T15:53:40

Hello Gunnar! On Thursday 21 August 2008 Gunnar Thielebein wrote: When you look at the subversion sources, you see in subversion/svn/main.c:1936 the call svn_cmdline_set_up_auth_baton() use opt_state.config_dir. This gets set by svn_config_ensure() some 400 lines above - which has a scary comment there. In the FSVS sources there's racallback.c - which has the call to svn_cmdline_setup_auth_baton() in it. This call gets a config directory (currently only NULL), and a "svn_config_t *cfg" - which is something different from the "apr_hash_t *cfg" that svn_ra_open() wants (in url.c). The config hash already gets fetched by svn_config_get_config(), like svn does - but again with a config directory as parameter. Now I'm not sure whether that should be the same path (/etc/fsvs/auth), or has to be different, or what exactly is expected here. So the job is to go through the subversion sources to see how the config theme is handled there - unless you find some good documentation. Regards, Phil

saving auth credentials

Gunnar Thielebein — 2008-08-21T09:32:08

Hi Phi, For our convenience I moved this topic to dev. I digged a little in svn hacking and when looking the functions svn_config_get_config and preparing auth_baton it seems all correct to me when comparing with e.g. svn client. From my gross understanding is that dir ~/.subversion is already taken for credentials. There is only a problem in creating the initial directory structure if its missing and saving the auth file with the credentials. This should be done automagically via svn_cmd_setup_auth_baton. When you say try a meaningful cfg value, what do you mean with that? I can't find another way getting cfg_hash. Could you shed some light on it? Regards, Gunnar

[ANNOUNCE] FSVS 1.1.16 released, and [Alert]

Philipp Marek — 2008-06-18T15:22:08

Hello everybody, here's a fresh release. Important: if you're versioning your /etc/ *and* you're using the DynDNS registration client, please change your password; the filtering script used for ddclient.conf was wrong, and so your password might be stored in your repository. It's been about 3 months since the last feature release; it took a bit longer because I changed some internal conventions and shoved code around. The major changes since 1.1.15 are: - FSVS_WARNINGS removed. Use FSVS_WARNING. - Handling of FSVS_WAA and FSVS_CONF now via the normal option handling, to reduce code size. Now it's possible to use "-oconf=..." on the command line, too. (But it's not possible to override the paths from the config file.) - Bugfix for error after commit, when $EDITOR returned an 0 byte file as commit message. - "fsvs diff" changed to recursive behavior, as "svn" does. - Fixed "fsvs diff -rX" to print only changed entries, not the whole list. - "fsvs diff -rX:Y" reimplemented, too; performance could get optimized. - Rewrote entry fetching from the repository. Previously a file with bad mode (like 0111) couldn't get diffed. - "update -rX" and "diff -rX" (but not "diff -rX:Y") now use the per-url override syntax (see "-u"). - New option to set maximum number of revisions on "fsvs log". - Fixed diff for symlinks and devices. - New option "stop_change" for use in scripts. - New option "author" for commit (but that doesn't work for svn+ssh://) - Changed from "merge" to "diff3" as merge program; seems to be more common. - Committed the distclean patch from Sheldon Hearn. - "make install" implemented. For the full list please see http://fsvs.tigris.org/source/browse/fsvs/tags/fsvs-1.1.16/fsvs/CHANGES?rev=1774&view=markup There are some known issues, too: - Multi-URL code has been found to be not perfect - removing a (common) directory removes files from all URLs. - "fsvs sync" now tries to fetch all file sizes and entry types from the repository, too. For that we have to do a recursive listing, and fetch special entries; encoded files with only a few kB are looked at, too. Needs more time and bandwidth. - "fsvs revert -rX" still doesn't work if types of entries would get changed (dir -> file, symlink -> device, etc.) - that's next on the list. For this release "diff" was done, and I hope it works better. So there's still some work left; stay tuned :-) Please fetch the sources from http://freshmeat.net/projects/fsvs/, or ask your distributions for packages - at least for debian and fedora you should be fine. If there are any questions, problems or suchlike, don't hesitate to ask on the mailing lists. Regards, Phil

Re: fsvs in Fedora

Philipp Marek — 2008-06-04T18:49:53

Hello David! On Wednesday 04 June 2008 David Fraser wrote: You're welcome - and welcome to FSVS :-) Thank you; I'll mention that on the webpages. That's because 1.1.15 had just a single script changed (vs. 1.1.14) - it was a security update, and all three mailing lists got an alert. Well, welcome on board, I sure hope you enjoy the journey! Regards, Phil

fsvs in Fedora

David Fraser — 2008-06-04T14:05:53

Hi everyone Just wanted to say thanks for writing fsvs - been waiting for something like this for years. I've got fsvs included in Fedora - 1.1.15 is now in the repositories for Fedora 7 and upwards, and I'm the package maintainer. I presume the announcements mailing list is the place to lurk to look for new releases - but there didn't seem to be a notice of the release of 1.1.15... Cheers David

Re: bugreport: fsvs diff

Philipp Marek — 2008-05-25T08:10:40

Hello Shuvaev! On Friday 23 May 2008 Nast wrote: ... ... Yes, you're right. "fsvs diff -rX:Y" is broken. I already completely rewrote it - to solve other problems - but this one is new. Thank you for reporting. I added this case to my test list -- let's hope for the best in the next release :-) Regards, Phil

Re: bugreport: fsvs update looses attributes

Philipp Marek — 2008-05-25T08:02:14

Hello Shuvaev! Thank you for your email. On Friday 23 May 2008 Nast wrote: ... I think I already got it fixed; are you in a hurry, or can you wait for 2 - 3 weeks for the next release? Regards, Phil

bugreport: fsvs diff

Nast — 2008-05-23T05:45:26

Hello, dev< at >fsvs.tigris.org I'd like to inform you about bug in fsvs (FSVS (licensed under the GPLv3), (C) by Ph. Marek; version fsvs-1.1.14:1496) under Debian Linux 4.0 (etch) This bug appears when I try to get differense between 2 revisions: First I create new repository #svnadmin create /mnt/1/repository2 I want to commit directory /tmp/1 into repository #cd /tmp/1 #ls -all --- -rw-r--r-- 1 root root 16 May 23 13:17 somefile lrwxrwxrwx 1 root root 10 May 23 13:18 symlink_to_somefile -> ./somefile --- As you can see this dir contains 2 files, file 'somefile' and symlink to this file #fsvs urls file:///mnt/1/repository2 Try to commit: #fsvs commit -m 'first' /tmp/1 --- Committing to file:///mnt/1/repository2 N... 16 /tmp/1/somefile N... 10 /tmp/1/symlink_to_somefile committed revision 1 on 2008-05-23T05:24:32.185924Z as root --- committed success And now I try to commit this dir again (I did not make any changes in /tmp/1), so: #fsvs commit -m 'first' /tmp/1 --- Committing to file:///mnt/1/repository2 committed revision 2 on 2008-05-23T05:26:55.112246Z as root--- --- committed successefully again and now time to show you the bug: #fsvs diff -v -r1:2 . --- diff -u somefile.r1 somefile.r2 An error occurred: File exists (17) in up__handle_special: symlink(./somefile, ./symlink_to_somefile.XXXXXX) 01:28:19 root< at >sip:1# fsvs diff -v -r1:2 diff -u somefile.r1 somefile.r2 -Mode: 0644 +Mode: 0600 -MTime: Fri May 23 13:17:51 2008 +MTime: Fri May 23 13:28:33 2008 Owner: 0 (root) Group: 0 (root) An error occurred at 13:28:33.283: File exists (17) in up__handle_special: symlink(./somefile, ./symlink_to_somefile.XXXXXX) in rev__get_file in df__do_diff in df__action in ac__dispatch in waa__update_tree in waa__partial_update in waa__read_or_build_tree in df__work: No working copy base found? in main: action diff failed --- This error occurs every time I try to merge 2 revisions in repository which contain symlinks to any files. As far as I understand command diff -rN:N+M can't work with symlinks. When I try to make diff using something like #fsvs diff -v -r1 . --- diff -u ./somefile.r1 ./somefile.local -Mode: 0600 +Mode: 0644 -MTime: Fri May 23 13:34:20 2008 +MTime: Fri May 23 13:17:51 2008 Owner: 0 (root) Group: 0 (root) diff -u ./symlink_to_somefile.r1 ./symlink_to_somefile.local Mode: 0777 MTime: Fri May 23 13:18:12 2008 Owner: 0 (root) Group: 0 (root) --- so, when I choose only one revision I get no any errors. I hope my message will help you to resolve this bug. If you need any other information, mail me at vitaliy< at >gkh-kemerovo.ru

bugreport: fsvs update looses attributes

Nast — 2008-05-23T06:36:08

Hi, dev< at >fsvs.tigris.org I have got bug in fsvs then I try to execute fsvs update I'am using Debian Linux 4.0 (etch) and fsvs -V FSVS (licensed under the GPLv3), (C) by Ph. Marek; version fsvs-1.1.14:1496 First I create new repository #svnadmin create /mnt/1/repository2 I want to commit directory /tmp/1 into repository #cd /tmp/1 #ls -all --- -rw-r--r-- 1 root root 18 May 23 14:09 somefile --- directory has only one file #fsvs urls file:///mnt/1/repository2 Committing: #fsvs commit -m 'first' /tmp/1 --- Committing to file:///mnt/1/repository2 N... 18 /tmp/1/somefile committed revision 1 on 2008-05-23T06:11:50.954017Z as root --- committed success Now, I would like to checkout into /tmp/2 #mkdir /tmp/2 #fsvs checkout /tmp/2 file:///mnt/1/repository2 --- ...... 18 /tmp/2/somefile ...... dir /tmp/2 Checked out file:///mnt/1/repository2 at revision 1. --- everything seems to be ok Try to check attributes: for source: # ls -all /tmp/1 --- -rw-r--r-- 1 root root 18 May 23 14:09 somefile --- and for destination # ls -all /tmp/2 --- -rw-r--r-- 1 root root 18 May 23 14:09 somefile --- attributes are the same for source and destination dirs So, lets update source file I have modified file /tmp/1/somefile (added some bytes to it) and take a look at attributes again --- # ls -all /tmp/1 -rw-r--r-- 1 root root 20 May 23 14:12 somefile # ls -all /tmp/2 -rw-r--r-- 1 root root 18 May 23 14:09 somefile --- of course mdate and size has been changed Commit 2nd revision into repository: # fsvs commit -m 'second revision' /tmp/1 --- Committing to file:///mnt/1/repository2 .mC. 20 /tmp/1/somefile committed revision 2 on 2008-05-23T06:13:33.472190Z as root --- Ok, Now I would like to update my destination directory to 2nd revision (and as I understand after this operation destination directory /tmp/2 must contain file 'somefile' with mdate 14:12, size equal to 20 bytes and the same permissions as it was in source dir) So, updating: #fsvs update /tmp/2 --- Updated file:///mnt/1/repository2 to revision 2. .mC. 20 /tmp/2/somefile --- and checking attributes in destination: #ls -all /tmp/2 --- -rw------- 1 root root 20 May 23 14:12 somefile --- as you can see file size is 20 bytes and mdate is right too, but look at the permissions was 644, but now it changed to 600. Why? This permissons looses gets not every time, sometimes everything ok. And if I will make fsvs checkout again permissions will be correct, so error occurs only then I update local copy and only for files which has been changed last time ----- Best regards, Shuvaev Vitaliy

Re: FSVS 1.1.14 and up and CentOS 4.4: Error on make run-tests

Philipp Marek — 2008-05-08T15:27:34

Hello Ben, On Wednesday 07 May 2008 Benjamin M. wrote: thank you ... I'll come back to you. Regards, Phil

Re: FSVS 1.1.14 and up and CentOS 4.4: Error on make run-tests

Benjamin M. — 2008-05-07T18:08:52

Hi Phil, Philipp Marek wrote, On 06/05/08 03:33: Take all the time you need ... no hurry here and don't hesitate if you want me to test patches. Ben,

Re: FSVS 1.1.14 and up and CentOS 4.4: Error on make run-tests

Philipp Marek — 2008-05-06T07:33:11

Hello Ben! On Friday 02 May 2008 Benjamin M. wrote: Great, so the build problems are sorted out. I'll yet have to find and fix some bugs - some tests fail for me, too. But then again ... That I don't understand ... Do you have an immediate need to get a newer FSVS running, or could you wait for the next release (planned in 2 weeks, will probably be 3 or 4 weeks :( )? Regards, Phil

Re: FSVS 1.1.7 and up: apr_md5.h: No such file or directory (CentOS 5)

Philipp Marek — 2008-05-04T11:52:53

Hello Ben! ... Sorry, that was bad advice. The tests run with FSVS_WAA=/tmp/fsvs-test-0/waa FSVS_CONF=/tmp/fsvs-test-0/conf and I forgot to mention that these should be set, so I can see a bit of detail about the "Unknown resolver error (6490552)". Could you please - run "make run-tests" (and get an error, but have directories initialized), - set these two environment variables, and do - cd /tmp/fsvs-test-0 - echo file:/// | /usr/src/fsvs-1.1.7/src/fsvs urls load -d -v and send me the output? Thank you very much. Regards, Phil