gmane.comp.version-control.bazaar-ng.general

Re: [MERGE][bug #303538] Limit readv hunks to 100MB

Martin Pool — 2008-12-02T00:16:36

Martin Pool has voted approve. Status is now: Approved Comment: Well done. For details, see: http://bundlebuggy.aaronbentley.com/project/bzr/request/%3C49347330.7000205%40arbash-meinel.com%3E Project: Bazaar

Re: [rfc] 'bzr reconfigure --stacked-on/--no-stacked'

Martin Pool — 2008-12-02T00:15:39

I think it should do what set_stacked_on() does, which is add the stacking pointer but not remove any data from the repository (which I agree would be problematic.) This is still useful, at least in the case where the branch is empty.

Re: bzr version numbers

Jelmer Vernooij — 2008-12-02T00:15:06

That one should be fixed now. bzr shouldn't actually be reminding you to run upgrade in that situation though, perhaps just warn that the transfer is slower than it could be. Cheers, Jelmer

Re: RFC: commit-to-branch

Martin Pool — 2008-12-02T00:01:29

So (after a bit of off-mail conversation), it looks like Robert's considering a command or option that will essentially take the uncommitted changes, merge them into another thread or branch where they must cleanly apply, commit that, then merge that back into the current thread (stepwise up the loom if necessary), then commit that change onto the original branch, then update the tree onto the new merge commit. It perhaps sounds longwinded like that, but it's a reasonable case, if you've discovered maybe just one hunk that ought to be committed elsewhere. I do have a feeling that this is adding one large CISC-type operation, and there'll be some users who want to do something similar but not quite the same. For instance they might want to inspect the result of the merge before committing it, or deal with conflicts there. I realize it's common not to, but it'd also be reasonable to do it. I have some inclination to accommodate the simpler cases first, so they can be more flexibly combined, and because it can help user understanding to see them exposed. However, if you're motivated to do this specific case first, do it.

[Success!] [MERGE] test_sprout_uses_bzrdir_branch_format

Bundle Buggy — 2008-12-02T00:01:10

This change has been merged. Previous status: Approved For details, see: http://bundlebuggy.aaronbentley.com/project/bzr/request/%3C492310F2.9000101%40arbash-meinel.com%3E Project: Bazaar

Re: Git support for pip. What about Bazaar?

Martin Pool — 2008-12-01T23:55:11

On Tue, Dec 2, 2008 at 8:13 AM, Eduardo O. Padoan gmail.com> wrote: So, just try it - ask here, or on irc, if you need help with the apis, and someone will help you out. Yesterday on IRC a new API user said how easy they were to learn and use.

Re: bzr version numbers

Martin Pool — 2008-12-01T23:53:16

On Tue, Dec 2, 2008 at 7:18 AM, Russel Winder concertant.com> wrote: I think they should be primarily known by numbers, unlike Ubuntu. If the RM or a developer cares to stick a nickname on it, fine. Bug Jelmer in public. :-) It's a tough tradeoff though - pulling from a knit repository is going to be substantially slower and it seems wrong to let people just suffer that with no indication why. Perhaps the message could be better. We learned our lesson here. We now use sequential numbers for the classes , and the top-level user interface is to say --1.6 or --1.9, making it obvious which version can read it. (Using names like 'dirstate' or 'knit' has the problem that there's no apparent ordering, and that they're named by the technology used as one part of the format.) There are still infelicities here in that it's a bit hard to for the user to understand which option gives you eg Branch6, but there are bugs discussing how to fix them, and they will get done. The relationship is close enough that I don't know if it's worth drawing the distinction. Does it make a difference to users if we say 'will be kept for six approximately-monthly releases', or 'will be kept until the next release after six months have passed'? In any case it is not a precise science: for APIs, it's hard to define in Python just what is public or not (unilke Java), and for both formats and APIs we're likely to leave it supported until there's a positive reason to remove it.

Re: [MERGE][bug #303538] Limit readv hunks to 100MB

Bundle Buggy — 2008-12-01T23:29:43

Bundle Buggy has detected this merge request. For details, see: http://bundlebuggy.aaronbentley.com/project/bzr/request/%3C49347330.7000205%40arbash-meinel.com%3E Project: Bazaar

[MERGE][bug #303538] Limit readv hunks to 100MB

John Arbash Meinel — 2008-12-01T23:28:48

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 It turns out that our _coalesce_offsets code already had the ability to limit one range, but no callers were using it. max_size=0 was used to indicate "unlimited", but it seems that you can get a 20GB pack repository, and then unlimited is ~20GB, which is a bit too unlimited :). This changes it to be 100MB, still large, so it shouldn't effect any normal operations, but it gives some sort of cap to prevent us from reading 20GB all at once. It also changes RemoteTransport to fix a possible loophole. At the moment the Remote code was saying "pack no more than 50 requests together, and then break up those chunks into 5MB RPC requests" but if each request was 10MB, it would still create a 500MB RPC. Now this should help capping a single RPC to the size of 1 hunk, or possibly 10MB. (If two request ranges are 4.9MB, I think the combining code will combine them since the first was still under the 5MB limit.) I didn't add tests, as I wanted to do a quick fix (it has already taken 30min or so). I suppose we could factor out the constant into a class variable, and then have a test that sets the value significantly lower and then asserts the result. Actually, we can't do a 'self' variable because _coalesce is a static function, but I suppose we could use Transport._max_XXX. John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkk0czAACgkQJdeBCYSNAAOSSACffNqHJkKWriYCy1s/z8ZPZjy9 GpgAoLPa6lD1YsFft67ATcHFn9FVsByp =BuBp -----END PGP SIGNATURE----- # Bazaar merge directive format 2 (Bazaar 0.90) # revision_id: john< at >arbash-meinel.com-20081201232022-3vu5ekhfvnnzyujr # target_branch: http://bazaar-vcs.org/bzr/bzr.dev # testament_sha1: 803289bb52bef32888140cf0308d1a3407008bb9 # timestamp: 2008-12-01 17:26:39 -0600 # source_branch: http://bzr.arbash-meinel.com/branches/bzr/1.11\ # /303538_max_readv # base_revision_id: pqm< at >pqm.ubuntu.com-20081201201721-zconkq0v7pow8nmw # # Begin patch === modified file 'NEWS' --- NEWS2008-12-01 18:40:08 +0000 +++ NEWS2008-12-01 23:20:22 +0000 < at >< at > -23,6 +23,11 < at >< at > * Give proper error message for diff with non-existent dotted revno. (Marius Kruger, #301969) + * ``Transport.readv()`` defaults to not reading more than 100MB in a + single array. Further ``RemoteTransport.readv`` sets this to 5MB to + work better with how it splits its requests. + (John Arbash Meinel, #303538) + DOCUMENTATION: API CHANGES: === modified file 'bzrlib/transport/__init__.py' --- bzrlib/transport/__init__.py2008-11-07 18:10:32 +0000 +++ bzrlib/transport/__init__.py2008-12-01 23:20:22 +0000 < at >< at > -797,6 +797,10 < at >< at > cur = _CoalescedOffset(None, None, []) coalesced_offsets = [] + if max_size <= 0: + # 'unlimited', but we actually take this to mean 100MB buffer limit + max_size = 100*1024*1024 + for start, size in offsets: end = start + size if (last_end is not None === modified file 'bzrlib/transport/remote.py' --- bzrlib/transport/remote.py2008-11-21 00:23:54 +0000 +++ bzrlib/transport/remote.py2008-12-01 23:20:22 +0000 < at >< at > -323,7 +323,8 < at >< at > sorted_offsets = sorted(offsets) coalesced = list(self._coalesce_offsets(sorted_offsets, limit=self._max_readv_combine, - fudge_factor=self._bytes_to_read_before_seek)) + fudge_factor=self._bytes_to_read_before_seek, + max_size=self._max_readv_bytes)) # now that we've coallesced things, avoid making enormous requests requests = [] # Begin bundle IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWbxflasAA15/gHRQAAhZ//// f///sP////BgBw+9tLEAD27114A9cxkh0MkmpmiaMSnmp6ak/ITJ6piZqZpH6UPSPTKYgzap+qYn kEJQGgep5RpoGIADIAAGjQAADQDJJP0VPympsp6jR6mGoA9Q0DQ0NGgAAABoJEkApkym8kj0NNR4 mo08kBoNNNAAAaBoHMJoyNDQyGEaGQ00aADEZMgGEAwCUITIyEwmkxJjQm0jQI0Jp6TG1GgjHqAZ I5ECgNuSaPe31zcEDFBSbKzg7nsLFz62P4/HHwMg44ITAAgHr786a8KVbUkIDoLpY59rKy6PLV2I IkED/uHhwsjwD4K1NttNnPxfUOfcoqnoJsu9VbUcTxM25zfvUk4mkOwWhif+NQTSGtRm52OmvPRP DucqYVyuzuRVvr3f+5E/ll4WFBXVMZXXzehXg+9bJ1CUaTOL3J7Y29GXny+jpwXo1Dv/Jy2LJyXf Y1YoJCskl1ZAFPMONHk3JSkebGp7Rnlg7eeLN1fQEDWaTKJhOMYQSEgUQTAHEPkI1h4YJOYEqRjA L9Nj319vWdmySnwBQTp/ns0LxTQPTh1BlCOdrJ3/YJUXdmtBiab3GuoPHQYxeNlLSCSvE5WTJIxF kCVlJObsQFZQGcmgoEywkSVhZUTMpJwROfscxXSX4CSqpS7QcEF48l4vcn9Dz9jZC13DUpC7eT3K n+zRtDLR+YrjJkAUG4sCVGk2lNiRdx9oKjFDtJkrRglEmfTKrluSjVMH9b8aGovHjFyuIU8SR1Ak l0RBL0AeKFhAEliVhCuUgS5+dYwI8irq03jOM2fHrurNQqJmcpmZEIGJg11pbp9fBe/DUmh8Libw RSxFAbebLs0Y0bhExBJVV0cRxMdpu1ldwl5jtv3BTz3j61UrwNpE1g4cOgKTn4dcfC/THQzWyCtg ZmJcKRFPQl+vMxNiEaj0wJCk4DjVqLB5YQFOgZdCwEeMajunIm+7zFySKBV4qmNTMOzxurJla4Dz u0Q/i79vGaHr6p76GwuNwsNQPWsXHPE5DEYmWGDEywoQKido8toTE3kS0wHOPKFBuSlaqxYrrXsy IHAvwcPZxQiQJDXEWsIj63Ey86QV1dmHUtN6PMSg8YdEkenPEgQFct1+YpVGsrB8SAQBUCICYlc0 S2RVnZOhNKHbintvC2WDjW7GrlChhXNLf1Bm25TRJ1TRNEklLuhhq7tmSnpz1thr+0SDMBeFxEIQ oP4NdnhTi7X+tdub/eP5TqgU2+AqJ3A5EKKKHM5g8i3FzHgkEwSo0OYklY9e7IXCFEwFKByP+JKt ti82fu8cr2qEOVaNpbYBO2/hUOR3f0aRHgdPQjuWv93pad2/IKpRoRyE8S0QDqbrnngaGFFrgemL YLyByiUdAjEI9B2btP4suNcoUTgiHWTdxU8l8hYZdcgYoFLghBNsEYv9wB7e9yluX3/W8Sl3ss0P mU+UyU3nqOo1kwDyxIn4lZx456+kgdrhfm5w5R5WWiGW4sMC41mo+d4op9xkeqHVwvkSKHE7C8Uv HPMqh2ya9TbQQpxB5gsmbUR3GlVRMiTQzKhGIYlbKBVUVnhhUXVWmraDqeJg8mJ7uxlTr4fPzeV0 E/AwNpuIzNoxsUWKX+qrxEssE8iC+UuqTAXyQdDBFRMekU1qhZKhzC14jvUpA5fRXrH6d5QRMpgm YBWV+6UXDcqMgYaYc12x0/qgI7rFoKvC+NzWgnJEDdK6zJqyA4vMDmOc5DeSOI0B/Id1m8cKcpch 4zUGz8E7U4htmwN/frOgEk9aDrmNpb2nwlGCo6MRw5ytiW618ohBLM+k9QJmsAhrj9/x7USOjduS r4VcAInv3Bb0IDzyF5vxROcAkfchH257YKAYZwuEiwuQxZKkXnM8la35a4S++Qw6ABs0SJlTrb06 gByyebEAwuUtQgzAxvpIFCiWGO+ATk1MkmdvACNAEKy0fRq2ES0iUW57TkZ1b9VxVkDEFMjORS2a mKQlbAVoxxTiqSiUAHLZGh7TJ4imQPwBg0grLzFJKlgUB8ZLYBacoZnxmTymgS7EEKwHqC+yx4OR JAb561Db40SCSqNIQD85w1wU6OjnE+MqCFglmHaiN8gb931B5qJdcimXATzBpQDLLdX+cjZcIcQo W0By2B6DJ1ioSotfT1vDA3dQImt7xCualx4arEYKyVwonnSvwQwaoW4PhmiJyL3ir9Ctyln7G++0 AgK4MKgg3wuTRQIEFCYjdYlDPWxNEVZULMpPxeJH6c/RWd8qz4hL8y1wsMDxlSJYf8g4vUiwxYoZ 91ER6TzNausMrYvIzYSFXazgsxqCIZ+kEqdzhzdny9YbATj54Fpq98Qyp7HJS/kAUtBqiOEbxItg HsAMc/wsXGgFWlml2+u1Rnszv7UI3LaBr5th0dwkqoNAaff6ys+QEiZhaietZ/ZvBP/i7kinChIX i/K1YA==

Re: [MERGE][#299313] --local commits are accessing the network

Robert Collins — 2008-12-01T23:11:03

If we make get_nick public, we should deprecate branch.nick. -Rob

Re: [CHK/MERGE] CHKMap.iter_interesting_nodes() and fetch updates

Robert Collins — 2008-12-01T22:57:21

Yes. I'm currently working on commit to make it have a safe, single-path iter_changes based core. I think its important we do a bench test with a hash'd key. I'm not convinced that it will be better or worse (though there is experimental evidence that its better - which is another reason to test it). For length 2 key-tuples, I suggest that the mapping be (hash(key[0]), hash(key[1])), to give locality of reference. Ah yes, exactly what I just said above :). Yup, definitely worth doing. One thing to rememer is that nodes are always defined bottom-up, so when you pull out such things, be sure not to make them start depending on a top-down provision of data. -Rob

Re: [rfc] 'bzr reconfigure --stacked-on/--no-stacked'

Aaron Bentley — 2008-12-01T22:39:05

I certainly agree with "reconfigure --no-stacked". What would --stacked-on do? Stack the branch and replace its repository with one that only had the bare minimum? Aaron

Re: [rfc] 'bzr reconfigure --stacked-on/--no-stacked'

Robert Collins — 2008-12-01T21:16:29

Complete agreement here. -Rob

Re: BzrEclipse and CPU load

Robert Collins — 2008-12-01T20:21:51

I think calling a single helper is much better, as every IDE will probably want this, and not all IDE's are java. -Rob

Re: Whygitisbetterthan.com, some advocacy from GitHub

Elliot Murphy — 2008-12-01T22:00:15

Hurrah! Things just keep getting better.

Re: [MERGE] Re-order the packs during fetch

John Arbash Meinel — 2008-12-01T21:47:09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andrew Bennetts wrote: One nice thing about this patch is that it helps both pack-0.92 repos as well as --1.9 format branches. Here are some specific numbers, of how long it takes me to update my local repository from bzr.dev 3874 to bzr.dev 3876 37.2s bzr1.9 from --format=pack-0.92 18.2s bzr.dev from --format=pack-0.92 16.6s bzr1.9 from --format=1.9 10.1s bzr.dev from --format=1.9 So for the existing repositories, 'bzr pull' of a small amount of data is almost 2x as fast, and for 1.9 format it is about 1.6:1. And that is still 1.8:1 faster than 0.92 format. (So you get almost 4:1 upgrading your repository to 1.9 and your client to 1.10rc1.) And this is all done over plain HTTP with a latency around 200ms. The next biggest win is probably the smart fetching that doesn't have to probe the indexes directly. But there is also a big win in getting the smart verbs for opening the branch and getting its information. It takes 3.2s (30%) before we have Branch.last_revision and are thinking about fetching. Getting streaming fetch will still be a big win. Of the 37kB we download (from 1.9) only 5.5kB or so is actual .pack data. John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkk0W10ACgkQJdeBCYSNAAOBmwCbBX0a6Z8tRsqBs5AWYJKy4Qba 7UcAoNDBCPeaH6t4ShBE5xHYXVB4ajON =MyGU -----END PGP SIGNATURE-----

Re: Whygitisbetterthan.com, some advocacy from GitHub

John Arbash Meinel — 2008-12-01T21:13:58

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Elliot Murphy wrote: It is in 1.10rc1. :) With a caveat that it doesn't quite work right on Windows because of file locking issues. John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkk0U5YACgkQJdeBCYSNAAOf7ACgkxWMeYFa4qA04hCIPtjAJQJO HYYAnAxMTPQIvubWx+f03JbbgOEXpocZ =4Rm5 -----END PGP SIGNATURE-----

Git support for pip. What about Bazaar?

Eduardo O. Padoan — 2008-12-01T21:13:32

Hi folks! Jannis Leidel, on his branch at github [1] has added support to pip [2] to install from git repositories. It already supports svn repos. Anyone interested on adding bzr support to it? I would do it, but I'd need some time to actually learn some useful quantity of bzrlib to do any good work :P, which I intend to do, some day... So if there is someone more motivated, please speak up! Thanks! [1] http://github.com/jezdez/pip/tree/git-support [2] ex-pyinstall,py, ease_install sucessor http://blog.ianbicking.org/2008/10/28/pyinstall-is-dead-long-live-pip/

Whygitisbetterthan.com, some advocacy from GitHub

Elliot Murphy — 2008-12-01T21:11:36

I while I use bzr and launchpad every single day, I really respect the work done at GitHub, and just saw this site: http://whygitisbetterthanx.com/ Git is listed as better than bzr in 4 areas: - cheap local branching - git is fast - the staging area I don't understand why git is supposed to be better than bzr at cheap local branching, perhaps this is a case of not having a shared repo by default causes a surface evaluation to make git look better here. I haven't investigated the claim about speed, but the numbers are posted and the benchmarking was done with publicly available branches of the django project, so it would be interesting to get confirmation on these. I disagree with the claim that the staging area makes git better than bzr but I can see how it's a subjective thing - for me, shelve works great to handle this. Of course, shelve is not in the core commands that bzr ships... I disagree that GitHub is the only social network for code because I use launchpad the same way that GitHub is described here, but GitHub has plenty of neat features that I drool over and would like to see in launchpad (and launchpad has plenty of features that I would miss sorely if I used GitHub exclusively). So I'd say this one should be a tie, and it's totally fair for the GitHub folks to pitch GitHub as a reason to choose git. Anyway, interesting stuff to look at.

[Success!] [Merge][#301969]Give proper error message for diff withnon-existent dotted revno

Bundle Buggy — 2008-12-01T20:29:39

This change has been merged. Previous status: Approved For details, see: http://bundlebuggy.aaronbentley.com/project/bzr/request/%3C418c22640812010943k3b765e05k77b585bbbbb3d2c%40mail.gmail.com%3E Project: Bazaar

Re: bzr version numbers

Russel Winder — 2008-12-01T20:18:42

John, On Mon, 2008-12-01 at 12:50 -0600, John Arbash Meinel wrote: Avaricious Aardvark Abused Abalone Accurate Actuary (One could go on for potentially many years like this, but is it worth it. :-) I think that makes it an agreed thing, names are only useful where the rate of change is relatively low. Also the time required to choose the names will detract for the important work of evolving Bazaar. As a user, I have to reiterate that I haven't a clue how to deal with all these -- but this has already been aired on the email list. The real problem is, of course, deprecation. For example (sorry Jelmer, but...): Format for http://people.samba.org/bzr/jelmer/bzr-rebase/.bzr/ is deprecated - please use 'bzr upgrade' to get better performance As a user, I have no control, just vast irritation as I get messages about which I can do nothing. There is an alternative: Format 1, Format 2, Format 3, etc. The integers make a very good sequence numbering that carries little or no baggage. An alternative would be Knit, Dirstate, Pack, Stack, BTree, but this opens up the door to silly sniping: why BTree when RedBlackTree is quicker -- this is a fatuous comment on my part of course as in this case BTrees are likely the best data structure, but I think you'll get what I mean. OK, this is the one that got me to do a response, the above is just (hopefully humorous) intro. Deprecations should not be determined by time unless there is a rigid relationship between time and releases. Deprecations should be determined by formal releases. So three releases between announcement and removal for example. Any temporal relationship is generally purely accidental. Announce deprecation, reinforce deprecation, remove -- so not in next, it's too soon. As we know Sun have a very poor, even bizarre record of deprecation with the JDK -- nothing ever actually gets removed, and some deprecated things get un-deprecated! Fortunately is was a sane decision to undeprecate the things they did.