gmane.comp.version-control.bazaar-ng.general

[MERGE/RFC] CHKMap._check_remap()

John Arbash Meinel — 2008-12-03T03:00:25

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The attached patch is targeted to the split-inventory work. The important bit is adding CHKMap._check_remap() which does some work to see if child entries can be packed into a single page. This is important to ensure canonical form even when we only create a new CHKMap via a delta. I also found a small bug in plain map() when a split is triggered and one of the keys is identical to the common prefix. There is a test for it, and we just add '\0' to the end of that key's prefix in serialized form. I'm approx 90% confident that this is enough to ensure canonical form. At the end of the conversion I did: python -c "import bzrlib.branch b = bzrlib.branch.Branch.open('.') b.lock_write() b.repository.start_write_group() rt = b.repository.revision_tree(b.last_revision()) i = rt.inventory.id_to_entry print i.key() chk = bzrlib.chk_map.CHKMap(b.repository.chk_bytes, None) chk._root_node.set_maximum_size(4096) for k, v in i.iteritems(): chk.map(k,v) print chk._save() b.repository.abort_write_group() b.unlock() " And it comes out with the same final sha1 sum. So after 21k revisions that add/remove/rename/etc, we still have canonical form. ('sha1:67a41796266642d4714b40d9708f61af70d003af',) ('sha1:67a41796266642d4714b40d9708f61af70d003af',) John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkk19kgACgkQJdeBCYSNAAOhcwCfTyNEOUruYJ+9CNA7yljo1ZT3 tb0An1HAJI7foSFkuFnCFkXhjcGF2Mr9 =Vnnr -----END PGP SIGNATURE----- # Bazaar merge directive format 2 (Bazaar 0.90) # revision_id: john< at >arbash-meinel.com-20081202235625-h5fo44xy2hxeopo2 # target_branch: http://bzr.arbash-meinel.com/branches/bzr/brisbane\ # /core # testament_sha1: f960e83a830403ae01592d914e31ef5079fca48a # timestamp: 2008-12-02 20:40:30 -0600 # source_branch: http://bzr.arbash-meinel.com/branches/bzr/brisbane\ # /chk_map # base_revision_id: robertc< at >robertcollins.net-20081201043625-\ # 77tcktr1dfxc7w5l # # Begin patch === modified file 'bzrlib/chk_map.py' --- bzrlib/chk_map.py2008-11-20 20:34:37 +0000 +++ bzrlib/chk_map.py2008-12-02 23:56:25 +0000 < at >< at > -22,7 +22,7 < at >< at > with internal nodes of 8-bit fan out; The key tuples are mapped to strings by joining them by \x00, and \x00 padding shorter keys out to the length of the longest key. Leaf nodes are packed as densely as possible, and internal nodes -are all and additional 8-bits wide leading to a sparse upper tree. +are all and additional 8-bits wide leading to a sparse upper tree. Updates to a CHKMap are done preferentially via the apply_delta method, to allow optimisation of the update operation; but individual map/unmap calls are < at >< at > -90,7 +90,7 < at >< at > Node that this does not update the _items dict in objects containing a reference to this node. As such it does not prevent subsequent IO being performed. - + :param node: A tuple key or node object. :return: A node object. """ < at >< at > -108,6 +108,7 < at >< at > """Return the tree in a string representation.""" self._ensure_root() res = self._dump_tree_node(self._root_node, prefix='', indent='') + res.append('') # Give a trailing '\n' return '\n'.join(res) def _dump_tree_node(self, node, prefix, indent): < at >< at > -131,7 +132,7 < at >< at > < at >classmethod def from_dict(klass, store, initial_value, maximum_size=0, key_width=1): """Create a CHKMap in store with initial_value as the content. - + :param store: The store to record initial_value in, a VersionedFiles object with 1-tuple keys supporting CHK key generation. :param initial_value: A dict to store in store. Its keys and values < at >< at > -176,7 +177,7 < at >< at > # after reading the 'a' subtree, but it is encountered in the first # tree immediately. Variations on this may have read internal nodes like this. # we want to cut the entire pending subtree when we realise we have a common node. - # For this we use a list of keys - the path to a node - and check the entire path is + # For this we use a list of keys - the path to a node - and check the entire path is # clean as we process each item. if self._node_key(self._root_node) == self._node_key(basis._root_node): return < at >< at > -210,7 +211,7 < at >< at > # Note that this is N^2, it depends on us trimming trees # aggressively to not become slow. # A better implementation would probably have a reverse map - # back to the children of a node, and jump straight to it when + # back to the children of a node, and jump straight to it when # a common node is detected, the proceed to remove the already # pending children. bzrlib.graph has a searcher module with a # similar problem. < at >< at > -358,7 +359,8 < at >< at > def unmap(self, key): """remove key from the map.""" self._ensure_root() - self._root_node.unmap(self._store, key) + unmapped = self._root_node.unmap(self._store, key) + self._root_node = unmapped def _save(self): """Save the map completely. < at >< at > -420,7 +422,7 < at >< at > class LeafNode(Node): """A node containing actual key:value pairs. - + :ivar _items: A dict of key->value items. The key is in tuple form. """ < at >< at > -491,23 +493,46 < at >< at > # then that can be done via the C extension return 2 + len('\x00'.join(key)) + len(value) + def _map_no_split(self, key, value): + """Map a key to a value. + + This assumes either the key does not already exist, or you have already + removed its size and length from self. + + :return: True if adding this node should cause us to split. + """ + self._items[key] = value + self._size += self._key_value_len(key, value) + self._len += 1 + if (self._len > 1 + and self._maximum_size + and self._current_size() > self._maximum_size): + return True + return False + def map(self, store, key, value): """Map key to value.""" if key in self._items: self._size -= self._key_value_len(key, self._items[key]) self._len -= 1 - self._items[key] = value - self._size += self._key_value_len(key, value) - self._len += 1 self._key = None - if (self._maximum_size and self._current_size() > self._maximum_size and - self._len > 1): + if self._map_no_split(key, value): common_prefix = self.unique_serialised_prefix() split_at = len(common_prefix) + 1 result = {} for key, value in self._items.iteritems(): serialised_key = self._serialised_key(key) prefix = serialised_key[:split_at] + # TODO: Generally only 1 key can be exactly the right length, + # which means we can only have 1 key in the node pointed + # at by the 'prefix\0' key. We might want to consider + # folding it into the containing InternalNode rather than + # having a fixed length-1 node. + # Note this is probably not true for hash keys, as they + # may get a '\00' node anywhere, but won't have keys of + # different lengths. + if len(prefix) < split_at: + prefix += '\x00'*(split_at - len(prefix)) if prefix not in result: node = LeafNode() node.set_maximum_size(self._maximum_size) < at >< at > -576,7 +601,7 < at >< at > class InternalNode(Node): """A node that contains references to other nodes. - + An InternalNode is responsible for mapping serialised key prefixes to child nodes. It is greedy - it will defer splitting itself as long as possible. """ < at >< at > -723,6 +748,10 < at >< at > # new child needed: child = self._new_child(serialised_key, LeafNode) old_len = len(child) + if isinstance(child, LeafNode): + old_size = child._current_size() + else: + old_size = None prefix, node_details = child.map(store, key, value) if len(node_details) == 1: # child may have shrunk, or might be a new node < at >< at > -730,7 +759,15 < at >< at > self._len = self._len - old_len + len(child) self._items[serialised_key] = child self._key = None - return self.unique_serialised_prefix(), [("", self)] + new_node = self + if (isinstance(child, LeafNode) + and (old_size is None or child._current_size() < old_size)): + # The old node was an InternalNode which means it has now + # collapsed, so we need to check if it will chain to a collapse + # at this level. Or the LeafNode has shrunk in size, so we need + # to check that as well. + new_node = self._check_remap(store) + return new_node.unique_serialised_prefix(), [("", new_node)] # child has overflown - create a new intermediate node. # XXX: This is where we might want to try and expand our depth # to refer to more bytes of every child (which would give us < at >< at > -845,26 +882,107 < at >< at > """Remove key from this node and it's children.""" if not len(self._items): raise AssertionError("cant unmap in an empty InternalNode.") - serialised_key = self._serialised_key(key) children = self._iter_nodes(store, key_filter=[key]) - serialised_key = self._serialised_key(key) if children: child = children[0] else: raise KeyError(key) self._len -= 1 unmapped = child.unmap(store, key) + self._key = None + serialised_key = self._serialised_key(key) if len(unmapped) == 0: # All child nodes are gone, remove the child: del self._items[serialised_key] + unmapped = None else: # Stash the returned node self._items[serialised_key] = unmapped if len(self._items) == 1: # this node is no longer needed: return self._items.values()[0] - self._key = None - return self + if isinstance(unmapped, InternalNode): + return self + return self._check_remap(store) + + def _check_remap(self, store): + """Check if all keys contained by children fit in a single LeafNode. + + :param store: A store to use for reading more nodes + :return: Either self, or a new LeafNode which should replace self. + """ + # Logic for how we determine when we need to rebuild + # 1) Implicitly unmap() is removing a key which means that the child + # nodes are going to be shrinking by some extent. + # 2) If all children are LeafNodes, it is possible that they could be + # combined into a single LeafNode, which can then completely replace + # this internal node with a single LeafNode + # 3) If *one* child is an InternalNode, we assume it has already done + # all the work to determine that its children cannot collapse, and + # we can then assume that those nodes *plus* the current nodes don't + # have a chance of collapsing either. + # So a very cheap check is to just say if 'unmapped' is an + # InternalNode, we don't have to check further. + + # TODO: Another alternative is to check the total size of all known + # LeafNodes. If there is some formula we can use to determine the + # final size without actually having to read in any more + # children, it would be nice to have. However, we have to be + # careful with stuff like nodes that pull out the common prefix + # of each key, as adding a new key can change the common prefix + # and cause size changes greater than the length of one key. + # So for now, we just add everything to a new Leaf until it + # splits, as we know that will give the right answer + new_leaf = LeafNode() + new_leaf.set_maximum_size(self._maximum_size) + new_leaf._key_width = self._key_width + keys = {} + # There is some overlap with _iter_nodes here, but not a lot, and it + # allows us to do quick evaluation without paging everything in + for prefix, node in self._items.iteritems(): + if type(node) == tuple: + keys[node] = prefix + else: + if isinstance(node, InternalNode): + # Without looking at any leaf nodes, we are sure + return self + for key, value in node._items.iteritems(): + if new_leaf._map_no_split(key, value): + # Adding this key would cause a split, so we know we + # don't need to collapse + return self + # So far, everything fits. Page in the rest of the nodes, and see if it + # holds true. + if keys: + # TODO: Consider looping over a limited set of keys (like 25 or so + # at a time). If we have to read more than 25 we almost + # certainly won't fit them all into a single new LeafNode, so + # reading the extra nodes is a waste. + # This will probably matter more with hash serialised keys, + # as we will get InternalNodes with more references. + # The other argument is that unmap() is uncommon, so we don't + # need to optimize it. But map() with a slightly shorter + # value may happen a lot. + stream = store.get_record_stream(keys, 'unordered', True) + nodes = [] + # Fully consume the stream, even if we could determine that we + # don't need to continue. We requested the bytes, we may as well + # use them + for record in stream: + node = _deserialise(record.get_bytes_as('fulltext'), record.key) + self._items[keys[record.key]] = node + nodes.append(node) + for node in nodes: + if isinstance(node, InternalNode): + # We know we won't fit + return self + for key, value in node._items.iteritems(): + if new_leaf._map_no_split(key, value): + return self + + # We have gone to every child, and everything fits in a single leaf + # node, we no longer need this internal node + return new_leaf def _deserialise(bytes, key): === modified file 'bzrlib/tests/test_chk_map.py' --- bzrlib/tests/test_chk_map.py2008-12-01 04:36:25 +0000 +++ bzrlib/tests/test_chk_map.py2008-12-02 23:44:34 +0000 < at >< at > -70,6 +70,73 < at >< at > self.assertEqual("chkleaf:\n0\n1\n0\n", self.read_bytes(chk_bytes, root_key)) return root_key + def assertMapLayoutEqual(self, map_one, map_two): + """Assert that the internal structure is identical between the maps.""" + map_one._ensure_root() + node_one_stack = [map_one._root_node] + map_two._ensure_root() + node_two_stack = [map_two._root_node] + while node_one_stack: + node_one = node_one_stack.pop() + node_two = node_two_stack.pop() + if node_one.__class__ != node_two.__class__: + self.assertEqualDiff(map_one._dump_tree(), + map_two._dump_tree()) + if isinstance(node_one, InternalNode): + # Internal nodes must have identical references + self.assertEqual(sorted(node_one._items.keys()), + sorted(node_two._items.keys())) + self.assertEqual(node_one._prefix, node_two._prefix) + node_one_stack.extend(node_one._iter_nodes(map_one._store)) + node_two_stack.extend(node_two._iter_nodes(map_two._store)) + else: + # Leaf nodes must have identical contents + self.assertEqual(node_one._items, node_two._items) + + def assertCanonicalForm(self, chkmap): + """Assert that the chkmap is in 'canonical' form. + + We do this by adding all of the key value pairs from scratch, both in + forward order and reverse order, and assert that the final tree layout + is identical. + """ + items = list(chkmap.iteritems()) + map_forward = chk_map.CHKMap(None, None) + map_forward._root_node.set_maximum_size(chkmap._root_node.maximum_size) + for key, value in items: + map_forward.map(key, value) + self.assertMapLayoutEqual(map_forward, chkmap) + map_reverse = chk_map.CHKMap(None, None) + map_reverse._root_node.set_maximum_size(chkmap._root_node.maximum_size) + for key, value in reversed(items): + map_reverse.map(key, value) + self.assertMapLayoutEqual(map_reverse, chkmap) + + def test_assert_map_layout_equal(self): + store = self.get_chk_bytes() + map_one = CHKMap(store, None) + map_one._root_node.set_maximum_size(20) + map_two = CHKMap(store, None) + map_two._root_node.set_maximum_size(20) + self.assertMapLayoutEqual(map_one, map_two) + map_one.map('aaa', 'value') + self.assertRaises(AssertionError, + self.assertMapLayoutEqual, map_one, map_two) + map_two.map('aaa', 'value') + self.assertMapLayoutEqual(map_one, map_two) + # Split the tree, so we ensure that internal nodes and leaf nodes are + # properly checked + map_one.map('aab', 'value') + self.assertIsInstance(map_one._root_node, InternalNode) + self.assertRaises(AssertionError, + self.assertMapLayoutEqual, map_one, map_two) + map_two.map('aab', 'value') + self.assertMapLayoutEqual(map_one, map_two) + map_one.map('aac', 'value') + self.assertRaises(AssertionError, + self.assertMapLayoutEqual, map_one, map_two) + self.assertCanonicalForm(map_one) + def test_from_dict_empty(self): chk_bytes = self.get_chk_bytes() root_key = CHKMap.from_dict(chk_bytes, {}) < at >< at > -120,6 +187,8 < at >< at > (None, ('bba',), 'target2'), (None, ('bbb',), 'common')]) root_key1 = chkmap1._save() + self.assertCanonicalForm(chkmap1) + chkmap2 = CHKMap(chk_bytes, None) chkmap2._root_node.set_maximum_size(10) chkmap2.apply_delta([(None, ('bbb',), 'common'), < at >< at > -128,6 +197,7 < at >< at > root_key2 = chkmap2._save() self.assertEqualDiff(chkmap1._dump_tree(), chkmap2._dump_tree()) self.assertEqual(root_key1, root_key2) + self.assertCanonicalForm(chkmap2) def test_stable_splitting(self): store = self.get_chk_bytes() < at >< at > -136,13 +206,15 < at >< at > chkmap._root_node.set_maximum_size(30) chkmap.map(('aaa',), 'v') self.assertEqualDiff("'' LeafNode None\n" - " ('aaa',) 'v'", + " ('aaa',) 'v'\n", chkmap._dump_tree()) chkmap.map(('aab',), 'v') self.assertEqualDiff("'' LeafNode None\n" " ('aaa',) 'v'\n" - " ('aab',) 'v'", + " ('aab',) 'v'\n", chkmap._dump_tree()) + self.assertCanonicalForm(chkmap) + # Creates a new internal node, and splits the others into leaves chkmap.map(('aac',), 'v') self.assertEqualDiff("'' InternalNode None\n" < at >< at > -151,8 +223,10 < at >< at > " 'aab' LeafNode None\n" " ('aab',) 'v'\n" " 'aac' LeafNode None\n" - " ('aac',) 'v'", + " ('aac',) 'v'\n", chkmap._dump_tree()) + self.assertCanonicalForm(chkmap) + # Splits again, because it can't fit in the current structure chkmap.map(('bbb',), 'v') self.assertEqualDiff("'' InternalNode None\n" < at >< at > -164,8 +238,19 < at >< at > " 'aac' LeafNode None\n" " ('aac',) 'v'\n" " 'b' LeafNode None\n" - " ('bbb',) 'v'", + " ('bbb',) 'v'\n", chkmap._dump_tree()) + self.assertCanonicalForm(chkmap) + + def test_map_splits_with_longer_key(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 1 key per LeafNode + chkmap._root_node.set_maximum_size(10) + chkmap.map(('aaa',), 'v') + chkmap.map(('aaaa',), 'v') + self.assertCanonicalForm(chkmap) + self.assertIsInstance(chkmap._root_node, InternalNode) def test_deep_splitting(self): store = self.get_chk_bytes() < at >< at > -176,7 +261,7 < at >< at > chkmap.map(('aaaaabaa',), 'v') self.assertEqualDiff("'' LeafNode None\n" " ('aaaaaaaa',) 'v'\n" - " ('aaaaabaa',) 'v'", + " ('aaaaabaa',) 'v'\n", chkmap._dump_tree()) chkmap.map(('aaabaaaa',), 'v') chkmap.map(('aaababaa',), 'v') < at >< at > -186,7 +271,7 < at >< at > " ('aaaaabaa',) 'v'\n" " 'aaab' LeafNode None\n" " ('aaabaaaa',) 'v'\n" - " ('aaababaa',) 'v'", + " ('aaababaa',) 'v'\n", chkmap._dump_tree()) chkmap.map(('aaabacaa',), 'v') chkmap.map(('aaabadaa',), 'v') < at >< at > -202,7 +287,7 < at >< at > " 'aaabac' LeafNode None\n" " ('aaabacaa',) 'v'\n" " 'aaabad' LeafNode None\n" - " ('aaabadaa',) 'v'", + " ('aaabadaa',) 'v'\n", chkmap._dump_tree()) chkmap.map(('aaababba',), 'v') chkmap.map(('aaababca',), 'v') < at >< at > -223,7 +308,7 < at >< at > " 'aaabac' LeafNode None\n" " ('aaabacaa',) 'v'\n" " 'aaabad' LeafNode None\n" - " ('aaabadaa',) 'v'", + " ('aaabadaa',) 'v'\n", chkmap._dump_tree()) # Now we add a node that should fit around an existing InternalNode, # but has a slightly different key prefix, which causes a new < at >< at > -249,8 +334,247 < at >< at > " 'aaabac' LeafNode None\n" " ('aaabacaa',) 'v'\n" " 'aaabad' LeafNode None\n" - " ('aaabadaa',) 'v'", - chkmap._dump_tree()) + " ('aaabadaa',) 'v'\n", + chkmap._dump_tree()) + + def test_map_collapses_if_size_changes(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 2 keys per LeafNode + chkmap._root_node.set_maximum_size(30) + chkmap.map(('aaa',), 'v') + chkmap.map(('aab',), 'very long value that splits') + self.assertEqualDiff("'' InternalNode None\n" + " 'aaa' LeafNode None\n" + " ('aaa',) 'v'\n" + " 'aab' LeafNode None\n" + " ('aab',) 'very long value that splits'\n", + chkmap._dump_tree()) + self.assertCanonicalForm(chkmap) + # Now changing the value to something small should cause a rebuild + chkmap.map(('aab',), 'v') + self.assertEqualDiff("'' LeafNode None\n" + " ('aaa',) 'v'\n" + " ('aab',) 'v'\n", + chkmap._dump_tree()) + self.assertCanonicalForm(chkmap) + + def test_map_double_deep_collapses(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 3 small keys per LeafNode + chkmap._root_node.set_maximum_size(40) + chkmap.map(('aaa',), 'v') + chkmap.map(('aab',), 'very long value that splits') + chkmap.map(('abc',), 'v') + self.assertEqualDiff("'' InternalNode None\n" + " 'aa' InternalNode None\n" + " 'aaa' LeafNode None\n" + " ('aaa',) 'v'\n" + " 'aab' LeafNode None\n" + " ('aab',) 'very long value that splits'\n" + " 'ab' LeafNode None\n" + " ('abc',) 'v'\n", + chkmap._dump_tree()) + chkmap.map(('aab',), 'v') + self.assertCanonicalForm(chkmap) + self.assertEqualDiff("'' LeafNode None\n" + " ('aaa',) 'v'\n" + " ('aab',) 'v'\n" + " ('abc',) 'v'\n", + chkmap._dump_tree()) + + def test_stable_unmap(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 2 keys per LeafNode + chkmap._root_node.set_maximum_size(30) + chkmap.map(('aaa',), 'v') + chkmap.map(('aab',), 'v') + self.assertEqualDiff("'' LeafNode None\n" + " ('aaa',) 'v'\n" + " ('aab',) 'v'\n", + chkmap._dump_tree()) + # Creates a new internal node, and splits the others into leaves + chkmap.map(('aac',), 'v') + self.assertEqualDiff("'' InternalNode None\n" + " 'aaa' LeafNode None\n" + " ('aaa',) 'v'\n" + " 'aab' LeafNode None\n" + " ('aab',) 'v'\n" + " 'aac' LeafNode None\n" + " ('aac',) 'v'\n", + chkmap._dump_tree()) + self.assertCanonicalForm(chkmap) + # Now lets unmap one of the keys, and assert that we collapse the + # structures. + chkmap.unmap(('aac',)) + self.assertEqualDiff("'' LeafNode None\n" + " ('aaa',) 'v'\n" + " ('aab',) 'v'\n", + chkmap._dump_tree()) + self.assertCanonicalForm(chkmap) + + def test_unmap_double_deep(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 3 keys per LeafNode + chkmap._root_node.set_maximum_size(40) + chkmap.map(('aaa',), 'v') + chkmap.map(('aaab',), 'v') + chkmap.map(('aab',), 'very long value') + chkmap.map(('abc',), 'v') + self.assertEqualDiff("'' InternalNode None\n" + " 'aa' InternalNode None\n" + " 'aaa' LeafNode None\n" + " ('aaa',) 'v'\n" + " ('aaab',) 'v'\n" + " 'aab' LeafNode None\n" + " ('aab',) 'very long value'\n" + " 'ab' LeafNode None\n" + " ('abc',) 'v'\n", + chkmap._dump_tree()) + # Removing the 'aab' key should cause everything to collapse back to a + # single node + chkmap.unmap(('aab',)) + self.assertEqualDiff("'' LeafNode None\n" + " ('aaa',) 'v'\n" + " ('aaab',) 'v'\n" + " ('abc',) 'v'\n", + chkmap._dump_tree()) + + def test_unmap_double_deep_non_empty_leaf(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 3 keys per LeafNode + chkmap._root_node.set_maximum_size(40) + chkmap.map(('aaa',), 'v') + chkmap.map(('aab',), 'long value') + chkmap.map(('aabb',), 'v') + chkmap.map(('abc',), 'v') + self.assertEqualDiff("'' InternalNode None\n" + " 'aa' InternalNode None\n" + " 'aaa' LeafNode None\n" + " ('aaa',) 'v'\n" + " 'aab' LeafNode None\n" + " ('aab',) 'long value'\n" + " ('aabb',) 'v'\n" + " 'ab' LeafNode None\n" + " ('abc',) 'v'\n", + chkmap._dump_tree()) + # Removing the 'aab' key should cause everything to collapse back to a + # single node + chkmap.unmap(('aab',)) + self.assertEqualDiff("'' LeafNode None\n" + " ('aaa',) 'v'\n" + " ('aabb',) 'v'\n" + " ('abc',) 'v'\n", + chkmap._dump_tree()) + + def test_unmap_with_known_internal_node_doesnt_page(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 3 keys per LeafNode + chkmap._root_node.set_maximum_size(30) + chkmap.map(('aaa',), 'v') + chkmap.map(('aab',), 'v') + chkmap.map(('aac',), 'v') + chkmap.map(('abc',), 'v') + chkmap.map(('acd',), 'v') + self.assertEqualDiff("'' InternalNode None\n" + " 'aa' InternalNode None\n" + " 'aaa' LeafNode None\n" + " ('aaa',) 'v'\n" + " 'aab' LeafNode None\n" + " ('aab',) 'v'\n" + " 'aac' LeafNode None\n" + " ('aac',) 'v'\n" + " 'ab' LeafNode None\n" + " ('abc',) 'v'\n" + " 'ac' LeafNode None\n" + " ('acd',) 'v'\n", + chkmap._dump_tree()) + # Save everything to the map, and start over + chkmap = CHKMap(store, chkmap._save()) + # Mapping an 'aa' key loads the internal node, but should not map the + # 'ab' and 'ac' nodes + chkmap.map(('aad',), 'v') + self.assertIsInstance(chkmap._root_node._items['aa'], InternalNode) + self.assertIsInstance(chkmap._root_node._items['ab'], tuple) + self.assertIsInstance(chkmap._root_node._items['ac'], tuple) + # Unmapping 'acd' can notice that 'aa' is an InternalNode and not have + # to map in 'ab' + chkmap.unmap(('acd',)) + self.assertIsInstance(chkmap._root_node._items['aa'], InternalNode) + self.assertIsInstance(chkmap._root_node._items['ab'], tuple) + + def test_unmap_without_fitting_doesnt_page_in(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 2 keys per LeafNode + chkmap._root_node.set_maximum_size(20) + chkmap.map(('aaa',), 'v') + chkmap.map(('aab',), 'v') + self.assertEqualDiff("'' InternalNode None\n" + " 'aaa' LeafNode None\n" + " ('aaa',) 'v'\n" + " 'aab' LeafNode None\n" + " ('aab',) 'v'\n", + chkmap._dump_tree()) + # Save everything to the map, and start over + chkmap = CHKMap(store, chkmap._save()) + chkmap.map(('aac',), 'v') + chkmap.map(('aad',), 'v') + chkmap.map(('aae',), 'v') + chkmap.map(('aaf',), 'v') + # At this point, the previous nodes should not be paged in, but the + # newly added nodes would be + self.assertIsInstance(chkmap._root_node._items['aaa'], tuple) + self.assertIsInstance(chkmap._root_node._items['aab'], tuple) + self.assertIsInstance(chkmap._root_node._items['aac'], LeafNode) + self.assertIsInstance(chkmap._root_node._items['aad'], LeafNode) + self.assertIsInstance(chkmap._root_node._items['aae'], LeafNode) + self.assertIsInstance(chkmap._root_node._items['aaf'], LeafNode) + # Now unmapping one of the new nodes will use only the already-paged-in + # nodes to determine that we don't need to do more. + chkmap.unmap(('aaf',)) + self.assertIsInstance(chkmap._root_node._items['aaa'], tuple) + self.assertIsInstance(chkmap._root_node._items['aab'], tuple) + self.assertIsInstance(chkmap._root_node._items['aac'], LeafNode) + self.assertIsInstance(chkmap._root_node._items['aad'], LeafNode) + self.assertIsInstance(chkmap._root_node._items['aae'], LeafNode) + + def test_unmap_pages_in_if_necessary(self): + store = self.get_chk_bytes() + chkmap = CHKMap(store, None) + # Should fit 2 keys per LeafNode + chkmap._root_node.set_maximum_size(20) + chkmap.map(('aaa',), 'v') + chkmap.map(('aab',), 'v') + chkmap.map(('aac',), 'v') + self.assertEqualDiff("'' InternalNode None\n" + " 'aaa' LeafNode None\n" + " ('aaa',) 'v'\n" + " 'aab' LeafNode None\n" + " ('aab',) 'v'\n" + " 'aac' LeafNode None\n" + " ('aac',) 'v'\n", + chkmap._dump_tree()) + # Save everything to the map, and start over + chkmap = CHKMap(store, chkmap._save()) + chkmap.map(('aad',), 'v') + # At this point, the previous nodes should not be paged in, but the + # newly added node would be + self.assertIsInstance(chkmap._root_node._items['aaa'], tuple) + self.assertIsInstance(chkmap._root_node._items['aab'], tuple) + self.assertIsInstance(chkmap._root_node._items['aac'], tuple) + self.assertIsInstance(chkmap._root_node._items['aad'], LeafNode) + # Unmapping the new node will check the existing nodes to see if they + # would fit, and find out that they do not + chkmap.unmap(('aad',)) + self.assertIsInstance(chkmap._root_node._items['aaa'], LeafNode) + self.assertIsInstance(chkmap._root_node._items['aab'], LeafNode) + self.assertIsInstance(chkmap._root_node._items['aac'], LeafNode) def test_iter_changes_empty_ab(self): # Asking for changes between an empty dict to a dict with keys returns < at >< at > -506,7 +830,7 < at >< at > " ('aab',) 'value2'", " 'b' LeafNode sha1:67f15d1dfa451d388ed08ff17b4f9578ba010d01", " ('bbb',) 'value3'", - ]), chkmap._dump_tree()) + ""]), chkmap._dump_tree()) def test__dump_tree_in_progress(self): chkmap = self._get_map({("aaa",): "value1", ("aab",): "value2"}, < at >< at > -523,7 +847,7 < at >< at > " ('aab',) 'value2'", " 'b' LeafNode None", " ('bbb',) 'value3'", - ]), chkmap._dump_tree()) + ""]), chkmap._dump_tree()) class TestLeafNode(TestCaseWithStore): < at >< at > -859,10 +1183,8 < at >< at > # children. result = node.unmap(chkmap._store, ('k23',)) # check the pointed-at object within node - k2 should now point at the - # k22 leaf (which should not even have been paged in). - ptr = node._items['k2'] - self.assertIsInstance(ptr, tuple) - child = _deserialise(self.read_bytes(chkmap._store, ptr), ptr) + # k22 leaf (which has been paged in to see if we can collapse the tree) + child = node._items['k2'] self.assertIsInstance(child, LeafNode) self.assertEqual(1, len(child)) self.assertEqual({('k22',): 'bar'}, < at >< at > -1017,7 +1339,7 < at >< at > " 'aab' LeafNode sha1:10567a3bfcc764fb8d8d9edaa28c0934ada366c5\n" " ('aab',) 'new'\n" " 'c' LeafNode sha1:263208de2fce0a8f9db614c1ca39e8f6de8b3802\n" - " ('c',) 'common'", + " ('c',) 'common'\n", CHKMap(self.get_chk_bytes(), target)._dump_tree()) # The key for the internal aa node aa_key = ('sha1:2ce01860338a614b93883a5bbeb89920137ac7ef',) # Begin bundle IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWQeUcsEALFd/gHxURCJ5//// f////r////pgNQ3V8Dj7zZ532zX3nck1p1Ylezpe++9z5TrT7s7Pb5fe+et718Be9zrVasx9soO2 ovrB7MCgFFu4dffE+NAGgV7uLvPddWPXR3lzrlPHuzju8fPDd8ccPvS33dx8+HvTzXux1vs9OzbW aWtIkBVWmaMj13ZRree9y8N4xWhyATu954RUKYhoCYmTQwjTUYmmJiYJhNGqe9QU1P1T9U81NTE9 JtQJQgAEEETJoqej1MU8yp4o9I9T1GgPUGnqepo9QAABiAhIjTU2phMqek8nlQBmoyaeoyPUAAAM jQGgJNJIQlP0mmin6EaapvVDaepsJNGkHqNBo9QDE0aABpoESiAQARqYmQSZtNTaTAKegJsmSPU0 9U8FDamINMJEggEyGgmSGaimm1MmJiM0mRoBiAAaBpoNVkQMQqEYqKDg2oDCH2jGBqiBtJpRy6V6 PDE+Ad0LAYg0PDwl/F9LWWsPFfUIPo8e3nXyv8acdN7qFVH82jPb9wp0Ad1NbcZR8PDghrPz9NWb nGfk6sFf8PJFMgP4hmbrLfKYorqEM3zBk/xh1Sv/gvK+hWlieWE7yqrm/pcK+OeMxNefW8+rSHPx dP+UozvlFjnSNhe/L0gA2s6Ow0/9In1xLMDv0jaM78NzWAj61QRAjVi3sVWm0pmmT8Jr/GRd2r0c reEWqGYsNWtjNzNtmZOdu+CcEFGERAIkMQnyETuosTtx2Dyr6LL/qcY9c4/3Gchq1wNNmSdnbjrb slqRbacGtn/bsb1QjWH9cSLB9W5ULypIVhUzzoUu4uE0xnNJFVuYQk00DIDAIMIKFcAzUmx6c/8f 6jNuWKNxH98mz+n39R6r32r8H438gcaoUKsFQtowR/b879fk9X+Kf8+jAf2s27eXdjn5DN6ieaaU 2EjBIEGEuDqBxA1IRFsf2/GmMj/f4nRwdZq7t4KmmKGyygSDISftEF57YAbeKt22Olw3u+V42gKM tnOFeMqZdlCBCpJI6ILCxqZeyGSe9ahNqq0QcCgD6qINwAcfT8d18/3Nd9eiPZUot2bINijpz9qq KrzIakvgEAiIsEQooitSqh/7T4vEj5OIz/lcHR9FALCFIGebr7DL0y/0VqsACCpFhJ2keizbjQpJ nfmyyM13ySewgesgMIpFigqgLERYsAWKoKsAVQFm/ygd6B87IE+Tvvu7+4g74IPEg9/COzbHpt9R GQGQ8qDvu9L+lsVeGpCcNkkiXQvO6llkM11S9xrq5Ry7KWcz88PnSJR4ROzqHXONjqTehVZw6exN nqmnxJw9st9vU79YybIGCBLDZWyxcyMiInswAKEoaQxqSritn1uaqrLBhF4qQLVuSKIcEIhvPc79 ooPkFFe9gwo0rE3K+0mAc5pSfZXlPI+VA5+hj3vx6u71iBCcD4XfDsfVFuCO5Te3mQHZ2pyCw92/ KJSo2UiG0KmWFpOainyArPQ/QIa8LK/HkfIIxFaNdOKPTWxkOSYH+6M40cHdijcIlrnvR8ojRqbU fOqFFdERs2W42BYQpmeLVyldDPTny9W4PbsPXtCOpPXpFqM0lquZiM5X2qzijIHAZEdYZO+qdN2o stpwW0NsCdJrndItATwHX4sPYh3oVQxEZREbc24NMYQbgzo5xzoTeRkXmi51mANCVlKK02RXFXvI E+6XgqI+CWnbu1bK9DdU5WUKI4KCYniMOzyw5TJ1WlGfvIkfgx36TI80m9WG8gAR0O/dwRjg08tz zJBQTbbjG+ImOC5V92+CM1bkYQRhblp/6obwoTDfvFcsU6PXxUbc0XS4c0wYYhsphWihdjBNJR2Q rkTC7wHJlW8VxzfWIPMQZ5u1TvnDUZiKUdbvH/ZcbFMhzOzt5KA+nSNRzCwYuip5Myc857zXNxyo 20wKOfe8HoshualEOww7joIOzVHFlbQVu24GL7N9iFzQ5omN+nyYM54z0THR5T1Td4j3M1G2CCog 45df0OqmbkyEQ88uO1EBSuhuW6fpPYaZPbO+VkkQ1WnIkttQ8KT5MuPCKvodedzrhpgXWuNiBpQk MniSEysIcCo6b54YXUvLELqIOV9jwTHkQ0zR0mpGizm1eu5IoqdZV4hNdpUbmjTYr3ul8LVbithZ wZM1c/BC3TJl8jHNqRDiDt5s5oqKP6YePm397edztrF5mejL8y1cGzZylYbLVR69xAUvREGidHSO z4JWHr7PtpfIgtGSjO6l2Ocitsstgg3MbGxsbGxsbGxsbGwdSzHz2ZsuJ3LRowsukTLIIL5e44E7 uogl20xTRCMHvN0Q4GcPLw9fTbpanLkq9GR5887DlwyldJknHoXS3AJwGn9Mjhzywrrw4zKcNOaR YT0BA826+n03vfBUPLWVbaLbtrXZoviOh3q6DO6u3o0CbO7tPBV2Fnoi00Yi9lhH2diFs0Rp2alc IOdMtIVw+eWm9VMLw9eBBLlpcrmV3DW6YZ4GwfMzmi0wxDqnYbk2Pbu5aWWGy+GowN7NspC0txDv w7su0R6wg7JjNqKc4jvH+FBLZQzsFVBwhuO282c7f2aTjFBnBvGe4Ui+Mh7M9aze4Qyzm763DR2P i2ZGG51zwLdBJnVxzp9/S+kMZlG+GGFCwtN1eLd5TI5EigHwEA/nFDh4vH3pGHJzEoTdHBNRDqz3 ipj6Oubh7OrL+yNtG0JSBYELtyDTbG2xs/RX1nV971aKPa6u96O10z3ab7M7XuhuZeZwxjW+2+/R sG6/LzC0d9sm/NuIX45ljHbjdoPnxJ1+VPw9MkH9Kp2SgvOIeZIEhLO2CJu4+PhTZo4XbeHC7aj3 kdAaEYxFA2g/D/YpvK0oUpSFWLaLbhUwP70Nft7GATGuxS7GWONLp1Uwhjj+GE3hPD9j74ecBBIh QPXIEm+34ylpYbG7kXYLzUB+KL1UoJSNEz9WGOS9OxxFFTAUiIMpbWwasMtEFKfNS3UDDWuVxfzX SGSs7uwJukF+J7mJETDuvLrML530JFJSmoUOwX1mIQGghSBgYZFUsJMBlxshgIMURpEqtCkokZSj 4dKtBAa5hc3hPxVwEjIFGSUIjbChGQQUTxYQgwO1e8wokGSYwR/hoUIzpCaMPNASE3poz7e+xRqu tkd+dOsHUEQwSFwIzoO1qMPjOmoXWRL3YCxoCZT2JEntICkiUjjLn3UWeaFIkUOLBgfUzywOUDcN kiigZoxrsb3dwdROW4cGzAvFpWptAKvDVcrwUPJnKVOu7yiLffboa97TqqjdEC8b6TKhSlKghPWX 04sz0hSEOuUIi6IGjkp0t3i793uSXLy/Lw4x4oDH318PkJiW9UYAZoa3cR7xhCRBCFOcd98o0IOU 7py4ur0mKS61HJ6WlJ4R5RWRNeoIq9+cqVcKWTNYVNYmlXG45NHprZL5tBB9QqHYYCofNUPNUM6O yJhgJtIwANIlIFwENiKQEFMSPtE9ohoR6VWuBmgjUA0I8yPNqAAewjEePVmAJgAefMAQEG4g0O5c NYHZC4152qm9r4LxSZC6PQ7FQvaxUcarTJ3103s10+yhTjUc5Qha79+GT3bNo/CobhzAJaEcZWBK NsI25UJHLJrNOtTC5dxIAdwAE/CcYQ243DQnGhdBATTtEmScmFGQtSERh/jczqv5+PXn5buHJNc4 Ovrn9VzDXiRtLbLaW2W0tpbS2y2y2y2lt/khPX9eAQpln38ccY/F5vMeG4aO3HtEgK0liPz1Ixl0 BBlFbId+N1Lqi5QrASyKkghfFLILILGcUsQrIPzI7SokJkGDkHBBzAjJEADKuSPizSgYghnzm8re IYI2FAVHWePukxIAgNBBxJdh44EDwKED76EzRLBZ0wCJF44YQEQrvUQ3FIFKIhK4xMeThEQoMIJZ CkakjFY0jRlEKACgQKCoBF4B7LRQRokSQqbNIYe8uglAIjwHiEzYYvowosdPELXY0XIYQ0UlgERB K6MZMkTWiB8PuoPimwhEsQIG6Iib88/uJETBsbWOSF7lhjPA0gKHTmh6q9SuTNyBUQrye2JJNiho +pBP59kE6yR0o0/68SOxHSj2Ul3uRzCVsTjij/9FXN/kwqehOtlkI5pbs+jybzLm2iG/DZ03SE7Z ezT0oT4YQOr9U0K8EFJ8RCIzoByTJKVJjOIixJQvcoIeJLrvc6Hc5SOrTwhNRARCiCR9qCWMmR9U dtmE75vKvmVXfeUIGoyjS062wthB7JqxNt1ZDWQzksbSqNyrtRaXihQ8uVofjUiOOLVMDWcUOTnM ZsgCrGICoJMQVBbIIQrxk6HTIuuruxKGFTUvt2zuAG0JtAbFyh+f+2kCVE2ogjuw4qbxIaMcEckE kgmVREUt2Xh7vN+dbjbUZtORPhyJHehWTNTdLgX9BkGyR1GkVMHczA5LD4GneiCKWOyHYgT+ip82 dFCtC7SbuMPy+bcj/Sy2H4xA18DbfvP3EkD0s4PaXXux9nmgHlK7gS1vN8V8O021lCMYU0BbZxdV 8Gbhq5QbQDXhbfLjNb8YZ7M4qLRMSKCk7mkRYMa6jT1mSsQt4HUoI3baioZ5IVwhbykISo0JdT/M 8eFBCNABghTGz0UQoZiZmRDuc5aoIC04qdYPjY2MYIHqFsKKcyeKPK1YKlUEmjKUeNomKSeo6EH4 HOR5wQOhZNGCsTCkiN6ly6nXiAAxMqgm4g1JEKxUwlPDUhF4eT+fT6t7h18GBirruvLEYC/cnKNg 4eORse1GuT49kqYjHkcpj7IvJFhaHebDhiheuBX2vb4U46dF5jeDXfiB4SdsP9r5hJGjB1Fh5sSh e/uCeMNyOMiFMzQSWHNFnDOLZgpJER4XOUEfBR87fEvUecoJ0rf2kggxJwkrIzyIBKpSnpQrss6I VxBMmB41apU9CPKDxHESqDk8hZCpc1mEx56elSAGx2QdgXF90EVFFlngwVG3IjxU2QS9S5LepAAj gmMhrXQZvi9VYz51qBwinmKvBiZE4Jgd+hwRlY7khUY8jpgZPUASO5WXc2gblD9kA/UgmfcAeXHG 9u2u2IbhF8ndmZ1XQIK6cJtBmWsAKq4I0wy50a5uhzmUyEhiq20MeRqEggkYlTGLEJXt1CXMmSmW RwtQpeS65ASmIHJt3JwN9FXUJoiYHYBAigkOwoIGxQ5AqSbQtEgohau82GGAjkUKbsx3X6UvozbH kptUEChCIuuEEcMbGTIx8J7Gd+rPOsDG1jqXJcnAbb1XfoIZLu4IZHkpuqWQQSSIiX/39vpf/LtC Zc+YQx1wTdrJspA8DocyOtRXGx4FmBjr41NbjleR2QtELxQvsAH4ELpeB5c7Ls3o1zizusrHB+Uu AC11KDwGmcZgLo6QQMKETyw4KIKADAJ2xjfJxchMBnhGAjynm6GkwDMy6FbaAP2oTeGTAEpKmj1O be0BN4EiIs0Evd5OMYunwYKldiIp42/W9pwYeZOGVAKQOo4AwxeZFSH6EEEma2kaK36lSZktA6EO HoJzHYBjbckdC9EAo+ew44C4w+xCPF4YkTNGicS7yhuRH2oi7bFfTx/RlME6GiZQiBXodfDTjW5Y UcHNJHAhrzlBUMU5OGuSkRahaw3S6eIGVHvT+PUquz3N25Gkt/LRDlOOtJVjuiqAuc3fXg5bhFNZ +NiFUC1CipkXswqhXl50PIUxKiKHW8fXrsA4pY3MDFRXmcOwgjYitRBX6UhtEnIBSSojwNRI0LkZ zBXFEE8RnkcbkijsQybXIOkNc0M49nUkYMnkOoS2MDohsYNFaHIqcb5N99hYjiJqcTR6QImCmFyb DXWJa0TRsKYAD3++vrvkdMqYOg8NiwrCnydCWihRx32FLkjByXM1MDDgIkhQyFiUB5DrAm4wT2RE /z/NuCD2OftelC3elNGuWXQ14V4IrRUEUnK20lZ1tcoG6R5WqyWMAQG8JPp5oPIj0E79zk9PZQiX wpkZap5ZTck2YWs8eZ3QD83t2KkonmXGOrim8rGZ+sVkEjMQ9kbFiUDbboOF4NjkeUGNjdaiDcLA 2M8froUHnPatzE9D0OgqIlKKsBSVi9SI/SjqH2aQERgx9f8VCRySQSQ2Te5A8TR3MDirG46hMcQK izkpJvIdciOIjuhEocEojETJaIBjhBMILBD3exBJY1Uxfsu98lHc1V76YAaUfprcAhL0pbglVFuo pQ8UErd9F8W0CIgkLzOpubLuBRBmPRqRyx6aIW1E9oBmaxjbOJk0K8RIcLvEhHLQe4YcALztIi98 X87kGujSGKFsFTcmLuSKbrw8sZHMPCiwuRQS0bmhZVkRHlmEEdj8fn/d+HH8fBudcdTngJgOM8jJ ubkKDzs8H8tSYdu0HAvLzuJDJGNO6CWZnkWGJcM937qH2AGfTLR7QDJjAMSNnMQs35F7WzIz4KKL 0hwRojEummXJKTR0zEOgQ+BpiDQIHaJGQRivMLeBnBogwyCUu4y6riMCUDDqO154hUh45NDZkRy0 nMaKEoj6CsZKFPiAThom4GRijhxwS3Man9HJkiRTTyNY8DnGIIbRHWLvJb3IlDuASHRRlnQmMb2N jg2HqrPJzOTn9XyYn9ny7yEN+dj4sdyqbiOqF27bNShxOZwOZ0yg2kHQzK29D6bCpadBh5R6VLMb 8qRPgAfX+Wv5HgHZBPcAfcCH3n1daGNZafN/no7U1zuGCD+OffWEfhCXUCrtR4LWAPMI7+VEYq/D egsIFtir0CLdpEek6g+8rvrWSQR60YoJ/wjBHxl4Af3iUgvhjxo7cEYFAkGZhGg1Vh6YRJYTykLS FIUg+DFiMWIxYsWIxGIxRZDAEDs5IT2Qniq5UbqMkaEYiNVGhIoyBqIwqAAOhJvBH/dvl7EdSNGj e77VbZl9ZyNG/80f6I/+0I47QjomRyeGfag4ehGXuuXb88r+mHwmgfOdHKamzI+Gnsd01oskUeOA HtlYBAGEJriP9hHp1atrDZ1Ybnw/nVs1kZuQgLPm80f88Wwjd9wQ0IzQjIltV8hGJ7/DMjr3UcqP bql6Pch09NVXlx2snv8kdSMEeDhR4JI6yPryiPVwI+6xxI6kb1zb1aFHeebRvX0frT9pclWxx9G3 D9syNb50aNKMPP1o+kR99hUJ1Q4r3TXpvRF55vQjw+CBvo2aM31xyI7HHCn4WFQ4hH6Wv63eLHZH eh3oMBH+IZ4BUQd/X8e7xrfH3TwR2f5qd9yfvRwSv0/Po8u588tPYged6W/Wq7u3GtGXj319SGQF M0BGxAjXiYo08Gi3zdX9eeuj4x6MvKgelCerBHFe2a3Ni+XtEfY8iN8Rw3UYo9c+/j26P6V+rq6e XW1lOwIEkYDFUWQYiRViCgp5oB9giJuwZvPTuddu/IC+hcvYdHX2VLJ7eCNzv5zIqHB3U7O5ZRrK hTT5edSTGPPL39eO9j9iMdG304/RqsYt8Rij8c+Hlb/hpIlPTb5Plmja5EcPhXBHyRwI+fnnmcWP fkJ8YX/COzxYsPD750JxHRw48uaezAPrCPwtybW9x18PX3Y+YR7e717fHK0ZfHW8PZzXLPdT0e7y oRtVO3Nt3U3vR5S2bO18t6vxEOnehT39V76cPP3bXsRtI1c82BHsz9W3u9We5Vp3PKa3v7vXU5JV ogO5B/TLRE+X9M/IDBkiKxYjEYpBkgWv0jEkfirHd1E6p5MvuUn/BDafMSFa2EwSECc+gkqBR/DM JTEHnZYTBiKiJpIIAgCMaSlogXQPgc1G4Rftx9SaYtpbbpRAztjlayRPRzDkaWBBZ/bZ2Ak0dFZF EGRDYvxcsi9YTsgbB28G8JDqhe3lFVVEVgqKj/ZrpNTCBWkDt/GHTDliicS+0ErFkIESBbIGDXKd ZsfjgDsih1wL0yg92QEM/n3sDpOcTWMKr2nxMpn/5onBTJ5Clj3FgdZHjPU9R836SRGczYye9SiQ 1r8CxuXwq2MmjgdY0EDaJx6jYkkBiAfW7+CZ8oKAxoCRsUIJDDUxNuO4oa6wWiQSlACk3BZEYERM CCaiPkFlgESKBIzWUtMqbRiqkqFELv7FhAOqqMSbs4cY5CNtQnNszWrBTKARSO3dC0hoaGstwxl2 lKJEAKtgWsVqKEHU5kAt9FAKNS81N5KwwsAQWWtzShm778gqCMdAiwHrRyK3kajYhDd2EhR9r+YB 1fLyEGX0aHlRmVFaI1AD7F51vjIH8p4IffMEOTKq+uHrUD322xzqukRuNskTUDSFspYgFyNr+0c8 R/Yg3Jev0I4tnoZ3oqE/q1SV9qQkokEkSBiEkD4gCI1Qio1YuPpKWQLkfSr1dmI9n/IACyRkGYd0 BJkqIQKHqHiUnnDhjNWfF7KNhAAH80dZ/Y/vecQMgAyDhRymNd+atkpBCkBssEeHA1lCva2+fnsI jVcQ6mC72ZRc5twVLaf0Qn8eNHyZu/4qc4pbEkkRkR9InFyHs7DoeqlKhTTKHOxe2g1zmMXsVjVJ iqb57ZAQHlT7ft1sQF2MkY7JA3JET8XEwNaJkxu5qBQ7EBcj9juxGQpkcSPriBm5M/DRQsR1+mGO m3j47FUgXiaWE8PzeO8w6Ekls30UGyjIwwsZAPl2GlElgeY6jGMjdM54FhYczQQQVQu86EzcetqI llyLscNDyFet7uWsGf2gYTYSQvFCs2ttwBF/hB6xJLhMPZzAi5O2weXaFCQiNtMGl8/HfpYd1hzq d51QqrmdShImcTLA75YlcFIkShIUnTBIgUHFyhRi4QLYHFBxMoKFx/XyO6FwMXImlFwViVhkDyIy EIDG/50ern81+t2pwLzKDmkHAEcxh73hAu7utb3HCB6ncgeBg8/lhyeJY6nQqdQtYuYLDxYGV6fS zDCANAuVLEz2+e1e1joffyaueJnodDyQT14Oig2EJfRD1sDuQKAh+dP8hCa0Qp6SFA95CkKAfohO Am3EUFH/uIyFITCGS0hBOfq8x2xy0lQoNhLD1YQr4odh7OgBEhJOwAB85IGi9FCqmFGIdLu2Bc5U sNiEH3vFesowGTTlOZbT7dY7KlDWCiCiYQ3OLs2QXWt6TNVUdttGCaU2akx2YUDK8cvmSiv7O4nO KHGKrxKM50zDIbnHc0efx4vyKfPCGtH5D3H5T4yri4emDTj5JuJONFmHFB3hEeXNaeM1yA9sGA7Z gcXTR2ACJg0c1HE/jHoCIyCWOc6KSLCCVOhMmWMEDbwAU+PkXXGGOwYcN8tF2bXbxaL90mMBVKaa hsHK0wK8DLtGZTFRnUEg1YJCy6QPiWxJPQ5ltfnQ3AOnZs2nuBDQbjoDugHA2h2pySa5pBSy+LsN OmhwMk6vrVfBreeaSEnj4I6FdJEYHiLxL85TafWAbb1D6bx39PyEnL2pZ9fXN0KznqD1JUMff59q UZlBh5taQGMP66iFl3OAQt3kGSgFEEEjNlboAS3PT6syTWcC4INW5LsxZL5EBzhlgUSbprSogNmU KE5VqfW8n23VBgkqkM3F0+yUZUvC0iUGcF7TyEFUNLHEmSMYJEe4ct9Bs8xtDI+MnM5l1VMSfd7F 3vXD0s+NRh4RwsVstQvdz0pttBw/6bQJwYgqVwyb3vSylDZp6eWo2y/ZsswS2h3+9nXihDC6GkdU GEGgOsRg/Mwz1nIsNNCN7kgiLqkwdMaVTYNJwQPLY015NGKa0kx3RZqBVembuI4yakw5cWHNOWt7 5dHXpdPHMLFAiHROTMMSBnYqiqKqqKKKvznjNeq9e8+A5x28fcQ4j+/D4dPbCrqSWRWlaVsXPeTP R2wh8ZLbqUGFOCnzAVS0SZ5EcuXbnfvy0WOixjNyv6N1rcfSENGjJsOKClDgiPHjjwBA9gx85L1q HckYG8PA2ALDJHUXlbsCnJYaTmdseblR7Ah2VXmEeBlAOdGCSQCQiwhnR3cMeSCJY5aApBDJRbqq ZylwVQtNZwNmw7BznyZntwxg/L8tQPthIEiwBCxKcG5KleU5jkOwbUudR69goLxDAWEGhKoIZdO1 ZkUOvjZT1NGThrlcrrVwrMkRf595JdOxcSFckjtgaWS5Ggizf61oeDRehgprkLJCc1BZmJ0Ds0AH ajlAAbps3EccAAG6qAXSpU6/kavM25y2hmgXizyT7MEEoPQeEFo2NrZCisNqVBTC2rZYR/GiJqVx vsIjftpVE5QuyVC6xQTugaDkhfYeBHNTB8ua3giODrsRafT3wK72Ep8G2uA+KIP9JYDuoCt1wLMl auRVSeMZD70ZgULFkScRkwEVBBc3RosaXHGA99pZNUEXKCkQBLbbKGbhGNlt1dLdTFuyEcSlsEWq PzVwMZmSKUVnJkuZwssvQNWzksI8Vo0EgKuYSsUcbB0VYKv1lSw76QigH36EdnSZw3ROruc/mm/N yca0yjcKZNSkKMO+2YDdQlZCw2wuswfp+nQcSR6SG6FiChFiIQQUPaWLnwLjh3KCe8le9NEiZz8/ vPqfUkKFlPkukipAK3YaHnpPQcM4vbp9X1UkRL8EzdXlcmDRvqcYsVJ/ft/jcBA7/o/w+zyKl8bn im50Dd3VJHCPOvgTd9u06HYwVmXcdSdTzFIGhgmeBY7UQRYl0kq2ScrEp/TfVl7YF5GQAXbJq4/7 iHwEOIg9H4YAXvaBDEP2PbAHwArNer1eoiUohuGok8K1SDyI9mlGY3ogWZfHgEY9l0EMgBuJRlY/ TRK6ghBP3059ZH6I83vVs+j7ffRY9U+fT4XXjRl8gqkSch9H0A0+VMOZuvt5OAtUZ28EdgBqvbZg j/lHBPZpu9ftR9CNT6fV7qcEbaMkjKNQREJuAAFCF5QPGSFr9Oz9HPneX2pCBkM0QN61QCH8oAeO EjUQc5ETquW+QZA6b7H3ARgrCcZmZLKQEmwQbIZASBGPoR0zAoO1otooO+AdYIU3Ra8EuEgUArSH sRoV+5M4HbV6Wk9AKkS3l5OZH91bkbPDlt7q+IS3TWHoRuc7aSCyGsbr9xYjxakfKohycQaChuDm 40dIjd5/PRZT0EgknL5hsnObhn+sT+6S+AANTDx4bUL4IM+PJjCRWwQkBC8J5+IStyLeQw8IJw2A t0LlVGJGu/IRbQztjQigyzVbR89v5uCtDJNYJa2XkXPKg+2hMUwlZlwiyY7WMZIPDqQghwBg8w7o h8OTH60dyNqOCNhEYRBYwEZpJome3TAIQOeUZCMYJIw0pY57FwEZHU+Ms87Egh73QwyS0nmnnzWg pKwpHlsz8Sb88W+Cv701YAft2ToRqnqhzcOvNsmzbErNqNKe7sdU7WTcLACHn+3GwCRFMTd7QCiP fb4e2juqo7XrAI1dP+ker9HF9QjIIMpBGQQZZCmQZmBcBmZCmQZmBcBmYFwGZgXAZmQpkGZkKZBm ZCmAzMC4Ea1ad4AqOSGuL61fvptIRJB6oBQgJxiayECAQ861Wb7fDT2ZZptGLdrTo7CqFCFlJ6vT /AeGx3e0YRPdfcksYip/tmph8Ek/Bou6HZRQIvRtfdHIjw38AIYEVaRrYLuXFKSQBC7t+/oR4VJA kHsQaDGQjBkTyfVUHrgWXSAAP7kF8CP4/jddAVx+g2WI6ybt2ob5CJ8dCoMLBIPgSkBDmy/VMIUg bQpUKiEMMIQIyMX7qUYJbHaqr9veRoBqEt39y9zJrTalW5BsqUR+GNbGEDbQG1SKGZ8/khPysNtS EVkOZD3qzxoWK2YAy0DnszfSpEYbv1RjWUbJFQNaEt7tMJJMQ2+Sr0+vZHb6ZJr4coTpngVhMkKj ZAiUPwRzo9y3ZqROPnId8HNqrWta06wCpFJAUFgohla0uChZoJqfGUJIJIRASTYX6Ptv2ahcDcGr 8LZu133u8n/NL7qF6WwYbQWF4BkQp9s079sSkhOd3Ktr7QAG2qPmp6UfajejhEfFx5alsMUYdN4h ULT9YncwDPoRih9Crh481xaARRO8k69Nggn24T8W/YeveZhZW3UJxkHNETGo0onZCUrDLLowMxdG aIM046cpShKyhVr/hVSB3IsjUbRZCURe/nRnV7/4cP06kdOKMirqAAf8UbuXqw57QE176C5DWSAB xqs7SPGIFu8oj2IgUL6A0Ggi2C/XRKSKhxxHVBU7ua0IPdjRFBwnxJVT44yJO/9VU6WJT8D9MBXv owU0AR/bVtLGNiOlPp8nxFyuhyRv7HApayijok7ABnTD9eAeFSqbq8cKqcRRjQCipIIWon0Pm22Q 34w0mh1qhraU/wQrIJGaaGSuiplGBHbDyJIfxANl+6FE3HBodaDCZn5U2QB0WEPcM2DFhuskjgdd XIEVCQESfgjM8VDBbp8/Nkfn8ZT7LF/nE/OoUJpkmDCsn2kNCshlsEGhN5FB4wNXIAUgfnkmALO7 soWUrvUGMGElKqiQSEPuxLbQba21GIAqyMBIsWMhEQgCgsBVhruDwh7mAtdQIfqY7/Ij7QAH9MtZ tejP8h004adwhrIgA8hIkUOhH6i+hTQcAkiUqohxxEbeBNCDpJBK7KKP6xiQRPqjERIhiUrK0Nj+ UWw1AD2+MlBR4/jAwLIcTIDKP4UuaiLilTNusEiSRRkL96xHBGKvGjxk0VS5UK1C/BHQaTOjRFL9 nuu0ttPdwDDWyTqLKkKIWzzm2ocwTnMIX6+3QfU2dm3VaW0qfuJ3eSE9/zhPawUV508jAIiSeiSA JSKKHvQkWAHg/W2qNyPNRC6ygsgDYoRAy1nQR20ofdBDej7kaoLdYYKvHv5wM8NTcJoK7DfARoyX HJvUzGd77QjsheeeHihYAYAGS+MiAoIwYbACPXxo/TwOkjWe+i8iPIIxHMfVEdRsOu89O0UecA2H x/WKe+HpRsK4tj78Hw6vX8a4bavNam5eJZ4AmmPgfRCfk9mielkFiCsYMQfKlFRTwsoSMCVkRiZJ zbgDiR60bUcuHyfLYOOYRYHTXPzI+IA8m49SNqADjGMgfdKZ4UIMiMvMzPBq6iSGhgWEU9fKfX1J e0Oz6ah4J7OvPeqURsOIXTVD2WdsS99VwhGKCWOu7CpT55H1+R/YsPUXaJGYHlR/IRqIeZGKhVHk 06158bjpsTueUAwWv7xIUAWLN4BEBIFo/FwP7psc++WwBoFHG1g9bosJfggTSjRowESjbRFpBHu5 z1yE32paxIk4KQlgcvss3f0a/jcr4MuV5Rilj7fr1kHFFh0QeiYDlkJiHGELcMBwDPUogSRAhryw +HasRjwxOXl9NtEQ0V6s4uWLeRzSDlgIYEYYDCAG075KTpBNmG6FPO+yZYLBHj9EqUtaGAjBGjl+ WBBx7n58I0gNZgUniSbBvSBcrE13+Jnl+1KM+XaXSaEqXzuTcmXLOAD4EiiB973Wo5RajUJkSRSS e+M0Q+6BSRlZMfouKoUtgo8VeYAozAqNVVijWVbhHF7RYjXGI5KpsRigm+59wBiFzIRUkUPu8yPY QWxw2tHhBZF4f59KNtgZBALYPpr/XiLgWPqkgoEdjRnONmglIUIol6EEdqAZBfW2pwEXlEpoDAoW SVlnpyw49B0eWsUPb0nXrxrXXTYiQ5m+RBQFdcSPpMMOGs0UHUJMUaILl4EAJwZRtAZYucO9qBZr BqcZQgvb+NEGHsyA9FQzFyuhhJRYLF7bQWQFBFXpvy0NnJZmwEwDFLK2RmFDiR6P6o/Z82bFH8gD nAMuhHoR0VEffxI7khJGK/ZEQ8e/cb1KANxiIULpYrpRSw1NYIUYIWAFAttqqtEvb2EORM2CDdvI w4IIkzHPlEGSBRMkkDTZb1A3cz8wE6gPpAgFFTCAhBHrR8xxPB8fZcbSnAHuQIoE9jCE3O6cXep4 9tffQLEdbfBq/CLbLQM1+4JiI0tRfCcfFZoC5x22Us+Lx17VpmCSMBCRJFZAyLQggNVQrutTktsY E6VCN1ShTDfGRc9RHRWiC/EAaaoyaEMcVKWIzEQyAZijfrijCi7oR/Xz9dvNhijXxU/99QLjQ17Q ZECqygQAYQAhWwgh4VKpVxghE0h90d7dsqBbJAvms6LKAmmrunn2qGYA5fRvoUGJ0Ug08vn8vnLK T/mBwUmj7JWMRhBD7PkDj4F5zTvBCRhvDgS3fBPppSQjJFO920bkfIBtREmSPaRv738bdXxdHViP igGZIjz3ghu1wkWQJAjFyDRiHcVCgUkkHAmIxC5sYlAVZo5leAIfoAXXBGMjEKd9IAfP5/zu9++3 XzXXIOv5E10BPNFXuQ7uv2akfnRtOYg+mB3oVnTDoIJvR8tBqT0XFCu0zI6DUA2on4vn3QCbpgh3 QD77zmgp+pgtdOfnyo0AGFHTvgHy4uzv/6q/cELWmsnRN4AUzAk5+8AtQRA19KFIAfAfzmene3s9 eLuaQ+0wpt+ZE+4F4U9mJCKVCCDwZB0bjompnEJTTjTgpVHV58/4iA2Mci1qGATBBeTsmWVmhOzs g1448bUKvteqOv2aC7QjR5VdN5GE7aPdor1camOALxg5Ia8cjep3gDs/39uXDk0i05ACKBiEUPYA Q1sN+sjvo3hPrLbAMP/3/4u5IpwoSAPKOWCA

Current details for split-inventory work

John Arbash Meinel — 2008-12-03T02:39:44

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Now that I've done some updates to the CHKMap.map/unmap functions to help ensure canonical form, I have been able to complete a conversion of bzr.dev into --development4 format. It took around 40min. I didn't profile it, but I would guess a most of time was spent deserializing each Inventory from the source. Anyway, I thought some people might be interested in the results. $ wbzr repository-details bzr.dev/ Commits: 20916 Raw % Compressed % Objects Revisions: 11298 KiB 0% 8170 KiB 9% 20916 Inventories: 3097450 KiB 61% 38976 KiB 45% 20916 Texts: 1937574 KiB 38% 35677 KiB 41% 44248 Signatures: 3401 KiB 0% 3205 KiB 3% 9031 Total: 5049725 KiB 100% 86029 KiB 100% 95111 $ time wbzr repository-details d4 Commits: 20916 Raw % Compressed % Objects Revisions: 11300 KiB 0% 8170 KiB 5% 20916 Inventories: 159899 KiB 7% 89580 KiB 65% 170075 Texts: 1937574 KiB 91% 35677 KiB 26% 44248 Signatures: 3401 KiB 0% 3205 KiB 2% 9031 Total: 2112176 KiB 100% 136633 KiB 100% 244270 Extra Info: count total avg stddev min max internal node refs 77927 1240750 15 13.2 2 35 internal p_id refs 4743 51586 10 9.1 2 23 inv depth 64955 197639 3 1.4 1 8 leaf node items 64955 376262 5 5.3 1 18 leaf p_id items 1534 17771 11 11.0 1 44 p_id depth 1534 5595 3 2.6 1 7 So the raw size of the chk inventory is very impressive, dropping from 3GB down to 160MB. Some other bits to extract from this info: 1) On average we have 2.1 files changed for every revision. (44k texts, 21k revisions.) 2) These 2.1 changed files trigger an average of 8.1 inventory pages to change. (1 is the inv info, the other 7.1 are chk pages.) This is also borne out by the average inv depth of 3 and the p_id depth of 3. So the changes trigger 3 file_id=>entry pages to be updated, and 3 p_id,basename => file_id pages to be updated. I'm not 100% sure about those details, though, as I haven't probed deeply. I'll note that we do get all the way to a depth of 8, which means a single change to that leaf must create 8 new pages. Also, I did a quick hack to try to enable knit-delta compression. Just by using the fact that we generally use "apply-delta" which means we have an existing key, and we modify the page generating a new key. Anyway, with the quick hack in place the "Compressed" size of the inventory is cut in half. (1.8M => 1.0MB.) If I hacked it a bit more to make the actual inventory records multi-line, then I see another ~10% reduction (948kB). If that stayed consistent, it would drop the bzr.dev pages down from 90MB down to 45MB. Which would change it from 65% to 48.8%, which is actually very close to the current pack-0.92 size of 45%. (It would be 45MB compressed, rather than 39MB compressed.) I think we could still do better, but I have the feeling that if we add in some of: a) Hashed prefixes (giving us denser InternalNodes, and much shallower trees) b) A layout that splits the inventory records across multiple "lines", to allow knit deltas to compress away the bits that don't change much (like file_id and parent_id). c) A delta compressor that can compress less-than-line (xdelta) d) multi-parent compression, this could even be mixed with (c) e) common-prefix extraction (this is a big win for the parent_id,basename => file_id map, I'm not as sure about the file_id=>entry maps). John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkk18XAACgkQJdeBCYSNAAObUwCdF/8Pka2TTEzze1Exd0PUKfde ud8AoJAypYlsjGlqY4gDS2LFj3WFzQWK =/3dD -----END PGP SIGNATURE-----

svn-import performance analysis

Jelmer Vernooij — 2008-12-02T18:48:19

I've done some work analysing the critical points in importing from Subversion using bzr-svn 0.5. The main culprits appear to be: * Inventory.copy() (33%) * Repository.add_revision() (49%) (most of which is spent serialising the inventory) While fetching a revision delta, bzr-svn makes a copy of the inventory of the parent revision and applies the changes it sees to that. It needs the previous inventory as well to figure out renames and the like in roundtripped revisions. I'm at a bit of a loss as to how I can optimize this further. Does anybody have any ideas? Also, I would expect CHK inventories to be of help here - is that correct? Cheers, Jelmer

[PATCH] doc/en/user-guide/undoing_mistakes.txt: position of --opions

Jari Aalto — 2008-12-02T16:30:06

Few fixes: 2008-11-26 Jari Aalto cante.net> * undoing_mistakes.txt (Correcting a tag): Move option --force after 'tag' command. (Clearing a tag): Move option --delete after 'tag' command. doc/en/user-guide/undoing_mistakes.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/en/user-guide/undoing_mistakes.txt b/doc/en/user-guide/undoing_mistakes.txt index b3e7e4c..142e00e 100644 --- a/doc/en/user-guide/undoing_mistakes.txt +++ b/doc/en/user-guide/undoing_mistakes.txt < at >< at > -149,7 +149,7 < at >< at > the ``tag`` command to redefine it. For example:: bzr tag 2.0-beta-1 (oops, we're not yet ready for that) (make more commits to include more fixes) - bzr tag 2.0-beta-1 --force + bzr tag --force 2.0-beta-1 Clearing a tag -------------- < at >< at > -159,5 +159,5 < at >< at > If you have defined a tag and no longer want it defined, use the bzr tag 2.0-beta-4 (oops, we're not releasing a 4th beta) - bzr tag 2.0-beta-4 --delete + bzr tag --delete 2.0-beta-4

en/user-guide/svn_plugin.txt: Adjust exmaples to use same (comment)noation

Jari Aalto — 2008-12-02T16:27:31

This patch unifies few examples to use same notation as elsewhere int he documentation: (comment) Jari doc/en/user-guide/svn_plugin.txt | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/en/user-guide/svn_plugin.txt b/doc/en/user-guide/svn_plugin.txt index bce0c1c..02bd467 100644 --- a/doc/en/user-guide/svn_plugin.txt +++ b/doc/en/user-guide/svn_plugin.txt < at >< at > -40,9 +40,9 < at >< at > default repository format. Next, create a feature branch and hack away:: bzr branch beagle-trunk beagle-feature1 cd beagle-feature1 - *changes* + (hack, hack, hack) bzr commit -m "blah blah blah" - *changes* + (hack, hack, hack) bzr commit -m "blah blah blah" When the feature is cooked, refresh your trunk mirror and merge < at >< at > -78,9 +78,9 < at >< at > Here's the recipe from above updated to use a central Bazaar mirror:: bzr branch bzr+ssh://bzr.gnome.org/beagle.bzr/trunk beagle-trunk bzr branch beagle-trunk beagle-feature1 cd beagle-feature1 - *changes* + (hack, hack, hack) bzr commit -m "blah blah blah" - *changes* + (hack, hack, hack) bzr commit -m "blah blah blah" cd ../beagle-trunk bzr pull

[PATCH] en/user-guide: Change 'X-repo/X-trunk' to 'project/trunk'

Jari Aalto — 2008-12-02T16:24:40

I found it a little hard to follow dashed names like 'X-repo/X-trunk', so here is patch proposal to use simple 'project/trunk' notation. Jari doc/en/user-guide/organizing_branches.txt | 6 ++-- doc/en/user-guide/publishing_a_branch.txt | 34 +++++++++++++++++------------ doc/en/user-guide/reusing_a_checkout.txt | 12 +++++----- doc/en/user-guide/using_checkouts.txt | 4 +- 4 files changed, 31 insertions(+), 25 deletions(-) diff --git a/doc/en/user-guide/organizing_branches.txt b/doc/en/user-guide/organizing_branches.txt index c2c7fe6..5da43c7 100644 --- a/doc/en/user-guide/organizing_branches.txt +++ b/doc/en/user-guide/organizing_branches.txt < at >< at > -14,9 +14,9 < at >< at > To create a mirror branch, set-up a shared repository (or ``checkout``) command to create the mirror. For example:: - bzr init-repo X-repo - cd X-repo - bzr branch sftp://centralhost/srv/bzr/X-repo/X-trunk + bzr init-repo project + cd project + bzr branch sftp://centralhost/srv/bzr/project/trunk Task branches ------------- diff --git a/doc/en/user-guide/publishing_a_branch.txt b/doc/en/user-guide/publishing_a_branch.txt index 7c54548..4e522fa 100644 --- a/doc/en/user-guide/publishing_a_branch.txt +++ b/doc/en/user-guide/publishing_a_branch.txt < at >< at > -14,7 +14,7 < at >< at > Note that central shared branches typically only want to store history, not working copies of files, so their enclosing repository is usually creating using the ``no-trees`` option, e.g.:: - bzr init-repo --no-trees sftp://centralhost/srv/bzr/X-repo/ + bzr init-repo --no-trees sftp://centralhost/srv/bzr/project/ You can think of this step as similar to setting up a new cvsroot or Subversion repository. However, Bazaar gives you more flexibility < at >< at > -34,25 +34,31 < at >< at > content: Here is an example of the first way:: - bzr init-repo X-repo - bzr init X-repo/X-trunk - cd X-repo/X-trunk - cp -ar ~/X . (copy files in using OS-specific tools) - bzr add + bzr init-repo project (prepare local repository) + bzr init project/trunk + cd project/trunk + (copy development files) + cp -ar ~/project . (using OS-specific commands or tools) + + bzr add (populate repository; start version control) bzr commit -m "Initial import" - (local branch has content - publish it centrally now) - bzr push sftp://centralhost/srv/bzr/X-repo/X-trunk + (publish to central repository) + bzr push sftp://centralhost/srv/bzr/project/trunk Here is an example of the second way:: - bzr init-repo X-repo - cd X-repo - bzr init sftp://centralhost/srv/bzr/X-repo/X-trunk - bzr checkout sftp://centralhost/srv/bzr/X-repo/X-trunk - cd X-trunk - cp -ar ~/X . (copy files in using OS-specific tools) + bzr init-repo project (prepare local repository) + cd project + (prepare remote central repository) + bzr init sftp://centralhost/srv/bzr/project/trunk + (branch from central repository to local) + bzr checkout sftp://centralhost/srv/bzr/project/trunk + cd trunk (copy development files) + cp -ar ~/project . (using OS-specific commands or tools) bzr add bzr commit -m "Initial import" + (publish to central repository) + bzr push sftp://centralhost/srv/bzr/project/trunk Note that committing inside a working tree created using the ``checkout`` command implicitly commits the content to diff --git a/doc/en/user-guide/reusing_a_checkout.txt b/doc/en/user-guide/reusing_a_checkout.txt index 15e961d..5b49446 100644 --- a/doc/en/user-guide/reusing_a_checkout.txt +++ b/doc/en/user-guide/reusing_a_checkout.txt < at >< at > -61,10 +61,10 < at >< at > One possible setup is to use a lightweight checkout in combination with a local tree-less repository. This lets you switch what you are working on with ease. For example:: - bzr init-repo --no-trees X-repo - cd X-repo - bzr branch sftp://centralhost/srv/bzr/X-repo/X-trunk - bzr checkout --lightweight X-trunk my-sandbox + bzr init-repo --no-trees project + cd project + bzr branch sftp://centralhost/srv/bzr/project/trunk + bzr checkout --lightweight trunk my-sandbox cd my-sandbox (hack away) < at >< at > -74,11 +74,11 < at >< at > a tree-less repository. You can grab or create as many branches as you need there and switch between them as required. For example:: (assuming in my-sandbox) - bzr branch sftp://centralhost/srv/bzr/X-repo/X-1.0 ../X-1.0 + bzr branch sftp://centralhost/srv/bzr/project/X-1.0 ../X-1.0 bzr switch ../X-1.0 (fix bug in 1.0) bzr commit -m "blah, blah blah" - bzr switch ../X-trunk + bzr switch ../trunk (go back to working on the trunk) Note: The branches may be local only or they may be bound to diff --git a/doc/en/user-guide/using_checkouts.txt b/doc/en/user-guide/using_checkouts.txt index 356a696..e29c365 100644 --- a/doc/en/user-guide/using_checkouts.txt +++ b/doc/en/user-guide/using_checkouts.txt < at >< at > -7,7 +7,7 < at >< at > Turning a branch into a checkout If you have a local branch and wish to make it a checkout, use the ``bind`` command like this:: - bzr bind sftp://centralhost/srv/bzr/X-repo/X-trunk + bzr bind sftp://centralhost/srv/bzr/project/trunk This is necessary, for example, after creating a central branch using ``push`` as illustrated in the previous section. < at >< at > -56,7 +56,7 < at >< at > as doing so: To get a history-less checkout in Bazaar, use the ``--lightweight`` option like this:: - bzr checkout --lightweight sftp://centralhost/srv/bzr/X-repo/X-trunk + bzr checkout --lightweight sftp://centralhost/srv/bzr/project/trunk Of course, many of the benefits of a normal checkout are lost by doing this but that's a tradeoff you can make if and when it makes sense.

Problems with olive-gtk on Ubuntu Hardy

David Ingamells — 2008-12-02T11:34:36

Running olive on Ubuntu Gutsy Gibbon worked OK, but with Hardy Heron I am getting the following error: /data/users/david.ingamells/bazaar/bzr-gtk-0.95.0$ olive-gtk Traceback (most recent call last): File "/data/id/release/bzr/current/bin/olive-gtk", line 94, in app = OliveGtk() File "/data/id/release/bzr/bzr-1.9/lib/python/bzrlib/plugins/gtk/olive/__init__.py", line 91, in __init__ self.window = OliveGui(calling_app = self) File "/data/id/release/bzr/bzr-1.9/lib/python/bzrlib/plugins/gtk/olive/window.py", line 37, in __init__ self.set_icon_list(gtk.gdk.pixbuf_new_from_file(icon_path("oliveicon2.png")), TypeError: pixbuf_new_from_file() argument 1 must be string, not None It looks like oliveicon2.png is not being found. When I look in the bzr-gtk code of __init__.py (as installed) I see: def data_basedirs(): return [os.path.dirname(__file__), "/usr/share/bzr-gtk", "/usr/local/share/bzr-gtk"] This is wrong as I installed from the source package with: python setup.py install --home xxxxx where xxx is not /usr/... It looks like the install of bzr-gtk is not correctly using the --home option. thanks, David.