http://blog.gmane.org/gmane.comp.web.zope.zodb hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://comments.gmane.org/gmane.comp.web.zope.zodb/12166 <pre>Hi All, I've been chatting with Shane about this off-list but wanted to see if anyone else has experience of this and has found a solution. So, the problem is that, with a Zope 2 app server, any revisions made to an object (say, a Page Template) don't show in the history tab of that object *until* the app server is restarted. I've verified this on both MySQL and Postgres with trunk of RelStorage as well as 1.5.2 and 1.4.2. It appears to be some kind of cursor re-use / relational database transaction isolation bug given that the following hack fixes the problem: --- a/relstorage/storage.py +++ b/relstorage/storage.py &lt; at &gt;&lt; at &gt; -1025,7 +1025,7 &lt; at &gt;&lt; at &gt; class RelStorage( self._lock_acquire() try: self._before_load() - cursor = self._load_cursor + conn, cursor = self._adapter.connmanager.open() oid_int = u64(oid) try: rows = self._adapter.dbiter.iter_object_history( &lt; at &gt;&lt; at &gt; -1053,6 +1053,7 &lt; at &gt;&lt; at &gt; class RelStorage( </pre> Chris Withers 2012-05-22T05:37:20 http://comments.gmane.org/gmane.comp.web.zope.zodb/12164 <pre>Good day comunity. My project is a ZOPE3/BlueBream project, that runs mainly of ZODB using ZEO. I use wsgi to get multiple instances(4) of my project and connect to ZEO Server from them. The database is around 3 GB in unpacked state and around 800 MB in packed. I use 4 processes with 7 threads each for serving data WSGIDaemonProcess doba.ua user=www group=www processes=4 threads=7 maximum-requests=2000000 I have little space on /tmp partition, so the cache of ZEO is only 40 MB pool-size 30 &lt;zeoclient&gt; server localhost:8200 storage 1 # ZEO client cache, in bytes cache-size 40MB # Uncomment to have a persistent disk cache #client zeo1 &lt;/zeoclient&gt; The server has those specs: 8GB ram. 8 CPU 3.2 Ghz. Raid-2 in a mirror mode. The load of instances: 1 GB per instance - 8 GB total. Other apps consum up to 1.5 GB. So there is 2.5 GB in inactive mode all the time. The problem, that I have: The server has a Raid-2 in a mirror mode, so I only have 1 hard drive, which contain all</pre> Войнаровський Тарас 2012-05-18T13:53:00 http://comments.gmane.org/gmane.comp.web.zope.zodb/12161 <pre>Hi All, In case anyone else is interested, I've git svn clone'd svn://svn.zope.org/repos/main/relstorage and pushed the result up here: https://github.com/cjw296/relstorage It's only a master branch, sine git svn clone does svn branches and separate remotes by default and I didn't want to do my manual hack-fix steps since Shane likely wants to stay with svn: cp -Rf .git/refs/remotes/tags/* .git/refs/tags/ rm -rf .git/refs/remotes/tags cp -Rf .git/refs/remotes/* .git/refs/heads/ rm -rf .git/refs/remotes cheers, Chris </pre> Chris Withers 2012-05-15T12:30:42 http://comments.gmane.org/gmane.comp.web.zope.zodb/12157 <pre>Hi, Previous threads on this subject didn't seem to go anywhere. I know it's not all about money but if we were to sponsor development of microsoft sql server support for relstorage, is there someone who knows how and has an estimated cost and available time? --- Dylan Jay Technical Solutions Manager PretaWeb: Multisite Performance Support P: +612 80819071 | M: +61421477460 | twitter.com/djay75 | linkedin.com/ in/djay75 _______________________________________________ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev&lt; at &gt;zope.org https://mail.zope.org/mailman/listinfo/zodb-dev </pre> Dylan Jay 2012-05-09T00:34:22 http://comments.gmane.org/gmane.comp.web.zope.zodb/12154 <pre>Hello all, I'm using Repoze.BFG, with paster to launch the webserver. This is a similar issue to the one I emailed about before titled "server stops handling requests - nowhere near 100% CPU or Memory used" The situation is the same. I used z3c.deadlockdebugger , and what I notice is that, when the server is blocked, there are about 100 threads running (as opposed to 15 or so when the server has just started), and all their stack traces look like this: Thread 140269004887808: File "/usr/lib/python2.6/threading.py", line 504, in __bootstrap self.__bootstrap_inner() File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner self.run() File "/usr/lib/python2.6/threading.py", line 484, in run self.__target(*self.__args, **self.__kwargs) File "/home/tsa/env/lib/python2.6/site-packages/paste/httpserver.py", line 878, in worker_thread_callback runnable() File "/home/tsa/env/lib/python2.6/site-packages/paste/httpserver.py", line 1052, in &lt;lambda&gt; lambda: self.process_requ</pre> Claudiu Saftoiu 2012-05-07T15:38:21 http://comments.gmane.org/gmane.comp.web.zope.zodb/12151 <pre>Hey all, I'm using a thread to do some server-side work. The thread will be asked by different requests to do the same thing at the same time, so I want the thread to do all the work once and return the data to the requests. The problem is that the requests each have their own transaction and the thread essentially has none. I can communicate between the two only using identifiers - not persistent objects - that way the thread can process data in a different transaction than the requests yet still return a meaningful reply. However, the thread has to start a new transaction each time it processes something - which I know how to do: while True: #wait until asked to do something import transaction transaction.begin() However, the thread needs access to the root object in order to turn the identifiers gotten from the requests into persistent objects... how would I go about accessing the root object in such a circumstance? Thanks, - Claudiu _______________________________________</pre> Claudiu Saftoiu 2012-05-02T16:49:32 http://comments.gmane.org/gmane.comp.web.zope.zodb/12147 <pre>-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am receiving the following weird error with a fresh Plone 4.2b2 sandbox during development tests. At least the "Versions" error message confuses me since the installation including ZODB Data.fs file is usually fresh. Andreas File "/home/ajung/.buildout/eggs/Chameleon-2.8.4-py2.6.egg/chameleon/template.py", line 190, in render raise_with_traceback(exc, tb) File "/home/ajung/.buildout/eggs/Chameleon-2.8.4-py2.6.egg/chameleon/template.py", line 172, in render self._render(stream, econtext, rcontext) File "e9d937ec11d78bdffa5760f426680e870e7641d6.py", line 42, in render File "/home/ajung/.buildout/eggs/z3c.pt-2.2-py2.6.egg/z3c/pt/expressions.py", line 58, in render_content_provider cp.update() File "/home/ajung/.buildout/eggs/zope.viewlet-3.7.2-py2.6.egg/zope/viewlet/manager.py", line 112, in update self._updateViewlets() File "/home/ajung/.buildout/eggs/zope.viewlet-3.7.2-py2.6.egg/zope/viewlet/manager.py", line 118, in _updateViewle</pre> Andreas Jung 2012-04-24T09:52:59 http://comments.gmane.org/gmane.comp.web.zope.zodb/12137 <pre>Hello all, I recently made a lot of changes to my ZODB app, and now I'm experiencing some puzzling behavior. Normally I have ~15 clients making requests to the server every few seconds. Some of these requests commit new data to the database, while others just process existing data. I also have a 'heartbeat' client, which access the view called '/is_alive', which is entirely described by this function: def is_alive(request): return True Once a day or so, I will get a report from the heartbeat client that the server is down - no heartbeat has succeeded in the past 10 minutes. Indeed, any URL I go to on the server, will simply not load, but take forever - even this '/is_alive' view. I can still SSH into the server, however. Running 'top', I see that the server is not taking any CPU time, and is not taking any large amount of memory. The computer itself runs just fine. I can even access the database with the paster shell, make requests, commit things, etc., without any exceptional delays. If I CTRL+C th</pre> Claudiu Saftoiu 2012-04-19T15:20:56 http://comments.gmane.org/gmane.comp.web.zope.zodb/12130 <pre>-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi List, We need to migrate an existing test environment running Plone 4.1.4 with ZEO to a new environment on RelStorage. I have configured zodbconvert to import the data.fs however I cannot for the life of me find doco on how to migrate the blob (bushy layout) into RelStorage. I am working with content exported via collective.recipe.backup If anyone could give an assist with this one would be a life saver! Cheers, Sam -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) iF4EAREIAAYFAk+HcmYACgkQFdt86iEfl/cgUAD/Z8Rl8ju30WPvrQWPGoAXHlOc l1SNV9cEhlgImApeufQBAKs9O840XuVRpldMyt37YtQ/nomXca1hy4yyZJp6IlqO =PpMq -----END PGP SIGNATURE----- _______________________________________________ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev&lt; at &gt;zope.org https://mail.zope.org/mailman/listinfo/zodb-dev </pre> Sam Wilson 2012-04-13T00:25:25 http://comments.gmane.org/gmane.comp.web.zope.zodb/12129 <pre>We have been running an in house application on ZODB 3.8.2 and recently upgraded to 3.10.5. We run a ZEO server with FileStorage. It seems like everything works great, no actions on our part, but I wondered if it is suggested to some how migrate the on disk FileStorage to take advantage of new features or optimizations in 3.10? Thank you. _______________________________________________ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev&lt; at &gt;zope.org https://mail.zope.org/mailman/listinfo/zodb-dev </pre> Paul Warner 2012-04-12T20:49:04 http://comments.gmane.org/gmane.comp.web.zope.zodb/12123 <pre>Hello all, I have an HTTP request which has roughly two parts to it. One part requires a lot of processing with a small chance of a DatabaseConflict, while the other part requires little processing with a high chance of a DatabaseConflict. I'd like to do something like this: def my_view(request): transaction.begin() slow_no_conflict() transaction.commit() for avar in something: transaction.begin() fast_yes_conflict(avar) transaction.commit() My question is: how will automatic retrying work? Most of my views are simply: def regular_view(request): do_stuff() and, if something conflicts, the whole thing is just retried. In `my_view`, what will happen if the `slow_no_conflict()` function has a conflict? What will happen if the `fast_yes_conflict(avar)` function has a conflict? What if its in the first iteration of the loop, or the last? I'm not quite sure how to properly think about these things yet. Thanks, - Claudiu ____________________________________________</pre> Claudiu Saftoiu 2012-03-29T18:28:12 http://comments.gmane.org/gmane.comp.web.zope.zodb/12110 <pre>I am getting an error when installing ZODB with easy install and python 2.5. The error is coming from transaction and ZODB fails to fully install. Here is the error: Searching for transaction&gt;=1.1.0 Reading http://pypi.python.org/simple/transaction/ Reading http://www.zope.org/Products/ZODB Best match: transaction 1.2.0 Downloading http://pypi.python.org/packages/source/t/transaction/transaction-1.2 .0.tar.gz#md5=48bb47e69df085d01ebec185705b3a7f Processing transaction-1.2.0.tar.gz Running transaction-1.2.0\setup.py -q bdist_egg --dist-dir c:\users\00081528\app data\local\temp\easy_install-t6bjyi\transaction-1.2.0\egg-dist-tmp-9hvdng File "build\bdist.win32\egg\transaction\tests\savepointsample.py", line 26 class SampleDataManager(object): ^ SyntaxError: invalid syntax File "c:\python25\lib\site-packages\transaction-1.2.0-py2.5.egg\transaction\te sts\savepointsample.py", line 26 class SampleDataManager(object): ^ SyntaxError: invalid syntax Adding transaction 1.2.0 to easy-ins</pre> Bahr, Eric 2012-03-28T19:02:05 http://comments.gmane.org/gmane.comp.web.zope.zodb/12105 <pre>The transaction package offers a nice feature, where you can say: import transaction for attempt in transaction.attempts(3): with attempt as t: ... do something ... If "do something" raises a ConflictError (or any other retryable error), the next attempt is tried, until all attempts have been exhausted, at which point, it gives up and the exception is raised. But I think it may be slightly broken. Here's the definition of the "attempt" context manager: class Attempt(object): def __init__(self, manager): self.manager = manager def __enter__(self): return self.manager.__enter__() def __exit__(self, t, v, tb): if v is None: self.manager.commit() else: retry = self.manager._retryable(t, v) self.manager.abort() return retry "do_something" within the body of an "attempt" context manager usually doesn't raise a retryable exception (it's business logic), but the "self.manager.commit()" wi</pre> Chris McDonough 2012-03-28T06:08:24 http://comments.gmane.org/gmane.comp.web.zope.zodb/12101 <pre>I have a Plone 4.1 site using RelStorage to store the zodb in MySQL 5.5 server. The site has 700 objects cataloged and the database is less than 300MB after packing. I won't consider it's a large site. When I tried to "update catalog" or "clear &amp; rebuild" the catalog, I got the following error: Traceback (innermost last): Module ZPublisher.Publish, line 134, in publish Module Zope2.App.startup, line 301, in commit Module transaction._manager, line 89, in commit Module transaction._transaction, line 329, in commit Module transaction._transaction, line 443, in _commitResources Module ZODB.Connection, line 567, in commit Module ZODB.Connection, line 623, in _commit Module ZODB.Connection, line 682, in _store_objects Module relstorage.storage, line 571, in store Module relstorage.adapters.mover, line 484, in mysql_store_temp Module relstorage.adapters.batch, line 67, in insert_into Module relstorage.adapters.batch, line 74, in flush Module relstorage.adapters.batch, line 110, in _do_inserts Module MySQLdb.cu</pre> J Z 2012-03-27T16:32:46 http://comments.gmane.org/gmane.comp.web.zope.zodb/12086 <pre>Hello ZODB List, (This is also a stackoverflow question - you might prefer the formatting there: http://stackoverflow.com/questions/9810116/zodb-database-conflict-fail ) I have a server, and a client. A client sends a request. The request has a certain key associated with it, e.g. `a-1`, `a-2`, `b-1`, `b-4`. If two requests for the same key come in at once, there will be a conflict error, as the same data structures are being modified. I can adapt the client to simply not send two requests of the same key at once. However, I'd like this system to work with multiple clients, too. It seems silly to have the clients coordinate what they send to the server. Instead, I'd like the server to simply block on a request of a certain key if that key is already being modified, until the other requests with that same key are done. To this end, I've created a locking system. At the beginning of the function on the server, I do: key = ... print "Acquiring %s lock..." % (key,) KEY_LOCKS[key].acquire() </pre> Claudiu Saftoiu 2012-03-21T22:54:58 http://comments.gmane.org/gmane.comp.web.zope.zodb/12083 <pre>It is possible to open a ZODB in a thread and share it to other threads via a filesystem socket or pipe [rather than a TCP conntection]? I've searched around and haven't found any reference to such a configuration. </pre> Adam Tauno Williams 2012-03-20T16:52:07 http://comments.gmane.org/gmane.comp.web.zope.zodb/12071 <pre>I am updating an item in a separated thread but when it finishes the new value is not updated in the same object in the main thread. In the example below root['counter'] starts in 0, is incremented in the new thread but it remains in 0 in the main thread. #!/usr/bin/python from ZODB.FileStorage import FileStorage from ZODB.DB import DB from BTrees.OOBTree import OOBTree import transaction from threading import Thread import persistent class MyThread(Thread): def __init__(self, db): Thread.__init__(self) self.db = db def run(self): self.connection = self.db.open(transaction.TransactionManager()) self.root = self.connection.root() print "Starting thread" self.connection.transaction_manager.begin() self.root["counter"] += 1 self.connection.transaction_manager.commit() print "self.root['counter'] =", self.root['counter'] filename = 'test.fs' storage = FileStorage(filename) db = DB(storage) connection = </pre> Sebastian Wain 2012-03-19T17:04:34 http://comments.gmane.org/gmane.comp.web.zope.zodb/12064 <pre>I'm pretty happy with how zc.zlibstorage has worked out. Should I build this into ZODB 3.11? BTW, lz4 compression looks interesting. The Python binding (at least from PyPI) is broken. I submitted an issue. Hopefully it will be fixed. Jim </pre> Jim Fulton 2012-03-14T17:47:54 http://comments.gmane.org/gmane.comp.web.zope.zodb/12062 <pre>Hi Shane, What does this exception mean: Traceback (innermost last): Module ZPublisher.Publish, line 135, in publish Module Zope2.App.startup, line 291, in commit Module transaction._manager, line 89, in commit Module transaction._transaction, line 329, in commit Module transaction._transaction, line 426, in _commitResources Module ZODB.Connection, line 776, in tpc_vote Module relstorage.storage, line 783, in tpc_vote Module relstorage.storage, line 817, in _vote Module relstorage.storage, line 689, in _prepare_tid Module relstorage.adapters.locker, line 104, in hold_commit_lock StorageError: Unable to acquire commit lock What can cause it? What are the recommended solutions? cheers, Chris </pre> Chris Withers 2012-03-04T23:16:38 http://comments.gmane.org/gmane.comp.web.zope.zodb/12058 <pre>Hi list, to have the ZODB3 tests run in a Python virtualenv successfullly I had to pip-install zope.testing separately. Is it the declared intention of the ZODB3 developers? Because it is not a "real" bug, I'm reporting it here instead of on Launchpad. Below is my complete bash session in Sphinx format. For zodb developers, the interesting reStructuredText section might be the third one, "Running the ZODB3 tests". I'm Using Ubuntu Lucid Lynx. Thanks. Aroldo. Installing ZODB3 in a virtualenv ================================ Aroldo Souza-Leite 2012-02-28 The ZODB3 tests raise an ImportError. They succeed after the intallation of zope.testing. Preparing the virtualenv ------------------------ .. code-block:: bash aroldo&lt; at &gt;aroldo-laptop:~$ virtualenv --no-site-packages tmp-env Using real prefix '/usr' New python executable in tmp-env/bin/python Installing setuptools............................done. Installing pip.....................done. After minor editing of tmp-en</pre> Aroldo Souza-Leite 2012-02-28T08:43:11 http://comments.gmane.org/gmane.comp.web.zope.zodb/12055 <pre>Hi all, I'm getting an error when trying to run zeopack (zeopack.py) from the command line. This is a Linux server running the following "Zope Version": "Zope 2.10.9-final, python 2.4.6, linux2". The Plone version is 3.3.1 The error message is "ImportError: No module named ZEO.ClientStorage". Here's my attempt to run the command: myhost:/soft/Plone/Zope-2.10.9-final-py2.4/lib/python/ZEO/scripts # /soft/Plone/Python-2.4/bin/python /soft/Plone/Zope-2.10.9-final-py2.4/lib/python/ZEO/scripts/zeopack.py Traceback (most recent call last): File "/soft/Plone/Zope-2.10.9-final-py2.4/lib/python/ZEO/scripts/zeopack.py", line 33, in ? from ZEO.ClientStorage import ClientStorage ImportError: No module named ZEO.ClientStorage I saw some threads about this error that suggested to set / expand the PYTHONPATH environment variable. I did not have that PYTHONPATH environment variable defined. So, I tried to define that variable: myhost:# export PYTHONPATH=/soft/Plone/Zope-2.10.9-final-py2.4/lib/python/ZEO:/soft/Pl</pre> Ricardo Dias Marques 2012-02-17T11:20:42 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.web.zope.zodb