gmane.lisp.clsql.devel

Re: [Patch] Update view-database only if thetransaction succeeded

Kevin Rosenberg — 2007-09-13T21:22:41

Yes, I did. Thanks for the discussion on the issues.

Re: [Patch] Update view-database only if thetransaction succeeded

Kevin Rosenberg — 2007-09-13T21:21:24

I had only looked at your patch in isolation. I briefly reviewed transactions.lisp and had a few ideas. Rather than a special variable, what about putting the view-object rollback information in the transaction object as an additional slot? Is the a great reason not to put the view-object rollback code in database--start-transaction and database-rollback, rather than in with-transaction? With-transaction is of course nice because you get rollback when if you abort out of the unwind-protect. But, given that CLSQL works to provide full transaction support with start-transaction and rollback, moving these the view-object code so that it is also supported by those functions seems like a win.

Re: [Patch] Update view-database only if thetransaction succeeded

Kevin Rosenberg — 2007-09-12T20:15:34

I see. So, only by using with-transaction (and not with start-transaction/rollback) will one have support for rolling-back the database slots of the objects. If using with-transaction buys the user better rollback support than available by using the individual transaction functions, that should go in the documentation as well.

Re: [Patch] Update view-database only if thetransaction succeeded

Saurabh Nanda — 2007-09-12T20:11:20

Will do. Thanks for the pointer. Yup. I think that's a better idea. Until and unless the transaction is rolled back (or aborted) the current DB connection is going to "see" the objects in the DB. So what you've suggested is the right thing to do. Heh. Just my cruft to make code more readable. Coloured markers in Emacs for free! Would've done that. The only problem being that doing so forces me to bind the symbol name to something. I've explained why I want *transaction-insert-list* to be unbound below. My solution is based on the assumption that most use-cases for transactions will use the with-transaction macro. My solution works optimally only in such a scenario because I'm binding the *transaction-insert-list* variables in the let form in the macro. By doing so I get the Lisp magic of dynamic scoping. With dynamic scoping *transaction-insert-list* will always be bound correctly in a multi-threaded environment or in nested transactions or whatever. On second thoughts I can probably bind *transaction-insert-list* to nil and give that doc-string and depend on something else to determine whether *transaction-insert-list* was bound in a dynamic context or not. Hope you understand what I'm trying to say. Explaining these abstracts concepts is hard on email :-) Saurabh.

Re: [Patch] Update view-database only if thetransaction succeeded

Saurabh Nanda — 2007-09-12T19:43:57

No. I thought of that. *transaction-insert-list* is bound only when you're using the with-transaction macro. However, someone may use start-transaction/commit/rollback separately which is going to break everything. If and only if the dynamic context for *transaction-insert-list* is set, should we push objects into it. Saurabh.

CLSQL Mail List Changes

Kevin Rosenberg — 2007-09-12T19:19:08

Sorry for the cross-posting -- eliminating cross-posting that is actually a goal of this message. Since requests for help are often intermingled with development issues (and to a lesser degree in the other direction), and to reduce the need to subscribe to multiple lists, I've merged the CLSQL-devel and CLSQL-help mail lists into a single, new mail list: CLSQL. I'll still keep CLSQL-announce as a low-traffic list for announcements, though, an announcement hasn't been made in years. I'd be happy to merge that list as well, but I think it does make sense to have a separate list for traffic-sensitive subscribers. One potential issue with the merge is that if you have subscribed to CLSQL-devel and CLSQL-help with two *different* email addresses, both addresses will be included in the merge so you might get duplicate messages. But, mailman does make it easy for you to unsubscribe one of the email addresses. A second issue is that archives will remain available on clsql-devel and clsql-help web sites. I will look into merging those archives into the new mail list's web site. But, I'm not sure that will be feasible. Barring a good argument against (or at least a few desparate pleas), I plan on implementing this change in the next week. However, if you have concerns or objections to this change, please send me email directly (off-list).

Re: [Patch] Update view-database only if thetransaction succeeded

Kevin Rosenberg — 2007-09-12T19:05:44

Rather than relying on the bound status of the variable, can't you just check if in-transaction-p (and initialize the variable value to nil)? Kevin

Re: [Patch] Update view-database only if thetransaction succeeded

Kevin Rosenberg — 2007-09-12T18:33:18

Hi Saurabh, Given that you have expressed interest in submitting future improvements to CLSQL, I have some general and specific suggestions so that you can best contribute to the project with this bug fix and future improvements. Your identification of the bug is an excellent find. I'd like to commit your final patch using git-am which automatically notes authorship and commits the changes into the repository. Please take a look at the git-am manual page for more info about on formatting of your submission, especially about how the body of the message becomes the commit message (see below). Add a ChangeLog entry so that can be in the same commit as your patch. As you may have seen looking at the repository, I typically use the ChangeLog addition as the actual commit message. See [1] below -- disregard the subversion revision "r11761: " prefix which was added by git-svnimport would not be part of the ChangeLog or future commit messages. To preserve backward compatibility with applications that expect the database slot to be immediately updated after a store, as well as be more consistent with the semantics of a SQL transaction, I think the right thing to do is to fill the database slot of objects immediately after a store. However, if a transaction ends before it is committed, then at that time the database slot of the modified objects should be returned to their initial states. This variable declaration should be at the top of the transactions.lisp file as it has nothing to do with what's in the database.lisp file. Note in clsql.asd that the functional module is loaded before the object module. Remove the empty comment line ";;;" as it provides no value. However, please add a documentation string to the defvar form. It would be better to change the variale name to something more descriptive and to leave the type out of the name. I think *view-objects-transaction-rollbacks* is a better name than *transaction-insert-list*. This variable should hold a stack of lists to properly handle nested transactions. The commit should come with an addition to the test suite in tests/test-oodml.lisp that demonstrates the correctness of your patch. Single and nested transactions with commits and rollbacks should cover theq possibilities. The "Side Effects" section of the ROLLBACK documentation should note what happens to view objects that have had their database slot modified during a transaction. If you need help generating the html and pdf documentation after modifying the DocBook XML, please let me know. Thanks again for spotting the problem and coming up with an initial idea on a fix. If the above seems like more work that you can do, let me know and I'll make an addition to the BUGS file until someone has the time to extend your idea. If you do have time to fix the bug optimally, that'd be great!

Re: [Patch] Update view-database only if thetransaction succeeded

Kevin Rosenberg — 2007-09-12T15:51:02

That's a good catch, Saurabh. Thanks for the patch. I've review it after I'm done working on fully supporting the strings as table names. The format of the patch looks quite usable. But, I'm aware that git has significant support maintaining metadata for email patches. I'll let you know if I find possible improvements in the format of your path. Kevin

[Patch] Update view-database only if the transactionsucceeded

Saurabh Nanda — 2007-09-12T05:55:39

Hi Kevin, I've modified the behaviour of update-records-from-instance and with-transaction to not set the view-database slot in all the new objects inserted when inside a transaction until the transaction is fully committed. Earlier, even if a transaction failed, and the objects were not actually inserted in the DB, the view-database slot was being set. This was causing problems while trying to re-insert the records - update-records-from-instance would try to update the records instead of inserting. (This is the first time I'm using git so please guide me about the correct way of submitting patches.) Saurabh.

Re: CLSQL source code repositories

Kevin Rosenberg — 2007-09-10T14:36:20

I think it should look like what you what the object to accomplish. From reading your statement below, it appears that you want to have an object that contains a sequence/field name so that a single primary key object can be automatically assigned and stored with that object. I think your suggestion below is probably best. But, assuming you want to have only one sequence name for all objects of that class, the proper way would be #1 above. I understand. Since the sequence can be opaque, and you'll have only one sequence per base class, then using a an internal sequence name would work. Perhaps something like __CLSQLSYS_EMPLOYEES_SEQ__ That would avoid having the complexity of subclassed view-class object No, I'm talking about using sequence-next on db-backends which don't support autoincrementing keys. But on sql engines that do support autoincrementing primary keys (like mysql), it is more efficient just to store the object then ask the db what was the primary key of the object you stored compared to reading then storing a sequence table and then storing the object. Plus, with the atomic autoincrementing, you don't need to lock the table. Very good. As you might guess for this project that has been ongoing for 5 years, not breaking things for existing users is of great importance. Simple has its advantages. Typically, I tend to trade that for flexibility, consistency, and efficiency. If I have an idiom that needs repeating, I tend to embed that in a function or macro around the FDML layer. I did bang my head on the CL ObjectStore library which had a very simple interface which was also much more powerful that CommonSQL. But, after 3 schema rewrites and numerous support conversations with Franz and still getting out-of-memory failures, I found using CLSQL with FDML gave me a highly efficient, easy to use, densely-linked 18GB SQL database. That said, there are some simple one-key tables where automatically getting it's primary key and storing it in the next primary key in the in-memory object would cut down on end-user coding and your modifications would be most welcome.

Re: [CLSQL-Help] CLSQL source code repositories

Kevin Rosenberg — 2007-09-06T14:55:14

You're very welcome. That's great and is the sort of creativity I was hoping the distributed repository would enable and encourage. I'll look forward to seeing your patches.

Re: When is fault-join-slot called?

Saurabh Nanda — 2007-08-17T05:38:31

Sorry. My mistake. fault-join-slot is NOT called in the above form. But any answers to the above? I couldn't figure out how MOP hooks into this function. Nandz.

When is fault-join-slot called?

Saurabh Nanda — 2007-08-16T10:53:57

Hi, Quick question: is fault-join-slot also triggered in a setf form? For example, will the following form also trigger fault-join-slot? (setf (slot-value parent-obj join-slot) join-object) I'm seeing weird behaviour in some of my code, which is expecting me to believe that this is the case. Also, where exactly is the MOP configured to hook into this function? Nandz.

Possible memory leak - LW/ODBC/Win32/Microsoft SQLServer 2K5

Matthew Lamari — 2007-07-20T23:51:13

I am using CL 3.8.5 (uffi-1.5.18) on Lispworks 5/Win32 to connect to a Microsoft SQL Server 2K5 database. I connect with: (connect (list nil nil nil :connection-string "Driver={SQL Server};Server=MyServer;Database=MyDB;Uid=myuser;Pwd=MyPassword;") :if-exists :new :database-type :odbc ) And run a query test with: (loop for i from 1 to 1000000 do (query "select CAST('hello' as varchar(4096))")) This loop basically starts eating up ram from the process. What is of interest is that if I break out and GC, (room t) does not indicate any rise in memory usage, implying that the ram is being eaten up in foreign allocations? Eventually, it runs to Lispwork's 1.5G limit and fails with: Error: Failed to allocate foreign object of size 4002 [allocator STATIC] 1 (abort) Return to level 0. 2 Return to top loop level 0. Disconnect doesn't clean it up either. Where should I go about tracking down this leak? I realize that it is plausible that the leak may be at the driver level or UFFI, not necessarily CLSQL. However, I was wondering if this is something that is a known issue for some aspect of the platform I'm using, or if this is something that devs versed in CLSQL could trivially track down, or at least point me in the right direction. Thanks, Matt

Re: ODBC BIGINT hacking

Kevin Rosenberg — 2007-07-20T22:02:20

Since not all Lisp implementations support long-long integers, the usual way to read 64-bit integers with UFFI is separatey access the two 32-bit halves of the long-long and then combine the results. I'd have to delve back into the ODBC backend to figure would how to do that double access, but I won't be doing that in the very near future. Sounds like you have an adequate work-around for now. Kevin

Re: SBCL calculating key-slots andfinalize-inheritance

Kevin Rosenberg — 2007-07-20T21:56:41

I'm not quite sure while this would be needed for SBCL, either. But, it seems safe enough and will solve your compilation issue, so I'll go ahead and include your patch in CLSQL 3.8.6

Re: ODBC -> MSSQL transactions (HACK)

Kevin Rosenberg — 2007-07-20T21:53:56

Thanks for the patch. A modified version of your patch will appear in CLSQL 3.8.6. Kevin

Re: [patch] with-database bug

Kevin Rosenberg — 2007-07-20T16:55:13

Thanks once more for the report. This will be fixed in CLSQL 3.8.6. Kevin

Re: find-and-load-foreign-library and Oracle

Kevin Rosenberg — 2007-07-20T16:41:19

Thanks for the additional information, Ricardo. I'll have a new version of CLSQL (3.8.6) soon that should fix the problem in the oracle-loader.lisp file.

Re: find-and-load-foreign-library and Oracle

Ricardo Boccato Alves — 2007-07-20T13:01:31

_______________________________________________ CLSQL-Devel mailing list CLSQL-Devel-0lovp2JerKU< at >public.gmane.org http://lists.b9.com/mailman/listinfo/clsql-devel