gmane.comp.lang.jython.devel

Summary of Jython tracker Issues

Jython tracker — 2008-09-05T16:10:12

------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/_______________________________________________ Jython-dev mailing list Jython-dev< at >lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jython-dev

Summary of Jython tracker Issues

Jython tracker — 2008-08-29T16:10:07

nowalker branch

Frank Wierzbicki — 2008-08-27T16:07:53

Hi all, The nowalker branch is nearing completion -- I have merged Python.g and PythonWalker.g into just a Python.g. The unit tests now pass as well as they do in trunk -- the new Python.g is (at least in my opinion) *much* simpler than the 2 grammar approach which was really only necessary because of some limitations in Anltr 3.0 (which I really where just limitations in my understanding of Antlr 3.0 -- I think now that I've gone through this process I could have done it in 3.0). The imaginary tokens that I was using are now gone, leaving only the INDENT and DEDENT tokens that where there in the original example Python.g from the Antlr project. I now have much finer control over the offsets, so I should be able to get almost all of these to match CPython (there are one or two differences I've seen that look like they *may* be odd implementation details). My internet connectivity is intermittent today and tomorrow -- so I'll wait until Friday or Saturday to merge this branch back into trunk. If anyone wants to give it a test it is here: https://jython.svn.sourceforge.net/svnroot/jython/branches/nowalker -Frank ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

GSoC Final Report: Django on Jython

Leo Soto M. — 2008-08-24T18:53:06

Looks like many people already know about my SoC results. But I wanted to "officially" finish the SoC reports. Django works on Jython without special patching. The tutorial can be followed without any problems. And Django projects are extremely easy to deploy on a Java Application Server, thanks to modjy. I'd even say that deploying Django/Jython on Java appservers is easiest than Django/CPython on apache/fastcgi/whatever. On the test suite, I wasn't able to fix all of the failures. Django developers did a great job on the testing front, growing from ~200 tests when I started the SoC to more than 400 now. That's a good thing, as it made possible to spot some bugs on Jython, but it makes the task of having fully passing test suite a bit more difficult. By the end of the SoC coding period, we had 92% of Django tests passing. A good number anyway, but should be improved. There is also more work to do on the DB-backends front[1]. I think that we may be able to easy the deployment of projects which mix Django/Jython and Java code. I'm not disappearing now. I want to continue coding and growing the support of Django on Jython. That's why I've started the django-jython project[2]. By now, it doesn't have a separate mailing list, since I don't see why it should. I think we should keep the Django/Jython usage discussions on the jython-users list. Also, I'm now a Jython developer, and have a lot of pending tasks to do on the Jython front which were motivated by my work on Django integration. I won't have as much time as I had before, but I expect to be hanging around here for a long time. Thanks to everyone which helped me on this great experience! [1] http://code.google.com/p/django-jython/wiki/DevelopingNewDatabaseBackends [2] http://code.google.com/p/django-jython/

Summary of Jython tracker Issues

Jython tracker — 2008-08-22T16:10:08

asm branch moved to trunk. asm branch now closed.

Frank Wierzbicki — 2008-08-19T22:57:51

A Release_2_3maint branch was created in case anyone wants to advance the javacc/old compiler work. I decided against tagging it since the branch is probably enough. The asm work is now merged into trunk, and the asm branch is closed. I think I'm going to wait a bit before I try to put an alpha together just to make sure things have settled. I also created a nowalker branch to hold the work I've done to eliminate PythonWalker.g -- but it's pretty rough still. -Frank ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Parsing and non-ASCII Input

Leo Soto M. — 2008-08-16T00:25:47

This was mostly planned as a note-to-myself document, as I spent most of this day with this issue but had to let it on hold for a few days. I now think it is good idea to post it here, for review of more experimented Jython developers. Problem ======= We have problems parsing non-ascii input under some circumstances. It is mostly visible on doctest right now, but I'm afraid that the current bugs can show up in other circumstances too. Here is a minimal example: 'f\xf6\xf6' 'b?r' Cause ===== The problem can be tracked to the use of String.toBytes() before reaching the parser, without taking into account the case when the incoming String[*] contains non-ascii characters. The offending code is on * ParserFacade#parse(String string, String kind): Unused, but dangerous IMHO, so should be removed/reworked * Py#compile_flags(String data, String filename, String type, CompilerFlags cflags): The entry point for most eval()/exec operations. [*] For clarity, When I say String, I'm specifically refering to java.lang.String. If I want to refer to Python strings, I say PyString. Potential Solution ================== Now, this could be solved on many ways. For example, we could say that the users of this methods must make sure that the String doesn't contain any value > 255. Then, every user of this method should have to encode the string using UTF-8, wrap the results on a String, and then we could have the original String back when ParserFacade#parse(InputStream stream, String kind, String filename, CompilerFlags cflags) decodes the InputStream. Clumsy, but it is similar to how CPython solves the issue, and it is the reason for PyCF_SOURCE_IS_UTF8 existence. But I think that we could (should?) do better. CPython is constrained by char[] and doesn't have a rich string type. We do have it. For me, it sounds weird to use String as a bytestring container (outside of the PyString code). The parser should accept Strings (and Readers?) in addition to an InputStream. InputStream means "the input is bytes, do encoding detection" and String/Reader means "just parse this (and validate that no encoding declaration is present). And, we have to make sure that when the original source for parsing is a PyUnicode, we take the String/Reader approach. If it is PyString, the InputStream approach is in order. If it is a String, obviously we go for String/Reader. Problems with the proposed Solution =================================== It is not straightforward to refactor the ParserFacade#parse to accept a String or a Reader, because universal newline support is done wrapping the InputStream. I'm not sure if universal newline is needed when parsing from a String, but I'd bet that it is. Also, I don't know where the DONT_IMPLY_DEDENT support belongs. Currently it is recognized on Py#compile_flags(String data, String filename, String type, CompilerFlags cflags), which accepts an String assuming that it is a bytearray in disguise. But that means that it won't be have any effect if the parsing is started by calling Py#compile_flags(InputStream istream, String filename, String type,CompilerFlags cflags), used on PythonInterpreter.

Summary of Jython tracker Issues

Jython tracker — 2008-08-15T16:10:07

Plan for moving asm to trunk part deux

Frank Wierzbicki — 2008-08-14T20:18:29

Even more than the last time I suggested this, most development has moved to the asm branch, and the asm branch has probably reached "good enough" to become trunk. Philip Jenvey pointed out some problems that have since been resolved. I still want to check and make sure that this is the right thing to do. My plan for this move is the same as before: 1) Announce a freeze of trunk 2) make a Release_2_3maint branch against the current trunk, and a Release_2_3a0 tag to mark the last of the javacc based Jythons -- just in case someone needs to continue with that 3) merge asm into trunk using svnmerge.py 4) soon after release a proper alpha2 with a proper Release_2_5maint branch, a proper tag, etc -- which would have been difficult for alpha1 because I made it from a branch and not trunk. I think that's it... -Frank ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Summary of Jython tracker Issues

Jython tracker — 2008-08-08T16:10:11

asm branch needs a clean if you rebuild

Frank Wierzbicki — 2008-08-06T02:16:46

Hey all, I'm mucking with tokens again -- so if you rebuild the asm branch do a "clean" first. I did try to get the build.xml to notice the change, but haven't succeeded yet. -Frank ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

fatal bug in 2.2.1 PyStringMap serialization

Colin Evans — 2008-08-05T04:04:40

Hi all! I just wanted to give a heads-up that I found a nasty fatal bug when serializing and deserializing PyStringMap instances in Jython 2.2.1. Essentially, if the size of the PyStringMap is one of a small set of prime numbers and the instance is deserialized and serialized, then calls to __finditem__ may result in an infinite loop. The details and a script to recreate the problem are here: http://bugs.jython.org/issue1098 ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Summary of Jython tracker Issues

Jython tracker — 2008-08-01T16:10:07

On String, PyString and PyUnicode

Leo Soto M. — 2008-07-31T22:46:42

Hi folks, Short History: I don't think that java Strings should be mapped to PyString. They should be mapped to PyUnicode. Any obvious reason why I may be wrong? Whole History: I was trying to implement unicodedata.normalize. It ought to be straightforward, by delegating to java.text.Normalizer.normalize. Here is the code: from java.text import Normalizer; _forms = { 'NFC': Normalizer.Form.NFC, 'NFKC': Normalizer.Form.NFKC, 'NFD': Normalizer.Form.NFD, 'NFKD': Normalizer.Form.NFKD } def normalize(form, unistr): """ Return the normal form 'form' for the Unicode string unistr. Valid values for form are 'NFC', 'NFKC', 'NFD', and 'NFKD'. """ return Normalizer.normalize(unistr, _forms[form]) But it isn't right. Normalizer.normalize returns a String, which is mapped to PyString. I need a PyUnicode. And unicode(Normalizer.normalize(unistr, _forms[form])) won't work, because it will try to decode the PyString using ASCII-7. For this particular case I think I'll just write the tempral unicodedata module in Java, or something like that. But it worries me how this shows that the whole String -> PyString mapping seems to be wrong. There is no obvious way to get back the original information, in the general case. For example: '\u1234' 1 4660 Traceback (most recent call last): File "", line 1, in UnicodeDecodeError: 'ascii' codec can't decode byte 0x52 in position 0: ordinal not in range(128) Traceback (most recent call last): File "", line 1, in UnicodeDecodeError: 'latin-1' codec can't decode byte 0x52 in position 0: ordinal not in range(256) u'' Traceback (most recent call last): File "", line 1, in File "/home/lsoto/src/jython.svn.asm/dist/Lib/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x52 in position 0: ordinal not in range(255) Note that for this case, the normalizing is a no-op, so the information I was trying to rescue was u'\u1234'.

Jython settings - input requested

Nicholas Riley — 2008-07-31T19:02:14

Hi folks, We're trying to clean up the multitude of ways Jython reads settings (properties, options) and would appreciate your input on what bits of customization you use and which ones you'd like to see added or removed. Jython reads settings/options/properties at startup from several sources. From the standpoint of a user of the interactive interpreter, they are: - Java system properties (-J-Dxxx=yyy with new launcher script) - Registry (prop=value in file jython_home/registry or $HOME/.jython) - Jython properties (-Dxxx=yyy) There exist several partial attempts at documenting these settings. I've attempted to enumerate them all here: Some cases in which I'm pretty sure Jython is not behaving as designed or expected: 1. If a registry file is present, the Java system properties are ignored completely. 2. If both registry files are present, the "registry" one is ignored completely. 3. If a property exists in the registry, it takes precedence over a command-line option (e.g., python.verbose trumps -v[vv]). Some changes I'm considering making: 1. Merge system properties and registry files into the registry, rather than replacing them outright. 2. Make command-line options take precedence over registry options. 3. Read from the JYTHONPATH environment variable, replacing python.path in the registry (or in system properties?). This is analogous to IronPython's IRONPYTHONPATH. CPython understands a bunch more environment variables (e.g. PYTHONDEBUG, PYTHONINSPECT) but I'm not sure we really need to support them, either in place or renamed. Should the JYTHONPATH mapping happen in the launcher scripts or Jython itself? One last thing: in CPython, setting PYTHONPATH places the items after the current directory, but before the standard library. Setting python.path in Jython places the items after the standard library; you must use python.prepath to get the CPython behavior. It looks like the reason for this change is no longer applicable: as the Python standard library is now included with Jython by default, so perhaps python.prepath should be removed and python.path take its place? Or do we need a python.postpath? Does anyone take advantage of this ordering?

Summary of Jython tracker Issues

Jython tracker — 2008-07-25T16:10:07

GSoC Weekly Report (#7 and #8): Django on Jython

Leo Soto M. — 2008-07-23T14:59:59

Looks like the impact of realizing that I'm going to speak on DjangoCon[1] about Django/Jython made me forget some things, like continue with the weekly reports. Now I'm fixing this :) The first thing I did was change my workflow and organize my code as patch queues, which are better suited to follow and update the patches I submit to Django devs, and also my work in progress on the Jython side. The details, and short instructions to get the latest Django/Jython code is on . Note that I haven't be able to publish the results I'm getting from hudson, the continuous integration server I started to use. So I'll try to also update the ad-hoc status page[2] until I get some Java hosting space, maybe on Imagemaker's servers. On the last days most of the patches to Django needed for Jython integration has been commited, some of them after a bit of tweaking. The notable exception is #7560, but I'm confident that it will be eventually committed now that Django is on the road to 1.0-beta, and critical must-have features have been integrated to trunk. The great news is that the size of my patch queues have been greatly reduced, which means that shortly Django should run on top on Jython without any special patching :-) The not-so-good news is that I'm still chasing the test suite. Django moves really fast, and new failing tests cases arise from time to time. Anyway, looks like most failures are concentrated on Jython text-encoding handling, the new manage.py tests and some obscure remaining incompatibilities with CPython. So, we may have a ~15% failure rate, but a few critical fixes should kill most of the failures. Finally, here is some current work in progress: - sqlite3/zxJDBC backend. Jim Baker pointed me to the cool Zentus SQLiteJDBC driver[4], which includes a pure Java implementation. By now it doesn't work but it already served as a good practical example to extract code that should be reused by other zxJDBC drivers out of the PostgreSQL/zxJDBC backend. - modjy integration. Needs some final touches to modjy, like correctly setting SCRIPT_PATH. - WAR distribution. I have a proof-of-concept prototype, now going to rewrite and integrate it to Django, as a management command. And some important things which haven't be done yet but should to be addressed on the remaining GSoC weeks: - Reloading support for the dev server - Some kind of IDE integration - Documentation And a final note: on the past two weeks I managed to finish all my university exams, so now there is more time for the GSoC project :-) [1] http://djangocon.org [2] http://dojstatus.leosoto.com [3] http://code.djangoproject.com/ticket/7560 [4] http://www.zentus.com/sqlitejdbc/

Jython help ?

Baroudi Malek — 2008-07-21T10:22:23

Some modjy questions/proposals

Leo Soto M. — 2008-07-21T04:01:42

Hello list, and specially Alan Kennedy ;) I have been playing with modjy the weekend and have two small questions/proposals: 1. I see that SCRIPT_NAME is directly set from getServletPath(). This ignores the context path, which from what I understand from the WSGI spec[1] ("The initial portion of the request URL's "path" that corresponds to the application object, so that the application knows its virtual "location""), should also be included. Also, looks like the JRuby guys are also prepending the context root[2]. Or am I missing something? 2. What about having a predefined directory analogous to WEB-INF/lib, but for python code (i.e, to get added on sys.path). How about WEB-INF/lib-python? This way, we could have the following hierarchy inside WAR files: . |-- WEB-INF | |-- classes | |-- lib | | |-- jython.jar | | |-- modjy.jar | | `-- postgresql-jdbc3-8.2.jar | |-- lib-python | | |-- django | | |-- jython-lib.jar | | `-- my_django_project | `-- web.xml `-- application.py Note that, as the entries of sys.path are also scanned by the Jython's classloader, they will also be scanned for java classes, but WEB-INF/lib/* won't be scanned for python modules. To me this feels consistent with typical Jython behaviour. Last but not least, getting modjy working has been quite easy, so I want to thank Alan for his great work! I can't wait to see it included into Jython itself :) [1] http://www.python.org/dev/peps/pep-0333/ [2] http://github.com/nicksieger/jruby-rack/tree/e440ea1334dcf8a9ad540728278ed89d21ef4077/src/main/ruby/rack/handler/servlet.rb#L111

Plan to move asm to trunk

Frank Wierzbicki — 2008-07-21T00:51:53

So it looks like most development has moved to the asm branch, and that the asm branch has probably reached "good enough" to become trunk. Still, I want to check and make sure that this is the right thing to do. My plan for this move would look something like this: 1) Announce a freeze of trunk 2) make a Release_2_3maint branch against the current trunk, and a Release_2_3a0 tag to mark the last of the javacc based Jythons -- just in case someone needs to continue with that (say a GSOC student that still needs this version, or someone in the future that cannot use ANTLR for some reason...) 3) merge asm into trunk using svnmerge.py 4) soon after release a proper alpha2 with a proper Release_2_5maint branch, a proper tag, etc -- which would have been difficult for alpha1 because I made it from a branch and not trunk. I think that's it... Thoughts? -Frank ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/

Possible Bugs in Jython 2.5a1 array

2008-07-19T08:29:43