gmane.org.osaf.devel

[Chandler-dev] Checkin 9/5/2008

Mimi Yin — 2008-09-05T19:09:21

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Update on Notifications Feature Work

Mimi Yin — 2008-09-04T22:27:17

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Checkin 9/4/2008

Mimi Yin — 2008-09-04T18:17:51

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Chandler Desktop 1.0.1 test build available fordownload

Grant Baillie — 2008-09-04T18:08:24

The Chandler Desktop 1.0.1 test build is now available for download at: http://builds.osafoundation.org/chandler/milestones/1.0.1/ Alternatively, you can also use the "Tools >> Check for Test Updates" feature to get to the correct download in your web browser. I'll be checking the downloads work on various platforms, but if you have time to give it a spin on your machine, feedback is welcome. Aslo, as mentioned earlier, we'll be having a QA session on IRC today to verify bugs and validate the build. --Grant _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Post-1.0 Chandler Desktop Rearchitecture Project

Grant Baillie — 2008-09-04T17:20:13

For an html version of the following, see --Grant Introduction ------------ Now that Chandler Desktop 1.0 has shipped, the time has come to deal with more long-term issues with the Chandler code base, as we shift our energies to attracting a volunteer developer community to maintain the app and build new features. When Chandler Desktop shipped its Preview release in 2007, we had extensive discussions on this list about how to deal with issues of performance, scalability, extensibility, stability and code maintenance. For a summary, see ``_. Although OSAF's organizational goals are somewhat different now, I still believe it is true that the rearchitecture project is the only way to make Chandler sustainable by a volunteer developer community. This missive is intended to describe how we can utilize our remaining months of paid developer time to move forward on the rearchitecture. Of course, with any substantial undertaking, there are higher-level issues, like assessing risks and non-technical goals: those will be addressed in a separate email (or possibly blog post). In what follows, I'll recap some of OSAF's thinking last year regarding technical goals of the rearchitecture, and then follow up with a summary of work we did on a pilot project (before focusing on releasing 1.0). I'll then outline the major of areas of work a complete rearchitecture would involve. Lastly, I'll throw out what subset we think is achievable with remaining OSAF paid developer time. However, once the basic infrastructure pieces are in place, there is considerable flexibility as to what we tackle afterwards, so I'll mention some alternate strategies (or ideas for strategies). .. _toc: .. contents:: **Table of Contents** Rearchitecture -------------- Ideally, a complete rearchitecture involves creating a codebase that: 1. Has separate layers for storage, domain model, interaction and presentation. This helps parallel development by allowing, for example, development of interaction components for a new feature before the domain model is ready. It also enables trying different storage back ends for performance investigations, since it implies there would be a well-defined and limited set of APIs a new storage layer would have to implement. Similarly, different presentations (via different UI libraries) are possible, even though this wasn't (and still isn't) a goal for the rearchitecture project. 2. Is "clean room" unit tested. By this, I mean PJE's goal of having every line of code in the project be covered by at least one unit test. The main idea here is to reduce the risk of widespread changes (for instance, the kind of work that often needs to be done to fix performance problems). Together with #1, this also implies that unit tests should test a single layer at a time, which should make tests fast. In the Chandler 1.0 situation, if you write a test of the osaf.app package, you end up having to run code to create repository objects for the entire UI, leading to a set-up time of several seconds for that single test case. 3. Is thoroughly documented. This decreases the ramp-up barrier for both internal and external developers, of course. The Architecture Pilot Project (APP) ------------------------------------ In November/December of 2007, Phillip, Jeffrey and I spent some time on a pilot project for rearchitecting Chandler. The proposed scope for the project can be found in `this thread `_, while some of the technical details can be found in the various README files in the chandler rearchitecture branch, viz. #. `Top Level `_ #. `Platform `_ #. `Application `_ #. `Debugging `_ At a high level, the primary goals of the pilot project were to demonstrate feasibility of rearchitecture, and also to drive adoption of its ideas amongst OSAF developers. (Ultimately, the plan was to have individual developers move over from the old code base to the new as we ported over people's areas of expertise). As a result, we spent more time on visibly demoable features than if we were trying to implement 1--3 above in a vacuum. Although the pilot never really ran to completion, as a result of OSAF's organizational restructuring, we did accomplish the following: - We adopted the `Trellis API `_ for dependency management (i.e. to have changes between different parts of the app's object graph propagate correctly). - The demo contains a pluggable architecture for the platform, (incorporating wx/twisted), for the Chandler-specific pieces and for debugging via PyCrust. - There is demo UI for sidebar, calendar, preview area and minicalendar, but not detail view. - The domain model layer was intentionally left very minimal (i.e. just enough to support demoing the GUI). However, propagation of domain- level changes to the running UI was demoable by using the debugger. - The code contained the beginnings of a calendar data model, as originally `outlined `_ by Phillip. - There were some tests of presentation-level wx code via the `Python Mocker `_ library. This was partially done as an investigation of what it takes to write "clean room" unit tests for wx code. What we need to implement ------------------------- Here, I have a broad breakdown of the areas we can contemplate working on. This list is roughly in an implementation order, where we build a complete application from the "bottom up". However, once the basic infrastructure is built (i.e. the first three items) we can envision changing the order, as well as precisely which user features we want to implement in each area. So, as you move down the list, the descriptions become more vague. More details will be forthcoming as we undertake earlier items, and plan new ones! For completeness, I've included some parts I doubt we'll get to. #. **[Domain]: Infrastructure/Metamodel** Basics for bootstrapping plugins and for having persistable objects. In other words, support for adding types via plugin entry points, for some notion of what is currently called ``Kind``, for extending existing objects (akin to today's ``Annotation`` and ``Stamp`` concepts), and for making references to persistable objects by name. (This doesn't include support for actually persisting objects, but does support a kind of core schema that the Storage layer can implement). #. **[Domain]: Item and DashboardEntry components** Phillip and I sketched out a design for separating out what is currently the ``Item`` class (i.e. basic persistable object) from what is now ``ContentItem``. The latter currently encompasses several different concepts ("Item created by the domain model" is one), but the one we wanted to focus on is "Item viewable in the Detail View". (Possibly ``ViewableEntry`` or something similar is a better name than what we came up with at the time). The main idea here is to have each ``DashboardEntry`` object be generated dynamically by an ``Item``, and thereby have a way to deal with cases like recurring events. There still remain some design questions at this level that need to be resolved. For example, in the plugin-centric implementation of the rearchitecture pilot, we see core Items as being composed of "extensions" or "parts" (coming from various plugins), but it's unclear as to whether you can have more than one part for a given item type (or whether items even really have a type). However, these questions can be resolved after implementing things like DashboardEntries (and Reminders, below). `Requires: [Domain]: Infrastructure/Metamodel` #. **[Domain]: Reminders** At its core, a ``Reminder`` will be a separate type (i.e. database table) indexed by fire date. With a flexible storage layer, it would be possible to have this "table" be implemented outside the core database (e.g. via a directory full of entries), allowing for a small, long-running reminder daemon. The ``Reminder`` system will work a little more like the current chandler ``osaf.startup`` module: A single entry will point at a piece of code to be run when the reminder is due. This will allow both user- level reminders (i.e. ticklers or event-relative alarms) as well as tasks scheduled by plugins (including the Chandler application). `Requires: [Domain]: Infrastructure/Metamodel` #. **[Interaction]: Basic infrastructure** The basic design can be found `here `_. This design is still incomplete, and so will need to be fleshed out with the usual round of doctests and implementation use cases. `Requires: (None)` #. **[Domain]: Calendar** In the past, we spent some time looking at a scheme for implementing calendar queries in reasonable time (i.e. so that most of the work can be done by database indexes, without too much time being spent at the Python level). In the rearchitecture pilot, Jeffrey had written some code at the interaction layer where ``Month`` objects cache events for a calendar. In practice, things will be complicated by factors such as "floating" timezones. Probably the best approach will be to have slightly "fuzzy" system, where we store and index event _dates_, but have queries expand the date range by a day on each end, and filter out events outside the range. Effectively, we are making the "display time" of events be dynamic, while the stored value is static. This does require extra computation to retrieve any event, but some database backends (e.g. sqlite) can be configured to add a date conversion function, and thereby do the filtering/sorting themselves. (This function could be written in C, and therefore be "fast"). `Requires: [Domain]: Item and DashboardEntry components` #. **[Storage]** In the intervening months, Phillip has spent some time investigating how to bind a Trellis-based object system onto a SQL back-end. After investigating and discarding a SQLAlchemy-based approach, he's settled on something that allows python expressions to be used for queries, and which will translate themselves as needed to SQL queries. This is flexible enough to allow you to write higher-layer tests that run entirely in memory. (Clearly, there will eventually be a need to write functional tests that operate on a "real" database). #. **[Interaction]: Table model** During the pilot project, we started developing an interaction model for Tables, in particular, the user-sortable kind like the Dashboard in Chandler. In fact, Phillip added some support for this to the Trellis (via the `SubSet `_ and `Observing `_ types), but I (Grant) had yet to adopt them. `Requires: [Domain]: Basic infrastructure` #. **[Interaction]: Detail view** Some kind of "editing context" ("save buffer" is another term) is needed here, to model how changes make their way from the UI to lower layers. In particular, there would need to be a way to hook into the save mechanism to handle confirmation of recurrence changes. A save buffer can also be used to detect sharing conflicts in the case where you are busy editing a value that is changed remotely. `Requires: [Domain]: Basic infrastructure, [Interaction]: Basic infrastructure` #. **[Domain] EIM Import/Export** The difficulty with porting the current Chandler EIM layer is that it is tied to Chandler's data model (in particular, with respect to recurrence). On the plus side, it has a lot of unit tests to ease the transition. In favor of porting EIM are two other factors: - EIM (or, more correctly, .chex backups) is the path for moving users away from Chandler 1.0 to the rearchitected project. - As evidenced by the current Chandler ICalendar import/export code, it is also a useful bridge for supporting other formats. #. **[Presentation] Styling framework** For details, see this `PEAK list posting `_. This is the layer that supports the "P" in what is currently "CPIA" in Chandler. While it is something we want long-term, it's not a strict requirement for implementing demo-level user interfaces, which can be wired up directly as before. #. **[Domain] Sharing** The current sharing code base has two drawbacks so far as porting over to the new architecture: - The basic classes are not thoroughly unit tested; mostly there are functional tests whose coverage of the code base is unclear at the moment. - The API isn't well-suited to using non-blocking (twisted) APIs for the network pieces. As a result, it will be a considerable amount of work to port over all the different kinds of shares currently supported by Chandler. However, it may well be possible to implement one particular kind of conduit. `Requires: [Domain] EIM Import/Export` #. **[Interaction] Sharing** Mostly, this involves modelling interactions around how conflicts are presented to the user (assuming we even want to implement sharing conflicts at first), as well as issues like visibility of elements that show syncing status. #. **[Domain] ICalendar Import/Export** We do have good unit tests for this functionality in current Chandler, and the strategy of implementing this on top of EIM has worked out well. However, it's unclear whether full ICalendar support is necessary for the "core" app we are trying to build, so this might be on opportuntiy for outside development. #. **[Presentation] Remaining tasks** While it is nice to have demos (even to attract volunteer developers), it's unlikely we'll have time to make a primary focus of reproducing Chandler's UI. There is some new calendar code that is demo- worthy, and in fact represents a good area for developers interested in helping out with the rearchitecture project. #. **[All] Email Support** The existing mail service layer in Chandler is implemented in a fairly well-tested way, but attempting to support the gamut of IMAP/POP/ SMTP servers is a testing and developer effort that I don't think we have resources for at the moment. #. **[All] Printing support** Phillip recently mentioned that we could possibly use the `PythonReports `_ project as a means for implementing some specialized printing layouts for Chandler 1.0 that could be relatively straightforwardly ported to the rearchitecture branch, but this remains a project for non-core OSAF developers. 1.0: What can we achieve from here? ------------------------------------------- Basically, continuing down the pilot project path (i.e. filling in the pieces of a lookalike of the current app) doesn't really make sense, because it would lead to an improved (i.e. more tested/testable) version of the Chandler 1.0 app, but without #2 in particular, we would still face similar reliability challenges as before. Another consideration is that our organization and target audience have changed considerably since last year. Rather than having a large paid staff to port the code, we will be relying on more limited resources, and ultimately the goal is to have the code live on as a volunteer-run project. So, it makes sense to implement the original rearchitecture goals, (i.e. 1--3 above), while not necessarily focusing on a complete reimplementation of the current Chandler. In fact, leaving out features, or implementing them minimally can be seen as creating opportunities for volunteer developers. For example, only supporting sharing via the Hub leaves it open to others to add support for other Cosmo servers, or for CalDAV in general, or for particular Atompub-based servers. With this in mind, a goal that Jeffrey, Phillip and I agreed would be achievable using OSAF's remaining paid staff time is: Achievable Goal ~ ~ ~ ~ ~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Create fully-tested interaction, domain and storage layers for Chandler:** This would include support for list and calendar views, stamping and the detail view, but would not include support for email/sharing. Besides achievability, this will will also provide promise that the full rearchitecture is feasible. It will also leave room for outside developers to volunteer, and/or build custom applications. As mentioned earlier, there is flexibility as to what we implement once the first three or four items above. As a result, we can envision: Possible Different Goals ~ ~ ~ ~ ~ ~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here are a couple of possible different routes we could take: **Sharing/"Chandler Lite":** Cut back, say, on some calendar features, but on a full application (i.e. up to presentation layer) that can sync a task list with Cosmo. **Calendar UI:** Implement enough of a calendar presentation layer to allow other developers to work on a small, self-contained calendar (including a detail view). There is also a parallel view of the rearchitecture, where we can look at user-level features that can be omitted in a first pass at either the interaction or domain layer. (Examples might be the minicalendar or preview area in current Chandler). This should obviously be discussed separately, especially once the basic infrastructure is in place. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Chandler Desktop IRC QA session for 1.0.1,Thursday September 4, 1:00 PM PDT

Grant Baillie — 2008-09-04T00:45:01

After tracking down a regression in one of the wx patches I had committed recently, I'm about to go ahead and roll a 1.0.1 build. So, we'll have an QA session to test 1.0.1 on IRC tomorrow, in #chandler, from 1-2 PM Pacific time. It would be good to verify the bugs that have been committed to this build: Also, we should go through the acceptance tests for Desktop builds: so any help I can get with the above will be seriously appreciated ;). Cheers, --Grant _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] First-round tbox shutdown proposal: 4 machines

Jared Rhine — 2008-09-03T18:29:56

Grant, in particular, but anyone's who is interested in tinderbox: As a followup to the other "post-tbox planning" thread, I propose we go ahead and power off: - dev-imac2 - mana - koa - hokulele These boxes appear to be not used directly in the production of Desktop builds. Shutting them off starts the process of tbox shutdown (hitting about a third of our boxes) without affecting the ability to produce Desktop releases at all. It'll give us a dry run of what tbox power-offs will do to the tinderbox pages, etc, and let me go in and collect these machines physically at my discretion.

[Chandler-dev] Checkin 9/3/2008

Mimi Yin — 2008-09-03T18:20:20

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] 1.0.1 build status

Grant Baillie — 2008-09-03T16:03:39

A quick update on 1.0.1: All of the required changes are ready to roll, but it looks as if one of the wx changes[1] is causing crashes on Linux, so unless I have a brilliant idea soon I'll be backing that one out, building a new wx, and then rolling 1.0.1. It'll take a couple of hours after that to produce new wx tarballs, but after that the actual 1.0.1 build will take 20-30 minutes. --Grant [1] https://bugzilla.osafoundation.org/show_bug.cgi?id=12231 (Dialogs should pop up on mouse-up instead of mousedown) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] IRC daemons moved

Jared Rhine — 2008-09-03T12:08:48

In preparation for a particular machine shut-off, the IRC daemons have been moved and rejiggered. Please report any problems you find. The archiving hasn't been carefully checked, but seems to work. [See below for details, but only "bug XXXX" and "~mark" are functional with soup. twisted-soup is dead. I don't think anyone had used it in ages; let me know if otherwise. soup and osaf-log have moved to a new server. (The "archive" server, which collects logs files and runs the dashboard and metrics.) We're running the Debian installs of "supybot" and "eggbot" (respectively), so some bugs probably have been fixed and we should now have security- fix support. supybot is now running (the default) twisted because the old "socketDrivers" setting doesn't work any more. [Time passes] Ok, things got complicated. Our soup plugins (courtesy of Mike, and some other alumni), were written for supybot 0.80, while there were API changes afterwards. These changes are minor, but to get rolling again, I needed to disable a bunch of local plugins and hack the code for the rest. Bear popped into IRC half-way through my coming to understand this situation and gave me the history and offers for help in the morning. Rolling back to 0.80 was/is an option, but in the mean time, I just pressed forward to port the "bug XXXX" functionality and "~mark" functionality. The tinderbox notification is down, but I'm not sure that's critical given current plans for the tbox farm. I'll sleep on next steps. Bear, for reference, both soup and osaf-log are managed via runit, so the "screen" hack is dead and both bots will auto-start on boot; check the runit logs for runtime output.

[Chandler-dev] Checkin 9/2/2008

Mimi Yin — 2008-09-02T19:54:28

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Next moves?

Marcelo de Moraes Serpa — 2008-09-01T06:54:48

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Simplifying the menus: Bug 12318

Mimi Yin — 2008-08-29T22:51:06

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Checkin 8/29/2008

Mimi Yin — 2008-08-29T18:36:32

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Checkin 8/28/2008

Mimi Yin — 2008-08-28T18:22:15

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Patch Hub?

Mimi Yin — 2008-08-28T18:21:31

Hi Jared, When do you think would be a good time to patch Hub? and/or roll another release of Chandler Server? There are a couple of important bug fixes that should go in: Sort triage status by TriageStatusChangedDate (Right now the NOW section is sorted upside down ;) - https://bugzilla.osafoundation.org/show_bug.cgi?id=11851 Add support for CTag, Apple's extension of ETag to a Calendar Collection - https://bugzilla.osafoundation.org/show_bug.cgi?id=12279 Jeffrey thought there might have been other fixes committed by Randy and Travis since the 1.0 release. Not sure if they are on trunk or in the 1.0 branch. (Not - sez Randy about his stuff.) Mimi _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Fix "Recover" pages?

Mimi Yin — 2008-08-28T12:18:51

Hi Jeffrey, Is there an easy way for me to fix the 2 recover pages for account activation and login? https://hub.chandlerproject.org/account/activation/recover https://hub.chandlerproject.org/account/password/recover I'm concerned that they don't look enough "a part" of the site which is unfortunate because they are both "security / identity" related pages. Also on Mac, the layout is screwy, which adds to the "Is this the right page?" feeling. Mimi _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Checkin 8/27/2008

Mimi Yin — 2008-08-27T18:46:32

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Hi all new at list!

Frank — 2008-08-27T02:05:01

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Chandler Hub is Down "Message"

Mimi Yin — 2008-08-26T18:58:02

http://chandlerproject.org/hub_down Hi Jared, Would it be possible to show the "Hub is down" message on a Chandler Hub page? (say the same one we use for account activation?) I can fix it if you point me to the right files. Mimi _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev

[Chandler-dev] Checkin 8/25/2008

Mimi Yin — 2008-08-25T18:47:38

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Open Source Applications Foundation "chandler-dev" mailing list http://lists.osafoundation.org/mailman/listinfo/chandler-dev