gmane.comp.search.xapian.devel

Re: GSoC xapian node binding

Liam — 2012-05-26T01:23:37


Actually, we can leave these function definitions in the header file; just
de-static them.

If we use a single DECLARE_POOL(Class) per class, the _convert function
will have to
  delete (GetMset_data*) data
since _done won't know the type to cast to.
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: Xapian license change

Wichert Akkerman — 2012-05-25T07:48:48

Hi Dan,

On 05/17/2012 06:29 PM, Dan Colish wrote:

I certainly wouldn't mind. One note I have to make is that I am going on 
a 4-week holiday in a couple of weeks which does limit my availability a 
bit.

Regards,
Wichert.

Re: GSoC xapian node binding

Liam — 2012-05-25T00:08:29


And a few more!

Use "that" instead of "pThis" as the name for the unwrapped args.This()

There are currently tests for mBusy in both _do_async and AsyncOp::AsyncOp.
We can remove the latter, and also make AsyncOp::AsyncOp take the unwrapped
object, rather than do the unwrap itself.

Use "Type aVar = value;" where possible, since we're removing try blocks
that required separate "Type aVar;".

The data argument for the _do functions should be named "data" vs "aData".

Could the DECLARE_POOL methods be defined once per class if the _process &
_convert functions are passed as args to the _do functions and then stored
in the AsyncOp object?
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: GSoC xapian node binding

Liam — 2012-05-23T20:35:02


Few more notes...

In op.h, tryCallCatch, GetInstance, sendToThreadPool, kBusyMsg should be
declared without static (since now used by multiple .cc files) and be
defined in util.cc or similar.

In the_do functions, since they no longer ThrowException()
  if (mBusy) throw Exception::Error(...)
similar to when _process returns error

Correcting previous code snippet to use Handle<Value>:
  Handle<Value> aResult;
  try { /// if we pull the try/catch from the _do functions
    aResult = aAsync ? GetMset_async(...) : GetMset_sync(...); /// or
GetMset_do(aAsync, ...)
  } catch (Handle<Value> ex) {
    ...
Local<Value> is for declaring a value in the current HandleScope. (Yes, my
orig code used Local in catch.)

GetMset_convert doesn't need a HandleScope, since callers have it.

Could we put the _data ptr into AsyncOp? That should eliminate the need for
OpInfo?

In your testing, make sure to trigger all the possible catch blocks. (You
might have to insert throws, e.g. to simulate a Xapian error.)
________________

Re: GSoC - Follow up Question about doc

Dan Colish — 2012-05-23T04:37:13

Ok, so I was really suggesting that most of the documentation that lives outside of our doxygen generated API documentation is consider to be higher level user docs. The actual user can vary widely from a new xapian user to core developers, but the intention of that documentation should be to describe in high level language the objects being documented. Describing all the methods might not be useful in that scenario. What's important is to capture the critical code path's and interfaces, and preferably show a few examples as well.
I don't think what you have do has been a waste at all; it just needs a little tweaking. All of the documentation that you've created to this point can be used, the question is where it really belongs.  As I said above, I think you need to split it into a few parts and focus on the audiences involved. For basic docs, you're usually writing for xapian end-users. For docs that cover internals at a high-level, you're writing for xapian developers. For docs that describe precise implem

Re: GSoC xapian node binding

Liam — 2012-05-23T03:12:14


Re latest code...


Commit message format:
  [add|fix|clean]: filename method (, ...) (, filename ...) - what was
added/fixed/cleaned

  add - create new functionality
  fix - repair existing functionality
  clean - no change in functionality
  If a LOT of methods are affected, the method list can be truncated to a
few key ones

  No need to change the messages you've already committed.


Should we create a single node-xapian.h header file? (Which I guess could
include the contents of op.h)


Code style:
   if/else and function opening brace on same line! (see op.h macro)
   local vars begin with letter a, to differentiate from function arguments


DECLARE_POOLS

  In the _do functions, perhaps the try/catch should move to the caller,
which would ThrowException() and delete aData on catch? (The only throw is
in AsyncOp::AsyncOp.) I'm not sure of the implications of returning
ThrowException to scope.Close() -- may be harmless; something to look into.
Even so, it's probably clearer to ThrowException from the

GSoC - Follow up Question about doc

Sehaj Singh Kalra — 2012-05-22T20:36:20

Dan Colish,

I am facing major issues in the documentation. Have invested quite some
time but not able to properly figure out what really needs to be done in
the documentation.

Can you please once again, clearly mention the objective of  a "user
documentation".
Also, can you please guide as to how to give an overall view? I am not able
to figure out what really needs to be added.

Also, it seems that I have wasted much of my time and energy in explaining
the things which were not meant to be explained. Have tried to removed them
from the doc.
This could have been avoided if I had come up with smaller chunks rather
than coming up with a single large chunk.

Have also added some replies regarding the comments you made on my github
commit. Reply to them when you can.
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: Xapian license change

Olly Betts — 2012-05-22T05:47:01

I'm rather too busy to at the moment, but I'm happy for others to work
on such a page (assuming it doesn't create a maintenance burden which
I end up shouldering, or spread misinformation which I then have to
spend time clearing up after).


Cool.


I guess you mean this:

http://trac.xapian.org/wiki/ProjectIdeas#Replacegetoptwithpoptorsimilar

That used to incorrect say "GPL" but I fixed in back in February.


Not quite.  We've not been asking longer term contributors (who have
already given us permission to relicense their contributions) to
dual-license up front.  This isn't a carefully thought out policy, there
just didn't seem a reason to do so.

It's a fairly subtle distinction, but for example a third party can't
just take code in Xapian written after a given date and use in under
MIT/X without investigating who wrote it.  (This isn't some deliberate
masterminded policy, just a consequence of the above).

But we do have agreement from most of the longer term contributors that
we can relicense, and mor

Re: Standing of the project

Parth Gupta — 2012-05-21T20:47:04

Hi Rishabh,



Sounds good. It makes sense that you and Gaurav share the same  evaluation
framework so that at the end we do not end up with two different evaluation
modules.

Of course, in our case the input to the evaluation module would be
Letor::Ranklist. So, if you overload the method like evaluate() which
returns the NDCG< at >k or MAP supplying a Letor::Ranklist (in our case) and
Xapian::MSet (in Gaurav's case). You can coordinate among yourselves.
Basically we need to use it a lot. While training, we optimise a loss
function and validate the model using the same evaluation module.



Great. Btw did you have look at the Matlab/Octave code written by Geoffrey
Hinton that is used in the Science paper. That's very informative and
understandable. Recently I played with it in some experiments and its quite
easy to follow.

Currently I am in Istanbul attending a conference so this week you might
not find me much around but most of the days I will try to answer e-mails
in the night.

Parth.

_____________________

Re: Iterators

Michael Uvarov — 2012-05-21T06:27:19

skip_to moves a cursor forward only.
That is why, I created a wrapper around iterators.

Here is a base class for them:
<https://github.com/freeakk/xapian/blob/master/c_src/termiter_gen.h>

And then I created few realizations of this interface:
<https://github.com/freeakk/xapian/blob/master/c_src/termiter_doc_gen.h>
<https://github.com/freeakk/xapian/blob/master/c_src/termiter_spy_gen.h>

After all, I can create an exemplar of one of this objects and store a
pointer on it as a property of the TermQlcTable.
<https://github.com/freeakk/xapian/blob/master/c_src/qlc_table.h>
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: Iterators

Olly Betts — 2012-05-21T01:30:18

You should be able to do skip_to("") on a TermIterator with no effect.
If it's the common case that first_term_name will be empty, then it's
probably worth checking for it to avoid the overhead of the no-op call:

    if (!first_term_name.empty())
it.skip_to(first_term_name);


Not sure I follow why this makes a difference...

Cheers,
    Olly

Re: Xapian-devel Digest, Vol 85, Issue 17

Sehaj Singh Kalra — 2012-05-19T13:28:13

James,

Thanks for pointing it out, will keep that in mind for future usage.

Thanks,
Sehaj Singh Kalra

On Sat, May 19, 2012 at 4:30 PM, <xapian-devel-request< at >lists.xapian.org>wrote:

_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: GSoC - QueryParser Information

James Aylett — 2012-05-19T07:52:45

The problem is how some email clients figure out URI/URL boundaries; to ensure it works enclose in <>, eg <http://trac.xapian.org/wiki/GSoC2012/QueryParser/Notes>. (This is in the URI RFC I think.)

J



On 19 May 2012, at 05:37, Sehaj Singh Kalra <sehaj.sk< at >gmail.com> wrote:

_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: GSoC - QueryParser Information

Sehaj Singh Kalra — 2012-05-19T04:37:11

The link doesn't seems to work. I think the suffix "." caused the problem.
Anyways, sending the link again.
http://trac.xapian.org/wiki/GSoC2012/QueryParser/Notes

On Sat, May 19, 2012 at 10:00 AM, Sehaj Singh Kalra <sehaj.sk< at >gmail.com>wrote:

_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

GSoC - QueryParser Information

Sehaj Singh Kalra — 2012-05-19T04:30:53

Have added information regarding queryparser at
http://trac.xapian.org/wiki/GSoC2012/QueryParser/Notes.
It is a bit informal at present but have a look, and let me know your
views. Will be improving it and adding some more things.
The aim as I talked with dcolish, is to explain the working of QueryParser
along with appropriate examples.
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: GSoC xapian node binding

Marius Tibeica — 2012-05-18T08:36:20


I tend to agree. The hybrid approach is even more complicated.

template <class dataType, class className>
class PoolHelper
{
public:
  ...
  template <Handle<Value> (*convertFunction)(className *data)>;
  static void execute_done(eio_req *data)
  {
    HandleScope scope;
    OpInfo* aInfo = (OpInfo*) req->data;
    dataType *aData = (dataType*)aInfo->data;
    AsyncOp<className> *aAsOp = (AsyncOp<className>*)aInfo->op;
    Handle<Value> argv[2];
    if (aAsOp->error) {
      argv[0] =
Exception::Error(String::New(aAsOp->error->get_msg().c_str()));
    } else {
      argv[0] = Null();
      argv[1] = convertFunction(aData);
    }
    tryCallCatch(aAsOp->callback, aAsOp->object->handle_, aAsOp->error ? 1
: 2, argv);
    delete aData;
    delete aAsOp;
    delete aInfo;
  }
};


#define DECLARE_POOLS(func,classn) \
...
static int func##_done(eio_req *req) {\
  HandleScope scope;\
  PoolHelper<func##_data,classn>::execute_pool<func##_convert>(req);\
  return 0;\
}\


I will leave the macro as it is.


The que

Re: Xapian license change

Dan Colish — 2012-05-17T16:29:25

Fair enough, maybe it was just my interpretation of the thread. Olly has be excellent at keeping track of the licensing issue since it does come up every so often.

This is a pretty good point. I had not thought about it much from the other side. I think I am inclined to use Olly's guideline that the rewritten code should be an improvement. That way we are able to gain more control our licensing and improving our code. It makes sense that just gaining more control is a worthwhile effort, but I would like to see the code improve if we're already making the effort to rewrite it. 

I should say that I'm not against relicensing. I just want it to be done for the right reasons and with a clear process. How would you feel about spearheading the organization of this effort? There are some good notes on this thread from Olly that could be rounded up into a working document.

Re: Xapian license change

Dan Colish — 2012-05-17T16:14:18

True, I think sponsored development is a good thing in most cases, but its a bit hard to do without a clear proposal. PyPy has done most of their work through academic grants and the money is always attached to very clears goals and steps. I would be hesitant to accept money without going through that same process.

Re: Xapian license change

Wichert Akkerman — 2012-05-17T15:47:39

Unfortunately not totally unexpected.


I do not think I am telling anyone how to do things, and that certainly 
is not my intention. So far I have only been asking for information 
about the status quo and which might need to be done for relicensing to 
happen, and Olly has been extremely helpful with providing that. I did 
already offer to do work myself in this area if I can help, as much as 
my time and skillset will allow. I have no expectation at all that other 
people will do anything just because I or someone else is asking them to.

I would also ask that you consider my point of view. With the current 
license unfortunately Xapian is not useful for me, so please do not 
expect me to invest energy into Xapian when that is not helping to 
scratch an itch I have. I am willing to work on Xapian, within and with 
the community as long at it helps me as well. That does include writing 
patches, updating documentation and everything else that might be needed.

Regards,
Wichert.

Re: Xapian license change

Charlie Hull — 2012-05-17T15:23:40

To rewind a little, at least one of the people asking about this is from 
a company interested in using Xapian in their products but unable to 
because of the GPL. They have indicated they may be able to fund some 
development towards this end. Surely sponsored development of Xapian is 
a good thing?

Cheers

Charlie

Re: Xapian license change

Dan Colish — 2012-05-17T15:16:26

The more I read on this thread the more I am confused. I'm starting to wonder who's asking for relicensing here because its starting to read like we, the Xapian developers, are asking how to relicense and what work we'd need to do. That is not the case. A request like this comes up once or twice a year and the requestor usually disappears a week later. 

You need to be a part of the community to have this discussion. I'm sure anyone can appreciate what it is like for an outsider to tell you how to do things. So please, join the community, get to know its members, and write patches or create documentation. Help us improve Xapian for a bit, then we can talk about why the licensing needs to change and how to go about it. Remember, our time is an extremely precious resource and I would like to use that time to make Xapian better rather than rewriting code so it can be relicensed.