gmane.comp.search.xapian.devel

Re: Xapian license change

Olly Betts — 2012-05-22T05:47:01

I'm rather too busy to at the moment, but I'm happy for others to work
on such a page (assuming it doesn't create a maintenance burden which
I end up shouldering, or spread misinformation which I then have to
spend time clearing up after).


Cool.


I guess you mean this:

http://trac.xapian.org/wiki/ProjectIdeas#Replacegetoptwithpoptorsimilar

That used to incorrect say "GPL" but I fixed in back in February.


Not quite.  We've not been asking longer term contributors (who have
already given us permission to relicense their contributions) to
dual-license up front.  This isn't a carefully thought out policy, there
just didn't seem a reason to do so.

It's a fairly subtle distinction, but for example a third party can't
just take code in Xapian written after a given date and use in under
MIT/X without investigating who wrote it.  (This isn't some deliberate
masterminded policy, just a consequence of the above).

But we do have agreement from most of the longer term contributors that
we can relicense, and mor

Re: Standing of the project

Parth Gupta — 2012-05-21T20:47:04

Hi Rishabh,



Sounds good. It makes sense that you and Gaurav share the same  evaluation
framework so that at the end we do not end up with two different evaluation
modules.

Of course, in our case the input to the evaluation module would be
Letor::Ranklist. So, if you overload the method like evaluate() which
returns the NDCG< at >k or MAP supplying a Letor::Ranklist (in our case) and
Xapian::MSet (in Gaurav's case). You can coordinate among yourselves.
Basically we need to use it a lot. While training, we optimise a loss
function and validate the model using the same evaluation module.



Great. Btw did you have look at the Matlab/Octave code written by Geoffrey
Hinton that is used in the Science paper. That's very informative and
understandable. Recently I played with it in some experiments and its quite
easy to follow.

Currently I am in Istanbul attending a conference so this week you might
not find me much around but most of the days I will try to answer e-mails
in the night.

Parth.

_____________________

Re: Iterators

Michael Uvarov — 2012-05-21T06:27:19

skip_to moves a cursor forward only.
That is why, I created a wrapper around iterators.

Here is a base class for them:
<https://github.com/freeakk/xapian/blob/master/c_src/termiter_gen.h>

And then I created few realizations of this interface:
<https://github.com/freeakk/xapian/blob/master/c_src/termiter_doc_gen.h>
<https://github.com/freeakk/xapian/blob/master/c_src/termiter_spy_gen.h>

After all, I can create an exemplar of one of this objects and store a
pointer on it as a property of the TermQlcTable.
<https://github.com/freeakk/xapian/blob/master/c_src/qlc_table.h>
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: Iterators

Olly Betts — 2012-05-21T01:30:18

You should be able to do skip_to("") on a TermIterator with no effect.
If it's the common case that first_term_name will be empty, then it's
probably worth checking for it to avoid the overhead of the no-op call:

    if (!first_term_name.empty())
it.skip_to(first_term_name);


Not sure I follow why this makes a difference...

Cheers,
    Olly

Re: Xapian-devel Digest, Vol 85, Issue 17

Sehaj Singh Kalra — 2012-05-19T13:28:13

James,

Thanks for pointing it out, will keep that in mind for future usage.

Thanks,
Sehaj Singh Kalra

On Sat, May 19, 2012 at 4:30 PM, <xapian-devel-request< at >lists.xapian.org>wrote:

_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: GSoC - QueryParser Information

James Aylett — 2012-05-19T07:52:45

The problem is how some email clients figure out URI/URL boundaries; to ensure it works enclose in <>, eg <http://trac.xapian.org/wiki/GSoC2012/QueryParser/Notes>. (This is in the URI RFC I think.)

J



On 19 May 2012, at 05:37, Sehaj Singh Kalra <sehaj.sk< at >gmail.com> wrote:

_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: GSoC - QueryParser Information

Sehaj Singh Kalra — 2012-05-19T04:37:11

The link doesn't seems to work. I think the suffix "." caused the problem.
Anyways, sending the link again.
http://trac.xapian.org/wiki/GSoC2012/QueryParser/Notes

On Sat, May 19, 2012 at 10:00 AM, Sehaj Singh Kalra <sehaj.sk< at >gmail.com>wrote:

_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

GSoC - QueryParser Information

Sehaj Singh Kalra — 2012-05-19T04:30:53

Have added information regarding queryparser at
http://trac.xapian.org/wiki/GSoC2012/QueryParser/Notes.
It is a bit informal at present but have a look, and let me know your
views. Will be improving it and adding some more things.
The aim as I talked with dcolish, is to explain the working of QueryParser
along with appropriate examples.
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: GSoC xapian node binding

Marius Tibeica — 2012-05-18T08:36:20


I tend to agree. The hybrid approach is even more complicated.

template <class dataType, class className>
class PoolHelper
{
public:
  ...
  template <Handle<Value> (*convertFunction)(className *data)>;
  static void execute_done(eio_req *data)
  {
    HandleScope scope;
    OpInfo* aInfo = (OpInfo*) req->data;
    dataType *aData = (dataType*)aInfo->data;
    AsyncOp<className> *aAsOp = (AsyncOp<className>*)aInfo->op;
    Handle<Value> argv[2];
    if (aAsOp->error) {
      argv[0] =
Exception::Error(String::New(aAsOp->error->get_msg().c_str()));
    } else {
      argv[0] = Null();
      argv[1] = convertFunction(aData);
    }
    tryCallCatch(aAsOp->callback, aAsOp->object->handle_, aAsOp->error ? 1
: 2, argv);
    delete aData;
    delete aAsOp;
    delete aInfo;
  }
};


#define DECLARE_POOLS(func,classn) \
...
static int func##_done(eio_req *req) {\
  HandleScope scope;\
  PoolHelper<func##_data,classn>::execute_pool<func##_convert>(req);\
  return 0;\
}\


I will leave the macro as it is.


The que

Re: Xapian license change

Dan Colish — 2012-05-17T16:29:25

Fair enough, maybe it was just my interpretation of the thread. Olly has be excellent at keeping track of the licensing issue since it does come up every so often.

This is a pretty good point. I had not thought about it much from the other side. I think I am inclined to use Olly's guideline that the rewritten code should be an improvement. That way we are able to gain more control our licensing and improving our code. It makes sense that just gaining more control is a worthwhile effort, but I would like to see the code improve if we're already making the effort to rewrite it. 

I should say that I'm not against relicensing. I just want it to be done for the right reasons and with a clear process. How would you feel about spearheading the organization of this effort? There are some good notes on this thread from Olly that could be rounded up into a working document.

Re: Xapian license change

Dan Colish — 2012-05-17T16:14:18

True, I think sponsored development is a good thing in most cases, but its a bit hard to do without a clear proposal. PyPy has done most of their work through academic grants and the money is always attached to very clears goals and steps. I would be hesitant to accept money without going through that same process.

Re: Xapian license change

Wichert Akkerman — 2012-05-17T15:47:39

Unfortunately not totally unexpected.


I do not think I am telling anyone how to do things, and that certainly 
is not my intention. So far I have only been asking for information 
about the status quo and which might need to be done for relicensing to 
happen, and Olly has been extremely helpful with providing that. I did 
already offer to do work myself in this area if I can help, as much as 
my time and skillset will allow. I have no expectation at all that other 
people will do anything just because I or someone else is asking them to.

I would also ask that you consider my point of view. With the current 
license unfortunately Xapian is not useful for me, so please do not 
expect me to invest energy into Xapian when that is not helping to 
scratch an itch I have. I am willing to work on Xapian, within and with 
the community as long at it helps me as well. That does include writing 
patches, updating documentation and everything else that might be needed.

Regards,
Wichert.

Re: Xapian license change

Charlie Hull — 2012-05-17T15:23:40

To rewind a little, at least one of the people asking about this is from 
a company interested in using Xapian in their products but unable to 
because of the GPL. They have indicated they may be able to fund some 
development towards this end. Surely sponsored development of Xapian is 
a good thing?

Cheers

Charlie

Re: Xapian license change

Dan Colish — 2012-05-17T15:16:26

The more I read on this thread the more I am confused. I'm starting to wonder who's asking for relicensing here because its starting to read like we, the Xapian developers, are asking how to relicense and what work we'd need to do. That is not the case. A request like this comes up once or twice a year and the requestor usually disappears a week later. 

You need to be a part of the community to have this discussion. I'm sure anyone can appreciate what it is like for an outsider to tell you how to do things. So please, join the community, get to know its members, and write patches or create documentation. Help us improve Xapian for a bit, then we can talk about why the licensing needs to change and how to go about it. Remember, our time is an extremely precious resource and I would like to use that time to make Xapian better rather than rewriting code so it can be relicensed.

Re: GSoC xapian node binding

Liam — 2012-05-17T15:02:58


OK, no need to bend over backwards. Simplicity is preferable after all.
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: GSoC xapian node binding

Marius Tibeica — 2012-05-17T09:38:36

I have tried multiple approaches.
The difficulty comes from the fact that each class can have multiple
sync/async functions. For example, Enquire has atleast get_mset,
get_matching_terms and get_eset.
Another problem is that from the pool methods (which should be templatized)
I have to call methods from the class (GetMset_process, GetMset_convert,
GetMatchingTerm_process, ...). This means that I also probably have to pass
a function as a template parameter.
The thirt problem is that func##_pool and func##_done need to be (int
(eio_req *req)), so I can not pass a function pointer.

At the moment I am trying a hybrid approach (bot templates and macroes).

On Wed, May 16, 2012 at 8:10 PM, Liam <xapian< at >networkimprov.net> wrote:

_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: Xapian license change

Charlie Hull — 2012-05-17T08:41:50

<snip>

I would suggest that the least pain-free route is to consider rewriting 
these parts. Knowing like you do the people involved I doubt it will be 
easy to get a clear answer or agreement.

I would suggest the same approach although I don't know the people involved.

Cheers

Charlie

Re: Xapian license change

Wichert Akkerman — 2012-05-17T08:04:18

Hi Olly,

there is a lot of information in your email that I was not able to find 
anywhere else. Perhaps it would be useful to create a relicensing-FAQ 
page on the wiki with this information?


On 05/17/2012 04:07 AM, Olly Betts wrote:

I don't know exactly how much time I can contribute and I certainly will 
not claim to be an expert on the underlying technology, but I am 
interested and willing to help.


That helps. There was a wiki page which explicitly mentioned wanting to 
replace getopt with something like popt for licensing reasons, which is 
why I mentioned it.


Looking at commit logs that means everything since November 13 2009 is 
already dual licensed.


Do you have a list of the code still owned by those three, so that can 
be turned into a TODO list? A quick grep shows 100 files with a 
BrightStation PLC copyright but nothing more recent than 2001, which 
suggests that a lot of that code has already been replaced over time.


Are those dependencies required or optional? If the intention is t

Re: Iterators

Michael Uvarov — 2012-05-17T05:04:10

Thanks for help.
I just use another iterator and move in the beginning of it with
skip_to(first_term_name);  // Can first_term_name be just ""?
It allows to use one class for term iteration both for ValueCountMatchSpy
and Document.
_______________________________________________
Xapian-devel mailing list
Xapian-devel< at >lists.xapian.org
http://lists.xapian.org/mailman/listinfo/xapian-devel

Re: Xapian license change

Olly Betts — 2012-05-17T02:07:44

The main way people can help is to contribute replacements for things
which we can't relicense.  That really requires development expertise
(or funds to pay for development expertise).

So far most of the progress has happened when an better replacement
has been written for some section of the code (e.g. recently I replaced
Xapian::Query - the main motivation was that the old implementation had
several rather fundamental problems, but this also means that
Xapian::Query can now be relicensed).

What's best to work on depends on what your skills and interests are.
If anyone is seriously interested in contributing in this way, it
probably makes most sense to simply discuss what's best for you to
work on, either here, or on IRC.


Actually, GNU getopt is LGPL v2 or later (or at least so says the
licensing boilerplate on the version we're using).


I don't think we have a totally firm decision currently, but it is
likely we'd go for either LGPL v2+ or MIT/X (with the latter probably
being the favourite).  We cur

Re: Iterators

Olly Betts — 2012-05-16T23:33:54

The input iterators like TermIterator have the standard C++
input iterator semantics, so advancing x invalidates y.  There's not
a direct way to clone an iterator - you'd have to get a fresh one from
the parent object and call skip_to() on it to advance it to the same
position.

The bidirectional iterators (like MSetIterator and ESetIterator) can 
be cloned just by making a copy.

Cheers,
    Olly