gmane.comp.search.snowball

Re: Fwd: How to add Tamil Support to stemmer?

Damodharan Rajalingam — 2013-04-18T20:56:59

Hi Martin,

I have replaced simple or conditions with among() command as suggested.  I tested the modification against the data in https://github.com/rdamodharan/snowball-data/tree/master/tamil and the results were same. I have sent a pull request for it - https://github.com/snowballstem/snowball/pull/3

Thanks
Damu



________________________________
 From: Martin Porter <martin.f.porter< at >gmail.com>
To: Damodharan Rajalingam <r.damodharan< at >ymail.com> 
Cc: snowball-discuss< at >lists.tartarus.org 
Sent: Saturday, March 30, 2013 8:26 PM
Subject: Re: [Snowball-discuss] Fwd: How to add Tamil Support to stemmer?
 

I've added to clear link to the gmane archive of snowball-discuss on
the snowball home page --- long overdue.

To Damodharan Rajalingam: I would look into introducing among(...)
expressions into your snowball script. See the earlier message by me
forwarded to the list by Shrinivasan T

Re: Porter stemming not dealing well with -gist-gists

Martin Porter — 2013-04-17T05:47:21

Hi Marc,

The "Porter stemmer" is now frozen in time, but this could be a
something to add to the snowball "English stemmer". It's connected to
the fact that -ist is not, in general, removed. -ologist -> olog
could, however, be added.

I've put your suggestion on the snowball-discuss list,

Thanks, Martin



On Tue, Apr 16, 2013 at 6:29 PM, Marc Schipperheijn
<m.schipperheyn< at >gmail.com> wrote:

Snowball Plugin for Pygments Syntax Highlighter

Hajime Senuma — 2013-04-14T18:16:17

Hello all,

I wrote a Snowball lexer plugin for Pygments, a popular syntax highlighter
written with Python:
https://github.com/hajimes/pygments-snowball.

For ease of implementation, the lexer has a defect that it assumes
stringescapes are curly brackets, but it'll work for most Snowball codes.
I would be glad if you could use this to make codes in your html documents
more readable.
I'm also willing to receive any suggestions before I commit the lexer to
the official Pygments repo.

Below are two output examples of Pygments Snowball Plugin.

Default(-styled) Porter Snowball:
http://htmlpreview.github.io/?https://github.com/hajimes/pygments-snowball/blob/master/sample_output/porter.html

Colorful Russian Snowball (far more vivid than colorless green):
http://htmlpreview.github.io/?https://github.com/hajimes/pygments-snowball/blob/master/sample_output/russian_colorful.html

Thanks,
Hajime
_______________________________________________
Snowball-discuss mailing list
Snowball-discuss< at >lists.tartarus.org
http://li

Re: Adding "cela" to the list of French stopwords

Adrien Grand — 2013-04-11T22:02:20

On Thu, Apr 11, 2013 at 12:22 PM, Martin Porter
<martin.f.porter< at >gmail.com> wrote:

Thank you very much Martin!

Regarding the 0xA0, in French, some punctuation marks must to preceded
by a non-breaking space[1]. Maybe J-C Deschamps copied it from an
existing text and didn't notice he copied the non-breaking space too.

[1] http://en.wikipedia.org/wiki/Punctuation#Other_languages

--
Adrien

Re: Adding "cela" to the list of French stopwords

Martin Porter — 2013-04-11T10:22:57

changes in place (Can't imagine where the 0xA0 cam from.)

Re: Adding "cela" to the list of French stopwords

Martin Porter — 2013-04-07T11:00:47

Thanks Adrien. I'll make the changes in the next few days. M.

Re: Adding "cela" to the list of French stopwords

Adrien Grand — 2013-04-07T08:55:22

Hi Martin,

On Sun, Apr 7, 2013 at 9:20 AM, Martin Porter <martin.f.porter< at >gmail.com> wrote:

Although incorrect, "celà" is very common. Maybe we should rather have
both "cela" and "celà" in the stopwords list?

I think the non-breakable space (0xA0) right after "celà" should be removed too?


There is nothing to excuse, your French is perfect!

Re: Adding "cela" to the list of French stopwords

Martin Porter — 2013-04-07T07:20:47

Adrien,

Dans la liste à http://snowball.tartarus.org/algorithms/french/stop.txt il y a,

               | Later additions (from Jean-Christophe Deschamps)
ceci         |  this
celà         |  that

mais je pense que "celà" n'est pas juste. A la page
http://fr.wikipedia.org/wiki/Pronom_d%C3%A9monstratif je lis,

"Du point de vue orthographique, contre toute attente, le pronom «
cela » s'écrit avec un « a » sans accent grave"

Je pense que je dois inclure "cela" et éliminer "celà".

Voulez vous le confirmer?

Anyway, I'm sure you'll excuse my bad French! It is a strange error,
because I took the word as an extra from J-C Deschamps, who must be a
French speaker, and also no one has commented on it before.

Martin

Adding "cela" to the list of French stop words

Adrien Grand — 2013-04-06T14:54:16

Hi,

I am working on Lucene and Solr, which use Snowball's stop words lists
for their analyzers. One of our users, Pierre Kobylanski, suggested to
add "cela" to the list of French stop words:
  https://issues.apache.org/jira/browse/LUCENE-4911

I thought you would be interested to add it to
http://svn.tartarus.org/snowball/trunk/website/algorithms/french/stop.txt
as well. Don't hesitate to let me know if I can help get this
included.

Best regards,

--
Adrien

Re: JavaScript version of the stemmer

Martin Porter — 2013-04-02T08:45:44

It is a bit confusing, but the snowball system is a later development.
snowball goes back to 2002, the porter stemmer page was put up some
time in the 1990s, I can't recall exactly when.

Re: JavaScript version of the stemmer

Martin Porter — 2013-04-01T12:38:01

By "the stemmer" do you mean the Porter stemmer? If so, there's a link
to the andargor/mckenzie authored one from

http://tartarus.org/~martin/PorterStemmer/index.html

On Sun, Mar 31, 2013 at 9:38 PM, John Gage <jsmgage< at >gmail.com> wrote:

JavaScript version of the stemmer

John Gage — 2013-03-31T20:38:53

Is there a JavaScript version of the stemmer lurking out there?  Please
note that there is a server and a client/browser version of JavaScript, the
former being nodejs.

John Gage
_______________________________________________
Snowball-discuss mailing list
Snowball-discuss< at >lists.tartarus.org
http://lists.tartarus.org/mailman/listinfo/snowball-discuss

Re: Fwd: How to add Tamil Support to stemmer?

Martin Porter — 2013-03-30T14:56:48

I've added to clear link to the gmane archive of snowball-discuss on
the snowball home page --- long overdue.

To Damodharan Rajalingam: I would look into introducing among(...)
expressions into your snowball script. See the earlier message by me
forwarded to the list by Shrinivasan T

Re: Fwd: How to add Tamil Support to stemmer?

Damodharan Rajalingam — 2013-03-28T10:29:41

Hi Richard,
I have added documentation on the algorithm here - https://github.com/rdamodharan/tamil-stemmer/tree/master/docs. I have also added vocabulary set for Tamil and sent a pull request - https://github.com/snowballstem/snowball-data/pull/1.

Please let me know if anything is missing or wrong :)

Thanks
Damu



________________________________
 From: Damodharan Rajalingam <r.damodharan< at >ymail.com>
To: Richard Boulton <richard< at >tartarus.org>; Shrinivasan T <tshrinivasan< at >gmail.com> 
Cc: "snowball-discuss< at >lists.tartarus.org" <snowball-discuss< at >lists.tartarus.org> 
Sent: Wednesday, March 27, 2013 9:35 PM
Subject: Re: [Snowball-discuss] Fwd: How to add Tamil Support to stemmer?
 

Hi Richard,
I am working on getting the sample dataset. I did this work for my Master's thesis 4 years back so I am having difficulty in spotting some files :) . I will cleanup the dataset I used for evaluation and add it to the git repo along with the documentation on evaluation results.

Thanks
Damu




___________________________

Re: Fwd: How to add Tamil Support to stemmer?

Damodharan Rajalingam — 2013-03-27T16:05:06

Hi Richard,
I am working on getting the sample dataset. I did this work for my Master's thesis 4 years back so I am having difficulty in spotting some files :) . I will cleanup the dataset I used for evaluation and add it to the git repo along with the documentation on evaluation results.

Thanks
Damu




________________________________
 From: Richard Boulton <richard< at >tartarus.org>
To: Shrinivasan T <tshrinivasan< at >gmail.com> 
Cc: snowball-discuss< at >lists.tartarus.org 
Sent: Wednesday, March 27, 2013 6:31 PM
Subject: Re: [Snowball-discuss] Fwd: How to add Tamil Support to stemmer?
 
On 27 March 2013 12:03, Shrinivasan T <tshrinivasan< at >gmail.com> wrote:

"rdamodharan" has actually done exactly what's needed for this, by
submitting a pull request on github to our repository;
https://github.com/snowballstem/snowball/pull/2  Unfortunately, I
haven't had a chance to look at this so far; I will make sure to make
time to do so over the next few days.

I have no way of evaluating the results of this stemmer, but am
wil

Re: Fwd: How to add Tamil Support to stemmer?

Shrinivasan T — 2013-03-27T13:07:26

Thanks for the update.

I sent this to damo.

Hope me may work on this further.

Re: Fwd: How to add Tamil Support to stemmer?

Richard Boulton — 2013-03-27T13:01:38

"rdamodharan" has actually done exactly what's needed for this, by
submitting a pull request on github to our repository;
https://github.com/snowballstem/snowball/pull/2  Unfortunately, I
haven't had a chance to look at this so far; I will make sure to make
time to do so over the next few days.

I have no way of evaluating the results of this stemmer, but am
willing to take the word of Tamil speakers as to whether the algorithm
is of use.  There may be some changes to the code that should be made
to improve performance, as Martin mentioned.  One thing that would be
of great use is a sample dataset, similar to that in
https://github.com/snowballstem/snowball-data/blob/master/english/voc.txt,
together with a sample file containing the corresponding expected
output.

Re: Make the list archives public

Shrinivasan T — 2013-03-27T12:51:17


http://snowball.tartarus.org/lists.php

This page does not contain the valid links to the gmane archives.
they lead to mailman page only.

Please check the links in that page and fix them.

Thanks.

Re: Make the list archives public

Richard Boulton — 2013-03-27T12:45:47

The archives on lists.tartarus.org are private explicitly because we
don't want to host them - mailman's web archive format isn't very
usable; gmane is the official site for snowball archives (as far as
I'm concerned!)

We link to the archives on gmane from
http://snowball.tartarus.org/lists.php , and don't deliberately link
to http://lists.tartarus.org/mailman/private/snowball-discuss/ - if
there's a link to that on the snowball site somewhere, let me know and
I'll redirect it to gmane!

Re: Make the list archives public

Martin Porter — 2013-03-27T12:38:13

Richard Boulton must answer that one, but meanwhile all of
snowball-discuss can be found at

http://news.gmane.org/gmane.comp.search.snowball

Make the list archives public

Shrinivasan T — 2013-03-27T12:05:37

Hello all,

I am new to the list.

I am wondering why our list archives are private?
http://lists.tartarus.org/mailman/private/snowball-discuss/

Please make them public so that it reaches all the public.

Thanks.