gmane.comp.gnu.aspell.user

Re: Hyphens and apostrophes in words

Lars Aronsson — 2013-05-21T20:40:27

I think that you started out with the goal of making a
spell checker, and for that you need tokenization, and
so you need to distinguish letters and space and
punctuation. But for grammar checking or translation,
you need to be able to find phrases and patterns, not
just tokens. Perhaps the basic layer of a larger, more
generic natural language processing library should have
a matching algorithm that isn't based on tokenization.
I have no idea if such libraries exist or what the current
state of the art is. But I think it's natural for a dictionary
to contain both words (cat, dog) and phrases (kill two
birds with one stone), so that "kill two dogs with one
stone" will generate a warning that perhaps the user
meant to write "birds" there. That would certainly be
another type of application than Aspell, but perhaps
a more useful one.

Re: Hyphens and apostrophes in words

Kevin Atkinson — 2013-05-21T14:55:53

Hi All,

Aspell author here.

Aspell does not support what I think you are after very well. That is 
why it is not enabled by default in the English dictionary.  For some 
insight on why see: 
http://aspell.net/man-html/Words-With-Symbols-in-Them.html

Changing the language data file should be able to force Aspell to accept 
them when compiling a dictionary but Aspell won't necessary be able to 
check them within a document.

Rather than modifying the data file, when compiling a dictionary you can 
force Aspell to accept any word by using the "--dont-validate-words" 
argument.  To spellchecker with these special words you need to keep 
Aspell from trying to tokenize a string into words (i.e, "hello world!" 
gets split into "hello" and "world").  I think you need to use the C API 
for this.

I hope this all makes sense.

Kevin.

Re: Hyphens and apostrophes in words

Ciarán Ó Duibhín — 2013-05-21T14:29:31

Thanks, Carlo, I really appreciate your help.


If you mean "aspell create", I did that:
I didn't remake the whole English dictionary yet, but I'm not using it in my 
tests, only a tiny dictionary called "variant" with half-a-dozen words.


That was helpful and prompted me to look at the Italian dictionary.  But 
what I found was not nice.  There is nothing in "it.dat" about "special" 
treatment for the apostrophe.  There are no words in aspell's Italian 
dictionary with an apostrophe in them.  But Italian words ending in 
apostrophe are included in the dictionary MINUS the apostrophe, whether or 
not that is a valid word.  So it seems that the validation by aspell of 
Italian words ending in apostrophe depends on this very behaviour, which I 
deduce must therefore be the same in all versions of aspell - at least by 
default.

The question then reduces for me to whether there is an option of setting 
aside this default behaviour of aspell, so as to keep word-marginal 
apostrophes when breaking up text into words for comparison with the 
dictionary.

Ciarán Ó Duibhín.

Re: SPAM Re: Hyphens and apostrophes in words

Carlo Traverso — 2013-05-20T16:38:34

It is my understanding that changing just the .dat file is not enough,
one has to recompile the dictionary from the (modified) sources. 

I know by experience that the Italian aspell dictionary recognizes
words with trailing apostrophes, the French dictionary recognizes
words with embedded hyphen, and the Latin and German dictionaries
split words at the embedded apostrophe, conforming to the description
of their .dat files. I suppose that the English dictionary does not
allow leading or trailing apostrophes because of the current English
use of apostrophe as quote character (while in other languages single
quotes are unusual).

Carlo Traverso

Re: Hyphens and apostrophes in words

Ciarán Ó Duibhín — 2013-05-20T15:39:56

On to testing problem no 3.  It's much the same process, using aspell 0.50.3 
for Windows as before.

I open the command prompt window,
go to folder C:\Program Files\Aspell\data,
change "en.dat" from "special ' -*-" to "special ' ***".

I go to C:\Program Files\Aspell\bin,
put the words "some odd words well more" in a text file "variant.dat",
run "aspell --lang en create master ./variant < variant.dat" ,
run "copy variant. ..\dict\variant."

I put some test text into "testaspell.txt":
      some odd words xtekt well 'well more

Now I run "aspell check --master variant testaspell.txt" and only the word 
xtekt is queried.  The word 'well is not queried.

What problems 2 and 3 have in common is that the dictionary seems to be OK, 
the right words go into it. But when the text is being checked against the 
dictionary, it would explain the experiments if the apostrophe is being 
removed from the text word before comparing with the dictionary (despite the 
special setting in en.dat).  I don't see any trace mode to verify this.  If 
this IS what is happening, is there something I can set to prevent it?

It would be really helpful if someone could try these tests with aspell 0.60 
and tell me what happens.

Ciarán Ó Duibhín.

Re: Hyphens and apostrophes in words

Ciarán Ó Duibhín — 2013-05-19T14:56:52

Thanks for replying, Carlo.

I will repeat my tests,  I begin with problem no 2. Remember, I am using 
aspell 0.50.3 for Windows.

I open the command prompt window, go to folder C:\Program Files\Aspell\data, 
I change in "en.dat" from "special ' -*-" to "special ' ***".

I put the words "some odd words 'twas more" in a text file "variant.dat" and 
run "aspell --lang en create master ./variant < variant.dat" followed by 
"copy variant. ..\dict\variant".

I put some test text into "testaspell.txt": some odd words xtekt 'twas more

Now I run "aspell check --master variant testaspell.txt" and two words are 
queried: xtekt and 'twas.  For 'twas, the suggested replacement is 'twas !

This is the same result I got before. Have I left something out? Would this 
work differently on other systems or versions?

Ciarán Ó Duibhín.

----- Original Message ----- 
From: "Carlo Traverso" <traverso< at >posso.dm.unipi.it>
To: <ciaran< at >oduibhin.freeserve.co.uk>
Cc: <aspell-user< at >gnu.org>
Sent: Sunday, May 19, 2013 11:19 AM
Subject: Re: [Aspell-user] Hyphens and apostrophes in words


> Carlo Traverso

Re: Hyphens and apostrophes in words

Carlo Traverso — 2013-05-19T10:19:14


    ciaran> I'd like to know which, if any, spellcheckers can be
    ciaran> configured to act like this.  (The examples are from
    ciaran> English but the real need comes from other languages.)
    ciaran> Asking here about aspell particularly, of course.

    ciaran> First, if necessary, allow the dictionary to contain words
    ciaran> with apostrophe "'" and hyphen "-" in any position. (I am
    ciaran> aware of the side-effects of this and am not worried by
    ciaran> them.)

    ciaran> Now, when checking text:

    ciaran> 1. Accept a word containing a hyphen if EITHER the
    ciaran> dictionary contains the whole word including the hyphen
    ciaran> ("hotch-potch") OR if the dictionary contains both parts
    ciaran> separately ("half-moon").

    ciaran> 2. With a dictionary containing "'twas" but not "twas",
    ciaran> accept "'twas".

    ciaran> 3. With a dictionary containing "well" but not "'well",
    ciaran> not accept "'well".

aspell can do 2 and 3, (but you have to recompile the English
dictionary after changing the handling of ' in the .dat file; and of
course add the acceptable words; this is the aspell way to do your
"First" point).

For 1, you should modify the .dat file again allowing - in the middle
of a word, add the composed words, and pass the spell-checker twice,
once with the modified dictionary, (to accept the words with -) once
with the original one (or rather the one modified in the first step)
to accept the two components. The first pass will refuse the words
with - not included, the second pass will split their components and
check again.

I don't think that it is possible to do it with one pass, combining
the two dictionaries in one .multi file since the .dat have to be
different (and hence the word tokens will be different). 

Carlo Traverso

Hyphens and apostrophes in words

Ciarán Ó Duibhín — 2013-05-18T23:14:51

I'd like to know which, if any, spellcheckers can be configured to act like this.  (The examples are from English but the real need comes from other languages.)  Asking here about aspell particularly, of course.

First, if necessary, allow the dictionary to contain words with apostrophe "'" and hyphen "-" in any position. (I am aware of the side-effects of this and am not worried by them.)

Now, when checking text:

1. Accept a word containing a hyphen if EITHER the dictionary contains the whole word including the hyphen ("hotch-potch") OR if the dictionary contains both parts separately ("half-moon").

2. With a dictionary containing "'twas" but not "twas", accept "'twas".

3. With a dictionary containing "well" but not "'well", not accept "'well".

That's what I would like to happen, and it seems to me not unreasonable.

In some experiments, Aspell (0.50-3 in Windows) did not do what I wanted in any of these cases.  But perhaps I'm omitting to do something, or perhaps other implementations behave differently, or later versions.
The spellchecker in MSWord passes all the tests, but unfortunately it is not available for use in other applications.
I haven't been able to test Hunspell.

I would be interested to hear anyone else's evidence or experiences.

Ciarán Ó Duibhín.

Can't build for MSWindows

Vincent Belaïche — 2013-04-28T17:25:44

Hello,

I am on MSWindows with MSYS installed. I tried to install the aspell
English dictionnary, but I got the following error:

-----------------------------------------------------------------------
/c/Programme/GNU/installation/aspell6-en-7.1-0>./configure
Finding Dictionary file location ... c:/Programme/GNU/Aspell/dict
Finding Data file location ... c:/Programme/GNU/Aspell/data
/c/Programme/GNU/installation/aspell6-en-7.1-0>make
/usr/bin/prezip-bin -d < en-common.cwl | /usr/bin/aspell  --lang=en create master ./en-common.rws
Error: The key "affix" is unknown.
make: *** [en-common.rws] Error 1
-----------------------------------------------------------------------

Then I though that maybe my aspell is too old, my current version is:

-----------------------------------------------------------------------
< at >(#) International Ispell Version 3.1.20 (but really Aspell 0.50.3)
-----------------------------------------------------------------------

but I could not install the latest version 0.60.6.1, attached is my
build log.

The configure steps gives only a warning, but the make fails. 

Any feedback or help is welcome...

   Vincent.

_______________________________________________
Aspell-user mailing list
Aspell-user< at >gnu.org
https://lists.gnu.org/mailman/listinfo/aspell-user

User Instructions

Parker,Christopher S — 2013-04-18T17:09:13

Please improve the instructions  of how to install this. I consider myself a pretty savvy computer user and I could not figure out how to install this spell checker into Notepadd++. it took several Google searches and a YouTube video for me to finally find out how to install this. This should not be that dificult.

Christopher Parker
CITE Web Developer
Teachers Assistant
(630)551-6662
s505214< at >mail.nwmissouri.edu
259 South Complex
Maryville, Mo 64468

en-common.rws problem

Ron Catterall — 2013-03-11T01:14:12

just loaded aspell-0.60.6.1
and aspell-en-7.1-0
(No problems reported on installs)

when I try it I get:

sh-3.2# pwd
/Applications/Utilities/aspell/aspell-0.60.6.1

sh-3.2# ./aspell --validate-words -d 
/Applications/Utilities/aspell/aspell-0.60.6.1/aspell6-en-7.1-0/american.alias 
check README
Error: The file 
"/Applications/Utilities/aspell/aspell-0.60.6.1/aspell6-en-7.1-0/en-common.rws" 
can not be opened for reading.
sh-3.2#

I have ...../aspell6-en-7.1-0/en-common.cwl in my directory
but I do not have en-common.rws anywhere on my machine

Mac OS 10.7.5, (running as root)

Any suggestions welcome

thanks

Ron

spellchecker

Shlomo Aran — 2013-03-03T19:52:04

Hi,
I am using jquery-spellchecker on Windows (with winamp).
I downloaded an Hebrew dictonary from
ftp://ftp.gnu.org/gnu/aspell/dict/0index.html
but I cannot figure out how ton install it.
I found this command: word-list-compress.exe d he.cwl | aspell --lang="he"
create master ./he.rws
in
http://www.sevenever.com/glog/Sevenever/post/Install_aspell_dictionary_on_Windows
But it does not work - it dows not create the he.rws

Thanks for any help.
Shlomo;

french typographic apostrophe

damien.thiriet77 — 2013-03-02T19:17:30

Hello,


I would like how can I make aspell French spelling understand, that french typographic apostrophe ’ is not part of words but a word separator. When spell-checking French words, I have a lot of warnings that make me loose much time, for example l’Europe d’ici and so on are highlighted as errors.
I could not understand which aspell option might be used to introduce ’ as not being part of a word. As far as i understood, aspell reads lEurope, that is indeed a mistake. Note that this typographic apostrophe should be used in French, not the English one, and that it is never used for quoting anything.
Thank you for any help,


Damien Thiriet

Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ?
Je crée ma boîte mail www.laposte.net

_______________________________________________
Aspell-user mailing list
Aspell-user< at >gnu.org
https://lists.gnu.org/mailman/listinfo/aspell-user

Cannot add words with spaces in the dictionary

Huren.nl Daniel — 2013-03-01T12:09:26

Hi Aspell,

 

We installed Aspell.net in our Sphinx search. Now we have the problem that
we cannot add words with spaces in the dictionary. It does not recognize it,
you get an error. This means that we can never give a suggestion (Did you
mean) of a 'product' that exists of multiple words. 

 

For example

Hydraulic breaker

Truck-mounted crane 

Rough terrain crane

 

This is the mistake I get:

Warning: pspell_new_config(): PSPELL couldn't open the dictionary. reason:
/home/sites3/site342/web/upload/custom.pws: The word "hydraulische
breekhamer" is invalid. The character ' ' (U+20) may not appear in the
middle of a word. in
/home/sites3/site342/web/templates/products_listing.tpl.php on line 135
Warning: pspell_suggest(): 0 is not a PSPELL result index in
/home/sites3/site342/web/templates/products_listing.tpl.php on line 138 

 

How can we solve this? That it recognizes a space. Thanks for the help.  

 

Kind regards,

 

Daniel Sloof

Re: Aspell question?

William H. Magill — 2012-12-31T22:48:29

On Dec 30, 2012, at 10:59 PM, Kevin Atkinson <k< at >kevina.org> wrote:


Duh ... what I get for using search instead of grep :)


Worked like a charm.
Thanks.


T.T.F.N.
William H. Magill
# iMac11,3 Core i7 [2.93GHz - 8 GB 1067MHz] OS X 10.8.2
# MacBook Pro4.1 Core 2 Duo [2.5GHz - 4GB 667] OS X 10.6.8
# Macmini6,1 Intel Core i5 [2.5 Ghz - 4GB 1600MHz] OS X 10.8.1
magill< at >mac.com
whmagill< at >gmail.com

Re: Aspell question?

Kevin Atkinson — 2012-12-31T03:59:09

First off sorry about the other email, the mailing list software removed the 
attachments and converted your message to plain text.  I got the attachments 
in the copy you sent direct to me.  Notes below.

On Sun, 30 Dec 2012, William H. Magill wrote:


Have another look of the file, the line "personal_ws-1.1 en 635 " is also in 
the middle of the file.


There are affix flags that the personal dictionary can't handle.  Something 
like this should fix it by expanding the affixes; for example "Absalom/M" will 
become "Absalom" and "Absalom's":

   cat ~/A-ispell_english | aspell --lang=en expand | tr ' ' '\n' \
     > A-ispell_english.fixed

The key part is "aspell expand".  The "tr" command is used to convert spaces 
to newlines.

-Kevin_______________________________________________
Aspell-user mailing list
Aspell-user< at >gnu.org
https://lists.gnu.org/mailman/listinfo/aspell-user

Re: Aspell question?

Kevin Atkinson — 2012-12-31T01:52:08

Again, I need to see the problem file to know what is going on.

-Kevin

On Sun, 30 Dec 2012, William H. Magill wrote:

Aspell-user mailing list
Aspell-user< at >gnu.org
https://lists.gnu.org/mailman/listinfo/aspell-user

Re: Aspell question?

Kevin Atkinson — 2012-12-30T16:41:38

Hi,

I'm not sure what the problem is.  If you could attach the .aspell.en.pws I 
might have a better idea.

Thanks,
Kevin
Aspell Maintainer

On Sat, 29 Dec 2012, William H. Magill wrote:

Re: Aspell question?

William H. Magill — 2012-12-30T21:21:16

I'm not sure what the problem is. If you could attach the .aspell.en.pws I 
might have a better idea.
 
I've done a bit more "research" and have noted the following:

1- if I delete .aspell.en.pws and re-run aspell-import
the error message does not appear. 
Then running "ispell" on the emacs buffer, it appears to work with no issues.
The file in question which I am editing is an html file.

2- 
A) If I then (using emacs) delete the first line and save the file as s.1
B) then sort s.1>s.2
C) read s.2 and tack on the original first line from the original file and save it again as .aspell.en.pws
D) the error occurs:
Error: /Users/magill/.aspell.en.pws: The word "personal_ws-1.1 en 635 " is invalid. The character '\
_' (U+5F) may not appear in the middle of a word.
E) the file size is the same in either situation.

3- During a run of aspell-import, I get a number of errors about /M and similar ispell suffixes.

The Aspell file attached is the "non-working" version.

Note: the attachments are re-named to allow OSX Mail to include them -- it doesn't see the ".file" -- even though they are visible (enabled) to the Finder.  121118.html is the file I am attempting to spell-check.

The two "obvious" questions are:
1- Why doesn't it like the header after the file has been sorted?
2- Why doesn't aspell-import like valid ispell nominclature?

T.T.F.N.
William H. Magill
 # iMac11,3 Core i7 [2.93GHz - 8 GB 1067MHz] OS X 10.8.2
 # MacBook Pro4.1 Core 2 Duo [2.5GHz - 4GB 667] OS X 10.6.8
# Macmini6,1 Intel Core i5 [2.5 Ghz - 4GB 1600MHz] OS X 10.8.1

 magill< at >mac.com
 whmagill< at >gmail.com




_______________________________________________
Aspell-user mailing list
Aspell-user< at >gnu.org
https://lists.gnu.org/mailman/listinfo/aspell-user

Aspell question?

William H. Magill — 2012-12-29T23:52:17

This is probably strictly an Aspell question, not mac ports...
I'm a long time ispell user converting to Aspell.   I've installed Aspell and its dictionary via mac ports.

(This is also on a new (Black Thursday) Mac-mini with OSX 10.8.2 and OSX Server 2.2)

sudo port install aspell
sudo port install aspell-dict-en

both apparently installed with no errors.

to my .emacs file, I've added:
 (setq-default ispell-program-name "aspell")
 (setq-default ispell-extra-args '("--reverse"))

I ran "aspell-import
( that process complained about /M and similar inclusions in my .ispell_english file, which surprised me.)

Now, when I run ispell on my emacs buffer, I get:

"Error: /Users/magill/.aspell.en.pws: The word "personal_ws-1.1 en 635 " is invalid. The character '_' (U+5F) may\
not appear in the middle of a word."

This is "sort of true" -- that is the header line created by aspell, and is as is documented at aspell.net

So the question is: Which is it... an issue with the macports aspell installation or an issue with aspell itself?


T.T.F.N.
William H. Magill
# iMac11,3 Core i7 [2.93GHz - 8 GB 1067MHz] OS X 10.8.2
# MacBook Pro4.1 Core 2 Duo [2.5GHz - 4GB 667] OS X 10.6.8
# Macmini6,1 Intel Core i5 [2.5 Ghz - 4GB 1600MHz] OS X 10.8.1

magill< at >mac.com
whmagill< at >gmail.com

Notepad++ v5.9.8 VERSUS aspell-en-0.50-2-3.exe

roger jones — 2012-12-05T21:08:53

Notepad++ v5.9.8   VS.   aspell-en-0.50-2-3.exe

I am running XP pro.  I cannot get Notepad++ v5.9.8 to accept the this 
speller. NP++ wants to see _*aspell-15.dll*_ and wants a path to it.
After running the aspell-en-0.50-2-3.exe , there is now a folder under 
program files, but there are no dll's .  I searched to entire HD and 
found none

Any help is appreciated.

gmane.comp.gnu.aspell.user

Re: Hyphens and apostrophes in words

Re: Hyphens and apostrophes in words

Re: Hyphens and apostrophes in words

Re: ***SPAM*** Re: Hyphens and apostrophes in words

Re: Hyphens and apostrophes in words

Re: Hyphens and apostrophes in words

Re: Hyphens and apostrophes in words

Hyphens and apostrophes in words

Can't build for MSWindows

User Instructions

en-common.rws problem

spellchecker

french typographic apostrophe

Cannot add words with spaces in the dictionary

Re: Aspell question?

Re: Aspell question?

Re: Aspell question?

Re: Aspell question?

Re: Aspell question?

Aspell question?

Notepad++ v5.9.8 VERSUS aspell-en-0.50-2-3.exe

Re: SPAM Re: Hyphens and apostrophes in words