http://permalink.gmane.org/gmane.comp.nlp.moses.user hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.nlp.moses.user/8925 <pre>hi kan you must subscribe to the mailing list to post to it. You can subscribe here http://mailman.mit.edu/mailman/listinfo/moses-support you answer you question-i think you need to install the package sudo apt-get install libbz2-dev also, make sure you have these packages installed too zlib1g-dev libboost-all-dev ---------- Forwarded message ---------- From: &lt;moses-support-owner-3s7WtUTddSA&lt; at &gt;public.gmane.org&gt; Date: 22 May 2013 12:29 Subject: Moses-support post from kanzhang.jerry-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org requires approval To: moses-support-owner-3s7WtUTddSA&lt; at &gt;public.gmane.org As list administrator, your authorization is requested for the following mailing list posting: List: Moses-support-3s7WtUTddSA&lt; at &gt;public.gmane.org From: kanzhang.jerry-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org Subject: MOSES ON Ubuntu 12.04LTS ERROR HELP! Reason: Post by non-member to a members-only list At your convenience, visit: http://mailman.mit.edu/mailman/admindb/moses-support </pre> Hieu Hoang 2013-05-22T12:59:36 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8924 <pre>Hi Johannes An alternative to adding the terminological information to the phrase table is to use Moses xml-input mechanism to override the model. http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc7 I think you will only find the best approach by experimenting, cheers - Barry On 21/05/13 10:44, Johannes Hellrich wrote: </pre> Barry Haddow 2013-05-22T09:13:22 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8923 <pre>NLP People news: 1. We continue to publish the guidelines on career path selection. The article we just made available at NLPPeople.com is called “Why does it really take to make it in academia?”. We and the author (Paige Harris) will very much appreciate if you could share your opinion on the topics discussed in the article. The direct link to the publication: https://nlppeople.com/index.php/publications/articles/111-what-does-it-really-take-to-make-it-in-academia 2. Hot NLP job openings: - talented NLP researcher for a leading software company based in Cambridge (UK):https://nlppeople.com/index.php/job/2083/nlp-researcher-cambridge - lead researcher position within Samsung Advanced Research Institute in Bangalore (India) https://nlppeople.com/index.php/job/2059/researcher-bangalore - NLP software developer advertised by Venbest Recruiting in Kyiv (Ukraine): https://nlppeople.com/index.php/job/2049/software-developer-(nlp)-kyiv 3. Current machine learning and data mining openings: - entry level progr</pre> Alexander Raginsky 2013-05-22T09:05:08 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8922 <pre>Hi Johannes There's 2 main models you should update 1. Language model 2. Translation model For the language model, you can concatenate the (in-domain) biomedical data to any other target-language corpus you might have. Or you can build language models for in-domain and out-of-doman data, the interpolate them to produce 1 overall LM. Or you can use both LMs separate during decoding. For translation models, there are a few choices. Again, you can concatenate the parallel data. Or you can subsample the parallel data from each corpora. Or you can create 2 separate phrase tables and use them both during decoding. If you do this, you can configure them to compete against each other. Or make 1 the primary phrase-table, and 1 a backoff table which is only used if the primary doesn't have translation. There's no explicit support for placeholders in Moses, but you can do it externally. I'm not sure what is current best practise, I suppose it just depends on what is easiest to implement and gives you the d</pre> Hieu Hoang 2013-05-22T08:57:31 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8921 <pre>I used train-model.perl script to get all the steps(1 to 9) done. Now everything is fine. Thank you. On Thu, May 16, 2013 at 5:10 AM, Hieu Hoang &lt;Hieu.Hoang-5WhEfG1TI8k&lt; at &gt;public.gmane.org&gt; wrote: </pre> Nithya 2013-05-22T05:13:30 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8920 <pre>Hi, This was a recursive issue ... libmoses needed libmira to compile and libmira needed libmoses to compile. However, I was able to solve this issue in rather naive way. Instead of using files from mira/ directory I copied all the files that I wanted to use in a separate directory and added the path to the moses/Jamfile. Now everything works perfectly fine. Thanks for the support though, -- Prashant On 05/16/2013 09:00 PM, Hieu Hoang wrote: _______________________________________________ Moses-support mailing list Moses-support-3s7WtUTddSA&lt; at &gt;public.gmane.org http://mailman.mit.edu/mailman/listinfo/moses-support </pre> Prashant Mathur 2013-05-21T11:35:12 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8919 <pre>Hi all, I want to incorporate a (biomedical) multilingual-terminology into a SMT pipeline. I have no current plans to use disambiguation, so I could simply extract a dictionary from my terminology. But how do I incorporate this dictionary with my pipeline? The common solution seems to be adding its entries to the training data. Would on over-sample it or add the entries only once? I also wondered if one couldn't treat it like a transliteration and replace the known translations in the training data with placeholders (and add some kind of wrapping around Moses to insert the correct ones) - this could be more useful if I use disambiguation later on. Are there any best-practices if failed to find or would this be worth a thorough analysis? Thank you very much, Johannes Hellrich ---------------------------------------------------------------- This message was sent through https://webmail.uni-jena.de </pre> Johannes Hellrich 2013-05-21T09:44:42 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8918 <pre> Thanks, Barry. It helps to know where to look. On 2013-05-21 14:50, Barry Haddow wrote: unusual characters in the corpus. If you vocab file (target side - es.vcb I think) then problem is. wrote: MGIZA++ failed below. I think this anyone confirm this or = -1 (fixed value for parameter p_0 in IBM-5 (if negative then it determined in training)) () fertility for fertility models) executed, use 0 if you just want all (fixed value for parameter p_0 in IBM-3/4 (if negative then determined in training)) pegging) from:/opt/domy/TRAININGS/alignments/align-dell2_full-en-es/giza.classes/en.vcb Reading vocabulary file from:/opt/domy/TRAININGS/alignments/align-dell2_full-en-es/giza.classes/es.vcb Source vocabulary list has 85970 unique tokens list has 84643 unique tokens corpus /opt/domy/TRAININGS/alignments/align-dell2_full-en-es/giza.classes/es-en-int-train.snt Reading more sentence pairs into memory ... 118049 is not in the vocabulary list _______________________________________________ list http://mailman.mit</pre> Tom Hoar 2013-05-21T08:03:10 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8917 <pre>Hi Tom Yes, this could be caused by unusual characters in the corpus. If you search for 118049 in the vocab file (target side - es.vcb I think) then you may see what the problem is. cheers - Barry On 20/05/13 14:40, Tom Hoar wrote: </pre> Barry Haddow 2013-05-21T07:50:24 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8916 <pre> moses-support-request&lt; at &gt;mit.edu编写： _______________________________________________ Moses-support mailing list Moses-support&lt; at &gt;mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support </pre> Chen Kehai 2013-05-21T07:31:57 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8915 <pre> On 20/05/2013 20:39, j0hn wrote: the counts are fractional counts, so they can be decimals, not just integers. I think fractional count is describe in 1 of David Chiang's Hiero paper. The probability is calculated as you described, unless a discount method (Good-Turing etc) is applied. _______________________________________________ Moses-support mailing list Moses-support-3s7WtUTddSA&lt; at &gt;public.gmane.org http://mailman.mit.edu/mailman/listinfo/moses-support </pre> Hieu Hoang 2013-05-20T22:26:47 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8914 <pre>Hello everyone, I'm trying to understand the format on the rule table from the tree based models and I'm a little bit confused about the counts. I use the following corpus to train: *Spanish:* en mi casa esta es mi casa *English:* in my house this is my house Then I use freeling to get the tree based tagging and I use that to train moses. The resulting rule table is here: http://pastebin.com/4k50avpt I'm trying to understand how the probabilities are calculated, for example, I know that the direct phrase translation probability is calculated as: count(target &amp; source) / count(source) but in rule table I get rules like this one: esta es [sn][sn-chunk] [S] ||| this is [sn][sn-chunk] [S] ||| 1 1 1 1 2.718 ||| 2-2 ||| 0.5 0.5 Note that the counters of source and target are decimal numbers... in that case how are the direct and indirect translation probability calculated? Is there any documentation where all of this is explained? Thanks! _______________________________________________ Moses-support mail</pre> j0hn 2013-05-20T19:39:44 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8913 <pre>The train-model.perl script from Beta 0.91 configured for MGIZA++ failed on step 2.1b in the reverse direction with the error below. I think this might be a result of inadequate cleaning. Can anyone confirm this or offer an alternate reason? Thanks. m5p0 = -1 (fixed value for parameter p_0 in IBM-5 (if negative then it is determined in training)) manlexfactor1 = 0 () manlexfactor2 = 0 () manlexmaxmultiplicity = 20 () maxfertility = 10 (maximal fertility for fertility models) ncpus = 1 (Number of threads to be executed, use 0 if you just want all CPUs to be used) p0 = 0.999 (fixed value for parameter p_0 in IBM-3/4 (if negative then it is determined in training)) pegging = 0 (0: no pegging; 1: do pegging) reading vocabulary files Reading vocabulary file from:/opt/domy/TRAININGS/alignments/align-dell2_full-en-es/giza.classes/en.vcb Reading vocabulary file from:/opt/domy/TRAININGS/alignments/align-dell2_full-en-es/giza.classes/es.vcb Source vocabulary list has 85970 unique tokens Target vocabulary list</pre> Tom Hoar 2013-05-20T13:40:40 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8912 <pre>Dear all, The test sets for all subtasks can be downloaded from http://www.quest.dcs.shef.ac.uk/wmt13_qe.html We will extend the deadline for submitting system results in two days: June 2nd instead of May 31st. Best, </pre> Lucia Specia 2013-05-17T17:23:04 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8911 <pre>MT Marathon 2013 and Open Source Convention http://www.statmt.org/mtm13 Organized and sponsored by MosesCore EU project, Machine Translation Marathon 2013 is a week long gathering of machine translation researchers, developers, students and users. It features: - MT Lectures and Labs covering the basics and tutorials. - Invited talks from experienced researchers and practitioners - Technical Talks about open source tools. - Hacking Projects to advance tools or research in one week. The Machine Translation Marathon 2013, MTM2013, will take place in Prague, Czech Republic in September 9--14th, 2013. The registration is now open and includes the option to book accommodation for students (limited number of places). The MT Marathon will again host an Open Source Convention to advance the state of the art in machine translation. We invite developers of open source tools to present their work and submit a paper of up to 10 pages that describes the underlying methodology and includes instructions on how to </pre> Ondrej Bojar 2013-05-17T13:51:46 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8910 <pre>I agree with Tom. If boost is available using apt-get, you should use it. If it fails to install, you need to figure out why. Otherwise you could be making the situation worse by installing multiple versions of boost from different places. On 17 May 2013 05:33, Tom Hoar &lt;tahoar-cB/ihCqopeG7AKbOTpHwwjx6VEFkrI8ampATvIKMPHk&lt; at &gt;public.gmane.org&gt; wrote: </pre> Hieu Hoang 2013-05-17T10:19:41 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8909 <pre>Installing libboost-all-dev with apt-get and compiling moses on Ubuntu 12.04 64-but works. Your apt-get command is correct. You'll need to troubleshoot why apt-get failed to install libboost-all-dev. On 05/17/2013 09:35 AM, Eziz.Tursun wrote: _______________________________________________ Moses-support mailing list Moses-support-3s7WtUTddSA&lt; at &gt;public.gmane.org http://mailman.mit.edu/mailman/listinfo/moses-support </pre> Tom Hoar 2013-05-17T04:33:56 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8908 <pre>I have installed giza++ and irstlm successfully, but there are some problem while compile Moses on Ubuntu 12.04 64bit. there is one of error messages: "lm/filter/bin/gcc-4.6/release/debug-symbols-on/link- static/threading-multi/main.o:/usr/local/include/boost/thread/detail/thread.hpp:180: more undefined references to `boost::thread::start_thread_noexcept()' follow collect2: ld returned 1 exit status ...failed gcc.link lm/filter/bin/gcc-4.6/release/ debug-symbols-on/link-static/threading-multi/filter... .... .... " I have installed boost as follows: "sudo apt-get install libboost-all-dev" after failed first time,then I installed boost lib 1.5.03 like this: ./bootstrap.sh ./b2 install Unfortunately, there was no progress. What's my problem? could tell me detailed steps for solve this problem? Best regards, Eziz _______________________________________________ Moses-support mailing list Moses-support-3s7WtUTddSA&lt; at &gt;public.gmane.org http://mailman.mit.edu/mailman/listinfo/moses-support </pre> Eziz.Tursun 2013-05-17T02:35:40 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8907 <pre>i saw that it wasn't addressed too but my solution to difficult linking problems is to avoid bjam/automake/cmake/make. ie. anything that tries to help, but makes it more difficult to understand. I would just execute a direct g++ command, eg g++ -L [path] -lmira -o whatever.exe I did this with mgiza after fruitlessly trying to fiddle with cmake. http://mgizapp.svn.sourceforge.net/viewvc/mgizapp/trunk/mgizapp/manual-compile/ It's prob not a fast way, but it's the easiest way for me On 16 May 2013 19:48, Nick Ruiz &lt;nicruiz-0WaB7DoCAz4&lt; at &gt;public.gmane.org&gt; wrote: </pre> Hieu Hoang 2013-05-16T19:00:43 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8906 <pre>Hi all, I saw this message from last month that doesn't look like it's been addressed. This is somewhat of a common problem for people writing code in Moses and new to compiling with bjam. Does anyone have any advice? Thanks, Nick On 04/29/2013 04:16 PM, Prashant Mathur wrote: _______________________________________________ Moses-support mailing list Moses-support-3s7WtUTddSA&lt; at &gt;public.gmane.org http://mailman.mit.edu/mailman/listinfo/moses-support </pre> Nick Ruiz 2013-05-16T18:48:54 http://permalink.gmane.org/gmane.comp.nlp.moses.user/8905 <pre>how exactly did you run giza++? Did you also run scripts/training/giza2bal.pl This take the giza++ file and creates the file aligned.grow-diag-final-and The EMS will do all of this for you. Or step 3 of scripts/training/train-model.perl will also run giza2bal.pl for you too. _______________________________________________ Moses-support mailing list Moses-support-3s7WtUTddSA&lt; at &gt;public.gmane.org http://mailman.mit.edu/mailman/listinfo/moses-support </pre> Hieu Hoang 2013-05-15T23:40:13 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.nlp.moses.user