http://permalink.gmane.org/gmane.comp.statistics.madlib.user hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.statistics.madlib.user/292 <pre>Samer, Windows is not currently an officially supported platform for MADlib, so you are in uncharted territory trying to get it working. There's no reason it shouldn't be able to work on the platform since none of the code is particularly OS port sensitive, but especially in the area of the installation scripts there may be some issues. Within the built library the main installation tool is 'madpack' which is a python script. Assuming you have python installed it should be able to work correctly, however it is possible that some changes would be needed especially in the area of path resolution. In the $INSTALLPATH/bin directory the main madpack.py file is called from a shell script, which requires something like cygwin to provide shell script execution on Windows. If you find an installation process that works for you please let us know and we can extend installation guides appropriately. Regards, Caleb On Mon, Jun 3, 2013 at 10:25 PM, Samer Abu-Nasser &lt;candlesurfer-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;p</pre> Caleb Welton 2013-06-04T17:23:55 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/291 <pre>Hi, I was able to compile madlib successfully into the cygwin directory usrs/local/madlib... 1. I copied this file madlib in to c:/ directory of windows can I do this? 2. How do I register madlib to a postgres db In windows (9.2 version of postgres using a windows installer not cygwin) - all tutorials suggest UNIX commands -----Original Message----- From: "user-request&lt; at &gt;madlib.net" &lt;user-request&lt; at &gt;madlib.net&gt; Sent: ‎6/‎3/‎2013 10:03 PM To: "candlesurfer&lt; at &gt;gmail.com" &lt;candlesurfer&lt; at &gt;gmail.com&gt; Subject: Welcome to the "User" mailing list Welcome to the User&lt; at &gt;madlib.net mailing list! To post to this list, send your email to: user&lt; at &gt;madlib.net General information about the mailing list is at: http://madlib.net/mailman/listinfo/user If you ever want to unsubscribe or change your options (eg, switch to or from digest mode, change your password, etc.), visit your subscription page at: http://madlib.net/mailman/options/user/candlesurfer%40gmail.com You can also make such adjustments via email by sendi</pre> Samer Abu-Nasser 2013-06-04T05:25:51 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/290 <pre>Hello Franck, There have been no published benchmarks since our VLDB paper last year. Regards, Caleb On Mon, May 13, 2013 at 6:27 AM, Franck Dernoncourt &lt; _______________________________________________ MADlib user discussion mailing list User&lt; at &gt;madlib.net http://madlib.net/mailman/listinfo/user</pre> Caleb Welton 2013-05-16T08:32:05 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/288 <pre>Hi, The paper "The MADlib Analytics Library or MAD Skills, the SQL" (VLDB 2012 - http://www.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-38.html) says "We defer macro-benchmarking of MADlib’s current methods to future work, that will focus on specific algorithm implementations". Has there been any MADlib benchmark since then? Thanks, Franck ---- Franck Dernoncourt franck.dernoncourt&lt; at &gt;gmail.com http://franck-dernoncourt.com/ _______________________________________________ MADlib user discussion mailing list User&lt; at &gt;madlib.net http://madlib.net/mailman/listinfo/user</pre> Franck Dernoncourt 2013-05-12T22:27:43 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/285 <pre>I offer this screen dump to describe how I got madpack to work for me: bash-4.1$ /usr/local/madlib/bin/madpack -p postgres -c madmin-savSHZN5Fh8qMp+WYRx65w&lt; at &gt;public.gmane.org:5432/testdb install Password for user madmin: madpack.py : INFO : Detected PostgreSQL version 8.4. madpack.py : INFO : *** Installing MADlib *** madpack.py : INFO : MADlib tools version = 0.5 (/usr/local/madlib/bin/../madpack/madpack.py) madpack.py : INFO : MADlib database version = None (host=127.0.0.1:5432, db=testdb, schema=madlib) madpack.py : INFO : Testing PL/Python environment... madpack.py : INFO : &gt; Creating language PL/Python... madpack.py : INFO : &gt; PL/Python environment OK (version: 2.6.6) madpack.py : INFO : Installing MADlib into MADLIB schema... madpack.py : INFO : &gt; Creating MADLIB schema madpack.py : INFO : &gt; Creating MADLIB.MigrationHistory table madpack.py : INFO : &gt; Writing version info in MigrationHistory table madpack.py : INFO : &gt; Creating objects for modules: madpack.py : INFO : &gt; - array_ops madpack.py : INFO </pre> Audrey Lee 2013-03-29T00:27:53 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/284 <pre>Caleb, You offer a strong clue. Thanks for taking time to type it up. I offer an example of a pg_hba.conf file which will be useful to other newbs: # TYPE DATABASE USER CIDR-ADDRESS METHOD # "local" is for Unix domain socket connections only local all all trust # IPv4 local connections: host all all 127.0.0.1/32 md5 # IPv6 local connections: host all all ::1/128 ident I 'activated' the pg_hba.conf file via this shell command: pg_ctl reload I found that madpack was picky about how I specify localhost. I needed to use the IP address: /usr/local/madlib/bin/madpack -p postgres -c madmin-savSHZN5Fh8qMp+WYRx65w&lt; at &gt;public.gmane.org:5432/testdb install Another small wrinkle is that madpack depends on testdb, it does not create testdb. I used this shell command to create testdb: createdb testdb Now I see this behavior from madpack: bash-4.1$ /usr/local/madlib/bin/madpack -p postgres -c </pre> Audrey Lee 2013-03-29T00:20:36 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/283 <pre>Hi All I am trying to use the MADLIB logregr function to do logistic regression. I have only categorical variables and I have encoded them as 0 or 1. I have 1 million data points and the predictors are arrays of size 2000 elements (each element is either 0 or 1) . The predictors constitute a sparse matrix. When I run logregr, it takes a long time, more than 10 minutes. I have not yet seen it finish. I am using Greenplum 4.1.2 and Madlib 0.5 on a Quad core machine with AMD Opteron 2.7 MHz CPUs. Is there debugging information for logregr? Many things can go wrong but I have no diagnostic information to work with. Also, instead of creating a table with arrays full of zeros, istere a way to feed data to logregr with a table where the first index is the row number, second index is the column number and the third column is the value? Mehmet </pre> Mehmet Yunt 2013-03-28T17:50:58 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/282 <pre>Hello Audrey. This is almost certainly has to do with how you have postgres configured. See http://www.postgresql.org/docs/8.4/static/auth-pg-hba-conf.html And more details for ident authentication http://www.postgresql.org/docs/8.4/static/auth-methods.html#AUTH-IDENT Given this pg_hba.conf file the same issue can be demonstrated with just psql: local all all trust host all all 1.1.1.1/0ident --&gt; Connecting using psql via local unix sockets works fine bash$ psql sql (8.4.13) Type "help" for help. postgres=# --&gt; Connecting using psql via a TCP/IP connection fails bash$ psql -h localhost psql: FATAL: Ident authentication failed for user "postgres" The easiest solution will be to configure the pg_hba.conf file to use something other than ident authentication. Note that ident authentication is generally considered insecure for TCP/IP connections, so by fixing this you will also make your server more secure. Regards, Caleb On Mar 26, 2013, at 3:34 </pre> Welton, Caleb 2013-03-27T00:17:23 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/281 <pre>Hello world. I need help installing madlib. I list below the steps which I have followed: Install 64b-Linux Install Postgres 8.4 Install madlib rpm: http://www.madlib.net/files/madlib-0.5-Linux.rpm Here is a screen dump of where I'm at: -bash-4.1$ -bash-4.1$ id uid=26(postgres) gid=26(postgres) groups=26(postgres) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 -bash-4.1$ -bash-4.1$ -bash-4.1$ psql psql (8.4.13) Type "help" for help. postgres=# SELECT count(*) AS cnt FROM pg_tables; cnt ----- 49 (1 row) postgres=# SELECT count(*) AS cnt FROM pg_tables WHERE schemaname='madlib' AND tablename='migrationhistory'; postgres-# cnt ----- 0 (1 row) -bash-4.1$ -bash-4.1$ /usr/local/madlib/bin/madpack -p postgres -c postgres&lt; at &gt;localhost:5432/testdb install Password for user postgres: madpack.py : ERROR : SQL command failed: SQL: SELECT count(*) AS cnt FROM pg_tables WHERE schemaname='madlib' AND tablename='migrationhistory' psql: FATAL: Ident authentication failed </pre> Audrey Lee 2013-03-26T22:34:59 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/280 <pre>Hi I am trying to do logistic regression with categorical independent variables using MADlib. I have encoded these variables as 1-0 in their appropriate columns. column belongs to a categorical variable or not once properly encoded for computational purposes. I would like to make sure there is nothing I missed in this case. Is there anything I should be aware of when using madlib with categorical independent variables? Thanks Mehmet </pre> Mehmet Yunt 2013-03-22T19:28:30 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/279 <pre>Forwarding to the general user base. Begin forwarded message: From: Mehmet Yunt &lt;mehmet.yunt-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&lt;mailto:mehmet.yunt-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&gt;&gt; Subject: Re: [MADlib-user] How to store the output of logregr in a table Date: March 20, 2013 6:28:54 AM PDT To: "Iyer, Rahul" &lt;Rahul.Iyer-mb1K0bWo544&lt; at &gt;public.gmane.org&lt;mailto:Rahul.Iyer-mb1K0bWo544&lt; at &gt;public.gmane.org&gt;&gt; I tested the work around all seems to work. I also wrote the version wrong. I was using 4.1.1.2 Thanks very much for the quick reply. Mehmet On Mar 19, 2013, at 5:27 PM, "Iyer, Rahul" &lt;Rahul.Iyer-mb1K0bWo544&lt; at &gt;public.gmane.org&lt;mailto:Rahul.Iyer-mb1K0bWo544&lt; at &gt;public.gmane.org&gt;&gt; wrote: Hi Mehmet, Thanks for your email. MADlib has been undergoing some changes and we are striving to keep all our documentation up-to-date. The query you posted works for us on GPDB 4.2.2.0 build 3. Some older versions of GPDB, however, have exhibited the problem you face. We have a workaround function that allows you to c</pre> Iyer, Rahul 2013-03-20T17:37:56 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/278 <pre>Mehmet, You may have hit a limitation in the version of Greenplum that you are currently using. We do provide a workaround function within MADlib to assist in working past the limitation, see: http://doc.madlib.net/v0.5/compatibility_8sql__in.html This particular limitation has been mostly resolved in later versions of the Greenplum Database, so it is also possible that you could address this without the workaround by upgrading to a more recent DB version. Regards, Caleb On Mar 20, 2013, at 9:41 AM, Qian, Hai wrote: </pre> Welton, Caleb 2013-03-20T17:23:02 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/277 <pre> Logregr returns a composite type data. So CREATE TABLE mytest AS SELECT (madlib.logregr('artificiallogreg', 'y', 'x', 20, 'irls', 0.001)).coef; In the next release of MADlib (v0.6), the API of logregr will be changed so that the result is store directly into a table. Hai On 3/20/13 9:00 AM, "user-request-QC320f16cZmsTnJN9+BGXg&lt; at &gt;public.gmane.org" &lt;user-request-QC320f16cZmsTnJN9+BGXg&lt; at &gt;public.gmane.org&gt; wrote: </pre> Qian, Hai 2013-03-20T16:41:13 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/276 <pre>I am new to MADlib and I was going through the demos from github (they seem to require updating) . I am using Greenplum 4.2 and MADlib 0.5. I get the following error when I try to create a table from tje output of logregr: analytics=&gt;create table mytest as select coef from madlib.logregr('artificiallogreg', 'y', 'x', 20, 'irls', 0.001); NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column(s) named 'coef' as the Greenplum Database data distribution key for this table. HINT: The 'DISTRIBUTED BY' clause determines the distribution of data. Make sure column(s) chosen are the optimal data distribution key to minimize skew. WARNING: PL/Python: in PL/Python function "compute_logregr" (entry db dev-gp03.undertone.net:5432 pid=13508) DETAIL: &lt;class 'plpy.SPIError'&gt;: unrecognized error in PLy_spi_execute_query CONTEXT: SQL statement "SELECT ( SELECT madlib.compute_logregr( $1 , $2 , $3 , $4 , $5 , $6 ) )" PL/pgSQL function "logregr" line 6 at assignment ERROR: function cannot execute on s</pre> Mehmet Yunt 2013-03-19T18:27:29 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/275 <pre>Vatsan, Looks like a great option for people looking for more of a local client programming interface. It's great seeing the different ways that people use and interact with MADlib. Thanks, Caleb On Mar 9, 2013, at 3:01 PM, Srivatsan Ramanujam wrote: </pre> Welton, Caleb 2013-03-11T16:14:14 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/274 <pre>Dear MADlibers, I just released my python wrapper for MADlib. Now you can have the number crunching power of MADlib along with the cool visualizations of Matplotlib, NetworkX and all your other Python libraries. Let MADlib do all the heavy-lifting on your Greenplum Appliance, while you program in your favorite language - Python! Check out http://vatsan.github.com/pymadlib/ for more information on PyMADlib including installation instructions, gallery and more! Your comments and feedback are welcome. Thank you, Vatsan </pre> Srivatsan Ramanujam 2013-03-09T23:01:36 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/273 <pre>Hi Audrey, Thanks for your valuable feedback about the MADlib documentation. We are always looking to improve the users' experience with MADlib. As for the usage of SVM, apologies for the confusion caused by the documentation. Yes, the "label" column represents the vector's labeled class. Here's an example run of your IBM stock example using the data points you mentioned: madlib_test=# create table ibm_stock(id int, ind float8[], label float8); madlib_test=# insert into ibm_stock values (3,array[0.001,28.5],1), (33,array[0.002,38.5],-1), (333,array[-0.001,30.5],1); madlib_test=# select madlib.svm_classification('ibm_stock','ibm_model',false, 'madlib.svm_dot'); svm_classification ------------------------------------------------------------ (ibm_model,ibm_model,3,2,-0.156592798394,0.042264973081,2) (1 row) madlib_test=# select madlib.svm_predict('ibm_model','{0.0015,40.2}') &gt; 0; ?column? ---------- t (1 row) As for the t(x) function mentioned in th</pre> Gee, Grace 2013-02-27T17:50:35 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/272 <pre>Hello World. I'm new to madlib. My experience with ML on databases is limited to running SVM on Oracle. Oracle does an excellent job of explaining how to setup an SVM classification system. Also they provide a decent comprehensive example related to some sample marketing data. The Madlib docs do a good job of explaining how to install the Madlib. They do a poor job of explaining how I might use Madlib after I install it. I studied this page: http://doc.madlib.net/v0.5/group__grp__kernmach.html The page displays this syntax: t(x) = if x[1] &gt; 0 and x[2] &lt; 0 then 1 else -1; What kind of syntax is this? Where/How do I run it? Why do I want to run it? I tried this: oracle&lt; at &gt;vbox3:/pt/s/r/madlib$ psql rsinp psql (9.1.7, server 9.1.8) Type "help" for help. rsinp=# t(x) = if x[1] &gt; 0 and x[2] &lt; 0 then 1 else -1; ERROR: syntax error at or near "t" LINE 1: t(x) = if x[1] &gt; 0 and x[2] &lt; 0 then 1 else -1; ^ At this point I'd suggest the doc-authors toss this discussion about t(x) in the trash</pre> Audrey Lee 2013-02-27T17:00:30 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/271 <pre>Hi Welton, Thanks for you response, we are using 32bit virtual box running GP for learning purpose, and 32bit MADlib against GP will be very convient. Thanks. Xiaobo Gu On Tue, Feb 26, 2013 at 6:35 AM, Welton, Caleb &lt;Caleb.Welton-mb1K0bWo544&lt; at &gt;public.gmane.org&gt; wrote: </pre> Xiaobo Gu 2013-02-26T01:33:14 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/270 <pre>Xiaobo, Thank you for the instructions on installing rpm based MADlib builds against Greenplum. Our general support model for Greenplum offering is via the gppkg downloads available via subscribe.net. We currently have no plan to make an official rpm based distribution for GPDB. Further 32bit centos is not a supported Greenplum distribution. If you want to fix the issue you are seeing and submit it to us for review then the team will review the submission for inclusion, but there is no plan to do development work for unsupported versions at this time. Thanks, Caleb On Feb 23, 2013, at 12:59 AM, Xiaobo Gu wrote: </pre> Welton, Caleb 2013-02-25T22:35:36 http://permalink.gmane.org/gmane.comp.statistics.madlib.user/269 <pre>Thank you in advance. Xiaobo Gu </pre> Xiaobo Gu 2013-02-25T12:54:29 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.statistics.madlib.user