gmane.comp.lang.ruby.modules.mechanize.user

Re: Error trying to reload serialized page

Eric Hodel — 2012-05-23T20:24:16

A String is not an acceptable substitute for a Net::HTTPResponse or Hash (representing the headers of a response).


Try:

  p2 = Mechanize::Page.new p1.uri, p1.response, p1.body, agent

While Mechanize::Page defaults all these to nil, this doesn't actually work for Page.  I'll see what I can do to fix this.


A Mechanize instance is consulted to determine how <meta> tags should be found, so you must pass it in.

Error trying to reload serialized page

Jack Royal-Gordon — 2012-05-23T19:40:15

What is the correct way to serialize an HTML page gotten from Agent::get, and then to reload the serialized page back into Mechanize?  I tried the following code:
agent = Mechanize.new
p1 = agent.get(url)
=> #<Mechanize::Page … >

p2 = Mechanize::Page.new(p1.uri.to_s, p1.response.to_s, p1.body)

and got the error:
Mechanize::ContentTypeError Exception: Mechanize::ContentTypeError

I also tried:
p2 = Mechanize::Page .new(p1.uri, p1.response, p1.body)

and got the error:
INTERNAL ERROR!!! undefined method `follow_meta_refresh' for nil:NilClass
/Users/jackrg/Documents/Novelty-Stats/vendor/local/ruby/1.9.1/gems/activesupport-3.1.3/lib/active_support/whiny_nil.rb:48:in `method_missing'
/Users/jackrg/Documents/Novelty-Stats/vendor/local/ruby/1.9.1/gems/mechanize-2.4/lib/mechanize/page.rb:350:in `meta_refresh'
/Users/jackrg/Documents/Novelty-Stats/vendor/local/ruby/1.9.1/gems/mechanize-2.4/lib/mechanize/page.rb:139:in `block (2 levels) in pretty_print'
/Users/jackrg/.rbenv/versions/1.9.2-p290/lib/ruby/

Re: [ANN] mechanize 2.5.1 Released

Kevin Kornemann — 2012-05-16T21:12:45

right, sorry.

On Wed, May 16, 2012 at 4:58 PM, Godfrey Chan <godfreykfc< at >gmail.com> wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: [ANN] mechanize 2.5.1 Released

Eric Hodel — 2012-05-16T21:05:16

It's possible that google has already picked up the email from the list archive page. I'm unsure if there are any mirrors of this list, but it's possible your password has been transferred far beyond this list by now.

Changing the password should be sufficient.

Re: [ANN] mechanize 2.5.1 Released

Brian Kennedy — 2012-05-16T21:00:41

The cat's shredded the bag.  it's out.   I'm a scrupulous security
professional, but like he said, it's a public list. You can guarantee that
somebody on the list is using your access already and exploring what a
raps.org is.  If what it is needs to be secure then you need to be hurrying
with your password change.

You're not getting out of it.   If you've messed up then you admit your
mistake and work to correct it.   as a hiring/firing manager I can say: Do
that and everything goes well, try to cover it up and get your poop smelled
later and it's likely you're fired.


On Wed, May 16, 2012 at 4:54 PM, Kevin Kornemann <kevin< at >proofimaging.com>wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: [ANN] mechanize 2.5.1 Released

Godfrey Chan — 2012-05-16T20:58:36

I don't think you understand how a mailing list works. Basically you just sent an *email* out to hundreds of email addresses. So the short answer is, no.

Godfrey

On 2012-05-16, at 1:54 PM, Kevin Kornemann wrote:


_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: [ANN] mechanize 2.5.1 Released

Kevin Kornemann — 2012-05-16T20:50:55

changed, but didn't get the rest.

On Wed, May 16, 2012 at 4:47 PM, Brian Kennedy <sirbeep< at >gmail.com> wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: [ANN] mechanize 2.5.1 Released

Kevin Kornemann — 2012-05-16T20:54:58

If possible, can you delete the email in your inbox? I mean surely a posted
can be deleted somehow?

On Wed, May 16, 2012 at 4:50 PM, Kevin Kornemann <kevin< at >proofimaging.com>wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: [ANN] mechanize 2.5.1 Released

Brian Kennedy — 2012-05-16T20:47:28

It's in my email inbox,  so nope.  :)

Cat's already shredded the bag and is working on the curtains.  Change your
password quick, you don't know who's on the list that may have taken it and
is running with it already.

On Wed, May 16, 2012 at 4:38 PM, Kevin Kornemann <kevin< at >proofimaging.com>wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: [ANN] mechanize 2.5.1 Released

Kevin Kornemann — 2012-05-16T20:38:03

shit, didn't realize this was public. Can I delete this post?

On Wed, May 16, 2012 at 4:25 PM, Eric Hodel <drbrain< at >segment7.net> wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: [ANN] mechanize 2.5.1 Released

Eric Hodel — 2012-05-16T20:25:33

You're going to need to emulate what the javascript does when you click on the request page.  You can use browser debugging tools to compare what query a browser makes with what query mechanize makes (through use of the debugger).

PS:  This is a public list so you will want to change the password.

Re: [ANN] mechanize 2.5.1 Released

Kevin Kornemann — 2012-05-16T18:35:59

cool, thanks! I'm trying to scrape a page that requires login, and then
clicking on form check boxes and then hitting search. I managed to get this
far with mechanize but the search results I get are garbage (not what I get
when I do the same with a browser), so I was grasping at straws that it was
a js script not being executed problem, but I really don't know. Looking
into other options but realy nothing I found was as easy or nice to use as
mechanize.

If it's not too much trouble, any hints are appreciated... to re-woord my
problem, search results are not as expected, even wrong page is being
returned from what I can tell. This is my script:

require 'nokogiri'
require 'mechanize'

agent = Mechanize.new

agent.follow_meta_refresh = true
agent.user_agent_alias = 'Mac Safari'

page = agent.get('http://connect.raps.org/RAPS/Directory/AdvancedSearch/')

login_form = page.form()

login_form.LoginTextBox = 'alexa coppola'
login_form.PasswordTextBox = 'goodfriend123'

page = agent.submit(login_form, login_form.

Re: [ANN] mechanize 2.5.1 Released

Eric Hodel — 2012-05-16T18:26:26

It handles JavaScript exactly the same as every other version, you can download them and display them, but not execute them.

Execution of JavaScript requires a JavaScript interpreter and DOM implementation atop Nokogiri, both of which are beyond the scope of mechanize.

Re: [ANN] mechanize 2.5.1 Released

Kevin Kornemann — 2012-05-16T04:39:53

Does this handle js better than older versions?

On Tue, May 15, 2012 at 10:30 PM, Eric Hodel <drbrain< at >segment7.net> wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: way to get response headers only?

shig odani — 2012-04-30T17:45:33

Thanks. I didn't realize that there were methods in correspondence with the
http verbs.

Message: 7
Date: Mon, 30 Apr 2012 08:32:17 -0600
From: Matt White <mattw922< at >gmail.com>
To: Ruby Mechanize Users List <mechanize-users< at >rubyforge.org
Subject: Re: [Mechanize-users] way to get response headers only?
Message-ID:
       <CA+BUVh0UXgkEObUSc6CW8Qpdkz8WnfMjHLXJk=ONwQgVhT1qtA< at >mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Just use .head instead of .gethttps://
www.goldstar.com/shows/555643/checkout/new

On Mon, Apr 30, 2012 at 8:21 AM, shig odani <sodani< at >gmail.com> wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: way to get response headers only?

Matt White — 2012-04-30T14:32:50

Sorry for the copy/paste.

On Mon, Apr 30, 2012 at 8:32 AM, Matt White <mattw922< at >gmail.com> wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: way to get response headers only?

Matt White — 2012-04-30T14:32:17

Just use .head instead of .gethttps://
www.goldstar.com/shows/555643/checkout/new

On Mon, Apr 30, 2012 at 8:21 AM, shig odani <sodani< at >gmail.com> wrote:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

way to get response headers only?

shig odani — 2012-04-30T14:21:12

Hello, I'm interested in grabbing the final uri of a given uri after any
redirects without retrieving the response body of this final uri. For
example, if uri1 redirects to uri2, which in turn redirects to uri3, I'd
like to just grab the response header of uri3.

I know that I can get the full response (header and body) by doing
Mechanize.new.get(uri1).uri, but again, I'm looking to make just a header
request.

Thanks.
_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Resource time out error handling

Ross Cameron — 2012-03-27T05:25:27

Hi

I have a long running process and eventually get the following error:

/home/linker/.rvm/gems/ruby-1.8.7-p358/gems/net-http-persistent-2.5.2/lib/net/http/persistent.rb:821:in 
`request': too many connection resets (due to Resource temporarily 
unavailable - Timeout::Error) after 0 requests on 1777511481920, last 
used 1332824337.78363 seconds ago (Net::HTTP::Persistent::Error)

I have not found a way as yet to trap Net:HTTP errors through Mechanize.

Would someone please point me in the right direction for dealing 
/handling this situation.

Thanks
Ross



_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

Re: Mechanize::ResponseCodeError: 500

Eric Hodel — 2012-02-18T02:11:13

Mechanize automatically uses SSL, there's no need to set anything.

Without working SSL a 500 could not be returned.

Re: Mechanize::ResponseCodeError: 500

Thiago Imolesi — 2012-02-18T01:59:14

Do u set to use ssl???

Att.
Thiago Imolesi

Em 17/02/2012, às 23:20, Eric Hodel <drbrain< at >segment7.net> escreveu:

_______________________________________________
Mechanize-users mailing list
Mechanize-users< at >rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users