gmane.network.wwwoffle.user

www.consortiuminfo.org/standardsblog/ blockswwwoffle?

Martin Bähr — 2008-07-09T11:36:13

hi, is anyone here able to access www.consortiuminfo.org/standardsblog/ through wwwoffle? i keep getting this message below, and i just don't get why they would block a proxy, and also how they detect something anyways. what does wwwoffle do to the request? is it possible to make wwwoffle send out the request exactly as it received it from the browser? ----------------- error message: --------------------- Precondition Failed We're sorry, but we could not fulfill your request for /standardsblog/article.php?story=20080708052706429 on this server. We have established rules for access to this server, and any person or robot that violates these rules will be unable to access this site. To resolve this problem, please try the following steps: * Ensure that your computer is free of viruses, Trojan horses, spyware or any other sort of malicious software. * If you are using any sort of personal firewall or browser privacy software, check to ensure that its settings do not cause your web browser to i

DontCache still ends up in outgoing

2008-03-01T18:11:30

Complaint: offline both $ wwwoffle SomeDontGetURL $ wwwoffle SomeDontCacheURL both say Requesting ThatURL and return $?=0 to the shell, even though WWWOFFLE intends to do no such fetching. At least one can do # grep 'not to get' /var/log/syslog wwwoffles[5218]: The URL 'http://example.net/f.jpg' matches one in the list not to get. to know about the former, but what about the latter? The latter still ends up in http://localhost:8080/index/outgoing/ Clicking on it there, still here offline, says Your request for URL http://en.wikipedia.org/w/index.php?title=List_of_thinking_errors&action=edit failed because it is on the list of hosts and/or paths that are not to be cached and cannot be requested when offline. Well, OK, then it should be barred from ending up in outgoing too. OK, to check for the latter one would do, after fetching, # less +/not\ possible /var/log/syslog (note I use maximum debug level for my messages) Anyway, if the shell returned 1 and a message for both, one could much easie

executives still use WWWOFFLE to keep track ofwhat's already read

2007-11-27T18:46:50

Gentlemen, assuming that one has "the works" in the latest computing equipment and network connections, why would one still use WWWOFFLE? Well, certainly one cannot keep track of what articles one has already read on a sites with many articles. So with Purge { age=-1 } one will always know that one has already read which articles by looking at their link colors, (etc. as I mentioned in one of my previous postings.)

404 with wwwoffle, page delivered without

Karsten Kruse — 2007-10-06T18:28:43

Hi, when i visit a certain URL i get error 404 with wwwoffle and the page without a proxy. Here is the URL: http://www.instructables.com/id /Bicyle-Power-for-Your-Television,-Laptop,-or-Cell-/?relatedLink (note that i broke the URL to fit it in here, it should read ...id/Bi...) Can anyone confirm and/or explain that? Here is my environment: Server - NetBSD 4.0_BETA2 i386 - wwwoffle version 2.9a Client - Debian 4.0r1 i386 - Mozilla Firefox 2.0.0.5 Thanks for your time. MFG, Karsten Kruse

Not quite wwwoffle - for mobiles and low bandwidthuse.

Ian Stirling — 2007-09-02T11:49:47

I've used wwwoffle for some years, and it 'just works' for me. So, naturally, when I realised a need for a possibly related bit of software, and lacking results from google. I started wondering if anyone had thought of, or knows of an implementation or proof of concept with wwwoffled. Basically, it's a two part web proxy to drastically reduce web usage bandwidth. One part resides on a mobile device with a (usually) poor bandwidth link, but relatively large amount of storage, that may occasionally be plugged into a high speed network. The other part is on server, connected via a fast connection to the internet. To quote a page I wrote describing this. ------------------------------------- "This is a brief page describing a web proxy optimised for use on devices with a reasonable amount of persistant storage, and very limited bandwidth. Once, each page linked to a subpage of contents, which remained static, and could be easily refreshed if it changed based on dates in the HTTP headers. Now,

Dan Jacobson does not fall for "SpamBLK"

2007-09-01T13:33:31

e> To: exp-IjSeDpznLsezQB+pC5nmwQ< at >public.gmane.org e> Subject: [WWWOFFLE-Users] only-same-host-frames e> The person you tried to send this email to is using SpamBLK... e> If you don't click the link above your previous email will be deleted. Wait, it must be a trap Dan! Don't click! It must be a spam-bot subscribed to the list, hoping to harvest "confirmed live ones". Well, I'll have him know that I just happen to be College Educated -- no easy mark. Ha!

only-same-host-frames

2007-09-01T01:08:43

Gentlemen, it's me again with another brilliant idea. You know when those sites pull in those ad frames, URL='http://news.com.com/Developing+nations+losing+spam+battle,+ Default Recursive Fetch options: stylesheets=0 images=0 frames=2 Frame=http://view.atdmt.com/M0N/iview/cntcmssc0770000080m0n/dire Frame=http://view.atdmt.com/MRT/iview/cntnkinf0250006355mrt/dire Well, there needs to be an only-same-host-frames variable, on the model of only-same-host-images, for those of us that use frames=yes.

SSL certificates

Andy Rabagliati — 2007-08-19T14:43:55

Folks, I have a slightly different use-case for wwwoffle - I scoop websites on one machine with wwwoffle, tar up the files, and pass them to another machine via UUCP. It allows me to provide web services for disconnected networks. I am upgrading to the latest version of wwwoffle, and have bumped into a problem or two. First, using Ubuntu Feisty and wwwoffle 2.9a-2 the creation of root certificates is not reliable on startup - I often get an empty file. There seems to be some discussion of this on the list, and maybe I need a newer version. Second, the path to those certificates seems to be hardcoded into the binary, at /etc/wwwoffle/certificates. I don't really like this, as I create dynamic wwwoffle instances on the fly, in /var/tmp/wwwoffle2345/* as a different user (uucp) and now it tangles up with my 'upstream' wwwoffle instance on the same machine. First prize for me is a way to disable all the SSL stuff completely, as all this happens unattended so SSL is not

extra spaces being added

Paul Slootman — 2007-07-19T10:26:40

While investigating http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=395009 (which is a wierd bug...) I noticed that pages received via wwwoffle were larger than directly received pages. Wwwoffle is adding spaces before /> strings. To demonstrate: # echo ""|wwwoffle-write --addheader http://foo.bar/bla # wwwoffle-read http://foo.bar/bla | cat -vet HTTP/1.0 200 OK^M$ Content-Type: text/html^M$ ^M$ $ # http_proxy=http://localhost:8080/ GET http://foo.bar/bla | cat -vet $ Now this shouldn't be a problem for properly written browsers (I've checked that it doesn't happen with e.g. text/plain content), but I suspect an off-by-one error somewhere... Paul Slootman

is it possible to have freshness information inresponse headers?

Miernik — 2007-07-03T17:35:20

Here is what I want to do: I want to turn off add-cache-info which I currently use to see if the page I am browsing is fresh (x minutes ago added to the page HTML), and instead I want some Firefox extension to present that information in Firefox statusbar. That's because the AddCacheInfo thing often breaks the page layout. For that I would need WWWOFFLE to have the "x minutes ago" information in some HTTP header in the response it gives to the browser. Is that possible, and if not now, can you consider to implement it?

URL-specification `default'

Albert Reiner — 2007-07-01T09:22:57

I just wanted to change the setting for purging from -1 to an age of 2m. From I followed the link to the description of the URL-Specification, , where I found this example: ,---- | *://*/* | Any protocol, Any host, Any port, Any path, Any args (This | is that same as saying 'default'). `---- Apart from the wording (should be "THE same", I suppose), when I entered "default" for the URL and requested the change to be performed, I got a page complaining that the change failed as a URL specification had been expected but "default" received. (It worked with "*://*/*".) User bug? Documentation bug? Or should wwwoffle be changed to allow that syntax? This is with version 2.9a. Best regards, Albert.

Recursive fetch with links to identical items

Albert Reiner — 2007-06-23T15:04:18

I just wanted to make sure my understanding is correct: Suppose I am doing a recursive fetch with a high depth, and many of those pages are interlinked or contain, e.g., references to the same graphics files (logos etc.). Will wwwoffle be so smart to skip the pages it already downloaded in this batch, or will it simply continue following links until the depth is exceeded? My assumption is that wwwoffle is smart enough not to fetch the same file more than once during a single run. On a related note[*]: Is there any defined relation between the order of the output of `wwwoffle-ls outgoing' and the order in which pages are fetched? Thanks in advance for any light you may shed, Albert. [*] The relatedness stemming from the fact that the same graphics file URL kept appearing and vanishing again from the top position of the list of outgoing URLs, i.e., `wwwoffle-ls outgoing | head -1'.

bug: disable-script can miss ""

Maxim Kirillov — 2007-06-19T04:13:10

for this document: ---------------- aaa ----------------- setting disable-script = on produces the following: ---------------------- ---------------------- this is wrong. the real world example is here: http://www.microsoft.com/downloads/details.aspx?FamilyID=aea55f2f-07b5-4a8c-8a44-b4e1b196d5c0&displaylang=en looks like parser cannot detect the script end

is an URI with & and no ? valid?

Miernik — 2007-06-17T05:03:46

I have a problem with a website: Sniffed direct connection headers: GET /register.html&termsread=1&agree_to_terms=1 HTTP/1.0 User-Agent: Wget/1.10.2 Accept: */* Host: www.moneymakergroup.com Connection: Keep-Alive HTTP/1.1 200 OK Date: Sun, 17 Jun 2007 04:45:20 GMT Server: Apache/2.0.52 (Red Hat) X-Powered-By: PHP/5.2.2 Connection: close Content-Type: text/html; charset=UTF-8 Now the same through wwwoffle: GET /register.html%26termsread=1%26agree_to_terms=1 HTTP/1.0 User-Agent: Wget/1.10.2 Accept: */* Host: www.moneymakergroup.com Connection: close HTTP/1.1 301 Moved Permanently Date: Sun, 17 Jun 2007 04:44:50 GMT Server: Apache/2.0.52 (Red Hat) X-Powered-By: PHP/5.2.2 Location: http://www.moneymakergroup.com/register.html&termsread=1&agree_to_terms=1 Content-Length: 0 Connection: close Content-Type: text/html; charset=UTF-8 GET /register.html%26termsread=1%26agree_to_terms=1 HTTP/1.0 User-Agent: Wget/1.10.2 Accept: */* Host: www.moneymakergroup.com Connection: close HTTP/1.1 301 Moved Permanently

Access to proxy< at >8080 or 8082

Joshua Fein — 2007-06-16T06:24:31

The way I use wwwoffle is to have two images running at the same time; one configured to be permanently off-line(8082), the other(8080) permanently on-line, but taken off & on periodicly to keep the indexes from getting too large. What I am wondering is if there is a way (say on the browser bar) to address one or the other to fetch a url explicitly; i.e. http://localhost:8082/get?http://news.bbc.co.uk/ to get the cached version or http://localhost:8080/get?http://news.bbc.co.uk/ to get an on-line version. Pardon if this is a naive question: How does htdig do it? Does wwwoffle have to be offline when htdig is running?

multi-letter state symbols for info/content

2007-06-15T03:07:57

In http://localhost/info/content? 'X' if the URL is in the DontGet section of the configuration file, '+' if the URL is in the cache, '~' if it has been requested and '-' if it is not cached. Actually there should be several state indicators: [DC] in DontGet, but already in the cache, [dC] not in DontGet, but already in the cache. Wait, there are also images=no images, so a third field, "not to be fetched", is called for, etc. As many fields as necessary to nail down (fully describe) the status of the URL. As this isn't a page we look at all day, it's OK to fill it up with more fields. Otherwise users will be baffled as to why images show up as "+", even though one has images=no (today, but not last week!) P.S., just like "ls -l" "drwxr-xr-x" the future fields should line up nicely, no matter what their toggle/state.

lining up the columns in index/monitor

2007-06-15T00:12:26

Gentlemen, how should the WWWOFFLE author best line up the URLs etc. in index/monitor? # [L=10:6;N=31:0] http://traffic.tccg.gov.tw/ # [L=105:5;N=31:0] http://wiki.debian.org.tw/ # [L=33:2;N=25:17] http://wiki.debian.org.tw/index.php/CurrentEvents # [L=105:2;N=17:17] http://wiki.debian.org.tw/index.php/Main_Page # [L=165:7;N=15:17] http://wiki.debian.org/OngoingTransitions # [L=13:2;N=16:17] http://wiki.boringhost.com/index.php/Talk:MediaWiki Perhaps by using a printf(3) statement with constant widths for the numbers? Hmmm, currently we see

[L=126:0;N=31:0] route instead of

. Surely in http://www.useit.com/alertbox/ somewhere it must say "Non-lined up columns are bad news. 66% of users over the age of 66 reported nausea and confusion."

how much space is needed for htdig index?

Miernik — 2007-06-06T08:37:11

Can someone who uses htdig with wwwoffle can tell me for example how much disk space the htdig index etc files take for a corresponding wwwoffle cache size? My wwwoffle cache is 1.85 GB and I have only 0.35 MB of free disk space, and I wonder if I need to get more free disk space before starting the indexing, or will it be fine? I never ran htdig before, so I don't have an estimate how much it might be.

could WWWOFFLE not make AAAA DNS requests if itdoesn't bind to any IPv6 addresses?

Miernik — 2007-06-02T10:24:54

Currently I think WWWOFFLE makes AAAA DNS requests even if it doesn't bind to any IPv6 addresses, and there a no IPv6 addresses on the system at all. These AAAA queries are completely useless, just waste a bit of time. At least could there be an option in the config file not to make any AAAA DNS queries? As on a system without IPv^ connectivity (as most systems are), it could't use the result anyway even if it got a response from the DNS server (in most cases it doesn't). On systems with long RTT to the Internet (think satellite connection), superflous DNS queries generate significant delays.

POST value is lost if URL aliased?

Miernik — 2007-06-02T19:26:08

In my Alias section I have: http://www.linuxbios.org/* = http://linuxbios.org/* When I go to http://linuxbios.org/mailman/options/linuxbios and I fill in the Email address: and Password: the form is posted by the browser to the URL with www:

Release of sho_title0.9

Joshua Fein — 2007-03-18T12:41:24

The indispensable Post-Script: P.S The download URL is: https://sourceforge.net/project/platformdownload.php?group_id=182008 Thanks again, Joshua Fein