gmane.comp.web.varnish.misc

Re: Unable to purge using CURL on VARNISH

Stephen Wood — 2013-05-23T19:49:48

Are you sure you're getting back a "200 purged" message, or just a 200?


On Thu, May 23, 2013 at 12:32 PM, <me-OOeHgxleHsGsTnJN9+BGXg< at >public.gmane.org> wrote:

_______________________________________________
varnish-misc mailing list
varnish-misc-nxvTQTwvqOZYtebHI+S8e9i2O/JbrIOy< at >public.gmane.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Unable to purge using CURL on VARNISH

2013-05-23T19:32:14

hi,


maybe you need to set a Host header with curl's -H option. if that's not
the problem, try to use varnishlog and see, what's going on.

good luck,
 tobias florek

Unable to purge using CURL on VARNISH

Puneet — 2013-05-23T17:06:42

Hi All,

I am unable to purge objects in varnish using CURL

CURL command: curl -X PURGE localhost/public/stylesheets/main.css -v the 
curl command is returning 200 OK response. the object is not getting purged

config file in VARNISH : default.vcl

     sub vcl_miss {
         if (req.request == "PURGE") {
             purge;
             error 404 "Not in cache.";
         }
         if (req.http.user-agent ~ "spider") {
             error 503 "Not presently in cache";
         }
         return (fetch);
     }

     sub vcl_hit {
         if (req.request == "PURGE") {
             purge;
             error 200 "Purged.";
         }
         return (deliver);
     }

     sub vlc.recv{
         if (req.request == "PURGE") {
             if (!client.ip ~ purge) {
                 error 405 "Not allowed.";
             }
         return(lookup);
         }
     }

Kindly, help on this

Thanks
Puneet

_______________________________________________
varnish-misc mailing list
varnish-misc-nxvTQTwvqOZYtebHI+S8e9

reg: cache hits taking longer duration

Ambati, Sugunakar — 2013-05-20T05:03:24

Hi,

we have been using varnish for the past few years.

Recently I observed that few requests are taking longer than expected. I am hitting my website say www.abc.com<http://www.abc.com>.

OS: CentOS 5.8 Linux kernel: 2.6 Varnish version: 2.1.4-1

Here is the summary of my test:
1.            Send requests one by one to www.abc.com using curl
2.            Measure the time using curl utility
3.            Total requests sent 2000 ; time taken to send them 30-40 seconds
4.            Observartions: most of requests get served within 0.3 seconds and few take more than 1 seconds and 2-3 requests take up more than 3 seconds
5.            The requests taking 3 seconds are served from cache as I could see the X-varnish headers with 2 numbers.

Need your help in understanding why it takes 3 seconds to serve from cache?

Thanks
Sugunakar

_______________________________________________
varnish-misc mailing list
varnish-misc-nxvTQTwvqOZYtebHI+S8e9i2O/JbrIOy< at >public.gmane.org
https://www.varnish-cache.org/lists/mailma

varnishstat reference

Sean Allen — 2013-05-19T20:20:28

is there is good reference source to learn about what all the different
stats in listed by varnishstat are other than reading the code?

Error 503 : Junk after gzip data

Florian Nitzsché — 2013-05-15T22:25:04

Hello,

I got a 503 error with the following line in logs :

9 FetchError   c Junk after gzip data

The page works fine out of varnish, does anybody why it happens ?
I've made a gzip test on my page and there are no encode errors.

Thank you very much

_______________________________________________
varnish-misc mailing list
varnish-misc-nxvTQTwvqOZYtebHI+S8e9i2O/JbrIOy< at >public.gmane.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: varnish-cache-3.0.3-plus increase %CPU and crashed by "errno = 12(Cannot allocate memory)"

Xianzhe Wang — 2013-05-15T08:21:22

Hi,
  I use the order:
      ps H -e -o pid,tid,pcpu,cmd --sort=pcpu |grep varnish

I get the message:
12255 12280  0.1 /opt/ace/cache/varnish-cache-3.0.3-plus/sbin/varnishd -f
/opt/ace/cache/varnish-stream.vcl -s
persistent,/opt/ace/file/varnish_cache_persistent.data,12G -T 127.0.0.1:2000-a
0.0.0.0:8080 -h classic,500009 -p thread_pool_min 200 -p thread_pool_max
2000 -p thread_pool_add_delay 2 -p session_linger 100 -p thread_pools 2 -p
http_req_hdr_len 32768 -p http_resp_hdr_len 32768 -p http_max_hdr 256 -p
lru_interval 3600
12255 30175 22.6 …… （omit startup command）
12255 26385 87.2 …… （omit startup command）
12255 30478 89.1 …… （omit startup command）
12255 21623 96.3 …… （omit startup command）
12255 12504 96.7 …… （omit startup command）

There are several threads consume about 100% CPU.

Then I debugged varnish, attach the threads above. I found they lingered
in in VGZ_WrwGunzip .
I think there may be  some *endless loop* or *deadlock* issues.

There are some debug log

varnish-cache-3.0.3-plus increase %CPU and crashed by "errno = 12(Cannot allocate memory)"

Xianzhe Wang — 2013-05-15T02:44:48

Hi,
    I used varnish-cache-3.0.3-plus(get from
https://github.com/mbgrydeland/varnish-cache/tree/3.0.3-plus).
I use persistent storage and "set beresp.do_stream = true".
The %CPU  is increased from 4% to 800% (8 cpu), and varnish crashed
sometimes by errno = 12 (Cannot allocate memory).

my startup command is:
/opt/ace/cache/varnish-cache-3.0.3-plus/sbin/varnishd -f
/opt/ace/cache/varnish-stream.vcl -s
persistent,/opt/ace/file/varnish_cache_persistent.data,12G -T 127.0.0.1:2000-a
0.0.0.0:8080  -p thread_pool_min 200 -p thread_pool_max 2000 -p
thread_pool_add_delay 2 -p session_linger 100 -p thread_pools 2 -p
http_req_hdr_len 32768 -p http_resp_hdr_len 32768 -p http_max_hdr 256

Then I change the command  to:
/opt/ace/cache/varnish-cache-3.0.3-plus/sbin/varnishd -f
/opt/ace/cache/varnish-stream.vcl -s
persistent,/opt/ace/file/varnish_cache_persistent.data,12G -T 127.0.0.1:2000-a
0.0.0.0:8080 *-h classic,500009* -p thread_pool_min 200 -p thread_pool_max
2000 -p thread_pool_add_delay 2 -p session_linger 100 -

Re: benchmarking varnish

Miguel González — 2013-05-14T18:07:58



On 13/05/2013 05:33, Miguel González wrote:

It seems one of the issues was that pingdom was misleading because It 
always reported all tests were resulting into a MISS instead of a HIT. 
Checking with wget -S gave some real information.

Also normalizing user agents could help when it comes to caching images 
and certain files:

http://serverfault.com/questions/365056/varnish-only-cache-assets-from-single-session

this seems to have increased the hitrate

Regards,

Miguel

Re: benchmarking varnish

Miguel González — 2013-05-13T03:33:46

Sorry I haven't replied for so long. I was trying to make sense of the docs.

 From my munin monitoring and varnishstat it seems I'm getting around 
80% of hitrate in my Varnish cache. This Varnish cache has a remote 
Apache backup (Varnish is in a server in Europe while Apache is in a 
server in the US.

How can I pinpoint why is not working fast?

My default.vcl and /etc/sysconfig/varnish

backend default {
   .host = "XXX.XXX.XXX.XXX";
   .port = "8000";
   .connect_timeout = 600s;
   .first_byte_timeout = 600s;
   .between_bytes_timeout = 600s;
}

acl purge {
   "localhost";
}

acl ban {
   "localhost";
}

sub vcl_recv {
   // Strip cookies for static files:
   if (req.url ~ 
"\.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|pdf|txt|tar|wav|bmp|rtf|js|flv|swf|html|htm|xml)$") 
{
     unset req.http.Cookie;
   }

   // Remove specific cookies:
   set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", 
"");
   set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", 
"");
   set r

Varnish and mod_pagespeed

Miguel Gonzalez — 2013-05-10T17:52:21

Dear all,

   I have Varnish as a front-end and an Apache/PHP running a Symfony application as a backend.

   I had mod_pagespeed enabled in Apache long time before I decided to put Varnish in front of it. It worked well.

   Since we decided to put Varnish in front, we are getting many errors logged in error_log file of this kind:

   Fri May 10 08:29:15 2013] [warn] [mod_pagespeed 1.3.25.4-2941 < at >17719] Fetch 
failed for http://www.midomain.com/images//image2.jpg.pagespeed.ic.t1diDHbOw9.jpg, 
status=404

  and others like:
   

   [warn] [mod_pagespeed 1.1.23.2-2258 < at >28956] Fetch timed out: http://127.0.0.1:8000/media/cache/thumb_image_user6.jpg (35) waiting for 50 ms

   the result is that the webpage is not loading as fast as it did without varnish.

   I have googled around and I found webpages like this one:

   http://serverfault.com/questions/290776/mod-pagespeed-varnish-and-apache-cache-issues-after-new-code-pushes

   but since I'm quite new to Varnish I don't really know what I

Re: file storage engine overflow and 503 error

Sean Allen — 2013-05-10T13:20:58

For the edification of future parties, check out the nuke limit option. We
were pulling back some objects that were larger than 50x average object
size so nuking didn't create enough space.


On Wed, May 8, 2013 at 5:22 AM, Sean Allen <sean-c0x2keDUgVLIFemRkCxrTXtgZLXuOG4l< at >public.gmane.org>wrote:

Re: Does code belong in vcl_recv & vcl_fetch?

Per Buer — 2013-05-10T06:45:27

You do two totally different things in recv and fetch. In recv you can
modify the req object, changing the _request_. In fetch you modify beresp,
the backed response. Cookie is a request header, so change it in recv. If
you where to alter Set-Cookie OTOH, you would do so in fetch.

I would recommend to spend some time on the Varnish Book og the tutorial
"Using Varnish". Or, if you are in NYC on the User Group meetup around the
end of May there will also be a training session.

Does code belong in vcl_recv & vcl_fetch?

John — 2013-05-09T20:47:34

Using Varnish 3.0.2, I would like to cache a specific URL path. When
Varnish sends this path to the backend, Apache gathers data from external
services like Twitter.

Do I need to place this code in both the vcl_fetch and vcl_recv? I've
currently got it in both places, but I do not want to duplicate code
unless it's required (or a best practice).

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

# Respond to incoming requests.
sub vcl_recv {
[...]
# Always cache the following URLs to prevent frequent requests to external
# services. For example: Twitter at /ext/twittersearch
# This code appears twice, once here and again in vcl_fetch so make sure you
# edit both and keep them equal.
if (req.url ~ "^/ext/") {
  unset req.http.Cookie;
}

[...]

# Code determining what to do when serving items from the Apache servers.
# beresp == Back-end response from the web server.
sub vcl_fetch {
[...]
# Always cache the following URLs to prevent frequent requests to external
# services. For example: Twitter at /ext/twittersearch
# Thi

RE: Keep URL after rewriting

Rangel, Raul — 2013-05-08T15:57:58

So you said you set the Site-URL to www.example.com/corporate which I'm assuming makes urls like http://www.example.com/corporate/foo.php. You also said that the DocumentRoot is /. So by removing the corporate you end up with //foo.php. So you are expecting to find foo.php in the DocumentRoot. It might be the double slash that is breaking it. But I think apache will handle that correctly. Change your regsub to replace /corporate and see if that helps. Otherwise use varnishlog to look at your logs.

Raul

-----Original Message-----
From: Hauke [mailto:mailman-Pl9ukmwD5a7PTwkrwQOX7A< at >public.gmane.org] 
Sent: Wednesday, May 08, 2013 2:43 AM
To: Rangel, Raul; 'varnish-misc-nxvTQTwvqOZYtebHI+S8e9i2O/JbrIOy< at >public.gmane.org'
Subject: Re: Keep URL after rewriting

Hi Raul,

thanks for your advices.

for wordpress I configured the Site-URL to www.example.com/corporate. That will fix my hyperlinks.

But now I have the following problem:

As shown in the config

if (req.http.host ~ "example.com" && req.url ~ "^/corpora

Varnish URL rewrite having integers with scientific nos in URL

kevin fernandes — 2013-05-08T14:05:28

Hi All,

I would like to know how can i rewrite/convert a URL containing number is
scientific form to normal integer.

For eg:

I have something like this coming in the URL

http://domain.com/public/domain/app/1.024e+06/1028422/1028422_b24db5c8391ec5c5d93cd0ac083acbdf

I would like to have to have the URL having a normal integer so the output
should be

http://domain.com/public/domain/app/1024000/1028422/1028422_b24db5c8391ec5c5d93cd0ac083acbdf

Playing around something like this for testing

    if (req.http.Host ~ "domain.com") {
        set req.url = regsub(req.url,
"^/public/domain/app/[+|-]?\d\.?\d{0,}[E|e|X|x](10)?[\^\*]?[+|-]?\d+/(.*)/(.*)$",
"/public/domain/app/\d+/(.*)/(.*)$");
    }


Any ideas or any clues if the syntax for the matching scientific nos is
correct?

Regards,
Kevin
_______________________________________________
varnish-misc mailing list
varnish-misc-nxvTQTwvqOZYtebHI+S8e9i2O/JbrIOy< at >public.gmane.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

file storage engine overflow and 503 error

Sean Allen — 2013-05-08T09:22:48

Varnish 3 series.

We recently switched one of our varnish caches from being having a malloc
based storage to file based. When the cache overflowed using the malloc
based one, an older item was nuked and the requested item was returned.

When we overflowed the file storage engine cache yesterday, varnishstat
showed the items being evicted but would return a 503 for the requested
item and in varnishlog, was showing it couldn't allocate storage during the
deliver phase for the requested item.

What do I need to do to change to not have a cache overflow result in 503
errors?

Re: how to flush director dns ttl

Gao Yongwei — 2013-05-08T09:14:47

Soga,  looks very hard for me now but it may be very interesting ,I'll have
a try .Thanks again : )
_______________________________________________
varnish-misc mailing list
varnish-misc-nxvTQTwvqOZYtebHI+S8e9i2O/JbrIOy< at >public.gmane.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: how to flush director dns ttl

Per Buer — 2013-05-08T09:06:40

Hi Gao,

On Wed, May 8, 2013 at 10:58 AM, Gao Yongwei <itxx00-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:


You would need to create a VMOD that kills off the cache in the DNS
director. For this to work you would probably need to modify the DNS
director in the Varnish Cache source.

Re: how to flush director dns ttl

Gao Yongwei — 2013-05-08T08:58:09

The whole point of the DNS director and it's TTL settings is to have

Thanks,perbu ,I can achieve  the first method ,  and could you please teach
me more about the second method , how does this vcl look like ?
many thanks , :-)
_______________________________________________
varnish-misc mailing list
varnish-misc-nxvTQTwvqOZYtebHI+S8e9i2O/JbrIOy< at >public.gmane.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Keep URL after rewriting

Hauke — 2013-05-08T08:43:01

Hi Raul,

thanks for your advices.

for wordpress I configured the Site-URL to www.example.com/corporate. That will 
fix my hyperlinks.

But now I have the following problem:

As shown in the config

if (req.http.host ~ "example.com" && req.url ~ "^/corporate") {
    set req.http.host = "corporate-stag.example.local";
    set req.url = regsub(req.url, "corporate", "");
  }

I remove the _corporate_ part in req.url. That's because the webapp lies 
directly under the DocumentRoot of _corporate-stag.example.local_. If I click 
some hyperlink in the webapp like _http://www.example.com/corporate/foo.php_ 
this does not work because of removing the _corporate_ from the req.url.

Is there any other option for my case except of moving the whole webapp from 
DocumentRoot to DocumentRoot/corporate and don't delete the _corporate_ part 
in req.url? I tried this and this will work, but it's a little bit messy at 
all.

Thanks,