gmane.comp.sysutils.puppet.devel

Re: Understanding/minimizing config retrieval time

Eric Sorenson — 2013-05-24T20:10:58

On May 24, 2013, at 7:33 AM, christian.le.flamm< at >gmail.com wrote:



At that time the catalog is being compiled on the master, so you should look there -- in puppet 3.2 you can add the '--profile' flag to your agent run which will cause additional profiling information to be logged on the master.

See this link for more info:  http://docs.puppetlabs.com/puppet/3/reference/release_notes.html#better-profiling-and-debugging-of-slow-catalog-compilations

Eric Sorenson - eric.sorenson< at >puppetlabs.com
#puppet irc: eric0

Re: Understanding/minimizing config retrieval time

Christian Flamm — 2013-05-24T19:11:20

I'll try that, thanks.
Am 24.05.2013 21:00 schrieb "Dustin J. Mitchell" <dustin< at >v.igoro.us>:

Re: Understanding/minimizing config retrieval time

Dustin J. Mitchell — 2013-05-24T19:00:50

Does strace help to figure that out?

Dustin

Re: Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Josh Cooper — 2013-05-24T18:23:45


Filed as https://projects.puppetlabs.com/issues/20901 and targeted for 3.2.2



Josh

Understanding/minimizing config retrieval time

2013-05-24T14:33:11

Hi,
while trying to find things in our puppet setup I could tune I had a look 
into /var/lib/puppet/state/last_run_summary.yaml and saw the time consumed 
by "config_retrieval" which is always around 11 sec.

---
  resources:
    total: 983
    scheduled: 0
    restarted: 0
    changed: 2
    skipped: 6
    failed: 0
    out_of_sync: 2
    failed_to_restart: 0
  events:
    total: 2 
    success: 2 
    failure: 0 
  changes: 
    total: 2 
  time:
    user: 0.015028 
    total: 27.2075000184937 
    mount: 0.005127 
    group: 0.004601 
    cron: 0.001611 
    package: 1.009123 
    exec: 11.764566 
    filebucket: 0.000147 
    last_run: 1369400811
    service: 0.405391
    file: 2.528648
    config_retrieval: 11.4732580184937
  version: 
    config: 1369400578 
    puppet: "3.1.1"

Here - 
http://grokbase.com/t/gg/puppet-dev/1253sjpxkb/trying-to-isolate-performance-issues-with-config-retrieval - I 
read about catalog caching and that it might take a while. So I inserted 
the recommended extra bit of debug

Re: Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Andy Parker — 2013-05-23T18:08:29


Thanks for tracking it down to that. Charlie and I traced it a bit further
and found that inside webrick it loads webrick/config.rb which sets up a
hash that contains the servername that is determined by calling
WEBrick::Util.getservername which does these reverse lookups. That explains
the mystery of simply loading some code causing a DNS lookup to occur.

Looking at that patch, I'm not certain if we should simply revert it or
move that to another place in the code. My current thinking is that we
should simply revert it because it is only needed to support 2.6 and
earlier clients, which Puppet 3 does not support anyway (the policy is that
a master only supports one "major" version back for agents, and in the case
of 2.6 and 2.7 that means only 2.7 is supported, not 2.6).

Re: Announce: Puppet 3.2.1 Available

Matthaus Owens — 2013-05-23T00:08:42

I neglected to mention in the announcement email, to support the
"Future" parser, we have added dependencies on ruby-rgen to our puppet
packages that support dependency resolution (gem, deb, rpm). It is
available as a gem
on rubygems.org and packaged as a library in our apt and yum
dependencies repos.

On Wed, May 22, 2013 at 12:54 PM, Matthaus Owens
<matthaus< at >puppetlabs.com> wrote:

Re: Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Wil Cooley — 2013-05-22T23:56:56

On Wed, May 22, 2013 at 10:10 AM, Charlie Sharpsteen
<chuck< at >puppetlabs.com> wrote:


Regardless of which state /etc/hosts is in, it always returns the
short name: "coolair".

Wil

Re: Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Charlie Sharpsteen — 2013-05-22T17:10:44


On Tuesday, May 21, 2013 3:03:28 PM UTC-7, Wil Cooley wrote:

Thanks a bunch for the info Wil! One additional question: What is the 
hostname that Ruby is using to perform the lookup? I.E., what is the return 
value  of:

  ruby -rsocket -e 'puts Socket.gethostname'

-Charlie

Re: Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Josh Cooper — 2013-05-22T16:53:49


I agree this should be restricted to code paths that actually use webrick



In the case of the windows-specific monkey patches for openssl, there is no
difference putting them in lib/puppet/util/monkey_patches.rb or
lib/puppet/util/windows.rb, as both are always loaded via require 'puppet'.

The reason for not moving them to the puppet's http client code, e.g.
lib/puppet/network/http/connection.rb, is because not all modules use
puppet's http client for making connections. For example, the module tool
uses open-uri, as do other providers. With that said, we could move the
code to only load the root certs if we're going to launch an application,
and do so before we invoke the application.


Josh

Re: Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Brice Figureau — 2013-05-22T13:28:29

[snip]

Yes, I concur that it's a bad thing to load webrick on a command-line
puppet (or even the agent if it doesn't listen). Those monkey patches
should be splitted depending on the puppet use case.

I suggest you file a redmine ticket about this issue (and I'm sure
you'll certainly won't be the first to be hit by this dns resolution
issue).

Re: Is Puppet::Provider::Package#query required?

JE Fritz — 2013-05-22T00:49:26


Ah!  Thanks.  I'd missed that query call. (After a fair amount of looking,
too!  But, for that file, only with eyeballs, not grep. Ah well.)

-Fritz

On Tue, May 21, 2013 at 1:38 PM, Adrien Thebo <adrien< at >puppetlabs.com> wrote:

Re: Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Wil Cooley — 2013-05-21T22:03:28

On Tue, May 21, 2013 at 9:53 AM, Charlie Sharpsteen
<chuck< at >puppetlabs.com> wrote:



Sure; in my /etc/hosts I had the following for my host-only vbox net:

192.168.56.1   coolair.local coolair

With that in place:

$ time ruby -rsocket -e "Socket.gethostbyname(Socket.gethostname)"
real 0m10.013s

And when I change it to:

192.168.56.1 coolair

I get:

$ time ruby -rsocket -e "Socket.gethostbyname(Socket.gethostname)"
real 0m0.032s

After a little tracing, it looks like it's the result of loading
webrick in puppet/util/monkey_patches.rb:

/Library/Ruby/Site/1.8/puppet/util/monkey_patches.rb:419::-: require 'webrick'

Which was added in ec462b7b: (#18781) Be more tolerant of old clients
in WEBrick server

Since this is just twiddling a constant (does that count as
monkey-patching?), wouldn't this maybe be better done in
lib/puppet/network/http/webrick.rb? If I understand it correctly,
these monkey-patches get applied for every invocation of puppet, even
if it's just "puppet --version", so it seems like it shou

Re: Is Puppet::Provider::Package#query required?

Adrien Thebo — 2013-05-21T17:38:41

It looks like PackageProvider#query is being called in the #properties
method at (
https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package.rb#L19).
I deleted that method and ran the specs to see what blows up:

  1) Package provider gem should be able to get a list of existing packages
     Failure/Error: package.properties[:provider].should == provider.name
     NoMethodError:
       undefined method `properties' for
(provider=gem):Puppet::Type::Package::ProviderGem
     # ./spec/integration/provider/package_spec.rb:39:in `block (5 levels)
in <top (required)>'
     # ./spec/integration/provider/package_spec.rb:37:in `each'
     # ./spec/integration/provider/package_spec.rb:37:in `block (4 levels)
in <top (required)>'

  2) Puppet::Type::Package::ProviderPip query should return a hash when pip
and the package are present
     Failure/Error: < at >provider.query.should == {
     NoMethodError:
       undefined method `properties' for
(provider=pip):Puppet::Type::Package::ProviderPip
     # ./l

Is Puppet::Provider::Package#query required?

John E Fritz — 2013-05-17T19:30:41

All the package providers supplied with Puppet seem to implement or inherit 
the query method, but as far as we can tell, this isn't used in other parts 
of the system.  Is it required?

Until June 2007, it was in used by the package type (in 
lib/puppet/type/package.rb) in the exists? and retrieve method, but 
commit 73502a7 got rid of those (but left a comment mentioning query above 
the exists? method).

We don't want to leave out something people rely on, but we also don't want 
to do unnecessary cargo-culting.

John Fritz and Jim Toth

Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Charlie Sharpsteen — 2013-05-21T16:53:46


On Monday, May 20, 2013 11:21:48 PM UTC-7, Wil Cooley wrote:

Hi Wil,

Could you provide a bit more information on what the entry in your 
/etc/hosts file looked like? It would be interesting to reproduce this 
behavior and see if everything is functioning as intended.

Thanks!

-Charlie

Re: Performance regression between 3.2.1-rc1 and 3.1.1?

Wil Cooley — 2013-05-21T06:21:48


After realizing that even 'puppet --version' took ~10s, I ran the ruby
profiler, noticed there were differences in the number of  calls, extracted
& diff'd those, noticed the addition of Socket#gethostbyname and
Socket#gethostname. Eventually I noticed that my /etc/hosts had
myhostname.local instead of just myhostname which (I presume) was causing a
DNS/mDNS lookup.

So unless anyone else has this problem, forget I said anything.

Wil

Performance regression between 3.2.1-rc1 and 3.1.1?

Wil Cooley — 2013-05-21T05:22:26

I am noticing a ~10s slow down with 3.2.1-rc1 running 'puppet apply' with a
manifest that is a single notice() function. I think I noticed it first
w/'puppet resource' and maybe also with other sub-commands. This slow down
might be just statistical noise in runs with a non-trivial manifest, but
it's intrusive while doing development.

I can provide the outputs of the 'puppet apply' below, but the pause seems
to happen before any output starts and all of the times that are reported
look normal (0.00s for a bunch of built-in automatic resources), with
catalog runs ending with "Finished catalog run in 0.0[56] seconds".

In fact, it almost looks like someone put in a sleep(10) somewhere while
debugging and forgot to take it out.

This is on Mac OS X 10.7.5.

$ uname -a
Darwin coolair 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT
2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64

$ ruby --version
ruby 1.8.7 (2012-02-08 patchlevel 358) [universal-darwin11.0]

$ facter --version
1.7.1

$ shasum puppet

Re: Defining a standard mechanism for template specialization

Henrik Lindberg — 2013-05-18T16:40:25

There are features in ARM-3 (Puppet Templates) that will help with 
several of the issues. One such feature is the ability to pass 
parameters to the template, and that a template can nest other templates.

https://github.com/puppetlabs/armatures/blob/master/arm-3.puppet_templates/puppet_templates.md#passing-arguments-to-the-template

The ARM-8 (Data in Modules) is also important and work is underway to 
define a successor to Hiera ("Hiera 2") with a more general binding scheme.

Puppet Templates is implemented and can be tried out (information in the 
ARM).

regards
- henrik

Re: Defining a standard mechanism for template specialization

Jeff Goldschrafe — 2013-05-18T15:34:20

A $content variable works great in certain instances, but there are others 
where this particular abstraction falls apart and I've been unable to 
determine a better one without a sane mechanism for template 
specialization. For example, if you're creating a defined type that 
contains a file resource, and you want the template to use variables of 
that defined type instance (such as $name/$title, for example) this isn't 
an easy thing to accomplish if you're calling template() from within a 
different scope in order to pass in content. In the case of this particular 
problem, the only semi-sane way to approach it is to create a wrapper 
defined type, which has a really bad code smell to me. And even then, you'd 
better hope you only need the variables actually passed into the type, and 
not any derived variables ($whatever_real, etc.).

I'm not sure I buy the point about the wide variety of paths to be searched 
making this a very difficult problem -- this is exactly what we're doing 
today with Hiera, and

Re: Defining a standard mechanism for template specialization

Daniel Pittman — 2013-05-17T21:02:32

I was thinking of either of these two when I said concatenation:

    $one = template(...)
    $two = template(...)
    notice("${one}${two}")  # concat!

    $three = concat($one, $two)  # with the obvious implementation

...but, yeah, the module also satisfies. :)

Basically, the language already allows you to do string pasting
without too much hardship.  Might as well do that the same way you do
other string pasting.

--
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons