gmane.comp.sysutils.puppet.devel

Re: Defining a standard mechanism for template specialization

Henrik Lindberg — 2013-05-18T16:40:25

There are features in ARM-3 (Puppet Templates) that will help with 
several of the issues. One such feature is the ability to pass 
parameters to the template, and that a template can nest other templates.

https://github.com/puppetlabs/armatures/blob/master/arm-3.puppet_templates/puppet_templates.md#passing-arguments-to-the-template

The ARM-8 (Data in Modules) is also important and work is underway to 
define a successor to Hiera ("Hiera 2") with a more general binding scheme.

Puppet Templates is implemented and can be tried out (information in the 
ARM).

regards
- henrik

Re: Defining a standard mechanism for template specialization

Jeff Goldschrafe — 2013-05-18T15:34:20

A $content variable works great in certain instances, but there are others 
where this particular abstraction falls apart and I've been unable to 
determine a better one without a sane mechanism for template 
specialization. For example, if you're creating a defined type that 
contains a file resource, and you want the template to use variables of 
that defined type instance (such as $name/$title, for example) this isn't 
an easy thing to accomplish if you're calling template() from within a 
different scope in order to pass in content. In the case of this particular 
problem, the only semi-sane way to approach it is to create a wrapper 
defined type, which has a really bad code smell to me. And even then, you'd 
better hope you only need the variables actually passed into the type, and 
not any derived variables ($whatever_real, etc.).

I'm not sure I buy the point about the wide variety of paths to be searched 
making this a very difficult problem -- this is exactly what we're doing 
today with Hiera, and

Re: Defining a standard mechanism for template specialization

Daniel Pittman — 2013-05-17T21:02:32

I was thinking of either of these two when I said concatenation:

    $one = template(...)
    $two = template(...)
    notice("${one}${two}")  # concat!

    $three = concat($one, $two)  # with the obvious implementation

...but, yeah, the module also satisfies. :)

Basically, the language already allows you to do string pasting
without too much hardship.  Might as well do that the same way you do
other string pasting.

--
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons

Re: Defining a standard mechanism for template specialization

Ashley Penney — 2013-05-17T20:56:04


For what it's worth, this is something I would like to see.  I found the
concatenation to be more of a pain than anything else, especially since the
concat module came about.  Selecting the first entry would be an overall
improvement.

Re: Defining a standard mechanism for template specialization

Daniel Pittman — 2013-05-17T20:08:16

On Fri, May 17, 2013 at 8:23 AM, Jeff Goldschrafe
<jeff< at >holyhandgrenade.org> wrote:

[...]


I think the best solution would be to change the `template` function
to select the first entry, rather than concatenate, and allow users to
manually concatenate multiple `template` calls if they wanted it.

I understand the attraction of having an automatic, rather than
manual, model for searching.  I don't think it is a realistic goal,
however, because of the wide variety of "search paths" that could be
desired.  Explicitly spelling out what is searched seems like the best
balance of flexibility and extensibility to me.

--
Daniel Pittman
⎋ Puppet Labs Developer – http://puppetlabs.com
♲ Made with 100 percent post-consumer electrons

Re: Defining a standard mechanism for template specialization

Trevor Vaughan — 2013-05-17T19:50:52

In this particular scenario, I tend to lean toward having a 'content'
variable in my class or define.

class foo (  $content => template('foo/awesome.erb') ) { ... }

Someone wants to override that:

class { 'foo':
  content => template('bar/my_awesome.erb')
}

I'm not sure if that covers 100% of the cases that you're concerned with,
but it tends to work well for me as a pattern.

Looking at many of the Forge modules, the main issue appears to me to be
that the authors are solving problem X and don't want to create a module
that handles X^Y configuration options due to time, cost, and manpower
issues (all of which are completely understandable). Also, the effort
required to understand and extend an elegant module may be more than people
are willing to put in to solve what is, otherwise, a 30 minute problem.

Finally, you run into the situation where someone did take the time to
create a completely awesome module but it's so granular that using quite a
few of them like that trends toward compiled graph bloat

Defining a standard mechanism for template specialization

Jeff Goldschrafe — 2013-05-17T15:23:59

There are a few tickets floating around, like #7635, that deal with the 
inability to specialize templates in standard Puppet. Among other things, 
this is a limitation that makes it somewhat difficult for Forge module 
authors to distribute truly reusable modules without providing a cadre of 
non-interoperable custom functions to address this concern (see: 
multi_source_template). These tickets propose very specific solutions in 
terms of semantics. I'd like to open a broader feature discussion about how 
templates can/should be specialized in Puppet manifests, if people think 
they should be specialized at all. (I'm primarily concerned with templates, 
but there's probably no reason this discussion couldn't also apply to 
sourced files as well.)

I'm focused on the following three stories that I've seen come up on 
lists/IRC from time to time:

   - As a user, I want to provide different versions of a template 
   depending on some characteristic of the system so I don't need to encode 
   this differentia

Re: Could not retrieve ipaddress on Windows2003

2013-05-17T02:56:10

this must be your agent box's hostname contain some invalid characters.

 --puppet option will load sys/ruby/lib/ruby/1.8/ipaddr.rb on windows, and 
this will override IPSocket.getaddres method. it will check the hostname
and the hostname must match [-A-Za-z\d._]+  pattern.

On Wednesday, March 27, 2013 4:38:49 PM UTC+8, windops wrote:

Re: #18393 - diff args

Alex Harvey — 2013-05-16T06:01:24


On Thursday, May 16, 2013 3:31:09 AM UTC+10, Jeff Weiss wrote:

Fair enough, but if you're going to keep this approach in the code and 
willing to pay the price of calling facter, then it would be trivial to 
have it just work the same way on all platforms - after all you may end up 
with a paying customer who just happens to have some old AIX5.2 lying 
around or legacy Solaris.

Re: #18393 - diff args

Jeff Weiss — 2013-05-15T17:31:09


The pull request that Andy mentioned and that Chris merged is to provide an
adequate default diffargs on all of our commercially supported platforms.
Diff is off by default. If one goes into puppet.conf to enable diff, then
on our commercially support platforms it will just work. On anything
outside our commercially support platforms (as is the case for HP-UX and
legacy Solaris), one is in puppet.conf anyway to turn on diff and can
therefore set diffargs to something reasonable on the unsupported platform.

Re: #18393 - diff args

Chris Price — 2013-05-15T16:44:13

Andy,

Sorry, for some reason when I was reviewing that pull request I was
thinking it was targeted at our AIX branch. I probably should have checked
with your team before merging it.  If you guys decide that we should back
that out and solve the problem a different way, we will be happy to work
with you on that.


On Wed, May 15, 2013 at 9:35 AM, Andy Parker <andy< at >puppetlabs.com> wrote:

Re: #18393 - diff args

Andy Parker — 2013-05-15T16:35:17

Right, I hadn't noticed at first that AIX was not part of the original
report. I had read the issue after it had been updated to say AIX and
had missed the HP-UX part.


My problem with all of these solutions is that they all increase the
startup time for everything. The one that was submitted calls facter
on startup, and your solution would execute diff (and presents other
issues related to getting what the diff program is since that is
configurable). That doesn't seem ideal, and granted, it also isn't
ideal that puppet can't run with the default settings on some
platforms.


I would be interested in seeing the proposal, but I almost get the
sense that the installation process should really just be laying down
the right default puppet.conf.




--
Andrew Parker
andy< at >puppetlabs.com
Freenode: zaphod42
Twitter: < at >aparker42
Software Developer

Join us at PuppetConf 2013, August 22-23 in San Francisco -
http://bit.ly/pupconf13
Register now and take advantage of the Early Bird discount - save 25%!

Re: #18393 - diff args

Alex Harvey — 2013-05-15T04:30:20


On Wednesday, May 15, 2013 4:54:33 AM UTC+10, Andy Parker wrote:

I have two objections to that solution.

Firstly, AIX5.3 isn't the only operating system not supporting diff -u - 
it's a problem on all HP-UX, as well as on some legacy Solaris.

Secondly, if diff -u isn't available it may make sense to try diff -c 
instead.  The output from that is I think more readable.

If we were making Puppet smarter, perhaps the thing to do is make an 
additional call to 'diff' to see if it supports the default args - if it 
doesn't it will return exit code 2.

I suppose I could submit a patch that does this if the solution sounds 
good.  That said, I think being able to configure the diff_args in 
puppet.conf is also an adequate solution.

Facter 1.7.x branch deletion

Josh Cooper — 2013-05-14T19:50:01

Hello,

With the release of Facter 1.7.1[1], we'd like to delete the `1.7.x` branch 
in
an effort to streamline the contribution process to Facter by using 
statically
named branches, `stable` and `master`. Moving forward, any bug fix that 
would
have gone into `1.7.x`, should instead be targeted at `stable`, while new
features and functionality should be targeted at `master`.

Fortunately, there are no open PRs against Facter `1.7.x`, so this process 
should
be painless.

If you have any questions or concerns, please let us know on the mailing 
list
or #puppet-dev on irc.freenode.net.

Thanks,
Josh

[1] https://groups.google.com/d/topic/puppet-users/pwo1YgHeqEA/discussion

Re: #18393 - diff args

Andy Parker — 2013-05-14T18:54:33

It looks like the PE team hit this problem for AIX and created a fix
where they change the default diff_args based on the platform:
https://github.com/puppetlabs/puppet/pull/1637

So it looks like puppet is going to try and be smart by default.

On Fri, May 3, 2013 at 10:32 PM, Alex Harvey <alexharv074< at >gmail.com> wrote:

Re: Puppet 3 considerably slower than Puppet 2.6

Allan Yung — 2013-05-14T14:39:42

Hi Erik

The 2.6 master uses mongrel with an apache loadbalancer in front, 3.1 uses 
passenger (switching between passenger and webrick has no effect on the 
evaluation speed)

The masters are in the same physical location. I even switch a 2.6 client 
between masters and experience the slowness only on the 3.1 master.

Does the evaluation time include time to retrieve metadata from the master? 
It seems unlikely that the client is establishing an SSL connection and 
doing the md5 comparison in 0.02 seconds (2.6 master). Maybe when talking 
to the 3.1 master it has to establish a new SSL connection everytime?

Regards,
Allan

On Tuesday, 14 May 2013 12:23:49 UTC+1, Erik Dalén wrote:

Re: Puppet 3 considerably slower than Puppet 2.6

Erik Dalén — 2013-05-14T11:23:49

Do you have the exact same apache and passenger settings on both masters?

Having too few workers for example would lead it to waiting a lot for every
file metadata lookup.

It is also quite sensitive to network latency for similar reasons, are all
these hosts located at the same physical location?


On 14 May 2013 11:13, Allan Yung <allanyung< at >gmail.com> wrote:

Re: Puppet 3 considerably slower than Puppet 2.6

Allan Yung — 2013-05-14T09:13:27

Hi Andrew,

That was from the 3.1 agent talking to the 3.1 master.

If I point a 2.6 agent to the 3.1 master I get the same slow behaviour. 
Pointing the 2.6 agent back to the 2.6 master gives the fast performance as 
before.

As part of moving to puppet3/hiera etc the modules have been refactored, so 
there are differences in the class and method names. They essentially 
perform the same tasks though.

Here is a trace from a 2.6 agent running against the 2.6 master. As you can 
see the evaluations happen in a fraction of the time:

May 14 10:05:55 host1 puppet-agent[9526]: Starting Puppet client version 
2.6.6
May 14 10:05:55 host1 puppet-agent[9526]: Retrieving plugin
May 14 10:05:56 host1 puppet-agent[9526]: Loading facts in get_esd_env
May 14 10:05:56 host1 puppet-agent[9526]: Loading facts in 
configured_ipaddress
May 14 10:05:56 host1 puppet-agent[9526]: Loading facts in get_env_group
May 14 10:05:56 host1 puppet-agent[9526]: Loading facts in get_vlan
May 14 10:05:56 host1 puppet-agent[9526]: Loading f

Re: Question about COMMITTERS.md

Jeff McCune — 2013-05-14T03:50:18

Got it, yeah I have been operating under the assumption the two methods
yield equivalent results, and the merge-up variation feels somehow cleaner
to me personally, so that's what I've been doing as a matter of preference.

As an FYI, the team has a meeting tomorrow morning so we'll hopefully be on
the same page tomorrow afternoon and can talk more on the dev list then.

For the time being I whipped up a pull request that contins the difference
in the example process and how the history looks as a result of the
difference in the process.  This PR is a request for comments and not
necessarily what we're going to merge:
https://github.com/puppetlabs/puppet/pull/1645


On Mon, May 13, 2013 at 8:57 AM, Andy Parker <andy< at >puppetlabs.com> wrote:

Puppet Dev Community Summary (2012-05-13)

Jeff McCune — 2013-05-14T00:02:28

 Puppet Dev Community Summary

   - Finished Card Total: *15 vs 18 last week*
   - Merged: 12 vs 13 last week
   - Not Merged: 1 vs 2 last week
   - Other: 2 vs 3 last week

Please see the team's current work in progress, outstanding pull requests,
and backlog on the Puppet Dev Community
board<https://trello.com/board/puppet-dev-community/50bd46a84c27cb74100035f5>
.
 <https://gist.github.com/gepetto-bot/5166341#merged>Merged
<https://gist.github.com/gepetto-bot/5166341#pr-facter439-20617-add-support-for-zfsonlinux--fixtures-marc-fournier>(PR
facter/439) (#20617) add support for zfsonlinux + fixtures [Marc
Fournier]<https://trello.com/card/pr-facter-439-20617-add-support-for-zfsonlinux-fixtures-marc-fournier/50bd46a84c27cb74100035f5/995>

Merged into master in ee95b764e62. This should be released in 2.0.0.
<https://gist.github.com/gepetto-bot/5166341#pr-puppet1607-add-a-boolean-parameter-type-dustin-j-mitchell>(PR
puppet/1607) Add a boolean parameter type [Dustin J.
Mitchell]<https://trello.com/card/pr-puppet

Re: How to package Hiera 1.2.1 backends?

windowsrefund — 2013-05-13T19:25:05

I solved this by repacking hiera-redis from a gem to a deb

gem2deb ftw! :)