gmane.comp.lang.perl.modules.module-build

How to create META.json using Module::Build and how to link to the VCS?

Gabor Szabo — 2013-01-03T11:04:33

Hi,

recently I wrote an article how people could include a link to their
version control system in the META.yml file.
http://perl5maven.com/how-to-add-link-to-version-control-system-of-a-cpan-distributions

After getting a few comments I found out that beside the link to the
repository one could also include the type of
the repository (git/svn/etc...), but as far as I can see only
META.json holds that information.
At least in the distributions I checked only META.json had this.
All these distributions were generated using Dist::Zilla.

So I wonder how can ask Module::Build to generate META.json as well,
and how can I tell it
to include the other fields of the repository as well?

regards
   Gabor

How To Build A Perl Package Database

Michael G Schwern — 2012-12-15T22:59:03

We have a lot of serious problems because we lack a database of installed
distributions, releases and files.  There are serious problems with
implementing one given A) the limitations of the standard Perl install and B)
wedging it into existing systems.  But I think I have a solution.  Its similar
to how meta data was slipped into the ecosystem without requiring authors to
rewrite their releases or install a bunch of extra modules.  It just happens
as part of the normal CPAN module upgrade process.

I've been thinking that a minimal package database could be created by putting
some hooks into ExtUtils::Install::install(), which every Perl build system
ultimately uses, to record what gets installed.  That way when
ExtUtils::Install is upgraded, the user gets a build database without
upgrading everything else.

This would be a fairly straight forward process at install time...

1) Copy everything to a temp directory
2) Record everything in that temp directory
3) Copy everything from temp into the real location

You could probably optimize this by skipping the copy to temp and just have
install() record stuff as it goes by, but this is the dumb, simple, robust way
to do it.

Storage is a problem.  The only reliable "database" Perl ships with is DBM, an
on disk hash, so we can't get too fancy.  It might take several DBM files, but
this is enough to record information and do simple queries.  What are those
queries?

* What version of the database is this?
* What distributions are installed?
* What release of a distribution is installed?
* What files are in that release?
* What version is that release?
* What location was a release installed into? (core, vendor, site, custom)
* What are the checksums of those files?

And the basic operations we need to support.

* Add a release (ie. install).
* Delete a release (and its files).
* Delete an older version of a release (as part of install).
* Delete an older version of a release, only if its in the same release
  location.  This is so CPAN installs don't delete vendor installed modules.
* Verify the files of a release.
* List distributions/releases installed.

It would also store the MYMETA data which gives us a lot of information (such
as dependencies) for free.

This is all totally doable, and efficient enough, with a small pile of DBM
files and Storable.  Where to put the database is a bit more complicated, see
the list of open problems below.

There's lots and lots and lots of additional information which could be stored
and queries and operations to allow, but if we can get the basics working
it'll allow a heap of new solutions.  And I think this is a SMOP.


Future possibilities include...

* Auto-upgrade to SQLite if ExtUtils::Install::DB::SQLite is installed.

If a special module is installed we can offer SQLite support (or whatever) for
a more advanced database.  At install time it would copy the existing DBM
system into its own database.

In general, more functionality can be added as more optional (or bundled)
dependencies are available to the system.  Through it all the basic DBM
database would continue to be redundantly maintained to provide a fallback
should those optional modules break or go away.

* Extra hooks into the install system.

ExtUtils::Install is sort of a black box.  If it started to do more than just
copy files it would need a more interesting API.  Rather than trying to cram
more options into install() it would be worthwhile to write a new API.  Build
systems can check for the existence of the new API and use that if available
and do more interesting things with the database.  This would be necessary to
support uninstall.


Problems include...

* Anything installed before the new ExtUtils::Install is lost.

Just have to live with that.  It will slowly go away as the new
ExtUtils::Install gets into core, Perl is released and vendor Perls update
their Perl or core modules.  It'll take time, but we're in the long run here.

* Anything installed outside the normal blib process is lost.

Initially, this is acceptable loses.

Ideally the install process would be expanded to better deal with things which
are not Perl libraries or programs.  Build systems which have their own
methods of installing these things could add them directly using the install
database API.  A lot of hand waving here.

* Upgrading the database.

I'd like to put some thought into how things are laid out initially to avoid a
lot of major revisions, and thought into what information should be recorded
so its available later, but eventually we're going to want to change the
"schema", such as it is with DBM files.

I figure this can happen as part of upgrading ExtUtils::Install.  It checks
what version of the database you have and performs the necessary transforms to
bring it up to the current version.  We know how to do this, just have to keep
it in mind and remember to implement it.

* Where to put the database?  What about non-standard install locations?

$Config{archlib} would seem the obvious location, but it presents a
permissions problem.  If a non-root user installs into their home
directory, you don't want them needing root to write to the installation
database.  There's several ways to deal with this.

One is to simply not record non-standard install locations, but this loses
data and punishes all those local::lib users out there.

Another is to have a separate install database for non-standard install
locations.  This makes sense to me, but it brings in the sticky problem
of having to merge install databases.  Sticky, but still a SMOP.  Once you
have to implement merging anyway, it now makes sense to have an install
database for each install location.  One for core.  One for vendor.  One for
perl.  And one for each custom location.  This has a lot of advantages to
better fit how Perl layers module installs.

    * allows separation of permissions
    * allows queries of what's installed based on what's in < at >INC

That second one is important.  When a normal user queries the database, they
want to get what's installed in the standard library location.  When a
local::lib user queries the database, they want to get what's installed in the
standard library locations AND their own local lib.


In summary...

Not perfect, but gets us off the ground.  Its not a great database, but it
does the important job of recording the critical install-time data for later
use.  Its implementable within the current system.  It doesn't require a bunch
of dependencies, just one upgrade.  It works with most existing module
releases.  It solves a major design problem with the Perl module system.

I think it's a Simple(?!) Matter Of Programming in ExtUtils::Install to get it
off the ground.  IMO the most important bit of coordination is putting some
thought into what the basic database should look like so we don't have to
worry about complicated upgrades later.

Thoughts?

Uninstalling obsolete module files

Alberto Simões — 2012-12-15T17:20:47

Hello

It would be cool if a module could obsolete other module.

For instance, I noticed that Net::DNS no longer installs 
Net::DNS::Domain. This means the Domain.pm file will be hanging in the 
system until a full reinstall of the perl module tree is done.

Just like one can specify a list of PM files in the distribution, it 
would be cool if we could specify a list of modules that get obsolete 
with this installation, something like:

     obsoletes_modules => [ 'Net::DNS::Domain' ],

One could maintain this information for a few releases, and the install 
action would remove the Domain.pm file if it finds it.

Thoughts on this?

Cheers,
ambs

The get_options Argument to new()

John M. Gamble — 2012-11-17T00:53:36

I managed to get around this using the args() method, so this isn't a
critical problem. But I was wondering if I used the get_options argument
incorrectly.

What I had was:

my $test_all = 0;

my $build = Module::Build->new(
my $build = $class->new(

license => 'perl',

get_options => {
Testall => {store => \$test_all},
},
#... etc.
);

$build->create_build_script;

It creates the Build file correctly, but when I try the command

Build test --Testall

I get the error message

Modification of a read-only value attempted at
C:/Perl64/lib/Getopt/Long.pm line 561.

Is this an error in Module::Build handling or an error on my part?

I'm using Activestate perl v5.14.1 built for MSWin32-x64-multi-thread, and
version 2.38 of Getopt-Long.

Thanks,
     -john

Passing Options to Test Files

John M. Gamble — 2012-11-07T01:23:31

I have tests in a module that take a long time to run. Strictly speaking,
the CPAN testers don't need to run all of them, so it makes sense to have
them automatically skipped, letting me run them manually when I want.

Reading the Module::Build::API document, I see that I can create my own
option (e.g., --Testall) using the get_options parameter. But it looks
like I can only pass information to the test files via tap_harness_args
to TAP::Harness.  Currently, all but two of my test files use Test::Simple,
with the remaining files using Test::More.

Do I have to re-write my test files to use TAP::Harness?

Thanks,
     -john

subclassing Action_install

Boyd Duffee — 2012-10-11T16:49:55

Hi again,

Trying to learn more about custom install.  I ran across the Modifying an
Action section in Module::Build::Cookbook and tried to shoehorn that bit of
code into my script.  When I run
perl Build.PL
I get
Can't use string as a HASH ref while "strict refs" in use
when I call the create_build_script() method.  I've attached the Build.PL
file.  What am I doing wrong?

many thanks,
Boyd

I'm using M::B v0.4 / Fedora 13 / perl v5.10.1 x86_64-linux-thread-multi

This is me stepping through the build process with the debugger

Module::Build::Base::write_config(/usr/share/perl5/Module/Build/Base.pm:1318):
1318:  File::Path::mkpath($self->{properties}{config_dir});
  DB<1>
Can't use string ("Module::Build::Custom") as a HASH ref while "strict refs"
in use at /usr/share/perl5/Module/Build/Base.pm line 1318.
 at /usr/share/perl5/Module/Build/Base.pm line 1318
Module::Build::Base::write_config('Module::Build::Custom') called at
/usr/share/perl5/Module/Build/Base.pm line 1956
Module::Build::Base::create_build_script('Module::Build::Custom') called at
Build.PL line 56
Module::Build::Base::CODE(0x17d23d0)(/usr/share/perl5/Module/Build/Base.pm:1258):
1258:    for my $f ( map glob($_), < at >{ $unlink_list_for_pid{$$} || [] } ) {

added check for shell in installed script

Boyd Duffee — 2012-10-10T16:42:51

Hi there,

I found the following line added to a script that I created with
Module::Build 0.4 and used Module::Build 0.38 to install

eval 'exec /usr/bin/perl -wT -S $0 ${1+"$< at >"}'
    if 0; # not running under some shell

I wasn't expecting that and I'm not sure that I want it.  It doesn't mention
this behaviour in the docs.  This is my Build.PL script.

my $builder = Module::Build->new(
    module_name         => 'Keele::Utilities::Database',
    license             => 'perl',
    dist_author         => q{Boyd Duffee <b.duffee at isc.keele.ac.uk>},
    dist_version_from   => 'lib/Keele/Utilities/Database.pm',
    requires    => {
        'DBI'   => '1.52',
        'Keele::Utilities'  => '0.03',
        'Date::Manip'   => 0,
    },
    build_requires => {
        'Test::More' => 0,
    },
    configure_requires => {
        'Module::Build' => 0.38,
    },
    add_to_cleanup      => [ 'Keele-Utilities-Database-*' ],
    create_makefile_pl => 'traditional',
    script_files        => {
        'script/get_my_network_usage.pl'    => 0,
    }
);

if ( $builder->install_destination('script') ne '/usr/local/bin' ) {
    my $script_install = $builder->prompt('Script location', '/usr/local/bin');
    $builder->install_path('script' => $script_install);
}


thanks,

added check for shell in installed script

Boyd Duffee — 2012-10-11T07:37:01

Hi there,

I found the following line added to a script that I created with
Module::Build 0.4 and used Module::Build 0.38 to install

eval 'exec /usr/bin/perl -wT -S $0 ${1+"$< at >"}'
    if 0; # not running under some shell

I wasn't expecting that and I'm not sure that I want it.  It doesn't mention
this behaviour in the docs.  Can someone explain why?  This is my Build.PL
script.

my $builder = Module::Build->new(
    module_name         => 'Keele::Utilities::Database',
    license             => 'perl',
    dist_author         => q{Boyd Duffee <b.duffee at isc.keele.ac.uk>},
    dist_version_from   => 'lib/Keele/Utilities/Database.pm',
    requires    => {
        'DBI'   => '1.52',
        'Keele::Utilities'  => '0.03',
        'Date::Manip'   => 0,
    },
    build_requires => {
        'Test::More' => 0,
    },
    configure_requires => {
        'Module::Build' => 0.38,
    },
    add_to_cleanup      => [ 'Keele-Utilities-Database-*' ],
    create_makefile_pl => 'traditional',
    script_files        => {
        'script/get_my_network_usage.pl'    => 0,
    }
);

if ( $builder->install_destination('script') ne '/usr/local/bin' ) {
    my $script_install = $builder->prompt('Script location', '/usr/local/bin');
    $builder->install_path('script' => $script_install);
}


thanks,

MYMETA Files Not Getting Packed

John M. Gamble — 2012-09-06T21:12:26

Are MYMETA.yml and MYMETA.json not meant to be part of a perl package? I
admit to adding them to the MANIFEST without considering it. What I found
more interesting is that the MYMETA files were not included in the
package.

I'm assuming this is an out-right error -- even if the MYMETA files are
unnecessary, their presence in the MANIFEST file should at least merit a
warning, right?

Thanks,
     -john

Question about documentation targets

Chris Maloney — 2012-04-28T13:46:15

Good day, all!

This is a newbie question.  I am trying to set up a build/development
environment for my CPAN module under Windows.  I'm switching between
cygwin and Strawberry Perl, and haven't settled on one or the other
yet -- and in fact maybe I'll continue to use both, so as to provide a
little more testing.

Under strawberry, the problem that I'm having now, is that when I type
"build" in my module's development directory, it is not building any
of my module's documentation:  I get no bindoc, libdoc, binhtml, or
libhtml in blib.

According to the documentation, the 'build' action is supposed to
build 'docs'.  This is consistent, in that at least, when I type
'build docs', I also get no documentation directories.  However, when
I type 'build html', I do get the two html directories, and when I
type 'build manpages', I get the two doc directories.

I would have thought that 'build docs' would do the same as 'build
html' and 'build manpages'.  Why doesn't it?

The documentation also says that, for the html action, nothing will be
done "if there are no binhtml or libhtml installation targets defined
...".  I can't find anywhere about how to define installation targets.
 Maybe I am just confused by the terminology, but there's nothing
about how to define installation targets in the Module::Build page,
and I can't find these keywords in the Config.pm documentation,
either.  If it just means setting the installation path, then I still
have the question, why does "build html" know where to put the files,
but "build docs" does not?

How can I get "build" to build the html and man pages by default on my
strawberry system?

Note that the cygwin behavior is different.  It does build all four of
these by default.  I'm having a different problem there, though, that
I haven't really investigated yet:  the conversion to html is taking
ridiculously long.

Thanks for any help you can give me!

Ending Build.PL if there are missing modules

Alberto Simões — 2012-02-03T21:50:46

Hello

Builder->new seems to check if all required modules are available. But 
if any fails, it continues to process the Build.PL file.

Is there any way to make it stop?

Thanks

Stopping a Pointless C Compiler Warning

John M. Gamble — 2012-01-11T02:36:38

I don't know which upgrade brought this, but currently when I use Build.PL
(and the resulting Build files) I get (in reverse video no less) the
message:

It looks like you don't have a C compiler on your PATH, so you will not be
able to compile C or XS extension modules.  You can install GCC from the
MinGW package using the Perl Package Manager by running:

    ppm install MinGW

I've become somewhat numb to it, but it's a completely pointless and
unhelpful message, since this machine I'm working on will never develop
anything in C. I haven't been able to find anything in Module::Build
documentation that tells me how to tell it that this is a pure Perl
module. Can anyone steer me in the right direction?

I'm using ActiveState perl version 5.14.1, and Module::Build version 0.3800.

Thanks,
     -john

ExtUtils::MakeMaker question, before I post to a larger audience

Phil Kasten — 2011-12-14T15:25:08

I'm probably just doing something really dumb, but I cannot figure out 
what...

I am using the cpan tool to install modules on a FreeBSD system that I 
do not have root permission on.  I have a provided a PREFIX for the 
makepl_arg, to install everything in my local path.  Almost everything 
works fine.

The modules (e.g. foo.pm) and the man1 pages are correctly installed 
relative to the PREFIX path.  However, man3 pages seem to ignore PREFIX. 
  In fact, even if I try using INSTALLMAN3DIR, and whether or not I 
specific INSTALLDIRS=perl, nothing seems to cause man3 man pages to be 
re-routed.  Instead, they try to be installed in the root distribution 
and the install fails:


In other words, with the PREFIX variable set (to ~/perl), man1 man pages 
are correctly placed in ~/perl/man/man1, but man3 man pages seem to want 
to go to /usr/local/lib/perl5/5.8.8/man/man3.  And, this does not change 
even if I also specify INSTALLMAN3DIR=~/perl/man/man3, with or without 
INSTALLDIRS=perl.

Here's the line from my ~/.cpan/CPAN/MyConfig.pm:
   'makepl_arg' => q[PREFIX=~/perl],

perl is 5.8.8 and ExtUtils::MakeMaker 6.62 is installed on my system. 
And the directory ~/perl/man/man3/ exists.

Am I doing something wrong?  Should I send this question somewhere else?

Thanks!
Phil

Build.PL But No Meta.yml (or Meta.json)

John M. Gamble — 2011-10-23T00:25:35

So I have a new laptop, and I install all the latest (I think) versions of
perl (Activestate v5.24.1

New Module::Build patch/release manager

David Golden — 2011-10-21T00:05:30

I'm very, very pleased to announce that Leon Timmermans has
volunteered to be the new M::B patch/release manager and received his
co-maint bit today.

Leon has been working on abstracting the Build.PL API through his work
on Module::Build::Tiny and has gotten very familiar with the guts of
Module::Build as a result.  He is also working to extract more of the
reusable bits of Module::Build for other parts of the toolchain to
reduce duplication.

In my discussion with him about the role, he said that he is not
planning on any major work to Module::Build, but has agreed to
shepherd the future release process to ensure that his refactoring and
other incremental changes are tested and released for CPAN and blead.

Thank you very much, Leon!

Official Module::Build repository has moved

David Golden — 2011-10-20T22:57:26

To help spread the maintenance work further, I've created an
"official" M::B repo in the "Perl-Toolchain-Gang" organization on
github.com.  (Thank you to Schwern for sponsoring this organization
first with EU::MM.)

https://github.com/Perl-Toolchain-Gang/Module-Build

Please follow that repo instead of the dagolden one.  At some point in
the future, I'll be deleting my personal github repo entirely.

If you are a regular patch submitter for M::B, please email me
off-list about getting direct access.

Thanks,
David

Possible bug?

Fields, Christopher J — 2011-10-17T03:08:06

All,

I have been attempting to push a release to CPAN for a few bioperl sub-distributions and ran into an odd issue where the distribution was not being indexed in CPAN, documented here:

http://www.nntp.perl.org/group/perl.modules/2011/10/msg78021.html

This appears to be due to the version for the modules being set to '0' in META.yml/json instead of not being defined (yes, we know this is a problem with the bioperl code, but Andreas indicated that lack of a module version probably should not default to '0'.  Any reason this is occurring?

chris

Needing some help with Module::Build and the Intel compiler.

Steve Mayer — 2011-09-29T04:49:29

Hello,

  I'm attempting to perform a Perl build on Windows 2003 64-bit using the Intel compiler (version 10.1.032) and I'm seeing issues with those modules that are using Module::Build.  Basically when I attempt to build any of these Module::Build based modules, I'm seeing the following type of error:

perl Build.PL
        compilet-qQxoU.
       (0): internal error: backend signals

Has anyone seen this type of issue or has more experience with using the Intel compiler (along with VS 2005) to build Perl with?

I'm not a C programmer, so I'm flying in the dark here...

Thanks in advance,

Steve Mayer
smayer69< at >me.com

"Food is an important part of a balanced diet." -- Fran Lebowitz

'undefined value as a HASH reference' failure on installing Module-Build 0.38

Henrik Markarian — 2011-09-15T20:49:40

Hi,

I¹m having trouble installing Module-Build on a dedicated Linux Server
(Debian 4.0).  Here¹s the output that I¹m getting:


CPAN.pm: Building D/DA/DAGOLDEN/Module-Build-0.3800.tar.gz

# Converted to config
installman1dir=/homepages/33/d280429716/htdocs/bugzilla/lib/man/man1
# Converted to config
installman3dir=/homepages/33/d280429716/htdocs/bugzilla/lib/man/man3
# Converted to config
installbin=/homepages/33/d280429716/htdocs/bugzilla/lib/bin
# Converted to config
installscript=/homepages/33/d280429716/htdocs/bugzilla/lib/bin
# running Build.PL --installdirs site --config
installsitelib=/homepages/33/d280429716/htdocs/bugzilla/lib --config
installprivlib=/homepages/33/d280429716/htdocs/bugzilla/lib --config
installarchlib=/homepages/33/d280429716/htdocs/bugzilla/lib/i486-linux-gnu-t
hread-multi --config
installsitearch=/homepages/33/d280429716/htdocs/bugzilla/lib/i486-linux-gnu-
thread-multi --config
installman1dir=/homepages/33/d280429716/htdocs/bugzilla/lib/man/man1
--config 
installman3dir=/homepages/33/d280429716/htdocs/bugzilla/lib/man/man3
--config installbin=/homepages/33/d280429716/htdocs/bugzilla/lib/bin
--config installscript=/homepages/33/d280429716/htdocs/bugzilla/lib/bin
--installdirs core
Created MYMETA.yml and MYMETA.json
Creating new 'Build' script for 'Module-Build' version '0.3800'
CPAN: Module::CoreList loaded ok (v2.55)
/usr/bin/perl Build --makefile_env_macros 1
Building Module-Build
Can't use an undefined value as a HASH reference at lib/Module/Build/Base.pm
line 5163.
make: *** [all] Error 9



I¹ve looked for hours online and haven¹t been able to find a solution.  Can
someone help?

Thanks,


Henrik Markarian

Docs for meta_add, meta_merge

Ron Savage — 2011-05-12T00:10:37

Hi Folks

I couldn't find any docs for meta_add etc in Module::Build. I did find
them by googling. Can't say exactly where they should go, but I'd like
to see at least a reference to them in there somewhere.

Tests acting differently inside or outside CPAN

Alberto Simões — 2011-04-25T12:29:11

Hello

I have a module (Text::BibTeX) that is cleanly installing when using 
cpan, but fails two tests when I try to install manually, using 
Strawberry Perl.

The tests are using a C library, getting the results through XS, and 
converting them to UTF8. From three tests that use this behavior, only 
two have that problem :-S

Is there any kind of variables or environment set by CPAN that can be 
causing this behavior?

Thank you for any hind (and yes, I know this description is too vague, 
but I am not able to explain what is going on in a btter way).

Thanks
ambs