gmane.linux.redhat.fedora.buildsystem

mockchain branch in mock

seth vidal — 2012-05-23T18:04:03

I added a mockchain to mock's git repo today which includes
modifications to include mockchain in mock - include specfile, makefile
and man pages.

Clark, let me know if there is anything else you need to merge this in.

Thanks,
-sv
--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: pungi --discs option

Amit Saha — 2012-05-22T05:02:17

Just so it helps, please find the modified manual page sources
(doc/pungi.8) with the --discs switch removed.

Best,
Amit

Re: pungi --discs option

Bill Nottingham — 2012-05-21T16:59:42

Amit Saha (amitksaha< at >fedoraproject.org) said: 

I do believe split media support was removed.

Bill
--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: pungi --discs option

Jesse Keating — 2012-05-21T16:44:20

Man page is likely dated, I don't think it's autogenerated.  Pungi 
dropped support for split media a number of Fedora releases ago, when 
anaconda did the same.

pungi --discs option

Amit Saha — 2012-05-20T10:20:44

Hello:

I just signed up for this list.

The pungi manual page seemes to suggest a --discs option, whereas if
specified, as a command line argument, says: "pungi: error: no such
option: --discs".

Perhaps the manual page is dated?

Best,
Amit

Re: Unable to finish the creation of source ISO on f17

William F. Acker WB2FLW +1 303 722 7209 — 2012-05-11T00:59:41


Makes sense. We're never going to boot the SRC disk.

           Thanks as always.

Re: Unable to finish the creation of source ISO on f17

Jesse Keating — 2012-05-10T22:06:19

Uh, isohybrid really shouldn't be being ran on source isos.  Sounds like 
some code got added that isn't careful about that.

koji wsgi support

Mike McLean — 2012-05-10T21:59:03

I just pushed a large commit adding wsgi support to koji-hub and 
koji-web. This feature had been on the back burner for a while, but the 
recent retirement of mod_python in Fedora has forced the issue.

Here's the summary I wrote for the (squashed) commit:
Support wsgi in koji-hub and koji-web
  - mod_python still supported, but deprecated
  - mod_wsgi is the default
  - koji-web now configured via web.conf
  - new wsgi-friendly publisher for koji-web
  - koji-web now has logging

This is a fairly large change. I've gone out of my way to maintain 
compatibility with mod_python, but migration issues are still possible.

Ideally, though, folks will move to mod_wsgi. This will mainly be a 
matter of adjusting httpd.conf files and moving koji-web config from 
http conf to the new web.conf file. I'll write a longer doc on this 
later, but for now:
- see the comments in the example http.conf files for hub and web
- see the example web.conf
- generally the web.conf options have the same name as the old koji-web 
PythonOptions

I'm not quite ready to cut a release with this yet, but soon. There 
needs to be more testing.

If you opt to try out the new code, please report bugs in our trac:
https://fedorahosted.org/koji/report
--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Unable to finish the creation of source ISO on f17

William F. Acker WB2FLW +1 303 722 7209 — 2012-05-02T14:03:06

Hi all,

      When I try to do a Pungi run, my first for f17, I get this.
Traceback (most recent call last):
   File "/bin/pungi", line 256, in <module>
     main()
   File "/bin/pungi", line 160, in main
     mypungi.doCreateIsos()
   File "/usr/lib/python2.7/site-packages/pypungi/__init__.py", line 1148, in doCreateIsos
     pypungi.util._doRunCommand(isohybrid, self.logger)
   File "/usr/lib/python2.7/site-packages/pypungi/util.py", line 36, in _doRunCommand
     raise OSError, "Got an error from %s: %s" % (command[0], err)
OSError: Got an error from /usr/bin/isohybrid: isohybrid: /home/wacker/bm/17/source/iso/Fedora-17-source-DVD.iso: could not find boot record


My command line is:
sudo pungi -c /home/wacker/.pungi/fedora-install-fedora.ks --ver=17 --sourceisos

      I actually do get the ISO, but, of course, the checksum file isn't 
created.

      I also should mention that I get a lot of unresolvable dependencies 
errors, some would go away if I stopped excluding javadoc.

          Any ideas?


           Tia.

Re: SCM Root

Mike McLean — 2012-04-30T16:12:03

A 'make sources' target that cares about the name of the checkout 
directory is arguably broken.
--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

SCM Root

Moray Henderson — 2012-04-26T09:43:12

Ah! SCM checkout and "make sources" are run in /tmp/scmroot, not
/builddir/build/SOURCES.  That might help people writing a custom "make
sources".


Moray.
"To err is human; to purr, feline."






--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

RE: Dist tag question

Moray Henderson — 2012-04-25T15:00:36

(There was a typo in the above: slspkg should read mypackage.  Sorry.)

Never mind - got it.  Yum needs the correct package name, it doesn't match
on what the package provides.

Still getting to grips with the Koji and Koji hub interfaces.  I found the
root.log of the task and that gave me the clue:

  DEBUG util.py:257:  Installing:
  DEBUG util.py:257:   bash                   x86_64  4.1.2-8.el6.centos
build  902 k
  DEBUG util.py:257:   curl                   x86_64  7.19.7-26.el6_2.4
build  192 k
  DEBUG util.py:257:   make                   x86_64  1:3.81-19.el6
build  389 k
  DEBUG util.py:257:   redhat-rpm-config      noarch  9.0.3-34.el6
build   57 k
  DEBUG util.py:257:   rpm-build              x86_64  4.8.0-19.el6_2.1
build  124 k
  DEBUG util.py:257:   shadow-utils           x86_64  2:4.1.4.2-13.el6
build  896 k

Some packages from srpm-build were missing from the install, because they
are in the repository under different names.  They got pulled in to the
buildArch buildroot as dependencies.  


Moray.
"To err is human; to purr, feline."







--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Dist tag question

Moray Henderson — 2012-04-25T14:26:40

Hi again,

I've got Koji to build rpms - but the task is still failing with the message

    GenericError: Unable to complete build: release mismatch (build: 2, rpm:
2.el6)

The different build steps are giving different results.  My .spec file
contains

  Release: %{release}%{?dist}

and my build and srpm-build groups both contain redhat-release, which
resolves to centos-release-6-2.el6.centos.7.x86_64 and contains
/etc/rpm/macros.dist to set "%dist .el6".

However buildSRPMFromSCM results in 

  Building target platforms: x86_64
  Building for target x86_64
  Wrote: /builddir/build/SRPMS/mypackage-1.0-2.src.rpm

while buildArch also builds a srpm before the binary rpm:

  Building target platforms: noarch
  Building for target noarch
  Wrote: /builddir/build/SRPMS/mypackage-1.0-2.el6.src.rpm

  Building target platforms: noarch
  Building for target noarch
  ...
  Wrote: /builddir/build/RPMS/slspkg-1.0-2.el6.noarch.rpm

My Koji hub lists the build as "mypackage-1.0-2".  If I get buildSRPMFromSCM
to use the dist macro, will the Koji hub pick up the right name?  How do I
get buildSRPMFromSCM to do that?


Moray.
"To err is human; to purr, feline."





--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

RE: How does...?

Moray Henderson (ICT — 2012-04-25T10:57:19

Thanks everyone - your replies have been very helpful.


Moray.
"To err is human; to purr, feline."


  OM International Limited - Unit B Clifford Court, Cooper Way - Carlisle CA3 0JG - United Kingdom
  Charity reg no: 1112655 - Company reg no: 5649412 (England and Wales)

--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: How does...?

Mike McLean — 2012-04-23T18:01:09

The allowed_scms option in kojid specifies which scms can be used and 
how kojid should use them. The format of the option is:

"""
a space-separated list of host:repository[:use_common[:source_cmd]] 
tuples.  Incorrectly-formatted tuples will be ignored.

If use_common is not present, kojid will attempt to checkout a common/ 
directory from the repository.  If use_common is set to no, off, false, 
or 0, it will not attempt to checkout a common/ directory.

source_cmd is a shell command (args separated with commas instead of 
spaces) to run before building the srpm. It is generally used to 
retrieve source files from a remote location.  If no source_cmd is 
specified, "make sources" is run by default.
"""

So...
- if you were using an old dist-cvs setup, you'd set use_common to yes 
and leave source_cmd as the default.
- with a dist-git setup, you'd set use_common to no and source_cmd to 
fedpkg,sources
- for simpler setups, you might simply set use_common to no, leave 
source_cmd as the default, and ensure that each package to be built 
includes a Makefile with a sources target that does the right thing.

Which of these options is best depends greatly on your situation.
--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: koji: small patch to fix warnings

Mike McLean — 2012-04-23T17:48:34

Our documentation is somewhat lacking unfortunately. I'm told the source 
is fairly readable, though as one of the authors I am not fit to judge :)

Of course, koji is a command line tool and it is quite legitimate to 
process its output in some ways. Your list-tagged example is fine, 
though you probably want the --quiet option to avoid printing the column 
headers.

The boundary where best practice shifts from standard shell output 
processing to writing a python script is somewhat fuzzy. It depends on 
what you're trying to accomplish.


Yeah, that's the one. Stray whitespace is disliked by many developers, 
myself included. The git-apply and git-am tools complain when they find it.

While not applicable to this case, leading whitespace can be an major 
issue in python, particularly when it comes to mixing spaces and tabs 
(we don't use tabs in Koji's code, though a few have slipped in over the 
years).

I use vim options to make visibly distinguish certain types of 
whitespace (e.g. "set list lcs=tab:>- lcs+=trail:-"). It's quite 
helpful. I think other development-friendly editors have similar features.
--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: How does...?

Adam Jackson — 2012-04-23T14:11:55


I wouldn't assume that the baseline configuration of the koji package in
Fedora necessarily matches how Fedora's koji instance is deployed.


Well, the way you do this as a fedpkg user is just run "fedpkg sources",
which grabs tarballs from the lookaside server based on the names and
md5sums in the 'sources' file in the checked-out package.  Which is how
"make sources" used to work, too.

From a quick look at the kojid source I'd assume you'd want to override
the 'source_cmd' attribute of the SCM to do whatever's appropriate.

- ajax
--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: koji: small patch to fix warnings

John Morris — 2012-04-22T08:14:26

Hi Mike,


Yes, I do need something better.  This raises a question I've had about 
workflow.  The koji CLI is very simple.  I think that most commands 
expose a single xmlrpc call (perhaps with a little sugaring).  I haven't 
seen a document yet that describes workflow beyond the most basic steps. 
  I often find myself doing cumbersome things, and think there must be 
an easier way.  This example could be called 'list-tagged --rpms --nosigs':

koji list-tagged --rpms --sigs fc16-cadcam | \
   sed '/^ /s/^/2ef86730/' | sort | uniq -u | sed 's/^[^ ]* //'

Is there some documentation I've missed somewhere?  There are several 
pages on the wiki, but nothing I've found is very complete (other than 
the source!).  Should I just not be using the CLI much and scripting my 
workflow instead?


I see that I left a space at the end of the middle 'warn(' line.  Is 
that what you mean?  I'm not a real dev, so this kind of clue is helpful.

John

--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: How does...?

Clint Savage — 2012-04-20T20:30:30

Hello Moray,

As Andreas pointed out there are a lot of pieces involved in getting
koji to build from git or hg or other revision control systems. I'll
try to explain the process from my POV as I've now done it a few
times. The project I work on is called GoOSe Linux, it's a similar
project to what CentOS is doing. It's possible I'm going into too much
information, but I figured it could not hurt to give you the pieces,
included the makefile and makefile.common parts.

Essentially, we have all of our rpms separated into two basic
components. First, we have a git repository for all of the rpms we
wish to build. They can be found at
http://github.com/gooselinux/<repo>. These consist of the spec, patch
and Makefiles along with a file called sources. This is exactly
similar to the method Andreas described in the previous mail. The
other part is the lookaside cache which holds the actual upstream
source code, usually in a tarball or zip file. These are all of the
pieces you need to make koji work properly.

Assuming you have your koji system setup and the package tagged, you
just have to tell koji to build your package. This can be done simply
by running 'koji build gl6-alpha
git://github.com/gooselinux/anaconda.git#HEAD' (your koji build line
will vary). As other discussions have probably explained, there's a
lot of magic that happens here, so I'll go through it step by step.

What happens next is that koji sets up a mock build environment to
build the source rpm. Once the mock environment is initialized, make
sources is called. Koji then takes the url you provided and checks it
out into the mock build environment. Then, the lookaside cache is
downloaded. You should have a look at our Makefile.common at
https://github.com/gooselinux/common. There is also a Makefile for
each rpm we have koji build, using anaconda as an example, you can
find the Makefile at https://github.com/gooselinux/anaconda. The
Makefile downloads the tarball from the lookaside cache and verifies
the sha256sum from the 'sources' file in the anaconda git repository
previously mentioned. If all of that completes successfully, then mock
builds the source rpm.

After building the source rpm, koji then creates the mock
environment(s) to build the binary rpm(s). The rpm(s) is/are built
from the source rpm (SRPM) built in the previous explanation.

A couple things you mentioned about kojid.conf and the allowed_scms
variable. I've attached a copy of my kojid.conf from one of the
builders to help you along your way. Additionally, Andreas'
Makefile.common is pretty long and does a lot of things, it's likely
got some very nice checking and additional things that the goose
Makefile.common doesn't. Ours is rather simple and just does the bare
minimum to make things work.

I sure hope this explanation and files has helped you on your way to
using Koji to build rpms!

Cheers,

Clint

On Fri, Apr 20, 2012 at 1:47 PM, Andreas Mack <andreas.mack< at >gmail.com> wrote:
--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: How does...?

Andreas Mack — 2012-04-20T19:47:52

Here's the Makefile.common that I use.

Andreas



--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys

Re: How does...?

Andreas Mack — 2012-04-20T19:45:08

I still use the old "make sources" way. Pretty much copied from the Fedora
set up, with the "lookaside" repository for the files.

Here's how I understood it, please correct me if I'm wrong:
Koji checks out the package subdir and executes "make sources" there. The
Makefile within the subdir of the package in svn references a
"Makefile.common" that lives in "trunk/common". The Makefile retrieves the
"Makefile.common" from svn and loads it. Makefile.common then retrieves the
sources file and the sources.
To download them it uses the REPOSITORY variable, i.e. the svn host. It
fetches the sources like this:
http://REPOSTITORY_HOST/repo/pkgs/<packagename>/<source file name>/<md5sum
of source file>/<source file name>. If the md5sum matches, "make sources"
returns and koji proceeds with building.

As Jesse wrote back then, if you don't want to do it that way, you're free
to specify any other command that will download the sources. For me it was
a good start without getting too much into fedpkg stuff.

Andreas.

On Fri, Apr 20, 2012 at 18:34, Moray Henderson <
Moray.Henderson< at >ict-software.org> wrote:

--
buildsys mailing list
buildsys< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys