gmane.linux.ubuntu.devel.distributed

DEB_VENDOR & package imports

Dmitrijs Ledkovs — 2012-05-03T13:49:19

Hello,

I am having fun times with package lp:debian/accountsservice

$ bzr cat lp:debian/accountsservice/.pc/applied-patches
Most recent Debian version: MISSING
0001-formats-locale-property.patch

0002-create-and-manage-groups-like-on-a-ubuntu-system.patch
0005-gdm_config_file_path_ubuntu.patch
0006-adduser_instead_of_useradd.patch
0007-add-lightdm-support.patch
0008-nopasswdlogin-group.patch
0009-language-tools.patch
0010-set-language.patch
0011-add-background-file-support.patch
0012-add-keyboard-layout-support.patch
0013-add-has-message-support.patch
1001-buildsystem.patch
2001-icon_reset.patch
3001-show_more_than_one_user_powerpc.patch

$ bzr cat lp:debian/accountsservice/debian/patches/series
Most recent Debian version: MISSING
0002-create-and-manage-groups-like-on-a-debian-system.patch

0005-gdm_config_file_path.patch
0006-adduser_instead_of_useradd.patch
0007-add-lightdm-support.patch
1001-buildsystem.patch
2001-icon_reset.patch
3001-show_more_than_one_user_powerpc.patch

It has both ./debian/patches/

Switching over to Quantal

Barry Warsaw — 2012-04-30T15:33:16

Now that Quantal is open for development, what do we need to do to get things
switched over, and UDD all happy-like?

$ bzr branch ubuntu:precise/debootstrap precise
bzr: ERROR: Revision {package-import-GeWIH/nMZzJKr9sWaVLiSSMbcUnWK/KXJQVSFeiJYKfRRQAXIyLz2A< at >public.gmane.orgd7p4} not present in "Graph(StackedParentsProvider(bzrlib.repository._LazyListJoin(([CachingParentsProvider(None)], []))))".
$ bzr branch ubuntu:debootstrap precise
Most recent Ubuntu version: 1.0.40
Packaging branch version: 1.0.38
Packaging branch status: OUT-OF-DATE

-Barry

Importer add-import-jobs cronjob temporarily disabled

Max Bowsher — 2012-04-28T09:52:44

See https://bugs.launchpad.net/udd/+bug/990394

jubany: Importer stopped

Max Bowsher — 2012-04-26T21:38:23

I've stopped the importer. It needs to be taught about quantal before it can proceed:

Traceback (most recent call last):
  File "/srv/package-import.canonical.com/new/scripts/bin/import-package", line 7, in <module>
    main()
  File "/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py", line 1177, in main
    only_before=options.only_before))
  File "/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py", line 1045, in _import_package
    versions = get_versions(lp, package, extra_debian=extra_debian)
  File "/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py", line 257, in get_versions
    vlist.sort()
  File "/srv/package-import.canonical.com/new/scripts/udd/icommon.py", line 265, in sort
    self.plist.sort(cmp=PackageToImport.import_order_comparator)
  File "/srv/package-import.canonical.com/new/scripts/udd/icommon.py", line 240, in import_order_comparator
    b_release = import_sequence_distro_releases[b.distro].index(b.release)
Va

Importer status: back up to date, running at 8 threads

Max Bowsher — 2012-04-22T19:53:20

The importer has now caught up from the massive backlog caused by
spurious fails of just about everything.

I've reduced the concurrency to 8, to reduce the level of API requests
being directed at Launchpad by the importer's fallback re-processing of
packages that it performs whilst idle.

Max.

Stormified importer has introduced unintended change in data beingwritten to sqlite

Max Bowsher — 2012-04-21T17:44:35

The stormification has introduced an unintended change in how the data
for testaments and revision ids are stored in the sqlite DBs.

They are now being stored as binary blobs rather than strings.

The importer is currently stopped whilst I investigate further.

Max.

Importer broken again

Max Bowsher — 2012-04-21T15:03:32

I just checked up on the importer status, and found it both locked up on
DB access again, and apparently it had been running with an improper
environment setup causing it to fail to import bzr-builddeb, so it had
put ~22k packages into failure state.

I've restarted it, properly, and requeued the failures, but that means
there is now a HUGE queue.

Ideally we'd increase the concurrency to churn through the queue
quickly... but we can't because of the db bug.

I REALLY think we should be reverting the introduction of storm.

Frankly, it's not fair that another effort wanting to re-use the UDD
code should impact the operational reliability of the UDD importer to
the degree the storm introduction already has.

By all means let's land infrastructural improvements - so long as they
are reasonably safe. Now that this one has been demonstrated not so,
let's push it out of trunk until it can be made safe.

Max.

How to ??

สุทธิพงษ์ ชัยสงคราม — 2012-04-20T05:25:35

Hi, my name is Sutthipong C. from Thailand. I have the problems for build Ubuntu source code (SRC). 

Now! I have Ubuntu 11.10 (source code) and download completely from this link (http://cdimage.ubuntu.com/releases/11.10/release/source/). 

How to build from source code, Help me please.

jubany package importer stopped?

Stephanie Miller — 2012-04-16T05:40:17

Hi folks,

Our monitoring system is currently showing an alert for jubany's package
importer process, saying that no python processes were found. Restarting
mass-import doesn't seem to resolve the alert. Additionally
/srv/package-import.canonical.com/new/logs/debug_log has lots of
messages about the database being locked.

Is this the current expected state? If not, what action should I take,
if any?

Thanks,
Stephane (Canonical IS)

UDD importer making a nuisance of itself with v3 source formatbranches

Max Bowsher — 2012-04-13T00:13:22

I've just had a conversation with cjwatson and slangasek on
#ubuntu-release about the importer making a nuisance of itself by
declaring a perfectly reasonable commit to be a collision / difference,
and replacing it with one of its own.

The key pain point here is the .pc/ directory.

It's practically impossible to maintain a .pc/ directory checked into
VCS without unnatural jumping through hoops. As a result, packages being
seriously developed in bzr by humans, rather than being primarily
imported, tend NOT to have a .pc/ :

<cjwatson> patches applied doesn't have to imply .pc in vcs; it's
unfortunate that the importer took that particular decision
<cjwatson> (I've been using patches-applied-in-bzr since well before the
importer did, *without* .pc)

I think, as a short term fix, we should modify the collision-is-clean
check to ignore the absence of a .pc directory in packager-committed
revisions.

Please have a think and let me know about any objections - we should
probably treat this as urgent since it caus

A huge waste of jubany diskspace...

Max Bowsher — 2012-04-10T00:11:07

pkg_import< at >jubany:/srv/package-import.canonical.com/new/tmp$ du -hsc  .
174G.

Yikes!

Anyone see any reason *NOT* to rm -rf ?

Max.

Importer stopped since Sunday?

Max Bowsher — 2012-04-10T00:08:25

It looks like someone stopped the UDD importer on Sunday?

James W. has a 'crontab -e' editor open from around that time.

Anyone know what's going on, and if it's safe to restart?

We're failing rather dismally at providing prompt imports at the moment.

Max.

bzr merge-upstream: why delete and add the same unchanged file?

Andreas Hasenack — 2012-03-20T19:20:29

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I'm experimenting with bzr merge-upstream and found what I think is an
odd behavior. I'm probably just using it incorrectly, but I can't find
out what I'm doing wrong.

I'm merging lp:landscape-client into ubuntu:landscape-client. Let's
say I'm preparing a new release.

This is the command-line:
$ bzr merge-upstream ~/canonical/source/landscape-client/trunk/
- --revision 531 --version 12.04.1

(~/canonical/source/landscape-client/trunk has lp:landscape-client)

https://pastebin.canonical.com/62652/ is the output

There are some conflicts, but I'm not worried about those for now
(unless they explain what I'm seeing).

This is my question:

$ bzr st LICENSE
removed:
  LICENSE
added:
  LICENSE

This happened to *all* files. There is not a single "modified" file in
the bzr status output.

Why is it removing and adding the same file? This file (and several
others) didn't change between ubuntu:landscape-client and
lp:landscape-client, it's exactly the same.

- --

What to do when a packaging branch is out of date

Andreas Hasenack — 2012-03-20T13:56:27

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

So apparently a package update can be prepared using udd, released
into the archive, and the packaging branch still be out of date:

"""
$ bzr branch
ubuntu:landscape-client landscape-client-12.04-0ubuntu1
Most recent Ubuntu version: 12.04-0ubuntu1


Packaging branch version: 11.07.1.1-0ubuntu2
Packaging branch status: OUT-OF-DATE
Branched 40 revisions.
$
"""

It's like it was released before being committed.

What should I do if I want to prepare another fix? Wait? Ping someone
to commit and then branch again? Do it with debdiffs and not worry
about branches anymore?

I emailed ubuntu-devel-discuss and someone told me that somebody has
to manually commit to the udd branch, and that this is normal.

- -- 
Andreas Hasenack
andreas-Z7WLFzj8eWMS+FvcfC7Uqw< at >public.gmane.org

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9ojIsACgkQeEJZs/PdwpDzAgCgpvRePROgSwZjqwS5uWiuYCzN

problem with recipe build

Scott Moser — 2012-03-02T22:15:24

I'm sure I'm doing something wrong, but not sure what.
I'm trying to just set up a launchpad build of two branches, one 'trunk'
that launchpad is pulling from an svn repo, and a packaging only branch.

I'm not quite sure why there would be expected to be a tag
'upstream-0.9.4+r4177' as that string is something that was created due to
this build recipe.  But the working-dir has in it a directory named
'madwifi-0.9.4+r4177' and debian/changelog in that says:
| madwifi (0.9.4+r4177-0ubuntu0+7) precise; urgency=low
|
|   * Auto build.
|
|  -- Scott Moser <smoser-GeWIH/nMZzLQT0dZR+AlfA< at >public.gmane.org>  Fri, 02 Mar 2012 16:58:43 -0500

Which seems sane to me.  The upstream source doesn't have tags named
'upstream-<version>', but only 'release-<version>'.

Help?


$ dpkg-query --show bzr-builder
bzr-builder 0.7.2-0ubuntu1

$ cat madwifi-daily.recipe
# bzr-builder format 0.4 deb-version 0.9.4+r{svn-revno}-0ubuntu0+{revno:packaging}
#/home/smoser/src/madwifi/trunk.dist
lp:~smoser/madwifi/trunk
#merge packaging /hom

UDD breakdown when building orig.tar.gz from upstream VCS

Barry Warsaw — 2012-02-15T13:21:37

I just want to put this out there for the historical record.  I think this is
a rare enough use case that UDD doesn't need to address, certainly not any
time soon, if ever.  OTOH, maybe there's an easy workaround.

I was working on an NBS for the fgfs-atlas package (LP: #903225).  The
solution was straightforward enough: upstream had all the necessary fixes in
their CVS repository, but hadn't done a release in a long time.  I twiddled
the packaging to build an orig.tar.gz from CVS, and the googlez helped find
some good general packaging information on how to do this.

Unfortunately, UDD is essentially useless here.  The problem is that after
creating the tarball from CVS, `bzr bd -S` can't be used because dpkg-source
will complain too much about deltas between the tarball and the source tree.
It'll warn about a lot of stuff, but then fail with some unrepresentable
changes to source.

I worked around this by unpacking the new orig.tar.gz and `cp -a` the debian/
directory from the precise version of the packag

xz tar imports failing?

Martin Pool — 2012-02-01T04:00:59

I haven't dug into whether these are new, or whether there is already
a bug for them, but they do seem to be new.  I see
<https://bugs.launchpad.net/udd/+bug/553668> about supporting xz in
bzr-builddeb: does that perhaps need to be deployed on Jubany?


---------- Forwarded message ----------
From: Cron Daemon <root< at >jubany.canonical.com>
Date: 1 February 2012 14:30
Subject: Cron <pkg_import< at >jubany> /usr/bin/python ${SCRIPTS_DIR}/email-failures
To: pkg_import< at >jubany.canonical.com




evince failed at 2012-01-30 13:42:01.725202 due to:
Traceback (most recent call last):
 File "/srv/package-import.canonical.com/new/scripts/bin/import-package",
line 7, in <module>
   main()
 File "/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py",
line 1177, in main
   only_before=options.only_before))
 File "/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py",
line 1078, in _import_package
   bstore, push, possible_transports=possible_transports)
 File "/srv/package-

Updates to the UPG for UDD

Barry Warsaw — 2012-01-31T23:18:08

Hi all,

I have a merge proposal to update the UDD documentation part of the packaging
guide.  I'm hoping this can be reviewed, merged, and published before my UDW
session tomorrow (it's okay if it can't).

The branch fixes some typos and formatting issues, but there are really two
substantive changes.

First, I've updated all the discussion of using `bzr merge-package` to just
use `bzr merge` as is now the case in Precise.  I do give minimum bzr and
bzr-builddeb version numbers for this, but don't dwell on the old way too
much.

Second, I've removed all the discussion of using looms to generate quilt
patches.  In going through the described procedures, I realized they just
don't work very well, and it makes things much more complicated.
Specifically, the problem comes down to when you are in the thread that
contains both the in-source change and the quilt patch, there is no way to
`quilt push` the patch, because the source tree already has the changes.

This is a problem in the non-loom approach (which I de

Improved quilt patch handling

Jelmer Vernooij — 2012-01-19T00:30:46

bzr-builddeb 2.8.1 has just landed on Debian Sid and Ubuntu Precise. 
This version contains some of my improvements from late last year for 
the handling of quilt patches in packaging branches. Most of these 
improvements depend on bzr 2.5 beta 5, which is also in Sid/Precise.

The most relevant changes (enabled by default) are:

  * 'bzr merge-package' is now integrated into 'bzr merge' (it's just a 
hook that fires on merges involving packages)
  * patches are automatically unapplied in relevant trees before merges
  * before a commit, bzr will warn if you have some applied and some 
unapplied quilt patches

Furthermore, you can now specify whether you would like bzr to 
automatically apply all patches for stored data and whether you would 
like to automatically have them applied in your working tree by setting 
'quilt-tree-policy' and 'quilt-commit-policy' to either 'applied' or 
'unapplied'.  This means that you can have the patches unapplied in the 
repository, but automatically have them applied upon c

Moving udd to django

James Westby — 2011-12-10T02:10:48

Hi,

I think there are a few reasons that we should consider moving udd to
django (more on what this actually means later.)

  1. We want to have remote commands so that we don't have to ssh in and
  run scripts for common tasks

  2. There are scripts in cron that generate html pages that are
  served. It would be nice to have these generated on request so that
  there is no latency.

  3. It would also allow for starting to move udd to an SOA, or at least
  make it easier.

  4. It would be nice to have a query builder, rather than all the
  hand-written sql.

So, what exactly do I mean by moving to django?

  * Moving all database access to go through django's ORM.

  * Moving all HTML generation to be done in django views.

This has fairly wide-ranging consequences though, and obviously needs to
be broken down in to manageable steps.

Here's my idea for what those steps should be.

1) Install python-django in production
   - So that it is there when it is needed.

2) Create a django project in the udd co

Using lp:udd beyond its original purpose

Jonathan Lange — 2011-12-02T14:18:19

Hello,

We're in the process of deploying lp:udd to do something that has very
little to do with Ubuntu distributed development. We're using it to
maintain a local database mapping from object files to symbols, for
our own nefarious purposes[1]. It works well for this, but there are
some changes that we will want to make soon.

There are two big changes that aren't necessary for us, but would help a lot:
  1. Decouple lp:udd from the package-import.canonical.com production deployment
  2. Split the package scanning and the branch importing parts of
lp:udd into separate projects

It sounds scary when I write it like that, but I don't think it's
actually that much code.

Neither of these things has to be done – we'll figure something out –
but I reckon these changes will result in less code that's better
tested.

What say you?

jml

[1] https://wiki.ubuntu.com/AutomagicBinaryPackaging