gmane.linux.redhat.fedora.infrastructure

Re: Yet another freeze break: reviewstatus

Stephen John Smoogen — 2012-05-24T23:11:11

+1

Re: Questions regarding Koji, fedpkg and SCM

Jeremy Davis — 2012-05-24T22:21:20

Thank you all for your assistance. Both of these items is what I was
looking for. Thank you for your time and have a great day!

On Thu, May 24, 2012 at 4:06 PM, Kevin Fenzi <kevin-+bl/7iUgRMUAvxtiuMwx3w< at >public.gmane.org> wrote:

_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: Questions regarding Koji, fedpkg and SCM

Kevin Fenzi — 2012-05-24T22:06:00

On Wed, 23 May 2012 17:05:40 -0600
Jeremy Davis <jdavis4102-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:


As noted, the buildsys list might be better. ;) 

Yes, we use bugzilla to review packages, then we have a script that
takes the approved packages list and processes the new package requests
from it. It's available at: 

http://git.fedorahosted.org/git/?p=fedora-infrastructure.git;a=blob;f=scripts/process-git-requests/process-git-requests;h=37ebbc9c6b1fec0338d082e077bef8856fc1dd21;hb=29e87a22fac82df08e7b31563be5b4bca01f821d

It calls our package database and then goes to the packages machine and
creates the git repo with the needed branches. 

Hope that helps some. 

kevin
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: Yet another freeze break: reviewstatus

Toshio Kuratomi — 2012-05-24T22:02:42

+1

-Toshio
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Yet another freeze break: reviewstatus

Kevin Fenzi — 2012-05-24T21:45:02

I'd like to revert puppet commits: 

55103ec6ce3a80234391229f6b10e220fa13e72d
5b131d0e489f5349d5cef6a8e0ffdc2febda0441

This reverts review-stats.py to the version that we had before monday,
which was working with the old python-bugzilla (which we have already
reverted to). 

I've tested it locally and it does only make 6 bugzilla calls when used
with the python-bugzilla version we have reverted to, so we shouldn't
be causing undue load on bugzilla. 

I'd also like to: 

diff --git a/modules/review-stats/manifests/init.pp b/modules/review-stats/manifests/init.pp
index 9a016a5..0376a77 100644
--- a/modules/review-stats/manifests/init.pp
+++ b/modules/review-stats/manifests/init.pp
< at >< at > -57,7 +57,7 < at >< at > class review-stats::build inherits review-stats {
         command => "/usr/bin/python /usr/local/bin/pyGetReviewByFlags.py -t /usr/local/share/review-stats -d 
         user    => "apache",
         minute  => [ 0, 30 ],
-        ensure => "absent",
+        ensure => "present",
     }
 
 }

to re-enable it. 

+1s? 

kevin
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: Second [Change Request] Fix pkgdb (un)privileged operations forold python-bugzilla

Ricky Elrod — 2012-05-24T21:18:02


+1

_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: Second [Change Request] Fix pkgdb (un)privileged operations forold python-bugzilla

Kevin Fenzi — 2012-05-24T21:15:00

On Thu, 24 May 2012 14:11:41 -0700
Toshio Kuratomi <a.badger-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:


Here's your +1 mallet of bugzilla wack a moling. :) 

kevin
--

_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Second [Change Request] Fix pkgdb (un)privileged operations for oldpython-bugzilla

Toshio Kuratomi — 2012-05-24T21:11:41

The change below broke bugz.fedoraproject.org.

Backing out the change to the unprivileged user but leaving the privileged
user change in appears to fix things (although we haven't tested everything
as stg doesn't talk to bugzilla as much as production does.)  Since
bugz.fedoraproject.org is fairly visible, I'd like to apply the following
hotfix to fix that.  I'll alert the cvsadmin to let me know if this
re-breaks processing of scmadmin requests.

diff --git a/modules/hotfix/files/pkgdb/utils.py b/modules/hotfix/files/pkgdb/utils.py
index 8c121c5..ef0001b 100644
--- a/modules/hotfix/files/pkgdb/utils.py
+++ b/modules/hotfix/files/pkgdb/utils.py
< at >< at > -51,7 +51,7 < at >< at > from cherrypy import request
 from sqlalchemy import select, and_
 from sqlalchemy.exceptions import DataError
 
-from bugzilla import RHBugzilla3
+from bugzilla import RHBugzilla3, Bugzilla
 import memcache
 
 # The Fedora Account System Module
< at >< at > -218,7 +218,7 < at >< at > def get_unprivileged_bz():
     bz_server = config.get('bugzilla.queryurl', config.get('bugzilla.url',
         'https://bugzilla.redhat.com'))
     bz_url = bz_server + '/xmlrpc.cgi'
-    _unpriv_bugzilla = RHBugzilla3(url=bz_url, cookiefile=None)
+    _unpriv_bugzilla = Bugzilla(url=bz_url, cookiefile=None)
     return _unpriv_bugzilla
 
 def get_bz():

-Toshio




_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: [PATCH] - add db-fas01 to dns - add to puppet - based in largerpart off of db01's recipe

Ricky Elrod — 2012-05-24T19:37:13

+1


_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: [PATCH] - add db-fas01 to dns - add to puppet - based in largerpart off of db01's recipe

Kevin Fenzi — 2012-05-24T19:35:49

+1. 

Note that before we switch to this, we should also make sure monitoring
and backups are in place. 

kevin
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: [PATCH] - add db-fas01 to dns - add to puppet - based in largerpart off of db01's recipe

seth vidal — 2012-05-24T19:27:59

On Thu, 24 May 2012 19:28:22 +0000
skvidal< at >fedoraproject.org wrote:

Need +1's to this patch, please

-sv






_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

[PATCH] - add db-fas01 to dns - add to puppet - based in larger partoff of db01's recipe

2012-05-24T19:28:22

From: Seth Vidal <skvidal< at >fedoraproject.org>

---
 manifests/nodes/db-fas01.phx2.fedoraproject.org  |   18 ++++++++++++++++++
 modules/bind/files/master/126.5.10.in-addr.arpa  |    4 ++--
 modules/bind/files/master/phx2.fedoraproject.org |    3 ++-
 3 files changed, 22 insertions(+), 3 deletions(-)
 create mode 100644 manifests/nodes/db-fas01.phx2.fedoraproject.org

diff --git a/manifests/nodes/db-fas01.phx2.fedoraproject.org b/manifests/nodes/db-fas01.phx2.fedoraproject.org
new file mode 100644
index 0000000..ac90bf9
--- /dev/null
+++ b/manifests/nodes/db-fas01.phx2.fedoraproject.org
< at >< at > -0,0 +1,18 < at >< at >
+node db01-fas.phx2.fedoraproject.org {
+    $bacula5 = "true"
+    # re-enable this once we have db's setup. 
+    # $backup_db_often = true
+    include phx
+    include db
+    include appDB-el6
+    collectd::collectd { 'log02': }
+    collectd::postgres { 'postgres':
+       databases => ['fas']
+    }
+
+    # Workaround for bug 768055
+    semanage_fcontext { '/usr/lib64/nagios/plugins/check_disk':
+        type => 'nagios_unconfined_plugin_exec_t'
+    }
+
+}
diff --git a/modules/bind/files/master/126.5.10.in-addr.arpa b/modules/bind/files/master/126.5.10.in-addr.arpa
index 39e2b4d..9dbb2e2 100644
--- a/modules/bind/files/master/126.5.10.in-addr.arpa
+++ b/modules/bind/files/master/126.5.10.in-addr.arpa
< at >< at > -2,7 +2,7 < at >< at > $TTL 86400
 < at >    IN    SOA    126.5.10.in-addr.arpa. hostmaster.fedoraproject.org.  (
 ; Make SURE to update the serial # when changes are made. Format is YYYYMMDDNN
 ; (NN is just a counter to handle multiple changes in one day).
-                2012031900 ; Serial
+                2012052400 ; Serial
                 28800      ; Refresh
                 14400      ; Retry
                 1000000    ; Expire
< at >< at > -109,7 +109,7 < at >< at > $TTL 86400
 96       IN        PTR      download04.phx2.fedoraproject.org.
 97       IN        PTR      download05.phx2.fedoraproject.org.
 98       IN        PTR      darkserver01.phx2.fedoraproject.org.
-99       IN        PTR      unused.
+99       IN        PTR      db-fas01.phx2.fedoraproject.org.
 100      IN        PTR      unused.
 101      IN        PTR      unused.
 102      IN        PTR      unused.
diff --git a/modules/bind/files/master/phx2.fedoraproject.org b/modules/bind/files/master/phx2.fedoraproject.org
index c0ebff5..707eae8 100644
--- a/modules/bind/files/master/phx2.fedoraproject.org
+++ b/modules/bind/files/master/phx2.fedoraproject.org
< at >< at > -3,7 +3,7 < at >< at > $TTL 3600
 ; Make SURE to update the serial # when changes are made. Format is
 ; YYYYMMDDNN (NN is just a counter to handle multiple changes in one
 ; day). 
-                2012051001 ; Serial
+                2012052401 ; Serial
                 3600      ; Refresh
                 600      ; Retry
                 1000000    ; Expire
< at >< at > -62,6 +62,7 < at >< at > db5             IN      CNAME   db05
 db-wikiINCNAMEdb05
 db-smoltINCNAME   db05
 db-insightINCNAMEdb05
+db-fas01INA10.5.126.99
 dhcp01INA10.5.125.43
 download01INA10.5.126.93
 download02INA10.5.126.94

Summary/Minutes from today's Fedora Infrastructure meeting(2012-05-24 18UTC)

Kevin Fenzi — 2012-05-24T19:04:05

============================================
#fedora-meeting: Infrastructure (2012-05-24)
============================================


Meeting started by nirik at 18:01:02 UTC. The full logs are available at
http://meetbot.fedoraproject.org/fedora-meeting/2012-05-24/infrastructure.2012-05-24-18.01.log.html
.



Meeting summary
---------------
* Robot Roll Call  (nirik, 18:01:02)

* New folks introductions and Apprentice tasks.  (nirik, 18:03:37)

* bugzilla and database issues this week  (nirik, 18:08:33)
  * ACTION: nirik to file outage tickets for these two things so we have
    record of them.  (nirik, 18:20:26)
  * plan to move fas db to it's own db server.  (nirik, 18:20:40)
  * adjust freeze document  (nirik, 18:20:51)

* two factor auth status  (nirik, 18:23:09)

* Fedora 17 Release tickets  (nirik, 18:24:25)
  * release 2012-05-29  (nirik, 18:24:45)
  * LINK: https://fedorahosted.org/rel-eng/ticket/4963   (nirik,
    18:30:08)

* Applications status / discussion  (nirik, 18:31:57)
  * LINK: http://findingscience.com/mod_auth_openid/ if anyone wants to
    look it over.  (nirik, 18:40:51)

* Upcoming Tasks/Items  (nirik, 18:42:16)
  * 2012-05-08 to 2012-05-29 FINAL FREEZE  (nirik, 18:42:16)
  * 2012-05-29 - F17 release  (nirik, 18:42:16)
  * 2012-06-01 - nag fi-apprentices.  (nirik, 18:42:16)
  * 2011-06-03 - gitweb-cache removal day.  (nirik, 18:42:16)
  * 2012-06-04 - class B reboots?  (nirik, 18:42:16)
  * 2012-06-05 - class A reboots?  (nirik, 18:42:17)
  * 2012-06-08 OOW: osuosl01.fedoraproject.org  (nirik, 18:42:19)
  * 2012-06-17 OOW: sign-vault02.phx2.fedoraproject.org  (nirik,
    18:42:20)
  * 2012-06-21 to 2012-07-04 Kevin is off on trains and boats.  (nirik,
    18:42:22)
  * sometime in June hardware will show up in Phx for new systems
    (smooge, 18:44:44)
  * mediawiki 119 upgrade coming.  (nirik, 18:51:02)
  * koji upgrades coming  (nirik, 18:51:09)
  * will apply fix for mod_wsgi hash stuff soon  (nirik, 18:51:25)
  * trip out onsite to phx2 sometime in the next few months.  (nirik,
    18:51:41)
  * mirrormanager 1.4 coming too  (nirik, 18:52:23)

* Open Floor  (nirik, 18:52:56)
  * LINK: https://aws.amazon.com/amis   (skvidal, 19:01:02)

Meeting ended at 19:02:51 UTC.




Action Items
------------
* nirik to file outage tickets for these two things so we have record of
  them.




Action Items, by person
-----------------------
* nirik
  * nirik to file outage tickets for these two things so we have record
    of them.
* **UNASSIGNED**
  * (none)




People Present (lines said)
---------------------------
* nirik (140)
* skvidal (34)
* dgilmore (33)
* mdomsch (31)
* marcdeop (18)
* abadger1999 (10)
* zodbot (10)
* cyberworm54 (10)
* relrod (10)
* lmacken (9)
* pingou (8)
* smooge (5)
* threebean (4)
* codemaniac (2)
* inode0 (2)
* jds2001 (2)
* fcami__ (1)
* ricky (0)
* CodeBlock (0)
--
18:01:02 <nirik> #startmeeting Infrastructure (2012-05-24)
18:01:02 <nirik> #meetingname infrastructure
18:01:02 <nirik> #topic Robot Roll Call
18:01:02 <zodbot> Meeting started Thu May 24 18:01:02 2012 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:02 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:01:02 <zodbot> The meeting name has been set to 'infrastructure'
18:01:02 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean
18:01:02 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean
18:01:30 * abadger1999 here
18:01:36 * lmacken 
18:01:38 * threebean is here
18:01:42 * skvidal is here
18:01:47 <dgilmore> hola
18:01:51 * jds2001 around
18:02:13 <relrod> here
18:02:32 <cyberworm54> hello evrybody
18:02:57 <nirik> welcome everyone.
18:03:33 <nirik> ok, lets go ahead and dive in then
18:03:37 <nirik> #topic New folks introductions and Apprentice tasks.
18:03:37 <nirik> If any new folks want to give a quick one line bio or any apprentices
18:03:37 <nirik> would like to ask general questions, they can do so now. Anyone?
18:03:56 <cyberworm54> i will
18:04:09 * marcdeop is sorry to be late
18:04:20 <nirik> no worries.
18:04:32 <cyberworm54> My name is Ivan garcia I've been a linux kvm admin for 4 years now
18:04:50 <cyberworm54> living in the USA
18:05:18 <nirik> cyberworm54: welcome. :) Are you most interested in the sysadmin side of things? or application devel/programming?
18:06:10 <cyberworm54> thank you :) I am more in the infrastructure side/sysadmin, I have proggramming skills and scripting too such as python and bash
18:06:27 * codemaniac is here
18:06:35 <nirik> great. Come see me in #fedora-admin later this afternoon and we can see about getting you setup in the apprentice group...
18:06:42 <nirik> welcome codemaniac
18:07:00 <codemaniac> \o nirik .Good to see you .
18:07:03 <nirik> so, any other general questions from apprentices or new folks? Or shall we move on?
18:07:08 <cyberworm54> that would be excellent I am really excited about this opportunity  :) thank you
18:07:37 * marcdeop welcomes the new fellow!
18:08:09 <cyberworm54> :)
18:08:17 <cyberworm54> thanks marcdeop
18:08:17 <nirik> ok, I'd like to diverge from the agenda I sent out for a bit and talk about issues we have been hitting this week...
18:08:33 <nirik> #topic bugzilla and database issues this week
18:08:53 <nirik> So, the first issue we ran into was that bugzilla.redhat.com was upgraded last saturday.
18:09:14 <nirik> We (in retrospect foolishly) updated python-bugzilla on our servers.
18:09:47 <nirik> The new python-bugzilla has some behavior and cases where it causes the same query as before to generate a LOT of additional queries with the new interface
18:10:11 <nirik> We have now downgraded back to the old stable python-bugzilla and (almost) everything should be back to normal.
18:10:28 <nirik> What can we do to better avoid this kind of thing moving forward?
18:11:07 <nirik> One thing I can think of is to better test in stg with partner-bugzilla.
18:11:21 <skvidal> nirik: to be fair
18:11:25 <nirik> Red Hat runs a bugzilla instance at partner-bugzilla.redhat.com that has email turned off.
18:11:30 <skvidal> nirik: the limited scale of tests in staging
18:11:30 <marcdeop> easnt here. tezting bugzilla for some weeks?
18:11:39 <skvidal> might not have tipped off the issues
18:11:47 <skvidal> and the load issues may not have been visible
18:12:03 <nirik> skvidal: true. Also, we didn't know until RHIT contacted us and blocked us out for causing too much load. ;)
18:12:08 <skvidal> nirik: indeed
18:12:37 <nirik> marcdeop: they updated the testing one a while back, but only updated the production one (bugzilla.redhat.com) last saturday
18:13:12 <nirik> Anyhow, I will probibly send a email on this to the list too with more info about what happened, etc.
18:13:31 <nirik> Right on the heels of fixing the bugzilla issues, we ran into some database problems.
18:13:39 * pingou (late)
18:13:55 <nirik> This was caused we think by a fedora-tagger update that hit the database hard every few hours.
18:14:25 * fcami__ is here (sorta)
18:14:26 <marcdeop> nirik: were those performance problems?
18:14:50 <skvidal> marcdeop: ultimately, that's what we _think_ it was
18:14:51 <nirik> marcdeop: yeah, worse, it would fill up all our database connections so our account system couldn't authenticate anyone.
18:15:10 <marcdeop> nirik: we could check our database connections
18:15:15 <marcdeop> with nagios
18:15:21 <marcdeop> and warn us in a threshold
18:15:37 <nirik> marcdeop: yeah, but sadly when the account system goes down, that means ALL our web apps that need it to authenticate go down.
18:15:50 <nirik> So, nagios was sending us about 150-200 emails anytime this happened.
18:16:01 <nirik> another nagios check isn't going to help, unless it reduces that. :)
18:16:17 <marcdeop> well,I was thinking more about *identifying* the problem
18:16:24 <marcdeop> as you remarked the work _think_
18:16:32 <marcdeop> s/work/word
18:16:37 * nirik is still not 100% sure that was the problem, but reverting that seems to have fixed it.
18:16:56 <skvidal> marcdeop: the issue looks like a tagger/packages update beating up db connections
18:17:10 <skvidal> marcdeop: we're taking multiple steps right now to help solve this for the future
18:17:16 <skvidal> 1. the rollback is a temp solution
18:17:29 <skvidal> 2. moving fas' db to its own instance - away from everything else
18:17:54 <mdomsch> when I get MM 1.4 solidified, it'll be much nicer to its database.  Still needs a boatload of connections for the crawler though
18:17:57 <nirik> yeah, currently out account system db is on the same db host with other things that may be less important
18:18:14 <skvidal> marcdeop: I agree that figuring out what was triggering it is a good idea - but moving fas db is important independent of any failures from today
18:18:24 <skvidal> today just makes it acute, rather than a chronic concern
18:18:30 <nirik> 3. increasing what is in the freeze, so we don't make changes like this that might affect the larger picture.
18:18:32 <marcdeop> skvidal: that's right
18:19:03 <mdomsch> skvidal: +1
18:19:13 <mdomsch> nirik: we can't control RH IT with our freezes
18:19:15 <dgilmore> skvidal: indeed
18:19:28 <nirik> mdomsch: very true...
18:19:39 <nirik> but we could have not been updating packages/tagger now.
18:20:04 <skvidal> I just posted a freeze break request to infra list - +1's for the general process and I'll post more for the puppet changes
18:20:26 <nirik> #action nirik to file outage tickets for these two things so we have record of them.
18:20:40 <nirik> #info plan to move fas db to it's own db server.
18:20:44 <pingou> nirik: have we actually had outage ?
18:20:51 <nirik> #info adjust freeze document
18:20:57 <pingou> or were we quick enough to reboot the pg server every two hours ?
18:20:59 <nirik> pingou: well, depends on how you define it.
18:21:18 <nirik> there's a short window where things were not authenticating. (less than a minute I'd guess)
18:21:20 <pingou> nirik: let's say outage longer than few min
18:21:24 <pingou> ok
18:21:34 <nirik> also, for the bugzilla thing, maintainers couldn't add bugs to their updates...
18:21:42 <nirik> and bugs.fedoraproject.org didn't work
18:22:34 <nirik> so, it was sorta kinda an outage of not so important things.
18:22:53 <nirik> anyhow, just wanted to talk about those items real quick... will move along now.
18:22:56 * marcdeop wonders if bugs works as of right now
18:23:04 <nirik> marcdeop: it should, yeah
18:23:09 <nirik> #topic two factor auth status
18:23:18 <nirik> any news on the two factor front this week?
18:23:40 <skvidal> nope
18:23:42 <skvidal> next
18:23:47 <skvidal> actually..
18:23:51 <skvidal> I haven't seen wolfkit in a bit
18:24:21 <nirik> yeah, everyone is busy. :) No worries...
18:24:25 <nirik> #topic Fedora 17 Release tickets
18:24:36 <nirik> we have release tickets lined up for the release on tuesday.
18:24:45 <nirik> #info release 2012-05-29
18:25:07 <nirik> .ticket 3285
18:25:08 <zodbot> nirik: #3285 (Fedora17 Final - new website) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/3285
18:25:11 <nirik> .ticket 3286
18:25:16 <zodbot> nirik: #3286 (Fedora17 Final: verify mirror space) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/3286
18:25:21 <nirik> .ticket 3287
18:25:23 <zodbot> nirik: #3287 (Fedora17 Final - release day ticket) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/3287
18:25:29 <nirik> .ticket 3288
18:25:30 <zodbot> nirik: #3288 (Fedora17 Final - verify permissions on content) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/3288
18:25:34 <nirik> .ticket 3289
18:25:35 <zodbot> nirik: #3289 (Fedora17 Final - modify Template:FedoraVersion on wiki) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/3289
18:25:43 <nirik> .ticket 3290
18:25:45 <zodbot> nirik: #3290 (Fedora17 Final - update stats gathering scripts for new release) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/3290
18:25:56 <nirik> I think we are pretty much on track with them all.
18:26:08 <nirik> as usual, we can't check bits until rel-eng stages them
18:26:28 <mdomsch> nirik: any guess when that is?
18:26:35 <mdomsch> today or tomorrow?
18:26:40 <nirik> dgilmore: ^ ? :)
18:26:53 <dgilmore> nirik: ill be staging today
18:26:55 * mdomsch needs to be sure the move-to-release script doesn't blow up again like it did last time
18:26:59 <dgilmore> and disabling buildbranched
18:27:08 <nirik> great.
18:27:50 <dgilmore> likely ill flip the bits for the mirrors today,  but it might be in the morning
18:27:57 <nirik> smooge: you set to take the verify ones you usually do? or would you like someone else to?
18:28:08 <dgilmore> going to tripple check that hardlinking etc is right before opening up
18:28:29 <smooge> I am ready
18:28:34 <nirik> dgilmore: are we nuking drpms? I think there was a request to do that last cycle?
18:28:48 <smooge> I will start checking after he has staged. We have plenty fo disk space still
18:29:13 <dgilmore> nirik: i dont know
18:29:22 <dgilmore> nirik: it feels like we shouldnt nuke it
18:29:47 <nirik> they don't do much good
18:30:08 <nirik> https://fedorahosted.org/rel-eng/ticket/4963
18:30:11 <dgilmore> no they dont
18:30:14 <nirik> anyhow, just thought I would mention it.
18:30:20 <nirik> anything else for release?
18:30:22 <dgilmore> and skvidal swears that yum wont break if they dont exist
18:30:30 <skvidal> what did I do?
18:30:33 <skvidal> drpms?
18:30:41 <skvidal> oh
18:30:42 <skvidal> yah
18:30:47 <mdomsch> that's save a few GB
18:30:47 <dgilmore> skvidal: if we drop drpms from Everything tree
18:30:57 <skvidal> pretty confident in the opportunistic nature of yum on that
18:31:05 <skvidal> well
18:31:07 <skvidal> mostly confident
18:31:08 <skvidal> :)
18:31:18 <dgilmore> skvidal: its the mostly that scares me
18:31:21 <skvidal> :)
18:31:25 <skvidal> LIVE IN FEAR
18:31:33 <marcdeop> die in peace!
18:31:38 <marcdeop> :P
18:31:47 <nirik> anyhow, shall we move along...
18:31:53 <dgilmore> yes
18:31:57 <nirik> #topic Applications status / discussion
18:32:09 <relrod> ooh me me pick me :)
18:32:13 <relrod> So the fedorahosted app I've started on is coming along well, threebean has helped a lot too :) It's moved to Fedora Hosted now (mirrored to github still): https://fedorahosted.org/fedorahosted/. I did my first test of things on a RHEL 6 vm last night, the CLI side of the app works fine, but python-flask isn't packaged for RHEL.
18:32:15 <nirik> lmacken / abadger1999 / threebean / pingou / relrod: any apps news?
18:32:18 <relrod> *so* my question is this. The web app has no tie to anything Fedora Hosted right now. It's just a web form and a db that stores stuff, that may end up having FAS auth so people can't create requests for $random_fas_username. Do we want to explore putting and leaving the web frontend on openshift? Or do we want to get Flask +deps in EPEL? I'm not sure what the general FI stance is, re: "production-ish" stuff on openshift.
18:32:26 * relrod may have had that typed out beforehand ;)
18:32:41 * lmacken did lots of fedora-packages/moksha work this week... nothing visible though.
18:32:59 <nirik> I'd perfer things like that we do host ourselves... just to avoid an external dependency.
18:33:20 <lmacken> apps.fp.o/{packages,tagger} are pretty much done though.
18:33:28 * threebean prepares for the python-flask-* packaging party
18:33:35 <relrod> nirik: ok. It was just a thought
18:33:40 * dgilmore has tasks for a webapp developer if they want something to do
18:33:51 <nirik> relrod: it looks like it has a epel branch, but just was never built for some reason.
18:34:27 * skvidal has been working on a coprs-submission front end using flask and openshift, too - it's just a test/wip
18:34:34 <nirik> lmacken: thats cool. Do we want to do some kind of announcement? perhaps after release?
18:34:40 <pingou> I got busy last week-end on mongodb vs postgresql for HyperKitty, but not much since
18:34:46 <lmacken> nirik: yeah, that sounds good.
18:35:22 <lmacken> nirik: I still need to track down why some new packages aren't getting indexed properly, so holding off on the apps.fp.o announcement until after the release is probably a good call.
18:35:25 <pingou> ah kinda fix fedora-active-user to the new bz :)
18:35:51 <nirik> lmacken: yeah, and that way we can doublecheck over things to make sure it's all set.
18:35:56 <nirik> without freeze issues, etc
18:36:22 <relrod> then to that end (re: packaging python-flask-*) if someone is bored and wants to get these in EPEL, I would love you: python-flask, python-flask-sqlalchemy, python-flask-wtf, python-wtforms
18:36:43 <threebean> relrod: thinking -- we should hammer out the FAS auth for flask but separate it out once we're done into python-fedora-flask
18:37:00 <abadger1999> <nod>
18:37:00 * threebean wonders if ianweller's fudcon app has already done this
18:37:07 <relrod> his app uses openid
18:37:08 <abadger1999> I think it uses openid
18:37:09 <skvidal> threebean: I was planning on just using openid + limited to fedoraproject openid only
18:37:11 <nirik> relrod: they appear to have epel branches. I'd try and contact the maintainer and see if they mind you managing those since they never built for it.
18:37:22 <skvidal> abadger1999: is there a good reason to NOT do that?
18:37:50 <abadger1999> skvidal: I like using openid rather than dedicated fas auth in general.  Seems to make things more flexible
18:38:07 <nirik> on the openid front... we may want to look at mod_auth_openid. It looks like it might work to replace our mod_auth_pg thing if we wanted to.
18:38:07 <abadger1999> the one issue is that I don't believe we have an active member of our dev team that knows openid server-side
18:38:18 <abadger1999> so if it's broken in fas, we won't be able to fix it easily.
18:38:21 <skvidal> abadger1999: good point
18:38:38 * mdomsch is making _slow_ progress on MM 1.4.  Bug fixes and perf improvements in the last couple weeks.  I need to do more testing before I unleash it on the world - well after F17 launch
18:38:41 <abadger1999> But we're already depending on it for several things without issue.
18:39:07 <abadger1999> so... more doesn't hurt a whole lot and it does give other sorts of flexibility.
18:39:34 <jds2001> abadger1999: you can learn in 2 hours, right? :)
18:39:55 <abadger1999> jds2001: Yeah.  two hours on the beach in the bahamas, you foot the bill ;-)
18:40:09 <pingou> sounds like a plan
18:40:12 <nirik> one advantage of mod_auth_openid is that you can run a script after the openid auth to determine if the auth is allowed or not, we could possibly hook that into the 2 factor stuff.
18:40:26 <abadger1999> nirik: Mmm that sounds nice.
18:40:38 * marcdeop joinns that plan
18:40:51 <nirik> http://findingscience.com/mod_auth_openid/ if anyone wants to look it over.
18:41:03 <nirik> it's under review (stalled), so we would need to finish that before we use it.
18:41:41 <nirik> anyhow, any more app news? or shall we move on?
18:41:58 <relrod> nothing else from me
18:42:08 <nirik> ok, upcoming items:
18:42:16 <nirik> #topic Upcoming Tasks/Items
18:42:16 <nirik> #info 2012-05-08 to 2012-05-29 FINAL FREEZE
18:42:16 <nirik> #info 2012-05-29 - F17 release
18:42:16 <nirik> #info 2012-06-01 - nag fi-apprentices.
18:42:16 <nirik> #info 2011-06-03 - gitweb-cache removal day.
18:42:16 <nirik> #info 2012-06-04 - class B reboots?
18:42:17 <nirik> #info 2012-06-05 - class A reboots?
18:42:19 <nirik> #info 2012-06-08 OOW: osuosl01.fedoraproject.org
18:42:20 <nirik> #info 2012-06-17 OOW: sign-vault02.phx2.fedoraproject.org
18:42:22 <nirik> #info 2012-06-21 to 2012-07-04 Kevin is off on trains and boats.
18:42:24 <nirik> I was looking at scheduling some mass rebooting for after freeze.
18:42:30 <nirik> get us up on the current kernel and updates.
18:43:11 <relrod> nirik: Historically the final freeze goes until one day after the release, not day-of
18:43:21 <nirik> yeah, sorry you are right.
18:43:45 * nirik adjusts
18:43:53 <nirik> anyhow, anything folks want to schedule or note?
18:44:26 <smooge> trip to Az in July is the only thing on my radar.
18:44:29 <inode0> voting will be happening June 1-7
18:44:44 <smooge> #info sometime in June hardware will show up in Phx for new systems
18:44:47 <dgilmore> nirik: going to look at doing some koji changes soon
18:44:49 <nirik> there's also town hall meetings coming up for the elections... everyone should go and ask hard questions. ;)
18:45:11 <nirik> smooge: cool. yeah, we should coordinate that with rhit folks and get it scheduled.
18:45:16 <nirik> dgilmore: whats on the slate?
18:45:23 * inode0 is hoping all the rebooting doesn't affect voting as we needed to extend once before for interruptions
18:45:45 <nirik> inode0: I could push them off a week...
18:45:48 <dgilmore> nirik: move to mod_wisgi
18:45:49 <nirik> and avoid that.
18:45:58 <dgilmore> wsgi
18:46:04 <smooge> mediawiki upgrade
18:46:24 <dgilmore> nirik: there has been somebiggish changes in koji
18:46:29 <dgilmore> upstream
18:46:35 <nirik> inode0: I'll do that. Thanks for the note.
18:46:53 <dgilmore> going to look at setting up some policies to limit certain builds to certain builders
18:46:53 <nirik> smooge: yeah, we need to get staging all updated so we can test... but would be good to get that done too.
18:47:18 <nirik> dgilmore: cool. Should we schedule that for when we do reboots? or will it be further down the road?
18:47:33 <dgilmore> nirik: maybe further down the road
18:47:38 <nirik> ok
18:47:52 <dgilmore> koji with mod_wsgi has not had a lot of testing yet
18:48:08 <dgilmore> its a pretty big change from mod_python
18:48:15 <nirik> yeah, but mod_python is dead.
18:48:24 <dgilmore> yerp
18:48:42 <lmacken> and we should secure our apps from hash collission attacks soon :)
18:49:01 <lmacken> **collision
18:49:10 <nirik> lmacken: yeah. Possibly we could do that at the same time as a reboot outage...
18:49:16 <nirik> although it shouldn't be much downtime
18:49:35 <lmacken> cool.
18:49:51 <nirik> if it's ready by then I guess. ;)
18:50:22 <lmacken> we should probably just take the environment variable approach first, since mod_wsgi upstream has been far from useful with regard to this patch
18:50:34 <nirik> yeah. seems easiest.
18:51:02 <nirik> #info mediawiki 119 upgrade coming.
18:51:09 <nirik> #info koji upgrades coming
18:51:25 <nirik> #info will apply fix for mod_wsgi hash stuff soon
18:51:41 <nirik> #info trip out onsite to phx2 sometime in the next few months.
18:51:47 <nirik> Anything else?
18:51:59 <mdomsch> MM 1.4 when I get around to it
18:52:23 <nirik> #info mirrormanager 1.4 coming too
18:52:40 * nirik wonders if we can get all this done before f18 freezes start. ;)
18:52:45 <nirik> time will tell
18:52:56 <nirik> #topic Open Floor
18:53:02 <mdomsch> S3 mirrors
18:53:03 * marcdeop encourages positive attitude!
18:53:07 <nirik> ok, anything for open floor? questions comments?
18:53:13 <mdomsch> I've started a new bucket for us-west-1
18:53:15 <nirik> mdomsch: oh yeah, whats the status there?
18:53:35 <mdomsch> and bapp02 is doing an s3cmd sync from s3://us-east-1 to s3://us-west-1
18:53:47 <mdomsch> so in reality s3 is doing the copy for us
18:53:50 <nirik> nice
18:53:56 <mdomsch> well...  S3 is slow...
18:54:03 <mdomsch> it's been running for 3-4 days now
18:54:06 <mdomsch> since Monday
18:54:06 <nirik> how many regions should we/can we support?
18:54:23 <mdomsch> we could do several, but the copying is _really_ slow
18:54:47 <mdomsch> and arguably we should move archive (f8) content into its own set of buckets, so s3cmd list doesn't have to walk that content
18:54:58 <mdomsch> on every sync run
18:55:27 <nirik> yeah.
18:55:33 <mdomsch> I presume we prefer to let s3 do the copying (initial copy costs the OpenShift team about $11)
18:55:43 <mdomsch> into each bucket, rather than us push from phx2 into each bucket
18:55:47 <nirik> is this something we want to announce/shout out? either as part of release or seperate?
18:55:50 <mdomsch> so it becomes a 2-stage sync
18:56:01 <mdomsch> I don't know if it'll be done in time for the release...
18:56:08 <nirik> yeah, I'd say let them do it unless it's just too slow to work
18:56:16 <mdomsch> phx2 -> us-east-1; us-east-1->us-west-1
18:56:32 <mdomsch> it's taking days to move 300GB
18:56:44 <dgilmore> mdomsch: id say phx2 to us-west-1
18:56:46 <mdomsch> but I forget how long the initial upload from phx2 -> us-east-1 was
18:56:51 <dgilmore> just because its physically closer
18:57:05 <nirik> do we have any amazon contacts we could ping and ask?
18:57:16 * nirik looks at spevack.
18:57:16 <dgilmore> nirik: spevack
18:57:43 <mdomsch> anyhow, I can bring that online whenever; it's just a DNS change once the content is in place
18:57:52 <mdomsch> and a s3sync change to copy to a second destination
18:58:07 <nirik> mdomsch: can you ping spevack and see if he can provide any suggestions on how we can better do it?
18:58:17 * nirik can if you like too
18:58:22 <dgilmore> mdomsch: do we now what kinda hits the ec2 mirror gets?
18:58:37 <mdomsch> dgilmore: we have the logs; I don't have a log parsing tool to count them
18:58:51 <dgilmore> mdomsch: ok, mostly just curious
18:59:14 <mdomsch> many many many small files with 1-50 hits in each file
18:59:52 <mdomsch> that's all I have
18:59:55 <dgilmore> mdomsch: ok, mostly im curious as there has been zero feedback on the AMIS for f17
19:00:05 <dgilmore> makes me wonder if they have even been used
19:00:33 <marcdeop> amis?
19:00:46 <dgilmore> marcdeop: the fedora images in EC2
19:00:48 <nirik> amazon images
19:00:58 <skvidal> marcdeop: Amazon Machine Images
19:01:02 <skvidal> https://aws.amazon.com/amis
19:01:03 <nirik> so someone can say: "make me a new instance... Fedora 17 please"
19:01:04 <nirik> ;)
19:01:10 <marcdeop> ohh yeah thanks
19:01:22 <cyberworm54> :)
19:01:30 <cyberworm54> EC< at >
19:01:47 <nirik> ok, any other open floor items? shall we call it a meeting?
19:02:34 <nirik> ok, thanks for coming everyone!
19:02:37 <dgilmore> nirik: think we are good
19:02:48 <nirik> as usual, hang out in #fedora-admin, #fedora-noc and #fedora-apps.
19:02:51 <nirik> #endmeeting
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: Freeze Break: shutdown search01-dev on virthost05 - setupdb-fas01 on virthost05

Dennis Gilmore — 2012-05-24T18:30:54

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 24 May 2012 14:22:55 -0400
Ricky Elrod <codeblock< at >elrod.me> wrote:


+1
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)

iEYEARECAAYFAk++fmQACgkQkSxm47BaWfdS3QCggUV3ixPWm0LdH2zOIvM3YJtv
n1YAoL6XswrCwUXMZFu7LGWlL8J+Lsyy
=xtbE
-----END PGP SIGNATURE-----
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: Freeze Break: shutdown search01-dev on virthost05 - setup db-fas01on virthost05

Ricky Elrod — 2012-05-24T18:22:55

+1

_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Freeze Break: shutdown search01-dev on virthost05 - setup db-fas01on virthost05

seth vidal — 2012-05-24T18:15:27

This will be a first step to migrate to a private fas postgres instance.

In order to free up mem and procs I would like to shutdown search01-dev
on virthost05 and bring up db-fas01 on there.

The goal would be a private postgres db server for fas alone.


I will be kickstarting a vm on virthost05 and then adding entries to
puppet/dns to accommodate this.

I need some +1's to break the freeze to make this happen.

thanks,
-sv
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: [Change Request] Fix pkgdb privileged operations for oldpython-bugzilla

Kevin Fenzi — 2012-05-24T15:23:18

+1 here as well. 

kevin
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: [Change Request] Fix pkgdb privileged operations for oldpython-bugzilla

seth vidal — 2012-05-24T15:06:08

On Thu, 24 May 2012 08:01:28 -0700
Toshio Kuratomi <a.badger-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org> wrote:


+1
-sv
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

[Change Request] Fix pkgdb privileged operations for oldpython-bugzilla

Toshio Kuratomi — 2012-05-24T15:01:28

As part of two situations I've pushed out a hotfix to pkgdb and would like
two +1's for the change.

The situations:
* Continuing work to get our apps working with the old python-bugzilla
  package from yesterday's problems between infrastructure and bugzilla
* Unknown causes of downtime for the fas servers.

We're currently trying to determine what's causing the second situation to
recur ~ every two hours.  As part of that we're trying to fix anything that
is out of the ordinary. So I've pushed this fix out and verified it works
with the reporter.  If I could get two +1's for after the fact application
during change freeze, that would be great.

The change:
When PackageDB makes changes to who owns a package or creates a new package
it needs to login to bugzilla and find out if the new owner has a bugzilla
account.  Those connections were failing for the same reason as fas's
export-bugzilla was failing yesterday: the older version of python-bugzilla
does not know how to get this information out of the new version of
bugzilla.  The fix is the same as yesterday's fix to export-bugzilla: for
now, use the RHBugzilla3 compatibility API to get this information.

https://fedorahosted.org/fedora-infrastructure/ticket/3304

diff --git a/modules/hotfix/files/pkgdb/utils.py b/modules/hotfix/files/pkgdb/utils.py
index 021a80f..8c121c5 100644
--- a/modules/hotfix/files/pkgdb/utils.py
+++ b/modules/hotfix/files/pkgdb/utils.py
< at >< at > -51,7 +51,7 < at >< at > from cherrypy import request
 from sqlalchemy import select, and_
 from sqlalchemy.exceptions import DataError
 
-from bugzilla import Bugzilla
+from bugzilla import RHBugzilla3
 import memcache
 
 # The Fedora Account System Module
< at >< at > -218,9 +218,8 < at >< at > def get_unprivileged_bz():
     bz_server = config.get('bugzilla.queryurl', config.get('bugzilla.url',
         'https://bugzilla.redhat.com'))
     bz_url = bz_server + '/xmlrpc.cgi'
-    _unpriv_bugzilla = Bugzilla(url=bz_url, cookiefile=None)
+    _unpriv_bugzilla = RHBugzilla3(url=bz_url, cookiefile=None)
     return _unpriv_bugzilla
-    
 
 def get_bz():
     '''Retrieve a connection to bugzilla
< at >< at > -236,7 +235,7 < at >< at > def get_bz():
     bz_url = bz_server + '/xmlrpc.cgi'
     bz_user = config.get('bugzilla.user')
     bz_pass = config.get('bugzilla.password')
-    _bugzilla = Bugzilla(url=bz_url, user=bz_user, password=bz_pass,
+    _bugzilla = RHBugzilla3(url=bz_url, user=bz_user, password=bz_pass,
             cookiefile=None)
     return _bugzilla


-Toshio
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Re: Hello to everybody

cyberworm — 2012-05-24T02:09:00

I am looking forward to introduce myself in the weekly meeting :)

Ivan Garcia 

On Mon, 2012-05-21 at 17:29 +0000,
infrastructure-request< at >lists.fedoraproject.org wrote:


_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure

Plan for tomorrow's Fedora Infrastructure meeting (2012-05-24 18UTC)

Kevin Fenzi — 2012-05-23T23:52:13

The infrastructure team will be having it's weekly meeting tomorrow, 
2012-05-24 at 18:00 UTC in #fedora-meeting on the freenode network.

Suggested topics:

#topic New folks introductions and Apprentice tasks.

If any new folks want to give a quick one line bio or any apprentices
would like to ask general questions, they can do so here. 

#topic two factor auth status

#topic Fedora 17 Release tickets

#topic Applications status / discussion

Check in on status of our applications: pkgdb, fas, bodhi, koji,
community, voting, tagger, packager, dpsearch, etc. 
If there's new releases, bugs we need to work around or things to note. 

#topic Upcoming Tasks/Items 

#info 2012-05-08 to 2012-05-29 FINAL FREEZE
#info 2012-05-29 - F17 release
#info 2012-06-01 - nag fi-apprentices.
#info 2011-06-03 - gitweb-cache removal day.
#info 2012-06-04 - class B reboots?
#info 2012-06-05 - class A reboots?
#info 2012-06-08 OOW: osuosl01.fedoraproject.org
#info 2012-06-17 OOW: sign-vault02.phx2.fedoraproject.org
#info 2012-06-21 to 2012-07-04 Kevin is off on trains and boats.

#topic Meeting tagged tickets: 
https://fedorahosted.org/fedora-infrastructure/report/10

#topic Open Floor

Submit your agenda items, as tickets in the trac instance and send a 
note replying to this thread.

More info here:

https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings

Thanks

kevin
_______________________________________________
infrastructure mailing list
infrastructure< at >lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure