gmane.comp.systems.beaker.devel

Re: Beaker road map updated

Nick Coghlan — 2013-05-21T00:04:17

Hmm, I should probably add a pointer to beakerlib in there somewhere - 
quite a few people write for beakerlib rather than using the beah/RHTS 
commands directly, and *that's* still open to new features and other 
changes.

Cheers,
Nick.

Beaker road map updated

Nick Coghlan — 2013-05-20T07:37:44

The Bugzilla downtime today ended up providing me with a good 
opportunity to bring the technical road map up to date.

The major changes are to set our objectives for Beaker 1.0 and to 
clearly document beah's current status as fully supported, but not 
actively developed.

* http://beaker-project.org/dev/tech-roadmap.html#objectives-for-beaker-1-0
* http://beaker-project.org/dev/tech-roadmap.html#the-status-of-beah

I also added several references to other projects (like Rickshaw, 
Ansible and cloud-init) that may be of interest for various parts of the 
road map.

The site update also covered the documentation and draft release notes 
for 0.13 changes that have already been merged:

* http://beaker-project.org/docs-develop/whats-new/#unreleased-changes

The full diff is at 
http://git.beaker-project.org/cgit/beaker-project.org/commit/?id=346da204962194b1a95f5aa44954c6589d097990

Cheers,
Nick.

The march towards Beaker 1.0

Nick Coghlan — 2013-05-13T07:43:33

Beaker has been doing 0.x releases for a long time. We've been talking 
about the release later this month being 1.0, but before we declare 
we've reached that milestone, I'd really like us to be at the point 
where someone could reasonably expect to grab the Beaker RPMs from 
beaker-project.org, and using just our documentation, set up and start 
running their own Beaker instance, including writing new tests using 
either beah or autotest.

We're not there yet, and we're not going to get there this month, so the 
next release will be 0.13 and we'll continue with regular 0.x releases 
until we're happy we have something we consider worthy of the "Beaker 
1.0" title. I don't think we're all that far away - we may need a 0.14, 
but I'll be surprised if we see 0.15.

For planning purposes, I'd like to keep marking the backlog as "1.0" - 
the 0.x releases will then just be snapshots that make useful chunks of 
functionality available to users, rather than having to wait until we're 
done with everything we want

Re: Defining a more useful component list in Bugzilla

Nick Coghlan — 2013-05-13T07:01:29

That makes sense. (And I guess I really should write that bug triage 
guide that has been on my todo list for a couple of months now...)


I quite like that. One concern I had with my suggestions was that the 
lines between categories are rather blurry, whereas package boundaries 
are nice and clear.

Since the documentation isn't packaged any more, we should keep that as 
a component, too.


Fedora has a "distribution" catch-all for bugs which can't be neatly 
allocated to one package. We would probably need something similar for 
feature requests and the like.

Cheers,
Nick.

Re: Defining a more useful component list in Bugzilla

Dan Callaghan — 2013-05-13T06:33:03

Excerpts from Nick Coghlan's message of 2013-05-13 13:44:24 +1000:

These sound more like bug "topics" or "feature areas" than actual 
components. I think it makes more sense to continue using a whiteboard 
field for that, since redefining them is not a big deal and we can 
easily add new ones -- unlike component.

There was some talk in the past that our internal release tooling would 
require us to use package names (or subpackage names) as our Bugzilla 
components. I'm not sure if that's accurate but it might be a reasonable 
way to split the components anyway.

    beaker-server
    beaker-lab-controller
    beaker-client
    beaker-integration-tests
    beah
    rhts

plus I guess an extra one for the tasks we maintain. It has the added 
advantage of making it clear which package needs to be updated for the 
fix. The downside is there is no way to handle fixes which cross 
multiple subpackages, which is probably quite a lot of them.

Re: Defining a more useful component list in Bugzilla

Amit Saha — 2013-05-13T06:09:40


----- Original Message -----


+1.


+ 1 to the above.


I am trying to understand exactly which bugs we classify into this category. The major user of the XML-RPC interface AFAIU is
the 'bkr' cli itself. So, are you suggesting that we confine the I part of the CLI to 'bkr cli' and anything to do with the interfacing 
with the server via XML-RPC go into this new category?


+ 1. And perhaps a comment suggesting the components it could span.

+ 1

+1 to Fedora compatibility.


Does it make sense to limit the scope of "scheduler" component to only bugs which involve
actually scheduling? (for example, a broken machine is chosen to be scheduled)  and have a 
separate category "server" for bugs such as "a group feature is broken" ? (I think, right now, we
would classify this bug in the scheduler category).

-Amit.

Re: Defining a more useful component list in Bugzilla

Raymond Mancy — 2013-05-13T05:59:14


----- Original Message -----

+1

_______________________________________________
Beaker-devel mailing list
Beaker-devel< at >lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel

Defining a more useful component list in Bugzilla

Nick Coghlan — 2013-05-13T03:44:24

The current component list in Bugzilla has a few limitations, with some 
bugs being thrown at a component more or less at random, since they 
don't fit any of the existing components. In other cases, the component 
is too broad, failing to make appropriate distinctions in the assignments.

Here's the current component list:

     command line
     Doc
     inventory
     qpid
     lab controller
     scheduler
     reports
     tests
     test harness
     web UI

Those actually aren't *that* bad, there are just some missing ones and a 
couple could do with a name change.

Suggested name/scope changes:

     qpid -> notifications (covering both AMQP sending and receiving and 
email sending)
     tests -> test suite
     inventory -> inventory task
     test harness -> beah test harness
     command line -> bkr CLI

Suggested new components:

     beaker-wizard CLI
     other tasks (covers all standard tasks other than the inventory task)
     harness API (for the new stable harness API)
     server XML-RPC (

Re: New Beaker API quirks

Dan Callaghan — 2013-05-13T01:39:09

Sorry for the very belated reply, Don!

Excerpts from Don Zickus's message of 2013-04-02 23:53:56 +1000:

Another user has just reported this as a bug:

https://bugzilla.redhat.com/show_bug.cgi?id=961300

It should be an easy fix on the lab controller side.


Yes I agree, that was an oversight really. Bug filed:

https://bugzilla.redhat.com/show_bug.cgi?id=962253


It should definitely be 409. Bug filed:

https://bugzilla.redhat.com/show_bug.cgi?id=962254

Handling exit status of the commands execution fromall the nodes

shwetha — 2013-05-07T05:48:20

Hi All ,

In a multi-node testing environment we have to fail a test case when 
certain commands on any one of the node fails . For example : if 
mounting fails on client node , that test case execution should not be 
continued. A message to be sent to all other machines i.e 
masternode/node/peer in this case to stop the execution.

How are we going to handle this situation. *This is the basic 
requirement for multi-node testing.* We want to check the exit status 
after every step. For example in most of our testcases the set-up includes :

1) peer probing
2) volume_create
3) volume_info
4) volume_start
5) volume_status
6) mount

In every step we should check the exit status from all the nodes and if 
any one node has non-zero exit status we should stop that test case and 
move on to next test case in that task.

-Shwetha
_______________________________________________
Beaker-devel mailing list
Beaker-devel< at >lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel

Technical roadmap updated

Nick Coghlan — 2013-05-03T06:56:06

I've just published an update [1] to the tech roadmap at 
http://beaker-project.org/tech-roadmap.html

Three changes of note:

- I moved the "Live Dashboard" down into the exploration section. It 
would still be nice to have it, but it turns out that Graphite's 
rendering API plus the ReloadEvery extension for Firefox ([2]) isn't a 
bad interim solution :)

- I moved the OpenStack integration up into the planned section. oVirt 
Engine is great if you're wanting to run a long lived high availability 
core service, but not so good if you're after throwaway VMs to run a 
single recipe. By contrast, OpenStack's reputation is pretty much the 
exact opposite (great for spinning up instances quickly, less great if 
you then want to keep them running forever), so it will hopefully be a 
better fit for our use case.

- I added Jenkins integration to the "exploration" section. While you 
*can* use Beaker for continuous integration (as Dan's patchbot shows), 
Jenkins is really the king of that space. I see a lot of pot

Patch merge criteria documented

Nick Coghlan — 2013-05-03T05:48:53

The criteria for marking patches as verified and for passing code review 
are now documented at 
http://beaker-project.org/dev-guide.html#reviewing-a-patch

We'd generally been applying these anyway, but now they're written down 
so people don't need to guess about what we're looking for (and they're 
a useful checklist for us as well).

===============
The "+1 Verified" marker indicates one of the following:

     If it’s a bug fix that is reproduceable and testable, the new test 
case has been verified to fail before the fix, and pass after the fix.
     If it’s a bug fix that is not amenable to an automated test, the 
patch has been verified to fix the bug through some other means (such as 
trying it out manually).
     If it’s a new feature, the feature has been verified to work as 
described.
     If it’s a code change, the test suite has been verified to pass in 
full.
     If it’s a docs change, the docs have been verified to build 
correctly and look right.
     On some rare occasions (for

Re: Revised Enhanced User Group proposal on Gerrit

Nick Coghlan — 2013-05-01T00:01:51

Right, the submit bot will be added to a separate "submission delegates" 
list. The bot account would then have access to modify jobs submitted 
through the bot, but no general access to other jobs submitted for that 
group. While I like the user level delegation idea, where the bot 
wouldn't have access to modify the job at all, it's a complex enough 
enhancement that I'd like to tackle it later.


For the initial iteration, we would need to expand the candidate system 
filter to include systems that are associated with the job's group, in 
addition to those that the delegate user has access to directly. Once we 
add system pools, that logic will all need adjusting anyway.

Cheers,
Nick.

Re: Revised Enhanced User Group proposal on Gerrit

Bill Peck — 2013-04-30T13:07:42

How does this actually work?

The doc you mention above says that all members of the group can 
ack/nack, change priority, delete job, etc..

Does this mean that the submit bot user should not be added as a member 
of the group?  Because if it were a member then it could modify any jobs 
submitted under the group.

But if the delegate user is not in the group then what systems will be 
used?

I do think this is going in the right direction, just need to make sure 
I understand it all. :-)


_______________________________________________
Beaker-devel mailing list
Beaker-devel< at >lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel

Re: Revised Enhanced User Group proposal on Gerrit

Jan Stancek — 2013-04-30T11:55:05




----- Original Message -----

Hi Nick,

Is there a field (other than whiteboard), which I could use to distinguish between jobs?
For example, say I have group "maintainers" who build packages for RHEL5/6/7. For each
package there is a beaker job. How can I make maintainer see only jobs for packages he built?

So, without this "submitting user", how can beaker determine pool of systems,
where my job can run? I assume my new ad-hoc group won't be used by any systems
other than public ones. Do I need to go over all of my systems and add it?
I assume the answer is "for now yes or wait until System Pools proposal is implemented".

Regards,
Jan

_______________________________________________
Beaker-devel mailing list
Beaker-devel< at >lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel

Revised Enhanced User Group proposal on Gerrit

Nick Coghlan — 2013-04-30T09:00:52

Rather than updating the design proposal directly on the site, I decided 
I wanted some feedback on the current draft first:

http://gerrit.beaker-project.org/#/c/1908/1/dev/proposals/enhanced-user-groups.rst

Key changes:

1. The "job modification" flag on group members is gone, replaced with 
the separate concept of a "submission delegate", who can submit jobs on 
behalf of the group, but cannot modify other group jobs
2. Unlike the previous design, submission delegates retain the ability 
to modify the jobs they submit. The design we discussed in the other 
thread (where you could change the actual submitting *user*) has been 
listed as a deferred feature (because it would require quite a bit more 
additional UI design work)

Another couple of points came up while I was writing it:

- should we allow *completed* single-user jobs to be converted to group 
jobs? We originally didn't allow this due to the problem with getting 
SSH keys installed, but that doesn't apply if the job is already 
completed, and i

Re: Clarification request for Enhanced User Groups

Nick Coghlan — 2013-04-30T08:32:32

In writing this up, I realised that since users can create ad hoc groups 
whenever they want, group-based delegation was all we needed to cover 
the use cases I had in mind.

User-based delegation is still an interesting idea, but I'll be flagging 
it as a deferred feature rather than something we try to do for 1.0.

I'll start a new thread once I have the revised design proposal published.

Cheers,
Nick.

Re: Clarification request for Enhanced User Groups

Nick Coghlan — 2013-04-30T07:25:37

Enabling that approach is actually where my original "Job Modification" 
design came from. It just turned out to have other issues that made it a 
bad idea :)

However, I think we can do something slightly different in the revised 
design that will still handle that case:

1. If SubmitBot is submitting an individual job with the nominal 
submitter as "Bob", then SubmitBot must be in Bob's delegation list
2. If SubmitBot is submitting a group job for "Bob's Team" with the 
nominal submitter as "Bob", then SubmitBot must be in Bob's delegation 
list OR in the delegation list for "Bob's Team"

The reason for the extra restriction is that users can be in multiple 
groups, so the group based delegations shouldn't bleed across in those 
cases.

Cheers,
Nick.

Re: Clarification request for Enhanced User Groups

Jan Stancek — 2013-04-30T07:01:40




----- Original Message -----

Could this also be a group option? I'd like to avoid running from one maintainer
to other asking them to update their settings. Something like:
- Admin of Bob's team adds "SubmitBot" to list of users that are authorised to submit
jobs on behalf of members in Bob's team.

Regards,
Jan

_______________________________________________
Beaker-devel mailing list
Beaker-devel< at >lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel

Re: Clarification request for Enhanced User Groups

Nick Coghlan — 2013-04-30T05:53:38

OK, that makes sense.

Getting back to the original Q in the thread, Dan and I think we've come 
up with a superior model that both simplifies the group model *and* 
simplifies the "automated account submitting jobs on behalf of users" 
use case.

Specifically, here's what I'm thinking:

1. The "job modification" privilege goes away. All group members can 
modify jobs submitted on behalf of that group.
2. Each user account gains the ability to specify other accounts that 
can submit jobs on their behalf
3. When submitting a job, you can nominate a different user that will 
appear as the job submitter. All other permissions (including future 
modifications to the job, and whether or not you can submit the job on 
behalf of a group) will be based on that user, rather than the automated 
account.

This may be clearer with an example:

We have a user "Bob", who is a member of the group "Bob's Team"
Bob has an automated account "SubmitBot" that he wants to use to submit jobs
On his user profile page, Bob adds "S

Re: Clarification request for Enhanced User Groups

Dan Callaghan — 2013-04-30T03:14:43

Excerpts from Nick Coghlan's message of 2013-04-30 11:51:32 +1000:

On the contrary, I think all that stuff should just go away entirely for 
"individually owned jobs" (which I assume to mean, a job submitted *not* 
on behalf of a group, that is <job group=""/>). For individually owned 
jobs, only the owner can modify them.

I'm fairly sure all the requests over the years for those extra 
permissions (ack/nack other group members' jobs, delete other group 
members' jobs) were necessary because we never had an idea of jobs 
submitted on behalf of an explicit group before.

One other thing occurs to me: IIRC the main reason for wanting to 
ack/nack and delete others' jobs in the past was for the job matrix 
report, which is effectively global (the whiteboard filter includes all 
matching jobs, regardless of who submitted them). So in order to fully 
satisfy the original use case, it might be necessary to add an extra 
filter in the job matrix to limit to jobs for a given group. Then users 
would construct thei